print logo
Main Content Anchor

Test Validation and Construction

Test Validation and Construction (TV&C) Program

The Test Validation and Construction (TV&C) Program provides professional expertise to decentralized departments in the development and validation of job-related selection procedures in accordance with the federal Uniform Guidelines on Employee Selection Procedures, the Civil Rights Acts of 1964 and 1991, California Government Code, the Americans with Disabilities Act and the professionally accepted standards of the American Psychological Association.

TV&C provides departments with the highest quality selection-related materials and a demonstrated commitment to excellence and innovation. TV&C also offers technical and customized training, consultation, and a wide variety of selection-related products and services such as job analysis and exam development.

The mission of the TV&C Program is to promote merit, diversity, and equity in civil service employment, consistent with California State and federal laws.

​TV&C Services

​TV&C provides a variety of products for departments and agencies that allow them to hire the best and maximize success.  To inquire about contracting with TV&C for these services, contact Christina Marquez Lopez at Christina.Lopez@calhr.ca.gov.

Job Analysis

 The purpose of a job analysis is to identify the critical and expected at entry tasks and knowledge, skills, and abilities (KSAs) needed for successful job performance. TV&C implements a content validation strategy to describe the content of the job.

Examination Planning and Consultation

Examination planning is critical, as it serves as the foundation for the examination process. TV&C can assist your department in addressing the following questions:

  • What testing method best serves your department's needs?
  • What rules and regulations must be considered?
  • What is the best way to validate the chosen selection process?
  • Which documents must be archived?

​Selection Instruments (Examinations)

Assessment Centers

An assessment center is a method of evaluating candidates through the use of a battery of test procedures that assesses the level of expertise and possession of requisite KSAs.

Qualification Appraisal Panel (QAP) Interview

A Qualification Appraisal Panel (QAP) is a formal, systematic, and standardized interview process in which all candidates are asked the same pre-established, job-related questions in the same order.

Situational Judgment Test (SJT)

A Situational Judgment Test (SJT) typically presents applicants with a job-related scenario. The applicants then choose an appropriate course of action to indicate how they would deal with such situations or what actions they would take in response to the hypothetical situations. Responses can be either verbal or written (e.g., multiple choice). These tests are particularly well suited for aptitudes requiring decision-making.

Training and Experience Evaluation (T&E)

A Training and Experience Evaluation (T&E) is a structured assessment procedure used to evaluate a candidate's job-related training, education, experience, and accomplishments, as it relates to the KSAs as specified in the classification specification. TV&C creates various types of T&E's to include narrative supplemental applications and online examinations.

Work Samples and Performance Tests

Work samples are a broad class of assessment techniques that replicate job tasks and/or job behavior, are administered in a standardized manner, and allow for the observation of candidate behavior in performing the prescribed tasks. Several examples include In-Basket Exercises, Writing Assessments, Oral Presentations, and Performance Tests.

Written Examinations

A written examination is a testing procedure in which a candidate's job-related knowledge, skills, abilities and other characteristics (e.g., personality, customer service orientation, adaptability, etc.) are assessed through the use of a variety of item formats that may evaluate subject areas such as mathematics, reading, comprehension, grammar, spelling, and/or technical knowledge. 

​Survey Research

​By measuring behaviors or gauging attitudes, values and other characteristics through survey methods, an organization can come to a better understanding of the activities or perspectives of specific groups influencing business processes (i.e., employees, customers). Are there specific aspects of organizational culture that you need to know more about? Would you like to know more about the attitudes of your customers? What about the job satisfaction of current employees? Surveys administered before and after a change process can possibly show the effectiveness of a program. By hiring professionals trained in survey research, you can avoid many pitfalls and collect more meaningful information about the workforce. TV&C customizes surveys to fit the employer's needs, or work the methodology into other projects (e.g., attitudes about the performance appraisal system). 

​Training

TV&C provides training and consultation to departments in the area of selection through the Selection Analyst Certification Program, TV&C's Learn, Apply & Build (LAB) Curriculum, TV&C's Test Talk Seminars, and specialized training.

Selection Analyst Certification Program

The Selection Analyst Certification Program provides selection analysts with the foundation to build knowledge and skillsets in the area of selection. Further information on this training program, please visit: Selection Analyst Certification Program

LAB

LAB is a hands-on training series designed to enhance the knowledge and expertise of testing professionals within the State. LABs are structured to guide participants through the job analysis and examination process.

Test Talk

Test Talk features presentations on selection topics, followed by an open discussion forum. This is an opportunity to share the successes and challenges of testing and selection from different agencies.

​Frequently asked questions (FAQ) 

 

 

01. What is a job analysis? 01. What is a job analysis? <p>​A job analysis is typically defined as a comprehensive, rigorous approach to identifying and describing the important aspects of a job. While the definitions of job analysis vary in detail and level of specificity, they share their emphasis of systematically analyzing and evaluating the important aspects of a job. The primary goal is to describe work behaviors in performing the job, along with the essential requirements of the job.</p>
02. Why conduct a job analysis?02. Why conduct a job analysis?<p>​Collecting job analysis data guarantees that an organization has the most reliable and up-to-date information about a job from which to make and legally defend important employment and management decisions. CalHR adopted Article 3.5, Section 50 in Title 2 of the California Code of Regulations which references the Merit Selection Manual. The Selection Manual provides exam analysts with the standards for decentralized departmental examination programs:<br>· The use of sound, job-related examination processes and individual selection procedures developed on job analytic data.<br>· The use of the appropriate selection procedures to assess those KSAs identified as important for successful job performance and required-upon-entry to the job.<br>· Documentation linking the content of the examination process and its individual selection procedures to the content requirements of the job classification.</p>
03. What is a job analysis used for?03. What is a job analysis used for?<p>​<br>A job analysis is used to support employment decisions such as:<br>· Training<br>· Personnel Selection<br>· Performance Appraisal<br>· Recruitment<br>· Screening<br>· Workforce Planning<br>· Workplace Accommodation<br>· Appeals </p>
04. Who is qualified to conduct a job analysis?04. Who is qualified to conduct a job analysis?<p>​A job analysis may be conducted by a human resources representative, a trained job analyst/consultant, or someone who is familiar with job analysis procedures.</p>
05. How is a job analysis conducted?05. How is a job analysis conducted?<p>A job analysis is a multi-step process and several tasks must be completed before arriving at a final product. There is no one correct procedure or method for a job analysis. However, there are several guided steps that should be taken by all job analysts to ensure quality standards and legal defensibility.</p>
06. What sources provide guidance for conducting job analyses?06. What sources provide guidance for conducting job analyses?<p>· <a href="http://www.uniformguidelines.com/uniformguidelines.html" target="_blank">The Uniform Guidelines of Employee Selection Procedures </a>promotes a uniform set of principles to help employers comply with Federal laws that prohibit discrimination.<br>· <a href="http://www.siop.org/history/crsppp.aspx" target="_blank">The Society of Industrial/Organizational Psychology’s Principles </a><br>· <a href="http://www.apa.org/ethics/code/" target="_blank">The American Psychological Associations Standards</a></p>
07. Should tasks be eliminated if the importance of the task is rated high but the frequency is low?07. Should tasks be eliminated if the importance of the task is rated high but the frequency is low?<p>​Typically, only those tasks that are performed most frequently and rated as important, are termed as being critical. However, there are still important tasks that are performed infrequently. Take for example, the ability of a police officer to fire a gun. Although this task may be performed only once in a while, it is still an important ability to possess. Therefore, analysts will have to use discretion regarding tasks with high importance but low frequency.</p>
08. Who is considered a subject matter expert (SME)?08. Who is considered a subject matter expert (SME)?<p>​SMEs are people who have a thorough knowledge of the work behaviors, activities, responsibilities, and the prerequisite KSAs for effective job performance. The SMEs should include persons who are fully knowledgeable about relevant organizational characteristics such as shift, location, type of equipment used, and so forth. This typically includes people at the current classification level or higher and supervisors that are familiar with the classification. SMEs can also be people who were previously part of the classification.</p>
09. How many task statements are necessary to describe a job?09. How many task statements are necessary to describe a job?<p>​According to the book “Job Analysis” written by Michael T. Brannick and Edward L. Levine (2002), the number of task statements necessary to describe a job is generally 30 to 100 tasks, organized into 5 to 12 major duty or function categories. Brannick and Levine further recommend that if the number of tasks is substantially more than 100, similar tasks should be combined whenever possible to reduce the number back to 100 or less. The authors advocate the same range of statements when compiling knowledge, skills, and abilities (KSAs) statement lists.</p>
10. What does KSAPC stand for?10. What does KSAPC stand for?<p>​The acronym KSAPC stands for Knowledge, Skill, Ability and Personal Characteristics, aka KSAs.</p>
11. What is the difference between abilities and skills?11. What is the difference between abilities and skills?<p>​Unfortunately, there is no professional or academic consensus regarding the definition of skills and abilities within a job analysis. Naming conventions change across professionals, and such naming conventions can be interchanged.<br>TV&C, however, utilizes a specific definition that closely ties to that stated in the Uniform Guidelines on Fair Employment and Selection. Abilities are defined as a present competence to perform an observable behavior or a behavior that results in an observable product. Abilities typically require knowledge, and describe the capacity to perform cognitive activities. For example, one may have the ability to balance accounts using generally accepted accounting principles. A skill is an observable competency to perform a learned psychomotor act which requires a degree of precision. For example, the skill to type 45 words per minute is a psychomotor behavior with a degree of precision. It is easy to identify a skill under this definition because of the manual manipulation that it involves, but definitions vary. In general, utilizing the definition in one way or another will not compromise the integrity of the job analysis.</p>
12. What should be done to a KSA that is deemed important, but not required upon entry?12. What should be done to a KSA that is deemed important, but not required upon entry?<p>​Any KSA that is deemed important, but not required upon entry is still an important job qualification. Although these KSAs are not appropriate to be used in the examination process, it can provide valuable information in other areas such as training and development of performance evaluation dimensions.</p>
13. What do you do when statements are removed from the final job analysis due to their ratings, but the subject matter experts (SMEs) are resistant and would like to keep them?13. What do you do when statements are removed from the final job analysis due to their ratings, but the subject matter experts (SMEs) are resistant and would like to keep them?<p>​When a task or a KSA statement is removed from the job analysis, it does not necessarily imply that they are unimportant to job incumbents or to specialized job assignments. Rather, it implies that they do not generalize well enough to be considered as representative of the job position overall, or that they may not be expected upon entry to the job.<br>It may be best to conceive of scale cut-off scores as guides with some room for interpretation/analysis. Nevertheless, if the data clearly indicate that one or more KSA statements should not be used for the purpose of developing selection instruments, then they should not be included based on verbal input from subject matter experts alone.</p>
14. How long does it take to complete a job analysis?14. How long does it take to complete a job analysis?<p>​The time it takes to complete a job analysis depends on many factors, including the size of the classification, resources, geographical locations of worksites, availability and number of subject matter experts, etc. Therefore, there is no general timeframe for how long a job analysis will take to complete. Contact TV&C to discuss possible job analysis timelines based on organizational needs.</p>
15. Is the job analysis material confidential?15. Is the job analysis material confidential?<p>​Job analysis data is not confidential and individual departments are welcome to share job analysis information/data with other departments. Therefore, it is not necessary for participants to complete security forms or clearances. However, once discussion crosses into examination development, all information is confidential; participants will need to sign appropriate security and confidentiality agreements. Similarly, since incumbents can share sensitive information regarding their jobs, it is imperative that analysts be respectful during this process. Although the information shared is not technically confidential, it should not be able to be linked back to any one individual. This helps to encourage incumbents to openly discuss and share information regarding the true nature of their job tasks, knowledge, skills, and abilities that they might not otherwise share if they are aware that their responses are not confidential. </p>
16. Does a department need to conduct a job analysis before administering an exam?16. Does a department need to conduct a job analysis before administering an exam?<p>​ Yes, the Selection Manual states that the job analysis shall serve as the primary basis for demonstrating and documenting the job-relatedness of examination processes conducted for the establishment of eligibility lists within the State’s civil service. Every examination developed by a department should be based on a properly conducted job analysis.Yes, the Selection Manual states that the job analysis shall serve as the primary basis for demonstrating and documenting the job-relatedness of examination processes conducted for the establishment of eligibility lists within the State’s civil service. Every examination developed by a department should be based on a properly conducted job analysis.</p>
17. If the KSAPCs identified in the job analysis aren’t in the class specification can they be tested for?17. If the KSAPCs identified in the job analysis aren’t in the class specification can they be tested for?<p>​It is CalHR's policy that KSAPCs used in examination development should fall under the intent and scope of the KSAPCs presented in the class specification. The class specification KSAPCs were written to be intentionally broad so that they would not be limiting. If it is discovered through a job analysis that the class specification no longer represents that job, then a revision of the class specification is recommended. In the case of a servicewide classification, this process should include collaboration from the other agencies that use the class.</p>
18. Can an employer be sued for administering an exam that is derived from an inadequate job analysis or from an exam that was not derived from a job analysis at all?18. Can an employer be sued for administering an exam that is derived from an inadequate job analysis or from an exam that was not derived from a job analysis at all?<p style="text-align:left;">​If the content validation approach was used, then yes. Departments must verify legal defensibility of selection processes. The Uniform Guidelines of Employee Selection Procedures requirements for conducting job analyses include identifying the critical work behaviors or tasks and KSAPCs that comprise successful performance of the job. Only those tasks and KSAPCs deemed important should be used as the basis of selection. Furthermore, Title VII of the Civil Rights Act of 1965 specifically prohibits making any employment-related decisions based on an employee’s race, color, religion, sex, or national origin. The Act further prohibits the use of a selection procedure that results in adverse impact unless the employer can demonstrate the job-relatedness of such a procedure.</p><p style="text-align:left;"> The passage of the Americans with Disabilities Act (ADA) in 1990 further emphasized the importance of conducting job analyses by requiring employers to evaluate candidates’ capabilities to perform the essential functions of the job. The ADA considers the preparation of a written description (job analysis) of the essential functions before advertising or interviewing applicants as evidence of those essential functions. By adhering to the principles of the Uniform Guidelines and the requirements of relevant case law, job analyses enable organizations to make valid and cost effective employment-related decisions while avoiding potential lawsuits.</p><p style="text-align:left;">The penalty for disregarding the validation requirements for selection instruments can be costly. A job analysis will help verify that selection procedures are defensible, which makes it imperative that departments conduct sound job analyses.</p>

 

 

Marquez Lopezhttp://www.calhr.ca.gov/state-hr-professionals/Lists/Contacts/DispForm.aspx?ID=4ChristinaMarquez LopezManagerchristina.lopez@calhr.ca.govCalifornia Department of Human Resources

  Updated: 2/8/2016
One Column Page
Link Back to Top