Subject Matter Expert Participation Requirements

The purpose of this document is to outline the policy and requirements for subject matter expert (SME) participation in the job analysis and examination development process for Merit Systems Services (MSS) classifications used to fill program positions within Interagency Merit System (IMS) county department.

Consistent with legal and professional standards for merit systems, valid job analyses as the foundation for the development of examinations for all job classifications are required by MSS.  All examinations for program positions must adhere to MSS selection standards of validity and job-relatedness and must link directly to a valid job analysis. Examinations designed to establish eligible lists must be competitive and effectively assess the qualifications and relative knowledge, skills, and abilities of candidates for the position they seek.

County departments are required to provide SMEs to actively participate in the job analysis and examination development for all allocated MSS classifications within their classification plan. 

This participation ensures comprehensive representation and valuable feedback from all user-county departments, resulting in a more reliable and valid examination process. The contribution of SMEs is vital for maintaining the relevance and accuracy of the job content and developing robust, job-related testing requirements.

When selecting SMEs to participate in the job analysis or exam development process, counties must ensure they mee the following criteria:

• SMEs are expert-level incumbents, supervisors, or individuals with in-depth knowledge of the job duties and requirements.
• They should be top performers in the role, first-level supervisors or hiring managers.
• SMEs must have direct experience/knowledge of the job and understand what is required to succeed in the role.
• They must have successfully completed their probationary period.

The level of county involvement in the job analysis process is determined by the total classification size (defined as the total number of positions allocated across all counties utilizing the classification) and the extent to which counties use that classification. For classifications widely used across many counties, participating in any phase of the job analysis process will satisfy the county participation requirement. Conversely, if a classification is utilized by only a few counties, participation in multiple phases becomes essential.

Participation in the job analysis survey is guided by the classification size, as outlined in the table below. The required percentage of job analysis respondents varies based on the number of incumbents in the classification. For example, if the classification size is 25, an 80% response rate is necessary to ensure sufficient data collection. MSS will communicate the specific number of complete surveys required from each county.

County departments that do not provide SMEs or fail to participate in the job analysis process will be prohibited from using MSS examinations. As a result, counties that opt not to participate may encounter significant challenges in recruiting for program positions.

Should a county department that has not participated in the job analysis process wish to utilize the MSS examination in the future, the county will be required to submit a valid job analysis that demonstrates a direct linkage to the MSS job analysis prior to being granted the ability to use the examination.

The extent of county involvement in the examination development process depends on the classification size and type of examination. MSS may request participation from four to ten SMEs across various counties. It’s important for counties to provide SMEs when requested. The examination development process is where the testing items, rating criteria, and passing score are developed. This process directly affects county candidate pools. The inclusion of a diverse group of SMEs bring a range of perspectives, enhancing the fairness and comprehensiveness of the exam content. Counties and other parties will not be permitted to review examinations or provide input outside of the examination development process.

• The extent of county involvement int he job analysis process is determined by the number of counties that are using a specific classification. If a classification is used by only a few counties, they will need to participate in multiple phases of the job analysis process. Conversely, if a classification is used by many counties, they will be required to participate in any phase of the job analysis process to fulfill the participation requirement.
• Job Analysis Survey: Participation in the survey is contingent on the classification size in each county. For instance, if the total classification size (sum of SMEs across all counties using classifications is 25, then an 80% response rate is necessary to gather sufficient data. MSS will send an email specifying the number of completed surveys required for each county.

• California Code of Regulations, Title 2, Section 17033
• Government Code, Section 19800
• Government Code, Section 19801