How To Get Started

Pro Tip from The Caviart Group: Many groups only interview industry professionals when conducting feasibility studies, but there’s another important group to interview: the “customer’s customers.” In other words, interview those who hire the professionals. The needs of those hiring the professionals will greatly impact the needs of the professionals.

Step 2: Design the Certification Program

The next step is to dive into the details of exactly how your certification program will function. A certification program is more than a test. It encompasses everything from marketing to professional standards. The program design helps to determine how every step in the credentialing process, from application to assessment and re-certification, should be structured to best serve the market and to work effectively and efficiently within your organization.

Design for Financial Success

While the financial feasibility of a program is assessed as a part of the feasibility study (Step 1), how the credential program is designed and implemented will determine the actual success or failure of the program.  In this step (Step 2) it is critical that detailed and realistic revenue and expense projections be made. On the revenue side it is important to consider the cost/benefit perception of potential candidates and to factor in the candidate credential adoption curve. That is, how long it will take for the industry to accept a new credential and for applicants to go beyond the early adopters. These factors will be greatly influenced on the quality and breadth of the credential promotion program. A well designed and executed marketing program is also a must. “If you build it, they will come” is just not true in certification.

This phase also includes the design of the assessment process. Although multiple-choice tests are often the least expensive to create, they may also be the least appealing for the potential candidate population. This is especially true for credentials aimed at upper-level professionals who need to be tested for professional judgment and decision making.

It is vital you have the right, experienced professionals involved in this stage of the process. A lot of organizations only seek testing advice for a new credential and frankly few certification consultants are qualified to do much more than that in this phase of the process.

We believe that the design phase is one of the most important parts of the credential development process and we are one of the few consulting companies that can provide comprehensive advice and guidance to help ensure our clients’ success.

Questions Answered During This Phase

Listed below are just a few of the high-level questions addressed in the program design, with several other questions beneath each. The program design is completed through meetings with stakeholders across the credentialing group to ensure all aspects of the program have been considered prior to development and launch.

1. How will the certification program be administered?

2. How will the examination be created and administered?

3. How will the program be marketed and promoted?

4. How will professional conduct issues be reviewed and resolved?

5. What is the financial feasibility of creating and maintaining a certification program?

6. What is the time frame for program development?

7. How will risk management be handled? (i.e. Legal issues/insurance, etc.)

Pro Tip from The Caviart Group: It typically takes a new certification program at least three years to be profitable. Groups must plan financially for supporting a certification program through its development to see success in the end.

Step 3: Develop The Certification Program

After the certification program design is completed, it’s time to put in place the necessary infrastructure. Many of the decisions made in the previous steps will be put into action now. Each component of the program is created and documented, including policies and procedures, administrative processes and systems, test delivery and scoring systems and marketing programs.

• Establish vendor relationships.  Select the vendor(s) for test development and test delivery.

• Start marketing the credential.  Marketing begins with the goal of driving awareness of the new credential, including launching a website with relevant information and developing excitement about what the credential means to the profession.   

• Set up the certification application and delivery system.  Systems related to the application and examination processes are developed and tested.

• Put professional rules in place. Rules are finalized and posted regarding everything from professional conduct, to continuing education/recertification and certification revocation/reinstatement processes.

Pro Tip from The Caviart Group: Certification management systems are quite different from membership management systems and organizations can waste a lot of staff time if the certification system is not well designed.

Step 4: Develop the Certification Examination

Now it is time to start developing the certification examination. To ensure legal defensibility of the examination, it is important to complete each of these steps to research and develop the test specifications, test design, the examination itself, scoring and test appeals systems.

• Perform a Job/Task Analysis – A job/task analysis (JTA) is the search for truth, as to what professionals actually do and need to know to perform competently. A JTA study must be designed not only to seek this truth but also to provide the information required to ensure the legal defensibility of the results.

• Design the Test – The results from the JTA are used to design a plan for the examination which identifies the topics that will be covered on the examination. The plan will also specify the number of test questions that will appear on the examination from each of the tasks or topics included in the outline. Following the specifications in the exam blueprint will ensure that every edition of the exam covers the same subject matter in the same proportions. During this stage, you will also determine the format of exam questions to best assess the knowledge, skills or abilities being tested. Don’t assume that the test will be in a multiple-choice format.  There are many different item types, and it is important to select the best format to accurately assess candidates in your profession.

• Develop the Test - After the test has been designed, subject matter experts (SMEs) are recruited to write the test questions, or “items.” After the test items have been drafted, they are professionally edited and then peer reviewed by SMEs. The editing process ensures that the items are consistent in style and format, that they are grammatically correct, and that they clearly present the problem that the examinee is to solve.

• Perform Standard Setting - Once the test is created, a cut score study needs to be performed to determine the minimum score needed to pass. This too must be performed correctly to ensure legal defensibility of the test results.

Pro Tip from The Caviart Group: Multiple-choice questions are often not the best format for measuring many skills, especially high-level skills like professional judgment. Item types like our Case Analysis Items are a better fit for accurately assessing high-level skills and provide more measurement opportunities than traditional multiple-choice questions. Candidates also report Case Analysis Items more accurately reflecting what they do on a day-to-day basis.

Step 5: Develop a Test Maintenance and Support Plan

Test Maintenance

Any certification examination only addresses a fraction of the knowledge and skill required to perform the tasks of the profession.  Almost all tests sample across the required knowledge and skill spectrum with the belief that the knowledge and skills are highly related. Therefore, if a person performs well on a given set of knowledge and skill questions, they most likely possess the related knowledge and skills. The problem is that this assumption is only true if the candidate does not have prior information about the specific content of the examination.

Unfortunately, from the first day a new certification examination is administered, the specific content of the examination begins to become dispersed across the candidate population, and the sampling assumption starts to degrade. In addition, all professions continue to change as new laws, regulations, technologies and practices evolve.

For both reasons, it is important that a certification examination be continuously monitored, refreshed and updated on a regular basis.

At the Caviart Group, our psychometricians monitor and analyze initial and ongoing test results to ensure that items are performing correctly and that the tests continue to perform as expected. We also plan for regular updates of our examinations, replacing poorly performing items with new items that have performed well in pre-testing. This process ensures that the tests remain valid and psychometrically sound while continually improving the statistical performance of the examination.

Program Support/Growth Plan

In addition to keeping the test refreshed, it is important to continue marketing and supporting the certification program to attract new candidates and promote the credential of certified professionals in the industry. The goal here is to move the credential to the point where it becomes the default prerequisite for entry to the profession.

There are three groups to focus on: new candidates and certified professionals (often known as the “customer”) and those who hire the certified professionals (the “customers’ customer”.) Marketing to new candidates should explain the benefits of the credential to their career growth.  Certified professionals should be encouraged to provide testimonials discussing how the credential has helped them advance.  They should also be encouraged to display and discuss their credential with other professionals. As they move into positions of authority, you want them to encourage or even mandate the credential for new employees.

Everything that an organization does to advance their credential program is built on the credential’s credibility and perceived worth. Credibility is earned through a valid, fair and rigorous examination process, a well-run certification program, and a reputation that the credential develops within the profession as proof of competence of the credential holders.

Pro Tip from The Caviart Group: It is as important to refresh your marketing as it is to refresh your exam. The first wave of candidates will be early adopters who are open to attaining an “unproven” credential. The next wave of candidates, however, will have a different mindset and will need to believe that the advantages provided by the credential are worth the time and investment required to obtain and maintain the credential.

Partnering for Success

At The Caviart Group, we have 40 years of experience walking groups through the process of exploring, building and growing certification programs, as well as licensure programs. While many consultants focus only on test development, we can guide your group through the whole process of program development. The thoroughness of this process produces a test designed to be legally defensible and valid, and a certification program that is built for success. Please contact us at any time to discuss your needs.