As previously announced, FITT will be applying for ISO 17024 accreditation. As part of the accreditation process, FITT’s certified program (CITP designation) is validated by an independent, unbiased accreditation body—the International Organization for Standardization (ISO) through the Standards Council of Canada —to ensure it operates in a consistent, defensible, and reliable manner.
The accreditation process is intentionally a long and laborious one. To create a certification program that will meet ISO’s conformity assessment, many steps must be completed to create fair and accurate standards of a world-class caliber, including independent third-party reviews, in-depth gap analysis, competency development, and program governance. The meticulous nature of ISO 17024 accreditation should come as no surprise as the goal is to truly showcase and prove an organization’s credibility and commitment to its industry. By doing this, the organization is proving itself as a trusted leader in the field.
As FITT is working diligently through the accreditation process for the CITP designation, we decided to give readers an inside look at what really goes on behind the scenes. To do that, we interviewed Lyne Marcil, Director of Psychometric Services at Emerit Consulting (Tourism HR Canada) and lead psychometrician assisting FITT in meeting ISO 17024 requirements—specifically activities related to the review/update of the FITT International Trade Competency Framework, FITT CITP Competency Profile, and CITP Professional Examination. Learn more about her and her work with FITT below:
Can you explain what psychometrics is as a field of study?
Psychometrics is a specialized branch of psychology concerned with the quantification and measurement of mental attributes, behaviour, performance, and the like, as well as with the design, analysis, and improvement of the tests, questionnaires, and other instruments used in such measurement (from American Psychological Association).
How does psychometrics as a study pertain to gap analysis work FITT is doing now in relation to the ISO 17024 process?
When developing assessment tools, like a certification program or exam, a very rigorous process must be followed to ensure the resulting tools are valid, reliable, and fair—the ultimate criteria to evaluate the quality of assessment tools. For the ISO 17024 accreditation, an independent third party reviews the assessment tool development process from the initial job analysis (competency development) through its administration, including program governance and all related policies and procedures, to confirm it meets all the criteria ISO set forth to ensure a high-quality certification program.
The official requirement to meet the ISO 17024 standard, as a first step, is to conduct a job analysis—that is to define the occupation/profession and identify the knowledge, skills, and abilities required by practitioners. In FITT’s case, given the fact that, one, the extensive and rigorous job analysis was conducted 6 years ago when the FITT International Trade Competency Framework (the Framework) was initially developed and, two, the field of international trade has remained relatively stable in terms of the skillset required from its practitioners, it was not necessary to complete a full job analysis by starting from scratch. Because the Framework was already developed, we opted to conduct a gap analysis first. It is not a requirement per se, but in this case made sense because the previous work conducted by FITT, the Framework, needed a refresh rather than starting from scratch.
What is the goal of a gap analysis?
The purpose of the gap analysis is to inform the Framework update by assessing how the field of international trade has evolved since the last iteration of the Framework, i.e., what changes took place and their impact on skills and knowledge required by the professionals in the field. Similar to the job analysis, we also want to be forward-thinking and identify trends and anticipated changes that may have an impact in the coming years on how the profession is practiced and the skillsets required by international trade professionals.
Want to learn more about The Certified International Trade Professional (CITP|FIBP) designation? Discover why it’s the world’s most recognized professional designation for those ready to take on the world.What were the main steps in performing this gap analysis?
The initial steps were to conduct secondary research by reviewing national and international standards related to global trade, recent reports on the state of international trade, and mainstream media reporting to identify changes that took place or that are imminent in the profession. A series of interviews with prominent Canadians involved in international trade was also conducted to confirm the research findings and obtain additional information. From this preliminary research and comparison against the Framework, a list of potential competencies to be added to the Framework was identified. These results were validated through consultation with the FITT Certification and Accreditation Committee (CAC) and FITT’s Content Advisory Panel (CAP).
What do you need to do to ensure the gap analysis is useful?
We use the gap analysis recommendations as a starting point for the next steps when updating the competency Framework. We know that some of these recommendations, either during the competency drafting stage or focus group review, will change and be adjusted. The competency development process is a very iterative process, with changes throughout all its steps, as we are consulting with various stakeholders along the way and incorporating their feedback and newly discovered information from additional research.
The gap analysis is definitely useful as it guides the development of additional competencies to be incorporated into the Framework. It allows us to focus on these competencies and be more efficient in the job analysis process.
What are the biggest benefits of doing a gap analysis and how are FITT’s focus groups integral to this process?
Focus groups are great and necessary opportunities to get input and seek feedback from a number of Subject Matters Experts (SMEs). In FITT’s case, they are practicing international trade practitioners, all with their unique experiences and backgrounds, working for various businesses in a wide range of settings and countries. The involvement of SMEs throughout a certification program development process is one of the key psychometric principles and a requirement for the ISO designation. We need to have enough consultation with SMEs built into the development process to ensure that, in the end, we have included a minimum of participants that are representative of international trade practitioners in the field.
Through a series of focus groups, each with various participants, all the competencies—both old and new —are reviewed in detail. This is an opportunity to update the “old” competencies and to make adjustments to reflect new terminology. We also update certain examples to be more current, clarify certain statements, and may add a performance indicator or knowledge requirement. Extensive discussions also take place for newly proposed competencies to confirm their relevance, their structure/placement in the Framework, and their definition. This is also an opportunity to answer questions or obtain clarifications for the analysts/facilitators. All the SME feedback is incorporated in the next iteration of the competencies. At times, additional research and further consultation are conducted to address issues raised during the focus groups.
Do you have any success stories from organizations that have used psychometric services and gap analyses to improve their processes?
Any licensure or professional certification program relies on, or should rely on, the expertise of psychometricians. The methodology for assessment development and all policies and procedures supporting a certification program must be carefully designed and adhered to at all stages to ensure a resulting program that is valid, reliable, and fair—regardless of whether the certifying body is applying for ISO accreditation or not. Decisions that have an impact not only on one’s career and livelihood but also on their clients and the general public, are made on the basis of the assessment process, therefore, we need to ensure this process is of high quality, meeting all the psychometric requirements.
Are there any learnings that you could pass on from this work?
An important learning would be highlighting how critical the involvement of SMEs is to the entire certification process. Not only is SME participation a psychometric requirement to ensure a representative sample of practitioners have been consulted to deliver a high-quality assessment process/tools, but it is also a great opportunity for them to contribute and shape their profession. The SMEs often become the biggest defenders and best champions for the program—they play a significant role in promoting the value of the certification program to the profession through informal discussions with their peers and colleagues.
To date, 60 subject matter experts have volunteered their time to the review and update of the Framework, and we expect another 100 or so to participate in the remaining ISO 17024 project activities! Thank you all!
Thank you for your insights. They really showcase the fastidious nature and depth of this entire process. We’re extremely proud to be part of it and can see that you are too.
Curious about next steps? Here they are!
Following the approval of the updated Framework by the Exam Development Committee, a thorough analysis will be completed to identify required updates to the CITP Competency Profile (Profile). Working in consultation with our network of CITPs, we will identify which skills and subskills (and related performance and knowledge pieces) should be represented in the revised Profile. Once the Profile updates are approved by the Exam Development Committee, we will move forward with updating the CITP Professional Exam blueprint and content. Subsequently, the FITTskills curriculum will need to be updated to ensure continued alignment with the CITP competencies.
All steps above are required as part of the ISO 17024 accreditation requirements and provide validation that the certification program operates in a consistent, defensible, and reliable manner.
To learn more about the upcoming changes to the CITP certification process please visit https://fittfortrade.com/changes-citpfibp-certification-process.
Contact certification@fitt.ca for any questions about the CITP designation or ISO accreditation.
*Certified International Trade Professional (CITP) is trademarked for use within Canada. FITT International Business Professional (FIBP) is trademarked for use internationally. Both reflect the same FITT-certified designation.
disqus comments