John DeFalco, Software Developer
March 1, 2017
Before computer-based testing became so prevalent, exam content was created on paper and locked in secure file cabinets to prevent it from being compromised. The author of this blog post explains what steps can you take to keep your content secure in the digital age.
In the testing and assessment industry, content is king. Content is what laymen outside the industry generically refer to as the “questions” and “answers” that comprise a test. Inside the industry, there are actually several specific terms that refer to these items. There may also be some stimulus information like a case history or a visual element that leads into the stem or question, which is the heart of what is being tested. In a multiple-choice exam, one of the choices is the correct answer and the other choices are called distractors. All of these parts of an exam are valuable content and they all require protection.
Creating exam content is generally not an easy process. Many hours go into the creation of fair and balanced items that test an individual’s cognitive abilities in a specific area. The lifecycle of developing an exam item includes work to define the type of content that is appropriate, item authoring by experts in the area of the credential, and review by experts in the subject area and in test development principals. The roles that support the creation of an exam are varied and include psychometricians, editors, rubric authors and subject-matter experts (SMEs). Due to the amount of work, the development process can be costly. Even if the subject matter experts are volunteers, writing items and maintaining inventory in the item bank can be costly in terms of time spent. Exam content is an investment that deserves protection from security breaches.
Exams can be categorized as high stakes, low stakes and no stakes. Focusing on the first category, high stakes exams are ones that carry with them important consequences for the test-taker. Examples include a certification, such as becoming a CPA, lawyer or doctor, or earning a driver’s license or a high school or college diploma. In certain professional occupations, passing a test brings with it a new job title and responsibilities, a new pay grade or bonus or all of the above. The positive effect that a certification can have on a career provides the motivation for some candidates to attempt to cheat on an exam by obtaining the exam content.
The intention of investing a large amount of time into developing content for high stakes exams is to ensure that individuals who pass the exams are qualified in the subject-matter that is being tested, ultimately protecting the public from candidates who are not qualified. I don’t think anyone would want to drive our nation’s roads if they handed out full-privilege driver’s licenses to every teenager who reached the age of 16 without requiring them to demonstrate safe driving skills. In the same line of reasoning, any sane person would greatly prefer their taxes to be prepared by a competent individual who understands how to properly file tax forms. The haphazard inclusion of figures on income tax forms would assuredly end in an unpleasant scheduled meeting with the good folks at the IRS. As you can probably understand by now, these are the reasons why securing exam content is incredibly important.
In the days before computer-based testing became so prevalent, exam content was created on paper and locked in secure file cabinets to prevent it from being compromised. Only select individuals had access to the content and stealing it would have involved breaking into the file cabinet and/or potentially taking pictures of the questions and answers. Fast-forward to today when almost every exam is either fully or largely created and processed by computers. You can even take an exam on your home computer through an online proctoring site. There is the ever-present threat of corporate espionage and international hacking, but the fact is, exam content is just as susceptible to internal theft as it is to external sources. The point that the Mission Impossible and Ocean’s Eleven movies have driven home in an entertaining fashion is that nothing is ever 100% secure. While that is true, there are steps that can be taken to put your content into the digital equivalent of Fort Knox.
Limit access
Just like in the days of paper, only give the “digital keys” to the people who require access. For example, why allow the receptionist who has no direct function in the exam’s creation to see the content? Sure, they might be under the same non-disclosure agreement as content creators but if they don’t require access, don’t risk it. This also includes the need to limit access to the content persisted on server databases such as SQL Server. Don’t permit every domain user to have administrator access to the database.
Segment the access by role
Content creators cannot create or edit reviews. Rubric authors cannot create or edit content. Reviewers cannot create or edit rubrics. This goes a long way to fragmenting the access and locking down individuals from seeing “the big picture”.
Checks and balances
In the days of paper, someone would have to spend a day at the copier to get your entire bank of content, which would almost definitely alert someone at some point in time. Nowadays, with a thumb drive and high-speed Ethernet, they can pick your bank clean inside of 15 minutes and go undetected in the process. Create a system of checks and balances for any function that allows the bulk access to content. For example, if there is a need to print content for an entire section of the exam in order to review it for obsolescence or provide to SMEs, gate this function with pre-approvals and limit the time in which the requester can print the content. In addition, in the worst case scenario that content does get stolen, there is also an electronic record of the action to submit in court.
Perform online reviews
When exams were only on paper, the content had to be photocopied or printed and distributed to reviewers, both internally and externally. This meant exam content was traveling around the country or world via various postal systems. While federal law mandates that tampering with mail a serious crime, it is better to be proactive than reactive. Performing online reviews with geographically disparate team members requires a much higher category of skills to pull digital information off the line. The use of VPN tunnels and SSL technologies makes the threat of theft almost non-existent.
In conclusion, while there are no foolproof methods to completely eliminate the theft of exam content, there are many precautions that can be taken to secure it. The football coach Vince Lombardi is quoted as saying “Perfection is not attainable. But if we chase perfection, we can catch excellence.” Strive to chase perfection in securing your exam content knowing that, if you merely attain excellence, you’ve gone a long way to maintaining the integrity and viability of your exam.
John DeFalco, Software Developer
March 1, 2017
Before computer-based testing became so prevalent, exam content was created on paper and locked in secure file cabinets to prevent it from being compromised. The author of this blog post explains what steps can you take to keep your content secure in the digital age.
In the testing and assessment industry, content is king. Content is what laymen outside the industry generically refer to as the “questions” and “answers” that comprise a test. Inside the industry, there are actually several specific terms that refer to these items. There may also be some stimulus information like a case history or a visual element that leads into the stem or question, which is the heart of what is being tested. In a multiple-choice exam, one of the choices is the correct answer and the other choices are called distractors. All of these parts of an exam are valuable content and they all require protection.
Creating exam content is generally not an easy process. Many hours go into the creation of fair and balanced items that test an individual’s cognitive abilities in a specific area. The lifecycle of developing an exam item includes work to define the type of content that is appropriate, item authoring by experts in the area of the credential, and review by experts in the subject area and in test development principals. The roles that support the creation of an exam are varied and include psychometricians, editors, rubric authors and subject-matter experts (SMEs). Due to the amount of work, the development process can be costly. Even if the subject matter experts are volunteers, writing items and maintaining inventory in the item bank can be costly in terms of time spent. Exam content is an investment that deserves protection from security breaches.
Exams can be categorized as high stakes, low stakes and no stakes. Focusing on the first category, high stakes exams are ones that carry with them important consequences for the test-taker. Examples include a certification, such as becoming a CPA, lawyer or doctor, or earning a driver’s license or a high school or college diploma. In certain professional occupations, passing a test brings with it a new job title and responsibilities, a new pay grade or bonus or all of the above. The positive effect that a certification can have on a career provides the motivation for some candidates to attempt to cheat on an exam by obtaining the exam content.
The intention of investing a large amount of time into developing content for high stakes exams is to ensure that individuals who pass the exams are qualified in the subject-matter that is being tested, ultimately protecting the public from candidates who are not qualified. I don’t think anyone would want to drive our nation’s roads if they handed out full-privilege driver’s licenses to every teenager who reached the age of 16 without requiring them to demonstrate safe driving skills. In the same line of reasoning, any sane person would greatly prefer their taxes to be prepared by a competent individual who understands how to properly file tax forms. The haphazard inclusion of figures on income tax forms would assuredly end in an unpleasant scheduled meeting with the good folks at the IRS. As you can probably understand by now, these are the reasons why securing exam content is incredibly important.
In the days before computer-based testing became so prevalent, exam content was created on paper and locked in secure file cabinets to prevent it from being compromised. Only select individuals had access to the content and stealing it would have involved breaking into the file cabinet and/or potentially taking pictures of the questions and answers. Fast-forward to today when almost every exam is either fully or largely created and processed by computers. You can even take an exam on your home computer through an online proctoring site. There is the ever-present threat of corporate espionage and international hacking, but the fact is, exam content is just as susceptible to internal theft as it is to external sources. The point that the Mission Impossible and Ocean’s Eleven movies have driven home in an entertaining fashion is that nothing is ever 100% secure. While that is true, there are steps that can be taken to put your content into the digital equivalent of Fort Knox.
Limit access
Just like in the days of paper, only give the “digital keys” to the people who require access. For example, why allow the receptionist who has no direct function in the exam’s creation to see the content? Sure, they might be under the same non-disclosure agreement as content creators but if they don’t require access, don’t risk it. This also includes the need to limit access to the content persisted on server databases such as SQL Server. Don’t permit every domain user to have administrator access to the database.
Segment the access by role
Content creators cannot create or edit reviews. Rubric authors cannot create or edit content. Reviewers cannot create or edit rubrics. This goes a long way to fragmenting the access and locking down individuals from seeing “the big picture”.
Checks and balances
In the days of paper, someone would have to spend a day at the copier to get your entire bank of content, which would almost definitely alert someone at some point in time. Nowadays, with a thumb drive and high-speed Ethernet, they can pick your bank clean inside of 15 minutes and go undetected in the process. Create a system of checks and balances for any function that allows the bulk access to content. For example, if there is a need to print content for an entire section of the exam in order to review it for obsolescence or provide to SMEs, gate this function with pre-approvals and limit the time in which the requester can print the content. In addition, in the worst case scenario that content does get stolen, there is also an electronic record of the action to submit in court.
Perform online reviews
When exams were only on paper, the content had to be photocopied or printed and distributed to reviewers, both internally and externally. This meant exam content was traveling around the country or world via various postal systems. While federal law mandates that tampering with mail a serious crime, it is better to be proactive than reactive. Performing online reviews with geographically disparate team members requires a much higher category of skills to pull digital information off the line. The use of VPN tunnels and SSL technologies makes the threat of theft almost non-existent.
In conclusion, while there are no foolproof methods to completely eliminate the theft of exam content, there are many precautions that can be taken to secure it. The football coach Vince Lombardi is quoted as saying “Perfection is not attainable. But if we chase perfection, we can catch excellence.” Strive to chase perfection in securing your exam content knowing that, if you merely attain excellence, you’ve gone a long way to maintaining the integrity and viability of your exam.