Gary Spivack, Senior Product Manager
July 9, 2024

Artificial Intelligence (AI) is revolutionizing the assessment industry, offering unparalleled efficiency, personalization, and security. Here are a couple of areas in which AI is making a difference:

Personalized Learning:

  • Enhance student success with AI-powered adaptive learning platforms that tailor assessments to individual needs and adjust in real-time based on performance.

Data-Driven Insights:

  • Leverage AI analytics to uncover trends and pinpoint areas for improvement, helping make informed impactful decisions.

Secure Testing:

  • Ensure the integrity of remote exams with AI-enhanced proctoring systems that monitor for cheating and irregularities using advanced facial recognition and behavior analysis.

Content Management:

  • Benefit from AI’s natural language processing capabilities to detect similarities, which aid in identifying enemy items.

Enemy Analysis

To identify enemy items, natural language processing (NLP) and other AI-driven methods are used. These technologies analyze the language and content of potential test items to detect similarities and dependencies that might not be immediately obvious to human test creators. By identifying these enemy items, test creators can ensure the assessments are fairer and more reliable.

The main goal is to maintain the integrity and accuracy of assessments by preventing inadvertent clues or redundancy in test items, which could compromise the validity of the test scores.

We’re thrilled to announce an exciting release that elevates Pro! to the next level of sophistication in Natural Language Processing (NLP). Our renowned Cosign Similarity feature has been upgraded with advanced Semantic Similarity. This is the first of many planned Artificial Intelligence (AI) enhancements utilizing large language models (LLM) to assist in creating and managing items and examinations.

Semantic Similarity Overview

Semantic similarity measures how similar the meanings of two documents or terms are, rather than just comparing the words they use. It’s a mathematical way to estimate how closely related different pieces of language, concepts, or instances are. Unlike semantic relatedness, which includes any type of connection between terms, semantic similarity focuses only on “is a” relationships. For example, “car” is similar to “bus” because both are vehicles, but “car” is also related to “road” and “driving” in a broader sense.

See the blog by Jay Myers Fortunato Favors the Bold: An Exploration of Enemy Analysis, which describes why automated tools are extremely useful when looking for potential item enemies.

Pro!’s Approach  

Unlock the power of advanced analytics with our cutting-edge semantic similarity feature! From a list of items or forms, effortlessly check the semantic similarity between items or passages to identify potential enemies. Choose your comparison options and model, and quickly obtain precise similarity ratings. Our process seamlessly handles combinations of item set stimuli/passages, associated items, and discrete items. You have complete control of checking specific items within the set or excluding traditionally friendly items. This secure, server-run process delivers a detailed spreadsheet with comprehensive comparison results and similarity settings. Elevate your item analysis today and stay ahead of the curve!

Future

As AI revolutionizes the assessment industry, making evaluations more personalized, efficient, and fair, Strasz Assessment Systems and Pro! are at the forefront of this transformation. We’re excited to develop and prototype cutting-edge AI-powered solutions that will elevate the testing industry. Stay tuned for future innovative advancements!

About Pro!

Our enterprise item bank offers collaborative and systemized management of the entire exam development process. Pro! enables unlimited users to author and edit unlimited items anywhere across the internet. Use Pro!’s item development workshop functionality, and allow your staff to communicate with item writing and review teams. Write traditional multiple-choice items or design complex simulation items with Pro!. Assign metadata to the items, including classifications, rationales, comments, references, and user-defined fields through user-friendly, easy-to-navigate screens.

Contact us for more information.

About the Author

Gary Spivack has worked in the testing industry since the mid-90s as a software engineer, developing complex software systems that span item authoring, examination delivery drivers, scoring, and reporting systems.  His drive to have systems be best-of-breed in the industry inspired him to become the product owner for Strasz Assessment System’s suite of products.  When not at work, Gary continues self-improvement through martial arts and, of course, follows the Philadelphia sports teams.