New Hallo Validation Study Shows Strong Correlation Between AI and Human Language Assessment Scores
How can universities and organizations evaluate whether a language assessment produces meaningful, dependable proficiency scores? At Hallo, we believe the answer starts with research and transparent validation evidence.
We are sharing new findings from our ongoing validation work on the Hallo Language Test (HLT). In this study, automated proficiency scores showed a strong relationship with ratings assigned by independent, third-party trained human evaluators. The results provide encouraging evidence about the quality of Hallo’s scoring, an important component of evaluating the assessment for educational and professional use.
Key Findings from Hallo’s Validation Study
Our November 2026 white paper, Human–AI Score Correspondence, examines one important aspect of assessment validity: whether Hallo’s automated scores reflect the language proficiency patterns recognized by independent, third-party trained evaluators. Across 234 paired responses, Hallo scores showed a strong positive relationship with average human ratings (Pearson r = 0.89). The human–human reference correlation was also strong (mean r = 0.85, based on 210 responses). These are correlation coefficients, not percentages of correct scores or measures of exact agreement.
Assessment | AI vs. Human (r) | Human vs. Human (r) |
Hallo Language Test (HLT) | 0.89 | 0.85 |
Duolingo English Test (DET) – Speaking | 0.84 | 0.85 |
Duolingo English Test (DET) – Writing | 0.85 | 0.87 |
Note: Results are from independent studies with different samples and methodologies and are not directly comparable. Correlation reflects scoring patterns rather than exact agreement. Published research from the Duolingo English Test provides an additional point of reference, reporting human–AI correlations of 0.84 for speaking and 0.85 for writing. While these studies used different samples and methodologies, the findings provide useful context for evaluating Hallo’s automated language scoring.
What These Results Tell Us About Scoring Quality
A language assessment must provide scores that can be interpreted with confidence. In this study, test responses receiving higher scores from independent, third-party trained evaluators generally also received higher scores from Hallo’s automated scoring system. This alignment is a promising piece of evidence supporting the scoring approach used in the Hallo Language Test.
Importantly, one correlation study does not establish overall test accuracy or validity for every purpose. Correlation does not show that individual scores always match, that all four skills have been separately validated, or that HLT scores predict academic outcomes. Those questions require complementary agreement, reliability, and validity studies.
How Hallo's Adaptive Language Assessment Works
Accurate assessment starts with asking the right questions. Instead of giving every test taker the same fixed set of questions, Hallo adapts the assessment to each person’s proficiency level.
A key part of this approach is the floor/ceiling rating method, based on research conducted by Brigham Young University (BYU). The method identifies the language skills a test taker can demonstrate consistently and the point where tasks begin to exceed their ability. For example, a test taker who performs well on intermediate questions may be given more advanced ones, helping the assessment capture the full range of their abilities.
The HLT combines automated scoring with computer-adaptive testing (CAT), which adjusts question difficulty based on each test taker’s performance. As Cambridge Assessment explains in its overview of CAT, this approach can make assessments more personalized and more efficient. Together, adaptive testing and AI-driven scoring give institutions a scalable way to measure English proficiency.
Making Quality Assessment More Accessible
Traditional language assessments often demand significant time, resources, and administrative effort. AI-powered testing can reduce that burden.
Time
Hallo offers flexible language assessments that can evaluate individual skills, such as speaking and listening, or all four language skills together. Comprehensive assessments can be completed in approximately one hour.
Resources
Hallo’s language assessments can be taken remotely using a compatible device with a camera and microphone, eliminating the need to travel to a testing center.
Administrative effort
Hallo reports language proficiency using the Common European Framework of Reference for Languages (CEFR), providing institutions with a familiar international proficiency framework. For more detailed evaluation, Hallo also provides numerical scores on a 0-10 scale.
Why This Research Matters for Universities
For universities, language proficiency results can inform decisions about admissions, placement, and student support. Institutions therefore need evidence about how assessment scores are produced, how consistently they behave, and whether they support the intended decisions. The new findings contribute to that evidence by examining Hallo’s scoring against trained human evaluations.
This is part of Hallo’s broader validation program, documented in the Hallo Language Test Technical Manual. Work on score concordance with established assessments, test–retest reliability, and additional measurement properties will help build a more complete validity argument for specific uses.
Research You Can Evaluate for Yourself
At Hallo, we believe that making assessment more accessible should never come at the expense of scoring quality or transparency. The human–AI scoring study is one part of a broader validation program that examines scoring consistency, evaluator agreement, and other aspects of test performance.
This work is documented in the Hallo Language Test Technical Manual, which outlines our assessment framework and supporting research. By continuing to evaluate and improve the technology, we aim to give universities, employers, and educators greater confidence in AI-powered language assessment.
What's Next
This white paper is an important milestone, not the end of the work. We plan to update and expand it as new findings become available, building a fuller picture of HLT scoring performance.
Upcoming research includes concordance studies examining how Hallo Language Test scores relate to established English proficiency assessments, including TOEFL, IELTS, and the Duolingo English Test, as well as test–retest reliability studies evaluating score consistency across multiple testing sessions.
Frequently Asked Questions
What did Hallo’s latest validation study find? In a sample of 234 paired responses, Hallo’s automated scores were strongly correlated with the mean ratings of independent, third-party trained human evaluators (Pearson r = 0.89). This offers encouraging evidence about scoring consistency, although correlation alone does not establish exact agreement or overall assessment validity.
What makes Hallo’s scoring approach effective? Hallo combines AI-powered scoring with a floor/ceiling rating method based on research conducted by BYU. By using questions of varying difficulty, the method identifies what a test taker can do consistently and where their abilities reach their limits, supporting more detailed proficiency evaluations.
What other research is Hallo planning? We are expanding our research and validation program with several studies planned for the near future. These include concordance studies comparing HLT scores with TOEFL, IELTS, and Duolingo English Test results, and test–retest reliability studies of score consistency across sessions.
How is the Hallo Language Test (HLT) different from traditional assessments? The HLT combines AI-powered scoring, adaptive testing, and research-based assessment methods to make English proficiency evaluation more accessible and efficient. Adapting questions to each test taker and scoring automatically allows Hallo to deliver assessments at scale, while ongoing research continues to strengthen the test.
Where can I learn more about Hallo’s assessment research? The Hallo Language Test Technical Manual provides detailed information on the test’s design, scoring framework, and supporting evidence. We will share new findings and updated publications through our Hallo Research and Validation resources as studies are completed.
Read the Full Validation Study
This new research adds to the evidence supporting Hallo’s language assessment scoring and reflects our commitment to accessible, scalable, research-informed testing.
Explore the methodology, findings, and limitations in our November 2026 white paper, Hallo Language Test (HLT): Human–AI Score Correspondence.
Copyright © 2026 Hallo Inc. All rights reserved.
Join the 10,000,000+ assessment community.
