Pay: 

• Qualified Bachelor’s students and graduates: $25 USD per hour.
• Graduate students (Master’s or PhD), experienced instructors, lecturers, and other qualified language professionals: $30 USD/hour

About Hallo.ai

Hallo.ai is an AI-powered language assessment platform headquartered in Provo, Utah. Our assessments are used by organizations worldwide, with millions of assessments administered annually.

To uphold the highest standards of quality, fairness, and accuracy, Hallo conducts annual validation studies comparing independent expert evaluations with AI-generated assessment results. Our language assessments are aligned with internationally recognized proficiency frameworks, including the Common European Framework of Reference for Languages (CEFR) where applicable.

As part of this work, we are establishing the Hallo Language Evaluator Network – a community of qualified language professionals who contribute to the validation and continuous improvement of Hallo’s language assessments.

Appointment as a Hallo Language Evaluator recognizes demonstrated expertise in language education, language assessment, or applied linguistics. Evaluators participate in an annual validation program alongside other qualified language professionals from around the world.

About the Appointment

Hallo Language Evaluators independently evaluate speaking and/or writing responses in their area of language expertise using standardized scoring rubrics.

Evaluations are completed independently, without access to AI scores or other evaluators’ ratings, ensuring objective and unbiased judgments. These expert evaluations support the validation of Hallo’s AI assessment models and contribute to ongoing research, quality assurance, and continuous improvement.

This is a paid annual appointment. Evaluators participate in in-person training and calibration in Provo, Utah before the annual evaluation period. Operational evaluations are then completed remotely on a flexible schedule over approximately 2-3 months.

The expected time commitment is approximately 50-100 hours during the annual evaluation period, depending on language, project needs, and evaluator availability.

Appointments are made annually and are not guaranteed from year to year. Returning evaluators may be invited to participate in future programs based on project needs and prior performance.

Responsibilities

• Evaluate speaking and/or writing responses in your area of language expertise using standardized scoring rubrics.
• Complete paid evaluator training and calibration before participating in operational evaluations.
• Independently evaluate assigned responses within established project timelines.
• Apply scoring criteria consistently, objectively, and fairly.
• Maintain confidentiality and adhere to standardized evaluation procedures.

Eligibility

We welcome applications from experienced language professionals with expertise in language education, language assessment, or applied linguistics.

Required Qualifications

• Native or near-native proficiency in the language(s) you will evaluate.
• Bachelor’s degree (or current enrollment in a graduate program) in Linguistics, Education, Languages, Translation, Literature, TESOL, Applied Linguistics, or a related field.
• Professional experience teaching, assessing, translating, interpreting, or otherwise working in the target language.
• Demonstrated ability to apply standardized evaluation criteria consistently and objectively.
• Successful completion of Hallo’s evaluator training and calibration program.

Preferred Qualifications

• Master’s degree or PhD (or current enrollment in a graduate program) in a relevant field.
• Experience with standardized language proficiency assessments (e.g., CEFR, ACTFL, IELTS, TOEFL, Cambridge English, DELE, DELF/DALF, Goethe, JLPT, TOPIK, HSK, CAPLE, CELPE-Bras, or equivalent).
• Previous experience as a language assessment evaluator, examiner, or rater.

Languages

We are building a global evaluator network supporting 80+ languages.

Applicants should indicate the language(s) they are qualified to evaluate and provide a brief summary of their relevant professional experience.

Recognition & Compensation

Hallo Language Evaluators receive:

• Appointment to the Hallo Language Evaluator Network upon successful completion of training and calibration.
• Qualified Bachelor’s students and graduates: $25 USD per hour.
• Graduate students (Master’s or PhD), experienced instructors, lecturers, and other qualified language professionals: $30 USD/hour.
• Paid in-person evaluator training and calibration.
• Flexible remote participation during the annual evaluation period.
• The opportunity to contribute to the validation of AI-powered language assessments used by organizations worldwide.
• Priority consideration for future annual validation studies and research initiatives.

Annual Evaluation Cycle

The Hallo Language Evaluator Program is conducted once each year.

• Application Period: August
• Training & Calibration: September
• Evaluation Period: September – November (approximately 2–3 months)

Exact dates may vary from year to year.

Location

Hybrid (Provo, Utah)

• Training and calibration: Conducted in person in Provo, Utah.
• Operational evaluations: Completed remotely on a flexible schedule.

Confidentiality

Selected evaluators will be required to sign a confidentiality agreement before participating in the evaluation program. Evaluation materials, scoring rubrics, candidate responses, and all related information are confidential and may not be shared or reproduced without prior written authorization from Hallo.

Application

Appointment as a Hallo Language Evaluator is based on professional qualifications and successful completion of Hallo’s evaluator training and calibration program.

Evaluators who successfully complete the program become members of the Hallo Language Evaluator Network and may be invited to participate in future annual validation studies and research initiatives.

To apply here.

About the Hallo Language Evaluator Network

The Hallo Language Evaluator Network is a community of language professionals committed to advancing the quality, fairness, and reliability of AI-powered language assessment. Through annual validation studies, members contribute independent expert evaluations that help ensure Hallo’s assessments continue to meet rigorous standards for language proficiency measurement across the globe.