Innodata is hiring an AI Voice Evaluation Specialist to assess and compare the conversational performance of AI models through real-time voice interactions. The role involves conducting consistent scenario-based conversations, evaluating responses against defined criteria, identifying errors, and documenting a reasoned model preference.
Review an assigned scenario and roleplay a natural conversation with two different AI models
Conduct comparable conversations with Model A and Model B, keeping the scenario, conversational approach, and overall interaction as consistent as possible
Maintain approximately the same number of conversational turns with each model to support a fair comparison
Record your voice during each interaction and pay close attention to the quality and behavior of the model's responses
Evaluate each model across five defined evaluation dimensions
Identify and categorize relevant error clusters that may occur in the audio or conversation
Compare the performance of Model A and Model B based on the evaluation criteria and your observations
Select the model that demonstrated the stronger overall performance according to the defined evaluation framework
Write a detailed rationale explaining your final preference, referencing specific examples and observations from both conversations
Apply evaluation guidelines consistently across different scenarios and model interactions
Qualification
Required
Bachelors Degree
Strong attention to detail and ability to notice subtle differences in conversational quality
Excellent listening and comprehension skills
Strong written communication skills, particularly the ability to explain observations clearly and objectively
Ability to follow detailed evaluation guidelines consistently
Comfort speaking naturally and roleplaying different conversational scenarios
Ability to compare two interactions fairly without allowing personal preferences to influence the evaluation
Strong critical-thinking and analytical skills
Reliability and consistency when completing structured evaluation tasks
Preferred
Familiarity with AI assistants, voice-based AI, or conversational systems
Benefits
Remote work, in Alabama, Alaska, Arkansas, Delaware, Florida, Georgia, Indiana, Maryland, Massachusetts, New Hampshire, New Jersey, New York, South Carolina, or Texas.
(NASDAQ: INOD) Innodata is a global data engineering company. We believe that data and AI are inextricably linked.
