RAG Model Evaluation Intern

About the job

AI Rudder is a software company that harnesses the power of AI voice automation to supercharge customer experiences. With AI voice assistants, your call center can make quality human-like calls at lightning speeds, collecting and analysing data automatically to reach and activate more customers. AI Rudder helps call centers reduce costs by automating repetitive tasks and lowering agent workload. This free up agents to focus on things only humans can do. Over the long term, AI Rudder aims to rethink the future of business communication.

Job Responsibilities

Deeply analyze the semantics and underlying needs of user inputs, enhancing the model’s ability to understand and respond to complex intents through data annotation and effectiveness validation.

Precisely evaluate the knowledge retrieval and matching efficiency of the retrieval model across multi-source data, continuously optimizing the accuracy and comprehensiveness of retrieval strategies.

Establish a scientific detection mechanism to identify content in model outputs that contradicts real-world knowledge, ensuring the reliability and authenticity of generated results.

Coordinate cross-linguistic consistency between Indonesian, Chinese, and English data, addressing issues such as translation discrepancies and semantic ambiguities to maintain the accuracy of multilingual knowledge bases.

Qualifications

RAG Knowledge: Understanding of RAG system architecture; 1+ years of experience in NLP data evaluation or RAG-related work.

Data Management Experience: Prior experience in managing data annotation projects, capable of independently developing annotation SOPs and handling communication/conflicts with outsourced teams.

Cross-Team Collaboration: Strong communication skills in Chinese and English, able to work efficiently with local Indonesian teams, algorithm engineers, and product managers to drive closed-loop resolution of data issues.

Preferred Qualifications

Familiarity with Chinese.

Experience in AI products for the Southeast Asian market or a background in the financial industry.

Industries:Computer Software

Function: Web Development