Domain Contributor - Evaluation Specialist
micro1 Β· 100% remote Β· Contract Β· Posted
- Pay
- $30β60/hr
- Location
- Worldwide
- Languages
- English
- Hours
- 15 hours per week
- Openings
- 20
- Level
- Expert
You clicked Apply on .
Already applied?
Summary
Design nuanced domain-specific tasks, author questions, assess AI outputs against primary sources, and document rationales for model shortcomings.
What you'll do
- Design nuanced, domain-specific tasks that probe AI model limits
- Author questions and scenarios demanding deep subject knowledge
- Assess AI-generated outputs, validating accuracy against authoritative sources
- Document clear rationales for model shortcomings
- Apply detailed task guidelines and evaluation rubrics consistently
- Submit completed deliverables independently on a rolling basis
Requirements
- Have proven experience with AI model training, annotation, RLHF, or evaluation
- Have deep expertise in history, law, finance, journalism, or technology
- Verify complex claims against authoritative sources
- Communicate findings and feedback clearly in writing and verbally
- Follow detailed instructions and evaluation frameworks precisely
Skills
- Domain Depth
- Source Verification
- Reasoning Clarity
- Evaluation Expertise
Full description
Role Title: Domain Contributor - Evaluation Specialist
Role Type: Contractor (~15 hrs/week)
Location: Remote
micro1 is engaging Domain Contributor - Evaluation Specialists to support an internal project evaluating and improving advanced AI models. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. Flexible, pay-per-task work across history, law, finance, journalism, and tech, for experts with prior AI training experience.
Scope of Work
- Design nuanced, domain-specific tasks that probe AI model limits and surface reasoning flaws.
- Author questions and scenarios demanding deep subject knowledge, beyond general understanding.
- Assess AI-generated outputs, validating accuracy against authoritative, primary sources in your area of expertise.
- Document clear rationales for model shortcomings, specifying missed reasoning, source errors, or nuances.
- Apply detailed task guidelines and evaluation rubrics consistently to each submission.
- Submit completed deliverables independently on a rolling basis, meeting project requirements for written and verbal clarity.
Preferred Qualifications
- Proven hands-on experience with AI model training, annotation, RLHF, or evaluation.
- Deep expertise in a core domain: history, archival or library science, law (paralegal, legal academic, compliance), finance (equity research, PE/VC, forensic accounting), journalism, or technology/industry history.
- Advanced research skills with a track record of verifying complex claims against authoritative sources.
- Exceptional written and verbal communication skills to articulate findings and feedback.
- Demonstrated ability to precisely follow detailed instructions and evaluation frameworks.
- Background in adversarial or βstomp the modelβ task design is advantageous.
- Graduate-level research, experience at a Fortune 100 company, or professional fluency in a non-English language a plus.
Compensation Structure
Compensation is output-based; experts are paid per task that meets the project specifications. The time required to complete work may vary depending on the expertβs experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.
Start Timeline & Availability
We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected, we expect you to start your first tasks within 24β48 hours of completing onboarding.
Similar jobs
Senior Vehicle Damage and Condition ExpertAuto Damage EstimatingVehicle Condition and Damage InspectionAccident Quality Auditing+2$20β60/hr
π Worldwide
Senior Cargo and Parcel Claims ExaminerAir Cargo ClaimsCustody DeterminationParcel Delivery DisputesRelease RulesFreight DocumentationLiability Assessment+5$20β60/hr
π Worldwide
Expert InterviewerRecruitingInterviewingTechnical VettingStructured Judgment+2$50β60/hr
πΊπΈ US
Expert Project ManagerGoogle SheetsProject ManagementData AnalysisLLM Training+3$50β60/hr
π Worldwide