[Remote] Domain Advisor Consultant (Subject Matter Expert)
Note: The job is a remote job and is open to candidates in USA. Meridial Marketplace, by Invisible, is seeking elite Subject Matter Experts (SMEs) to engage in high-impact AI evaluation and training projects. The role involves defining quality criteria for AI outputs, curating benchmark examples for model training, and providing expert analysis on model failures. This is a consulting opportunity for independent professionals with deep domain knowledge who want to shape the standards for AI in their fields.
Responsibilities
- Review and advise on the quality criteria used to evaluate AI outputs
- Create, edit, or certify "Golden Tasks"—benchmark examples of perfect prompts and responses that serve as the ground truth for model training
- Analyze model failures (hallucinations, logical fallacies) and provide technical breakdowns of why the model failed and how a human expert would have approached the problem
- Consult on the difficulty and nuance of specific prompts to ensure the AI is being tested against sufficiently rigorous scenarios
Skills
- Elite Domain Mastery: Advanced degrees (PhD, MD, JD) or equivalent industry-recognized status (e.g., Senior Principal Engineer, Published Author, Award-Winning Creative)
- Professional Autonomy: You currently operate as an independent consultant or freelancer and have your own workflows, equipment, and professional methodology
- Interest in Shaping Technology: While prior experience in AI evaluation is a bonus, it is absolutely not required. We want industry veterans who are eager to set the 'gold standard' for how AI should operate within their specific profession
- Articulate Reasoning: The ability to not just know the right answer, but to articulate the reasoning process clearly in writing
- Global Compliance: Candidates must be properly registered to operate as independent contractors in their country of residence
Company Overview