[Remote] AI Test Engineer
Note: The job is a remote job and is open to candidates in USA. Smart IT Frame LLC connects top talent with leading organizations across the USA, specializing in IT, healthcare, and professional roles. They are seeking an AI Test Engineer to validate and assure the quality of AIML systems for real-time contextual advertising on live video streams.
Responsibilities
- Validate AIgenerated outputs against labelled datasets and expected outcomes
- Perform content validation for native moments detection across video streams
- Design and implement automated testing frameworks Pythonbased
- Build validation harness for
- JSON parsing and comparison
- Structured output validation using defined schemas
- Conduct multimodal validation video audio transcript outputs
- Execute system integration and endtoend testing
- Track accuracy metrics defect logging and root cause analysis
- Automate regression and validation pipelines across datasets
- Collaborate with Applied Scientists to validate LLM and GenAI outputs
- Work within AWS environment using services such as
- S3 data storage
- Lambda processing
- CloudWatch monitoring
Skills
- Strong Python scripting for automation and testing
- Experience in AIML model validation and QA workflows
- Hands-on with JSON parsing schema comparison
- Data validation pipelines
- Experience in testing Multimodal AI systems vision NLP audio LLM outputs and generative AI pipelines
- Good understanding of Structured output validation
- AI accuracy evaluation techniques
- Familiarity with AWS cloud environment S3 Lambda logging/monitoring
- Ability to handle mixed manual automated testing approaches
- Experience with media/video content analysis workflows
Company Overview
Company H1B Sponsorship