[Remote] Junior Data Scientist

Note: The job is a remote job and is open to candidates in USA. Weekday AI is recruiting for an unnamed client for a Junior Data Scientist contractor role focused on evaluating and improving AI systems. The role involves assessing AI-generated data science responses, creating prompts, datasets, and reference solutions, and providing structured feedback on accuracy and reliability.

Responsibilities

• AI Model Evaluation: Evaluate AI-generated outputs on data science, statistics, machine learning, experimentation, and quantitative analysis tasks
• Problem and Prompt Development: Create challenging, well-defined prompts and technical problems that assess analytical reasoning and practical data science skills
• Dataset Preparation: Prepare and validate relevant datasets for testing AI models across statistical and machine learning use cases
• Reference Solution Development: Develop accurate reference answers, calculations, and analytical solutions to establish reliable evaluation benchmarks
• Statistical Validation: Identify statistical errors, incorrect assumptions, flawed methodologies, and inappropriate interpretations of data
• Quantitative Reasoning Assessment: Assess whether AI-generated solutions demonstrate sound mathematical reasoning, appropriate analytical methods, and evidence-based conclusions
• Structured Feedback: Provide clear, actionable feedback highlighting errors, missing steps, and opportunities to improve the accuracy and reliability of AI-generated outputs
• Quality Assurance: Validate analytical results, review solution consistency, and ensure that evaluation criteria are objective and reproducible
• Technical Documentation: Explain statistical concepts, analytical approaches, and evaluation findings in a clear and structured manner

Skills

• At least one year of professional experience at a well-established company in technology, finance, research, or a related field
• At least part of the relevant professional experience should have been gained within the past three years
• Foundational knowledge of data science, statistics, machine learning, and quantitative analysis
• Strong analytical and problem-solving skills, with the ability to evaluate data-driven conclusions critically
• Understanding of statistical methods, experimental design, data interpretation, and model evaluation
• Ability to identify incorrect calculations, methodological weaknesses, and unsupported conclusions
• Strong written communication skills, particularly when explaining technical and statistical concepts
• Ability to work independently, follow evaluation guidelines, and deliver accurate, high-quality assessments
• Data Science Fundamentals
• Statistics and Probability
• Machine Learning Fundamentals
• Quantitative Reasoning
• Data Analysis and Interpretation
• Statistical Methodology
• Analytical Problem-Solving
• Technical Writing and Documentation
• Python Programming
• SQL and Data Manipulation
• Pandas, NumPy, and scikit-learn
• Experimental Design and Hypothesis Testing
• Predictive Modeling and Model Evaluation
• Dataset Creation and Validation
• AI Model Evaluation and Benchmarking
• Experience in Technology, Finance, or Research Organizations
• Experience working at a recognized technology company, financial institution, or research organization
• Professional exposure to statistical analysis, machine learning projects, quantitative research, or data-driven decision-making
• Familiarity with evaluating AI-generated answers or validating analytical solutions
• Ability to communicate complex quantitative concepts in clear, concise English
• Candidates based in English-speaking countries are preferred

Company Overview