Job Closed

This listing is no longer active.

ItsaCheckmate logo
ItsaCheckmate

Menus. Orders. Simplified.

Senior Prompt Engineer – Data Science, Quality Analysis

Data ScientistData ScientistFull TimeRemoteSeniorTeam 201-500H1B No SponsorCompany SiteLinkedIn

Location

India

Posted

147 days ago

Salary

0

Seniority

Senior

Bachelor Degree5 yrs expEnglishPythonSQL

Job Description

Senior Prompt Engineer – Data Science, Quality Analysis

ItsaCheckmate

• Design, test, and optimize LLM prompts for conversational AI, text classification, and structured data extraction tasks. • Build evaluation pipelines to analyze prompt performance using quantitative metrics, human-in-the-loop feedback, and business KPIs. • Conduct prompt experiments and regression testing to ensure stability, accuracy, and safety as models evolve. • Collaborate with Machine Learning, Product, and Operations teams to translate business objectives into scalable, data-driven prompt-engineering strategies that enhance model accuracy, efficiency, and real-world usability. • Use Python/SQL to analyze model outputs, identify anomalies, and automate quality checks. • Document best practices and contribute to internal frameworks for prompt evaluation and continuous improvement. • Communicate findings effectively to technical and non-technical stakeholders, driving measurable business impact through insight-driven decisions.

Job Requirements

  • B.S. or higher in a quantitative discipline (Data Science, Computer Science, Engineering, or related field) or in a field relevant to language models (Linguistics, Philosophy, Cognitive Science, etc.).
  • 5+ years of relevant experience with a B.S. degree, or 3+ years of experience with a Master’s degree.
  • Demonstrated proficiency in Python for automation, evaluation, and experimentation with LLM workflows.
  • Proven experience in prompt engineering and working with LLMs (GPT-4, Claude, Gemini, and LLaMA) for text generation, reasoning, and structured data extraction.
  • Proficiency in Python and SQL for data analysis, evaluation scripting, and workflow automation.
  • Strong background in A/B testing, statistical analysis, and performance metric evaluation, with the ability to design experiments and interpret data-driven insights for continuous model optimization.
  • Familiarity with prompt-evaluation tools such as LangFuse or Galileo, and Weights and Biases for experiment management and regression testing.
  • Deep understanding of advanced prompting techniques, including few-shot prompting, reasoning-based prompting, multi-turn dialogue design, agentic orchestration, and DSPy/AdaFlow-style programmatic prompting frameworks.
  • Experience applying CO-STAR and TIDD-EC! prompting frameworks for structured reasoning, instruction design, and context control in production-grade LLM systems.
  • Excellent requirement-elicitation and communication skills, with the ability to translate business objectives into prompt-engineering solutions.
  • Analytical mindset with a process-driven approach to optimizing model behavior, data quality, and operational workflows.
  • Academic or applied research experience related to language models, prompt engineering, or LLM-based systems is a strong plus.
  • Familiarity with LLM architectures, embeddings, and fine-tuning techniques preferred.
  • Experience with LLM red-teaming, adversarial evaluation, or model safety testing is a plus.
  • Candidates must be flexible and work during US hours at least till 5 pm EST, which is essential for this role.

Related Categories

Related Job Pages

More Data Scientist Jobs

Knuddels logo

Team Lead Data Science – remote, Karlsruhe

Knuddels

Wir bringen Menschen näher zusammen.

Data Scientist147 days ago
Full TimeRemoteTeam 51-200Since 1999H1B No Sponsor

• You will take on a key role, working not only operationally but also strategically reshaping and embedding the Data Science function at Knuddels. • Leadership & team building: As the team is evolving, you will continue to expand its structure and foster your team members’ development through coaching and feedback. • Data strategy & culture: Together with company leadership, you will define the long-term data strategy. Your goal is to establish a genuine data-driven culture, empower colleagues, and make data understandable and accessible across the company. • Sparring partner & bridge builder: You act as the data-driven conscience but do not rely on numbers blindly. You link quantitative analyses with qualitative feedback from the community (e.g., through close exchange with Community Management) to form a holistic view and enable well-founded decisions. • Hands-on Data Science: You work operationally with our complex data (millions of events per day) to extract valuable insights using data mining, clustering, forecasting, recommendation systems, and classification. • Quality & processes: You ensure the correctness of our data across systems (DWH, analytics) and establish best practices (e.g., code reviews, documentation) for scalable data-science products. • Business impact: You derive concrete actions from your analyses that deliver the greatest value to our community and secure the long-term success of Knuddels.

Germany
Later logo

Principal Product Manager, Data Product, Data Science

Later

Level up your social media marketing strategy ✨ Schedule, plan, engage & grow 🌴

Data Scientist147 days ago
OtherRemoteTeam 51-200Since 2014H1B Sponsor

• Own and evolve the end-to-end creator data product strategy • Define and maintain a long-term roadmap that improves creator and audience data • Identify high-leverage data opportunities for Sales, Strategy, and Agency teams • Translate ambiguous business problems into clear data product bets

United States
$210K - $250K / year
Job Closed
Golden Prospects by YMP logo

Data Scientist, NLP

Golden Prospects by YMP

Connecting young mining professionals with golden opportunities!

Data Scientist148 days ago
OtherRemoteTeam 11-50H1B No Sponsor

• Design and develop models to extract entities, detect intents, and understand document structure • Tackle challenges like long-context reasoning, layout-aware NLP, and ambiguous inputs • Evaluate model performance where ground truth is partial, uncertain, or evolving • Shape the roadmap and success metrics for replacing legacy document processing systems with smarter, scalable solutions • Work with other data scientists and engineers to optimize machine learning models and insert them into end-to-end pipelines • Understand product use-cases and define key performance metrics for models according to business requirements • Set up systems for long-term improvement of models and data quality (e.g. active learning, continuous learning systems, etc.)

New York
Earnest logo

Data Scientist II

Earnest

At Earnest, we empower you to take control of your career so you can empower students to take control of their finances.

Data Scientist150 days ago
OtherRemoteTeam 201-500Since 2013H1B Sponsor

• Conduct experimentation and execute causal inference analyses on pricing, marketing, and conversion models to drive revenue optimization. • Develop Pricing Optimization algorithms to maximize the unit economics of our lending products while driving meaningful growth in our origination businesses. • Develop and maintain predictive ML models to assess potential risks and opportunities across our lending products, contributing to the enhancement of risk and marketing assessment procedures. • Conceptualize, research, and prototype data-driven solutions, effectively communicating their impact to the stakeholders. • Collaborate closely with cross-functional teams and stakeholders to accelerate solution iteration and achieve measurable outcomes. • Build funnel dashboards and perform root cause analysis to monitor and identify user behavioral patterns and areas of opportunity. • Collaborate with data and infrastructure engineers to deploy ML and Pricing pipelines, from data collection through model deployment. This includes automating training and ongoing monitoring utilizing BI tools. • Prepare technical designs and documentation using git and Confluence.

United States
$131K - $164K / year
Job Closed