Grid Dynamics Holdings

Role Description We are seeking an experienced ML Engineer to join our team to build and scale automated evaluation and synthetic data generation (SDG) capabilities that support safety assessments across languages and markets. You will play a crucial role in training automated judges, developing validation techniques, building automated performance checks, and creating scalable approaches to analysis and reporting. You will be part of a team who works closely with language experts and multi-lingual annotators to validate automated approaches to safety evaluations, across diverse linguistic contexts. - Automated Judge Development: - Train, fine-tune, and validate automated judge models that can reliably score AI system outputs for safety and policy compliance. - Develop calibration and agreement metrics to ensure judges meet human-parity benchmarks. - Validation Techniques: - Design and implement validation frameworks to assess the accuracy, reliability, and cross-linguistic consistency of automated evaluation systems. - Develop methods to detect drift, bias, and failure modes in automated judges across markets. - Synthetic Data Generation: - Develop and maintain synthetic data generation pipelines to augment evaluation coverage, stress-test safety boundaries, and support evaluation in low-resource languages. - Ensure synthetic data is diverse, representative, and validated against human-generated benchmarks. - Scalable Analysis & Reporting Automation: - Create automated pipelines for analysis and reporting that reduce manual effort, increase reproducibility, and enable rapid cross-market safety assessments. - Build tooling that integrates with existing dashboards and reporting workflows. Qualifications - 3+ years of experience in an ML engineering or applied ML research role, with hands-on experience building and deploying ML models and pipelines. - Strong proficiency in Python and ML frameworks (e.g., PyTorch, TensorFlow, Hugging Face Transformers). - Experience training, fine-tuning, and evaluating language models and/or classifiers, including prompt engineering and model calibration. - Experience building automated data processing, evaluation, or monitoring pipelines. - Comfortable with experiment design and statistical validation of model performance across segmented samples. - Able to work independently as well as collaboratively with minimal direction. - Organized, highly attentive to detail, and manages time well. - Experience with synthetic data generation techniques, including data augmentation, paraphrasing, and controlled generation methods. - Experience with multilingual NLP, cross-lingual transfer learning, or low-resource language modeling. - Familiarity with evaluation-as-a-service architectures or automated red teaming frameworks. - Experience with large-scale distributed computing (e.g., Spark, Ray, or cloud-based ML platforms). - Prior experience in AI safety, responsible AI, content moderation, or trust and safety domains. - Experience with CI/CD integration for ML model validation and deployment. - Advanced degree (MS/PhD) in Computer Science, Machine Learning, Natural Language Processing, or a related field. Benefits - Opportunity to work on cutting-edge projects. - Work with a highly motivated and dedicated team. - Competitive salary. - Flexible schedule. - Benefits package - medical insurance, vision, dental, etc. - Corporate social events. - Professional development opportunities. - Well-equipped office.

Role Description We are seeking a highly skilled Machine Learning Engineer who specializes in leveraging Large Language Models (LLMs) for automated evaluation and quality assessment. In this role, you will design and build systems that automatically measure and improve the accuracy, relevance, and consistency of model outputs. You will lead initiatives to create evaluation pipelines, develop metrics, and deliver actionable insights for continuous improvements. This position requires strong technical expertise, analytical problem-solving abilities, and the capacity to manage projects across multiple cross-functional teams. - Design and implement automated systems and pipelines for evaluating LLM outputs. - Develop metrics and KPIs to measure output quality, accuracy, and consistency using LLM-based evaluations. - Collaborate with Engineering teams to create automated logic checks and validation tools. - Partner with Data Scientists to analyze evaluation results and optimize prompt and task structures. - Provide feedback loops to ensure evaluation guidelines align with LLM-based assessments. - Investigate how LLM-derived evaluations can enhance product reliability and user experience. - Recommend refinements to prompt engineering, evaluation strategies, and automation tools. - Stay informed on emerging trends in LLM evaluation, automated quality assessment, and AI toolchains. - Continuously improve and expand automated evaluation processes based on industry best practices. Qualifications - 5+ years of experience in ML engineering, NLP, or AI/ML automation. - Hands-on experience in prompt engineering and designing LLM-based evaluation systems is preferred. - Strong understanding of machine learning principles with focus on NLP and advanced LLM capabilities (e.g., Chain-of-Thought, agentic workflows). - Expertise in building automated evaluation or QA pipelines. - Excellent analytical and problem-solving skills with experience in root cause and error pattern analysis. - Proven project management and cross-functional collaboration experience. - Excellent communication skills to convey complex insights to technical and non-technical audiences. - Detail-oriented mindset with a focus on evaluation metrics, prompt design, and automation. - Ability to quickly adapt to new business rules and evaluation guidelines across diverse product domains. - Strong programming skills in Python and SQL. - Experience with big data technologies like PySpark for data aggregation and sampling is a strong plus. - Bachelor's/Master's degree in Computer Science/Engineering or a related field. Benefits - Opportunity to work on cutting-edge projects. - Work with a highly motivated and dedicated team. - Competitive salary. - Flexible schedule. - Benefits package - medical insurance, vision, dental, etc. - Corporate social events. - Professional development opportunities. - Well-equipped office.

Role Description We are looking for Senior Big Data Developers to join our team across a variety of projects and technology stacks. Opportunity is fully remote, with a B2B setup in Turkey (sole proprietor). - Design, build, and maintain scalable data pipelines and data processing systems - Develop and optimize batch and real-time data workflows using distributed processing frameworks - Work with streaming and messaging systems to enable reliable data ingestion and event-driven architectures - Implement data integration solutions across various sources, ensuring data quality, consistency, and reliability - Collaborate with cross-functional teams to translate business requirements into technical data solutions - Build and maintain ETL/ELT processes supporting analytics, reporting, and downstream applications - Ensure performance, scalability, and fault tolerance of data platforms in cloud environments - Work with orchestration tools to schedule, monitor, and manage complex data workflows Qualifications - Proficiency in Python - Extensive hands-on experience across the AWS ecosystem - Experience with Apache Spark and Kafka - Hands-on experience with Apache Airflow - Advanced SQL skills and solid understanding of REST API integration Requirements - Previous experience operating in a Lead capacity, driving technical strategy, and mentoring junior engineering team members - Experience implementing modern data observability and automated data quality monitoring frameworks Benefits - Opportunity to work on bleeding-edge projects - Work with a highly motivated and dedicated team - Competitive salary - Flexible schedule - Benefits package - medical insurance, sports - Corporate social events - Professional development opportunities - Well-equipped office

Role Description Join a world-class team of engineers at Grid Dynamics as we build mission-critical engineering tools. We are hiring an ownership-driven Backend Developer to architect the Python-based workflow engines and state machines that bridge the gap between complex hardware requirements and streamlined digital approvals. - Design and implement high-performance, asynchronous Python backend services using FastAPI and asyncio to power complex review workflows. - Translate intricate business logic into robust state machines, managing requirement transitions from Draft through Active to Approved/Rejected states. - Build and maintain the integration layer with JAMA Connect's REST API, ensuring seamless data synchronization and handling edge cases like rate limiting and retries. - Serve as the primary architect for the client-server contract, collaborating with Swift/macOS engineers to define JSON schemas, error codes and status enums. - Design relational database schemas in PostgreSQL that support comprehensive audit trails and versioned records for every approval decision. - Maintain high code quality through rigorous pytest coverage and document the API lifecycle to ensure long-term maintainability for QA and future engineers. Qualifications - Proven experience with async I/O patterns (FastAPI, asyncio, or aiohttp) and a deep understanding of writing non-blocking, scalable code. - Strong background in designing and consuming RESTful APIs, including experience with pagination, error handling, and third-party service integrations. - Advanced skills in PostgreSQL, specifically in schema design for version tracking, complex migrations, and high-integrity audit logs. - Demonstrated experience building state machines or complex workflow engines involving multi-step approvals and status transitions. - A "test-first" mindset with professional experience using pytest for unit and integration testing against both mock and live endpoints. - Would be a plus: prior experience working with JAMA Connect, IBM DOORS, or similar ALM (Application Lifecycle Management) tools. Benefits - Opportunity to work on bleeding-edge projects. - Work with a highly motivated and dedicated team. - Competitive salary. - Flexible schedule. - Benefits package - medical insurance, sports. - Professional development opportunities.