Job Closed
This listing is no longer active.
Software for rapid military planning: make planning fast enough for today's environment
Data Engineering Lead
Location
United States
Posted
106 days ago
Salary
$205K - $255K / year
Seniority
Senior
Job Description
Data Engineering Lead
Onebrief
• Build and lead Battle Road’s data platform from the ground up. • Architect our AWS-based data warehouse and ingestion pipelines • Transform high-volume simulation outputs into clean, trusted datasets • Establish schema standards and data contracts with engineering • Enable scenario replay, comparison, and traceability • Support analytics and AI workflows across the platform • Hire and develop a high-performing data engineering team • Within your first year, you will stand up a scalable warehouse, integrate core simulation feeds, and deliver reliable insight into product usage and simulation behavior. • You will be modeling a dynamic world state, not just logs.
Job Requirements
- 5+ years of professional experience as a Data Engineer or Data Platform Engineer.
- Deep experience building production-grade batch and streaming pipelines
- Proficiency in Python, SQL, and at least one data processing framework.
- Experience with cloud-based data infrastructure (AWS, GCP, or Azure).
- Experience designing schemas and modeling complex systems
- Experience leading or mentoring engineers
- Clear, concise written and verbal communication skills
- Ability to reason about data as a product, not a byproduct.
Benefits
- Equity: Share in the company's success.
- Flexible Work Environment: Remote-first organization* with flexible work hours and unlimited PTO.***(*note that some roles are in-person, on-site with customers)*
- Comprehensive Health Coverage: Health, dental, vision, and life insurance.
- Retirement Plan: 401(k) plan with company match to secure your future.
- Parental Leave: 8 weeks at 100% regardless of state.
- Company Retreats: Annual company summit trips.
- Home Office Budget: $1,000 per year for home office improvements.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
THE TEAM & THE JOB We are seeking a talented and self-motivated Senior Data Scientist/AI Engineer to join our team. In this role, you will design, build, and deploy next generation AI-driven tools for market research. You will work at the intersection of data science and AI engineering. We develop models and predictions using a combination of surveys, voter databases, census microdata, behavioral tracking and other sources. We handle the most difficult data problems at our company, including massive datasets, missing data, self-selection, measurement error, fraud detection, attrition, and many more. This is a hands-on role that requires both analytical and programming skills at an advanced level. WHAT YOU WILL BE DOING Design and build AI-powered applications to analyze, summarize, and generate insights from survey data. Contribute to technical design and architecture decisions for AI/ML systems. Develop and optimize prompts and chains. Build applications to simplify analytics and improve the efficiency of data processing. Work with structured and unstructured data Collaborate with data analysts, product managers, UI designers, and software developers to create great products. Contribute to model evaluation and performance tracking using both qualitative and quantitative metrics. Stay up to date with recent developments in AI and machine learning, including LLMs, fine tuning, and RAG pipelines. Handling ambiguous, complex problems with minimal guidance Contribute to technical vision, architectural decisions, or to help setting technical standards. Contribute to business impact and strategic problem-solving at an organizational level as and when required. KEY COMPETENCIES Master’s or PhD in data science or related field. Demonstrable progressive experience in senior data science/AI engineering roles. Advanced knowledge in Python and/or R for data analysis, modelling, and visualization. Experience working with LLM APIs and building LLM-based applications and frameworks. Progressive experience with natural language processing, including topic modelling, text classification, and summarization. Experience in taking part in cross-functional technical projects. Ability to write clean, maintainable, well-documented code. Proven track record of delivering complex, high-impact solutions. Ability to cross collaborate with other teams across the business. For roles based in California, New York, Colorado or Washington State, the base salary hiring range for this position is USD 188,864 - USD 211,600. Compensation offered will be determined by multiple factors including location, job-related knowledge, skills and experience. Certain roles may be eligible for incentive compensation and additional benefits. All US based full time employees are eligible for the following benefits: Paid vacation, holidays and sick days Flexible working arrangement available Group medical, dental and vision insurance Company-paid life and disability insurance Paid parental leave 401(k) with company match Why join YouGov? Join our global team to help us achieve our social mission: to make millions of people’s opinions heard for the benefit of our local, national, and international communities. Understanding diversity of opinion requires diversity of background. Although our global panel of millions of people worldwide powers our research, our biggest asset is our people. If our research is to be truly representative of what the world thinks, we need people from all walks of life to be part of the team to bring their perspective to the work we do. Life at YouGov We are driven by a set of shared values. We are fast, fearless, and innovative. We work diligently to get it right. We are guided by accuracy, ethics, and proven methodologies. We respect and trust each other, bringing these values into everything that we do. We strive to provide YouGovers with best-in-class benefits to support their physical, financial, and emotional wellbeing. We want our employees to have a sense of belonging and uniqueness in a supportive workplace, so they can bring their full selves to work. Equal Opportunity Employer As an Equal Opportunity Employer, qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, gender identity or expression, and sexual orientation), parental status, national origin, marital status, age, disability, genetic information, HIV status, political affiliation, socioeconomic background, veteran status or any other characteristic protected by law or in line with our responsibilities as a fair and ethic employer. All employment decisions are made based on occupational qualifications, merit, and business need.
• Lead the foundational setup of new data environments, including Snowflake provisioning, SSO configuration, and initial system architecture. • Design, build, and manage scalable data pipelines and ETL/ELT processes to transform raw data into actionable insights. • Support the scaling of engineering teams to handle high volumes of commercial projects while maintaining business continuity during organizational transitions. • Develop and maintain robust DBT data models to support semantic layer build-outs and advanced analytics capabilities. • Partner closely with business leaders and stakeholders to bridge the gap between technical delivery and strategic objectives. • Provide engineering support for custom backend applications, ensuring seamless data integration and reliable exposure to end users. • Establish and enforce best practices for data engineering, architecture, and deployments that can scale across multiple organizations.
Data Engineer
Diverge HealthEmpowering primary care practices to deliver the highest quality care to underserved patients
• Build, optimize, and maintain data pipelines • Work closely with data scientists, analysts, and engineers • Design scalable and reliable data architectures • Collaborate across teams to implement data solutions
• Provide timely, well-documented responses to data questions from stakeholders across Engineering, Marketing, Business, Data Science, and Analytics, escalating when appropriate and keeping requesters informed. • Take on well-scoped, small data modeling requests in our warehouse: update or add dbt models and tests, write clear model documentation, and open PRs with thoughtful descriptions and checks. • Support the health of our data platform by pairing with the on-call engineer to monitor job runs and alerts, triage issues, and execute playbooks; escalate quickly when outside scope. • Proactively monitor nightly and hourly pipelines (e.g., Airflow DAGs, dbt Cloud runs, AWS DMS replication tasks) and remediate common failures (retries, backfills, simple config fixes) under guidance. • Help maintain data reliability by responding to Monte Carlo observability alerts, validating anomalies, and creating or refining data quality tests. • Partner with engineers to simplify and automate routine remediation steps; contribute small improvements that make our pipelines more robust and observable. • Document what you learn: improve runbooks, add troubleshooting steps to our internal knowledge base, and keep tickets updated with clear status and outcomes.




