Big Data Engineer
Location
United States
Posted
28 days ago
Salary
$104.6K - $156.8K / year
Seniority
Mid Level
Job Description
Big Data Engineer
AdvanSix
Role Description AdvanSix is seeking a Big Data Engineer to build and operate our enterprise Unified Data Layer (UDL) - spanning IT and OT - to deliver trustworthy, performant data products that power Finance, Operations, Supply Chain & Logistics, HSE, Commercial, and corporate analytics. You’ll engineer batch/CDC/streaming pipelines, model curated/semantic layers, and harden run-state with testing, CI/CD, security, and observability. You’ll partner closely with the data team and larger IT organization. Mission - Design and deliver scalable, secure data pipelines and data models that safely connect operational systems to analytics. - Ensure trusted and well‑governed data. - Enable repeatable delivery of BI, ML, AI, and automation solutions. Data Engineering & Modeling - Build ingestion pipelines (batch, CDC, streaming) from S/4HANA/DataSphere, PHD/historian, LIMS, TMS, HSE, and other sources into landing → curated → semantic layers. - Implement data contracts, schema/versioning, SCD handling, partitioning, and performance tuning (file formats, clustering, caching). - Develop dimensional/semantic models that back certified Power BI datasets and APIs for apps/agents. OT/IT Integration & Safety - Integrate OT data via OPC UA/MQTT, broker/DMZ patterns, read-only historian feeds, and event/batch frames—no control-net reads. - Collaborate with plant controls on change control, signal quality, and downtime windows. Quality, Security & Observability - Embed data quality rules, unit/integration tests, and validation checks (freshness, completeness, drift/PSI). - Instrument lineage and end-to-end monitoring; build alerting and on-call runbooks to minimize MTTR. - Enforce RBAC, secrets management, PII/HSE classifications, and retention aligned to Governance/MDM policies. CI/CD, Cost & Reliability - Automate build/test/deploy with Git-based CI/CD (environments, approvals, blue/green). - Track and optimize cost/performance (cluster sizing, autoscaling, cache strategy); contribute to FinOps reviews. Collaboration & Documentation - Partner with Reporting & BI on semantic model contracts, RLS, and performance SLAs; avoid direct system scraping. - Produce “readme” docs, data dictionaries, runbooks, and post-incident reviews; support knowledge transfer with vendors. Qualifications - Minimum 5 years' in data engineering building production pipelines at scale (batch/CDC/streaming). - Hands-on with Azure data stack: Databricks or Fabric/Synapse, ADF/Pipelines, ADLS/OneLake, Azure SQL/SQL MI, Key Vault. - Strong SQL and Python/PySpark; comfort with Spark Structured Streaming and performance tuning. - Experience implementing tests/observability (freshness, schema, expectations), and Git-based CI/CD. - Familiarity with SAP S/4HANA structures and SAP DataSphere semantic modeling. - OT concepts: historians (PHD/PI), OPC UA/MQTT, event/batch frames, ISA-95/99 basics. - Understanding of Power BI consumption (semantic models, RLS) and APIs for downstream AI/ML apps/agents. Preferred Qualifications - Time-series/data-quality tooling (e.g., Great Expectations or equivalent patterns), feature/metric stores. - MDM concepts (keys, survivorship), lineage/catalog tooling. - TMS/WMS, LIMS, Historian, HSE domain exposure; Lean/Six Sigma mindset; FinOps awareness. Benefits - We provide benefits that are industry competitive and focused on employee well-being. - Total Rewards program includes a competitive compensation, health, dental, vision & wellness programs, paid vacation, 401K with company matching, health savings programs, disability & life insurance, employee assistance program. - Tuition reimbursement for continued education, certifications, training, and development. - Work within a fast paced and innovative company, meeting passionate colleagues and partners with diverse backgrounds and experiences.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Technical Product Analyst – Travel Data Platform, Orders Management System
BCD TravelTravel smart. Achieve more.
• Partner with Technical Product Managers to understand product vision, roadmap priorities, and business objectives • Analyze and refine incoming development requests to clarify problem statements, scope, and success criteria • Analyze system interactions, integrations, APIs, and data flows to define expected behavior and outcomes • Create and maintain documentation such as process flows, data mappings, interface behavior descriptions, and business rules • Support backlog refinement by ensuring work items are well-defined, prioritized appropriately, and ready for delivery • Define acceptance criteria, test scenarios, and expected outcomes for validation activities • Support the TPM in stakeholder discussions by providing detailed analysis, examples, and clarifications, along with note taking and action item tracking
• Design and build scalable, secure, and cost-effective data solutions in Snowflake • Develop and optimize data pipelines using tools such as dbt, Python, CloverDX, and cloud-native services • Participate in discovery sessions with clients to gather requirements and translate them into solution designs and project plans • Collaborate with engagement managers and account teams to help scope work and provide technical input for Statements of Work (SOWs) • Serve as a Snowflake subject matter expert, guiding best practices in performance tuning, cost optimization, access control, and workload management • Lead modernization and migration initiatives to move clients from legacy systems into Snowflake • Integrate Snowflake with BI tools, governance platforms, and AI/ML frameworks • Contribute to internal accelerators, frameworks, and proofs of concept • Mentor junior engineers and support knowledge sharing across the team
Norwegian Data Annotator
Careerflow.aiCommitments Required: 40 hours per week with overlap of 6 hours with PST. Engagement type: Contractor (no medical/paid leave). Duration of contract: 6 months with opportunity to extend; expected start date is 1st week of Jun-2026. Location: North America and LATAM.
Role Description We are looking for native or fluent language speakers to help train AI systems by reviewing and annotating content in their language (Norwegian). This is a long-term, remote gig. No prior experience is required. On the job training will be provided. What You Will Do - Review content in your native language (e.g., newspaper articles, website screenshots, advertisements) - Draw bounding boxes around specific elements as instructed (e.g., highlight the commercial aspect of an article) - Answer structured questions about the content - Write short summaries when required - Follow clear annotation guidelines to ensure quality and consistency Qualifications - Native or highly fluent speakers of the target language - Students, freelancers, part-time workers, or anyone seeking flexible short-term work - People comfortable following written instructions on a computer Requirements - Strong reading comprehension in your language - Basic computer skills (no technical background needed) - Attention to detail - Ability to follow structured instructions Role Details - Duration: Minimum 6 months (Long-term Project) - Pay Rate: $8 USD / hour - Hours: 30–40 hours/week - Location: Fully remote - Work: Not voice-based — text and image annotation only - No Interview, only assessment: 15–20 minute language skills assessment You will be contracted by Careerflow on behalf of one of our clients.
Senior Data Engineer
MercorCincinnatus is an enterprise staffing company that partners with leading technology companies to source and employ highly skilled professionals for full-time and long-term contingent roles. Cincinnatus serves as the employer of record for these engagements, providing W-2 employment, payroll, benefits, and compliance, while placing employees directly within client teams to work on high-impact initiatives. Roles hired through Cincinnatus are not project-based or freelance engagements. They are structured, role-based positions that typically involve full-time or fixed-term commitments, close collaboration with a client's internal teams, and integration into standard enterprise workflows. Cincinnatus is a legal entity separate from Mercor. While opportunities may be discovered through Mercor's platform, employment, onboarding, payroll, and benefits for these roles are administered by Cincinnatus. Equal Employment Opportunity Cincinnatus is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, reproductive health decisions, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, political views or activity, or any other legally protected characteristic. Cincinnatus is committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans throughout the job application process.
Role Description Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey. Position: Data Engineering Expert Type: Contract Compensation: $90–$125/hour Location: Remote Role Responsibilities - Build long-horizon pipeline tasks with deterministic rubrics to grade agent performance against verifiable ground truth. - Develop ETL/ELT pipelines and dbt models to produce specified output tables with incremental logic and defined watermark behavior. - Create Airflow/Dagster DAGs that pass test suites and data quality tests with known pass/fail cases. - Design warehouse schemas matching defined contracts and performance targets tied to measured query-time budgets. - Work independently in long focus sessions to create challenging scenarios for agent evaluation. - Ensure tasks have checkable answers without open-ended essays or subjective judgment calls. Qualifications - Must-Have: BS or MS in CS or related field. - 3+ years in data engineering or analytics engineering. - Expertise in dbt model development, pipeline orchestration (Airflow, Dagster, Prefect), warehouse design (Snowflake, BigQuery, Redshift, Databricks), and data quality and testing. - Comfortable with data engineering artifacts: dbt models, DAGs, schema docs, data contracts, and test suites. - Clear written communication skills to articulate reasoning and encode it into deterministic rubrics. Requirements - Hourly contractor. - Strong contributors are promoted based on task quality and throughput. Application Process - Upload resume. - AI interview based on your resume. - Submit form. For details about the interview process and platform information, please check: Interview Process For any help or support, reach out to: support@mercor.com PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

