A people-first, purpose-driven, independent network of healthcare and scientific communications agencies, Vaniam Group describes itself as the healthcare communications partner of
Data Engineer, Enterprise Data, Analytics and Innovation
Location
United States
Posted
15 days ago
Salary
$110K - $125K / year
Seniority
Mid Level
Job Description
Data Engineer, Enterprise Data, Analytics and Innovation
Vaniam Group
Role Description Are you passionate about building robust data infrastructure and enabling innovation through engineering excellence? As our Data Engineer, your goal is to own and evolve the foundation of our data infrastructure. You will be central in ensuring data reliability, scalability, and accessibility across our lakehouse and transactional systems. This role is ideal for someone who thrives at the intersection of engineering and innovation, ensuring our data platforms are robust today while enabling the products of tomorrow. A Day in the Life - Lakehouse and Pipelines - Design, build, and operate reliable ETL and ELT pipelines in Python and SQL. - Manage ingestion into Bronze, standardization and quality in Silver, and curated serving in Gold layers of our Medallion architecture. - Maintain ingestion from transactional MySQL systems into Vaniam Core to keep production data flows seamless. - Implement observability, data quality checks, and lineage tracking to ensure trust in all downstream datasets. - Data Modeling and Governance - Develop schemas, tables, and views optimized for analytics, APIs, and product use cases. - Apply and enforce best practices for security, privacy, compliance, and access control, ensuring data integrity across sensitive healthcare domains. - Maintain clear and consistent documentation for datasets, pipelines, and operating procedures. - Integration of New Data Sources - Lead the integration of third-party datasets, client-provided sources, and new product-generated data into Vaniam Core. - Partner with product and innovation teams to build repeatable processes for onboarding new data streams. - Ensure harmonization, normalization, and governance across varied data types (scientific, engagement, operational). - Analytics and Predictive Tools - Collaborate with the innovation team to prototype and productionize analytics, predictive features, and decision-support tools. - Support dashboards, APIs, and services that activate insights for internal stakeholders and clients. - Work closely with Data Science and AI colleagues to ensure engineered pipelines meet modeling and deployment requirements. - Reliability and Optimization - Monitor job execution, storage, and cluster performance, ensuring cost efficiency and uptime. - Troubleshoot and resolve data issues, proactively addressing bottlenecks. - Conduct code reviews, enforce standards, and contribute to CI/CD practices for data pipelines. Qualifications - 5+ years of professional experience in data engineering, ETL, or related roles. - Strong proficiency in Python and SQL for data engineering. - Hands-on experience building and maintaining pipelines in a lakehouse or modern data platform. - Practical understanding of Medallion architectures and layered data design. Requirements - Familiarity with modern data stack tools, including: - Spark or PySpark - Workflow orchestration (Airflow, dbt, or similar) - Testing and observability frameworks - Containers (Docker) and Git-based version control - Excellent communication skills, problem-solving mindset, and a collaborative approach. Benefits - 100% remote environment with opportunities for local meet-ups. - Positive, diverse, and supportive culture. - Passionate about serving clients focused on Cancer and Blood diseases. - Investment in you with opportunities for professional growth and personal development through Vaniam Group University. - Health benefits – medical, dental, vision. - Generous parental leave benefit. - Focused on your financial future with a 401(k) Plan and company match. - Flexible Time Off policy for rest and relaxation. - Volunteer Time Off for community involvement. - Emphasis on Personal Wellness. - Virtual workout classes. - Discounts on tickets, events, hotels, child care, groceries, etc. - Employee Assistance Programs. - Salary offers are based upon several factors including experience, education, skills, training, demonstrated qualifications, location, and organizational need. The range for this role is $110,000 - $125,000.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Tech Lead – Data Engineering
ExperianWe're unlocking the power of data to help create a better tomorrow.
Role Description Buscamos um(a) profissional para atuar como referência técnica em Engenharia de Dados, liderando a evolução da plataforma de dados e garantindo a entrega de soluções escaláveis, eficientes e alinhadas às necessidades do negócio. - Atuar como referência técnica na arquitetura de dados, com foco em ambientes Databricks e AWS. - Ser responsável pelos pipelines críticos de dados, incluindo ingestão, processamento e disponibilização (serving). - Garantir a adoção de padrões de engenharia, assegurando qualidade de código, testes, observabilidade, governança e cumprimento de SLAs. - Liderar decisões de arquitetura e design técnico, considerando aspectos de escalabilidade, performance, segurança e otimização de custos. - Atuar ativamente em iniciativas de FinOps, promovendo a eficiência operacional e a melhor utilização dos recursos da plataforma. - Orientar e desenvolver engenheiros de dados por meio de mentoria, code reviews e disseminação de boas práticas. - Trabalhar em parceria com stakeholders de negócio, produto e tecnologia para traduzir demandas em soluções robustas e escaláveis. - Apoiar a priorização técnica do roadmap, equilibrando necessidades de entrega, sustentabilidade da plataforma e redução de dívida técnica. - Liderar a gestão e resolução de incidentes críticos, contribuindo para a evolução da maturidade operacional do ambiente de dados. - Promover a melhoria contínua dos processos, ferramentas e práticas da área de Engenharia de Dados. Qualifications - Experiência consolidada em engenharia de dados (+5 anos). - Experiência prévia em papel de liderança técnica ou como referência dentro do time. - Forte domínio de SQL, Python e/ou Scala. - Experiência com arquiteturas modernas de dados (Data Lake / Lakehouse). - Experiência prática com Spark / Databricks (ou similar). - Experiência com cloud (preferencialmente AWS). - Conhecimento sólido em modelagem de dados (analítica e operacional). - Experiência com CI/CD, versionamento e boas práticas de engenharia. Requirements - Experiência com governança em escala (ex: Databricks Unity Catalog). - Implementação de Data as a Product / Data Mesh / Embedded Data. - Experiência com streaming (Kafka, Kinesis). - Data Observability e Data Quality frameworks. - Experiência com FinOps aplicado a dados. - Uso de IA para aumento de produtividade em engenharia de dados. Soft Skills - Capacidade de influenciar sem autoridade formal. - Comunicação clara com áreas técnicas e de negócio. - Tomada de decisão baseada em trade-offs (custo, prazo, qualidade). - Capacidade de estruturar problemas complexos. - Mentalidade de dono (ownership) sobre a plataforma. Benefits - Cultura inclusiva e ambiente equilibrado entre carreira e compromissos pessoais. - Reconhecimento como uma das melhores empresas para se trabalhar no país. - Certificações de mercado, incluindo Great Place To Work™ e Top Employers. - Avaliação de 4,6 no Glassdoor.
Data Engineer
Malvern PanalyticalWe are a global leader in the analytics of material & life sciences.
• Design, own, and scale the data foundation powering AI-driven workflows • Architect the governed context layer for AI agents • Establish intelligent feedback loops to drive continuous learning • Deliver advanced analytical products to measure performance
Data Engineer
PartnerOneWe are the leaders in Big Data management through hyper-automation, virtualized cloud tiering, metadata and AI
**Data Pipeline Development:** - Design and build ETL pipelines using Microsoft Fabric (Dataflow Gen2, Notebooks, or equivalent tools) - Write optimized SQL queries and transformations for data ingestion from designated source systems - Apply data quality rules and validation logic at each pipeline stage - Implement incremental loads and manage refresh schedules for performance - Escalate to Lead for architectural decisions or complex transformation patterns **Data Quality & Validation:** - Define and implement data quality checks at ingestion, transformation, and output stages - Perform ongoing data validation to ensure pipeline outputs align with business logic and source system expectations - Identify, document, and escalate data quality issues with root cause analysis - Maintain data quality dashboards and SLA monitoring - Support UAT for new data sources or transformation logic **Transformation & Modeling:** - Build and maintain data transformations using Power Query, SQL, or Python as appropriate - Develop dimensional models and define aggregation logic aligned with analytics requirements - Optimize data structures for performance and maintainability - Document transformation logic, lineage, and assumptions per team standards - Collaborate with Lead to define semantic **Operational Support:** - Troubleshoot pipeline failures and performance issues; coordinate resolution with IT/Engineering - Respond to data discrepancy reports from business users and analysts - Maintain documentation of data sources, data dictionaries, and transformation specifications - Support capacity planning and optimization of Fabric environments and pipelines models and calculated metrics
• Drive execution against the product roadmap for core retail platforms, integrations, and data foundations; contribute to roadmap prioritization in partnership with the Director of Product. • Translate complex operational and business needs into scalable, reusable platform capabilities rather than bespoke solutions. • Lead product execution for OMS-adjacent capabilities including order lifecycle management, inventory visibility, fulfillment orchestration, routing, splits, cancellations, and returns. • Define and evolve integration patterns, APIs, and data contracts across OMS, POS, ERP, WMS, CEP, CDP, and analytics platforms. • Own requirements for eventing, instrumentation, and data quality standards; define and maintain the governance model for core data domains (customer, order, product, inventory). • Partner with Data and Analytics teams to enable reliable reporting, agent-enabled insights, self-serve analytics, and downstream activation for experimentation, measurement, and AI use cases. • Research, evaluate, and scorecard platform and vendor solutions with a critical eye on architecture, scalability, integration complexity, and total cost of ownership; provide feasibility analysis to inform build-vs-buy decisions. • Identify where AI or automation can reduce operational cost, improve data quality, or accelerate fulfillment outcomes. Evaluate solutions pragmatically and make the case for prioritizing them.
