Leader in technology solutions for the Taft-Hartley industry.
Senior Data Engineer
Location
United States
Posted
122 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
Bridgeway Benefit Technologies
• Design, develop, and maintain a scalable data warehouse/lakehouse environment. • Design and implement ELT pipelines to ingest, transform, and deliver high-quality data for analytics and reporting, incorporating current best practices, such as “pipelines as code”. • Ensure data security and compliance, including role-based access controls for security, encryption, masking, and governance best practices to ensure compliant handling of sensitive information. • Optimize performance of data workflows and storage for cost efficiency and speed. • Partner with engineers, analysts, and stakeholders to meet data needs; balance cost, performance, simplicity, and time-to-value while mentoring teams and documenting standards. • Define and implement robust testing frameworks, enforce data contracts, and establish observability practices including lineage tracking, SLAs/SLOs, and incident response runbooks to maintain data integrity and trustworthiness. • Monitor, troubleshoot, and resolve data & automation issues. • Collaborate within an Agile-Scrum framework and develop comprehensive technical design documentation to ensure efficient and successful delivery. • Serve as a trusted expert on organizational data domains, processes, and best practices.
Job Requirements
- 5+ years of experience in data engineering and ELT with a focus on large-scale data platforms
- 3+ years of experience with Databricks
- Advanced proficiency in analytical SQL, including ANSI SQL, T-SQL, and Spark SQL
- Strong Python skills for data engineering
- Expertise in data modeling
- Hands-on experience with data quality and observability practices (tests, contracts, lineage tracking, alerts)
- Practical knowledge of orchestration tools and CI/CD concepts for data workflows
- Excellent communication and a track record of technical leadership and mentoring
- Strong understanding of integrating data solutions with AI and machine learning models
- Strong problem-solving skills and attention to detail.
- Experience with version control systems like Git preferred
- Strong understanding of data governance and best practices in data management, with hands-on experience using Unity Catalog
- Hands-on experience in designing and managing data pipelines using Delta Live Tables (DLT) on Databricks
- Streaming and ingestion tools, such as Kafka, Kinesis, Event Hubs, Debezium, or Fivetran
- DAX, LookML, dbt; Airflow/Dagster/Prefect, Terraform; Azure DevOps; Power BI/Looker/Tableau; GitHub CoPilot knowledge is a plus
- Bachelor’s degree in Computer Science, Information Technology, or a related field. Master’s degree preferred
Benefits
- Preference given to East Coast candidates.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer – Integration Hub, Data Pipelines
Cuculus GmbHAffordable energy and water for everyone.
• Design, build, and maintain robust ETL/ELT data pipelines for batch and streaming workloads. • Implement data ingestion and transformation workflows using Apache Airflow, Apache NiFi, Apache Spark, and Kafka. • Integrate data from multiple sources including REST APIs, files, relational databases, message queues, and external SaaS platforms. • Optimize pipelines for performance, scalability, reliability, and cost efficiency. • Develop and operate a centralized data integration hub that supports multiple upstream and downstream systems. • Build reusable, modular integration components and frameworks. • Ensure high availability, fault tolerance, and observability of data workflows. • Design and manage data warehouses, data lakes, and operational data stores using PostgreSQL and related technologies. • Implement appropriate data modeling strategies for analytical and operational use cases. • Manage schema evolution, metadata, and versioning. • Implement data validation, monitoring, and reconciliation mechanisms to ensure data accuracy and completeness. • Enforce data security best practices, access controls, and compliance with internal governance policies. • Establish logging, alerting, and auditability across pipelines. • Automate data workflows, deployments, and operational processes to support scale and reliability. • Monitor pipelines proactively and troubleshoot production issues. • Improve CI/CD practices for data engineering workflows. • Work closely with data scientists, analysts, backend engineers, and business stakeholders to understand data requirements. • Translate business needs into technical data solutions.
• Design and develop conceptual, logical, and physical data models for various domains • Lead the development of data modeling standards, best practices, and guidelines • Develop end-to-end solution architectures for data-driven and AI-focused applications • Mentor and guide junior data modelers • Design data models to support enterprise and operational reporting • Collaborate with data scientists to develop features for machine learning models • Optimize data models for performance and scalability • Ensure data models comply with data governance policies and standards • Identify and define data quality checks and validation processes • Implement data quality monitoring and reporting mechanisms
• Análisis, diseño y desarrollo de soluciones de ingeniería de datos en entornos cloud. • Construcción y mantenimiento de pipelines de datos. • Tratamiento, integración y análisis de grandes volúmenes de información. • Optimización de procesos de datos en plataformas Microsoft Azure. • Data engineering y data analysis.
Senior Azure Data Engineer
SmartbridgeSimplifying business transformation through thought leadership and innovation. Bring your digital agenda to reality.
• Define the target-state Azure data architecture (ingestion, orchestration, storage zones, serving patterns), security/networking boundaries, cost/perf tradeoffs, and promotion strategy (Dev→Test→Prod). • Implement robust ELT/ETL with ADF/Synapse Pipelines (parameters, reusable templates, CI/CD). • Hands-on in T-SQL and Python/PySpark for transformations, utilities, and tests. • Physical/semantic modeling, partitioning, columnstore strategies, statistics management, query plan analysis, index design, concurrency & transaction isolation, workload management. • SLA/SLO definitions, Azure Monitor / Log Analytics / App Insights dashboards and alerts; error handling, retries/backoff, idempotency, CDC and schema drift strategies. • RBAC, Key Vault, managed identities, private endpoints/VNet, data masking patterns; document data contracts and access patterns. • Code reviews, PR discipline, mentoring, and crisp documentation/runbooks for client handoff.




