Job Closed
This listing is no longer active.
Senior Data Engineer – MSI
Location
United States
Posted
73 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer – MSI
MSI
• Design and manage data systems, pipelines, and management tools • Oversee junior data engineers, recommend and deploy data models, and lead projects involving data collection and storage systems • Collaborate with data architect, analysts, and stakeholders to identify and document needs/requirements for data pipelines and process automation • Design, build, document, test, and maintain data pipelines using industry best practices • Perform ETL, ELT operations in accordance with enterprise data governance and security standards • Develop data quality and governance process to ensure the accuracy and quality of the data through inspection, validation, processing, anomaly detection
Job Requirements
- 5-7 years of relevant experience required
- BA/BS in relevant discipline required
- Vast experience with multiple development methodologies
- Experience of building large, complicated data pipelines across different platforms, data sources, data structures
- Experience of working with relational databases such as SQL Server, Oracle, etc., and SQL scripting
- Experience with Power BI
- Experience of cloud platforms (like AWS, Azure, GCP) and its ETL tools and techniques of sourcing, maintaining, and updating data, data warehousing, data cleansing & transformation, etc.
- Programming experience in Python, Spark, and other similar languages
- Advanced programming knowledge and experience with large-scale processing engines are required, as well as a deeper understanding of database security and compliance tools.
Benefits
- Exceptional service through a dedicated team that makes rapid resolutions a priority
- Passion for crafting solutions for the important risks facing individuals and businesses
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer – MSI
Baldwin GroupBaldwin Group is an insurance distribution and advisory firm that is working to give its customers “peace of mind to pursue their dreams, purpose, and passion
• Collaborate with data architect, analysts, and stakeholders to identify and document needs/requirements for data pipelines and process automation • Design, build, document, test, and maintain data pipelines using industry best practices • Performs ETL, ELT operations in accordance with enterprise data governance and security standards • Develop data quality and governance process to ensure the accuracy and quality of the data through inspection, validation, processing, anomaly detection
• Analysis and Planning of Loads/Pipelines: • Assess the data warehouse architecture and requirements. • Map data, transformations and processes in GCP services (Cloud Storage, BigQuery, Dataproc). • Define data migration strategy (full load, incremental, CDC). • Develop a data architecture plan on GCP. • Design and Data Modeling on GCP: • Design table schemas in BigQuery, considering performance, cost and scalability. • Define partitioning and clustering strategies for BigQuery. • Model data zones in Cloud Storage (Bronze, Silver and Gold). • ELT/ETL Pipeline Development: • Create data transformation routines using Dataproc (Spark) or Dataflow to load data into BigQuery. • Translate business logic and existing transformations into GCP. • Implement data validation and data quality mechanisms. • Infrastructure Provisioning and Management: • Use IaC tools (Terraform) to provision and manage GCP resources (BigQuery datasets/tables, Cloud Storage buckets, Dataproc clusters). • Configure and optimize Dataproc clusters for different workloads. • Manage networking, security (IAM) and access in GCP. • Performance and Cost Optimization: • Optimize queries in BigQuery to reduce costs and improve performance. • Tune and optimize Spark jobs on Dataproc. • Monitor and optimize GCP resource usage to control costs. • Data Security and Governance: • Implement and ensure data security in transit and at rest. • Define and apply IAM policies to control access to data and resources. • Ensure compliance with data governance policies. • Monitoring and Support: • Troubleshoot performance and functional issues of data pipelines and GCP resources. • Documentation: • Document the architecture, data pipelines, data models and operational procedures. • Communication: • Communicate effectively with team members, stakeholders and other areas of the company. • Ensure clear communication between architecture definitions and software components, and the evolution and quality of the team's deliverables. • Jira / Agile Methodologies: • Be familiar with agile methodologies, their ceremonies, and be proficient with the Jira tool.
• Conduzir apresentações técnicas, workshops e treinamentos sobre soluções de dados na GCP; • Desenvolver e executar provas de conceito (PoCs) e demonstrações hands-on; • Apoiar clientes na construção de arquiteturas modernas de dados (data platforms); • Atuar na disseminação de soluções de Data Analytics, Machine Learning e AI; • Trabalhar em conjunto com Customer Engineers e Account Executives na qualificação de oportunidades; • Apoiar estratégias de adoção de produtos de dados da GCP; • Criar conteúdos técnicos e materiais de treinamento; • Interagir com clientes e parceiros em toda a América Latina; • Promover boas práticas em engenharia e arquitetura de dados na nuvem;
• Architect and optimize the **Snowflake data platform**, including warehouse sizing, cost optimization, storage strategy, and access controls • Design and own **dbt project structure**, including models, macros, testing, documentation, and scalable data contracts • Build and maintain **ELT pipelines** using Fivetran and orchestration tools, ensuring reliable data ingestion across multiple sources • Implement and manage **data quality and observability frameworks** (tests, SLAs, lineage, monitoring, incident response) • Translate **business requirements into scalable data models and reusable datasets** • Partner with **Analytics, Product, and Marketing teams** to deliver high-quality, self-service data solutions • Establish and enforce **data modeling standards** (dimensional and ER models) • Optimize **query performance and warehouse costs** in Snowflake, providing insights to stakeholders • Define and enforce **data governance policies**, including RBAC, masking, and PII handling • Own end-to-end delivery of **complex data initiatives**, from design to production • Participate in **code reviews and technical design discussions**, raising engineering standards • Identify and reduce **technical debt** across pipelines, models, and infrastructure



