Co-creating solutions for a better future
Big Data Engineer
Location
Mexico
Posted
8 days ago
Salary
0
Seniority
Senior
Job Description
Big Data Engineer
Stefanini LATAM
• As part of the Data Engineering team, you will be responsible for design, development and operations of large-scale data systems operating at petabytes scale. • You will be focusing on real-time data pipelines, streaming analytics, distributed big data and machine learning infrastructure. • You will interact with the engineers, product managers, BI developers and architects to provide scalable robust technical solutions.
Job Requirements
- Min 6-8 years of BIG data development experience.
- Demonstrates up-to-date expertise in Data Engineering, complex data pipeline development.
- Experience in agile models
- Design, develop, implement and tune large-scale distributed systems and pipelines that process large volume of data; focusing on scalability, low -latency, and fault-tolerance in every system built.
- Experience with Java, Python to write data pipelines and data processing layers
- Experience in Airflow & Github.
- English conversational
- Experience in writing map-reduce jobs.
- Demonstrates expertise in writing complex, highly-optimized queries across large data sets
- Proven, working expertise with Big Data Technologies Hadoop, Hive, Kafka, Presto, Spark, HBase.
- Highly Proficient in SQL.
- Experience with Cloud Technologies ( GCP, Azure)
- Experience with relational model, memory data stores desirable ( Oracle, Cassandra, Druid)
- Provides and supports the implementation and operations of the data pipelines and analytical solutions
- Performance tuning experience of systems working with large data sets
- Experience in REST API data service – Data Consumption
- Retail experience is a huge plus.
- Advanced english
Benefits
- Sueldo acorde a experiencia
- Prestaciones de Ley
- Días extra con goce se sueldo
- Convenios en escuelas de idiomas, entretenimiento, cursos y más
- Vales de despensa
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Semi Senior/Senior Ingeniero de Datos – Sector Bancario
DevsuDevsu is a technology agency that provides software development services, IT augmentation and staffing.
• Diseñar y desarrollar soluciones de datos alineadas con la visión de Arquitectura de Datos. • Crear y optimizar ETLs, ELTs y APIs para manejo eficiente de datos. • Implementar estrategias de testing para validar calidad funcional y no funcional. • Proponer mejoras continuas en procesos y productos de datos. • Resolver incidencias técnicas y documentar soluciones conforme a estándares. • Mentorizar y apoyar el onboarding de nuevos integrantes del equipo.
• Design, build, and optimize end-to-end ETL pipelines for legal data from multiple jurisdictions, including cleaning, transformation, chunking, validation, embedding, and ingestion into vector databases • Work extensively with XML-based legal data feeds: parse, validate, normalize, and transform XML structures into scalable internal schemas and unified document formats • Develop and maintain data models and storage schemas that support continuously updated datasets while ensuring consistency, scalability, and accuracy across diverse datasets and large amounts of data • Coordinate data handover and integration from multiple internal and external data providers, including official sources, APIs, and web scraping pipelines, ensuring reliable and timely updates • Implement and continuously refine metadata enrichment strategies to maximize searchability, ranking quality, and relevance of legal information in vector databases. • Build and maintain a high-performance search and retrieval infrastructure enabling agent-based systems to call search functions and retrieve the most relevant legal information efficiently • Collaborate with product, AI, and legal domain experts to deliver high-quality, reliable data solutions • Own the data integration of one jurisdiction end-to-end
Data Architect
AxiomWhere legal teams can find the right talent for everything from routine in-house tasks to complex outside counsel work.
• Design, document, and maintain Axiom's enterprise data model, ensuring coherence and consistency across disparate systems and platforms. • Establish and enforce data governance policies, data quality standards, and associated assurance mechanisms to ensure the reliability and integrity of organizational data assets. • In coordination with the VP, Enterprise Applications, define and maintain the data architecture roadmap, including effort scoping, dependency identification, and alignment with enterprise technology strategy. • Directly manage the Data Engineer; provide technical mentorship, set priorities, and support professional development. • Serve as the primary data architecture liaison to the Product, Operations, and Data Science functions. • In coordination with the VP, Enterprise Applications, evaluate and determine the technology stack for AI solutions and other data initiatives. • Collaborate with the VP, Enterprise Applications and Integrations Lead to reconcile roadmap priorities and ensure integration-produced data meets enterprise data model standards. • Participate collaboratively in the design and implementation of Model Context Protocol integrations, enabling AI tooling to access Axiom's information assets. • Administer and govern the Databricks environment in coordination with the Data Engineer. • Establish and maintain data documentation standards: entity definitions, lineage, ownership, and access policies.
• Build and maintain ELT pipelines using Fivetran and custom API integrations via Dagster, including setting up new source connectors and troubleshooting. • Manage and improve Dagster orchestration — scheduling jobs, debugging failures, and keeping pipeline runtime tight. • Maintain and improve GitHub Actions CI/CD workflows to ensure safe, reliable deployment of dbt models into production. • Manage and maintain Snowflake infrastructure via Terraform — warehouses, roles, permissions, and resource monitors. • Write dbt models across staging, intermediate, and mart layers — clean, well-tested, and documented so the whole team can trust them. • Identify and address data quality issues before they become problems for the people depending on the data. • Handle ad-hoc analytics requests from stakeholders across the business — translate a business question into a query, turn a query into an insight. • Contribute to recurring deliverables: monthly growth reports, provider audits, forecasting documents, and ongoing Looker maintenance.




