Data Engineer
Location
Texas
Posted
2 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer
Sedgwick
• Designs, builds, and maintains resilient ETL/ELT pipelines that ingest data from on-premise systems, AWS services (S3, RDS), and Azure platforms (Blob Storage, Azure SQL), centralizing and curating data for consumption in Snowflake and downstream AI services. • Develops and maintains feature stores and analytically optimized datasets that support machine learning workflows, ensuring data is clean, versioned, reproducible, and statistically valid for Data Science teams. • Engineers data pipelines that enable generative AI use cases, including the automated extraction, transformation, chunking, and loading of structured and unstructured data into vector databases across AWS and Azure environments. • Acts as a Snowflake power user and technical lead, implementing advanced data modeling patterns, Snowpipe automation, and compute and storage optimization to support high-concurrency analytics and AI workloads. • Executes non-invasive data extraction strategies to unlock mission-critical data from decades-old legacy systems while preserving system stability and avoiding disruption to core business operations. • Designs and manages complex, cross-platform data workflows using orchestration tools such as Airflow, AWS Step Functions, and Azure Data Factory to ensure reliable, synchronized data movement across the organization’s multi-cloud architecture. • Partners closely with central IT, database administrators, infrastructure, and security teams to resolve connectivity and access challenges—including PrivateLink, IAM, network segmentation, and firewall controls—while securing production approval for new data integrations. • Implements automated data quality, validation, and observability frameworks to detect data drift, anomalies, and integrity issues that could negatively impact production analytics, machine learning, or AI systems. • Drives efficiency across the data ecosystem by optimizing storage, compute usage, and query performance in Snowflake, AWS, and Azure, ensuring responsible cost management and measurable ROI for Transformation Office initiatives. • Operates as a dedicated engineering partner to MLOps, Data Science, and AI teams, rapidly iterating on evolving data requirements and translating experimental use cases into scalable, production-ready data solutions.
Job Requirements
- Master’s degree in Computer Science, Data Engineering, or a related field from an accredited college or university preferred.
- Six (6) years of hands-on data engineering experience, with a track record of building production-grade pipelines for Data Science and AI in multi-cloud environments or equivalent combination of education and experience required.
- Expert-level proficiency in Snowflake architecture, including data sharing, performance tuning, and the integration of Snowflake with external cloud AI services
- Advanced, hands-on knowledge of AWS (S3, Glue, Lambda) and Azure (Data Factory, Synapse) data services
- Mastery of Python, SQL, and PySpark.
- Deep experience with data orchestration and containerization (Docker)
- Proven ability to interface with "old world" tech (on-premise SQL, Mainframe extracts, flat files) and transform it for modern cloud consumption
- A strong understanding of the specific data needs for Machine Learning (feature engineering) and Generative AI (vectorization and embedding pipelines)
- A "get-it-done" attitude, capable of navigating enterprise bureaucracy and technical debt to ship code at the speed required by a Transformation Office
- Ability to work in a team environment
- Ability to meet or exceed Performance Competencies
Benefits
- health insurance
- retirement plans
- paid time off
- flexible work arrangements
- professional development
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer I – Automated Content, Scientific Products
BDBD is a global medical technology company that is advancing the world of health. www.bd.com
• Apply data expertise and automation principles to generate accurate, customer-facing scientific content. • Serve as a subject matter expert (SME) for data handling, standardization, and automation. • Clean, validate, wrangle, and standardize product data and metadata. • Develop and refine data pipelines and content generation processes. • Optimize interactions with the custom in-house SQL database + VB.NET platform. • Identify and implement opportunities to eliminate redundant data entry and reduce errors. • Apply automation, scripting, and data technologies to streamline content development. • Lead continuous improvement initiatives for data flow and best practices in scientific content creation. • Support new product initiatives by ensuring data readiness and accurate generation of materials. • Independently drive and manage project scope, roadmaps, and end-to-end deliverables.
Role Description We are looking for a Data Engineer to join a team of analytics and machine learning experts. The hire will be responsible for: - Building tooling to support analytics - Helping to extend our machine learning platform - Expanding and optimizing our data pipeline architecture - Supervising junior engineers - Interfacing with the Development team to create cross-team solutions The ideal candidate is an experienced data engineer and data wrangler who enjoys optimizing data systems and building them from the ground up. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. Experience in analytics and statistics is a major bonus. The right candidate will be excited by the prospect of optimizing or even re-designing our company's data architecture to support our next generation of products and data initiatives, as well as mentoring and guiding junior members of the team. Qualifications - 3+ years of experience in a Data Engineer role - Bachelor's degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field - Advanced working SQL knowledge and experience with relational databases (including Postgres and MySQL) - Experience building data pipelines, architectures, and data sets from raw, loosely structured data - A history of focusing on test-driven design and results for repeatable and maintainable processes and tools - Experience building processes supporting data transformation, data structures, metadata, dependency, and workload management - Working knowledge of message queuing, stream processing, and highly scalable data stores - Strong project management and organizational skills - Experience managing junior engineers and guiding a team through project planning, execution, and quality control stages Requirements - Experience with object-oriented design in Python - Experience with data pipeline and workflow management tools - Experience with AWS cloud services: EC2, RDS, Redshift, Glue, S3 Additional Pluses - Strong analytic skills and understanding of statistical methodologies - Experience building machine learning models - Experience handling data from acquisition to usage in models - Experience building and maintaining RestAPI systems, Flask apps, and state machines - Experience with continuous integration, especially in a data science context - Experience with Ruby on Rails Benefits - Competitive salary and benefits package - Flexible, remote work - Fun, fast-paced work environment - Dynamic start-up culture - Ability to make an immediate impact in a growth stage company - Convenient downtown Chicago office located in the heart of the city - Equal opportunity employer
• Analyze and translate high-level customer requirements into detailed designs to solve complex business problems • Design solutions that align to the long-term plan for a service or product based on deep expertise and customer insights • Partner with various engineering teams to create and manage a long-term data and information architecture and execution roadmap for a product or service • Define data solutions and develop code across multiple products or services, as well as, influence or drive architectural changes • Ensure consistent, usable, forward-looking, maintainable test infrastructure; draw from a large base of design patterns, is an expert in available technologies, and is adept at identifying practices that work well • Identify code across multiple code bases to optimize, refactor, and reuse code to improve performance and maintainability while ensuring maximum efficiency, effectiveness, and return on investment • Lead code reviews across the product or service, understand the root causes of issues, and find ways to resolve them • Proactively identify performance and availability issues, troubleshoot, provide effective options, and resolve issues in production that could span multiple product areas • Develop and maintain thorough architectural documentation for the product or service • Design products or services by using secure programming patterns and by finding, fixing, and enhancing security in existing applications • Ensure security best practices are part of design and implementation of new features and applications • Estimate for projects that span multiple product areas including work, time, resources and skill needs • Proactively identify technologies or solutions that differ from current technology stack or represent innovative uses of existing technologies • Construct and deliver proposed solution strategies for potential new technologies and work with architecture to review and approve proposals • Mentor and coach other engineers by participating in design and code reviews and share best practices; proactively seek mentorship from others • Lead the effort in defining the engineering lifecycle and practices for team and associated teams in partnership with the principal data engineer • Drive collaboration across multiple teams; find ways accomplish more by enabling others • Anticipate business needs and present options to leadership and business stakeholders with product managers • Other duties or responsibilities as assigned according to the team and/or country specific requirements
Role Description UPLabs is a dynamic venture studio dedicated to building innovative startup companies from the ground up. Our team thrives on solving complex problems, driving technological advancements, and creating impactful digital products. We’re seeking a highly skilled professional to join our growing team and contribute to our mission of launching the next wave of AI powered solutions for enterprise customers. Technical Challenge: - Design, build, and evolve scalable data pipelines and data platforms to reliably power AI/ML models and agentic workflows for our customers. - Partner closely with forward deployed product and engineering teams to translate ambiguous requirements into robust, maintainable data solutions. Responsibilities: - Architect, build, and maintain scalable, production-grade data pipelines and data models to enable reliable ingestion, transformation, and delivery of data. - Own end-to-end pipeline reliability, including orchestration, monitoring, alerting, and incident response to meet data freshness and quality expectations. - Partner with product, engineering, and analytics stakeholders to define data requirements, translate them into clear technical specifications, and deliver iteratively. - Improve performance, scalability, and cost efficiency of data workloads by tuning SQL queries, optimizing storage/compute patterns, and standardizing best practices across projects. - Document data models, pipeline behavior, and operational runbooks to ensure maintainability and smooth knowledge transfer across accounts. Qualifications - Strong data engineering expertise, including designing and operating data pipelines, data models, and batch/stream processing workflows in a production environment. - Proficiency with Python for building data pipelines, automation, and data tooling. - Advanced SQL skills for data transformation, analysis, and performance tuning. - Experience working with PostgreSQL, including schema design, query optimization, and data integrity best practices. - Experience building data pipelines and workloads on Databricks. - Experience using dbt to develop, test, and maintain modular analytics engineering workflows. - Experience working with MongoDB or other document databases as part of modern data stacks. - Experience working with one or more major cloud platforms: AWS, Azure, or GCP. Preferred Skills - Experience using Snowflake for cloud data warehousing, modeling, and analytics workloads. - Working knowledge of Apache Spark for large-scale data processing. Company Description We build high-growth technology startups that enable faster, cleaner, and safer movement of people and goods. Our vision is to transform the moving world by pairing leading corporations and entrepreneurs with a proven methodology for launching and scaling software and hardware companies. We work with corporate investors over a multi-year period to launch a portfolio of mobility-focused ventures. Our team is dedicated to the first year of a new venture’s life cycle, from ideation to minimum viable product build (and beyond) to recruiting and hiring the full-time team who will scale the business. Location LATAM Remote




