Job Closed

This listing is no longer active.

Keyrus logo
Keyrus

#MakeDataMatter #HumanizingTheFuture

Senior Data Engineer – Azure and Databricks Specialist

Data EngineerData EngineerFull TimeRemoteSeniorTeam 1,001-5,000Since 1996H1B SponsorCompany SiteLinkedIn

Location

Brazil

Posted

96 days ago

Salary

0

Seniority

Senior

Job Description

Senior Data Engineer – Azure and Databricks Specialist

Keyrus

• Build and maintain data ingestion and processing pipelines from multiple sources • Develop extraction, transformation and load (ETL/ELT) processes using Python, Apache Airflow and Azure Data Factory • Structure and manage data in the Data Lake, ensuring organization and data governance • Implement transformations and data processing using Spark / SparkSQL on Databricks • Support, monitor and evolve the client’s data platform, ensuring stability and performance • Work with various databases and storage technologies, such as SQL Server, Oracle, MongoDB and Hadoop • Collaborate with multidisciplinary teams in a dynamic, data-driven environment

Job Requirements

  • Strong experience with Azure Databricks
  • Knowledge of Azure Data Factory and Azure Data Lake
  • Experience developing with Python
  • Knowledge of Apache Spark / SparkSQL
  • Experience with Oracle and SQL Server databases

Benefits

  • Culture of equality
  • Diversity and inclusion

Related Categories

Related Job Pages

More Data Engineer Jobs

Centene Corporation logo

Senior Data Engineer

Centene Corporation

Transforming the health of the communities we serve, one person at a time.

Data Engineer96 days ago
OtherRemoteTeam 10,001+Since 1984H1B No Sponsor

You could be the one who changes everything for our 28 million members by using technology to improve health outcomes around the world. As a diversified, national organization, Centene's technology professionals have access to competitive benefits including a fresh perspective on workplace flexibility. Position Purpose: Develops and operationalizes data pipelines to make data available for consumption (reports and advanced analytics), including data ingestion, data transformation, data validation / quality, data pipeline optimization, and orchestration. Engages with the DevSecOps Engineer during continuous integration and continuous deployment. - Helps guide the design and implementation of complex data management procedures around data staging, data ingestion, data preparation, data provisioning, and data destruction (scripts, programs, automation, assisted by automation, etc.) - Provides guidance to data engineers in the design, development, implementation, testing, documenting, and operating of large-scale, high-volume, high-performance data structures for business intelligence analytics - Designs, develops, and maintains real-time processing applications and real-time data pipelines - Ensure quality of technical solutions as data moves across Centene’s environments - Provides senior level knowledge and insight into the changing data environment, data processing, data storage, and utilization requirements for the company and drives the team toward viable solutions - Develops, constructs, tests, and maintains architectures using advanced programming language and tools - Drives ways to improve data reliability, efficiency, and quality and deploys a solution; use data to discover tasks that can be automated - Performs other duties as assigned - Complies with all policies and standards Education/Experience: A Bachelor's degree in a quantitative or business field (e.g., statistics, mathematics, engineering, computer science) and Requires 4 – 6 years of related experience. Or equivalent experience acquired through accomplishments of applicable knowledge, duties, scope and skill reflective of the level of this position. Technical Skills: - One or more of the following skills are desired. - - Snowflake - SQL - Informatica IDMC Pay Range: $87,000.00 - $161,300.00 per year Centene offers a comprehensive benefits package including: competitive pay, health insurance, 401K and stock purchase plans, tuition reimbursement, paid time off plus holidays, and a flexible approach to work with remote, hybrid, field or office work schedules. Actual pay will be adjusted based on an individual's skills, experience, education, and other job-related factors permitted by law, including full-time or part-time status. Total compensation may also include additional forms of incentives. Benefits may be subject to program eligibility. Centene is an equal opportunity employer that is committed to diversity, and values the ways in which we are different. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or other characteristic protected by applicable law. Qualified applicants with arrest or conviction records will be considered in accordance with the LA County Ordinance and the California Fair Chance Act

United States
$87K - $161K / year
Job Closed
Solventum logo

Senior Data Engineer – AI, ML Enablement

Solventum

Enabling better, smarter, safer healthcare to improve lives.

Data Engineer96 days ago
Full TimeRemoteTeam 10,001+H1B No Sponsor

• Architect, design and develop AI/ML solutions to solve novel business problems, taking ideas from problem framing through proof of concept, pilot, and full production deployment. • Implement both traditional machine learning models and modern AI techniques, including large language models (LLMs), generative AI, retrieval-augmented generation (RAG), and agentic AI systems. • Collaborate closely with product managers, engineers, data scientists, and business stakeholders to define project scope, technical approach, success metrics, and deployment requirements. • Build robust, scalable, and maintainable AI systems with a focus on accuracy, robustness, fairness, and explainability, adhering to responsible AI principles. • Develop and execute rigorous model evaluation frameworks to assess performance, generalization, bias, and resilience to edge cases or adversarial inputs. • Transition AI prototypes into production-ready systems by partnering with platform and engineering teams to ensure seamless deployment, monitoring, and lifecycle management. • Contribute to documentation, governance, and compliance processes to meet enterprise and regulatory standards, especially in healthcare and other regulated environments.

Mexico
First American logo

Senior Data Engineer (Remote in CA)

First American

First American is on a mission to deliver a variety of real estate-focused services and solutions. As an employer, First American has been recognized for its ex

Data Engineer96 days ago

Who We Are Join a team that puts its People First! As a member of the First American family of companies, First American Trust is a federal savings bank that has provided banking, wealth management, and trust solutions on a national, full-service basis for more than five decades. Since 1889, First American (NYSE: FAF) has held an unwavering belief in its people. They are passionate about what they do, and we are equally passionate about fostering an environment where all feel welcome, supported, and empowered to be innovative and reach their full potential. Our inclusive, people-first culture has earned our company numerous accolades, including being named to the Fortune 100 Best Companies to Work For® list for ten consecutive years. We have also earned awards as a best place to work for women, diversity and LGBTQ+ employees, and have been included on more than 50 regional best places to work lists. First American will always strive to be a great place to work, for all. For more information, please visit www.careers.firstam.com. What We Do First American Trust is seeking a Senior Data Engineer to help develop and grow our data engineering capabilities to enhance the Strategic Product Development Organization. You will help FA Trust’s Data Engineering team make real-time decisions based on the datasets, facilitate analytics on existing products, and provide insights on potential new opportunities for growth. You will partner with subject matter experts to help us solve business problems with data. You will mentor data engineers and junior data engineers by providing guidance, advice, and support in various facets of data engineering. What You'll Do: - Build high-performing cloud data solutions to meet our analytical and BI reporting needs. - Design, build, and operate secure, scalable cloud data pipelines that support analytics and data products. - Integrate new data sources into the central data warehouse and deliver curated data to applications and other downstream destinations. - Identify, design, and implement internal process improvements, such as automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability, etc. - Build and enhance a shared data lake that powers decision-making and model building. - Implement data governance best practices across pipelines and datasets, including data quality checks, metadata management, lineage, and access controls. - Partner with teams across the business to understand their needs and develop end-to-end data solutions. - Collaborate with analysts to perform exploratory analysis and troubleshoot issues. - Coach and develop the team by providing hands-on mentorship to engineers, lead code and design reviews, create training/onboarding content, and run knowledge-sharing sessions. - Manage and model data using visualization tools to provide the company with a collaborative data analytics platform. - Build tools and processes to help make the correct data accessible to the right people. - Participate in active rotational support role for production during or after business hours supporting business continuity. - Engage in collaboration and decision making with other engineers. - Lead presentations regularly for stakeholders. What You'll Bring: - 5+ years of development experience with any of the following software languages: Python and SQL with cloud experience (Azure preferred or AWS). - Hands-on data security and cloud security methodologies. Experience in configuration and management of data security to meet compliance and CISO security requirements. - Experience creating and maintaining data intensive distributed solutions (especially involving data warehouse, data lake, data analytics) in a cloud environment. - Proven experience with relational and non-relational databases (e.g., PostgreSQL, Oracle, Azure Cosmos DB, MongoDB), including schema design, indexing, query optimization, and operational management in a cloud environment. - Hands-on experience in modern Data Analytics architectures encompassing data warehouse, data lake etc. designed and engineered in a cloud environment. - Demonstrated experience applying data governance best practices, including data quality validation, cataloging/metadata, lineage, and role-based access controls. - Hands-on expertise with LLMs and generative AI workflows. Familiarity with AI governance and privacy-preserving techniques. - Proven professional working experience in Event Streaming Platforms and data pipeline orchestration tools like Apache Kafka, Fivetran, Apache Airflow, or similar tools - Proven professional working experience in any of the following: Databricks, Snowflake, BigQuery, Spark in any flavor, HIVE, Hadoop, Cloudera or RedShift. - Experience developing in a containerized local environment like Docker, Rancher, or Kubernetes preferred - Experience with or knowledge of Agile Software Development methodologies. Data Modeling - Design schema and data pipelines to ingest, standardize, and deliver data from various sources into the data warehouse or data lake using ETL/ELT patterns. - Create, maintain, and optimize database structures to efficiently store and retrieve large volumes of data. - Evaluate data trends and model simple to complex data solutions that meet day-to-day business demand and plan for future business and technological growth. - Implement data cleansing processes and oversee data quality to maintain accuracy. - Function as a key member of the team to drive development, delivery, and continuous improvement of the cloud-based enterprise data warehouse architecture. Pay Range: $129,300.00 - $172,300.00 Annually This hiring range is a reasonable estimate of the base pay range for this position at the time of posting. Pay is based on a number of factors which may include job-related knowledge, skills, experience, business requirements and geographic location. ** Note that the following statements only apply to candidates who will be working from an unincorporated area within Los Angeles County. ** First American will consider for employment all qualified applicants, including those with arrest or conviction records, in a manner consistent with the requirements of applicable state and local laws (e.g., the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act). First American intends to conduct a review of an applicant’s criminal history in connection with a conditional offer. First American reasonably believes that a criminal history may have a direct, adverse and negative relationship with the following material job duties for this position potentially resulting in the withdrawal of the conditional offer of employment: handling of confidential, proprietary or trade secret information belonging to First American or its customers, administrating or facilitating financial transactions, and the ability to meet customer-imposed criminal history requirements. What We Offer By choice, we don’t simply accept individuality – we embrace it, we support it, and we thrive on it! Our People First Culture celebrates diversity, equity and inclusion not simply because it’s the right thing to do, but also because it’s the key to our success. We are proud to foster an authentic and inclusive workplace For All. You are free and encouraged to bring your entire, unique self to work. First American is an equal opportunity employer in every sense of the term. Based on eligibility, First American offers a comprehensive benefits package including medical, dental, vision, 401k, PTO/paid sick leave and other great benefits like an employee stock purchase plan.

United States
$129K - $172K / year
Job Closed
Peraton Corporation logo

Data Engineer (Azure Synapse ETL/SQL)

Peraton Corporation

Peraton Corporation, a national security company headquartered in Herndon, Virginia, supplies solutions for mission-critical programs and systems. Founded in 2017, Peraton's missio

Data Engineer96 days ago

Responsibilities Peraton is seeking a Data Engineer (Azure Synapse ETL/SQL) to join our team of qualified, diverse individuals. This position will be located in Herndon, VA with some remote work possible. The qualified applicant will become part of Peraton’s Department of State (DOS) Consular Systems Modernization (CSM) Program, for the Bureau of Consular Affairs (CA). This initiative will modernize and consolidate the operational environment under a common technology framework in order to better support the services provided to CA’s customers. The Data Engineer is responsible for working with large datasets, verifying data is collected, stored, and processed in a reliable and optimized manner. The Data Engineer will develop ETL processes, manage data warehouses, and confirm data integrity. This role collaborates closely with data scientists and analysts to provide clean, structured data that supports business insights and decision-making. Strong problem-solving skills and expertise in big data technologies are key to success in this role. Roles and Responsibilities: - Key Responsibility includes building and maintaining ETL pipelines using Azure Synapse. - Work with the system engineering team to understand customer business needs and priorities. - Implement Azure cloud analytics and databases, leveraging tools such as Azure Synapse, Azure SQL, Synapse pipelines, Databricks, and related frameworks - Utilize data science tools such as Anaconda, R, Weka, and Synapse Analytics for advanced data processing modeling. - Define system and technical requirements, including both functional and non-functional, to support database management system development within an Agile environment. - Develop conceptual, logical, and physical designs for database solutions. - Develop and maintain database solutions for managing structured and unstructured data. - Develop and maintain database solutions for data ingestion (batch and streaming). - Develop and maintain database scripts supporting ETL processes and functions. - Monitor and tune database solutions for optimal performance. - Develop and maintain secure database solutions that are compliant with DISA STIG and NIST specifications (security controls). - Develop data analytic models, incorporating machine learning (ML), deep neural networks (DNN), and large language models (LLM) to support business objectives. - Support CI/CD initiatives and utilize static code analyzers to ensure high quality source code. - Author clear documentation for database solutions, including data dictionaries, data catalogs, and design artifacts. - Integrate database solutions with the software applications and/or platforms. - Support software testing activities, including IV&V, and provide timely software defect resolutions. DOSCSM Qualifications Basic Qualifications: - Bachelor's degree and eight (8) years’ experience. - At least 5 years of experience with developing relational database management systems (RDBMS) using Oracle, PostgreSQL, MySQL, or Microsoft SQL Server in a production environment. - Hands-on experience developing, testing and optimizing complex SQL scripts - Hands-on experience with Azure Synapse Analytics for pipeline orchestration and/or SQL/Spark processing. - Experience supporting database engineering activities for software projects specializing in development of enterprise web applications within an Agile environment. - Experience designing database solutions using conceptual, logical, and physical diagrams. - Experience with developing database solutions involving storage of structured and unstructured data, data ingestion via batch and streaming, and ETL processes. - Excellent communication skills, both verbally and in writing. - Must be able to obtain or have an Active Secret (or higher) clearance. Interim clearances acceptable. - Must be a US Citizen. Preferred Qualifications: - Experience with enterprise software system modernization efforts. - Experience developing database solutions in an incremental manner using the Agile methodology. - Experience in cloud-based environments including the development of database solutions that are based on a hybrid-cloud architecture. - Experience developing database solutions for high volume, highly available, and secure software system. - Experience developing database solutions in a secure environment that are required to meet DISA STIG and NIST specifications (security controls). - Ability to work independently and in a team environment. Peraton Overview Peraton is a next-generation national security company that drives missions of consequence spanning the globe and extending to the farthest reaches of the galaxy. As the world’s leading mission capability integrator and transformative enterprise IT provider, we deliver trusted, highly differentiated solutions and technologies to protect our nation and allies. Peraton operates at the critical nexus between traditional and nontraditional threats across all domains: land, sea, space, air, and cyberspace. The company serves as a valued partner to essential government agencies and supports every branch of the U.S. armed forces. Each day, our employees do the can’t be done by solving the most daunting challenges facing our customers. Visit peraton.com to learn how we’re keeping people around the world safe and secure. Target Salary Range $86,000 - $138,000. This represents the typical salary range for this position. Salary is determined by various factors, including but not limited to, the scope and responsibilities of the position, the individual’s experience, education, knowledge, skills, and competencies, as well as geographic location and business and contract considerations. Depending on the position, employees may be eligible for overtime, shift differential, and a discretionary bonus in addition to base pay. EEO EEO: Equal opportunity employer, including disability and protected veterans, or other characteristics protected by law.

United States
$86K - $138K / year
Job Closed