Data Engineer II

Location

Worldwide

Posted

1 day ago

Salary

0

Seniority

Mid Level

Job Description

Data Engineer II

Mrsool

Role Description We're looking for a passionate Data Engineer II to help build and scale the data platform that powers analytics, product insights, and data-driven decision making across the organization. You'll work on modern data technologies including Kafka, Maxwell, Spark, S3, Trino, BigQuery, dbt, and Metabase to build reliable, scalable, and high-performance data systems. From real-time data ingestion and distributed data processing to dimensional modeling and data platform development, you'll play a key role in shaping the foundation of our data ecosystem. As part of the Data Engineering team, you'll: - Design and build robust data pipelines. - Implement modern data lake architectures. - Develop reusable data models. - Create trusted datasets that empower Product, Analytics, Data Science, and Business teams. - Contribute to improving our engineering standards by focusing on performance, reliability, observability, automation, and developer experience. This role is ideal for someone who enjoys solving complex data engineering challenges, building scalable platforms, and continuously improving how data is collected, transformed, and consumed across the organization. If you're excited about building modern data platforms, working with large-scale distributed systems, and having a meaningful impact on the company's data strategy, we'd love to hear from you. Qualifications - 4+ years of hands-on experience designing and building scalable data platforms, data lakes, and data warehouses. - Strong proficiency in Spark (Scala, python) and SQL, with experience building production-grade data pipelines and distributed data processing applications. - Hands-on experience with Apache Spark and a solid understanding of distributed data processing, performance tuning, and optimization. - Experience building batch and streaming data pipelines using technologies such as Kafka, CDC/Maxwell, or similar event-driven architectures. - Strong understanding of modern data lake architectures, including Medallion Architecture, data modeling, partitioning, and storage optimization. - Experience working with cloud-native data platforms and technologies such as Amazon S3, BigQuery, Trino, or similar analytics engines. - Solid experience designing dimensional models, star schemas, and building reliable data marts that support analytics and business intelligence. - Hands-on experience with dbt, including developing reusable models, implementing automated testing, and maintaining documentation. - Strong knowledge of data quality, observability, lineage, and engineering best practices to build reliable and maintainable data products. - Experience optimizing large-scale data pipelines, SQL queries, and distributed processing jobs for performance, scalability, and cost efficiency. - Familiarity with CI/CD, Git-based development workflows, infrastructure automation, and modern software engineering best practices. - Excellent problem-solving skills with the ability to independently own projects from design through production. - Strong communication and stakeholder management skills, with experience collaborating across Product, Engineering, Analytics, and Business teams. - A passion for building scalable data platforms and continuously improving developer experience, platform reliability, and operational excellence. Requirements - Design, build, and maintain scalable batch and real-time data pipelines using Maxwell, Kafka, Spark, and dbt to power analytics and business-critical applications. - Develop and optimize data models following Medallion Architecture (Bronze, Silver, Gold) to create reliable, reusable, and high-quality datasets. - Build and maintain cloud-native data platforms using S3, Spark, Trino, and BigQuery, ensuring scalability, reliability, and cost efficiency. - Design robust data ingestion frameworks leveraging CDC (Maxwell), Kafka, and event-driven architectures to support near real-time data processing. - Create, optimize, and maintain data warehouses and data marts that enable fast, reliable reporting and self-service analytics. - Partner closely with Product Managers, Data Analysts, Backend Engineers, and Business stakeholders to translate business requirements into scalable data solutions. - Develop reusable dbt models, testing frameworks, and documentation to improve data quality, governance, and developer productivity. - Optimize Spark jobs, Trino queries, and storage layouts for performance, reliability, and cost efficiency. - Own the end-to-end lifecycle of critical data pipelines, ensuring high availability, monitoring, SLA adherence, and proactive incident resolution. - Build and enhance the core data platform by developing reusable frameworks, automation, CI/CD pipelines, and engineering best practices. - Ensure data quality through validation, monitoring, lineage, and observability while implementing best practices for security and governance. - Enable analytics teams by delivering trusted datasets, semantic models, and dashboards that power decision-making through Metabase. Benefits - Inclusive and Diverse Environment: We foster an inclusive and diverse workplace that values innovation and offers remote environments. - Competitive Compensation: Our compensation packages are highly competitive and include potential share options for certain roles. - Personal Growth and Development: We are committed to your personal and professional growth, providing regular training and an annual learning stipend to help you advance your career in a dynamic environment. - Autonomy and Mentorship: You'll enjoy a high degree of autonomy in your role, supported by mentorship and ambitious goals that pave the way for both your success and the company's growth.

Related Categories

Related Job Pages

More Data Engineer Jobs

Zensar logo

Cloud Data Engineering - Azure Data Engineering

Zensar

At Zensar, we’re “experience-led everything”. We are committed to conceptualizing, designing, engineering, marketing, and managing digital solutions and experiences for over 130 leading enterprises. We are a company driven by a bold purpose: Together, we shape experiences for better futures. Whether for our clients, our people, or the world around us, this belief powers everything we do. At the heart of our culture is ONE with Client - a set of four core values that reflect who we are and how we work: One Zensar, Nurturing, Empowering, and Client Focus. Part of the $4.8 billion RPG Group, we’re a community of 10,000+ innovators across 30+ global locations, including Milpitas, Seattle, Princeton, Cape Town, London, Zurich, Singapore, and Mexico City. We believe the best work happens when individuality is celebrated, growth is encouraged, and well-being is prioritized. We are an equal employment opportunity (EEO) and affirmative action employer, committed to creating an inclusive workplace. All qualified applicants will be considered without regard to race, creed, color, ancestry, religion, sex, national origin, citizenship, age, sexual orientation, gender identity, disability, marital status, family medical leave status, or protected veteran status.

Data Engineer1 day ago
Full TimeRemoteTeam 10,001

Role Description The DE&A - Core - Cloud Data Engineering role is a critical position within Zensar Technologies, responsible for driving cloud-based data engineering initiatives. The successful candidate will lead a team of data engineers, ensuring the efficient and effective delivery of projects. With a focus on Azure, they will also have experience with other cloud platforms, enabling them to provide diverse solutions to clients. Qualifications - Experience in cloud-based data engineering. - Proficiency in Azure and other cloud platforms. - Leadership experience in managing data engineering teams. Requirements - Strong understanding of data engineering principles. - Ability to deliver projects efficiently and effectively. - Experience with diverse cloud solutions. Benefits - Inclusive workplace culture. - Opportunities for personal and professional growth. - Commitment to employee well-being.

India
Full TimeRemoteTeam 10,001+H1B Sponsor

• Design, build, and maintain enterprise data architectures supporting Supply Chain execution and performance measurement • Develop data models, pipelines, and integration frameworks • Lead data migrations, conversions, and performance optimization efforts • Oversea testing, deployment, maintenance, and continuous improvement of data systems • Ensure adherence to data quality, security, and compliance standards

United States
$119K - $203K / year

Chapter Lead, Data Science

Commonwealth Bank of Australia

The Commonwealth Bank of Australia is an Australian multinational bank serving residents across New Zealand, Asia, the United States, and the United Kingdom thr

Data Engineer1 day ago

Title: Chapter Lead, Data Science (Operations) Location: Australia Full time Job requisition id REQ248859 Job Description: Permanent Full Time Opportunity Drive the future of AI innovation at scale You are a visionary leader with a growth mindset, combining deep technical fluency with the ability to inspire and scale impact across the enterprise. Were one of the largest and most advanced Data Science practices in the country Together we can build state-of-the-art Data Science models that power seamless experiences for millions of customers See yourself in our team At the heart of the Chief Operations Office (COO), we're reimagining how we serve customers in operations through the power of transformative AI solutions. As a Chapter Lead reporting to the Chapter Area Lead of Data Science in COO, you'll balance hands-on technical contribution with people leadership-guiding and growing a team of data scientists while working in cross-functional squads with engineers and other experts to design and deliver innovative, responsible AI solutions. You'll stay at the forefront of emerging technologies, fostering a culture of curiosity, inclusion, and continuous learning, and translating complex ideas into high-impact outcomes for customers and colleagues. If you're passionate about shaping the future of AI while helping others thrive, we'd love you to lead with us. Key Responsibilities Lead AI Innovation Drive the design, construction and deployment of a range of AI projects Accelerate adoption of advanced AI capabilities Expand Data & AI Impact Across the Bank Identify high‑value AI use cases and champion the integration of Data Science methods across processes Ensure models follow best-practice engineering, governance, and deployment standards Drive Continuous Improvement & Business Value through AI Partner closely with operations teams to understand challenges and apply transformative AI capabilities Drive redevelopment, optimisation and monitoring of AI models to support strategic business outcomes We're interested in hearing from people with the following skills: Technical & People Leadership Demonstrated capability in driving outcomes that are reusable and can be leveraged across the group Demonstrated intellectual curiosity with a passion for experimenting and exploring emerging technologies Demonstrated ability to lead and mentor teams while remaining hands-on, fostering a supportive and inclusive team culture that values learning, thoughtful decision-making, and sustainable delivery of real-world impact. Technical Expertise Data: Experience designing and delivering AI solutions using a range of structured and unstructured data, with particular strength in natural language data. Machine Learning: Strong grounding in classical machine learning and deep learning, with the ability to select appropriate techniques-from simpler models to more complex approaches-based on problem context, constraints, and impact. Generative AI: Hands-on experience designing, building, and deploying Generative AI solutions, including RAG and agentic systems, with a pragmatic understanding of their limitations, risks, and costs. Statistical Expertise: Experience defining fit-for-purpose evaluation metrics, guiding experimentation, and applying sound statistical judgement to technical decision-making across the AI lifecycle. Production AI & MLOps: Proven ability to take models from prototype to production, including version control, CI/CD practices, containerisation, deployment, monitoring, and ongoing iteration. Data Engineering & Architecture: Working knowledge of data pipelines and integration patterns, and experience collaborating on scalable, reliable data and AI architectures. Cloud Platforms: Experience designing or deploying AI solutions on cloud platforms such as AWS or Azure, with awareness of scalability, cost, and operational considerations. Technical Leadership & Delivery: Ability to lead and mentor teams while remaining hands-on, balancing curiosity and continuous learning with a strong focus on delivery, value creation, and real-world impact. Working with us At CommBank, we advocate and facilitate a culture of inclusion and respect, celebrating all cultures, abilities, genders, expressions of gender and sexual orientation Read more about our commitment to inclusion and diversity. Our people bring their diverse backgrounds and unique perspectives to build a respectful, inclusive, and flexible workplace. One where we're driven by our values, and supported to share ideas, initiatives, and energy. One where making a positive impact for customers, communities and each other is part of our every day. We're committed to being an accessible employer and want to make finding your dream job as easy as possible. If you experience any accessibility issues or if you require additional support, please contact HR Direct. We support our people with the flexibility to balance where work is done with a hybrid approach of at least half your time each month connecting in office. We also have many other flexible working options available including changing start and finish times, part-time arrangements and job share to name a few. Talk to us about how these arrangements might work for you. Here, you'll thrive. You'll be supported when faced with challenges and empowered to tackle new opportunities. We're hiring talent from across all of Australia and have opened technology hubs in Melbourne and Perth. We really love working here, and we think you will too. If you're already part of the Commonwealth Bank Group (including Bankwest, x15ventures), you'll need to apply through Sidekick to submit a valid application. We're keen to support you with the next step in your career. We're aware of some accessibility issues on this site, particularly for screen reader users.

Australia

Title: Lead, Data Governance (Master Data Management) Location: Hoboken United States Job Description: Workplace Type: Remote Job: Data Engineering Schedule: FULL_TIME Req ID: 24896 OCTO Global Data | Pearson Role Overview Pearson's data organization is in the middle of a genuine transformation and master data is the foundation it's being built on. We're not here to maintain the status quo. We're rethinking how a global education company defines, trusts, and activates its most critical data assets. We're looking for a Lead, Data Governance (Master Data Management) to lead this charge: setting the strategy, defining the architecture, and driving the execution of Pearson's enterprise master data capabilities. You'll start with the Customer domain, working through the legacy complexity while simultaneously building the blueprints for how we govern Product, Supplier, and other critical domains in a fundamentally different way. This is a senior, player-coach role for someone who can hold two things at once: the pragmatism to tackle what's broken today, and the ambition to reinvent how an enterprise thinks about data. You will be shaping Pearson's philosophy around trusted, reusable, governed data and building the systems, products, processes, and culture to make that real. You'll operate as a thought leader to the business, not just a technical function. That means influencing how Pearson's most senior stakeholders understand and value their data, advocating for data as a strategic product, and helping the organization move from treating data as a byproduct of operations to treating it as a first-class asset. This role sits within Global Data, a team of builders in the middle of a meaningful modernization. You'll work at the intersection of data engineering, data products, analytics, and business units across a global organization, with the ability to drive real change. Key Responsibilities MDM Strategy & Architecture - Drive MDM vision and roadmap: Define and execute the enterprise master data strategy across the Customer domain and beyond (product, supplier, employee, finance) with a phased roadmap for domain expansion. Challenge the status quo and question inherited ways of working, bring fresh thinking to how best practices are defined, and push the organization toward modern, scalable approaches that reflect where data management should be going. - Establish best practices: Design and implement MDM architecture standards that serve Pearson's global needs while enabling business units to achieve their goals. - Data governance: Build and operate a data governance framework including stewardship models, data ownership policies, and dispute resolution processes across BUs. - Architecture and Design: Define system-of-record vs. system-of-use vs. system-of-truth across Salesforce, Oracle, E-commerce, and other integrated platforms. Drive data model re-design reducing MDM data bloat and ensuring streamlined field use across all BUs and functions. - Metadata & lineage: Oversee metadata management and data lineage practices to ensure traceability and auditability across the data estate. - Automation: Identify automation opportunities as it relates to master data CRUD or Data quality. Customer360 & Self-Service Analytics - Lead Customer360 data product: Lead the design and build of holistic view of the Pearson customer, a foundational data product that powers self-service analytics across the business. - Innovative data access: Create and champion new approaches to accessing and consuming master data, enabling business users to answer their own questions without reliance on data teams. - Single source of truth: Drive the design and adoption of golden record and deduplication processes, match and merge logic that produces a trusted, unified view of each customer across all Pearson systems. 1. Data Quality & Trusted Data - Data quality ownership: Define and enforce data quality standards across master data domains, profiling, monitoring, cleansing, and continuous improvement. - Golden record & deduplication: Lead the design of match and merge processes to identify, resolve, and prevent duplicate records across source systems. - Data stewardship program: Stand up a data stewardship program that embeds accountability for data quality within business units, not just central IT. Production Support & Legacy Systems - Legacy system stabilization: Partner with engineering and operations teams to drive production support for legacy MDM systems while the organization transitions to modern platforms. - System integration: Ensure master data flows correctly across Oracle, Salesforce, and other enterprise systems, managing integration patterns, data synchronization, and API-based consumption. - ETL/ELT pipelines: Oversee the design and maintenance of data ingestion pipelines that feed master data from source systems into the MDM platform. - Master Data Migrations: Support master data migration programs ensuring data quality, standard processes, and approvals are in place before creating or updating data. 1. - Cross-Functional Partnership & Change Management - Stakeholder influence: Act as the primary MDM partner to business units, translating complex data capabilities into business value and influencing across a matrixed organization. - Change management: Drive the cultural and operational change required for MDM transformation, MDM programs succeed or fail based on adoption, not just technology. - Business & technical fluency: Serve as the bridge between technical data teams and business stakeholders, with a deep understanding of both Pearson's data landscape and the commercial needs of its business units. - Program Leadership: Serve as the central MDM lead across all major enterprise programs and ensure alignment of requirements and dependencies across the programs. - Performance Management and Continuous Improvement: Define and track KPIs tied to business outcomes and MDM value realization Qualifications Required - 10–15 years of experience in data management, with significant depth in Master Data Management - Hands-on experience with MDM platforms such as Informatica - Deep knowledge of Oracle and Salesforce as source systems and MDM consumers (SFDC – Data360 and Agentforce) - Proven experience designing match and merge logic, deduplication, and golden record processes - Large Scale Customer Master / Customer 360 transformations - Deep understanding of data governance frameworks, data stewardship, and data ownership models - Experience with ETL/ELT pipelines, API integration patterns, and cloud data platforms - Track record of driving data quality programs at scale in complex enterprise environments - Ability to lead cross-functional initiatives and influence senior stakeholders without direct authority - Experience managing legacy system environments while driving modernization - Strong communication skills - able to translate technical concepts for business audiences and vice versa Nice to Have - Experience with data product thinking and self-service analytics enablement - Familiarity with Customer360 or similar enterprise customer data initiatives - Experience with GCP (Looker) - Experience with metadata management tools and data lineage platforms - Knowledge of real-time and streaming data architectures Benefits at a Glance We know you’ll do great work, so we give a lot back with some of the best benefits in the business. We know one size doesn’t fit all, so our workplace programs meet the different needs of our diverse teams, and their families, too. See our Benefits. At Pearson, we’re transforming learning. If you’re a Technology leader who thrives in complexity, leads with purpose, and aspires to shape the future—we want to hear from you. Apply today and imagine the impact you can make. Compensation at Pearson is influenced by a wide array of factors including but not limited to skill set, level of experience, and specific location. The full-time salary range is between $200,000 - $220,000. This position is eligible to participate in an annual incentive program, and information on benefits offered is here. #LI-EB1 Pearson is an Equal Opportunity Employer and a member of E-Verify. Employment decisions are based on qualifications, merit and business need. Qualified applicants will receive consideration for employment without regard to race, ethnicity, color, religion, sex, sexual orientation, gender identity, gender expression, age, national origin, protected veteran status, disability status or any other group protected by law. We actively seek qualified candidates who are protected veterans and individuals with disabilities as defined under VEVRAA and Section 503 of the Rehabilitation Act. If you are an individual with a disability and are unable or limited in your ability to use or access our career site as a result of your disability, you may request reasonable accommodations by emailing Job: Data Engineering Job Family: TECHNOLOGY

New Jersey
$200K - $220K / year