Job Closed
This listing is no longer active.
MediaRadar, Inc. is a Software-as-a-Service company created to provide advertising insights in real time and help its clients gain the tools to outsmart their competition. The comp
Senior Data Engineer - Databricks
Location
India
Posted
70 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer - Databricks
MediaRadar
MediaRadar, An Industry Leader in Marketing Intelligence now including the data and capabilities of Vivvix, powers the mission-critical marketing and sales decisions that drive competitive advantage. Our next-generation marketing intelligence platform enables clients to achieve peak performance with always-on insights that span the media, creative, and business strategies of 5 million brands across 30+ media channels and 275 billion in media spend. By bringing the advertising past, present, and future into focus, our clients rapidly act on the competitive moves and emerging advertising trends impacting their business. About the Role We are looking for an experienced and strategic Senior Data Engineer to join our data team. In this role, you will be responsible for building and maintaining scalable, high-performance data solutions using Azure Databricks, Apache Spark, AKS, Airflow, Postgres, and modern data lakehouse architectures. You’ll play a key role in the full software development lifecycle—from design and implementation to deployment and documentation—while collaborating cross-functionally to support analytics, reporting, and operational data needs. This is an exciting opportunity to work along with a great team of data engineers, demanding technologies and an engaging work environment to help shape our data engineering best practices.
Job Requirements
- What You’ll Do:
- Involve in Design, development, and maintenance of scalable ETL/ELT pipelines on Azure Databricks using Apache Spark (PySpark/Spark SQL).
- Design and implement both batch and real-time data ingestion and transformation processes.
- Build and manage Delta Lake tables, schemas, and data models to support efficient querying and analytics.
- Consolidate and process large-scale datasets from various structured and semi-structured sources (e.g., JSON, Parquet, Avro).
- Write optimized SQL queries for large datasets using Spark SQL and PostgreSQL.
- Develop, schedule, and monitor workflows using Databricks Workflows, Airflow or similar orchestration tools.
- Design, build, and deploy cloud-native, containerized applications on Azure Kubernetes Service (AKS) and integrate with Azure services.
- Ensure data quality, governance, and compliance through validation, documentation, and secure practices.
- Collaborate with data analysts, data architects, and business stakeholders to translate requirements into technical solutions.
- Contribute to and enforce best practices in data engineering, including version control (Git), CI/CD pipelines, and coding standards.
- Continuously enhance data systems for improved performance, reliability, and scalability.
- Mentor junior engineers and help evolve team practices and documentation.
- Stay up to date on emerging trends, technologies, and best practices in the data engineering space.
- Work effectively within an agile, cross-functional project team.
- What You’ve Done:
- Proven experience as a Data Engineer, with a strong focus on Azure Databricks and Apache Spark.
- Proficiency in Python, PySpark, Spark SQL, and working with large-scale datasets in different data formats.
- Strong experience designing and building ETL/ELT workflows in both batch and streaming environments.
- Solid understanding of data lakehouse architectures and Delta Lake.
- Experience in Azure Kubernetes Service (AKS) is desired.
- Proficient in SQL and experience with PostgreSQL or similar relational databases.
- Experience with workflow orchestration tools (e.g., Databricks Workflows, Airflow, Azure Data Factory).
- Familiarity with data governance, quality control, and security best practices.
- Strong problem-solving skills and attention to detail.
- Excellent communication and collaboration skills, with a track record of working cross-functionally.
- Experience mentoring junior engineers and leading by example.
- Comfortable working in agile development environments and using tools like Git, CI/CD, and issue trackers (e.g., Jira).
Benefits
- At MediaRadar, we are committed to creating an inclusive and accessible workplace where everyone can thrive. We believe that diversity of backgrounds, perspectives, and experiences makes us stronger and more innovative. We are proud to be an Equal Opportunity Employer and make employment decisions without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, disability, genetic information, or any other legally protected status. This is a full-time exempt role with base salary plus benefits. Final compensation will depend on location, skill level, and experience.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Manager – Data Engineer, Digital Data Platforms
MashreqWe are Mashreq - Inspiring you to Rise Every Day
• design and maintain data pipelines of real-time and batch • create data strategy, data architecture, and best practices • lead the organization in strategic improvements in data engineering solutions • handle critical applications influencing management decision-making and regulatory reporting
At Mindera, we believe that software is built by people, for people with high performance systems that impact users worldwide. We are looking for a Senior Data Engineer to join an agile, collaborative team where your voice matters as much as your code. You won’t just be managing infrastructure, you’ll be a technical partner helping us build resilient systems that scale. We value empathy, self-organization, and the courage to take risks. If you love solving complex problems and believe that teaching others is the best way to learn, you’ll feel right at home here. We’re looking for someone who loves to build, learn, and guide others. You’ll be working closely with our leadership (CEO/COO) and our clients, so clear communication and a collaborative spirit are key. National and international expected traveling time varies according to project/client and organizational needs: 0%-15% estimated. 💪 How You’ll Contribute - Build & Maintain Pipelines: Design, build, and operate end-to-end data solutions , including both real-time and batch data pipelines. Ensure data flows correctly across architectures, from raw ingestion to modeling-ready layers. - Cross-Cloud Engineering: Leverage expertise across multiple cloud environments to build robust data ecosystems, whether utilizing AWS (S3 and core cloud services) , GCP (BigQuery and Google Cloud Storage) , or Azure (Databricks). - Automate & Orchestrate: Use deep knowledge of Apache Airflow for scheduling, orchestrating, and monitoring workflows. Build software services and automate processes using strong Python development skills. - Advanced Data Modeling: Apply the correct methodologies (Kimball, Inmon, Data Vault) to design Star Schemas, Semantic Models, and highly optimized platforms.
Big Data Engineer II
MetLifeMetLife is a leading insurance and financial services company based in New York, New York. The company and its affiliates specialize in employee benefits and life, accident, and he
Description and Requirements GG10.2 Data Engineer plays a critical role in data and analytics life cycle and significantly contributes to production grade data and analytics solutions. The role requires one to demonstrate Big Data, Engineering and Cloud expertise.It is an individual contributor role, expected to independently function 5-8+ years of relevant experience Bachelors degree in computer science, information technology or equivalent educational qualification • Design, build, and maintain robust ETL/ELT pipelines on cloud(Azure) or on-prem to collect, ingest and store large volumes of structured and unstructured data for batch/real time processing• Monitor, optimize, and troubleshoot data pipelines to ensure reliability, scalability, and performance• Ensure data processing, quality, security, and compliance guidelines, policies and standards are followed• Collaborate with multiple partners from Business, Technology, Operations and D&A capabilities (Data Governance, Data Quality, Data Modeling, Data Architecture, Data science, DevOps, BI & insights) • SQL, Python/Scala• NoSql and distributed databases (Hbase, Cosmos DB)• ETL pipleine development• Big Data Frameworks : Apache Spark, Hadoop, Hive• Cloud platforms: Azure data factory, Eventhub, Azure functions, Synapse, Databricks• Datawarehouses, data marts, data lakes• Medallion architecture• Performance tuning, optimization, and data quality validation• Real-time and batch data processing , streaming pieplines with Spark• Communication skills, analytical skills, structured problem-solving skills.,• Partner, Stakeholder engagement experience • DevOps practices: Git, AzureDevops, CI/CD pipelines• Unix shell scripting, MongoDB, Nifi• Exposure to Gen AI technology and tools About MetLife Recognized on Fortune magazine's list of the "World's Most Admired Companies" and Fortune World's 25 Best Workplaces™, MetLife, through its subsidiaries and affiliates, is one of the world's leading financial services companies; providing insurance, annuities, employee benefits and asset management to individual and institutional customers. With operations in more than 40 markets, we hold leading positions in the United States, Latin America, Asia, Europe, and the Middle East. Our purpose is simple - to help our colleagues, customers, communities, and the world at large create a more confident future. United by purpose and guided by our core values - Win Together, Do the Right Thing, Deliver Impact Over Activity, and Think Ahead - we're inspired to transform the next century in financial services. At MetLife, it's #AllTogetherPossible . Join us! #BI-Hybrid
Senior Data Engineer
ParexelParexel is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to legally protected status, which in the US includes race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or protected veteran status.
When our values align, there's no limit to what we can achieve. At Parexel, we all share the same goal - to improve the world's health. From clinical trials to regulatory, consulting, and market access, every clinical development solution we provide is underpinned by something special - a deep conviction in what we do. Each of us, no matter what we do at Parexel, contributes to the development of a therapy that ultimately will benefit a patient. We take our work personally, we do it with empathy and we're committed to making a difference. This role is responsible for collaborating with the R&D function within Corp IT Data and Analytics team, including software and infrastructure design, ensuring quality and performance standards are met. Key Accountabilities: Data analysis: - Applies data analysis, data modelling, and quality assurance techniques, based upon a detailed understanding of business processes, to establish, modify or maintain data structures and associated components (entity descriptions, relationship descriptions, attribute definitions). Advises database designers and other application development team members on the details of data structures and associated components. Database design: - Develops and maintains specialist knowledge of database concepts, object and data modelling techniques and design principles and a detailed knowledge of database architectures, software and facilities. Analyses data requirements to establish, modify or maintain object/data models. Evaluates potential solutions, demonstrating, installing and commissioning selected products. Programming/ software development: - Designs, codes, tests, corrects and documents complex programs and scripts from agreed specifications, and subsequent iterations, using agreed standards and tools, to achieve a well-engineered result. Takes part in reviews of own work and leads reviews of colleagues' work. Systems design: - Specifies user/system interfaces and translates logical designs into physical designs taking account of target environment, performance security requirements and existing systems. Produces detailed designs and documents all work using required standards, methods and tools, including prototyping tools where appropriate. Key Responsibilities: - Design and develop ETL mappings and workflows using Qlik or Informatica. - Develop technical specifications for data loading rules. - Mentor and provide technical guidance for Software Engineers. - Design, implement, and architect software products. - Design, implement, and architect product extensions for customer specific requirements. - Adhere to and produce artifacts (technical specifications, architectural specifications, data models, installation and qualification guides, and release notes) for the software development process. - Support technical consulting activities to clients in the pharmaceutical industry. - Design and support clinical trials information sharing solutions. - Provide application support for existing products. - Help with release management, using versioning tools such as SVN, and executing live deployments. Skills: - Willingness to work in a matrix environment and to value the importance of teamwork. - Understanding of common system architecture frameworks and concepts. - Technical awareness of Security and Application Software Development. - Excellent interpersonal, verbal and written communication skills. - Client focused approach to work. Knowledge and Experience: - Proven relevant experience in systems integration and implementation, or software development, or a related discipline. - Demonstrated ability to liaise with multiple stakeholders. - Experience with Microsoft Azure components in particular, Azure SQL, Azure Analysis Services, Synapse. - Experience implementing ETL solutions, particularly utilizing Qlik Replicate and Compose. - Strong Database Development Skills, particularly implementing SQL solutions in Azure SQL, Synapse or Databricks. environments. Good experience with large datasets. - Experience improving Data Quality in Data Warehouses, particularly utilizing Informatica Data Quality. - Experience testing ETL solutions, particularly utilizing Informatica DVO product. - Experience using Enterprise Architecture modeling tools, frameworks and methodologies such as TOGAF, SPARX EA, UML, BPMN, ArchiMate etc. - Experience Designing Data Warehouses, applying Kimball’s Dimensional Modeling Techniques, Architecting ETL systems. - Experience working in Agile development software methodology, authoring technical documents / specifications. - Experience debugging complex software systems. - Significant experience implementing data-driven systems with large-scale relational or object-oriented databases. - Ability to work in a high pressure, result/deadline-oriented environment. Education: - Bachelor's Degree in technical discipline (Math’s, Science, Engineering, Computing, etc.) or a related study, or equivalent project-related experience.




