The most dynamic global recruiting agency focused on the creative, product, marketing and technology industries.
Data Engineer
Location
India
Posted
80 days ago
Salary
0
Seniority
Lead
Job Description
Data Engineer
CLIQHR Recruitment Services
• Design and implement scalable data ingestion and processing pipelines using AWS services and modern data tools • Perform data ingestion, validation, and transformation using existing data frameworks • Develop and enhance data ingestion frameworks to support fast and reliable data onboarding • Process structured and semi-structured data using batch processing and store data in S3 • Transform and load data into Redshift and manage data operations using DBT • Build and manage workflow orchestration using Airflow or Step Functions • Export and integrate processed data with external systems via S3 • Implement data validation, auditing, and monitoring to ensure data quality and reliability • Create configurable data pipeline steps using JSON, tables, or similar methods for non-technical users • Support containerized deployments using Docker and contribute to deployment processes
Job Requirements
- 8–11 years of experience in Data Engineering or related roles
- Strong hands-on experience with AWS services such as Glue, Lambda, S3, and Step Functions
- Proficiency in Python, PySpark, and advanced SQL
- Experience with DBT for data transformation and modeling
- Expertise in workflow orchestration tools like Airflow or Step Functions
- Hands-on experience with cloud data warehouses such as Redshift, Snowflake, or BigQuery
- Experience handling structured and semi-structured data pipelines
- Knowledge of Shell scripting and containerization using Docker
- Basic understanding of data governance and DevOps deployment practices
- Familiarity with Terraform for infrastructure management (nice to have)
- Strong problem-solving skills and ability to work in fast-paced environments.
Benefits
- Remote work arrangement
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Staff Data Platform Engineer (ClickHouse | OLAP)
Growth Acceleration PartnersConsult • Design • Build • Modernize
WHAT WE DO Founded in 2007, Growth Acceleration Partners (GAP) is a consulting and technology services company. We consult, design, build and modernize revenue-generating software and data engineering solutions for clients. With modernization services and AI tools, we help businesses achieve a competitive advantage through technology. GAP’s remote, integrated engineering teams use end-to-end solutions to innovate and align with your business goals. We have 600+ English-speaking engineers based in Latin America and approximately 20 U.S.-based engineers. With some of the highest customer satisfaction scores in the industry, GAP’s focus is customer and employee success. GAP is a woman-owned company headquartered in Austin Texas. We are a values-based company focused on growing our people by investing in education, onsite English classes and training in the latest technologies, including AI, data analytics and machine learning. Our goal is to provide solutions for our customers that help them achieve critical business outcomes, while enabling our GAPSters and our communities to attain long-term success. Summary We are looking for a Staff Data Platform Engineer with deep expertise in large-scale analytical databases and real-time data platforms. In this role, you will lead the design, optimization, and scalability of high-performance OLAP systems that power analytics, AI pipelines, and intelligent connectivity platforms. You will architect and optimize distributed analytical databases such as ClickHouse while ensuring the platform can process multi-terabyte datasets efficiently. This role combines data architecture, performance engineering, and platform design, with a strong focus on real-time analytics and large-scale data processing. As a Staff Engineer, you will also guide engineering teams on best practices for data modeling, distributed query optimization, and system scalability, ensuring that data systems deliver reliable insights and support advanced AI-driven workloads. Education - Bachelor’s or Master’s Degree in Computer Science, Data Science, Engineering, or related field, or equivalent practical experience. Professional Experience - +10 years of experience building and operating data engineering or analytics platforms - Strong experience working with OLAP databases or analytical data systems - Experience designing distributed data systems handling multi-terabyte datasets - Proven experience optimizing high-performance analytical queries and data pipelines - Experience mentoring engineers and guiding data architecture decisions Key Responsibilities Data Platform Architecture - Architect and optimize large-scale ClickHouse or OLAP databases - Design distributed data systems capable of handling multi-terabyte datasets - Define best practices for analytical data models and query performance - Ensure high availability, scalability, and reliability of data systems Performance Optimization - Tune analytical queries for maximum performance and efficiency - Optimize storage structures, indexing strategies, and partitioning - Improve query latency for real-time analytics workloads - Monitor and address performance bottlenecks in distributed data systems Data Pipeline & Platform Integration - Integrate ClickHouse with analytics platforms, dashboards, and AI pipelines - Design efficient data ingestion and transformation pipelines - Support real-time analytics use cases across distributed systems - Enable downstream consumption by analytics and product teams Engineering Leadership - Mentor engineers on distributed data system design and performance optimization - Guide teams in building scalable analytics infrastructure - Contribute to architectural decisions across the data platform - Promote best practices for data architecture and data reliability Cross-Team Collaboration - Work closely with data scientists, analytics teams, and backend engineers - Ensure the data platform supports real-time analytics and AI-driven features - Communicate complex data architecture decisions clearly to technical teams Required Technical Skills Data Platforms & Databases - Deep experience with ClickHouse or similar OLAP databases - Experience with analytical databases such as Druid, DuckDB, or similar systems - Strong experience managing multi-terabyte data environments - Deep understanding of distributed data system architecture Data Engineering & Analytics Systems - Experience building and operating analytics platforms and data pipelines - Knowledge of high-performance query optimization - Experience designing analytical schemas for fast querying - Experience integrating data platforms with analytics or AI systems Nice to Have - Experience with real-time data streaming architectures - Experience working with vector databases or AI data pipelines - Experience with cloud-based data infrastructure - Experience supporting large-scale analytics platforms Soft Skills - Advanced English proficiency - Strong communication and collaboration skills - Builder mindset with strong curiosity and experimentation attitude - Ability to explain complex technical concepts clearly At Growth Acceleration Partners, we're an equal opportunity employer committed to building a diverse and inclusive team. We value everyone's unique background, and we provide equal opportunities regardless of race, color, creed, religion, sexual orientation, gender identity, age, national origin, disability, marital status, veteran status or any other personal right protected by law. We foster a culture of belonging and strive to provide a welcoming environment where everyone feels safe to contribute and grow.
Senior Data Engineer
TBC CorporationTBC has been a leader of innovative and value-creating business solutions in the mobility industry for over 65 years.
Lyric is an AI-first, platform-based healthcare technology company, committed to simplifying the business of care by preventing inaccurate payments and reducing overall waste in the healthcare ecosystem, enabling more efficient use of resources to reduce the cost of care for payers, providers, and patients. Lyric, formerly ClaimsXten, is a market leader with 35 years of pre-pay editing expertise, dedicated teams, and top technology. Lyric is proud to be recognized as 2025 Best in KLAS for Pre-Payment Accuracy and Integrity and is HI-TRUST and SOC2 certified, and a recipient of the 2025 CandE Award for Candidate Experience. Interested in shaping the future of healthcare with AI? Explore opportunities at lyric.ai/careers and drive innovation with #YouToThePowerOfAI. Applicants must already be legally authorized to work in the U.S. Visa sponsorship/sponsorship assumption and other immigration support are not available for this position. This role plays a pivotal part in ensuring the security, efficiency, reliability, and scalability of our cloud infrastructure and related operations. ESSENTIAL JOB RESPONSIBILITIES & KEY PERFORMANCE OUTCOMES - Primary responsibility will be to architect, implement, and optimize cloud-based solutions while leveraging industry best practices. - Provide mentorship and guidance to junior engineers and drive continuous improvement and innovation across our CloudOps and DevOps practices. - Collaborate closely with cross-functional teams to design and deploy robust, automated systems that support our organization's mission-critical applications and services. - Monitor and maintain the health of our cloud environment, identifying and resolving issues proactively to minimize downtime and incidents. REQUIRED SKILLS & EXPERIENCE Required: - Masterful understanding of advanced Kubernetes technologies and ability to design and diagnose production grade deployments at scale. - Proven AWS cloud experience with Azure and GCP being a bonus. - Infrastructure as Code (IaC) experience with Terraform and/or AWS CloudFormation. - Containerization experience with the docker ecosystem and related tooling. - Production grade coding experience with languages such as Bash, Python and Golang for scripting and automation of services. - Understanding of advanced AWS networking technologies such as VPC, TGW related technologies. - A solid grasp of cloud security and best practices and an understanding of compliance standards such as HIPAA and HITRUST. GENERAL COMPETENCIES Required: - Strong collaboration and communication skills. - Ability to design and present complex topics to the team. - Adept at critical thinking and problem solving. ***The US base salary range for this full-time position is: $125,241.00 - $187,862.00 The specific salary offered to a candidate may be influenced by a variety of factors including but not limited to the candidate’s relevant experience, education, and work location. Please note that the compensation details listed in US role postings reflect the base salary only, and does not reflect the value of the total rewards compensation. *** Lyric is an Equal Opportunity Employer that strives to create an inclusive environment, empower employees and embrace collaborative success.
• Serve as an expert for Data Analytics and Reporting: BI systems and reporting: Enablement, compliance, development, and maintenance (PowerBI). • ETL tools: Enablement, compliance, development, and maintenance (Alteryx, Python,, Boomi/Rivery, Airflow, Openflow, AWS Step Functions, AWS Lambda). • Data warehousing: Design, development, and maintenance of data models and structures (Snowflake, MySQL and Postgres). • Data pipelines: Build, monitor, and optimize data pipelines for efficient data flow • Enable cross-functional collaboration: Promote a data-driven culture by educating and supporting teams in using analytics tools and data effectively. • Improve operational efficiencies: Democratize data access to enable self-service analytics and data-driven insights. • Identify and implement cost-saving opportunities related to data infrastructure and processes. • Maintain and enhance a centralized repository of data sources, processes, and data flows. • Optimize data processes to reduce complexity and improve performance. • Document data workflows and processes for business continuity and transparency. • Adhere to data governance best practices. • Identify and address technical debt within data pipelines and systems. • Share knowledge and best practices for efficient code and architectural principles. • Design end-to-end solutions that deliver timely and accurate data for business consumption. • Manage and democratize data: Develop and maintain workflows to prepare and transform data for analysis. • Transform data into actionable insights for business stakeholders. • Understand data relationships, flows, and their impact on key performance indicators (KPIs). • Develop and maintain operational dashboards to monitor business performance. • Enhance and expand the data warehouse and pipelines to support the organization's analytical maturity. • Foster a data-driven community within the organization by sharing knowledge and promoting data reuse. • Strategic planning and support: Contribute to the analytical framework for strategic decision-making across business functions • Ensure successful data operations Deliver projects based on priorities set for the Data Analytics team • ITSM exposure - Incident/Change/Problem Management • SOX ITGC focus • Strong problem-solving skills for complex data challenges • Strong documentation skills • Excellent communication skills to convey technical information to both technical and non-technical audiences • High-energy, team-first, positive attitude
Data Architect - T7
NielsenIQNielsenIQ is an industry leader in data analytics and global measurement. The company delivers information to partners, retailers, and manufacturers through pow
Company Description NIQ is the world’s leading consumer intelligence company, helping brands and advertisers understand what people buy, watch, and engage with. MRI‑Simmons, a division within NIQ, runs one of the most comprehensive and detailed consumer insights studies in the U.S., collecting thousands of data attributes from 50,000+ respondents annually. Our Consumer Canvas product takes this data even further — mapping it to the broader U.S. population (250M+ adults) and enabling advanced audience targeting through partners like The Trade Desk, LiveRamp, TransUnion, and more. We are expanding our architecture team to drive the next evolution of Canvas — identity resolution, data enrichment, privacy‑first design, and scalable activation pipelines. Job Description We are seeking a Data Architect to own and evolve the end‑to‑end architecture behind the Consumer Canvas platform. You will work across Snowflake, Databricks, identity systems, ingestion pipelines, and activation layers — ensuring our data is scalable, secure, privacy‑compliant, and highly performant. You will partner closely with Data Engineering, Data Science, Machine Learning, and Product teams to build next‑gen solutions that power the future of consumer insights and advertising activation. This is not a migration or maintenance role — it is a strategic influence position with the opportunity to architect new systems, components, and methodologies from the ground up - Own the end-to-end architecture of the Consumer Canvas product. - Design systems across ingestion, identity spine creation, enrichment, governance, and activation workflows. - Collaborate with cross-functional partners (Data Scientists, ML Engineers, Data Engineers, Product Owners). - Define and enforce PII governance, privacy controls, and security best practices. - Translate complex business and technical requirements into scalable architecture. - Support technical teams with architectural guidance and model/structure design. - Participate directly in design, prototyping, and testing cycles within an Agile environment. - Act as a senior technical thought partner, providing mentorship (non‑manager role). Qualifications Must‑Have Skills - 5+ years of experience in Data Architecture or related technical leadership roles. - Hands-on experience with Snowflake (including Data Clean Rooms) and/or Databricks. - Strong SQL and experience with Python, PySpark, and Spark. - Proven experience working with PII data, privacy constraints, and governance frameworks (critical & non‑negotiable). - Data modeling expertise (conceptual, logical, physical). - Experience designing scalable data pipelines and cloud-native architectures. - Strong communication skills and ability to influence cross-functional stakeholders. Nice-to-Have Skills - Experience in Ad-Tech, Martech, Media, Consumer Data, or Identity Ecosystems - Familiarity with Medallion Architecture (Bronze/Silver/Gold). - Azure services (ADF, Azure Data Lake, Azure DevOps). - Containers & orchestration (Docker, Kubernetes). - Experience with audience activation platforms (e.g., Trade Desk, LiveRamp). Education - Bachelor’s degree in Computer Science, Engineering, or a related field required. - Master's degree or certifications (SnowPro, Azure Data Engineer) are a plus. - Equivalent professional experience is fully acceptable in place of formal credentials. You will thrive if you are: - A strategic thinker who enjoys architecting new systems, not just migrating old ones. - Passionate about privacy-first data design and operational excellence. - Skilled at turning ambiguous requirements into elegant architectures. - Excited by large-scale audience data (hundreds of millions of profiles). - Comfortable navigating complex stakeholder environments. Additional Information Why Join NIQ? - Build one of the most influential consumer identity data architectures in the U.S. - Work with cutting-edge cloud technologies (Snowflake, Databricks, identity graph frameworks). - Be part of an innovative team pushing the boundaries of consumer analytics and ad-tech. - Flexible, fully remote work environment. - Global collaboration & career growth opportunities. This role has a market-competitive salary with an anticipated base compensation of the following range: $130,000 - $138,000 USD Yearly. Actual salaries will vary depending on a candidate’s experience, qualifications, skills, and location. This role might also be eligible for a Sales-based incentive or performance-based bonus. Other benefits include a flexible working environment, comprehensive health insurance, industry-leading parental leave, life insurance, education support, and more. Our Benefits - Flexible working environment - Volunteer time off - LinkedIn Learning - Employee-Assistance-Program (EAP) NIQ may utilize artificial intelligence (AI) tools at various stages of the recruitment process, including résumé screening, candidate assessments, interview scheduling, job matching, communication support, and certain administrative tasks that help streamline workflows. These tools are intended to improve efficiency and support fair and consistent evaluation based on job-related criteria. All use of AI is governed by NIQ’s principles of fairness, transparency, human oversight, and inclusion. Final hiring decisions are made exclusively by humans. NIQ regularly reviews its AI tools to help mitigate bias and ensure compliance with applicable laws and regulations. If you have questions, require accommodations, or wish to request human review were permitted by law, please contact your local HR representative. For more information, please visit NIQ’s AI Safety Policies and Guiding Principles: https://www.nielseniq.com/global/en/ai-safety-policies. About NIQ NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK, bringing together the two industry leaders with unparalleled global reach. With a holistic retail read and the most comprehensive consumer insights—delivered with advanced analytics through state-of-the-art platforms—NIQ delivers the Full View™. NIQ is an Advent International portfolio company with operations in 100+ markets, covering more than 90% of the world’s population. For more information, visit NIQ.com Want to keep up with our latest updates? Follow us on: LinkedIn | Instagram | Twitter | Facebook Our commitment to Diversity, Equity, and Inclusion At NIQ, we are steadfast in our commitment to fostering an inclusive workplace that mirrors the rich diversity of the communities and markets we serve. We believe that embracing a wide range of perspectives drives innovation and excellence. All employment decisions at NIQ are made without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, disability, genetic information, marital status, veteran status, or any other characteristic protected by applicable laws. We invite individuals who share our dedication to inclusivity and equity to join us in making a meaningful impact. To learn more about our ongoing efforts in diversity and inclusion, please visit the https://nielseniq.com/global/en/news-center/diversity-inclusion - Career Site Team: Technology - Compensation: USD 130000 - USD 138000 - yearly




