Job Closed

This listing is no longer active.

Weekday (YC W21) logo
Weekday (YC W21)

We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent

Data Engineer

Data EngineerData EngineerFull TimeRemoteLeadTeam 11-50Since 2021H1B No SponsorCompany SiteLinkedIn

Location

India

Posted

66 days ago

Salary

₹1,000K - ₹2,000K / year

Seniority

Lead

Job Description

Data Engineer

Weekday (YC W21)

• Lead and manage a team of data engineers, fostering a culture of innovation, ownership, and continuous improvement. • Define and execute the data engineering roadmap aligned with organizational goals and product strategy. • Architect, design, and oversee the implementation of scalable, reliable, and high-performance data pipelines and data platforms. • Collaborate with cross-functional teams including Product, Analytics, Data Science, and DevOps to deliver end-to-end data solutions. • Establish best practices for data governance, data quality, security, and compliance. • Drive the adoption of modern data technologies and frameworks, ensuring systems are future-ready and cost-efficient. • Review system architecture and code to ensure scalability, maintainability, and performance optimization. • Manage stakeholder expectations, prioritize initiatives, and ensure timely delivery of projects. • Mentor team members, support career growth, and build leadership within the team. • Monitor and improve system reliability, uptime, and performance through proactive measures and observability tools.

Job Requirements

  • Min Experience: 12 years
  • Strong experience in data engineering, including building large-scale data pipelines and distributed systems.
  • Hands-on expertise with modern data stack technologies such as SQL/NoSQL databases, ETL/ELT tools, and data warehousing solutions (e.g., Snowflake, BigQuery, Redshift).
  • Proficiency in programming languages such as Python, Java, or Scala.
  • Experience with big data technologies like Apache Spark, Hadoop, or Kafka.
  • Deep understanding of cloud platforms such as AWS, Azure, or Google Cloud.
  • Strong knowledge of data modeling, data architecture, and data lifecycle management.
  • Experience implementing data governance, security, and compliance frameworks.
  • Familiarity with CI/CD pipelines, DevOps practices, and infrastructure-as-code tools.
  • Proven track record of leading and scaling high-performing engineering teams.
  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

Related Categories

Related Job Pages

More Data Engineer Jobs

Vertical Relevance is looking for a Data Architect, to join our team as a full-time employee and work remotely. This person is responsible for the end-to-end planning, building, and deploying of software systems. Candidate will be able to drive the programming of well-constructed, testable code. As a Data Architect, you will implement technical solutions as part of a team for customer engagements. This role requires strong teamwork, communication, patience and organizational skills needed to drive customer success. At Vertical Relevance we deliver with excellence through teamwork, automating everything, constantly learning and taking ownership for the outcomes our customers experience. Are you ready to join the team? Responsibilities - Help customers shape their journey to adopting the cloud and provide our customers with technical and strategic guidance on their “cloud journey”. - Consult, plan, design, and implement Data solutions on the cloud customers - Design AWS Data Lake implementations - Database Migration to AWS Managed Services - Design and Develop AWS Data Quality and Governance solutions - Become a deep technical resource that earns our customer's trust - Develop high-quality technical content such as automation tools, reference architecture, and white papers to help our customers build on the cloud - Innovate on behalf of customers and translate your thoughts into action yielding measurable results. - Support solution development by conveying customer needs and feedback as input to technology roadmaps. Share real world implementation challenges and recommend expansion of capabilities through enhanced and new offerings. - Assist with technical briefs that document solutions - Assist with reference architecture implementations - Support internal and external brand development through thought leadership: - Work with Marketing/Alliances to write blog posts - Work with Marketing/Alliances to develop internal case studies Qualifications - Professional experience architecting/operating Data / DevOps solutions built on AWS and Azure. - Experience in software/technology customer facing experience. Sample Activities You’ll Do AWS Data Architecture and Data Lake Implementation Assist Customer in building out a framework for data collection, data ingestion and catalog, data storage and data serving designed to optimize the efficiency in which information is accessed, curated, served and analyzed from up to two on-premises data sources. - Assist Customer with implementing Amazon Virtual Private Cloud (Amazon VPC), account, Amazon Simple Storage Service (Amazon S3) bucket strategy and capture impact for introducing critical data for a data lake. - Assist Customer with achieving modern big data lambda architecture, providing an environment for consumption and utilization of data coming from either batch or streaming sources. - Assist Customer with defining detailed data architecture including raw data, conformed, structured, enriched, and aggregated. - Provide Customer general best practices in building production quality data pipeline for ingestion, transformation and consumption to downstream applications through a serving layer. - Assist Customer with defining details to data catalog and data serving layer through application program interfaces (APIs) if required. - Provide Customer general best practices for Amazon S3 data storage approach. Build Ingestion and Orchestration Ingestion Based on Customer’s ingestion requirements provided. Assist Customer with building a reusable ingestion framework to onboard new data quickly based on one data source example. - Assist Customer with designing and building robust data ingestion solutions using monitoring, logging and alerting service with retry capabilities. - Assist Customer with server-less approaches where if applicable for the ingestion. - Assist Customer with building the framework for managing data updates to data stored in objects on Amazon S3. Orchestration Based on Customer’s ingestion, orchestration and operational requirements. Provide general best practices to Customer for implementing an extensible and supportable orchestration framework for onboarding data to lambda architecture for a representative data source. - Advise Customer on leveraging AWS Services tools for data ingestion including Amazon Kinesis, Amazon EMR, AWS Lambda or other tool defined as part of the assessment. - Advise Customer on orchestration options on AWS Services including but not limited to AWS Glue, AWS Step Functions, AWS Batch, Amazon EMR, AWS Data Pipeline, AWS Lambda or other AWS native orchestration tooling. - Experience with Apache Iceberg Sagemaker, SageMaker Lakehouse, SageMaker Catalog, Terraform. - Database migration factory - Aurora PostgreSQL - Experience with Athena, Redshift, EMR, Glue, DataBricks and/or Snowflake. - AWS Glue Data Catalog configuration and management. - Python/PySpark for metadata automation. - Data governance framework implementation. - Metadata ingestion pipeline development. - DCAM/DAMA framework experience. - Lake Formation security models. - Enterprise taxonomy design. - Assist Customer with building out the orchestration and state management processes to support data processing without data loss and exception and identify error handling. - Advise Customer on how the orchestration tooling will provide operational feedback to the support organizations in the event of a failure. Storage and Catalog Amazon S3 Data Storage Build Based on Customer’s usage requirements. Assist Customer with building the data lake storage approach for a representative data source. - Assist Customer with building data storage including buckets, prefixes, encryption, file types, partitioning. - Assist Customer with building automated data partition strategy. - Assist Customer with building Amazon S3 Bucket policies, logging and monitoring. - Assist Customer with building out data lifecycle management. - Assist Customer with building cross-region replication if required. Relevant Technical Tools - Primary Languages - Python - Tooling, Services & Libraries – Airflow, Kafka, Parquet, Spark, Metaflow, Git, Hadoop Relevant AWS Services - AWS Infrastructure Scripting – CloudFormation, AWS CLI, AWS CDK - AWS Storage Services – S3 - AWS Compute Services – Lambda, Batch - AWS Database Services – RDS, DynamoDB, Redshift, Aurora, Neptune - AWS Networking Services – VPC - AWS Management and Governance Services – CloudTrail, Service Catalog - AWS Analytics Services – Athena, EMR, Kinesis, Glue, Lake Formation, Data Pipeline - AWS Security, Identity, Compliance Services – IAM - AWS Application Integration Services – Step Functions The Company Vertical Relevance was founded to help business leaders drive value through the design and delivery of effective transformation programs across people, processes, and systems. Our mission is to help firms at any stage of their journey develop custom solutions for success and growth. We provide a full range of services from strategy and design through to implementation and training. Our collective industry expertise is our greatest asset - our professionals have an average of 20+ years’ experience within Financial Services, across Wealth Management, Asset Management, Insurance, and Banking. Within our Customer Experience practice, we add complementary industry expertise (technology and media) synergizing the most relevant and successful customer trends. We focus wholly on your success by first rigorously assessing your business and technology challenges, and then right-sizing solutions that provide a meaningful ROI. With our industry experts hitting the ground running and focusing on nimble, quality delivery, we can see rapid, tangible improvements with our clients in productivity and effectiveness. When it makes sense for your company, we leverage our product partnerships in the areas of CRM, Sales Acceleration, Predictive Analytics, Digital Knowledge Management, and Cloud Transformation. Vertical Relevance is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected Veteran status, age, or any other characteristic protected by law.

United States
Full TimeRemoteTeam 10,001

IQVIA is growing! Hiring multiple Clinical Data Management (CDM) Leads across our FSP (Functional Service Provider) and Traditional (Sponsor-Agnostic) delivery models. Apply now and have your resume considered for Lead Data Manager opportunities across multiple teams - allowing us to match you with the best-fit project and client environment for your experience and interests! IQVIA is seeking experienced Clinical Data Management Leads to oversee end-to-end data management delivery for global clinical trials. You’ll ensure quality, accuracy, and compliance while leading cross-functional teams that help bring new therapies to life. This is a remote/WFH position with all necessary equipment provided. What You’ll Do - Lead data management for complex, global clinical trials from study setup to database lock. - Serve as the main client contact for all data management deliverables. - Oversee SAE, PK, and external data reconciliation and database deployment. - Manage timelines, budgets, and resource allocation to meet customer expectations. - Mentor junior data management staff and promote best practices. - Drive process improvements and ensure regulatory and SOP compliance. Who We’re Looking For Ideal candidates will have.... - Bachelor’s degree in life sciences, health, clinical, biological, or mathematical field. - No less than 10 years of direct clinical data management experience, with 5+ years in a Lead Clinical Data Management role. - Your previous roles/job titles may include Clinical Data Manager, Senior Clinical Data Manager, Lead Data Manager, Principal Data Manager, and/or Data Management Project Manager. - Proven success managing large global studies (1000+ patients). - Expertise with Medidata Rave, Oracle InForm, or similar EDC systems. - Strong understanding of GCP, ICH, and clinical research processes. - Excellent communication and leadership skills. What IQVIA Offers - Fully remote role – company equipment provided - Competitive salary and performance-based incentives - Career growth and learning opportunities through mentoring and internal mobility - Inclusive, global culture focused on innovation and collaboration What to Expect After Applying - Recruiter video chat (20 min) - Hiring manager interview (1 hour, MS Teams - Offer decisions are typically made within one or two weeks of the final interview.) Apply now to join IQVIA, where your expertise shapes the future of data-driven healthcare! IQVIA is a leading global provider of clinical research services, commercial insights and healthcare intelligence to the life sciences and healthcare industries. We create intelligent connections to accelerate the development and commercialization of innovative medical treatments to help improve patient outcomes and population health worldwide. Learn more at https://jobs.iqvia.com IQVIA is committed to integrity in our hiring process and maintains a zero tolerance policy for candidate fraud. All information and credentials submitted in your application must be truthful and complete. Any false statements, misrepresentations, or material omissions during the recruitment process will result in immediate disqualification of your application, or termination of employment if discovered later, in accordance with applicable law. We appreciate your honesty and professionalism.

South Africa
Sungrow Europe - Welcome traveler! logo

Privacy & Data Protection Coordinator - Europe

Sungrow Europe - Welcome traveler!

We are a green tech company with young creative minds. We are fighting every day to make our mission happen – Clean Power for All! With 870+GW of clean power presently installed worldwide, we are on the road to success! Founded in 1997 by University Professor Cao Renxian, Sungrow is a leader in the research and development of solar inverters with the largest dedicated R&D team in the industry and a broad product portfolio offering PV inverter solutions and energy storage systems for utility-scale, commercial & industrial, and residential applications, as well as internationally recognized floating PV plant solutions, NEV driving solutions, EV charging solutions and renewable hydrogen production systems.

Data Engineer66 days ago
Full TimeRemoteTeam 501-1,000

Your mission Servus und hallo bei uns hier in Sungrow. We're thrilled to introduce a pivotal role that safeguards our most valuable asset: data, ensuring our operations align with stringent privacy and data protection standards across Europe. Curious what this is all about? Let’s dive right in: - Assist in the development, implementation, and maintenance of Sungrow’s data protection and privacy compliance program. - Monitor relevant data protection laws, regulations, and industry best practices, particularly GDPR, advising on their impact. - Support data protection impact assessments (DPIAs) and provide guidance on privacy-by-design principles for new projects. - Coordinate with internal departments to ensure data processing activities comply with established policies and legal requirements. - Assist in managing data subject access requests (DSARs) and other privacy-related inquiries. - Contribute to data protection training and awareness initiatives for employees across the organization. - On an ad hoc basis, assist with other branches of the compliance team, especially in the area of ESG. Your profile Alright, alright, that was intriguing, wasn't it? Now, in order to fit in this role and be happy for a long time, your ideal background should look a lot like this: - A Bachelor’s degree in Law, or a related field; a Master's degree or CIPP/E certification is a plus. - 1-5 years of experience in data protection, privacy, or legal compliance roles, preferably within a multinational environment. - Solid understanding of European data protection regulations, especially the GDPR. Knowledge in cybersecurity related laws such as CSA and NIS-2 is a strong plus. - Strong analytical skills with the ability to interpret legal texts and translate them into practical business solutions. - Excellent communication and interpersonal skills to collaborate effectively with diverse teams. - Business fluent English is required; Mandarin is a strong plus; German, Polish or other European languages are a plus. Why us? Our Legal & Compliance team is at the forefront of protecting Sungrow's integrity and ensuring our ethical operation in the digital age. As part of our team, you will play a crucial role in building trust with our customers and partners by upholding the highest standards of data privacy. You will gain exposure to complex international data protection challenges and contribute directly to our sustainable growth in the renewable energy sector. We offer a collaborative and supportive environment where your expertise will be valued, and you will have ample opportunities for professional development as you help us navigate the evolving landscape of data governance.

Germany
Unholster logo

Data Engineer

Unholster

El software adecuado puede transformarlo todo.

Data Engineer66 days ago
Full TimeRemoteTeam 11-50Since 2011H1B No Sponsor

• Construir plataformas de datos • Abordar la ingesta batch y streaming de data • Implementar entregables orientados a usuarios y aplicaciones • Coordinar multidisciplinariamente entre equipos

Chile