Conseil en stratégie spécialisé dans l’impact investing - Accompagne les projets de développement inclusifs et durables
Data Engineer
Location
India
Posted
129 days ago
Salary
0
Seniority
Mid Level
Job Description
Data Engineer
AVAHI
• Design, build, and maintain scalable AWS data platforms supporting batch and streaming pipelines, analytics, and AI/ML workloads, aligned with AWS Well-Architected best practices. • Build and operate data ingestion, transformation, and enrichment pipelines from internal systems and external APIs, handling structured, semi-structured, unstructured, and graph data. • Implement data normalization workflows to ensure consistent schemas, high data quality, and reliable analytics, BI, and ML use cases. • Design and enforce data governance including cataloging, lineage, access control, and auditability. • Build and maintain knowledge graphs to model relationships across core business entities, enabling advanced analytics and inference. • Identify data gaps, inconsistencies, and missing relationships using strong analytical and inference skills. • Integrate data from enterprise platforms such as CRM and ERP systems (Salesforce, HubSpot, SAP, NetSuite, Dynamics 365, Workday). • Design secure data access layers for analytics, BI, ML, and downstream applications. • Implement monitoring, observability, and data quality checks for freshness, completeness, and pipeline health. • Optimize data architectures for performance and cost efficiency using partitioning, indexing, compression, and storage tiering. • Build internal tooling, dashboards, and standardized scaffolding to improve visibility, maintainability, and onboarding. • Collaborate with cross-functional teams to deliver high-impact data solutions and share best practices, documentation, and technical guidance.
Job Requirements
- Strong experience designing and operating AWS data platforms, including S3, Glue, Lake Formation, Athena, Redshift, EMR, Kinesis/MSK, DynamoDB, OpenSearch, and Neptune.
- Strong Python skills for data engineering, focused on modular, testable, and maintainable code.
- Solid understanding of distributed data systems, including batch and streaming pipelines, fault tolerance, idempotency, and event-driven architectures.
- Experience with data warehouse and lakehouse architectures, ETL/ELT pipelines, and analytical query engines.
- Hands-on experience with Spark, Hadoop, Hive, or Flink.
- Strong data modeling skills, including normalized, denormalized, and graph-based models, with safe schema evolution.
- Advanced SQL skills for analytics and data engineering, including window functions, CTEs, and query optimization.
- Experience integrating external APIs and enterprise systems, especially CRM and ERP platforms.
- Knowledge of data governance, security, and compliance, including encryption, access control, and audit logging.
- Experience implementing monitoring, observability, and data quality checks using CloudWatch and CloudTrail.
- Comfort with Infrastructure as Code using CloudFormation or Terraform.
- Strong end-to-end ownership mindset, with a focus on scalability, reliability, and long-term maintainability.
- Professional-level English communication skills, able to explain data architectures and trade-offs to technical and non-technical stakeholders.
Benefits
- Remote-First Flexibility: Enjoy work-life harmony in a remote-first environment that allows you to work from anywhere.
- Innovative Culture: We embrace a startup mindset, encouraging creativity, agility, and growth.
- Career Development: Avahi is committed to your growth, offering mentorship and opportunities to advance your career.
- Purpose-Driven Mission: Join us in making a difference. Avahi is dedicated to championing diversity, supporting women in tech, and fostering sustainable practices.
- Global Collaboration: Work alongside a diverse, talented team, sharing insights and collaborating to create innovative solutions that make a real impact.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Lead Manager of Data Engineering
JobgetherWe use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time. #LI-CL1 We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more. Role Description This role involves leading a dynamic data engineering team dedicated to developing innovative solutions grounded in AI technologies. - Lead the data engineering team and manage all associated activities. - Drive the development of data architecture leveraging AI innovations. - Implement data engineering templates and architectures for optimized performance. - Collaborate with product and architecture teams to enhance data strategies. - Stay updated on AI trends to enhance team performance and relevance. - Manage internal stakeholders and represent the organization to clients as needed. - Oversee team performance evaluations and professional development initiatives. - Define results and take action to meet organizational objectives. Qualifications - Bachelor’s degree or equivalent work experience. - 7+ years of experience in data engineering/IT/Project Management, including management experience. - 2+ years leading projects with proven project management methodologies. - Experience with data integration/ETL tools like databricks, Hadoop, and AWS tech stack. - Familiar with AI engineering concepts and tools. - Proficiency in software development and database management. - Excellent communication and interpersonal skills. Benefits - Competitive salary ranging from $130,000 to $165,000. - Discretionary bonus eligibility. - Comprehensive benefits package including medical, dental, and vision. - 401(k) savings plans. - Generous paid time off policies. - Flexible remote working conditions.
• Design, develop, and maintain robust, scalable data pipelines using Python and Airflow • Collaborate with Analytics Engineers and other team members to deliver clean, well-modeled data to downstream users • Implement monitoring, logging, and alerting to ensure data pipeline reliability and performance • Ensure data quality and integrity through validation frameworks and testing • Optimize performance and cost-efficiency of cloud-based data systems • Contribute to architectural decisions around data storage, workflow orchestration, and infrastructure as code • Document data pipelines and processes
• Build and maintain ETL pipelines that ingest daily Excel/CSV trade files, transform and validate the data, and persist it to PostgreSQL • Design and evolve database schemas using SQLAlchemy ORM and Alembic migrations • Implement business logic services: shareholder balance tracking, sharebook reconciliation, movement detection, classification rules • Maintain and extend Elasticsearch sync services (incremental and full resync workflows) • Support business stakeholders with data questions using the ELK stack — writing and running queries, interpreting results, and surfacing relevant data from Kibana • Parse and process structured financial data files (Excel, CSV) using Pandas, including data validation and deduplication • Write robust, type-safe Python following project standards (mypy strict mode, Black, isort) • Write and maintain automated tests with pytest — unit tests, repository mocks, snapshot tests • Collaborate with the CTO on data flow design and review migration strategies before deployment
Manager for Data Engineering
JobgetherWe use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time. #LI-CL1 We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more. Role Description This role involves leading a dynamic data engineering team dedicated to developing innovative solutions grounded in AI technologies. Your leadership will be instrumental in shaping IT architecture that aligns with business needs while enhancing analytics capabilities. - Lead the data engineering team and manage all associated activities. - Drive the development of data architecture leveraging AI innovations. - Implement data engineering templates and architectures for optimized performance. - Collaborate with product and architecture teams to enhance data strategies. - Stay updated on AI trends to enhance team performance and relevance. - Manage internal stakeholders and represent the organization to clients as needed. - Oversee team performance evaluations and professional development initiatives. - Define results and take action to meet organizational objectives. Qualifications - Bachelor’s degree or equivalent work experience. - 7+ years of experience in data engineering/IT/Project Management, including management experience. - 2+ years leading projects with proven project management methodologies. - Experience with data integration/ETL tools like databricks, Hadoop, and AWS tech stack. - Familiar with AI engineering concepts and tools. - Proficiency in software development and database management. - Excellent communication and interpersonal skills. Benefits - Competitive salary ranging from $130,000 to $165,000. - Discretionary bonus eligibility. - Comprehensive benefits package including medical, dental, and vision. - 401(k) savings plans. - Generous paid time off policies. - Flexible remote working conditions.
