We make sense of data to drive your business forward. #MakeSenseofData #DriveYourBusinessForward #PartnerYourWay
Senior AWS Data Engineer
Location
India
Posted
6 days ago
Salary
0
Seniority
Senior
Job Description
Senior AWS Data Engineer
EXL
• Manage data engineering projects, ensuring alignment with business objectives. • Provide strategic guidance on data engineering best practices. • Oversee a team of data engineers. • Ensure continuous improvement of data processes. • Design, build, and maintain efficient, reusable, and reliable architecture and code for data pipelines and data applications on AWS. • Build robust data ingestion pipelines (from on-prem to AWS and within AWS) using AWS services such as Glue, Redshift, S3, Lambda, EMR/Spark, Kinesis, and SQS. • Develop and manage ETL/ELT processes to collect, process, and store data from multiple sources, ensuring data quality, integrity, and security. • Architect and implement end-to-end data solutions (ingestion, storage, integration, processing, access) on AWS, with a focus on data lakes and data warehouses. • Participate in the architecture and system design discussions for high-scale data engineering projects. • Independently perform hands-on development, unit testing, and participate in code reviews to ensure adherence to best practices. • Implement serverless applications using AWS Lambda, API Gateway, Step Functions, and other AWS technologies. • Migrate data from traditional relational databases, file systems, and APIs to AWS-based data lakes (S3), RDS, Aurora, and Redshift. • Implement high-velocity streaming solutions using Amazon Kinesis, SQS, and Kafka (preferred). • Architect and implement CI/CD strategies for enterprise data platforms. • Collaborate with product, operations, QA, and cross-functional teams throughout the software development cycle. • Stay abreast of new technology developments, implement POCs for new tools/technologies, and onboard them for real-world use cases. • Identify and resolve performance issues and continuously optimize for cost, reliability, and scalability.
Job Requirements
- Bachelor’s degree in Computer Science, Software Engineering, MIS, or equivalent combination of education and experience.
- 5+ years of experience implementing and supporting data lakes, data warehouses, and data applications on AWS for large enterprises.
- Strong programming experience with Python, Shell scripting, and SQL.
- Solid experience with AWS services: CloudFormation, S3, Athena, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, Secrets Manager.
- Experience in serverless application development and data pipeline orchestration.
- Experience in system analysis, design, development, and implementation of data ingestion pipelines in AWS.
- Knowledge of ETL/ELT, data modeling, and big data technologies.
- Familiarity with data warehousing concepts and cloud-based architecture.
- Strong problem-solving skills and attention to detail.
- Excellent communication and teamwork abilities.
Benefits
- Work From Home
Related Guides
Related Categories
Related Job Pages
More Cloud Engineer Jobs
• Migración y CMZ • Recrear y validar estructuras de tablas en CMZ • Migrar datos históricos por tabla con validación de integridad (volúmenes, frecuencias, reglas de negocio) • Reconfigurar scheduled queries que generan tablas derivadas desde EDW PROD / Data Lake • Identificar y documentar dependencias upstream/downstream por dominio • Pipelines e Ingesta • Desarrollar pipelines de ingesta en Airflow hacia capas Raw - Catalog - Cmp • Configurar y validar procesos de carga en el nuevo entorno gobernado • Reconectar integraciones de sistemas y reportes (Looker, Power BI, Tableau) a nuevas tablas • Calidad y Gobernanza • Completar metadata de tablas migradas (owner, descripción, linaje) • Ejecutar pruebas de validación y reconciliación de datos contra fuente original • Levantar tickets JIRA para solicitudes de nuevos datasets con justificación de negocio
• Build and ship internal applications, services, and shared libraries end to end from API to the data layer. • Design, query, and tune our managed data platforms while keeping schemas, migrations, and query performance healthy. • Maintain the internal SDK that routes apps through the model gateway, injects keys, and adds tracing and cost headers, so teams write less code and we keep control of prompts and guardrails. • Run and harden the model gateway used by every team, including high availability across zones and clean dev, staging, and production environments. • Build and operate the cloud services behind the platform, including container compute, managed databases, caching, storage, and secrets. • Build and run the CI/CD pipelines, container registry, and monorepo tooling that every app depends on. • Own platform security maintenance by patching CVEs and security findings quickly during business hours. • Work with the Security team to implement core safety controls, including image and code scanning, secret scanning, access control, identity and sign-on, and audit logging. • Keep the platform reliable and observable through logging, tracing, issue resolution, and steady improvement. • Support cost visibility by tagging usage, building spend and chargeback views, and helping keep spend sensible. • Write clear documentation and provide practical support so teams can adopt the platform without friction.
Senior Data Engineer, AWS, Python, SQL
DevsuDevsu is a technology agency that provides software development services, IT augmentation and staffing.
• Design, build, and maintain scalable data pipelines for analytics, ecommerce, logistics, and marketing. • Improve and maintain the company's data infrastructure and tooling, including Redshift/Snowflake, Airflow, and Fivetran. • Architect data processing systems capable of handling complex data flows and large-scale datasets. • Develop reliable, efficient, and cost-effective data solutions in an AWS environment. • Ensure high standards of data quality through automation and best engineering practices. • Optimize platform performance, resiliency, scalability, and operational costs. • Collaborate closely with software engineers, analysts, and business stakeholders. • Mentor engineers and analysts on data engineering best practices. • Participate in architecture discussions and technical decision-making.
• Architect and develop .NET systems with a focus on performance and scalability; • Collaborate with multidisciplinary teams on the design and implementation of solutions; • Maintain and evolve existing systems following engineering best practices; • Integrate systems with AWS services; • Contribute to the continuous improvement of team processes and practices.




