Job Closed
This listing is no longer active.
Senior Data Engineer
Location
United States
Posted
120 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
Indium
• This is a remote position • Ability to work effectively in ambiguous environments and handle unstructured problems while anticipating risks and issues • Experience designing and building end-to-end data processing pipelines for Data Lake and Data Warehouse implementations • Hands-on expertise in configuring and developing custom components for data ingestion, processing, and provisioning • Strong experience working on Azure Cloud, including Azure Synapse and SQL • Proven experience delivering at least one complete end-to-end data pipeline implementation • Expertise in architecting enterprise solutions using frameworks, reusable components, and enterprise design patterns • Strong knowledge of database design, development, performance tuning, memory optimization, and DB partitioning • Experience across DW-BI technologies such as MSBI, Oracle, and Teradata • Proficient in CI/CD automation and deployment practices • Programming experience in Python, PySpark, and Scala • Skilled in building dashboards and visualizations using Power BI
Job Requirements
- Ability to work effectively in ambiguous environments and handle unstructured problems while anticipating risks and issues
- Experience designing and building end-to-end data processing pipelines for Data Lake and Data Warehouse implementations
- Hands-on expertise in configuring and developing custom components for data ingestion, processing, and provisioning
- Strong experience working on Azure Cloud, including Azure Synapse and SQL
- Proven experience delivering at least one complete end-to-end data pipeline implementation
- Expertise in architecting enterprise solutions using frameworks, reusable components, and enterprise design patterns
- Strong knowledge of database design, development, performance tuning, memory optimization, and DB partitioning
- Experience across DW-BI technologies such as MSBI, Oracle, and Teradata
- Proficient in CI/CD automation and deployment practices
- Programming experience in Python, PySpark, and Scala
- Skilled in building dashboards and visualizations using Power BI
Benefits
- Diversity, equity, and inclusion (DEI) initiatives
- WE@IN women empowerment program
- Expert sessions and tailored training
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Optimize and modernize Amazon Redshift environments, including consolidation into Redshift Managed Storage (RMS). • Lead or support migration from Provisioned Redshift to Redshift Serverless with zero or minimal downtime. • Analyze and tune Redshift SQL for performance across batch and ad-hoc analytical workloads. • Review and enhance data ingestion pipelines from AWS sources such as RDS, DynamoDB, and Kinesis. • Design data delivery into Apache Iceberg tables and Redshift with minimal duplication and high efficiency. • Collaborate with stakeholders on architecture recommendations and best practices.
• Engineer Complex Data Transformations: You will lead the development of pipelines that translate unstructured XBRL and securities data into optimized SQL Server environments, making complex financial filings accessible and queryable. • Power High-Visibility Applications: You will architect the backend data layer that serves a suite of web applications, ensuring fast load times and data integrity for a diverse user base at a major federal regulator. • Optimize Cloud-Scale Processing: You will leverage AWS and big data tools to handle increasing data velocity and variety, ensuring our infrastructure scales alongside the evolving needs of the financial markets. • Ensure Data Precision: You will implement rigorous validation and cleansing logic within the pipeline to ensure that the data driving federal oversight is accurate, consistent, and audit-ready.
• Build and Maintain Bronze/Silver Layer Pipelines: You will ensure core data sources lands accurately, on time, and with full lineage. • Lead Data Ingestion, Transformation, and Enrichment: You will own the end-to-end pipeline from raw file landing through cleansed, conformed staging tables, including deduplication, standardization, code mapping, and entity resolution. • Develop Automated Ingestion Pipelines: You will use Snowpipe, Matillion, or custom solutions with reliability, observability, and minimal manual intervention in mind. • Implement dbt Models: You will write clean, tested, documented SQL that follows medallion architecture patterns and supports incremental processing at scale. • Establish Scalable Engineering Patterns: You will create reusable templates, macros, and testing frameworks that accelerate onboarding of new data sources without sacrificing quality. • Ensure Data Quality and Observability: You will implement dbt tests, freshness checks, and anomaly detection. You will own the pipeline reliability and respond to data quality incidents. • Collaborate with the Architect and Stakeholders: You will translate business requirements into technical specifications, participate in design reviews, and contribute to ADRs, and you will ensure pipelines align with enterprise standards.
• Build and Maintain Bronze/Silver Layer Pipelines: You will ensure core data sources lands accurately, on time, and with full lineage. • Lead Data Ingestion, Transformation, and Enrichment: You will own the end-to-end pipeline from raw file landing through cleansed, conformed staging tables, including deduplication, standardization, code mapping, and entity resolution. • Develop Automated Ingestion Pipelines: You will use Snowpipe, Matillion, or custom solutions with reliability, observability, and minimal manual intervention in mind. • Implement dbt Models: You will write clean, tested, documented SQL that follows medallion architecture patterns and supports incremental processing at scale. • Establish Scalable Engineering Patterns: You will create reusable templates, macros, and testing frameworks that accelerate onboarding of new data sources without sacrificing quality. • Ensure Data Quality and Observability: You will implement dbt tests, freshness checks, and anomaly detection. You will own the pipeline reliability and respond to data quality incidents. • Collaborate with the Architect and Stakeholders: You will translate business requirements into technical specifications, participate in design reviews, and contribute to ADRs, and you will ensure pipelines align with enterprise standards.




