Job Closed
This listing is no longer active.
👋 Welcome to InnovativeDev! We help startups and industry leaders to design and develop amazing software solutions.
Senior Data Engineer
Location
Pennsylvania
Posted
84 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
InnovativeDev
• Responsible for integrating, analyzing raw data, developing, and maintaining datasets and improving data quality and efficiency. • Develops and maintains scalable data pipelines, integration tools and builds out new API integrations to support continuing increases in data usage, volume, and complexity. • Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition. • Designs data integrations, data quality framework and integrate it with monitoring services. • Strong hands on capabilities across integration toolsets including ADF, Replication, CDC etc. • Integrate data from different SOR’s, combine raw information and create resulting models ready for consumption. • Interpret trends and patterns. • Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues. • Identifying, designing, and implementing internal process improvements including re-designing infrastructure for greater scalability, DB, SQL, and data pipelines performance tuning, optimizing data delivery, and automating manual processes. • Build solution prototypes as needed and write algorithms against data. • Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it. • Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility, and fostering data-driven decision making across the organization. • Works closely with a team of frontend and backend engineers, product managers, and analysts.
Job Requirements
- Bachelor's degree in data analytics or similar field.
- Hands-on 7+ years' experience with SQL, Data integration, Data analysis, Data Modelling, and design
- 4+ years of Python, NodeJs or Java development experience
- 4+ years of experience with schema design and dimensional data modeling
- Experience in technology platforms across Microsoft Azure, Cloud Computing, Software as a Services (SaaS), Integration Platform as a Service (IPaaS), Infrastructure as a Service (IaaS)
- Previous experience as a data engineer or in a similar role
- Technical expertise with data models, data mining, and segmentation techniques
- Experience working and implementing solutions with Microservices Architecture
- Experience working with Data warehousing and Data Lake solutions.
- Demonstrated ability to instrument Data Quality & Management standards and processes.
- 4+ years working with data integration and ETL tools, such as Azure Data Factory, CDC, Replication, Talend, Informatica etc.
- Strong problem-solving and analytical skills
Benefits
- Employee-friendly perks and benefits
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Engineer
LyftLyft, established in 2012 by Logan Green and John Zimmer, is a transportation network company offering a mobile application that promotes ride-sharing by connec
• Owner of the core data pipeline, responsible for scaling up data processing flow to meet the rapid data growth at Lyft • Evolve data model and data schema based on business and engineering needs • Implement systems tracking data quality and consistency • Develop tools supporting self-service data pipeline management (ETL) • SQL and MapReduce job tuning to improve data processing performance • Write well-crafted, well-tested, readable, maintainable code • Participate in code reviews to ensure code quality and distribute knowledge • Collaborate cross-functionally with product, engineering, data science, and marketing teams to understand business problems and align on prioritization and solutions
• Lead and manage data engineering projects from conception to deployment, ensuring alignment with stakeholder needs. • Design, build, and maintain scalable data pipelines to support data ingestion, transformation, and storage. • Collaborate with data scientists and analysts to ensure that data needs are met and provide technical guidance on best practices. • Implement and maintain data quality and integrity measures. • Utilize cloud-based data platforms and technologies, particularly Azure, to enhance data accessibility and usability. • Mentor and guide junior team members in data engineering concepts and practices. • Identify opportunities for process improvements and implement solutions that enhance efficiency, scalability, and performance.
• Apply an in-depth understanding of data structures and information content. • Select, deploy, and manage the systems and infrastructure required for a data processing pipeline in support of the project requirements. • Investigate, create, and maintain data flows, data content, data element definitions with a goal of enterprise Master data integration. • Determine technical breath in data profiling from different sources and determine whether and how data can support business and data requirements of its intended use. • Develop and maintain common business definitions and metadata criteria for consistent metrics reporting across the enterprise. • Design the architecture for new data and analytics platform to support analytics and data science and machine learning. • Design the data models and data movement processes that support analytics and data science. • Recommend and implement patterns and best practices for data engineering. • Ensure quality processes are built into the design of the platform. • Understand the architectural difference between solution approaches and communicate the advantages/disadvantages of your recommendation to both technical and non-technical audiences. • Design and develop analytics and interactive visualizations that create business insights and clearly communicate data and trends. • Develop complex SQL queries to obtain data from our source systems. • Perform data validation and quality assurance to ensure data integrity and accuracy. • Collaborate with IT and business partners to identify data sources and align data domains to authoritative sources of data. • Enforce tactical enforcement of Data Governance policies and rules. • Research new technologies while keeping up-to-date with technological developments in relevant areas of Data Governance, Master Data, and Data Quality.
Role Description Turn 10 Studios is seeking a Data Engineer to join the Data Pipeline team and help design, build, and support modern data platforms that power studio‑wide decision making. This role focuses on developing scalable, high‑quality data pipelines and analytics systems using Azure‑based technologies. The Data Engineer will partner closely with business, design, test, and development teams to shape how data is captured, modeled, and consumed, enabling test‑driven methodologies and a culture of data‑driven development. - Design, build, and maintain scalable ETL and ELT pipelines supporting real‑time and batch analytics workloads. - Develop and optimize data models within lakehouse and warehouse architectures to support analytics and reporting needs. - Implement data processing solutions using Databricks, Azure Data Factory, Azure Synapse Analytics, and related services. - Ensure data quality, reliability, and performance across ingestion, transformation, and consumption layers. - Partner with cross‑functional stakeholders to define data requirements and improve how data is captured and used. - Create and maintain clear, concise technical documentation for data pipelines, models, and processes. - Establish documentation and knowledge‑sharing practices that improve discoverability and reuse across teams and studios. - Advocate for modern engineering practices and contribute to the evolution of the studio’s data architecture. Performance will be measured based on: - Quality, reliability, and scalability of delivered data solutions - Adherence to project timelines and delivery commitments - Effectiveness of documentation and knowledge sharing - Collaboration with stakeholders and contribution to data‑driven decision making Qualifications - Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related technical discipline, or equivalent professional experience. - Demonstrated expertise in SQL for analytics and data engineering use cases. - Proven experience designing and implementing scalable ETL processes, including data movement, orchestration, and quality controls. - Hands‑on experience with modern big data analytics platforms, including data lakes, distributed processing frameworks, and columnar storage formats. - Experience building and operating cloud‑hosted data systems, with strong preference for Microsoft Azure. Requirements - Azure Data Factory, Databricks, Azure Synapse Analytics - Spark‑based data processing - Data lake and lakehouse architectures - Parquet and Delta Lake formats - Azure Data Explorer and Kusto Query Language - Data modeling for analytics and reporting - Data quality, governance, and observability practices Benefits - Opportunity to work on the front edge of modern data engineering using Databricks, Azure Synapse Analytics, and Azure Data Explorer. - Direct impact on studio‑wide decision making, test‑driven development, and data‑driven culture. - Exposure to large‑scale lakehouse and warehouse analytics systems handling both real‑time and batch data. - Collaborative environment where data engineering directly influences business, design, testing, and development outcomes. Assessment Process Candidates will be evaluated through a combination of technical interviews, practical problem‑solving discussions, and assessment of prior experience building and supporting modern data platforms. Emphasis will be placed on applied data engineering skills, architectural judgment, and documentation practices.



