Job Closed
This listing is no longer active.
Empowering Every Mind, Elevating Every Ability.
Senior Data Engineer – Real-Time Streaming
Location
United States
Posted
95 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer – Real-Time Streaming
Orbital AI
• Design and implement real-time streaming data pipelines for high-volume event data. • Develop and operate distributed data processing systems using technologies such as: • Apache Flink • Apache Kafka • Apache Druid • Build scalable ingestion pipelines capable of handling millions of events per second. • Design low-latency analytical data stores for operational dashboards and real-time analytics. • Optimize data pipelines for performance, scalability, and fault tolerance. • Work with product and analytics teams to translate business needs into real-time data models. • Build and maintain data observability, monitoring, and reliability frameworks. • Implement schema evolution and data quality controls across streaming pipelines. • Contribute to data platform architecture decisions and infrastructure design. • Mentor junior engineers and promote best practices in data engineering and distributed systems.
Job Requirements
- 7+ years of experience in data engineering or distributed systems development
- Strong experience building streaming data pipelines
- Hands-on experience with at least one major streaming framework
- Experience with real-time analytical databases
- Experience with large-scale distributed systems
- Strong SQL skills and experience designing analytical data models
- Experience building fault-tolerant, highly scalable pipelines
- Proficiency in one or more programming languages:
- Java
- Python
- Experience with AWS
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Design, build, and maintain scalable data pipelines that drive analytics, product insights, and operational reporting across the company • Develop data movement and storage strategies that meet security standards while managing cost and efficiency tradeoffs • Partner closely with data analysts to ensure value is being delivered in our reporting and alerting systems which are broadly consumed across the business • Collaborate with engineering, product, and operations teams to support our product roadmap and ongoing customer needs • Educate others on best practices and continually raise the bar when it comes to the production, application, and maintenance of data at Strike
• Write well-designed, testable, efficient code • Produce specifications and determine operational feasibility • Integrate software components into a fully-functional software system • Develop software verification plans and quality assurance procedures • Document and maintain software functionality and APIs • Configure and deploy software tools, processes and metrics • Adhere to industry standards and regulations • Embrace and evolve our agile development practices
Lead Data Migration Consultant – SAP S/4HANA
TSP - The Silicon Partners IncBringing SAP, RPA, AI/ML and Cloud Together!
• Lead end-to-end data migration activities for SAP S/4HANA implementation projects. • Design and execute data migration strategy, planning, and execution. • Hands-on experience with LTMC (Legacy Transfer Migration Cockpit) and DMC (Data Migration Cockpit). • Manage data extraction, transformation, cleansing, validation, and loading activities. • Work closely with business stakeholders, functional consultants, and technical teams to ensure data quality. • Lead data migration workshops with clients and gather migration requirements. • Perform data mapping, reconciliation, and validation. • Handle mock loads, trial migrations, and cutover planning. • Ensure data integrity, consistency, and compliance throughout the migration lifecycle. • Mentor and guide data migration team members. • Provide regular status updates to project stakeholders and leadership. • Troubleshoot and resolve data migration issues and defects.
Computer and Information Systems Manager
Fox CorporationFox Corporation produces and distributes content through leading brands, including: FOX News Media FOX Sports FOX Entertainment FOX Television Stations Tubi Media Group We empower a diverse range of creators to imagine and develop culturally significant content, while building an organization that thrives on creative ideas, operational expertise, and strategic thinking.
This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more. Role Description Fox Cable Network Services, LLC seeks a Computer and Information Systems Manager (Manager, Data Engineering), responsible for designing, developing, and maintaining comprehensive Data Lake and warehouse solutions. Tackle challenges that come with complex large-scale data having different streams of data sources. Specific duties include: - Lead the design, development, and implementation of data architectures, data pipelines, and data systems. - Manage a team of 3-5 data engineers and provide technical guidance and mentorship. - Build scalable data infrastructure and understand distributed systems concepts from a data storage and compute perspective. - Ensure the accuracy and availability of data to customers and understand how technical decisions can impact the business’s analytics and reporting. - Collaborate with cross-functional teams, business partners, stakeholders including senior leadership to understand business requirements, promote best practice and trade-offs, and to provide solutions that meet the organization's goals. - Develop, democratize and maintain Data as an asset. - Optimize data processes for efficiency, reliability, and scalability. - Act as a Subject Matter Expert to the organization for end-to-end data pipeline. - Ensure quality by performing root cause analysis and troubleshooting of defects. - Stay current with industry trends and advancements in Data Engineering. Telecommuting permitted from any location in the U.S. Qualifications - Bachelor’s degree in Computer Science, Computer Engineering or related field, plus two (2) years of data engineering or related experience executing multiple large data analytics and engineering projects. - In lieu of Bachelor’s degree, will accept four (4) years of data engineering or related experience executing multiple large data analytics and engineering projects. Requirements - Two (2) years of experience designing and developing Platform Architecture and ETL/Data pipelines. - Experience developing frameworks and reusable components for AWS cloud data migration and integration. - Experience analyzing and capturing batch, streaming and near real-time data (ETL) requirements for Enterprise Data Lake and Data Warehouse. - Experience working with various stakeholders on requirement gathering, defining project scope, roadmaps and short and long-term strategic goals. - Specific skills required include: - Programming languages such as Python or Java. - pySpark or Scala. - AWS Redshift, Athena, Spectrum, DynamoDB and RDS. - Amazon Web Services including EC2, Elastic Map Reduce (EMR), Glue, Lambda, Kinesis, IAM, ECS, S3, API Gateway, Kinesis and Kafka. - Agile Methodology, Software Development Life Cycle, Test Strategy and Performance Tuning. - Databricks, PL/SQL, Shell scripting, Tableau and Looker. - Jira, Bitbucket, Confluence and Git. Telecommuting permitted from any location in the U.S. Benefits - Medical/dental/vision insurance. - 401(k) plan. - Paid time off. - Other benefits in accordance with applicable plan documents. - Benefits for Union represented employees will be in accordance with the applicable collective bargaining agreement. Pursuant to state and local pay disclosure requirements, the pay rate/range for this role, with final offer amount dependent on education, skills, experience, and location is $200,970.00-221,067.00 annually.



