The world's trusted engineering network
Data Engineer
Location
Worldwide
Posted
8 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer
Castillians
• Improve code base and data model of the data estate • Assist with maintenance tasks and daily data operations • Monitor data loading process for Datawarehouse ETL/ELT • Migrate Central Backend to Central repository • Collaborate on migration of architecture to Microsoft Fabric • Involve in web scripting data analytics project • Enhance data pipeline and auditing processes • Support production issues across pipelines • Conduct ad hoc data analysis for business stakeholders
Job Requirements
- Senior level experience
- Strong knowledge on data and BI estate
- Familiarity with Microsoft Fabric
- Experience with ETL/ELT processes
- Ability to monitor data loading process chain
- Knowledge of data migration strategies
- Proficiency in data scripting and analytics
Benefits
- Professional development opportunities
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Software Engineer – Data Platforms
smartclipWe are shaping the future of online video and TV advertising
• Join our team at smartclip as a Software Engineer, Data Platforms. • You will design and build the engines that process, transform, and refine terabytes of TV and advertising data every single day. • Ensure our data platform is scalable, resilient, and cutting-edge. • Design and implement technical solutions for Big Data applications in an agile, high-performance environment. • Continuously develop and refine our data aggregation pipelines (TV data, advertising, and forecasting) primarily based on Spark and Scala. • Host, manage, and scale your solutions using Docker and Kubernetes within the AWS cloud. • Lead the way in test automation – from unit tests to complex integration tests. • Evaluate new frameworks and tools, share your findings with the team, and help us stay at the forefront of Big Data technology.
Master's Fellow – Data Engineering, Automated Ingestion, AWS S3, Data Governance
Sistema FibraPelo Futuro da Indústria | Pelo Futuro do Trabalho
• The fellow will support the structuring, development, and evolution of a data solution based on Retrieval-Augmented Generation (RAG), with a focus on organizing, governing, and intelligently providing access to the area's information.
• Design, develop, and maintain secure, scalable ETL/ELT pipelines processing structured and unstructured data from multiple sources • Build and optimize data models and processing workflows to support BI, analytics, and reporting needs • Develop monitoring and alerting solutions to ensure pipeline performance and data quality • Build insightful dashboards, reports, and datasets in Domo • Integrate data from external APIs and third-party systems (e.g., Salesforce) • Collaborate with Software Engineering, Architecture, Product, Analytics, and IT teams • Translate business requirements into robust technical solutions • Conduct code reviews and provide feedback to engineering team members • Mentor other data engineers and contribute to onboarding efforts • Participate in Agile ceremonies including standups, retrospectives, and planning
Data Engineer – Surveillance and Interoperability
ARHS GroupAt the heart of your IT Projects: Delivered. On Time, On Budget, On Scope. #WeAreCodeBlooded
• Design and develop technical specifications for an interoperability middleware based on client's SMART Guidelines. • Support subject matter experts in defining and validating data dictionary mappings. • Design mapping logic between surveillance systems such as DHIS2, SORMAS, Go.Data, OpenELIS, and other health information systems. • Identify interoperability gaps and propose scalable technical solutions. • Document architecture decisions, interoperability workflows, and design trade-offs. • Develop specification frameworks aligned with client's SMART Guidelines, ICD-11, LOINC, SNOMED CT, and other healthcare interoperability standards. • Contribute to the design of AI agent frameworks and orchestration layers supporting data integration. • Configure and optimize relational and graph/network database environments. • Develop scalable ingestion frameworks capable of operating in both cloud and on-premises environments. • Implement staging layers for data ingestion, validation, transformation, harmonization, and quality assurance. • Design synchronization mechanisms supporting low-resource environments and offline data collection. • Develop production-grade ETL/ELT pipelines to automate ingestion and processing of surveillance datasets. • Build AI-assisted workflows and agent-driven mechanisms for extracting and integrating data from systems such as DHIS2, SORMAS, EWARS, and other external sources. • Implement automated processes for data validation, cleansing, deduplication, normalization, and harmonization. • Ensure pipelines efficiently process heterogeneous datasets while delivering high-quality data for analytics and modelling teams. • Optimize pipeline performance, scalability, and reliability. • Develop automated workflows for weekly and monthly surveillance reports and situation reports (SitReps). • Build APIs and data export services supporting downstream analytics and modelling. • Develop clean analytical datasets optimized for threshold analysis and collaborative modelling. • Support dashboard development and data visualization initiatives. • Ensure reporting infrastructure meets security, performance, and interoperability requirements.




