Job Closed
This listing is no longer active.
The world's trusted engineering network
Senior Data Engineer
Location
Ireland
Posted
42 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
Castillians
• Design, build, and maintain scalable data pipelines (ETL/ELT) to support analytics and AI use cases • Develop and optimise data architectures, data lakes, and data warehouses • Work closely with stakeholders to translate business needs into data solutions and insights • Build and maintain data processing scripts and automation tools (Python, SQL) • Support and enhance AI/analytics platforms and data products • Implement monitoring, testing, and performance tuning of pipelines • Collaborate with cross-functional teams (data scientists, engineers, product teams)
Job Requirements
- 5+ years of experience in Data Engineering / Software Engineering / Data Platforms
- Strong programming skills in Python and SQL
- Experience with ETL/ELT pipelines and data modelling
- Hands-on experience with: Data warehousing (Snowflake, Redshift, BigQuery, etc.)
- Big data tools (Spark, Kafka, Hadoop)
- Experience with cloud platforms (AWS, Azure, or GCP)
- Knowledge of relational and NoSQL databases
- Familiarity with CI/CD and DevOps practices
- Experience with real-time/streaming data pipelines
- Exposure to AI/ML data pipelines or feature engineering
- Experience with containerisation (Docker, Kubernetes)
- Knowledge of data observability and monitoring tools (ELK, Splunk)
- Experience in large-scale distributed systems
Benefits
- Clear scope with no ambiguity over deliverables.
- Opportunity for repeat engagements based on performance.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• This is a remote position. • The Data Engineer works end-to-end, creating dashboards and is responsible for integrating data, applying business rules, and developing effective visualizations. • Must be proficient in Qlik Script, DAX, and functions in both Qlik Sense and Power BI, data modeling, SQL queries, and building interactive dashboards. • Collaborate with business teams to understand project requirements and objectives. • Design and develop data pipelines to collect, integrate, and transform data from multiple sources. • Create dimensional and relational data models to support complex analyses. • Develop and maintain reports and dashboards using BI tools such as Qlik Sense. • Perform integration testing and data validation to ensure the accuracy and consistency of information. • Optimize queries and ETL processes to improve system performance and efficiency. • Provide technical support to end users to ensure effective adoption of BI solutions. • Monitor data integrity and security, implementing data governance best practices.
• Desenvolver e manter pipelines de dados utilizando GCP (Google Cloud Platform) • Atuar com BigQuery para modelagem, consulta e otimização de dados • Trabalhar com ferramentas como Dataflow e Looker para processamento e visualização de dados • Implementar e garantir práticas de governança de dados, incluindo qualidade, catálogo e linhagem • Definir e aplicar políticas de privacidade e segurança de dados (LGPD e boas práticas) • Monitorar e garantir a integridade, confidencialidade e disponibilidade dos dados • Colaborar com times de negócio e tecnologia para entendimento de requisitos e entrega de soluções • Apoiar iniciativas de data governance e compliance • Documentar processos, fluxos e políticas de dados
Senior Data Architect – Common Data Model, Metadata Platforms
aKUBE*** Recruitment Simplified *** People | Process | Technology
• Lead current state assessment of metadata systems and identify gaps • Design and build common metamodel and enterprise data model • Develop and enhance metadata utility platform • Integrate multiple data formats and systems into unified model • Create data dictionary and standardized data definitions • Drive future state architecture and enhancements • Support testing, validation, and production deployment • Manage CI/CD setup and system environments • Conduct knowledge transfer and documentation • Coordinate with program manager and stakeholders on priorities and delivery
Role Description - Desenvolver e otimizar pipelines de dados em larga escala com foco em desempenho e confiabilidade; - Atuar na construção de soluções de streaming e batch para ingestão, transformação e consumo de dados; - Colaborar com arquitetos e desenvolvedores na definição de padrões técnicos e boas práticas; - Garantir observabilidade, versionamento e qualidade dos dados em produção; - Apoiar iniciativas de modernização e automação de processos dentro da tribo; Qualifications - Experiência sólida como Engenheiro(a) de Dados Sênior; - Proficiência em Scala, Spark (Streaming) e Python; - Conhecimento prático em arquiteturas distribuídas e microsserviços; - Vivência em processamento de dados em nuvem (AWS); - Capacidade analítica e pensamento voltado à eficiência e escalabilidade; Requirements - Experiência com ScyllaDB (ou Apache Cassandra); - Conhecimento em Kafka; - Vivência com serviços AWS como Lambda, SNS, SQS e Glue; - Experiência em ambientes de dados voltados a produtos digitais e times;


