The only account you'll ever need to secure, transact, and explore onchain like never before.
Senior Data Engineer
Location
United Kingdom
Posted
94 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
Brahma
• Feature engineering at scale • Architect unstructured data pipelines • Orchestrate ML workflows using Dagster or Airflow • Optimize compute & cost • Build "dataset-as-code" • Infrastructure ownership with Kubernetes for scalability
Job Requirements
- Expert-level Python
- Unstructured data experience
- Familiarity with multi-modal pipelines (video/audio/images)
- Practical experience with distributed computing tools like Ray, Dask, or Spark
- Comfortable wrapping code in Docker and deploying to Kubernetes
- Understanding of cloud storage classes, lifecycle rules and throughput limits
- Systems mindset focusing on cost-effectiveness and observability
- Proactive ownership of issues
Benefits
- High-performance engines that transform raw video into production-ready model fuel
- Feature extraction on an ad-hoc request basis
- Building multi-modal pipelines
- Deploying and managing workloads on Kubernetes
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Modernizing and developing e-commerce applications • Develop, test, and maintain high-quality Magento applications and extensions. • Customize Magento themes and templates to create tailored user experiences. • Integrate third-party modules and APIs to enhance platform functionality. • Optimize Magento sites for performance, scalability, and security. • Troubleshoot and debug issues to ensure seamless operation of e-commerce platforms. • Collaborate with designers and frontend developers to implement new features and enhancements. • Stay updated with the latest Magento developments, trends, and best practices to recommend improvements. • Participate in code reviews and share knowledge with team members to foster a collaborative environment.
Unstructured Data Engineer
LeidosLeidos is an innovation company rapidly addressing the world’s most vexing challenges in national security and health.
• Design, build, and manage end-to-end RAG pipelines for enterprise AI applications. • Lead preprocessing of unstructured data, including discovery, classification, cleansing, redaction, and metadata enrichment. • Develop and optimize document chunking, embedding, and vectorization strategies for structured and unstructured datasets. • Coordinate ingestion of curated datasets into vector databases and AI platforms. • Package curated unstructured datasets as governed, reusable data products for enterprise consumption. • Define and implement metadata tagging strategies to align with Collibra governance standards. • Partner with Data Governance and Data Quality teams to ensure AI-ready data meets enterprise standards for lineage, classification, and compliance. • Evaluate and optimize embedding models, retrieval strategies, and indexing performance. • Monitor and tune RAG pipeline performance, including latency, retrieval accuracy, and cost efficiency. • Implement automation for document ingestion, transformation, and publishing workflows. • Support integration with enterprise AI platforms (e.g., ChatGPT Enterprise, AskSage, Moveworks). • Conduct cost analysis and capacity planning for vector storage and processing workloads. • Provide technical guidance on AI data readiness and unstructured data lifecycle management. • Design, implement, and optimize enterprise-grade RAG and prompt engineering frameworks, including context engineering strategies (chunking, metadata enrichment, semantic filtering, dynamic context management) to improve retrieval accuracy, grounding, and response quality. • Develop and maintain scalable multi-modal data pipelines that ingest, preprocess, embed, and integrate text, documents, images, audio, and structured data into governed vectorized data products consumable by enterprise AI platforms.
• Wspieraniu klientów w tematyce inżynierii i analityki dużych zbiorów danych w środowisku chmurowym – tworzenie rozwiązań od koncepcji po wdrożenie oraz inne fazy procesu SDLC/DDLC • Definiowaniu rozwiązań Data Intelligence i zarządzaniu ich wdrażaniem pod kątem technicznym i metodycznym • Wspieraniu celów biznesowych naszych klientów poprzez opracowywanie i wdrażanie rozwiązań analitycznych • Budowaniu Proof of Concept (PoC) i nadzorowaniu architektury Microsoft dla naszych klientów • Dzieleniu się wiedzą i pełnieniu funkcji trenerskiej w zakresie Azure Data Intelligence, oferowaniu wsparcia pracownikom, związanego z tematyką BI i analityką danych • Współpracy z innymi działami biznesowymi w celu dostarczania przydatnych rozwiązań analitycznych • Proponowaniu nowych możliwości wykorzystania danych, wdrażaniu nowoczesnych rozwiązań analitycznych oraz dbaniu o najwyższą jakość analiz i raportów, które pomagają kształtować strategiczne decyzje biznesowe
• Design, develop, and maintain optimal data pipelines and workflows. • Provide technical guidance on architecture design and collaborate with Data Engineers for implementation. • Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs. • Perform data integration, transformation, and cleaning to create insightful and actionable data for the business. • Translate business requirements into technical specifications and finished products. • Participate in code reviews and contribute to team knowledge sharing. • Lead the development and maintenance of documentation for data engineering processes and systems. • Create ad-hoc reports requested by internal and external partners. • Work as a team to support and troubleshoot errors for 100+ data pipelines. • Work closely with the application development team to update data and data structures. • Provide analysis, recommendations, and feedback to business process owners, leadership team, and the Information Technology department. • Propose automated solutions to repeated development tasks.




