Generative media in the blink of an API.
Platform Data Engineer
Location
United Kingdom
Posted
15 days ago
Salary
0
Seniority
Senior
Job Description
Platform Data Engineer
Runware
• Design, build, and maintain **schemas and data models** • Optimize table layout, partitioning, indexing, and compression for high-volume data • Ensure fast, efficient querying for logs, requests, metrics, and performance traces • Maintain ingestion pipelines for billions of records • Build robust pipelines for: - API logs - Model inference logs - Error events - Usage & integration events - GPU & system metrics • Implement ETL/ELT workflows to transform raw data into analytics-ready structures • Ensure quality, reliability, and real-time availability of data sources • Build tooling to support large-scale **log analysis** • Enable deep investigation into latency, throughput, errors, and bottlenecks • Provide the raw data foundation for E2E inference-time monitoring • Help debug production issues using logs and traces • Work closely with DevOps, ML, and backend engineering • Integrate pipelines with monitoring tools (Prometheus, Grafana, Datadog, OpenTelemetry) • Automate ingestion and cleanup tasks • Build internal libraries or utilities to support monitoring and debugging workflows • Provide clean data interfaces for the Data Expert (dashboards, monitoring, analytics) • Support engineering teams by exposing the right logs and metrics • Contribute to debugging, RCA (root cause analysis), and performance optimization initiatives
Job Requirements
- Solid experience as a **Data Engineer** or similar role in a production environment
- Strong understanding of **data pipelines**, streaming vs batch processing, and data modeling
- Experience working with **analytical databases** (ClickHouse is a plus, but not mandatory)
- Comfortable digging through **logs, metrics, and platform data** to understand system behavior
- Familiarity with **event-based systems, monitoring, and observability concepts**
- Pragmatic mindset: you care about usefulness, reliability, and performance over theory
- Comfortable working cross-functionally with backend, infra, and data profiles
- Startup / scale-up experience is a plus
- Nice to Have
- Experience with high-throughput or realtime systems
- Exposure to cost monitoring, performance analytics, or platform observability
- Background in AI, ML platforms, or data-heavy products
Benefits
- Generous paid time off – vacation, sick days, public holidays
- Meaningful stock options – share in the upside you create
- Remote-first setup – work from home anywhere we can employ you
- Flexible hours – own your schedule outside core collaboration blocks
- Family leave – paid maternity, paternity, and caregiver time
- Company retreats – twice-yearly gatherings in inspiring locations
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Clinical Data Team Lead
Thermo Fisher ScientificThermo Fisher Scientific is a global biotechnology product development company whose mission is to make the world healthier, cleaner, and safer. Thermo Fisher Scientific leads a gl
Title: CDTL Location: Bangalore Karnātaka India Full time Remote Join Us as a Clinical Data Team Lead – Make an Impact at the Forefront of Innovation We have successfully supported the top 50 pharmaceutical companies and more than 750 biotechs, spanning 2,700 clinical trials across 100+ countries in the last 5 years. As part of our expert team, you’ll have the opportunity to ensure operational excellence that makes a real difference in organizational performance. As a Clinical Data Team Lead, you will act as the lead data manager for one or more projects or provide support to the lead data manager. You may lead all data management activities for individual, limited volume/complexity studies with support from senior team members. Additionally, you will act as an interdepartmental and client liaison for all data management study activities. What You’ll Do: • Applies relevant components of the project protocol to daily tasks and directs others on how to apply to their daily tasks. • Acts as a liaison to the Project Lead and/or client as required and communicates with management regarding all data management (DM) activities within their studies. • Develops and maintains data management project documentation files; performs independent reviews of data management deliverables following documented CDM guidelines. • Develops and delivers study specific training for DM project staff. • Produces project-specific status reports for management, Project Lead and/or clients on a regular basis. Monitors study metrics. • May participate in business development activities by assisting with bid preparation and/or representing data management at bid defense meetings, where required. • Assists with project forecasting of hours and identification of resource requirements and identifies potential out of scope activities to Project Lead and management and assists with the Contract Modification process. • Mentors junior level staff on all associated tasks within a study. • Assists with the administrative and financial management of allocated projects. Performs all assigned data management activities independently and efficiently, with attention to quality. Education and Experience Requirements: • Bachelor's degree or equivalent and relevant formal academic / vocational qualification • Previous experience that provides the knowledge, skills, and abilities to perform the job (comparable to 2+ years). In some cases an equivalency, consisting of a combination of appropriate education, training and/or directly related experience, will be considered sufficient for an individual to meet the requirements of the role. Knowledge, Skills and Abilities: • Ability to effectively apply knowledge and skills in a highly organized fashion while adhering to regulatory guidelines, SOPs and client expectations • Ability to use interactive computer programs • Good written/ verbal communication skills with a strong command of English language and grammar; good organizational, analytical/problem solving skills and attention to detail • Ability to work productively with minimal supervision • Ability to maintain a high degree of confidentiality with clinical data and client's proprietary data • Strong customer focus and excellent interpersonal skills • Proven flexibility and adaptability • Ability to work in a team environment and independently as needed • Ability to train and direct study team • Ability to set and meet timelines or be able to recognize and schedule changes in response to project demands
Data Engineer, MS
Codvo.aiBuilding Advance AI & Cloud Native Software Using The "Virtual Silicon Valley" Model. Let’s Talk AI, Cloud and Outcomes.
• Design and build ETL pipelines that ingest, validate, and standardize data from multiple sources with high accuracy and performance • Develop matching logic and fuzzy matching algorithms using tools like rapidfuzz to identify and reconcile data discrepancies across datasets • Create automated report generation systems that produce Excel and PDF outputs with complex formatting, formulas, and dynamic content • Implement data transformation workflows including date normalization, field mapping, and scenario classification based on business rules • Write comprehensive unit tests using pytest and maintain code quality through version control (Git) and code review practices • Translate business requirements into technical specifications, documenting matching rules, reconciliation scenarios, and data workflows directly from stakeholder discussions • Optimize database queries and data operations using relational databases (PostgreSQL, MySQL) and SQLAlchemy ORM for efficient data access • Collaborate with cross-functional teams to troubleshoot data issues, validate outputs, and continuously improve pipeline reliability and performance
Data Quality
CapcoCapco, a Wipro company, is a management & technology consultancy dedicated to the financial services & energy industries
Job Title: Data Quality About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. WHY JOIN CAPCO? You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry. MAKE AN IMPACT Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services. #BEYOURSELFATWORK Capco has a tolerant, open culture that values diversity, inclusivity, and creativity. CAREER ADVANCEMENT With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands. DIVERSITY & INCLUSION We believe that diversity of people and perspective gives us a competitive advantage. The Opportunity: - As an AVP – Finance Data Quality role-holder you will be accountable and responsible to support Data Services for Finance related Value Streams for policy and regulatory compliance and mitigating data risk within the bank. The ideal candidate will possess strong technical skills, experience with finance or financial regulatory reporting in a banking environment, and be comfortable collaborating closely with IT teams and business stakeholders. What you’ll do: Impact on the Business/Function - Working as an individual contributor, support Finance to become DMOV and DUSE compliant. - Support Finance and the business process owners to ensure their key Processes and Data (data flows including Right Sources and data definitions) are documented, and the data sets produced/consumed by their processes are controlled and monitored within key systems by using a range of tools. - Help Finance to identify & log Control Issues and DQ issues and on-board them to the data quality issue management process and operating model. This includes ensuring right level of information is captured in the Group Issue Management Tool to progress the issue towards remediation. Customers / Stakeholders - Build strong working relationships with the data SMEs in within Finance and DAO teams . - Build strong working relationships with ITSO ,DMP team and other technical support teams like Collibra , ARIS, EIM etc. - Within GDMO closely work with Data Governance, Data Architecture (Business) and Trusted Source teams. Operational Effectiveness & Control: - Maintain a continued focus on continuous improvement across areas of responsibility Leadership & Teamwork - Make a positive contribution to the Finance Data Team’s effort to meet objectives and help Finance to meet their policy KCIs. - Living and building values and building longer term shareholder value. - Contribute to an enjoyable, inclusive team atmosphere. - Take ownership and support the end-to-end implementation of strategic solutions Qualifications - Internal What you will need to succeed in the role: - Experience working in Finance, with a solid understanding of banking finance concepts including Liquidity, Risk Weight Assets, Treasury etc. - Knowledge of regulatory reporting requirements of a large diversified global bank (e.g., Basel III, BCBS239, CARR etc) - Desired banking knowledge / quick learning aptitude and ability of developing knowledge - Adaptive to multi data environment with quick learning ability to learn infrastructures. - Understanding / Experience of Alteryx, Python, SAS, SQL. Exposure / Experience to data visualization tools like Qlik Sense, Tableau. - Proficient in Microsoft Excel, PowerPoint. Visio, VBA. - Exposure to document and content management utilities like Rational Team Concert, Confluence etc. - Adaptive to change environment and capable of delivering in matrix organization. - Ability for logical thinking, eye for details - Effective written and verbal communication skills to work with multiple sites across geographies - Exposure to SDLC and Agile methodologies.
Data Architect
CapgeminiFounded in 1967, Capgemini is revered as one of the world's leading consulting, technology, and outsourcing agencies. In 2016 alone, the company reported global
• Design, configure, and maintain Teamcenter data models using BMIDE, including object types, attributes, relationships, and business rules. • Develop, deploy, and support BMIDE templates and extensions across development and production environments. • Translate Navy engineering, logistics, and lifecycle management requirements into scalable, maintainable PLM data structures. • Apply and enforce data modeling standards, naming conventions, and BMIDE best practices to ensure consistency, reusability, and compliance across the NPLM solution. • Develop and maintain comprehensive data documentation, including data dictionaries, schema diagrams, and metadata definitions, to support transparency, governance, and knowledge transfer. • Support the evolution of the Teamcenter data model architecture, ensuring alignment with enterprise objectives and interoperability requirements. • Conduct impact analysis and implement controlled updates to the data model to support system integrations and evolving mission needs. • Collaborate with solution architects, developers, and functional stakeholders to align data model design with system capabilities and business requirements. • Partner with data governance and architecture teams to promote a single source of truth, reduce data duplication, and align with enterprise data standards. • Support testing, validation, and troubleshooting of BMIDE configurations, including resolving schema conflicts, deployment issues, and data inconsistencies. • Performs other duties as assigned.




