Data Engineer
Location
Poland
Posted
81 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer
Inetum
• Design, implement, and operate MCP and OCR integrations, ensuring the overall stability and performance of the platform. • Focus on developing enrichment integrations for LLM prompt generation. • Optimize platform reliability (performance tuning, DB connection handling, LLM gateway operations, observability, autoscaling).
Job Requirements
- Design and implement secure, robust integrations with multiple MCP servers to fetch and aggregate customer data (API clients, batching, connection pooling, retries, rate limiting, caching) and feed enriched context into LLM prompt generation.
- Implement and operate OCR integrations for processing attachments (PDFs, images), including tool/API selection, preprocessing, quality handling, and text integration into mail classification and prompt pipelines.
- Ensure secure handling of customer and PII data used in LLM workflows.
- Scale and performance‑tune GenAI Python microservices running in Kubernetes.
- Improve PostgreSQL session and connection management, including detection and resolution of connection leaks.
- Operate and harden the LLM‑Gateway (LiteLLM), implementing load balancing, throttling, and fallback strategies required by GenAI mail‑processing workflows.
- Implement and maintain observability for enrichment and platform workflows (metrics, traces, structured logs) and contribute to runbooks.
- Optional participation in on‑call rotation.
- Hands‑on experience with Kubernetes, Helm, and Docker.
- Strong experience integrating backend APIs, including authentication (tokens/OAuth/OIDC), retries, rate limiting, and caching.
- Practical experience with OCR tools/APIs or document‑processing pipelines.
- Solid Python skills for integration and troubleshooting (FastAPI, Django, Pydantic).
- Familiarity with PostgreSQL connection handling and basic tuning.
- Exposure to observability tools (Prometheus, Grafana, ELK).
- Strong understanding of data privacy and PII handling in production environments.
Benefits
- Attractive financial benefits: A cafeteria system that allows employees to personalize benefits by choosing from a variety of options.
- Generous referral bonuses, offering up to PLN6,000 for referring specialists.
- Additional revenue sharing opportunities for initiating partnerships with new clients.
- Professional development and team support: Ongoing guidance from a dedicated Team Manager for each employee.
- Tailored technical mentoring from an assigned technical leader, depending on individual expertise and project needs.
- Community and Well-Being: Dedicated team-building budget for online and on-site team events.
- Opportunities to participate in charitable initiatives and local sports programs.
- A supportive and inclusive work culture with an emphasis on diversity and mutual respect.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Principal Data Engineer - Principal Oracle DBA
Metropolitan Council of the Twin CitiesFounded in 1967, the Metropolitan Council of the Twin Cities is a regional planning agency serving the seven-county metropolitan area of Minnesota. The organiza
Title: Principal Data Engineer (Principal Oracle DBA) - REPOST Location: Saint Paul, MN, United States Job Description: Salary $101,004.80 - $163,841.60 Annually Location 390 Robert St. N St. Paul, MN Job Type Full-Time Remote Employment Flexible/Hybrid Job Number 2025-00215 Division Regional Administration Department IS-Admin WHO WE ARE This is a repost due to an additional vacancy. We are the Metropolitan Council, the regional government for the seven-county Twin Cities metropolitan area. We plan 30 years ahead for the future of the metropolitan area and provide regional transportation, wastewater, and housing services. More information about us on our website. We are committed to supporting a diverse workforce that reflects the communities we serve. Information Services (IS) is the central IT department within the Regional Administration (RA) division that supports all divisions of the Metropolitan Council. Our 140 team members provide technology, practices, and innovative solutions that enable the Council's core services. How your work would contribute to our organization and the Twin Cities region: The Principal Data Engineer works with business and IS leaders and stakeholders to establish data engineering strategies that support the organization’s goals and objectives. They lead and influence decisions around data, collaborating with IS and business units to create solutions that make data available to systems and/or data consumers, such as system users, developers, data scientists, or business analysts. They create and socialize the data engineering technology roadmap and plan how to get there. They evaluate new technologies, tools, and techniques and work closely with the IS architects to build data engineering solutions on top of the overall technology architecture. They help establish data governance policies, procedures, standards, and best practices to ensure compliance with industry regulations, security and data privacy laws, and information security requirements. This role designs, develops, and maintains data solutions. This includes databases (DBs), data lakes, data warehouses, data marts and data movement (data pipelines/ETL/ELT), as well as integrating analytics and data science outputs into existing business processes or systems. They administer, monitor, and maintain data solutions and tools, ensuring their security, performance, and availability. They also identify and resolve data quality issues. The Principal Data Engineer requires the highest degree of expertise in database technologies, data modeling, structured and unstructured data, and strong programming skills. They often act as technical leads and work on small to large initiatives, advising and mentoring others. They play a critical role in ensuring the data is properly managed and leveraged to support strategic decision-making and to achieve business objectives. Full Salary Range: $48.56 - $78.77 hourly/$101,005 - $163,842 yearly This position is eligible for a hybrid telework arrangement (both remote and onsite). Candidate's permanent residence must be in Minnesota or Wisconsin. What you would do in this job Strategic Planning & Leadership - Develop data engineering strategies for operations, reporting, and analytics. - Create and socialize the data engineering roadmap to achieve the strategic plan. - Evaluate new technologies, tools, and techniques and make recommendations. - Develop data policies, procedures, standards, and best practices. - Mentor, influence, and advise data engineers on industry best practices. - May act as a technical lead on small to large data-related initiatives. Data Operations - Ensure data systems are available, scalable, secure, and supportable. - Administer, monitor, and support 24x7x365 operations, ensuring data security, performance, and availability. - Resolve incidents and meet Service Level Agreements (SLAs). - Ensure compliance with industry regulations, security, and data privacy laws. - Recommend performance improvements, optimizations, and automation. - Help fulfill data practices requests, complete audits, and respond to cybersecurity events. - Efficiently complete data service requests. - Identify and resolve data quality issues. Data Development - Design, develop, and implement data solutions for operational, reporting, and analytical systems. - Create solutions that make data available to systems and/or data consumers, including system users, developers, data scientists, and business analysts. - Build and leverage CI/CD development practices. - Integrate analytics and data outputs into business processes or systems. Cross-Functional Collaboration - Effectively collaborate with business teams on data needs and requirements. - Partner with IS teams to deliver available, secure, and supportable solutions for on-premise, hybrid, and cloud environments. - Mentor and train IS and business teams on data solutions. What education and experience are required for this job (minimum qualifications) Any of the following combinations of education (in Computer Science/Engineering, Information Technology, or related) and relevant experience: - Master's degree and four (4) years of experience - Bachelor’s degree and six (6) years of experience - Associate’s degree and eight (8) years of experience - High school diploma/GED and ten (10) years of experience Knowledge, Skills, and Abilities Required: - Developed data strategies & roadmaps - Defined data standards, best practices, policies, & procedures - Administered & supported DB operations 24x7x365 & metrics for SLAs - Developed data solutions for operations, reporting, & analytics - ITIL, Incident, Service, Change & Problem Management experience - Knowledge of & experience with data privacy laws, security, & industry regulations - Installation & configuration of DB software & drivers on servers & clients - Writing & tuning complex SQL queries - Oracle DB Administration - Oracle Exadata Administration What additional skills and experience would be helpful in this job (desired qualifications): - Oracle Database@Azure Experience - SQL Server Administration - Experience leading a technical team - Modern cloud data platform experience in Databricks - Developing & supporting data integrations (data pipelines/ETL/ELT) - Reporting & data visualization experience What knowledge, skills, and abilities you should have within the first six months on the job: - Knowledge of IS teams, roles & responsibilities, services, & general technologies. - Understand the data management team, group functions, & data technologies. - Knowledge of the various business units & the types of data each has and needs. - Understand & use incident, service, problem, & change management processes. - Strong interpersonal relationships within and across teams, to remove barriers, reduce risk, build trust and improve insights. - Skill and comfort in delivering strategic guidance to IS teams and leadership. What you can expect from us: - We offer the opportunity to make a difference and positively influence the Twin Cities metropolitan area. - We encourage our employees to develop their skills through on-site training and tuition reimbursement. - We provide a competitive salary, excellent benefits, and a good work/life balance. More about why you should join us! Additional information Union/Grade: AFSCME/Grade J FLSA Status: Exempt Safety Sensitive: No What your work environment would be: You would perform your work in a standard office setting. Work may sometimes require travel between your primary work site and other sites. If you are new to the Metropolitan Council, you must pass a background check, which verifies education, employment, and criminal history. If you have a criminal conviction, you do not automatically fail. The Metropolitan Council considers felony, gross misdemeanor, and misdemeanor convictions on a case-by-case basis, based on whether they are job-related and whether the candidate has demonstrated adequate rehabilitation. If you are already an employee of the Metropolitan Council, you must pass a criminal background check if the job you're applying for is in Information Services. Security Policy: This position involves direct access to Criminal Justice Information (CJI) as defined by the FBI CJIS (Criminal Justice Information Services) Security Policy. In accordance with section 5.12.1.1 of the FBI CJIS Security Policy, final candidates must agree to submit to a state-of-residence and national fingerprint-based record check. If the record check reveals criminal convictions, the Metropolitan Transit Police Department and/or the Minnesota Bureau of Criminal Apprehension will review the nature and circumstances of those convictions to determine whether access to Criminal Justice Information would be permissible. If it is determined that access to Criminal Justice Information would not be permissible, the candidate will no longer be eligible for this position. IMPORTANT: If you make a false statement or withhold information, you may be barred from job consideration. The Metropolitan Council is an Equal Opportunity, Affirmative Action, and veteran-friendly employer. The Council is committed to a workforce that reflects the region's diversity and strongly encourages persons of color, members of the LGBTQ community, individuals with disabilities, women, and veterans to apply.
Data Engineer
CapTechCapTech is a privately held IT management consulting company offering customer engagement, transformation, data, analytics, and tailored solutions for private a
Title: Data Engineer (AWS, Azure, GCP) Location: Denver United States Job Description: Company Description CapTech is an award-winning consulting firm that collaborates with clients to achieve what's possible through the power of technology. At CapTech, we're passionate about the work we do and the results we achieve for our clients. From the outset, our founders shared a collective passion to create a consultancy centered on strong relationships that would stand the test of time. Today we work alongside clients that include Fortune 100 companies, mid-sized enterprises, and government agencies, a list that spans across the country. Job Description CapTech Data Engineering consultants enable clients to build and maintain advanced data systems that bring together data from disparate sources in order to enable decision-makers. We build pipelines and prepare data for use by data scientists, data analysts, and other data systems. We love solving problems and providing creative solutions for our clients. Cloud Data Engineers leverage the client's cloud infrastructure to deliver this value today and to scale for the future. We enjoy a collaborative environment and have many opportunities to learn from and share knowledge with other developers, architects, and our clients. Specific responsibilities for the Data Engineer - Cloud position include: - Developing data pipelines and other data products using Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP) - Advising clients on specific technologies and methodologies for utilizing cloud resources to efficiently ingest and process data quickly - Utilizing your skills in engineering best practices to solve complex data problems - Collaborating with end users, development staff, and business analysts to ensure that prospective data architecture plans maximize the value of client data across the organization. - Articulating architectural differences between solution methods and the advantages/disadvantages of each Qualifications Typical experience for successful candidates includes: - Experience delivering solutions on a major cloud platform - Ability to think strategically and relate architectural decisions/recommendations to business needs and client culture - Experience in the design and implementation of data architecture solutions - A wide range of production database experience, usually including substantial SQL expertise, database administration, and scripting data pipelines - Ability to assess and utilize traditional and modern architectural components required based on business needs. - A demonstrable ability to deliver production data pipelines and other data products. This could be hands on experience, degree, certification, bootcamp, or other learning. Skills: Successful candidates usually have demonstrable experience with technologies in some of these categories: - Languages: SQL, Python, Java, R, C# / C++ / C - Database: SQL Server, PostgreSQL, Snowflake, Redshift, Aurora, Presto, BigQuery, Oracle - DevOps: git, docker, subversion, Kubernetes, Jenkins - Additional Technologies: Spark, Databricks, Kafka, Kinesis, Hadoop, Lambda, EMR - Popular Certifications: AWS Cloud Practitioner, Microsoft Azure Data Fundamentals, Google Associate Cloud Engineer Additional Information We want everyone at CapTech to be able to envision a lasting and rewarding career here, which is why we offer a variety of career paths based on your skills and passions. You decide where and how you want to develop, and we help get you there with customizable career progression and a comprehensive benefits package to support you along the way. Alongside our suite of traditional benefits encompassing generous PTO, health coverage, disability insurance, paid family leave and more, we've launched extended benefits to help meet our employees' needs. - CapTech is committed to providing a flexible work environment and helping our employees achieve a work-life balance that suits their individual needs. Employees must be available to work onsite in a client location or a CapTech office as requested. We allow CapTech employees to work remotely when compatible with CapTech and client needs. - Learning & Development - Programs offering certification and tuition support, digital on-demand learning courses, mentorship, and skill development paths - Modern Health -A mental health and well-being platform that provides 1:1 care, group support sessions, and self-serve resources to support employees and their families through life's ups and downs - Carrot Fertility -Inclusive fertility and family-forming coverage for all paths to parenthood - including adoption, surrogacy, fertility treatments, pregnancy, and more - and opportunities for employer-sponsored funds to help pay for care - Fringe -A company paid stipend program for personalized lifestyle benefits, allowing employees to choose benefits that matter most to them - ranging from vendors like Netflix, Spotify, and GrubHub to services like student loan repayment, travel, fitness, and more - Employee Resource Groups - Employee-led committees that embrace and incorporate diversity and inclusion into our day-to-day operations - Philanthropic Partnerships - Opportunities to engage in partnerships and pro-bono projects that support our communities. - 401(k) Matching - Generous matching and no vesting period to help you continue to build financial wellness CapTech is an equal opportunity employer committed to fostering a culture of equality, inclusion and fairness - each foundational to our core values. We strive to create a diverse environment where each employee is encouraged to bring their unique ideas, backgrounds and experiences to the workplace. For more information about our Diversity, Inclusion and Belonging efforts, click HERE. As part of this commitment, CapTech will ensure that persons with disabilities are provided reasonable accommodations. If reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please contact Laura Massa directly via email lmassa@captechconsulting.com. CapTech supports Equal Pay for all. In addition, in the State of Colorado, we are committed to Equal Pay for ALL in accordance with the Colorado Equal Pay for Equal Work Act. The base pay range for this role is: $90,000 - $200,000. At this time, CapTech cannot transfer nor sponsor a work visa for this position. Applicants must be authorized to work directly for any employer in the United States without visa sponsorship.
• Design and build production-grade data pipelines in Databricks using Spark/PySpark and SQL. • Develop and maintain an Analytics ID stitching pipeline using deterministic and probabilistic matching techniques across multiple customer data sources. • Build and manage modular data marts (Identity, Behavior, Demographics) with independent refresh cadences. • Implement and maintain a scalable feature store supporting downstream analytics and data science use cases. • Own the end-to-end data lifecycle: ingestion, transformation, validation, deployment, monitoring, and optimization. • Develop data quality frameworks including schema drift detection, anomaly monitoring, match-rate validation, and automated deduplication audits. • Implement CI/CD processes for multi-environment promotion (dev/staging/prod) in Databricks environments. • Coordinate orchestration workflows and manage dependencies using Databricks Workflows or similar tools. • Collaborate closely with Data Architects and Client stakeholders to translate business rules into scalable technical solutions. • Produce comprehensive technical documentation including data contracts, lineage maps, architecture diagrams, and operational runbooks.
• Design and build production-grade data pipelines in Databricks using Spark/PySpark and SQL. • Develop and maintain an Analytics ID stitching pipeline using deterministic and probabilistic matching techniques across multiple customer data sources. • Build and manage modular data marts (Identity, Behavior, Demographics) with independent refresh cadences. • Implement and maintain a scalable feature store supporting downstream analytics and data science use cases. • Own the end-to-end data lifecycle: ingestion, transformation, validation, deployment, monitoring, and optimization. • Develop data quality frameworks including schema drift detection, anomaly monitoring, match-rate validation, and automated deduplication audits. • Implement CI/CD processes for multi-environment promotion (dev/staging/prod) in Databricks environments. • Coordinate orchestration workflows and manage dependencies using Databricks Workflows or similar tools. • Collaborate closely with Data Architects and Client stakeholders to translate business rules into scalable technical solutions. • Produce comprehensive technical documentation including data contracts, lineage maps, architecture diagrams, and operational runbooks.

