Maximizing your ROI with AI-driven Real Estate insights.
Senior Data Engineer
Location
Colombia
Posted
18 hours ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
8020REI
• Own Big Data pipelines end to end. • Design, build, and optimize PySpark ETL/ELT pipelines on Amazon EMR and AWS Glue that process nationwide, county-partitioned property data (First American, BuildZoom permits, market comps) on daily and monthly cadences. • Run our lakehouse. • Operate and evolve our Bronze → Silver → Gold data lake on S3 with Apache Hudi and the AWS Glue Data Catalog, queried through Athena, including schema contracts, partitioning strategy, compaction, and performance tuning. • Orchestrate and automate. • Build reliable orchestration with AWS Step Functions, EventBridge, and Lambda; make reruns, backfills, and failure recovery boring and documented. • Enforce data quality. • Implement and extend our Data QA Audit Standard: layer contracts, write-audit-publish gating, quarantine flows, drift monitoring, and actionable Slack alerting, so bad data never reaches a client list. • Operate production databases and APIs. • Manage Aurora PostgreSQL and DynamoDB workloads, and run data-serving APIs (API Gateway, SQS-backed async workers), such as our Address Resolution Service, to production SLAs with dashboards and runbooks. • Own cloud cost. • Monitor, report, and reduce the AWS data-platform bill (EMR cluster sizing, Glue/Lambda usage, S3 lifecycle, Athena scan costs) as a first-class engineering responsibility. • Ship infrastructure as code. • Define infrastructure with Terraform and CloudFormation, delivered through GitHub Actions CI/CD with tests, linting, and coverage gates, we run a disciplined PR, branch-policy, and code-review culture. • Partner with Data Science. • Build the feature pipelines, training datasets, and serving paths behind our ML scoring and valuation models (scikit-learn/XGBoost-family stack), and co-own the handoff contracts between DS and DE. • Document like a pro. • Maintain runbooks, architecture docs, and data dictionaries (Confluence) so any teammate can operate what you build.
Job Requirements
- 4+ years of hands-on data engineering with large-scale distributed data systems and a track record of production ownership (not just development).
- Advanced PySpark : performance tuning, partitioning strategy, and cost-aware cluster sizing on real workloads (EMR or equivalent).
- Strong Python : clean, tested, production-grade code (we use pytest, ruff, mypy, and coverage gates in CI).
- Deep AWS experience : EMR, Glue, Lambda, S3, Athena, Step Functions, EventBridge, IAM, and VPC networking; comfort operating (not just deploying to) these services.
- Advanced SQL : complex analytical queries, query optimization, and data modeling on both a warehouse/lake engine (Athena/Presto) and PostgreSQL.
- Lakehouse experience : hands-on production work with at least one open table format (Apache Hudi strongly preferred; Iceberg or Delta Lake also valued) and medallion-style architecture.
- Data quality mindset : experience implementing validation, quality gates, monitoring, and incident response for production data.
- Infrastructure as Code : Terraform and/or CloudFormation in a CI/CD workflow.
- English and Spanish : professional working proficiency in both (B2+).
- Bachelor’s degree in Computer Science, Systems Engineering, Data Engineering, or equivalent practical experience.
Benefits
- Competitive Base Compensation
- Profit Share Bonus
- Flex PTO (up to 26 days per year)
- Home Office Upgrade Bonus
- HMO Bonus
- Full-time Remote Work
- Opportunity for growth and team-building potential
- Ongoing support and budget to develop new skills
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Role Description At Mindera, we believe that software is built by people, for people with high performance systems that impact users worldwide. We are looking for a Senior Data Engineer - Applied AI to join an agile, collaborative team where your voice matters as much as your code. You make the accelerators produce output that is correct, idiomatic, and adopted on the stacks our clients actually run. You encode what "good" looks like for each platform into validators and evals, build the platform adapters, and make the accelerators fit how delivery teams really work. We value empathy, self-organization, and the courage to take risks. If you love solving complex problems and believe that teaching others is the best way to learn, you’ll feel right at home here. National and international expected traveling time varies according to project/client and organizational needs: 0%-15% estimated. What will you own - The platform adapters that target each design partner's stack (Databricks, Snowflake, BigQuery, AWS), and the abstraction that emerges once a second stack forces it. - The domain side of the evaluation and verification layer: encoding each platform's idioms and correctness rules so generated output can be trusted. - The high-value generation targets with partners: ingestion and pipelines, transformation logic, warehouse and catalog scaffolding, orchestration, IaC and CI/CD, and the governance accelerators where partner pain is real (contracts, lineage, metadata, PII, data quality). - The fit into real delivery: reviewable diffs, integration with partner workflows, the path to adoption. We're Looking for Someone Who... - Has Senior-Level Impact: You bring a minimum of 5 years as a data engineer with a track record of end-to-end delivery in fast-paced environments. - Values Collaboration: You have professional fluency in English, are an excellent collaborator between developers and leadership, and have the ability to guide others. - Embraces Engineering Best Practices: You have a solid understanding of engineering and architectural principles, and you are comfortable collaborating on CI/CD workflows and using version control (Git). Your Technical Toolkit - Strong production software engineering: Python, testing, CI/CD, REST, Docker. You build reusable products, not one-off pipelines. - Real depth in modern cloud data platforms: You have gone deep on at least one of Databricks/Spark, Snowflake, or BigQuery, are strong in SQL and data modelling, and can reason confidently across the others and pick up a partner's stack fast. - Enough AI engineering to be a full builder on an agentic system: prompt design, structured outputs, function calling, and evaluation, with the Anthropic and OpenAI APIs. You co-own the AI work, you do not just hand requirements to your AI-systems counterpart. - Code-as-data fluency: codegen, templating, validating and parsing generated code, producing reviewable output. - You work close to delivery teams and design partners, and you care that the thing gets used. Recruitment Stages - Screening - Reviewing CVs and eligibility verification. - 1st Call - 30 minute call to assess Technical & Culture Fit with IT Recruiter. - Tech Interview - 2 hour call with two of our Cloud Engineers to assess your technical knowledge in depth. The interview includes a theoretical section based on test-case scenarios as well as live practical exercises. - Cultural Interview - 1-hour call with a Minder to assess cultural fit with Mindera and align expectations for both parties. - Project/Client Call - A 30 minute to 1 hour call with the client or project team to assess your technical fit for a specific project and align expectations. Benefits - We work in Self-organization culture and therefore we offer a flexible, supportive environment where you are trusted to manage your own time and career. - Well-being First: health insurance and "Open Holidays" because we trust you to know when you need a break. - Flexible Working hours and Work: Work from our vibrant hubs in Porto, Aveiro, or Coimbra, or choose a remote setup that works for you. As long as you are based in Portugal :) - Career Growth: A dedicated budget for training and conferences, plus a transparent profit-sharing model. - Tools for Success: Choose Laptop & Peripherals that best suit your needs, plus Hotspot with unlimited usage (PT), for work or Netflix ;) - Team work makes the dream Work: Mindera Annual Trip, Sports, and sharing groups to connect and have fun! Compensation The maximum monthly gross salary for this position is 6000EUR. Final offers will be based on experience and alignment with the role's requirements. Why Mindera? At Mindera, we use technology to build products we are proud of, with people we love. We believe diverse teams build better products. Mindera is a place where you can be your authentic self, regardless of gender, age, ethnicity, religion, or background. We strive to create an environment where everyone feels they belong and has an equal opportunity to grow. Most of all, you get to work with a group of great people in a politics free environment where the whole team owns the project together. Our culture reflects our lean and self-management mindset, encouraging colleagues to take risks, make decisions, collaborate openly, and communicate with everyone. Freedom and responsibility go hand in hand, and we value commitment, feedback, and empathy. We are proud of our work and love learning about everything as we navigate an Agile, Lean, and collaborative environment. Curious to know more? Check out our Blog and our Handbook! With offices located across Portugal, United Kingdom, United States, India, Romania, Brazil, Morocco, Spain, Australia, Vietnam join us as we expand and influence the future of software engineering worldwide.
Data Engineer
ClearCaptions LLCClearCaptions was founded in 2011 with a driving force to serve the hard-of-hearing to enable them to communicate again after they lose the ability to use the telephone. Since our inception, we have existed for one purpose, to improve the lives of our customers. Enabling communication is our passion. We believe in providing the highest levels of service and products to our customers. Our goal is to change lives by re-enabling social connections and independence through using the telephone today and through other services in the future. We are a technology service company that utilizes groundbreaking automatic speech recognition, human captioning, product development, and customer center marketing, sales, and service to deliver easy-to-use products to our mostly senior customer base. As a Federal Communications Commission (FCC) – certified telephone captioning provider, ClearCaptions follows the highest industry standards for privacy, security, and professionalism on all calls.
Role Description The Data Engineer will be responsible for collaborating with cross-functional teams to gather data requirements, developing efficient data solutions, and ensuring the accurate formatting and storage of enterprise data leveraging OneLake, Lakehouses, Data Pipelines, Dataflows Gen2, and Notebooks. The role will work closely with the reporting team to provide actionable insights based on reliable and repeatable data. In addition, the role will be involved in documentation, data governance, and providing level II/III support of ETL and/or warehousing solutions. This is a Remote/Work from Home position reporting to the BI Reporting and Development Manager. What you will do: - Collaborate with cross-functional teams to understand data and analytical requirements and translate them into effective Microsoft Fabric–based data solutions. - Design, develop and maintain end-to-end data ingestion and transformation processes using Microsoft Fabric Data Pipelines, Dataflows Gen2, and Notebooks. - Format, cleanse, and store data in a structured manner to ensure data quality and accessibility for reporting and analysis, following Medallion Architecture. - Work closely with reporting and analytics teams to ensure datasets are accurate, well-modeled, and optimized for Power BI and other analytical workloads. - Create comprehensive developer documentation for data pipelines, processes, and solutions. - Participate in data governance activities to ensure data integrity, security, and compliance. - Utilize source control to manage and track changes to data pipelines and codebase. - Provide level II/III support to troubleshoot and resolve data-related issues and inquiries. - Provide ancillary support for Machine learning (ML) and Artificial Intelligence (AI) processes. - Stay updated with industry trends and best practices related to data warehousing, ETL, and data engineering. Qualifications - Bachelor's degree in Computer Science, Information Technology, or a related field. - 3+ years experience in data engineering, ETL development, and data warehousing. - Experience with Microsoft Fabric, including OneLake, Lakehouses, Data Pipelines, Dataflows Gen2, and/or Notebooks. - Understanding of lakehouse architectures, data modeling, and modern analytics patterns. - Experience with data governance, data quality, and data security frameworks and standards. - Experience in Continuous Integration/Continuous Delivery (“CI/CD”). - Expertise in integrating Fabric solutions with Power BI, Azure services, and external data sources. - Expertise in developing data pipelines to support Machine Learning (ML) and/or Artificial Intelligence (AI) tools is a plus. - Excellent verbal and written communication skills, presentation, problem-solving skills, and attention to detail. - Self-starter with strong organizational and time management skills, self-directed and able to handle multiple priorities with demanding timeframes. - Ability to work independently and collaboratively with colleagues and staff to create a high-quality results-driven, team-oriented fast-paced environment. - Demonstrated ability to use discretion, make sound decisions, and maintain confidentiality. - Willingness and ability to work flexible hours. - Proficient in MS Office (including Excel, Outlook, PowerPoint, Visio, & Word), modern communication tools for virtual teams (i.e., MS Teams). Physical Demands - Employees may experience the following physical demands for extended periods of time: - Sitting, standing, and walking (95-100%) - Keyboarding (70-90%) - Viewing computer monitor, tablet and cell phone requiring close vision (70-90%) Work Environment - 100% Remote: Work environment is indoors at home. Compensation $120,000 to $135,000 plus 5% determined by competitive market analysis and internal equity considerations. Final compensation will be based on the candidate’s qualifications, experience, and business needs. For details on our comprehensive benefits program, visit www.clearcaptions.com/careers to explore our total rewards package. Company Description Since our founding in 2011, our mission has been to improve the lives of seniors and their caregivers. We are deeply passionate about communication and committed to becoming the foremost provider of services and solutions that enable seniors to lead more meaningful and independent lives. ClearCaptions is a Federal Communications Commission (FCC)-certified telephone captioning provider, adhering to the highest industry standards of privacy, security, and professionalism.
Principal Engineer, Domain Data Architecture
NagarroNagarro (Frankfurt: NA9) is a leader in digital product engineering and drives technology-led business breakthroughs.
Role Description We are looking for a professional with extensive experience in data architecture and governance within the financial services domain. - Implement data standards, policies, and guidelines to ensure data deliveries and architecture artifacts maintain integrity, quality, and consistency across the business area/domain. - Ensure alignment with Data Governance frameworks and IT initiatives within the business area/domain. - Oversee business area/domain data modelling and ensure its global alignment. - Facilitate communication and collaboration for the adoption and maintenance of data policies and frameworks. - Drive the management and adoption of data policies and frameworks within the business area/domain across EMEA and globally. - Drive data architecture decisions for Data Solutions/Products. - Collaborate with IT teams and business stakeholders to gain alignment on data policies and standards. - Contribute to data architecture discussions and lead decision-making in this area. - Stay informed about industry trends, emerging technologies, and best practices in data architecture and data management. - Use industry knowledge to build future-focused data architecture capability roadmaps. Qualifications - Bachelor’s or master’s degree in computer science, Information Technology, or a related field. Requirements - Total Experience: 12+ years. - Business knowledge/SME in Risk, Finance, Transactions, Compliance, etc. - Experience working in Financial Services organizations and dealing with business capabilities/processes for various products such as Loans, Deposits, Capital Market products (Derivatives, Bonds, Swaps, Options, etc.). - Extensive knowledge of regulatory reporting requirements in these areas across EMEA and the associated solution landscape. - Good understanding of the Trade Lifecycle, its actors/roles and responsibilities, and how data is managed throughout the lifecycle. - Specialization in regulatory reporting areas such as ILAAP, FINREP, STE, TTR, CCR, etc., will be an added advantage. - Proven ability to deliver enterprise data architecture policies and target states. - Experience establishing governance functions, drafting Terms of Reference (ToRs), and chairing governance forums. - Ability to create data flows across domains (target states, transition states, etc.). - Competence in reviewing data architecture artifacts and making decisions based on policies and standards. - Experience with data modelling and industry standards (e.g., FSLDM, BIRD). - Understanding of current data methodologies (e.g., Data Lake, Data Mesh, Data Fabric, Hybrid, Agentic). - Data modelling experience/knowledge. - Experience evaluating and recommending data tools. - Hands-on experience with Databricks, Snowflake, or similar platforms. - Familiarity with Collibra and Enterprise Architecture tools such as LeanIX, Ardoq, etc. Benefits
Associate Principal Engineer, Data Engineer
NagarroNagarro (Frankfurt: NA9) is a leader in digital product engineering and drives technology-led business breakthroughs.
Role Description - Define and drive the enterprise data architecture strategy, principles, standards, and target-state architecture. - Develop enterprise data architecture roadmaps and transition plans aligned with business and technology strategies. - Establish and lead enterprise data governance forums, architecture review boards, and decision-making processes. - Create and maintain enterprise data architecture artifacts, including conceptual, logical, and physical data models, data flows, and architecture blueprints. - Design enterprise data flows across business domains and ensure alignment with target-state architecture. - Review and approve data architecture deliverables, ensuring compliance with enterprise standards, governance policies, and best practices. - Collaborate with business, engineering, analytics, and architecture teams to design scalable and reusable data solutions. - Evaluate emerging data technologies, architecture patterns, and tools, providing recommendations aligned with organizational objectives. - Provide architectural guidance for cloud data platforms, including Databricks, Snowflake, and other modern data ecosystems. - Define and promote best practices for data modelling, metadata management, data quality, lineage, and governance. - Support enterprise transformation initiatives by ensuring consistency, interoperability, and scalability across the organization's data landscape. - Mentor architects and engineering teams on enterprise data architecture principles and governance practices. - Stay current with evolving data architecture methodologies, industry standards, and emerging technologies, driving continuous improvement across the enterprise. Qualifications - Bachelor’s or master’s degree in computer science, Information Technology, or a related field. Requirements - Total Experience 11+ years. - Strong experience in Data Architecture, Enterprise Architecture, or Data Engineering. - Proven experience defining and delivering enterprise data architecture strategies, target states, and architectural roadmaps. - Strong experience establishing and governing enterprise data architecture practices, including governance forums, operating models, and Terms of Reference (ToRs). - Experience creating enterprise data flows across business domains, including current state, transition state, and target state architectures. - Strong ability to review data architecture deliverables and ensure compliance with enterprise standards, policies, and governance principles. - Hands-on experience with enterprise data modelling and industry-standard data models such as FSLDM (Financial Services Logical Data Model), BIRD (Banks' Integrated Reporting Dictionary), or equivalent. - Strong understanding of modern data architecture paradigms, including Data Lake, Data Mesh, Data Fabric, Hybrid Data Architecture, Agentic Data Architecture. - Strong understanding of enterprise data governance, metadata management, data quality, lineage, and master data management concepts. - Experience collaborating with enterprise architects, business stakeholders, engineering teams, and governance bodies to deliver scalable data solutions. - Excellent analytical, problem-solving, and decision-making skills. - Strong communication, stakeholder management, and leadership capabilities. - Experience working in Agile delivery environments. - Experience designing enterprise-wide logical and conceptual data models. - Experience evaluating, selecting, and recommending enterprise data architecture and modelling tools. - Hands-on experience with modern cloud data platforms such as Databricks, Snowflake, or similar technologies. - Familiarity with enterprise data governance platforms such as Collibra. - Experience with Enterprise Architecture tools such as LeanIX, Ardoq, or equivalent platforms. - Experience working in regulated industries such as Banking, Financial Services, or Insurance is an advantage. Company Description We're Nagarro. We are a Digital Product Engineering company that is scaling in a big way! We build products, services, and experiences that inspire, excite, and delight. We work at scale — across all devices and digital mediums, and our people exist everywhere in the world (18,500+ experts across 39 countries, to be exact). Our work culture is dynamic and non-hierarchical. We're looking for great new colleagues. That's where you come in!


