Parceira que transforma desafios em resultados!
Engenheiro de Dados Sênior
Location
Brazil
Posted
21 hours ago
Salary
0
Seniority
Senior
Job Description
Engenheiro de Dados Sênior
Cadmus Soluções em TI
• Desenvolver pipelines de dados escaláveis utilizando AWS (S3, EMR, Redshift). • Atuar com processamento de dados em larga escala usando DBT, DuckDB e Spark. • Participar da construção de DAGs dinâmicas via Airflow e YAML. • Discutir requisitos com stakeholders e propor soluções técnicas. • Monitorar e garantir a qualidade dos pipelines e dados processados.
Job Requirements
- Experiência sólida com AWS (S3, EMR, Redshift).
- Domínio de Python orientado a objetos e SQL.
- Conhecimento em DBT, DuckDB, Airflow.
- Experiência com arquitetura de dados e interdependência de pipelines.
- Desejável experiência com observabilidade e monitoramento de pipelines.
Benefits
- Saúde
- Reembolso de despesas
- Trabalho flexível
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Staff Data Engineer - US (Remote)
Luxury PresenceDo it all with Luxury Presence. Build your brand, expand your network, & close more deals.
Luxury Presence is building the AI growth platform for real estate. Backed by Bessemer Venture Partners and other top investors, we're a Series C company that has hit $100M in annual recurring revenue. More than 90,000 real estate professionals, including over 30% of the WSJ Real Trends top 100 agents in the United States, use us to run and grow their business. About the Role We’re seeking a Staff Software Engineer to strengthen our real estate MLS data platform squad. You will build robust data pipelines and backend services that power: • High-quality MLS and property data across 400+ feeds • Property discovery and search on agent websites • Personalized listing recommendations and other data-driven features • Conversational and operational AI agents that streamline internal workflows • The evaluation and monitoring infrastructure that keeps these systems improving over time This role sits at the intersection of backend engineering, data infrastructure, and AI-powered products. Who is the Data Platform Squad? We make sure clean, reliable MLS listing records and user click-stream data are always available to our products and customers. Our current team—a mix of data engineers and software engineers—owns the entire listing pipeline: ingestion, transformation, and normalization across 400+ MLS feeds and other sources. We also extend the platform to capture user-activity data for user-facing features such as personalized listing recommendations, and we build AI agents that automate feed onboarding and listing-issue triage, reducing manual effort for internal teams and clients and shortening the path from data to business impact. What You’ll Do Technical leadership & architecture • Own the end-to-end architecture for MLS and property data: streaming and batch pipelines, microservices, storage layers, and APIs • Design and evolve event-driven, Kafka-based data flows that power listing ingestion, enrichment, recommendations, and AI use cases • Drive technical design reviews, set engineering best practices, and make high-quality tradeoffs around reliability, performance, and cost Backend, data & platform engineering • Design, build, and operate backend services (Python or Java) that expose listing, property, and recommendation data via robust APIs and microservices • Implement scalable data processing with Spark or Flink on EMR (or similar), orchestrated via Airflow and running on Kubernetes where applicable • Champion observability (metrics, tracing, logging) and operational excellence (alerting, runbooks, SLOs, on-call participation) for data and backend services Streaming & batch data pipelines • Build and maintain high-volume, schema-evolving streaming and batch pipelines that ingest and normalize MLS and third-party data • Ensure data quality, lineage, and governance are built into the platform from the start—supporting analytics, AI/ML, and customer-facing features • Partner with analytics engineering and data science to make data discoverable and usable (e.g., semantic layers, documentation, self-service tooling) AI agents & data products • Collaborate with ML/AI engineers to design and scale AI agents that automate MLS feed onboarding, listing discrepancy triage, and other operational workflows • Work with frameworks such as PydanticAI, LangChain, or similar to integrate LLM-based agents into our data and service architecture • Help define and implement evaluation, logging, and feedback loops so these agents and data-driven products continuously improve Cross-functional impact & mentorship • Collaborate closely with Product, Engineering, and Operations to shape the roadmap for our data platform, MLS capabilities, and AI-powered experiences • Translate ambiguous business and customer problems into clear technical strategies and phased delivery plans • Mentor and unblock other engineers; elevate the overall level of technical decision-making on the team via pairing, reviews, and design guidance What You’ll Bring Experience & scope • 10+ years of professional software engineering experience, including owning production systems end-to-end • Significant experience working with data-intensive or distributed systems at scale (high volume, high availability) • Prior experience in a senior or staff/lead role where you influenced architecture, standards, and technical direction Core technical skills • Strong programming skills in Python or Java, with experience building microservices and APIs (REST/GraphQL) Hands-on experience with Apache Kafka or similar event/messaging platforms (Kinesis, Pub/Sub, etc.) • Deep experience with: ◦ Spark or Flink for large-scale data processing, across streaming and batch pipelines (on EMR or similar big-data compute) ◦ Airflow (or equivalent orchestration tools) ◦ Kubernetes for running data/compute workloads • Strong SQL and data modeling skills; solid understanding of ETL/ELT patterns, data warehousing concepts, and performance tuning • Experience building on AWS (preferred) or another major cloud provider, with a good grasp of cost, reliability, and security tradeoffs AI agent experience • Experience building or integrating AI agents into production workflows (e.g., internal tools, support automation, operational triage, or data workflows) • Familiarity with frameworks such as PydanticAI, LangGraph, Claude Code or similar, and how they interact with backend services, vector stores, and LLM APIs • Comfort working with logs, telemetry, and evaluation metrics to monitor, debug, and iteratively improve AI-driven systems Leadership & collaboration • Demonstrated ability to lead technical initiatives across teams, from idea to production (alignment, design, implementation, rollout) • Track record of mentoring other engineers and raising the bar on code quality, testing, and design • Strong communication skills; able to clearly explain complex technical decisions to both engineers and non-technical stakeholders • Customer and product mindset: you care about how the data and services you build improve the end-user and client experience, not just the internals Nice to Have • Experience with any of: ◦ Iceberg, Hive, or other table formats/data lake technologies ◦ Snowflake, Athena, Redshift, or other cloud data warehouses ◦ dbt or similar transformation frameworks ◦ Data quality / observability tools (e.g., Great Expectations, Monte Carlo, Datafold) ◦ Vector databases / retrieval (e.g., LanceDB, Pinecone, Elasticsearch/OpenSearch) • Background in real estate, marketplaces, or other domains where data quality and freshness are highly visible to customers • Prior experience in a startup or high-growth environment where you’ve built or significantly evolved a data platform Join us in shaping the future of real estate The real estate industry is in the midst of a seismic shift, and the future belongs to those who break new ground. As one of the fastest-growing companies in the proptech and marketing sectors, Luxury Presence challenges the status quo of what technology can do for real estate agents, leaders, and brokerages. We're a team of agile and tenacious innovators working collaboratively to drive the industry forward. Together, we build game-changing products that empower modern real estate entrepreneurs to dominate their markets. From award-winning web design to agile SEO solutions to cutting-edge AI tools, we deliver tech that anticipates market shifts and keeps our clients ahead of their competition. Founded in 2016 by Stanford Business School alum Malte Kramer, Luxury Presence has grown to a global team ranked on the Inc. 5000 fastest-growing companies list three years in a row. We're backed by world-class investors, including Bessemer Venture Partners, NextEquity Partners, Toba Capital, and Switch Ventures, and have raised $89 million to date. More than 18,000 real estate businesses rely on our platform, including 30% of the Wall Street Journal RealTrends top agents and teams. Additionally, many of the industry's most powerful brokerages rely on Luxury Presence as a trusted business partner. Every year since 2020, Luxury Presence has ranked on BuiltIn's Best Place to Work lists. HousingWire named our founder and CEO a 2024 Tech Trendsetter, we've received several Tech100 Awards, and we just scored an Inman Innovation Award for Best AI-Powered Platform. Luxury Presence is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, or national origin.
Staff Data Engineer - CANADA (Remote)
Luxury PresenceDo it all with Luxury Presence. Build your brand, expand your network, & close more deals.
Luxury Presence is building the AI growth platform for real estate. Backed by Bessemer Venture Partners and other top investors, we're a Series C company that has hit $100M in annual recurring revenue. More than 90,000 real estate professionals, including over 30% of the WSJ Real Trends top 100 agents in the United States, use us to run and grow their business. About the Role We’re seeking a Staff Software Engineer to strengthen our real estate MLS data platform squad. You will build robust data pipelines and backend services that power: • High-quality MLS and property data across 400+ feeds • Property discovery and search on agent websites • Personalized listing recommendations and other data-driven features • Conversational and operational AI agents that streamline internal workflows • The evaluation and monitoring infrastructure that keeps these systems improving over time This role sits at the intersection of backend engineering, data infrastructure, and AI-powered products. Who is the Data Platform Squad? We make sure clean, reliable MLS listing records and user click-stream data are always available to our products and customers. Our current team—a mix of data engineers and software engineers—owns the entire listing pipeline: ingestion, transformation, and normalization across 400+ MLS feeds and other sources. We also extend the platform to capture user-activity data for user-facing features such as personalized listing recommendations, and we build AI agents that automate feed onboarding and listing-issue triage, reducing manual effort for internal teams and clients and shortening the path from data to business impact. What You’ll Do Technical leadership & architecture • Own the end-to-end architecture for MLS and property data: streaming and batch pipelines, microservices, storage layers, and APIs • Design and evolve event-driven, Kafka-based data flows that power listing ingestion, enrichment, recommendations, and AI use cases • Drive technical design reviews, set engineering best practices, and make high-quality tradeoffs around reliability, performance, and cost Backend, data & platform engineering • Design, build, and operate backend services (Python or Java) that expose listing, property, and recommendation data via robust APIs and microservices • Implement scalable data processing with Spark or Flink on EMR (or similar), orchestrated via Airflow and running on Kubernetes where applicable • Champion observability (metrics, tracing, logging) and operational excellence (alerting, runbooks, SLOs, on-call participation) for data and backend services Streaming & batch data pipelines • Build and maintain high-volume, schema-evolving streaming and batch pipelines that ingest and normalize MLS and third-party data • Ensure data quality, lineage, and governance are built into the platform from the start—supporting analytics, AI/ML, and customer-facing features • Partner with analytics engineering and data science to make data discoverable and usable (e.g., semantic layers, documentation, self-service tooling) AI agents & data products • Collaborate with ML/AI engineers to design and scale AI agents that automate MLS feed onboarding, listing discrepancy triage, and other operational workflows • Work with frameworks such as PydanticAI, LangChain, or similar to integrate LLM-based agents into our data and service architecture • Help define and implement evaluation, logging, and feedback loops so these agents and data-driven products continuously improve Cross-functional impact & mentorship • Collaborate closely with Product, Engineering, and Operations to shape the roadmap for our data platform, MLS capabilities, and AI-powered experiences • Translate ambiguous business and customer problems into clear technical strategies and phased delivery plans • Mentor and unblock other engineers; elevate the overall level of technical decision-making on the team via pairing, reviews, and design guidance What You’ll Bring Experience & scope • 10+ years of professional software engineering experience, including owning production systems end-to-end • Significant experience working with data-intensive or distributed systems at scale (high volume, high availability) • Prior experience in a senior or staff/lead role where you influenced architecture, standards, and technical direction Core technical skills • Strong programming skills in Python or Java, with experience building microservices and APIs (REST/GraphQL) Hands-on experience with Apache Kafka or similar event/messaging platforms (Kinesis, Pub/Sub, etc.) • Deep experience with: ◦ Spark or Flink for large-scale data processing, across streaming and batch pipelines (on EMR or similar big-data compute) ◦ Airflow (or equivalent orchestration tools) ◦ Kubernetes for running data/compute workloads • Strong SQL and data modeling skills; solid understanding of ETL/ELT patterns, data warehousing concepts, and performance tuning • Experience building on AWS (preferred) or another major cloud provider, with a good grasp of cost, reliability, and security tradeoffs AI agent experience • Experience building or integrating AI agents into production workflows (e.g., internal tools, support automation, operational triage, or data workflows) • Familiarity with frameworks such as PydanticAI, LangGraph, Claude Code or similar, and how they interact with backend services, vector stores, and LLM APIs • Comfort working with logs, telemetry, and evaluation metrics to monitor, debug, and iteratively improve AI-driven systems Leadership & collaboration • Demonstrated ability to lead technical initiatives across teams, from idea to production (alignment, design, implementation, rollout) • Track record of mentoring other engineers and raising the bar on code quality, testing, and design • Strong communication skills; able to clearly explain complex technical decisions to both engineers and non-technical stakeholders • Customer and product mindset: you care about how the data and services you build improve the end-user and client experience, not just the internals Nice to Have • Experience with any of: ◦ Iceberg, Hive, or other table formats/data lake technologies ◦ Snowflake, Athena, Redshift, or other cloud data warehouses ◦ dbt or similar transformation frameworks ◦ Data quality / observability tools (e.g., Great Expectations, Monte Carlo, Datafold) ◦ Vector databases / retrieval (e.g., LanceDB, Pinecone, Elasticsearch/OpenSearch) • Background in real estate, marketplaces, or other domains where data quality and freshness are highly visible to customers • Prior experience in a startup or high-growth environment where you’ve built or significantly evolved a data platform Join us in shaping the future of real estate The real estate industry is in the midst of a seismic shift, and the future belongs to those who break new ground. As one of the fastest-growing companies in the proptech and marketing sectors, Luxury Presence challenges the status quo of what technology can do for real estate agents, leaders, and brokerages. We're a team of agile and tenacious innovators working collaboratively to drive the industry forward. Together, we build game-changing products that empower modern real estate entrepreneurs to dominate their markets. From award-winning web design to agile SEO solutions to cutting-edge AI tools, we deliver tech that anticipates market shifts and keeps our clients ahead of their competition. Founded in 2016 by Stanford Business School alum Malte Kramer, Luxury Presence has grown to a global team ranked on the Inc. 5000 fastest-growing companies list three years in a row. We're backed by world-class investors, including Bessemer Venture Partners, NextEquity Partners, Toba Capital, and Switch Ventures, and have raised $89 million to date. More than 18,000 real estate businesses rely on our platform, including 30% of the Wall Street Journal RealTrends top agents and teams. Additionally, many of the industry's most powerful brokerages rely on Luxury Presence as a trusted business partner. Every year since 2020, Luxury Presence has ranked on BuiltIn's Best Place to Work lists. HousingWire named our founder and CEO a 2024 Tech Trendsetter, we've received several Tech100 Awards, and we just scored an Inman Innovation Award for Best AI-Powered Platform. Luxury Presence is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, or national origin.
Data Governance, Professional, Senior I
Lincoln ElectricLincoln Electric is the world leader in arc welding, robotic welding systems, plasma and oxyfuel cutting equipment.
Role Description We are seeking a Senior Master Data Governance Analyst to play a critical role in strengthening, standardizing, and scaling our global data governance framework. This individual will operate as a strategic business-facing partner, driving the adoption of enterprise-wide master data standards established by leadership. This is not an IT configuration role. Success requires strong business acumen, hands-on governance experience, and the ability to translate standards into practical, scalable business execution. Advanced English required, please submit your resume in English. Key Responsibilities - Global Data Governance Leadership & Business Partnership - Serve as a senior governance partner to global data owners, data stewards, and business leaders, driving adoption of enterprise master data standards and policies. - Lead the rollout and governance of global master data initiatives, ensuring consistency while accommodating regional business needs. - Facilitate high-impact workshops, governance forums, and working sessions across countries and functions to drive alignment, decision-making, and accountability. - Influence stakeholders at multiple levels, ensuring sustained adoption and continuous improvement of governance practices. - Partner with internal audit and compliance teams to ensure adherence to SOX and internal control requirements. - Standards Implementation & Process Excellence - Translate global governance standards into scalable, actionable business processes and operating models. - Act as a bridge between business and IT, ensuring governance requirements are clearly defined, documented, and implemented effectively. - Document end-to-end processes, data standards, and governance frameworks to support global consistency and clarity. - Identify and drive process optimization opportunities to improve efficiency, scalability, and data reliability. - Solution Enablement & Data Integrity - Collaborate with IT to design and enhance governance workflows (e.g., approvals, change requests, validations) within a master data governance solution. - Support data migration, validation, and reconciliation activities across systems, ensuring data integrity and compliance. - Define and monitor KPIs and dashboards to measure data quality, governance effectiveness, and adoption across regions. - Communicate data insights and governance performance to stakeholders and leadership teams. - Change Management & Adoption - Lead change management efforts to support governance adoption, including stakeholder engagement, training, and communication strategies. - Develop enablement materials and ensure successful onboarding of new governance processes and tools. - Act as a champion for data governance culture, promoting accountability and ownership across the organization. Qualifications - Bachelor’s degree in Business, Information Systems, Engineering, or a related field. - 5+ years of experience in data governance, master data management, data quality, or business process transformation. - Proven experience operating in a global or multi-regional environment, driving cross-functional initiatives. - Strong hands-on experience with master data domains (Customer, Vendor/Supplier, Material/Product, Finance). - Solid understanding of SAP ERP environments, including data structures and integration points. - Experience with a master data governance solution strongly preferred. Company Description Lincoln Electric is a high-performance industrial machinery and technology leader who helps customers manufacture and maintain vital equipment and infrastructure. Lincoln Electric’s innovative solutions enable higher quality and productivity across a variety of processes including welding, cutting, brazing, machining, process automation, and field repair. The Company leverages proprietary technologies and expertise in materials science, power electronics, automation, and intelligent software to help customers build better and achieve resilience in their operations. Headquartered in Cleveland, Ohio, Lincoln Electric operates 71 manufacturing and automation facilities across 20 countries and serves customers in over 160 countries.
Senior Clinical Data Engineer
ParexelParexel is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to legally protected status, which in the US includes race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or protected veteran status.
Role Description Are you a skilled data engineering professional with a passion for clinical research? Parexel is seeking a Senior Clinical Data Engineer to join our Global Data Operations team. In this pivotal role, you will leverage your technical expertise to support the conduct of clinical trials, serve as an internal subject matter expert, and lead cross-functional projects with sponsors and data management teams. As a Senior Clinical Data Engineer, you will work independently and collaboratively to manage electronic data, eClinical technologies, and systems while maintaining the highest standards of quality and regulatory compliance. You will also mentor team members and drive process improvements across the organization. Key Responsibilities - Project & Technology Management - Lead and implement Data Receipt Agreements with vendors through cross-functional collaboration - Program and configure import procedures for data ingestion using SAS, Workbench, or alternative technologies - Design and execute reconciliation checks to ensure accurate data transfer - Program offline listings and custom reports to provide actionable insights - Aggregate data across multiple sources and manage external data effectively - Handle missing values, raw data files, data structures, and SAS dataset operations - Review and analyze data outputs to provide insights to study teams and clients - Ensure first-time quality on all deliverables - Manage electronic data timelines and coordinate with project teams - Lead programming teams to successful study completion within timelines and budget - Deploy and manage data management technology (Workbench, SAS, R, etc.) - Documentation & Compliance - Maintain comprehensive study documentation in accordance with SOPs, guidelines, and work instructions - Ensure traceability and regulatory compliance across all activities - Document deviations and communicate them to project teams - Standards & Process Improvement - Participate in creating standards through tools (SAS macros, libraries, processes) - Develop and implement project-specific tools and templates - Improve efficiencies through standardized project structures and naming conventions - Support business development and bid defense meetings Qualifications - Bachelor's degree (or equivalent) in a relevant science discipline - Proficiency in at least one programming language (SQL, SAS, or R) - Strong analytical and problem-solving skills - Ability to make informed decisions in ambiguous situations - Application of Clinical Research Standards (CRS) concepts - Root cause analysis capability - Fluent written and oral English Requirements - Proficiency in at least one of the following: SQL, SAS, or R - Knowledge of SOPs, guidelines, work instructions, system life cycle methodologies, ICH-GCP, and applicable regulations (e.g., 21 CFR Part 11) - Proven practical application of regulatory and compliance frameworks - Experience working with at least two systems used to aggregate data in clinical trials (e.g., SAS, Workbench, Elluminate) - Strong background in clinical research industry or similar field



