• Extracting, cleansing, and loading data. • Building data pipelines using SQL, Kafka, and other technologies. • Investigate and troubleshoot issues with data and data pipelines. • Investigation and documentation of new data sets • Implement new data transformation configurations or ad-hoc reports • Triage incoming bugs and incidents. • Participation in sprint refinement, planning, and kick-off to help estimate stories, raise awareness and define additional implementation details. • Monitor data pipeline execution and performance. Resolve discovered issues collaboratively. • Perform and implement data quality checks to maintain consistent and accurate data.

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards. WHY JOIN US If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you! ABOUT THE ROLE WHAT YOU WILL DO - Build and maintain scalable, distributed, fault-tolerant data pipelines on GCP; - Develop and manage lakehouse layers and Delta Lake workflows using BigQuery and Dataproc; - Collaborate with stakeholders across data engineering, compliance, and business teams; - Design and implement pipelines to acquire, normalise, transform, and release large volumes of financial data; - Design and implement bitemporal data models on BigQuery for regulatory-grade time-series datasets; - Build and maintain testing frameworks for data pipelines and transformation logic; - Own end-to-end solutions including ingestion pipelines, QA workflows, correction management, and audit trails; - Contribute to shared platform services in a collaborative environment; - Support implementation of AI solutions including data ingestion, anomaly detection, and semantic search using Vertex AI. MUST HAVES - 6–8 years of experience in data engineering; - Proficiency in Python for data pipelines, transformation logic, and automation; - Proficiency in SQL with hands-on experience in BigQuery including partitioning, clustering, and time-series queries; - Experience with Cloud Composer (Apache Airflow) for pipeline orchestration; - Working knowledge of Dataproc (Apache Spark) for batch ingestion and incremental processing; - Experience with AI-assisted development tools such as GitHub Copilot or similar; - Experience with Git version control and collaboration workflows; - Familiarity with REST APIs for integrations; - Familiarity with GCP technologies (Cloud Storage, Pub/Sub, Datastream, Cloud Monitoring, IAM, VPC Service Controls); - Understanding of financial data concepts related to equities and& other asset classes; - Upper-intermediate English level. NICE TO HAVES - Knowledge of data libraries such as pandas or PySpark; - Experience with columnar storage and time-series analytics tools such as ClickHouse; - Familiarity with Dataplex for data governance and lineage; - Understanding of Change Data Capture (CDC) using Datastream; - Understanding of bitemporal data modeling concepts; - Knowledge of financial reference data such as equities, fixed income, or corporate actions; - Experience with BigQuery cost management techniques; - Experience with CI/CD pipelines and Terraform for infrastructure as code; - Exposure to LLMs and& Agentic AI using Vertex AI for data-related use cases. PERKS AND BENEFITS - Remote work & Local connection: Work where you feel most productive and connect with your team in periodic meet-ups to strengthen your network and connect with other top experts. - Legal presence in India: We ensure full local compliance with a structured, secure work environment tailored to Indian regulations. - Competitive Compensation in INR: Fair compensation in INR with dedicated budgets for your personal growth, education, and wellness. - Innovative Projects: Leverage the latest tech and create cutting-edge solutions for world-recognized clients and the hottest startups.

At IRIUM we want you to always chase your dreams. Here, prepare yourself to conquer your goals, while enjoying the journey. We are currently looking for. Data Engineer Requirements: WHAT WE'RE LOOKING FOR Expert in Apache Kafka: configuring, managing brokers, partitioning, replication factors, and performance tuning. Advanced proficiency in Kafka Streams and Kafka Connect to build real-time pipelines, integrating systems from various sources and destinations. Strong understanding of the Confluent ecosystem, including Schema Registry, KSQL, and monitoring tools to ensure robust operations. Expertise in event-driven architectures and patterns such as publish-subscribe for distributed systems. Skilled in developing and managing APIs and microservices to integrate applications with data platforms. Proficient in programming languages like Java, Scala, and Python for real-time data stream processing. Hands-on experience with complementary technologies like Spark Streaming, Flink, or other real-time processing tools. Knowledgeable in relational and NoSQL databases (e.g., PostgreSQL, Cassandra, MongoDB). Practical experience with cloud environments (AWS, Azure, GCP) focusing on big data solutions. Familiarity with CI/CD tools (e.g., Git, Jenkins, Docker, Kubernetes) for automation and continuous delivery. Technical leadership skills to coordinate and mentor data engineering teams. Strong analytical abilities to solve complex problems and identify bottlenecks in data processing systems. Excellent communication skills to translate business requirements into technical solutions. Proactive in exploring new technologies and crafting strategic approaches to optimize processes and architectures. Designed and implemented high-performance streaming architectures using Kafka and Confluent for enterprise-scale projects. Built real-time data pipelines to process large volumes, integrating multiple enterprise systems. Applied best practices for data governance, quality control, and compliance (e.g., GDPR). Led initiatives to migrate monolithic systems to Kafka Streams-based architectures. Certifications (plus): Confluent Certified Developer for Apache Kafka. Cloud certifications (e.g., AWS Certified Solutions Architect, Google Cloud Professional Data Engineer). Additional certifications in Big Data tools or related technologies. Location: remote, Portugal What do we offer? ➡ An innovative and growing company, with a lot of opportunities for professional development. ➡ Retribution according to your experience and performance. Access to flexible pay and medical insurance as a social benefit. ➡ Unlimited access to technological training in free mode. IRIUM is a company with dynamic and proactive professionals. Our values are responsibility and commitment to work quality. This is the spirit we are looking for at IRIUM, whatever your age is. If you recognize yourself in this, this is your company! We can build the future together. Let’s talk! Send your CV to: recrutamento@irium.pt At IRIUM we defend a world without stereotypes or limitations and we believe in equality for all, principles that we subscribe to in our Equality Plan and Code of Ethics, guaranteeing equal treatment and opportunities regardless of any personal, physical or social condition.

About Us LexisNexis Legal & Professional, which serves customers in more than 150 countries with 11,300 employees worldwide, is part of RELX, a global provider of information-based analytics and decision tools for professional and business customers. About the Role We are seeking a highly skilled Senior Data Engineer with expertise in Retrieval-Augmented Generation (RAG) and strong Python programming skills to join our innovative team. The ideal candidate will have a deep understanding of AI and machine learning principles, experience with RAG models, and a passion for developing cutting-edge AI solutions. Responsibilities: - Test, evaluate and implement AI models with a focus on Retrieval-Augmented Generation (RAG). - Strong prompt engineering and prompt testing skills to derive the desired outcome - Collaborate with cross-functional teams to integrate AI solutions into existing systems. - Optimize and fine-tune RAG models for performance and scalability cost saving. - Conduct research and stay up-to-date with the latest advancements in AI and machine learning. - Write clean, efficient, and maintainable code in Python. - Perform data preprocessing, feature engineering, and model evaluation. - Troubleshoot and debug AI models and applications in a mono-repo settings. - Document AI models, processes, and workflows. - Train entry-level data engineers as directed by department management, ensuring they are knowledgeable in critical aspects of their roles. - Keep abreast of new technology developments. - Design and works with complex data models. - Identify opportunities to apply automation or other tools to improve effectiveness or efficiency. - Work closely with other development team members to understand complex product requirements and translate them into data engineering and/or data management designs. - Mentor less senior data engineers on methodologies and optimization techniques. - All other duties as assigned Requirements - Proven experience as a Data Engineer/ AI Engineer or similar role. - Strong proficiency in Python programming. - In-depth knowledge of Retrieval-Augmented Generation (RAG) models. - Proven experience in prompt engineering - Experience with machine learning frameworks and libraries (e.g., TensorFlow, PyTorch). - Proficiency in data preprocessing and feature engineering techniques. - Excellent problem-solving skills and attention to detail. - Strong communication and teamwork abilities. Preferred Skills: - Experience with cloud platforms (e.g., AWS, Azure, Google Cloud). - Knowledge of natural language processing (NLP) techniques. - Familiarity with version control systems (e.g., Git). - Experience with deploying AI models in production environments. Work in a way that works for you We promote a healthy work/life balance across the organization. We offer an appealing working prospect for our people. With numerous well-being initiatives, shared parental leave, study assistance and sabbaticals, we will help you meet your immediate responsibilities and your long-term goals. Working for you We know that your wellbeing and happiness are key to a long and successful career. These are some of the benefits we are delighted to offer: - Health Benefits: Comprehensive, multi-carrier program for medical, dental and vision benefits - Retirement Benefits: 401(k) with match and an Employee Share Purchase Plan - Wellbeing: Wellness platform with incentives, Headspace app subscription, Employee Assistance and Time-off Programs - Short-and-Long Term Disability, Life and Accidental Death Insurance, Critical Illness, and Hospital Indemnity - Family Benefits, including bonding and family care leaves, adoption and surrogacy benefits - Health Savings, Health Care, Dependent Care and Commuter Spending Accounts - Up to two days of paid leave each to participate in Employee Resource Groups and to volunteer with your charity of choice About the Business LexisNexis Legal & Professional® provides legal, regulatory, and business information and analytics that help customers increase their productivity, improve decision-making, achieve better outcomes, and advance the rule of law around the world. As a digital pioneer, the company was the first to bring legal and business information online with its Lexis® and Nexis® services. U.S. National Base Pay Range: $78,800 - $131,300. Geographic differentials may apply in some locations to better reflect local market rates. This job is eligible for an annual incentive bonus. We know your well-being and happiness are key to a long and successful career. We are delighted to offer country specific benefits. Click here to access benefits specific to your location. We are committed to providing a fair and accessible hiring process. If you have a disability or other need that requires accommodation or adjustment, please let us know by completing our Applicant Request Support Form or please contact 1-855-833-5120. Criminals may pose as recruiters asking for money or personal information. We never request money or banking details from job applicants. Learn more about spotting and avoiding scams here. Please read our Candidate Privacy Policy. We are an equal opportunity employer: qualified applicants are considered for and treated during employment without regard to race, color, creed, religion, sex, national origin, citizenship status, disability status, protected veteran status, age, marital status, sexual orientation, gender identity, genetic information, or any other characteristic protected by law. USA Job Seekers: EEO Know Your Rights.