Senior Data Engineer

Data EngineerData EngineerFull TimeRemoteSeniorTeam 11-50

Location

Worldwide

Posted

21 hours ago

Salary

0

Seniority

Senior

Job Description

Senior Data Engineer

Blue Pearl PTY LTD

Role Description We are seeking two highly skilled Senior Data Engineers to join a fast-paced, data platform delivery team supporting a UK-based client. The successful candidates will play a key role in designing, building, and maintaining scalable data solutions on Microsoft Azure, specifically leveraging Azure Synapse and related services. This role requires experienced professionals who can operate independently, engage effectively with stakeholders, and deliver high-quality data engineering solutions in a distributed, remote environment. - Design and develop scalable data pipelines and data processing solutions on Azure - Build and optimize solutions using Azure Synapse Analytics - Integrate data from multiple sources into cloud-based data platforms - Ensure high-quality, reliable, and efficient data ingestion and transformation pipelines - Collaborate with stakeholders to understand requirements and translate them into technical solutions - Support and enhance existing data platforms and architectures - Troubleshoot and resolve data and platform-related issues - Deliver solutions aligned with best practices in data engineering and cloud architecture - Work independently while managing deliverables across the project lifecycle Qualifications - Strong hands-on experience with Microsoft Azure Data Services - Azure Synapse Analytics - Proven experience in Data Engineering (data pipelines, ETL/ELT processes) - Experience working with data integration and transformation tools - Data warehousing concepts and architectures - Exposure to cloud-based data platforms and modern data ecosystems Requirements - Ability to work independently and drive delivery without close supervision - Strong problem-solving and analytical thinking - Experience working in remote/distributed teams - Understanding of basic Business Intelligence concepts and workflows - Excellent communication skills (verbal and written English) - Ability to engage confidently with international (UK-based) stakeholders - Strong collaboration and stakeholder management skills - Self-motivated and delivery-focused mindset Benefits - Experience with additional Azure services (Data Factory, Databricks, etc.) - Exposure to large-scale or enterprise data platform projects - Experience supporting UK or international clients

Related Categories

Related Job Pages

More Data Engineer Jobs

Role Description Lead client SMEs through complex schema mapping and source-to-target validation, including automated reconciliation scripts across sandboxes and production cutovers. - Expert in Automation Bypass strategies and Identity Resolution rules to prevent duplicate records and performance degradation during high-volume loads. - Architect Zero Copy Federation via Salesforce Data Cloud (Data 360), mapping Databricks DLOs to Salesforce DMOs for 10–25 year historical datasets. - Lead "Receiving Pipes" development using Bulk API 2.0 and scripting (Python, Node, or SFDX) to ingest 10 years of active and historical records. - Perform complex object mapping from flat legacy tables into normalized Salesforce objects (Accounts, Cases, custom Disaster objects). - Define a hybrid schema bridging physically migrated (10-year) and Zero Copy historical (10–25 year) data. - Configure Match and Reconsolidation rules to link survivor records across 15+ years of Databricks history to active Salesforce cases. - Build Parent-to-Child load sequencing (e.g., Disaster before Case). - Manage Bulk API 2.0 concurrency limits and batch failures to ensure efficient, reliable dry runs and cutovers. Qualifications - Technical expertise in schema mapping and validation. - Experience with Salesforce Data Cloud and Databricks. - Proficiency in scripting languages such as Python, Node, or SFDX. - Strong understanding of data migration and object mapping. Requirements - Experience with Bulk API 2.0 and data ingestion processes. - Ability to configure Match and Reconsolidation rules. - Knowledge of managing data concurrency and batch processing. Benefits - Competitive salary. - Health and wellness programs. - Opportunities for professional development.

United States
Truist logo

Data Platform Engineer II

Truist

A purpose-driven financial services company, Truist Financial Corporation was established to deliver better banking for all people, businesses, and communities.

Data Engineer1 day ago

Title: Data Platform Engineer II Location: Raleigh, North Carolina, USA Job Description: The position is described below. If you want to apply, click the Apply Now button at the top or bottom of this page. After you click Apply Now and complete your application, you'll be invited to create a profile, which will let you see your application status and any communications. If you already have a profile with us, you can log in to check status. Need Help? If you have a disability and need assistance with the application, you can request a reasonable accommodation. Send an email to Accessibility (accommodation requests only; other inquiries won't receive a response). Regular or Temporary: Regular Language Fluency: English (Required) Work Shift: 1st shift (United States of America) Please review the following job description: - Responsible for developing, building, and maintaining data or BI or content management, document composition platforms on on-premise infrastructure and cloud (SaaS) instance, where appropriate. Designs and implements fault tolerant systems (High Availability, Disaster Recovery) and creating a cloud-native enterprise data platform that drives near real-time decision-making. - Defines and operationalizes Data or BI or Content Management, Document composition platforms platform and enables relevant use cases. - Develops, maintains, and improves deployment pipelines for the platforms. Maintains and ensures operational metadata (logs, performance metrics, etc.) are available for diagnostics. - Provides solution design for these platforms. Creates automated solutions and frameworks to deliver efficiency. - Implement effective technical solutions by providing problem analysis and resolution in a timely manner and explain and interpret complex, difficult, or sensitive information. - Build non-functional monitoring capabilities and provide escalated support for highly complex applications in production. - Build in and maintain security controls and monitoring in support of company standards. - Provide solutions to solve problems of moderate to high complexity and exercise judgment based on the analysis (e.g. modeling, testing, etc.) of multiple sources of information. - Lead small data engineering projects with manageable risks and resource requirements; plays significant roles in larger, more complex initiatives. Requirements - Must have a Bachelor's degree in Computer Science, MIS, Business, or a technical field. - Must have 4 years of experience in technical development or analyst positions performing the following: - Planning and managing IT projects - Providing direction and mentoring less experienced teammates - Interpreting and conveying complex, difficult, or sensitive information - Administration of IBM Content Manager on Demand (CMoD). - Data acquisition, transformation, and storage design using design principles, patterns, and best practices. - Working within a team environment and interacting with data professionals and business data SME's throughout the organization. - Utilizing experience with: Programming/Scripting languages including: COBOL, JCL, PL/I, REXX, SAS, Easytrieve, Bash, and UNIX; Mainframe; Linux; Storage Systems; SQL; Relational databases; ETL/ELT architecture; Data integration concepts; Big data concept; and Data, BI, content, and document composition platforms. - Position may be eligible to work hybrid/remotely but is based out of and reports to Truist offices in Raleigh, NC. Must be available to travel to Raleigh, NC regularly for meetings and reviews with manager and project teams within 24-hours' notice.. General Description of Available Benefits for Eligible Employees of Truist Financial Corporation: All regular teammates (not temporary or contingent workers) working 20 hours or more per week are eligible for benefits, though eligibility for specific benefits may be determined by the division of Truist offering the position. Truist offers medical, dental, vision, life insurance, disability, accidental death and dismemberment, tax-preferred savings accounts, and a 401k plan to teammates. Teammates also receive no less than 10 days of vacation (prorated based on date of hire and by full-time or part-time status) during their first year of employment, along with 10 sick days (also prorated), and paid holidays. For more details on Truist's generous benefit plans, please visit our Benefits site. Depending on the position and division, this job may also be eligible for Truist's defined benefit pension plan, restricted stock units, and/or a deferred compensation plan. As you advance through the hiring process, you will also learn more about the specific benefits available for any non-temporary position for which you apply, based on full-time or part-time status, position, and division of work. Truist is an Equal Opportunity Employer that does not discriminate on the basis of race, gender, color, religion, citizenship or national origin, age, sexual orientation, gender identity, disability, veteran status, or other classification protected by law. Truist is a Drug Free Workplace. EEO is the Law E-Verify IER Right to Work

North Carolina
ICF logo

Data Engineer

ICF

Founded in 1969, ICF is a global advisory and technology services company headquartered in Reston, Virginia. It delivers data-driven solutions across energy, environment, infrastru

Data Engineer2 days ago

Title: Data Engineer (Clearance Required) Location: Reston VA United States Full-time Job Description: The Work: We are seeking a Data Engineer to support federal government programs by designing, building, and maintaining scalable, secure, and compliant data solutions. This role partners closely with data scientists, analysts, developers, and government stakeholders to enable data-driven decision-making across mission‑critical systems. Job Location: This is a fully remote role which may require occasional travel to the DC Metro area as needed for onsite client meetings. Preference to candidates located within the Washington, DC Metro area. If you accept this position, you should note that ICF does monitor employee work locations and blocks access from foreign locations/foreign IP addresses, and also prohibits personal VPN connections. What You Will Do: - Design, build, and maintain data pipelines and ETL/ELT processes supporting large-scale federal data systems - Develop and optimize cloud-based data architectures in AWS or Azure environments (e.g., S3, Redshift, RDS, Databricks, Synapse) - Integrate structured and unstructured data from multiple government and third‑party sources - Ensure data quality, integrity, availability, and performance across systems - Support ATO, RMF, and federal security requirements, including documentation and audit support - Collaborate with DevOps teams on CI/CD, infrastructure as code (IaC), and deployment automation - Translate business and mission requirements into scalable technical solutions - Produce and maintain technical documentation for data flows, pipelines, and architectures - Participate in Agile ceremonies and work closely with cross‑functional teams What You Will Bring With You: - Active Top Secret clearance, with SCI eligibility - 2+ years of experience designing and supporting enterprise data solutions in federal or regulated environments What We Would Like You To Bring With You: - Bachelor's Degree in related field - Prior experience supporting federal or DoD programs, preferably DCSA - Strong experience with relational databases (e.g., Oracle, PostgreSQL, SQL Server, Azure SQL, or similar) - Knowledge of data modeling techniques, ETL/ELT processes, and data integration patterns - Familiarity with cloud platforms (Azure, AWS, or GCP) and modern data services - Experience supporting security, governance, and compliance requirements - Experience with data warehousing, reporting, and analytics platforms - Familiarity with DevSecOps, CI/CD pipelines, and infrastructure‑as‑code Working at ICF ICF is a global advisory and technology services provider, but we're not your typical consultants. We combine unmatched expertise with cutting-edge technology to help clients solve their most complex challenges, navigate change, and shape the future. We can only solve the world's toughest challenges by building a workplace that allows everyone to thrive. We are an equal opportunity employer. Together, our employees are empowered to share their expertise and collaborate with others to achieve personal and professional goals. For more information, please read our EEO policy. We will consider for employment qualified applicants with arrest and conviction records. Reasonable Accommodations are available, including, but not limited to, for disabled veterans, individuals with disabilities, and individuals with sincerely held religious beliefs, in all phases of the application and employment process. To request an accommodation, please email and we will be happy to assist. All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodations. Read more about workplace discrimination rights or our benefit offerings which are included in the Transparency in (Benefits) Coverage Act. Candidate AI Usage Policy At ICF, we are committed to ensuring a fair interview process for all candidates based on their own skills and knowledge. As part of this commitment, the use of artificial intelligence (AI) tools to generate or assist with responses during interviews (whether in-person or virtual) is not permitted. This policy is in place to maintain the integrity and authenticity of the interview process. However, we understand that some candidates may require accommodation that involves the use of AI. If such an accommodation is needed, candidates are instructed to contact us in advance. We are dedicated to providing the necessary support to ensure that all candidates have an equal opportunity to succeed. Pay Range - There are multiple factors that are considered in determining final pay for a position, including, but not limited to, relevant work experience, skills, certifications and competencies that align to the specified role, geographic location, education and certifications as well as contract provisions regarding labor categories that are specific to the position. The pay range for this position based on full-time employment is: $81,499.00 - $138,549.00 Nationwide Remote Office (US99)

District Of Columbia
$81.5K - $138.5K / year
Multitude Insights logo

Principal Data Engineer

Multitude Insights

BLTN | Real-time intel platform built for law enforcement

Data Engineer2 days ago
Full TimeRemoteTeam 11-50Since 2023H1B No Sponsor

• Design & Build: Architect and maintain the company’s core data infrastructure, including pipelines, warehousing, ingestion, transformation, and orchestration systems. • Own the Data Platform: Take end-to-end responsibility for the scalability, reliability, and security of the data platform ensuring high-quality data is accessible across the organization. • Set the Standard: Establish and own data engineering best practices, including data modeling, pipeline design, and observability patterns that the broader team can build on. • Champion Data Quality: Drive data quality and observability initiatives across the platform, ensuring stakeholders can trust the data they rely on. • Collaborate Cross-Functionally: Partner closely with engineering, product, and analytics teams to understand data needs and translate them into concrete technical solutions. • Lead & Mentor: Mentor engineers on data engineering principles and patterns, and contribute actively to hiring and team development.

United States