Resilinc logo
Resilinc

Strengthening the supply chains that power our lives.

Senior Data Engineer

Data EngineerData EngineerFull TimeRemoteSeniorTeam 201-500Since 2010H1B SponsorCompany SiteLinkedIn

Location

India

Posted

4 days ago

Salary

0

Seniority

Senior

Bachelor Degree5 yrs expEnglishApacheAzureCloudETLPythonSparkSQL

Job Description

Senior Data Engineer

Resilinc

• Partner directly with enterprise customers and business stakeholders to understand data challenges, define requirements, and deliver scalable solutions. • Own technical delivery for customer engagements from discovery and solution design through implementation and adoption. • Help customers maximize the value of Resilinc’s data and analytics capabilities through effective solution design and execution. • Present technical architectures, implementation plans, and business outcomes to both technical and executive audiences. • Design, build, and maintain scalable batch and real-time data pipelines and data products using Databricks, ClickHouse, Python, and distributed data processing frameworks. • Architect modern cloud-native data platforms leveraging Databricks, Delta Lake, Azure Data Services, and distributed processing technologies. • Develop robust ETL/ELT frameworks, data models, and data quality monitoring processes. • Optimize data storage, query performance, scalability, reliability, and cost efficiency across large-scale datasets. • Build and maintain data products that enable analytics, reporting, operational workflows, and AI/ML applications. • Collaborate closely with Product, Engineering, Solutions Consulting, and Customer Success teams to deliver customer outcomes. • Drive best practices in data governance, security, observability, reliability, and operational excellence. • Contribute to architecture decisions, technical design reviews, and platform evolution. • Mentor junior engineers and contribute to engineering best practices.

Job Requirements

  • BE/MS in Computer Science, Information Technology, Engineering, or a related field.
  • 5+ years of experience in data engineering, data platform development, or related disciplines.
  • Strong experience with Databricks, Apache Spark, Delta Lake, or similar distributed data processing technologies.
  • Experience with cloud-based data platforms and services, including Azure (preferred) or equivalent cloud ecosystems.
  • Experience designing and operating analytical data platforms using ClickHouse or similar high-performance analytical databases.
  • Advanced proficiency in Python and SQL, with strong software engineering fundamentals.
  • Experience building scalable ETL/ELT pipelines, orchestration workflows, and modern data architectures.
  • Strong understanding of data modeling, data warehousing, and analytics platform design.
  • Experience implementing CI/CD, infrastructure automation, and DevOps practices for data platforms.
  • Excellent communication and stakeholder management skills, with the ability to translate business requirements into technical solutions.
  • Ability to navigate ambiguity, drive outcomes independently, and succeed in customer-facing environments.

Benefits

  • Full-stack benefits for health, wealth and wellbeing

Related Categories

Related Job Pages

More Data Engineer Jobs

Ciklum logo

Senior Data Engineer

Ciklum

Experiences of Tomorrow, Engineered Together!

Data Engineer4 days ago
Full TimeRemoteTeam 1,001-5,000Since 2002H1B No Sponsor

Role Description Ciklum is looking for a Senior Data Engineer to join our team full-time in Pakistan. As a Senior Data Engineer, become a part of a cross-functional development team engineering experiences of tomorrow. Ciklum is developing the web solution - TAX platform built to enhance and unify the process of providing Tax and Financial Services. The team is developing an innovative automation platform for our client, a globally-operating, complex system that enhances and unifies the process of collecting, processing, and providing Tax and Financial services. The client's employees and external users actively use the platform daily, and the team continues to add new features that automate business flows. Responsibilities - Designing, developing, and deploying Power BI solutions, providing advanced data analytics, and creating dashboards and reports that deliver key insights to the business. - Data models design and implementation. - Optimize reports and dashboards for better performance by improving query speed and ensuring efficient data model design. - Contributes design, code, configurations, manage data ingestion, closed to real-time processing, batch processing, ETL across multiple data storages. - Responsible for performance tuning of complicated SQL and DAX queries. - Identify gaps and improve the platform’s quality, robustness, maintainability, and speed. - Cross-train other team members on technologies being developed, while also continuously learning new technologies from other team members. - Contribute in the Unit’s activities and community building, participate in conferences, and provide excellence in exercise and best practices. Qualifications - 3+ years of experience in developing BI solutions with Power BI. - Proficient in Power BI Desktop and Power BI Service, including report and dashboard creation, data transformations, and DAX formulas. - Strong expertise in DAX (Data Analysis Expressions) for complex calculations. - Experience with Power Query for data transformations and ETL processes. - Solid understanding of data modeling principles and best practices. - 3+ years of experience coding in SQL (Azure SQL DB or MSSQL preferable), solid CS fundamentals including data structure and algorithm design. - 2+ years contributing to production deployments of large backend data processing and analysis systems. - Experience in data integration and ETL processes. - Proficient with Azure Data Services (Azure Data Factory, Azure Synapse Analytics, etc.). - Strong problem-solving and analytical skills, with a keen attention to detail. - Excellent communication skills, both written and verbal, with the ability to present complex data in a clear and concise manner. - Experience in working with cross-functional teams and collaborating with stakeholders at various levels of the organization. - Knowledge of professional software engineering best practices for the full software development life cycle. - Knowledge of Data Warehousing, design, implementation, and optimization. - Knowledge of Data Quality testing, automation, and results visualization. - Knowledge of the development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations. - Experience participating in an Agile software development team, e.g. SCRUM. - Experience designing, documenting, and defending designs for key components in large distributed computing systems. - A consistent track record of delivering exceptionally high-quality software on large, complex, cross-functional projects. - Demonstrated ability to learn new technologies quickly and independently. - Ability to handle multiple competing priorities in a fast-paced environment. Requirements - Desirable: Experience with Power BI Embedded. - Experience with Power BI REST or JS APIs. - Power BI Premium Capacity Management. - Experience in backend development and deployment. - Experience in CI/CD configuration. - Experience with Azure DevOps for CI/CD. - Experience with Tabular Editor, DAX Studio, SQL Profiler. - Experience with Powershell. - Experience with Swagger, Postman. Benefits - Care: your mental and physical health is our priority. We offer a competitive benefits package that includes comprehensive medical coverage, life insurance, gym membership, fuel, internet and mobile allowances, and provident fund. - Tailored education path: boost your skills and knowledge with our regular internal events (meetups, conferences, workshops), Udemy license, language courses, and company-paid certifications. - Growth environment: share your experience and level up your expertise with a community of skilled professionals, locally and globally. - Flexibility: Own your schedule – you decide when to start your working day. - Opportunities: we value our specialists and always find the best options for them. Our Internal Mobility Program helps change a project if needed to help you grow, excel professionally, and fulfill your potential. - Global impact: work on large-scale projects that redefine industries with international and fast-growing clients. - Welcoming environment: feel empowered with a friendly team, open-door policy, informal atmosphere within the company, and regular team-building events.

Pakistan
Full TimeRemoteTeam 201-500H1B No Sponsor

• Define and execute a multi-year AI & Data platforms strategy and roadmap aligned with EP’s business priorities, operating model, and regulatory obligations. • Establish and run adaptable, effective AI and data governance: standards, patterns, intake/prioritization, exception management, and measurable value tracking. • Partner with Product and Business to translate platform investments into business outcomes. • Own the enterprise data platform operating model: architecture, engineering standards, scalability, reliability, and cost management. • Lead the evolution from “pipelines and reporting” to data products. • Serve as the primary technology counterpart to Salesforce for agentic + data integration. • Build EP’s “AI platform” capabilities that make AI usable at scale. • Partner with Cyber/InfoSec and Legal/Compliance to implement data protection and AI guardrails. • Establish repeatable operational practices for AI solutions. • Lead and mentor a high-performing AI & data organization.

United States
myPOS logo

Senior Data Engineer – Contractor

myPOS

By combining the latest payment technologies with imagination and expertise, we create a new world of payments.

Data Engineer4 days ago
ContractRemoteTeam 201-500H1B No Sponsor

• Design, build, and operate scalable, reliable data pipelines and data infrastructure • Ensure high-quality data is accessible, trusted, and ready for analytics and data science • Build and maintain data pipelines for ingestion, transformation, and export across multiple sources and destinations • Develop and evolve scalable data architecture to meet business and performance requirements • Partner with analysts and data scientists to deliver curated, analysis-ready datasets and enable self-service analytics • Implement best practices for data quality, testing, monitoring, lineage, and reliability • Optimize workflows for performance, cost, and scalability (e.g., tuning Spark jobs, query optimization, partitioning strategies) • Ensure secure data handling and compliance with relevant data protection standards and internal policies • Contribute to documentation, standards, and continuous improvement of the data platform and engineering processes • Ensure secure, compliant handling of data and models, including access controls, auditability, and governance practices • Build and maintain MLOps automation: CI/CD for ML, environment management, artifact handling, versioning of data/models/code

Serbia
Sportradar logo

Senior Data Engineer

Sportradar

Sportradar, the leading global sports technology company creating immersive experiences for sports fans and bettors.

Data Engineer4 days ago
Full TimeRemoteTeam 1,001-5,000Since 2001H1B Sponsor

Role Description THE CHALLENGE - Design, build, and implement generalized large-scale, sophisticated data pipelines using Nifi for downstream analytics and data science for our Sport Performance products. - Design and develop scalable Nifi ingestion pipelines within AWS cloud services to consume real-time and batch data from external sources. - Ensure the seamless integration of AWS-based tools for data storage, processing, and analytics. - Responsible for ETL development and warehousing using Python and Java. - Create data pipeline triggers and filters within ETL (extract, transform, and load) process to ensure appropriate optimization of data flowing through the system and resource usage. - Implement monitoring and error handling for all new parts of the data pipeline to ensure observability and alerting is available. - Establish rigorous unit testing across the data pipeline to ensure robustness of the system. - Design and create data models for use throughout the ETL system. - Utilize Kafka to efficiently and effectively store data to move throughout the data pipeline and for downstream data science and analytics usage. - Design data architecture and data models for both internal and external representations of data. - Build the data transforms within the data pipeline to convert data from external to internal representations. - Conduct data analytics and debugging of bad data by writing SQL queries. - Build automated cleaning of data to remove bad or unusable data from downstream consumers with logging to understand the frequency and depth of the underlying issues. - Collaborate with other engineering teams to adopt standard methodologies, drive scalability, and increase consistency across systems. - Maintain awareness of company standards and technology guidance; use JIRA, an Agile project management tool, to ensure efficient data development; collaborate with peers to align projects with overall direction. - Follow best practices across Data Engineering to ensure scalable, consistent data architecture and system. - Utilize Java language to build data processor in Nifi framework. - Utilize Docker to ensure consistent, repeatable, and isolated environment for software development and testing. - Work in a self-driven, independent fashion to meet Sport driven deadlines. - Position permits telecommuting from anywhere in the U.S. Qualifications - Master’s degree in Computer Science, Computer Engineering, or closely related field. - 1 year experience as a data engineer or related occupation. - 1 year of experience with: - Python - Java - Kafka - AWS - Docker - ETL Development and Warehousing - Analytic and debugging using SQL - Agile development environment - Designing data architecture Benefits - A collaborative environment with colleagues from all over the world (Offices in Europe, Asia and US) including various social events and teambuilding. - Flexibility to manage your workday and tasks with autonomy. - A balance of structure and autonomy to tackle your daily tasks. - Vibrant and inclusive community, including Women in Tech and Pride groups which welcome all participants. - Global Employee Assistance Programme. - Calm and Reulay app (leading well-being apps designed to support focus, quality rest, mindfulness, and long-term mental resilience). - Online training videos. - Flexible working hours.

United States