Job Closed

This listing is no longer active.

Ness Digital Engineering logo
Ness Digital Engineering

Ness is a full lifecycle digital engineering firm offering digital advisory through scaled engineering services.

Senior Data Engineer

Data EngineerData EngineerFull TimeRemoteSeniorTeam 1,001-5,000H1B No SponsorCompany SiteLinkedIn

Location

India

Posted

67 days ago

Salary

0

Seniority

Senior

Job Description

Senior Data Engineer

Ness Digital Engineering

Role Description - Databricks Development - Develop ETL/ELT pipelines using Spark (PySpark/Scala/SQL). - Work with Delta Lake for ACID-compliant data processing. - Build and optimize bronze–silver–gold architecture. - Handle large-scale batch and streaming data. - Data Integration - Ingest data from various sources (API, SQL Server, ADLS, Kafka, Event Hub, Service Bus). - Manage Databricks Jobs, Workflows, and Clusters. - Optimize pipelines for performance and cost. - Data Modeling & Storage - Create and optimize data models for analytics. - Work with Data Lake, Delta Lake, Hive Metastore. - Implement schema evolution, time travel, merge, and CDC. - Reporting & BI - Build dashboards in Power BI / Tableau / Looker. - Write optimized SQL for reporting layers. - Create semantic models, DAX measures, and calculated fields. - Publish reports and handle incremental refresh. - Governance & Quality - Implement CI/CD for Databricks using Git. - Apply data validation, logging, monitoring, and alerting. - Follow security best practices (RBAC, Key Vault, PAT tokens). - Excellent analytical and problem-solving abilities - Understanding of HR processes, compliance requirements, and industry best practices Company Description

Related Categories

Related Job Pages

More Data Engineer Jobs

Full TimeRemoteTeam 51-200Since 2021H1B No Sponsor

• Responsible for designing, implementing, and supporting modern data platforms using Microsoft Fabric and Databricks • Collaborate with client stakeholders, consultants, and cross-functional teams to deliver data platforms that support reporting and analytics • Design and implement data platforms leveraging Microsoft Fabric and Databricks • Build and optimize data ingestion, transformation, and orchestration pipelines • Develop scalable lakehouse architectures that support analytics, reporting, and downstream data products • Partner with analytics and business teams to deliver trusted datasets and dashboards

Illinois
Motive logo

Data Engineer

Motive

Motive combines IoT hardware with AI-powered applications to connect and automate physical operations.

Data Engineer67 days ago
Full TimeRemoteTeam 1,001-5,000Since 2013H1B Sponsor

Who we are: Motive empowers the people who run physical operations with tools to make their work safer, more productive, and more profitable. For the first time ever, safety, operations and finance teams can manage their drivers, vehicles, equipment, and fleet related spend in a single system. Combined with industry leading AI, the Motive platform gives you complete visibility and control, and significantly reduces manual workloads by automating and simplifying tasks. Motive serves nearly 100,000 customers – from Fortune 500 enterprises to small businesses – across a wide range of industries, including transportation and logistics, construction, energy, field service, manufacturing, agriculture, food and beverage, retail, and the public sector. Visit gomotive.com to learn more. About the Role: As a Data Engineer on our BI team, you’ll be a key player in Motive’s growth, delivering the data infrastructure for the AI era. You’ll act as the essential link between complex data and key business domains, delivering the high-quality datasets and semantic models that drive global strategy. This is an exciting opportunity to implement cutting-edge tooling, leverage AI to enhance your workflow, and master a modern data stack in a fast-evolving environment. This is the perfect role for a "jack of all trades" data practitioner. You’ll design data models, build robust pipelines, manage DevOps and automated systems, and implement AI-driven data tooling. You’ll even get your hands dirty building dashboards and performing deep-dive analysis. If you love working full-stack and owning the entire data lifecycle, you’ll love this role. What You'll Do: - Collaborate & Strategize: Partner closely with business stakeholders to understand their challenges and design end-to-end architecture that solves complex business problems. - Build & Maintain Data Models: Design, develop, and own robust, efficient, and scalable data models in Snowflake and Iceberg using dbt and advanced SQL. - Orchestrate & Automate: Build and manage reliable data pipelines and CI/CD workflows using tools like Airflow, Python, and Terraform to ensure data is fresh, trustworthy, and infrastructure is version-controlled. - Champion Data Quality: Implement rigorous testing, documentation, and data governance practices to maintain a single source of truth. - Enable Analytics & Workflows: Act as the Product Owner and Tech Lead for your data domains, taking responsibility for the end-to-end data product delivery– from raw ingestion to data models enabling analytics and data apps in tools like Tableau and Retool. - Innovate with AI: Help us build our next-generation data infrastructure by integrating AI capabilities (like Snowflake Cortex AI) to democratize analytics and empower the business. - Architect Observability: Implement monitoring and alerting frameworks (e.g., dbt packages or Monte Carlo monitors) to proactively catch "silent" data failures before stakeholders do. What We're Looking For: - 6+ years of experience in Analytics Engineering, Data Engineering, or a similar role. - Deep expertise in SQL and developing complex data models for analytical purposes (e.g., dimensional modeling). - Hands-on experience with: - Data Warehousing: High proficiency in Snowflake (preferred) and experience with Open Table Formats like Iceberg. - Data Transformation: dbt - Orchestration & ETL: Airflow, Fivetran, Airbyte - Cloud Platform: AWS - Programming/Ingestion: Python - Infrastructure as Code: Terraform - AI-Augmented Development: Proficiency using AI coding assistants (Cursor, Copilot, or Claude) to accelerate development and automate routine tasks. - A strong analytical mindset with a proven ability to solve ambiguous business problems with data. - Excellent communication skills and experience working cross-functionally. - Self-starter with the ability to self-project manage work - A user focus with the ability to understand how a data consumer will use the data products you build Bonus Points (Nice-to-Haves) - Experience building semantic models for natural language querying - Direct experience with advanced Snowflake features (e.g., Snowpark, Cortex AI). - Experience building visualizations and dashboards in tools like Tableau, Retool, or Thoughtspot. Pay Transparency Your compensation may be based on several factors, including education, work experience, and certifications. For certain roles, total compensation may include restricted stock units. Motive offers benefits including health, pharmacy, optical and dental care benefits, paid time off, sick time off, short term and long term disability coverage, life insurance as well as 401k contribution (all benefits are subject to eligibility requirements). Learn more about our benefits by visiting Motive Perks & Benefits The base compensation range for this role is: $127,000—$175,000 USD Creating a diverse and inclusive workplace is one of Motive's core values. We are an equal opportunity employer and welcome people of different backgrounds, experiences, abilities and perspectives. Please review our Candidate Privacy Notice here. UK Candidate Privacy Notice here. The applicant must be authorized to receive and access those commodities and technologies controlled under U.S. Export Administration Regulations. It is Motive's policy to require that employees be authorized to receive access to Motive products and technology.

United States
$127K - $175K / year
Job Closed
ioet logo

Senior Data Engineer

ioet

Diversity is a technical strategy.

Data Engineer67 days ago
Full TimeRemoteTeam 51-200Since 2014H1B No Sponsor

• Design and implement scalable, production-grade ETL/ELT pipelines • Take full ownership of the technical architecture, design, and implementation of the data platform and related tooling • Apply strong non-SQL programming skills (Python, PySpark, or similar) to data processing, automation, and pipeline development • Collaborate closely with stakeholders and engineering leaders to define and execute the data platform roadmap • Act as a subject matter expert, providing technical guidance and best practices to leadership and the wider engineering organization

Latin America
Job Closed
Capgemini logo

Senior Data Engineer

Capgemini

Get the Future You Want

Data Engineer67 days ago
Full TimeRemoteTeam 10,001+Since 1967H1B Sponsor

At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world’s most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and engineering services across all industries. Join us for a career full of opportunities. Where you can make a difference. Where no two days are the same. Job Description Your Role: - Plan, manage, and execute releases across all environments - from development to production. - Identify technical and functional interdependencies between various tracks. - Coordinate with all development pods and stakeholders across programs. - Review application code, documentation, and perform pre/post deployment tasks. - Define release process for new data engineering applications. - Change management communication to all required stakeholders. - Execute release packages, step-by-step deployment guides, and command line instructions on production environment. - Support project timelines, clearly setting expectations, and realigning expectations internally as priorities change. - Develop a sound understanding of client’s needs: data conversion and migration requirements, environment management and build processes, deployment planning and execution, solution designs, systems integrations, technical architectures, and infrastructure architectures. Your Profile: - Execute deployment scripts and step-by-step instructions in test and production environments. - Comfortable with Command Line Interfaces. - Understanding of CI/CD, Git branching, packaging and DevOps pipelines. - Working knowledge of data warehousing and reporting technologies: Azure, Databricks, Unity Catalog, Python, Spark, PySpark, Airflow, MicroStrategy, Tableau. #LI-DC10 #LI-Remote Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry expertise and partner ecosystem.

Colombia