Ember logo
Ember

Global energy think tank that uses data-driven insights to shift the world to clean electricity.

Data Engineer

Data EngineerData EngineerFull TimeRemoteMid LevelTeam 51-200Since 2017H1B SponsorCompany SiteLinkedIn

Location

Worldwide

Posted

1 day ago

Salary

£36.6K - £44.9K / year

Seniority

Mid Level

Job Description

Data Engineer

Ember

Role Description Use your coding and data skills to work on the most important challenge of our time: shifting the world to clean energy. Ember publishes the best-in-class dataset on global electricity generation and the data team is growing as we expand our datasets to new areas such as installed renewable capacity, price, battery storage, grids and flexibility, EV and heat pump deployment. We are looking for someone to use their data and coding skills to ensure we can quickly gather and curate data from new sources, as well as helping to run, maintain and document existing data pipelines. This role offers an exciting opportunity to learn from a knowledgeable and passionate team of engineers and analysts while applying your development skills towards the clean energy transition. Key Responsibilities - Develop new ETL scripts using Python to gather and validate data from a variety of sources e.g. APIs, web scraping. - Work with our data engineering team to deploy ETL scripts within our orchestrated data platform based on Dagster and BigQuery. - Help run, maintain, and improve existing data pipelines. - Help ensure that our pipelines are written using best coding and data practices. - Help ensure Ember’s data and output are of the highest standard. Qualifications - At least one year experience developing and deploying Python code. - Experience working with SQL databases. - Numerate and data literate, with excellent data extraction and transformation skills. - A thoughtful and selective approach to the use of AI coding tools and the ability to critically evaluate their outputs. - Fluent in spoken and written English. - Passionate about clean energy. - Driven and keen to learn. - Systematic with careful attention to detail. - Ability to work as part of a remote international team. Requirements - Experience developing data pipelines on an orchestration platform such as Dagster (preferred), Airflow, dbt or Prefect. - Experience with version control software such as Git. - Experience working on cloud platforms, such as GCP (preferred), AWS or Azure. - Experience working with business users to turn research questions into specific data requirements, and developing to those requirements. - Other language skills. - Previous experience within the power sector or clean energy sector. Benefits - We operate a nine-day fortnight meaning our full-time staff are given every other Friday off work with no reduction in pay. - 25 days holiday, plus UK bank holidays (unless local statutory minimums are higher) and for each year that you're part of the team at Ember you'll receive an additional day of holiday, up to a maximum of 5 additional days. - Generous paid maternity and paternity leave. - Flexible working conditions, including the opportunity for part-time work and home working. - Access to a local working space can also be arranged for employees. - Free annual eye tests. - Access to a counselling service. - Funding and allocated time for your training and development. - Paid volunteer day. - Four paid days off to enable low carbon travel. - Time off to donate blood. Company Description Ember is an independent, not-for-profit energy think tank that aims to accelerate the clean energy transition with data and policy. We gather, curate and analyse data on the power sector and coal mine methane emissions, and use our findings to improve energy and climate policy. See our work at ember-energy.org and @ember_energy .

Related Categories

Related Job Pages

More Data Engineer Jobs

ProArch logo

AI/Data Solutioning Engineer

ProArch

Consulting and technology- enabled by cloud, guided by data, fueled by apps, and secured by design.

Data Engineer1 day ago
Full TimeRemoteTeam 201-500H1B Sponsor

Role Description ProArch is seeking an experienced Senior AI Engineer to lead the design and development of AI/Data solutions, services, and reusable accelerators. This role is engineering-first, with approximately 50–75% of time focused on building solutions and 25–50% supporting presales activities including discovery, architecture definition, and client interactions. The ideal candidate will combine strong hands-on engineering expertise with solution architecture capability, with a focus on AI agents and modern data platforms. - Lead development of AI/Data solutions, accelerators, and reusable service offerings - Design and implement production-grade PoCs, MVPs, and scalable frameworks - Architect end-to-end solutions across data pipelines, storage, semantic models, and AI layers - Develop AI agents using Azure AI Foundry including orchestration, tool integration, and grounding - Implement advanced AI patterns including RAG and multi-agent workflows - Ensure solutions are scalable, secure, and governed (RBAC, monitoring, evaluation) - Drive reusable solution patterns and internal IP creation - Collaborate with delivery teams to transition solutions into production - Support pre-sales activities including workshops, proposals, and demos - Communicate solution value to both technical and business stakeholders Qualifications - Strong experience in Data Engineering, AI Engineering, or Full-Stack Engineering - Expertise in Python/.Net, SQL, and distributed systems (Spark preferred) - Experience building large-scale data platforms and pipelines - Hands-on experience with Azure Data + AI ecosystem (Fabric, Synapse, Databricks) - Experience with LLMs, embeddings, vector search, and RAG architecture - Experience with agent orchestration frameworks such as Semantic Kernel or LangChain or similar - Strong solution architecture and design thinking capability - Expertise of observability practices including Application Insights/OpenTelemetry - Ability to communicate clearly in written and verbal form; comfortable presenting technical concepts to technical and non-technical audiences - Ability to translate business problems into scalable technical solutions - Strong analytical and problem-solving abilities - Stakeholder management and collaboration skills - Proactive, self-driven, and customer-focused mindset Requirements - Preferred Qualifications: - Microsoft Fabric experience (Lakehouse, OneLake concepts, semantic models) - Experience with CI/CD for data/AI workloads and basic MLOps/LLMOps practices - Understanding of identity and access management patterns (Entra ID, managed identities, service principals) - Exposure to Agent evaluation frameworks

India
Anteriad logo

Data Engineer

Anteriad

Anteriad is an advertising services company that specializes in providing full-funnel business-to-business (B2B) marketing solutions to help its clients drive g

Data Engineer1 day ago

Title: Data Engineer Location: remote Job Description: Come Join Our Team At Anteriad and innovate the way B2B marketers make data-driven business decisions. About Anteriad We are not just another B2B solution provider. We're problem solvers. We believe that data is the key to unlocking effective solutions that span a range of marketing challenges - from customer acquisition to demand generation to account-based marketing. Data is at the core of everything we do. Our team works tirelessly to create powerful solutions that drive real results for our clients. Whether it's through innovative technology or deep analysis, we're committed to finding the best path to growth for every one of our customers. Why Join Our Data Integration Team? This is an excellent opportunity for an intelligent, energetic, and self-motivated individual to play a vital role within a growing part of Anteriad. As our Data Engineer, you will have an important role in receiving, organizing, and loading data into Anteriad’s Data Warehouse to be used by our clients and throughout the Anteriad organization. As an integral part of the Data Integration team, you will play a key role in building, managing, and optimizing data pipelines that support our marketing and analytics efforts. You will be a professional with deep expertise in SQL and hands-on experience with SSIS and cloud-based data solutions, especially within the Azure ecosystem. This role may require 1-4 on-site meetings at our NYC office per year. You must be willing to attend if scheduled. This role is not eligible for sponsorship, including current OPT and/or partner VISA status. Anteriad means “always moving forward” and we apply that to our company culture by tirelessly promoting an environment that allows our employees to thrive: - Work from home - Flexible PTO - Training & development with unlimited access to Percipio LMS - Mix of collaborative & independent work - Community outreach via Anteriad Cares - encouraging staff to take time to volunteer - Professional mentoring program - career guidance from leadership - Employee Resource Groups - collaborate with others that share your passions! - Benefits We Bring to You: - Comprehensive medical (choice of 3 plans), dental and vision coverage - Company paid short-term disability, long term disability, and life insurance - Optional supplemental life, accident, and critical illness insurance plans - 401K with company match - Flexible PTO and generous holiday schedule - Fully paid primary caregiver leave (12 weeks) & parental bonding leave (2 weeks) - What You’ll Do - Partner with technical and non-technical stakeholders to translate business requirements into scalable, reliable, and maintainable data engineering solutions. - Design, develop, and maintain data integration solutions using Microsoft SQL Server, Azure Data Factory, Azure Databricks, Microsoft Fabric, and related Azure services. - Build, manage, and optimize ETL/ELT pipelines using Azure Data Factory and Microsoft Fabric Pipelines, including pipeline orchestration, parameterization, scheduling, monitoring, and error handling. - Develop and maintain data transformation logic using Azure Databricks notebooks and Microsoft Fabric notebooks, with a focus on performance, scalability, reusability, and operational reliability. - Design and implement data ingestion, transformation, validation, and delivery processes across structured, semi-structured, and external data sources. - Develop and optimize data integration processes utilizing Azure Data Factory, Azure Databricks, Azure Storage Accounts, Azure Data Lake Storage, Azure Key Vault, Microsoft Fabric Lakehouses, Warehouses, Pipelines, and Notebooks. - Build and maintain SQL-based data models, stored procedures, views, automation scripts, and database objects to support operational and analytical workloads. - Support the modernization and migration of legacy data integration processes from SSIS and on-premises platforms to cloud-based Azure and Microsoft Fabric architectures. - Design and implement reusable pipeline patterns, data quality checks, validation controls, logging, alerting, and performance monitoring processes. - Support new client onboarding efforts by designing and implementing reliable data ingestion, transformation, reconciliation, and validation workflows. - Troubleshoot and resolve data pipeline, data quality, performance, and integration issues in collaboration with teams across Intelligence & Analytics. - Design, develop, and maintain datasets, semantic models, and data structures that support reporting, analytics, and visualization solutions across Power BI, Excel, Microsoft Fabric, and other business intelligence platforms. - Contribute to data engineering standards, documentation, best practices, and reusable frameworks that improve scalability, maintainability, and operational efficiency. - Research, evaluate, and recommend emerging technologies and platform capabilities that improve data architecture, automation, performance, and reliability. - What You’ll Bring - 3+ years of experience in data engineering, data integration, business intelligence engineering, or a related technical role. - 3+ years of experience writing, optimizing, and troubleshooting SQL, including T-SQL, stored procedures, views, indexing strategies, and query performance tuning. - Hands-on experience designing, building, and supporting ETL/ELT pipelines using Azure Data Factory and/or Microsoft Fabric Pipelines. - Hands-on experience developing data processing and transformation logic using Azure Databricks notebooks, Microsoft Fabric notebooks, PySpark, Spark SQL, Python, or similar technologies. - Strong Microsoft SQL Server and relational database design principles skills. - Experience with Azure data platform services such as Azure Data Factory, Azure Databricks, Azure Storage Accounts, Azure Data Lake Storage, Azure Key Vault, and related cloud data services. - Strong knowledge and in depth work with Microsoft Fabric components such as Pipelines, Notebooks, Lakehouses, Warehouses, Dataflows, and semantic models. - Experience building reliable data ingestion and transformation processes from APIs, flat files, databases, web services, cloud storage, and other external data sources. - Hands on experience implementing data validation, reconciliation, monitoring, logging, and alerting processes for production data pipelines. - Experience supporting production data workloads, including troubleshooting failures, optimizing performance, and improving pipeline reliability. - Prior experience modernizing or migrating legacy ETL processes from SSIS, SQL Server, or on-premises environments to Azure or Microsoft Fabric is strongly preferred. - Bachelor’s degree in Computer Science, Information Technology, Data Analytics, a related field, or equivalent professional experience. - Power BI, semantic models, or other data visualization and analytics tools experience is a plus. - Experience with Customer Data Platforms, CDPs, database marketing, marketing analytics, or client data onboarding environments is a plus. - Working knowledge with AWS or Google Cloud Platform is a plus. - Strong analytical, problem-solving, and critical thinking skills. - Excellent verbal and written communication skills, with the ability to explain technical concepts to both technical and non-technical audiences. - Ability to work independently on moderately complex data engineering tasks while collaborating effectively in a team-oriented environment. - Our Values: - Lead & Learn We lead with unrivaled vision, innovation and execution, always learning and embracing new ways of doing things to stay out in front​ - Collaborate & Celebrate We build great things when we work together as one Anteriad team, celebrating our achievements – both great and small – along the way ​ - Innovate & Inspire We are always looking for bold new ways to exceed the expectations of our customers and to inspire each other to even greater success​ - Do More & Do Good We go above and beyond in the service of our clients and colleagues, and the communities where we live - Notice to California Applicants: We collect and process personal information as described in our California Applicant Privacy Notice, in compliance with the California Consumer Privacy Act (CCPA/CPRA). Please review the notice here: Anteriad Privacy Policy

United States

Title: Platforms Engineer (Data) Location: Australia Job Description: We're the airline that's always done things a little different. Our way. The Virgin way. We're on a mission to make flying better than 'nice'. And to do that, it takes a team of talented people who are passionate about customer experience, doing the right thing always and putting safety first - in the air, on the ground and in our workplaces. We're a team that values diversity, inclusion, innovation and collaboration, backed by a trusted Australian brand with big ambitions. What you'll be doing As a Data Platforms Engineer, you'll design, build and evolve scalable cloud-based data platforms that power reporting, analytics and customer insights across the business. This role is suited to an engineer who enjoys building reusable frameworks, improving developer experience and applying modern software engineering practices to data platforms. You'll work closely with platform and analytics teams to establish scalable, reliable and secure engineering patterns that accelerate enterprise data delivery. You will: - Design and develop reusable data engineering frameworks and accelerators on Databricks - Build and optimise data pipelines across large and complex datasets - Optimise data workloads for scalability, performance and cost efficiency - Drive automation to improve platform reliability, deployment consistency and operational efficiency - Implement engineering best practices across CI/CD, testing, observability and code quality - Leverage modern engineering practices and AI-assisted tools to improve delivery quality and developer productivity - Maintain technical documentation, standards and operational processes - Collaborate with product owners and engineers to deliver platform capabilities and continuously improve our data platform You'll be great in this role if you have... - Proven experience building enterprise-scale data solutions on Databricks and AWS</li> - Hands-on expertise with Databricks, Python, SQL and distributed data processing - Experience implementing CI/CD pipelines, DevOps practices and engineering automation - Strong understanding of Infrastructure-as-Code principles and cloud platform engineering - Strong understanding of version control, branching strategies and engineering governance - Experience contributing to technical design decisions and engineering standards - A track record of writing clean, maintainable and well-tested code - Experience working within Agile cross-functional teams - Strong understanding of observability, governance and operational best practices - The ability to balance strategic thinking with hands-on engineering delivery Bonus points for - A degree (or equivalent experience) in Information Technology, Computer Science, Engineering or a related discipline - Relevant Databricks or AWS certifications - Experience working with enterprise-grade data platforms and solutions - Experience building internal engineering frameworks, developer tooling or platform capabilities We're committed to looking after you, with some of the best benefits and conditions in the industry - including (but not limited to): - Discounted flights for you and your loved ones (including $1000 worth of travel credits per year) - A flexible, hybrid work style. Work from home or the office, adapt your location to suit your style and needs - One of Brisbane's most admired head office locations, with South Bank train station, a supermarket, pharmacy, and lunch options downstairs plus a 5-minute walk to the Busway, a rooftop restaurant and onsite gym (with a low membership fee) and South Bank Parklands just across the road - Discounts on travel insurance, lounge membership, car hire, accommodation, and experiences worldwide - Cheaper hospitality, retail, technology, beauty and wellness services - Free counselling and access to a wellbeing app to support your physical, mental, social and financial health - 'Dress for Your Day' - enjoy the freedom to wear whatever makes you feel great for the type of work you do and the day you have ahead of you. When everyone is included, everyone wins We're all individuals. And we love that. That's why we encourage applications from Indigenous Australians, people with disability, those from diverse cultural backgrounds, our friends in the LGBTQIA+ community, and anyone else who wants to join our family.

Australia
Full TimeRemoteTeam 5,001-10,000H1B Sponsor

• Design, develop, and maintain scalable data pipelines using Databricks and Delta Lake • Build and manage transformations across bronze, silver, and gold layers • Optimize processing for performance, reliability, and cost • Integrate data from ERP, CRM, APIs, and other enterprise systems • Develop gold-layer datasets aligned to standardized business definitions • Translate business requirements into reusable data models • Ensure consistency of core metrics across reporting • Align Databricks outputs with Power BI semantic models • Implement automated data quality checks and validation rules • Build testable, production-ready pipelines • Support impact analysis using lineage tools • Participate in CI/CD and deployment processes • Monitor and optimize pipeline performance • Troubleshoot issues across environments • Ensure data consistency between dev, test, and prod • Support high-volume data workloads • Use AI-assisted tools for development (e.g., Copilot, Databricks Agents) (Preferred) • Explore AI agents for testing, lineage analysis, and optimization (Preferred) • Partner with BI and business teams • Support governance and cataloging efforts • Document data models and pipelines

India