The smartest solution for fresh
Senior Data Engineer
Location
Alabama + 20 moreAll locations: Alabama | California | Colorado | Florida | Illinois | Kentucky | Montana | Nevada | New Jersey | New York | North Carolina | Oregon | Massachusetts | Michigan | Missouri | Pennsylvania | Texas | Utah | Virginia | Washington | Wisconsin
Posted
68 days ago
Salary
$156K - $211K / year
Seniority
Senior
Job Description
Senior Data Engineer
Afresh
• Build tools and frameworks that streamline customer integrations, enabling faster onboarding and better handling of customer data. • Create robust ETLs in PySpark and DBT to process billions of records from customer datasets, ensuring data is accurate, reliable, and ready for downstream use. • Investigate and implement new technologies into the data platform, focusing on practical solutions that address current pain points and anticipate future needs. • Collaborate with product, engineering, and go-to-market teams to design and deliver data solutions for new products and features. • Identify and implement optimizations to improve ETL runtime and data processing scalability, reducing the time and effort required for integrations. • Solve real-world data quality challenges by working directly with messy, incomplete, or inconsistent customer data to extract the signal we need. • Support team members by mentoring engineers, leading technical discussions, and providing clear, actionable feedback.
Job Requirements
- Significant experience designing and maintaining ETLs that process large-scale datasets.
- Proficiency with Python, PySpark, SQL, and experience working on platforms/tools like Databricks, Snowflake, or DBT.
- Strong problem-solving skills and the ability to work with ambiguous or incomplete requirements to deliver concrete, impactful solutions.
- A focus on practical outcomes—you're skilled at balancing technical rigor with the need to get things done.
- Experience working directly with complex, unclean datasets and finding innovative ways to process and analyze them.
- A knack for identifying areas where tooling or automation can simplify workflows and reduce manual effort.
- Excellent communication skills—you’re able to explain your ideas clearly to both technical and non-technical audiences.
- Proven leadership in technical projects, with a willingness to mentor and help others grow.
Benefits
- Join a mission-driven company reducing millions of pounds of food waste in grocery stores per year.
- Work on challenging, real-world problems that have a direct impact on our customers.
- Be part of a collaborative, supportive team where your ideas are valued and acted on.
- Use cutting-edge tools and platforms to solve meaningful data challenges.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Design, implement, and maintain robust and scalable data pipelines using AWS, Azure, and containerization technologies. • Develop and maintain ETL/ELT processes to extract, transform, and load data from various sources into data warehouses and data lakes. • Collaborate with data scientists, analysts, and other engineers to ensure seamless data flow and availability across the organization. • Optimize data storage and retrieval performance by utilizing cloud services like AWS Redshift, Azure Synapse, or other relevant technologies. • Work with containerization tools like Docker and Kubernetes to ensure smooth deployment, scalability, and management of data pipelines. • Monitor, troubleshoot, and optimize data processing pipelines for performance, reliability, and cost-efficiency. • Automate manual data processing tasks and improve data quality by implementing data validation and monitoring systems. • Implement and maintain CI/CD pipelines for data workflow automation and deployment. • Ensure compliance with data governance, security, and privacy regulations across all data systems. • Participate in code reviews and ensure the use of best practices and documentation for data engineering solutions. • Stay up-to-date with the latest data engineering trends, cloud services, and technologies to continuously improve system performance and capabilities.
Principal Data Engineer – Team Lead
Dev.ProSoftware Development Partner. Result-driven. Quality-obsessed.
• Architect our data ecosystem, ensuring the platform powers frontend applications efficiently • Lead and mentor the team, driving technical excellence through code reviews and best practices in data modeling • Build and maintain automated DAGs and ELT workflows for reliable, 24/7 data availability • Design and optimize Snowflake datasets to support analytics and customer-facing features • Partner with DevOps to configure and manage AWS services for smooth data ingestion and transformation • Proactively monitor workflows, troubleshoot failures, and ensure reliability via dbt testing • Collaborate with developers to align Node.js backends and React frontends with the data architecture • Establish data standards and document pipelines, DAGs, and data models for onboarding and knowledge sharing
Data Engineering Architect
Wavicle Data SolutionsA trusted cloud, data and analytics consulting and development partner helping businesses get more value from data
• Provide top-quality solution design and implementation for clients. • Provide support in defining the scope and the estimating of proposed solutions. • Engage with our clients to understand their strategic objectives. • Responsible for translating business requirements into technology solutions. • Work with the client and engagement leaders to define a project plan to meet delivery expectations. • Utilize Big Data technologies to architect and build a scalable data solution. • Identify any gaps and work with the client to resolve in a timely manner. • Stay current with technology trends in order to make useful recommendations to match the client requirements. • Responsible for the design and execution of abstractions and integration patterns (APIs) to solve complex distributed computing problem. • Participate in pre-sales activities, including proposal development, RFI/RFP response, shaping a solution from the client’s business problem. • Mentor and coach a team of data engineers. • Act as a thought leader in the industry by creating written collateral (white papers or POVs), participate in speaking events, create/participate in internal and external events (lunch ‘n learns, online speaking events, etc.)
• Manage and Develop a High-Performing Team: Hire, mentor, and manage a team of Data Engineers. Provide clear goals, career development guidance, and ongoing performance feedback. Foster accountability, ownership, and engineering excellence. • Own Data Engineering Execution: Oversee the design, development, and operation of scalable data pipelines and platform capabilities across our cloud data environment. Ensure reliable ingestion, transformation, and availability of structured and unstructured data. • Enforce Platform Best Practices: Implement and enforce industry best practices for data modeling, pipeline orchestration, testing, monitoring, observability, and cost efficiency across Azure-based data infrastructure (ADLS, Databricks, and related tooling). • Operational Excellence & Reliability: Define and manage SLAs for production data processes. Ensure high standards for data quality, reliability, and performance. Proactively identify risks and engineering bottlenecks before they impact the business. • Collaborate Cross-Functionally: Partner with Analytics Engineering and business stakeholders (Finance, Operations, Product, Revenue) to translate business requirements into scalable technical solutions. Ensure alignment between engineering execution and business priorities. • Support Governance & Compliance: Operationalize data governance, data protection, and compliance frameworks (including GDPR and other global requirements) in partnership with leadership. Ensure secure and responsible data management practices. • Enable BI and AI/ML Capabilities: Ensure the data platform effectively supports analytics, reporting, machine learning, and AI workloads through well-structured, discoverable, and trusted datasets. • Elevate Organizational Maturity: Improve engineering processes, documentation standards, code review rigor, deployment practices, and cross-team coordination to elevate the overall maturity of the Data Engineering function. • Partner in Planning & Roadmapping: Collaborate with the Director of Data Engineering on roadmap planning, resource allocation, and prioritization to ensure successful execution of strategic initiatives. • This role requires working overlapping hours with U.S.-based colleagues, including evening or overnight hours in India aligned to Central Standard Time (CST). • Additional duties and responsibilities as necessary.




