The world's most productive AI Workspace for projects, tasks, chat, docs, and more. All software and humans - converged.
Senior Data Engineer
Location
United States
Posted
18 days ago
Salary
$139K - $181.5K / year
Seniority
Senior
Job Description
Senior Data Engineer
ClickUp
Role Description We're looking for a Staff Data Engineer to own the architecture and technical vision of our data platform. This is a high-leverage, high-autonomy role where you'll set the technical bar for the team, drive cross-functional alignment on data infrastructure strategy, and solve our hardest engineering problems. - Own the technical architecture of ClickUp's data platform, making design decisions that balance scalability, cost, reliability, and velocity. - Define and drive the technical roadmap for data infrastructure in partnership with leadership. - Design systems at scale: build frameworks, abstractions, and patterns that other engineers use daily. - Lead complex, cross-team technical initiatives spanning data engineering, analytics engineering, data science, and data analytics. - Drive cost optimization across cloud infrastructure and compute, turning efficiency into a competitive advantage. - Build and evolve our data pipelines using AWS serverless (Lambda, Fargate, Step Functions, Kinesis, S3, DynamoDB, Aurora), Snowflake, and dbt. - Establish and champion engineering standards: observability, testing, CI/CD, code review, and documentation practices. - Design and maintain infrastructure for AI/ML workloads, including LLM frameworks, feature pipelines, training data systems, and model monitoring. - Mentor senior engineers, provide technical guidance through design reviews, and raise the overall engineering quality of the team. - Influence org-wide technical decisions and represent data engineering in company-level architecture discussions. Qualifications - Significant professional experience in data engineering or backend/infrastructure engineering, with at least 3 years operating at a senior or staff level. - Proven track record of owning architecture for data platforms or large-scale distributed systems. - Deep expertise in AWS cloud services (Lambda, Fargate, Step Functions, S3, Kinesis, DynamoDB, Aurora) and infrastructure as code (Terraform and/or CDK). - Expert-level SQL and Snowflake (or equivalent cloud data warehouse) knowledge, including performance tuning and cost optimization. - Strong experience with dbt and modern ELT/ETL patterns at scale. - Advanced Python skills with emphasis on building reusable libraries, frameworks, and tooling. - Hands-on experience with orchestration frameworks (Airflow, Dagster, or Prefect) in production environments. - Experience building data infrastructure for AI/ML: feature stores, training pipelines, embedding pipelines, model serving, or LLM integration. - Deep understanding of streaming and event-driven architectures (Kinesis, Kafka, or equivalent). - Mastery of CI/CD, Git workflows, containerization (Docker), and deployment automation. - Strong communication skills: ability to write technical RFCs, influence without authority, and translate complex trade-offs for non-technical stakeholders. - Track record of mentoring and growing engineers, with a multiplier mindset. Requirements - Experience operating data platforms at high scale (petabyte-level warehouses, millions of events/sec). - Familiarity with data mesh or data product paradigms. - Experience with FinOps practices and cloud cost management at scale. - Prior experience in a technical leadership role without direct reports (staff/principal IC track). - Contributions to open-source data tools or technical communities. Benefits Unsure if you meet all the qualifications of this job description but are deeply excited about the role? We hire based on ambition, grit, and a passion for improving the way people work. If you think ClickUp is the company for you, we encourage you to apply! At ClickUp, we assess every candidate based on the potential impact they can have. We hire the best people for the job and support each person’s journey to build their boldest career. Equal Opportunity Employer ClickUp is an Equal Opportunity Employer, and qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, or national origin. Privacy Notice ClickUp collects and processes personal data in accordance with applicable data protection laws. You can find further details by viewing our Global Candidate Privacy Notice. Visa Sponsorship Please note we are unable to sponsor or take over sponsorship of an employment visa for roles outside of engineering and product at this time. Sponsorship for engineering and product roles is not guaranteed, but is instead based on the business needs for that specific role at that time. Please reach out to the recruiter with any questions. Fraud Alert ClickUp Talent Acquisition will only initiate contact via an @clickup.com email or through our official careers portal on clickup.com. We will never request fees, payments, or sensitive personal information. Please disregard any offers received outside these channels and report them to support@clickup.com. AI Processing Notice ClickUp may use artificial intelligence and machine learning technologies to help review and screen candidates' employment applications against role-related criteria. These tools support, but do not replace, human decision-making. If you have questions or need an accommodation in the recruitment process, please contact us at AskPeople@ClickUp.com.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Own and extend the Python-based ETL job orchestration engine: add new job types, monitor execution, and resolve production failures • Build and maintain data pipelines powering operational reporting for scheduling, finance, credentialing, and clinical operations • Integrate data sources into the warehouse - EHR (NextGen), CRM (HubSpot), HR platforms (ADP, Lever), credentialing (Modio), call center (Five9), and other third-party APIs • Optimize high-frequency SQL workloads • Support and extend custom AI agents • Maintain HIPAA compliance across all data handling: enforce access controls, audit logging, and PHI segregation in pipelines and reporting layers • Write and maintain version-controlled code in the GitHub repository
Senior Data Engineer – Databricks
SugarCRMSugarCRM is the CRM platform that makes the hard things easier. With Sugar, you let the platform do the work.
• Own Databricks production support for the Sugar Predict data platform, including monitoring, alerting, and incident response across all production data flows • Maintain and report on SLA performance metrics for data pipeline delivery, ensuring visibility into platform health and accountability across internal and external stakeholders • Identify and implement pipeline optimizations that reduce Databricks compute costs, improve throughput, and reduce processing windows while tracking impacts through measurable KPIs • Migrate legacy ETL/ELT pipelines to Databricks, building automation tooling to reduce manual intervention and ensure uninterrupted data delivery during transitions • Support new customers onboarding by provisioning, validating, and hardening tenant data pipelines that deliver reliable, isolated data from day one • Design and build high-performance Databricks pipelines that ingest, transform, and serve ERP and CRM data at scale across both Azure and AWS environments • Own the Delta Lake architecture including schema design, partitioning strategies, data quality enforcement, and incremental processing patterns • Enforce data security best practices across Databricks environments, including role-based access control, secrets management, and compliance requirements for enterprise CRM and ERP data • Implement data quality monitoring and observability across pipeline health and ML model inputs, ensuring data integrity that directly supports Sugar Predict prediction accuracy • Apply and enforce multi-tenant data isolation patterns ensuring reliable, secure data delivery across Sugar Predict enterprise customers • Partner with the Enterprise Architecture team to ensure Sugar Predict data pipelines integrate seamlessly with the broader SugarAI product ecosystem • Support a globally distributed operation through on-call rotation and after-hours incident response, meeting SLAs across multiple time zones • Maintain technical documentation, runbooks, and architectural decision records, contributing to team knowledge sharing and operational readiness across on-call and incident response scenarios • Apply CI/CD best practices to data pipeline development, including version control, automated testing, and deployment tooling to ensure reliable and repeatable pipeline delivery
• Assist in building and maintaining ETL/ELT pipelines for healthcare datasets including claims, eligibility, provider, risk adjustment, HEDIS, EHR, and clinical data • Support the development of data models and data transformations aligned with healthcare standards (e.g., HL7, FHIR, X12) • Contribute to data quality checks, validation rules, and documentation for healthcare data assets • Work with analysts and business users to understand data requirements and translate them into technical tasks • Assist in ingestion and integration of new data sources from EMR systems, CMS feeds, and vendor partners • Develop SQL queries, transformations, functions, and stored procedures to support reporting and analytics workflows • Support data platform tools such as Azure Data Factory, Databricks, Python/Spark jobs, and version control workflows • Participate in issue resolution related to data pipeline failures or data quality errors • Maintain data dictionaries, mapping files, and documentation as part of data governance processes • Collaborate with senior engineers to implement best practices in security, compliance (HIPAA), and architecture
Role Description The Data Engineering Practice Leader will serve as a senior pillar lead responsible for shaping data engineering strategy, growing a book of business, leading practice development, and overseeing delivery of enterprise-scale data transformation initiatives. This role sits within FormativGroup’s Data & Analytics practice and blends consulting leadership, executive advisory, technical oversight, commercial ownership, and team development. The position is approximately 25% billable delivery and 75% business development, practice ownership, and strategic leadership. As the Practice Leader, you will partner with executive client stakeholders, lead account growth, oversee solution architecture, guide delivery teams, and help define scalable, modern data solutions across Snowflake, Databricks, AWS, Azure, Microsoft Fabric, and related cloud data ecosystems. What You'll Work On - Serve as a senior pillar lead for the Data & Analytics - Data Engineering practice. - Own and grow a book of business through client relationship leadership, business development, and strategic account expansion. - Lead strategic data modernization discussions with executive and senior client stakeholders. - Shape solution strategy, future-state architecture, and transformation roadmaps for enterprise clients. - Oversee delivery of complex data engineering programs, ensuring alignment to business outcomes, architecture standards, and delivery quality. - Develop new data engineering offerings, accelerators, products, and reusable delivery assets. - Provide technical oversight across Snowflake, Databricks, AWS, Azure, Microsoft Fabric, and equivalent modern data platforms. - Guide solutioning for data lakes, data warehouses, lakehouses, advanced analytics enablement, and AI-ready data platforms. - Lead proposal strategy, pursuit support, client presentations, and commercial solution development. - Partner with internal leadership to define practice priorities, talent strategy, delivery methodology, and growth plans. - Manage, mentor, and develop senior managers, managers, architects, consultants, and cross-functional delivery teams. - Establish governance, data quality, metadata, lineage, security, privacy, and compliance standards across client programs. - Evaluate emerging data, analytics, and AI-enablement technologies and determine applicability to client and practice needs. - Monitor portfolio performance, delivery health, financial outcomes, and client satisfaction. - Represent FormativGroup as a trusted advisor in data engineering, analytics modernization, and cloud data transformation. Qualifications - 15+ years of experience in data engineering, analytics engineering, data architecture, cloud data platforms, or related technology consulting roles. - 10+ years of leadership experience within a traditional consulting firm, technology consulting firm, systems integrator, or professional services organization. - Bachelor’s degree in Data Analytics, Business Analytics, Information Systems, Computer Science, Statistics, Mathematics, Economics, or a related field. - Strong technical background with the ability to solution, oversee architecture, evaluate technical tradeoffs, and develop new data products or offerings. - Executive-level advisory experience across analytics strategy, data modernization, cloud transformation, and business value realization. - Strong understanding of data architecture, data modeling, ETL/ELT, pipeline design, data integration, governance, security, and compliance. - Experience leading large enterprise programs, multi-year engagements, or complex consulting portfolios. - Experience with proposal development, pursuit strategy, solution design, pricing support, and executive presentations. - Strong commercial mindset with the ability to balance client outcomes, delivery quality, and financial performance. - Exceptional communication, executive presence, facilitation, negotiation, and stakeholder management skills. - Experience leading account growth, business development, relationship leadership, and strategic account expansion. Benefits - Discretionary bonuses, commissions, or other incentive programs. - Comprehensive benefits package that includes medical, dental, vision, 401(k), paid time off, etc. Employment Eligibility Applicants must be authorized to work for ANY employer in the U.S. We are unable to sponsor or take over sponsorship of an employment visa currently. This is a remote role with approximately 50% travel. The preferred location is the Northeast U.S. To be considered for this position, candidates must reside in one of the following U.S. states or Washington, DC: AL, AR, AZ, CA, CO, CT, DE, FL, GA, IA, ID, IL, IN, KS, MA, MD, MI, MN, MO, NC, NH, NJ, NV, NY, OH, OK, OR, PA, TN, TX, VA, WI, or Washington, DC. Candidates residing outside these locations are not eligible for consideration currently. Compensation The estimated compensation range for this position is $185,000 — $265,000 USD. The actual compensation offered will be determined based on factors such as the candidate’s experience, skills, education, work location, and internal equity. Company Description FormativGroup operates within the critical middle layer of business technology, where applications and systems connect infrastructure to business processes. We are specialists who help the middle market take full advantage of their technology investments with deep, industry-centric expertise, all in one place, to unify fragmented systems. With deep technical expertise across cloud architecture, system integration, AI, and data strategy, we bridge the gap between business goals and modern platforms. FormativGroup is an equal opportunity employer providing opportunities to applicants and employees without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, or protected veteran status.



