Senior Data Engineer
Location
United States
Posted
2 days ago
Salary
$145K - $220K / year
Seniority
Senior
Job Description
Senior Data Engineer
Allocate
Role Description Allocate is looking for a Senior Data Engineer to help build out the data infrastructure that powers our analytics, reporting, and data-driven product features. In this role, you will partner closely with our data lead to model core financial entities, integrate internal and external sources, and build the pipelines and infrastructure that let our engineering and product teams make informed decisions and ship compelling features. This is a fully remote position where you will work alongside our backend team (C#/.NET) and frontend team (Node/Vue.js) to integrate data pipelines into our platform. Responsibilities - Build and Extend Data Architecture: - Build on and extend Allocate's data lakehouse on AWS. - Contribute to our knowledge graph that models key relationships (investors, funds, companies, etc.). - Integrate vector database for semantic search and retrieval across AI agents, models, and providers. - Develop Data Pipelines: - Create robust ETL/ELT pipelines to ingest, clean, and transform data. - Ensure both batch processing and real-time data streaming are handled. - Build pipelines with scalability and reliability in mind. - Enable AI/ML Capabilities: - Work closely with data science and engineering team to provision data and infrastructure for machine learning models. - Prepare training datasets and set up feature stores. - Orchestrate workflows that feed LLM-based agents with the necessary context. - Engineering Excellence and Collaboration: - Partner with data lead and engineering team to deliver data and AI infrastructure. - Raise the bar through code review, testing, and adherence to best practices. - Work in cross-functional squads to incorporate data-driven features into the product roadmap. - Infrastructure and DevOps: - Collaborate with DevOps engineers to deploy and maintain data services. - Containerize and orchestrate data tools using Docker/Kubernetes on AWS EKS. - Implement CI/CD pipelines for data workflows. - Monitor the health and performance of data platforms. - Continuous Improvement: - Stay up to date with the latest in data engineering and AI. - Evaluate and recommend new technologies to improve pipeline reliability. - Encourage rethinking how things are done to build a world-class platform. Qualifications - 5+ years of hands-on experience in data engineering or related fields. - Strong experience working with AWS cloud services for data. - Proficiency in SQL and relational database design. - Fluency in at least one major programming language used in data engineering. - Understanding of machine learning models and data consumption. - Solid understanding of containerization and deployment. - Strong analytical and problem-solving skills. - Experience working in a regulated SEC environment. - Excellent communication skills and a collaborative mindset. - Bachelor’s degree in Computer Science or equivalent practical experience. Requirements - Experience with tools like S3, EC2, ECS, EKS, Athena, Redshift, Glue, and Step Functions. - Experience with data warehouses or lakehouses (e.g., Snowflake, Databricks Delta Lake). - Experience with graph databases and knowledge graph schemas. - Experience with Docker and Kubernetes for running distributed jobs/services. - Experience with workflow managers (Airflow, Prefect, dbt, or similar). Benefits - Medical, dental, and vision insurance. - 401(k) plan. - Responsible vacation time (PTO). - Travel required for team/department offsites. - A Broadband internet connection required.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer, EIM
Vena SolutionsTake your entire business from reactive to proactive with the leading AI-Powered Complete FP&A Platform.
• Drive the execution of the technical vision of our internal Data Infrastructure. • Evolve the maturity and the level of technical excellence across your group and the entire EIM organization. • Build an all-inclusive data program that is source agnostic and brings together data from various business applications to provide product usage, marketing effectiveness, employee retention, LTV, all in one place. • Create true agility within the data team to work sprint-by-sprint and build progressively, embracing the concept of MVP and backlogs. • Design and implement strategies that will achieve growth and productivity through process automation of existing and new operating models. • Create data models, data mapping and the necessary ETL design. Build and validate activities for various data integrations. • Partner with key stakeholders teams to deliver a scalable next generation data architecture, and solid data engineering processes that align with corporate Security and Data Governance. • Foster data-driven culture and the creation of related competencies and data literacy across the organization.
Senior Data Engineer
Privia HealthA health management technology company, Privia Health is a national practice led by physicians. The company was founded in 2007 to provide physician groups with resources dedicated
Role Description The Senior Data Engineer will play a pivotal role in building, modernizing, and scaling our enterprise data platform. While our legacy footprint includes SQL Server on GCP, our future is centered on a modern stack powered by Snowflake and Python. - Pipeline Engineering: Design, develop, and maintain resilient, scalable, and efficient end-to-end data pipelines. - Stakeholder Collaboration: Partner directly with product owners, data analysts, and Analytics stakeholders to gather requirements, communicate technical roadmaps, and deliver data solutions aligned with business goals. - Architecture & Design: Play a key role in data modeling and architecture discussions to support high-volume, low-latency, and high-availability systems. - Technical Leadership: Act as a subject matter expert for the data layer; mentor junior engineers, advocate for code quality, and delegate project tasks effectively. - DevOps Culture: Champion a modern DevOps mindset by implementing CI/CD practices, automated testing, and robust monitoring/observability for data workflows. - SDLC Execution: Deliver clean, well-tested, and secure code within an Agile/Scrum framework (Jira), participating fully from discovery through to deployment. - Continuous Innovation: Evaluate emerging data technologies and framework optimizations to continuously improve platform performance and cost-efficiency. - Responsible for end-to-end project ownership. - Follow established processes and be a champion for best practices and continuous improvement. - Deliver timely, well-written, well-designed, testable, efficient code. - Ensure designs and code are in compliance with specifications. - Perform other duties as assigned. Qualifications - 5+ years of professional experience designing and implementing commercial data warehouses and data marts. - 4+ years of hands-on production experience with Snowflake (including performance tuning, Snowpipe, or secure data sharing). - 3+ years of strong development experience in Python for data manipulation and pipeline orchestration. - 3+ years of strong experience with DBT or similar transformation tool. - Expert-level proficiency in advanced SQL (including complex joins, window functions, and query optimization). - Experience integrating with external data sources via APIs and handling semi-structured data (JSON, Parquet). - Solid understanding of relational databases (MSSQL/MySQL). - Proven track record of shipping high-quality software backed by a strong belief in automated testing (unit, integration, and data quality testing). - Experience working within an Agile SDLC environment. - Must comply with HIPAA rules and regulations. - Must be willing and able to communicate with the team via webcam, webconf, and phone. - Must have access to private, quiet work space with high-speed internet to effectively work remotely. Requirements - Minimum of 5 MBPS for Download Speed and 3 MBPS for Upload Speed for remote work. - Employees who regularly work from home offices are eligible for expense reimbursement to offset internet costs. Benefits - Salary range: $105,000.00 to $120,000.00 in base pay, exclusive of any bonuses or benefits. - Eligible for an annual bonus targeted at 15%. - Includes medical, dental, vision, life, and pet insurance, 401K, paid time off, and other wellness programs.
Data Engineer
KyndrylWe design, build, manage and modernize the mission-critical technology systems that the world depends on every day.
• Architect behind the high-performance data infrastructure • Responsible for implementing identified components • Ensure that implemented components are appropriately documented, unit-tested, and ready for integration into the final product • Architect the solution, test the code, and deploy and build a CI/CD pipeline for it • Provide work estimates for assigned development work • Guide features, functional objectives, or technologies being built for interested parties
Senior Full-Stack Software Engineer, Data & Migrations
Redhorse CorporationRedhorse provides energy, environmental, technology, and intelligence services to support our government's needs.
• Lead the modernization of a mission-critical legacy system managing billions of dollars in international security cooperation. • Work as an "Agent-native" engineer using AI-led development tools daily to replace a decades-old Oracle-based platform. • Own the modernization ground, ensuring complex production data is migrated with financial-grade proof while building across the full stack (UI, API, and Infrastructure).



