The Motley Fool logo
The Motley Fool

Making the world smarter, happier, and richer through free and premium investing guidance.

Senior Data Engineer

Data EngineerData EngineerContractRemoteSeniorTeam 501-1,000Since 1993H1B No SponsorCompany SiteLinkedIn

Location

United States

Posted

6 days ago

Salary

$160K - $180K / year

Seniority

Senior

Job Description

Senior Data Engineer

The Motley Fool

• Design, build, and take full ownership of the data infrastructure powering our investment operations. • Own the complete data lifecycle, spanning ingestion, transformation, warehousing, analytics, and machine learning. • Architect ETL/ELT pipelines orchestrated by Apache Airflow (AWS MWAA). • Build and tune a Snowflake data warehouse fed by S3 data lakes. • Develop analytical models and dashboards that turn raw data into actionable intelligence. • Lead the transformation of our data infrastructure from manual processes to scalable, governed, automated data systems. • Replace manual file drops with event-driven Airflow DAGs and AWS Lambda functions. • Define all cloud infrastructure with Terraform or AWS CDK. • Establish clean, governed, and accessible datasets that feed AI agents and intelligent automation.

Job Requirements

  • 5+ years of professional Python development. Comfortable with object-oriented design, data manipulation libraries (pandas, NumPy).
  • Familiarity with financial research data vendors and feed/API products such as CapIQ Xpressfeed, FactSet, Bloomberg, Thomson Reuters/Refinitiv/LSEG, Russell or MSCI.
  • Familiarity with financial business data and feed/API products from Broadridge, Morningstar and custodian banks and fund administrators.
  • Proven experience designing and operating ETL/ELT pipelines. Apache Airflow and Lambda experience is a plus.
  • Deep expertise in Snowflake architecture (clustering keys, micro-partitions, Snowpipe, Stages). Able to write complex analytical SQL, window functions, CTEs, recursive queries, and optimize them for cost and performance.
  • Hands-on experience with AWS CDK or Terraform. You define infrastructure in code, not in the AWS Console.
  • Exposure to LLM integration patterns: markdown files and prompt engineering. Experience with RAG, knowledge bases, and embeddings is a plus.

Related Categories

Related Job Pages

More Data Engineer Jobs

Role Description The Data Team Supervisor is responsible for managing the daily activities of the data analysis team involved in designing, implementing, maintaining, and supporting data warehouse systems and projects. The role ensures that data reporting, data modeling, and data-related projects run efficiently, supporting organizational objectives and maintaining data quality. The supervisor also acts as the primary point of contact for all data reporting, analytics, and database support needs. Qualifications - Bachelor’s degree in computer science, information systems, or a related field required; equivalent experience may be considered. - 7+ years of experience in a data-focused technology role, with heavy exposure to SQL server databases and Power BI (or similar tools). - Previous experience in a supervisory or management role required. Requirements - Provide leadership and oversight in the full lifecycle of data warehouse projects, from planning and design to implementation and support. - Supervise a team of Data Warehouse Analysts and Database Administrators, ensuring that workloads are appropriately balanced and deadlines are met. - Establish and maintain auditing processes to ensure data accuracy and quality across all data warehouses. - Collaborate with business users and stakeholders to gather requirements and translate them into effective technical solutions for data reporting and analytics. - Develop, create, and modify query-based dashboards, reports, KPIs, and advanced analytics to support decision-making across the organization. - Stay current with advancements in data warehousing, business intelligence (BI) technologies, and analytics to bring innovative solutions to the team. - Be available for on-call support as assigned by the manager, providing prompt assistance to resolve data reporting and database-related issues. - Provide technical advice, assistance, and training to users, helping them understand and utilize data reporting tools effectively. - Document and maintain standard operating procedures, troubleshooting guides, and technical documentation for the data warehouse team. - Ensure that best practices for data management, storage, and retrieval are followed across all data warehouse projects. - Perform other duties as assigned by management to support the department’s and organization’s goals. Benefits - Medical Insurance - Vision Insurance - Dental Insurance - 401(k) Matching - Flexible Spending Accounts - Health Savings Accounts - Disability & Life Insurance - Employee Assistance Program - LegalShield - ID Shield - Commuter Reimbursement Plan - Tuition Reimbursement - Bonus Pay - This position falls under our “Corporate Bonus Plan” which entitles you to a potential bonus payout of up to 10% of your annual salary as an annual bonus, if offered employment. - Wellable membership - Telescope Health (telehealth) through Accresa - Intellect (mental health) application - Employee engagement activities, including voluntary events, raffles, book club, and more!

United States
$105K - $120K / year
Spoiler Alert logo

Senior Data Engineer

Spoiler Alert

B2B sales and supply chain software partnering with CPG brands to manage excess and short-dated inventory.

Data Engineer6 days ago
Full TimeRemoteTeam 11-50Since 2015H1B No Sponsor

• Architect, build, and maintain the infrastructure that moves and transforms Spoiler Alert's marketplace data reliably and at scale, effectively incorporating AI coding tools to build with quality and velocity • Own the analytical schema in our data warehouse that serves internal analytics, aggregate analytics, and customer-facing reporting in our data tools, ensuring accuracy, consistency, and documentation • Design and enforce data quality standards across the stack, defining what trusted data means at Spoiler Alert and developing the monitoring and testing that upholds it • Make intentional architectural decisions with the appropriate level of abstraction, documenting decisions and tradeoffs so the team can build confidently on your work, including data infrastructure for effective product applications of AI/ML • Contribute exemplary code in pipeline logic, SQL, and data modeling that other engineers learn from and build on • Share meaningful code reviews that evolve our team's patterns and practices • Debug and resolve data issues quickly, and drive root cause work that prevents them from recurring • Contribute to shared support responsibilities and help build a culture of reliability across the data platform • Take end-to-end accountability for data systems and roadmap themes, owning projects through from design to production to iteration, communicating tradeoffs and risks along the way • Partner with Product, Data, and Engineering teammates to ensure our pipelines, models, and tooling support reliable internal and external decision making using our data • Identify opportunities where better data infrastructure creates leverage for Product, Data, Engineering, and Operations teams, and advocate for those investments with clarity • Contribute meaningfully to architecture conversations and Engineering reflection, shaping technical decisions with your expertise

United States
$140K - $175K / year
Shift5 logo

Data Engineer

Shift5

The Data and Cybersecurity Platform for Planes, Trains, and Tanks

Data Engineer6 days ago
Full TimeRemoteTeam 51-200Since 2019H1B No Sponsor

• Design, build, and maintain robust, scalable batch and streaming data processing, storage, and integration pipelines. • Interpret requirements and design specifications, taking full ownership of building features from the ground up. • Write clean, well-documented, scalable, extensible, and testable code to ensure application quality and maintainability. • Partner with data scientists and engineers to create semantic data models representing complex vehicular systems, and integrate applications cleanly across other Shift5 componentry. • Build scalable data products for data scientists, transportation engineers, and executives to drive insights and decision-making. • Create efficient, reliable, cost-effective, dynamically scalable, and observable solutions utilizing AWS cloud services. • Analyze complex data sets to create data ontologies, verify data quality/integrity, and ensure data accuracy throughout pipelines. • Support the design process and occasionally travel to customer sites (estimated a few times per year) to collaborate with Field Engineers on data integration and deployment.

Virginia
$130K - $170K / year
Prop Firm Match logo

Analytics, Data Engineer

Prop Firm Match

The go-to source for anything prop firm-related. Compare trusted firms, verified reviews, spreads, payouts, and offers.

Data Engineer6 days ago
Full TimeRemoteTeam 51-200Since 2023H1B No Sponsor

• Own and scale the core data platform for reporting, attribution, product analytics, reconciliation, AI workflows, and decision-making. • Turn fragmented raw data into reliable, reusable business intelligence layers. • Manage BigQuery warehouse architecture and dbt models. • Oversee Airbyte pipelines and tracking infrastructure. • Apply orchestration tools for recurring workflows. • Monitor pipeline reliability and address breakages. • Build reliable business data models for trusted sources across various teams. • Improve data quality and reduce manual work through governance and automation. • Enable AI workflows and new integrations with structured data. • Collaborate with cross-functional partners such as Product, Marketing, Finance, Operations, Analytics, and Engineering.

United States
$3.5K - $5.5K / month