We don’t just get the job done: we constantly think about how to get it done better.
Data Engineer
Location
New York
Posted
1 day ago
Salary
$90K - $140K / year
Seniority
Senior
Job Description
Data Engineer
Mindex
• Assemble large, complex data sets that meet business requirements through extraction, transformation, and loading of data from a wide variety of data sources. • Provide operational support and troubleshooting for existing processes and systems. • Work closely with architects, solution leads, data owners, Data Scientists and key stakeholders to facilitate and coordinate the data platform backlog grooming process, triaging new feature requests in preparation for future project activities. • Deliver automation & efficient processes to ensure high quality throughput & performance of the entire data & analytics platform. • Ensure data extraction, transformation and loading data meet data security & compliance requirements. • Engage with data source platform leads to gain tactical and strategic understanding of data sources required by Agency Data Services AI/ML as well as Data Office standards. • Create data tools for data scientist team members that assist them in building and optimizing models.
Job Requirements
- BS degree in Computer Science, Data Science, Engineering, or equivalent software/services experience required.
- 4+ years working with SQL, Snowflake, Databricks, Spark, and other big data technologies; 4+ years using Python, SQL, PySpark, R, or similar languages and manipulating, processing, and extracting value from large, disconnected data sets.
- 4+ years building and optimizing data pipelines, architectures, and data sets to answer business questions and identify opportunities for improvement
- 2+ years supporting large-scale data processing and storage using Azure Data Factory, Integration Runtime, Data Lake, Databricks, Spark, Azure ML, and Cosmos DB.
- 2+ years addressing privacy, compliance, and security aspects of data storage and processing; and delivering data solutions in Agile environments.
- 2+ years with software development and CI/CD methodologies and tools for automated infrastructure code and MLOps and designing, implementing, and maintaining automation platforms and tools, including Ansible Tower, Azure, ARM, Terraform Enterprise, Azure DevOps, and GitHub Actions.
- 2+ years with Salesforce FSC and Salesforce Data Cloud.
Benefits
- Health insurance
- Paid holidays
- Flexible time off
- 401k retirement savings plan and company match with pre-tax and ROTH options
- Dental insurance
- Vision insurance
- Employer paid disability insurance
- Life insurance and AD&D insurance
- Employee assistance program
- Flexible spending accounts
- Health savings account with employer contributions
- Accident, critical illness, hospital indemnity, and legal assistance
- Adoption assistance
- Domestic partner coverage
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Platform Product Intern
G-PFind, hire and manage teams in days instead of months with the #1 Global Growth Platform.™
• Lead the end-to-end development of an internal-facing GenAI capability • Conduct competitive scans of the emerging Databricks ecosystem and adjacent tooling • Help define schemas, APIs, and contracts • Participate in biweekly Data Platform demos and engineering standups • Close collaboration and weekly 1:1s with the Principal Product Manager of Data Platform.
Data Engineering Lead
OpenSesameWe help companies develop the world's most productive and admired workforces.
• Own the Data Governance & Accessibility program • Design scalable data architecture • Improve pipeline reliability and mentor data engineers
• Architect, implement, and optimize data platforms and pipelines specifically designed to support LLMs, Retrieval-Augmented Generation (RAG), and sophisticated AI agentic systems at Exabyte scale • Drive the adoption and deployment of agentic workflows and agent harnessing techniques to create autonomous, data-driven security features • Design and implement highly scalable, fault-tolerant, and cost-effective data solutions, emphasizing rapid iteration and high-quality deployment • Write elegant, production-ready code with a focus on performance, maintainability, and testing rigor, ensuring the ability to ship fast without compromising quality • Provide technical leadership and deep expertise in data modeling, normalization, and semantic cataloging for AI/ML workloads • Establish best practices for MLOps/DataOps surrounding LLMs, including monitoring, observability, and zero-touch recovery mechanisms for AI services • Actively mentor engineers, conducting technical workshops, leading design reviews, and strengthening the team's knowledge in cutting-edge AI platform technologies • Collaborate across the organization with Data Scientists, Product Managers, and other engineering teams to transform research prototypes into robust, production-grade services • Own the end-to-end lifecycle of critical data services: development, testing, deployment, and monitoring
Senior Project Manager, Streaming Intelligence & Data Platform
PlayOn! SportsThe nation's leading high school media company providing live streaming and digital ticketing services.
• Serve as the primary delivery partner for the Streaming Intelligence product line and DPT, converting roadmap priorities into actionable project plans with clear milestones, owners, and dependencies. • Co-own quarterly and annual planning with the Principal PM—from ideation through go-to-market—ensuring engineering capacity, vendor timelines, and business priorities are aligned. • Track and report on product line KPIs defined by the Principal PM (accuracy benchmarks, processing latency, content coverage, downstream adoption) and surface risks or deviations early. • Partner with the data team on agentic AI enablement—coordinating efforts to surface data models and semantic views to the broader business through AI-powered interfaces, including usage tracking and accuracy assurance. • Manage delivery cadence for platform infrastructure initiatives including data governance rollouts, real-time streaming capabilities, and pipeline migration efforts—tracking phased delivery across multiple months with clear metrics of success. • Lead end-to-end program management for AI pipeline initiatives, including the game film analysis benchmarking program that evaluates vision models (e.g., Gemini Flash, Gemini Pro) against human-tagged ground truth across 28+ annotated fields per play. • Manage the cadence of model evaluation cycles: coordinating game imports, prompt engineering iterations, accuracy scoring, and cost analysis across multiple AI models. • Drive vendor integration workstreams—tracking deliverables, SLAs, and roadmap alignment with external computer vision and AI partners alongside the Principal PM. • Build and maintain project documentation for hybrid human-AI workflows, including escalation paths for ambiguous plays, quality review processes, and fallback handling. • Facilitate cross-team visibility across product, engineering, data, design, and QA to ensure aligned execution mapping back to PlayOn’s unified product objectives: Engaged Communities, Resilient Services, and New Customer Markets. • Own dependency mapping, risk management, and blocker resolution for strategic AI initiatives spanning multiple engineering teams. • Develop and maintain project dashboards, status reports, and executive-level communications that increase transparency of engineering progress to the broader business. • Provide governance and optimization for Atlassian tools (Jira, Confluence) to ensure standardized workflows, consistent estimation practices, and clear traceability from strategy to execution. • Champion operational improvements to planning cycles, stakeholder coordination, cross-org meetings, and feedback loops—creating transparency through standardized tooling and repeatable frameworks. • Actively incorporate AI tools (e.g., Claude, automation platforms) into your own delivery workflows: automating status reports, synthesizing meeting notes, generating risk analyses, and accelerating documentation. • Contribute to PlayOn’s broader AI maturity journey by demonstrating AI-augmented delivery practices that can be adopted across the PMO and engineering organization.




