Job Closed

This listing is no longer active.

Blackbird.AI logo
Blackbird.AI

Blackbird.AI helps organizations detect and respond to disinformation and manipulation that causes reputational and financial harm. Powered by their exclusive AI-Driven Constellation Platform, Fortune 500s and governments can proactively manage new information risks that were previously undetectable. Blackbird.AI was founded by a team of experts from artificial intelligence, behavioral psychology and national security, with a mission to defend authenticity and fight narrative manipulation. Recognized by Forrester as a "Top Threat Intelligence Company", the Blackbird Risk Index™(BRI) is an industry benchmark in the reputation enhancement space.

Staff Data Engineer

Data EngineerData EngineerOtherRemoteSeniorTeam 51Since 2017

Location

Texas + 2 moreAll locations: Texas | Washington | New York

Posted

123 days ago

Salary

$160K - $190K / year

Seniority

Senior

Job Description

Staff Data Engineer

Blackbird.AI

Blackbird.AI helps organizations discover emergent threats and stay one step ahead of real-world harm through our AI-powered Narrative and Risk Intelligence Platform. Our commitment is to prioritize safety and security, providing the tools to identify potential risks and ensure a safer environment proactively. No matter the job or where it's located, we're all connected by a shared vision: To lead and enhance the landscape of risk intelligence. As a Staff Data Engineer, you will play a critical role in architecting and scaling our data platform and AI/ML processing infrastructure. You'll be a technical leader responsible for our entire data ecosystem—from ingestion pipelines that process diverse data sources to the lakehouse architecture that powers our narrative analysis capabilities. You'll architect systems that seamlessly support batch and streaming data patterns while building real time alerting on generated insights. You'll work at the intersection of data engineering, AI-powered data transformation, and platform engineering, making architectural decisions that will shape our ability to detect misinformation, disinformation, and narrative attacks at scale while managing costs effectively. A key aspect of this role involves building intelligent pipelines that use traditional AI and generative AI to cluster, enrich, classify, and extract insights from data as it flows through our system. As a Staff Data Engineer you will: Design and implement scalable data platform architecture on Databricks, supporting both batch and streaming ingestion Build robust, fault-tolerant data ingestion pipelines that integrate with multiple third-party APIs and data providers Design and implement AI-powered enrichment stages within pipelines—applying ML clustering, generative AI summarization, classification, and entity extraction to transform raw data into actionable intelligence Build analytical systems with full-text search capabilities using Elasticsearch for rapid querying and analysis of enriched data Work with AI/ML researchers to implement, integrate and scaling AI processing Expose data platform capabilities as APIs and other interfaces for downstream consumption by applications and services Optimize data lake and lakehouse architecture for performance, cost-efficiency, and scalability Design and implement data quality frameworks, monitoring, and alerting systems Design efficient architectures for calling external AI APIs and managing rate limits, costs, and reliability Architect solutions with cost-efficiency as a first-class concern, implementing monitoring and optimization strategies for compute and storage Make critical build-vs-buy decisions and establish architectural standards for the data organization Mentor engineers and elevate the team's technical capabilities through code reviews, design discussions, and knowledge sharing Requirements Preferred Qualifications: Experience designing both batch and streaming/near real-time data architectures Proficiency with Elasticsearch for building analytical systems with full-text search capabilities Hands-on experience with LLM APIs and understanding of rate limiting and cost optimization Experience with Agentic AI, context engineering, and evaluation Background in trust & safety, security, or content moderation domains Experience with data observability tools and building comprehensive monitoring systems Prior experience at a startup or fast-paced environment Apply agentic coding tools for day to day development Familiarity with Databricks' Lakeflow, Agent Bricks, and vector databases What’s in it for you: Blackbird.AI is embarking on an exciting growth journey with numerous opportunities for career development within the company. You will join a nurturing, inclusive, and experienced team. Join us as we soar to new heights! Values: At Blackbird.AI, our core values shape how we work and make decisions. Our values inspire us to be authentic and continue improving. We embrace a strong sense of responsibility to society, recognizing the vital role our services play in empowering governments, communities, and individuals to foster critical thinking and empowerment. We believe in integrating personal and professional lives with societal needs, emphasizing the importance of creating an environment that attracts top talent and provides substantial growth opportunities. We are motivated by the potential of science and technology to impact humanity positively. Benefits Competitive compensation package, 401(k), and equity - everyone has a stake in our growth! Comprehensive health benefits for you and your loved ones, including wellness days and monthly wellness reimbursements - an apple a day doesn't always keep the doctor away! Generous vacation policy, encouraging you to take the time you need - we trust you to strike the right work/life balance! A flexible work environment with opportunities to collaborate with your team in person - you can have it all! Inclusion and Impact - soar to new heights! Professional development stipend - never stop learning! Location & Work Eligibility: We are only able to hire candidates currently residing in the U.S. Unfortunately, we cannot offer visa sponsorship for this role. Applicants must be legally authorized to work in the U.S. without future sponsorship. Candidates applying for this position should meet the residency requirement and be able to provide proof of U.S. work authorization. Pay Transparency: [NEW YORK ONLY] For individuals assigned and/or hired to work in New York, Blackbird.AI is required by law to include a reasonable estimate of the compensation range for this role. This compensation range is specific to New York. It takes into account the wide range of factors that are considered in making compensation decisions, including, but not limited to, skill sets, experience and training, licensure and certifications, and other business and organizational needs. At Blackbird.AI, it is not typical for an individual to be hired at or near the top of the range for their role, and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current compensation range for this position is expected to be $160,000 - $190,000. This range may vary for positions outside of New York and as it has not been adjusted for the applicable geographic differential associated with the location where the position may be filled. Regardless of location, candidates can expect during the first few conversations with Blackbird.AI’s Talent Team and Hiring Managers to share any approved budget. Apply Today Equal Opportunity Employer

Job Requirements

  • 8+ years of software engineering experience with 5+ years focused on data platforms or data engineering
  • Deep expertise with Databricks, Apache Spark, and data lakehouse architectures
  • Strong experience building and operating data pipelines at scale (handling TBs+ of data)
  • Experience integrating AI/ML capabilities into data pipelines (clustering, LLM APIs, classification, summarization)
  • Proficiency in Python, DBT, and SQL for data processing and pipeline development
  • Experience with both batch and streaming large scale data processing patterns
  • Strong understanding of cloud platforms (AWS, Azure)
  • Excellent communication skills and ability to mentor engineers

Related Categories

Related Job Pages

More Data Engineer Jobs

Baubap logo

Senior Data Engineer

Baubap

Smart microloan for everyone

Data Engineer123 days ago
Full TimeRemoteTeam 11-50H1B No Sponsor

• Maintenance and optimization of key data processing pipelines. • ML Infrastructure Scale up. • Implementation of Risk Monitoring Procedures to give oversight over internal pipeline. • Design, build and maintain data pipelines. • Data integrity assurance, monitoring and resolution. • Query Performance optimization. • Purpose built backend logic. • ML Ops Infrastructure management and scaling. • Documentation and knowledge sharing.

Latin America
Job Closed

About the Role We are looking for a talented Senior Data Engineer to build and scale the data infrastructure that powers Torus's mission. As a core member of our data team, you'll build and maintain our modern data stack (dbt, Redshift, Airflow, Fivetran, Metabase, and Streamlit), designing and implementing scalable data pipelines that ingest, transform, and serve data from our complex ecosystem of IoT devices, grid systems, and business applications. In this role, you'll collaborate on and support the infrastructure that enables our data team to operate efficiently. You'll build real-time data pipelines that process telemetry from thousands of energy storage devices, create robust ETL workflows that ensure data quality and reliability, and develop tools that make data accessible to both technical and non-technical stakeholders. Your work will directly enable machine learning models, analytics dashboards, and business intelligence that drive strategic decisions. Our products operate within complex ecosystems including IoT devices, the electrical grid, commercial and industrial buildings, and smart home systems. You'll work with high-volume streaming data, time-series sensor data, and diverse data sources to build the infrastructure that makes Torus a truly data-driven organization. Who You Are - Autonomous and ownership-oriented: You thrive with autonomy and end-to-end ownership. You take pride in owning your code, pipelines, and services from conception to production. You have strong technical judgment and the ability to span across the data stack—from pipeline development to data modeling to tooling. You have strong DevOps fundamentals and enjoy total ownership of your domain. - Collaborative architect: The team is small, so you'll be collaborating on architectural design and contributing to technical direction, not just implementing tickets. You work effectively both independently and as part of a team, actively collaborating with your data team colleagues and cross-functional partners, sharing knowledge and giving/receiving candid feedback. - Builder mindset: You strive to write elegant, maintainable code and are comfortable independently picking up new technologies to solve problems efficiently. You're passionate about building infrastructure and tools that empower data scientists, analysts, and business users to move faster and make better decisions. - Quality-focused: You have an eye for detail, good data intuition, and a passion for data quality and reliability. - Adaptable startup operator: You thrive in a startup environment with ambiguous requirements and rapidly changing priorities. If you prefer clearly defined requirements and established processes, this role may not be the right fit. If you're energized by scaling data systems and shaping how a growing company uses data, you'll excel here. You understand that our existing systems were built to solve real problems under real constraints, and you approach improvements with curiosity and respect rather than judgment. - Continuous learner: You're genuinely excited about learning new technologies and tackling unfamiliar problems. You may not check every box in our requirements, but you're confident in your ability to learn quickly and contribute meaningfully. You keep your ear to the ground for opportunities to improve data flows and aren't afraid to propose innovative solutions. - Mission-driven: You're passionate about using technology to combat climate change and transform how people consume energy. If you're someone who loves to learn, is supportive of your teammates, stays curious about where data processes can be improved, and is ready to roll up your sleeves to build better systems together—we want to hear from you, even if you don't meet every single requirement listed below. What You'll Own Data Infrastructure & Pipelines - Design, build, and maintain scalable batch and streaming data pipelines that handle high-volume IoT telemetry and business data - Develop robust ELT workflows to ingest, transform, and load data from diverse sources including APIs, databases, IoT devices, and third-party systems - Build and optimize our data warehouse using Redshift, implementing dimensional models that support analytics and machine learning use cases - Implement real-time data processing systems that enable immediate insights and rapid response to system events - Develop incremental SQL patterns in dbt for efficient data transformation Data Quality & Reliability - Build tools, processes, and pipelines to enforce, check, and manage data quality at scale - Develop monitoring and alerting systems to ensure pipeline reliability and data freshness - Create data validation frameworks and automated testing for data pipelines - Establish best practices for data governance, documentation, and lineage tracking Platform & Tools Development - Build frameworks that enable data scientists to deploy models to production efficiently - Develop self-service analytics capabilities and data access patterns for non-technical stakeholders - Create and enhance analytics tools to facilitate intuitive data consumption Infrastructure & Operations - Own the full software development lifecycle for data services, focusing on automation, testing, monitoring, and documentation - Develop and maintain infrastructure using AWS CDK and Terraform - Build and maintain CI/CD pipelines for data operations - Manage cloud infrastructure on AWS (ECS, Redshift, Lambda, S3) - Support ad hoc data requests and maintain core pipeline operations Required Experience - Typically requires a bachelor's degree in Computer Science, Engineering, Information Technology, Data Science, or a related technical field and 5+ years of experience building scalable data pipelines, but we value diverse learning paths and welcome candidates who demonstrate equivalent expertise. - Strong experience building batch and streaming data pipelines using distributed processing frameworks - 3+ years of experience designing and implementing ELT pipelines for data extraction, transformation, and loading from diverse sources - Expert proficiency in Python with strong software engineering fundamentals - Advanced SQL skills and experience with relational databases and data warehousing - Hands-on experience with data warehouse modeling, including dimensional modeling and schema design - Experience with cloud platforms (AWS preferred) and infrastructure-as-code tools - Practical experience owning production data systems with DevOps fundamentals - Experience with containerization (Docker) and orchestration concepts - Passion for data quality, monitoring, and building reliable systems Preferred Experience - Experience with modern data stack tools: dbt, Redshift, Airflow, Fivetran, Streamlit, or similar - Hands-on experience with AWS services (ECS, Lambda, Athena, S3, EC2, VPC) - Experience with Terraform and/or AWS CDK for infrastructure as code - Experience working with IoT data, time-series data, or sensor data at scale - Familiarity with data observability and lineage tools (OpenMetadata, Monte Carlo, Great Expectations, etc.) - Experience with monitoring platforms like Datadog - Experience with CI/CD pipelines (GitHub Actions or similar) - Knowledge of Kubernetes and container orchestration - Experience with authentication systems (AWS SSO, Okta, etc.) - Experience in energy systems, industrial IoT, or utilities domain - Background supporting machine learning infrastructure and MLOps practices - Experience in a high-growth startup environment - Familiarity with AI-assisted development tools (Cursor, Windsurf, etc.) Additional Details - Background Check: All candidates are subject to a background check - Location + Travel: The role is remote based in the US. Requires occasional travel to our South Salt Lake Headquarters - Schedule: Full-Time, Salaried - Compensation: $130,000 - $170,000 (Note: We have the flexibility to hire at different levels, which may impact the corresponding pay range) - Work Authorization: Applicants must already have the legal authorization to work in the US without requiring any employer sponsorship Physical Requirements - Constantly operates a computer and other peripheral office equipment such as a printer or mouse - Ability to communicate information so others can understand. Must be able to exchange accurate information in these situations - Must report to work reliably and with the ability to use full and unimpaired skills and judgment to safely execute your job - Proficiency in reading, writing, and speaking English required - When on the production floor, required to don personal protective equipment to include, but not limited to ear protection, gloves, eye protection and/or safety helmet. - When on the production floor, ability to observe, detect and respond to audible and visual machine malfunction warnings. Our Benefits and Perks Benefits eligibility is based on employment status. - Employee Rewards Package including Equity - 401(k) Retirement Savings Plan - Health Benefits Package: Choice between traditional PPO or HSA eligible medical plans; Dental insurance; and Vision insurance - Human-centered Paid Time Off including Unlimited Discretionary PTO or 10 days of accrued PTO; 10-days paid company holidays; Waiting period-free 100% paid parental leave - Torus paid Life and AD&D Insurance with option to purchase additional coverage - Voluntary Short- and Long-Term Disability Insurance - Peer Recognition Program Torus is proud to be an Equal Opportunity Employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran, or disability status.

United States
Job Closed
Doran Jones logo

Data Engineer – Automation-Led Modernization

Doran Jones

McLaren Strategic Solutions is a leading-edge global technology consulting firm, addressing critical challenges across industries such as retail, financial services, and healthcare. Integrating a powerful ecosystem of platforms with capital-efficient execution, McLaren specializes in digital transformation to help businesses optimize operations, accelerate revenue, and achieve scalable outcomes. McLaren’s expertise spans the development of customer-centric applications, modernizing systems for cost-effectiveness and security, and leveraging cloud scalability for future-ready architectures. With a deep commitment to operational excellence, McLaren provides comprehensive managed services, including application maintenance, cybersecurity, platform solutions, and AI-optimized operations, ensuring seamless, secure, and efficient performance. From supply chain automation to compliance and analytics, McLaren drives measurable impact: improving workforce productivity, reducing inventory costs, and cutting technology ownership expenses. With its emphasis on automation and zero business downtime, McLaren facilitates seamless migrations from legacy systems to modern platforms, enabling organizations to harness the full potential of digital transformation. Backed by strategic partnerships and a proven delivery model, McLaren empowers clients to innovate, modernize, and achieve lasting success in today’s digital economy. McLaren is a certified minority owned business through the NMSDC and has a mission to place more people from non-traditional backgrounds into sustainable technology careers. Through partnerships with non-profit technology programs in underserved communities and Veteran organizations, candidates transition from tech training programs into real IT careers at McLaren. Our unique recruitment policy allows us to create exceptional teams, bringing a broad spectrum of experience to our company and creating anything but a traditional consulting firm.

Data Engineer123 days ago

This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more. Role Description We are looking for a Senior Data Engineer / Data Architect to lead automation-led modernization of legacy data and reporting platforms at scale. This role drives operational automation, repeatable architecture, and AI-assisted tooling, eliminating manual work and accelerating delivery across multiple environments. A core focus is leading the migration of data pipelines and integrations from Informatica to Databricks, while building scalable, repeatable patterns used across states and teams. This is a hands-on, senior role with ownership across architecture, design, and execution. Responsibilities - Assess legacy data and reporting workloads to identify where automation replaces manual effort. - Lead the migration of Informatica-based data workloads to Databricks, ensuring performance, reliability, and data integrity. - Design and execute an automation-first modernization strategy for data pipelines, reporting systems, and analytics platforms. - Apply tool-assisted and AI-assisted techniques to accelerate modernization while maintaining compliance and control. - Build repeatable frameworks and patterns for: - Data ingestion, transformation, and orchestration; - Reporting and analytics modernization; - Data validation, reconciliation, and quality controls. - Establish governance to ensure modernization efforts are consistent, auditable, and scalable. - Partner with distributed data teams to deploy and scale modernization patterns across environments. - Provide hands-on technical leadership while remaining engaged in execution. Qualifications - 10+ years in data engineering and/or data architecture. - Proven track record of modernizing large-scale, complex, legacy data and reporting environments. - Hands-on experience migrating or modernizing Informatica-based data pipelines to modern platforms such as Databricks. - Demonstrated use of automation frameworks, accelerators, or AI-assisted tools to compress delivery timelines. - Hands-on experience with Azure, Cloudera, and Power BI. - Strong experience designing and operating ETL/ELT pipelines and analytics layers. - Deep understanding of data quality, lineage, reconciliation, and validation frameworks. - Experience working in large-scale or regulated enterprise environments. Preferred Experience - Experience in payer, healthcare, or other regulated domains. - Experience designing solutions that support multi-environment scalability. - Exposure to GenAI tooling for code generation and automation. - Experience as a technical thought leader for new platform initiatives. Company Description McLaren Strategic Solutions is a leading-edge global technology consulting firm, addressing critical challenges across industries such as retail, financial services, and healthcare. Integrating a powerful ecosystem of platforms with capital-efficient execution, McLaren specializes in digital transformation to help businesses optimize operations, accelerate revenue, and achieve scalable outcomes. - McLaren’s expertise spans the development of customer-centric applications, modernizing systems for cost-effectiveness and security, and leveraging cloud scalability for future-ready architectures. - With a deep commitment to operational excellence, McLaren provides comprehensive managed services, including application maintenance, cybersecurity, platform solutions, and AI-optimized operations, ensuring seamless, secure, and efficient performance. - From supply chain automation to compliance and analytics, McLaren drives measurable impact: improving workforce productivity, reducing inventory costs, and cutting technology ownership expenses. - With its emphasis on automation and zero business downtime, McLaren facilitates seamless migrations from legacy systems to modern platforms, enabling organizations to harness the full potential of digital transformation. - Backed by strategic partnerships and a proven delivery model, McLaren empowers clients to innovate, modernize, and achieve lasting success in today’s digital economy. - McLaren is a certified minority owned business through the NMSDC and has a mission to place more people from non-traditional backgrounds into sustainable technology careers. - Through partnerships with non-profit technology programs in underserved communities and Veteran organizations, candidates transition from tech training programs into real IT careers at McLaren. - Our unique recruitment policy allows us to create exceptional teams, bringing a broad spectrum of experience to our company and creating anything but a traditional consulting firm.

United States
Job Closed
Suno logo

Senior Data Engineer

Suno

Transformando a sociedade por meio da educação financeira.

Data Engineer123 days ago
Full TimeRemoteTeam 201-500Since 2016H1B No Sponsor

• Help structure pipelines, architectures, and tools that ensure the quality, security, and governance of our data at scale. • Build and orchestrate data pipelines (ELT/ETL), ensuring that data arrives organized and reliable. • Work on data modeling and ETL processes to scale our analytics. • Support the business by translating needs into technical solutions that align with our strategy. • Create frameworks that enable consistent analyses and support growth decisions across areas such as Subscriptions, Marketing, Asset, and new projects like Advisor. • Collaborate with BI analysts, software engineers, and internal stakeholders, acting as the bridge between technology and the business. • Ensure governance, quality, and documentation of data processes.

Brazil
Job Closed