Principal AI Threat Detection Engineer

Location

United States

Posted

5 days ago

Salary

$117.2K - $157.5K / year

Seniority

Lead

No structured requirement data.

Job Description

Principal AI Threat Detection Engineer

Blackbaud

Role Description We are looking for a savvy, high-performing Principal AI Threat Detection Engineer – Insider Threat to lead the strategy, design, and day-to-day management of Blackbaud’s Insider Threat program as it matures into an AI-augmented capability, protecting Blackbaud’s and our clients’ information. As a technical leader within Security Engineering, this individual will direct AI-driven and agentic tooling to investigate anomalous events and alerts, detect malicious and anomalous insider activity, and reverse engineer malware, while personally applying the contextual judgment, escalation authority, and accountability that AI cannot provide on its own. The Principal AI Threat Detection Engineer serves as the subject matter expert for Insider Threat tooling, User and Entity Behavior Analytics (UEBA), Security Orchestration Automation and Response (SOAR) platforms, and the AI/ML models underpinning them, validating model outputs, tuning detection logic, and partnering with leadership to report on the business impact of theft, destruction, alteration, or denial of access to information. As the program shifts more first-pass analysis to AI, this role increasingly focuses human effort where it matters most: resolving ambiguous or high-stakes cases, overseeing AI-assisted decisions, and troubleshooting complex threats that impact the information security infrastructure at the data, application, service, operating system, and network levels. Qualifications - 8+ years of Security Engineering and Analysis experience, preferably in Threat Detection and Response, including demonstrated experience supporting Insider Threat programs - 5+ years of IT or networking experience - Hands-on experience administering, tuning, and optimizing Insider Threat detection tools, User and Entity Behavior Analytics (UEBA) platforms, and the AI/ML models that power them - Experience with SOAR tools and playbook development, including explicit expertise applying AI-driven and agentic automation to security operations and critically evaluating AI-generated findings for accuracy - Experience with security metrics and reporting, preferably including automating recurring metrics and reporting processes - Demonstrated ability to critically evaluate AI/ML model outputs - Intermediate to Advanced Linux/Unix OS and Windows knowledge - Firewall rule and policy, and network routing fundamentals - Ability to manage parallel tasks and accurately document resolutions - Proven ability to implement automation through scripting (e.g., PowerShell, PERL, Python, Bash scripting), including integrating AI/ML APIs or agentic frameworks into detection workflows - Experience leveraging APIs to integrate third-party tooling into an existing tool stack - Familiarity with cyber security frameworks such as NIST and MITRE ATT&CK, and awareness of emerging AI governance and security frameworks (e.g., NIST AI RMF, MITRE ATLAS) - Industry-recognized professional certification such as Security+, CBROPS, CSA, CEH, GSEC, SSCP Requirements - Lead the maturation of Blackbaud’s Insider Threat detection program toward an AI-augmented capability, including administration, tuning, and optimization of Insider Threat tools, UEBA platforms, and the AI/ML models underneath them - Perform intrusion and insider risk analysis using SIEM technology, UEBA behavioral analytics, AI-generated insights, reports, data visualization, log analysis, and pattern analysis, applying human judgment to validate AI findings and separate true signal from noise - Direct AI-driven hunting tools and agents to identify threat actor groups (external and insider) and their respective tactics, techniques, and procedures, personally leading the investigation of cases that require nuanced human judgment - Serve as the human escalation point and first responder for security events that AI and automated tooling flag but cannot fully resolve, via email, phone, and tickets across corporate user networks, data centers, and cloud environments - Own remediation of information security incidents, including insider threat investigations, ensuring AI-assisted findings are verified and contextualized before action is taken - Document and communicate findings, escalate critical incidents, and interact with lines of business, HR, Legal, and other stakeholders on sensitive insider matters, exercising the discretion and judgment that AI systems cannot provide - Design and build AI-driven and SOAR-based automated workflows within Detection Engineering, defining the guardrails, escalation thresholds, and human checkpoints that keep automation safe, effective, and improve analyst performance - Champion responsible use of AI-driven analysis and automation to enhance detection accuracy and accelerate triage, while evaluating model accuracy, bias, and drift to keep detection logic explainable and trustworthy - Document automation and AI model deployment processes, to include defining pre-build requirements, validation criteria, and human review checkpoints for high-risk decisions - Utilize AI and automation to build metrics and dashboards supporting the Insider Threat and broader detection program, applying human interpretation to translate outputs into decisions leadership can act on - Serve as a thought leader on the evolving division of labor between AI and human analysts – determining which decisions should be automated, which require human review, and how that boundary should shift as the program matures – as well as on new alert content, data correlation, and anomaly thresholds - Improve and challenge existing processes and procedures in a very agile and fast-paced cyber security environment - Keep current on the threat landscape, insider risk trends, cyber security developments, and emerging AI/ML techniques relevant to detection engineering - Adapt to fluid infrastructures, and evaluate, pilot, and integrate new AI-enabled technologies and platforms - Act as peer reviewer and technical escalation point within the core security engineering team, including reviewing AI-assisted detection logic and automation before deployment - Advise and inform leadership on how to optimize the current toolset and evaluate future tools, including UEBA, SOAR, and AI-enabled analysis platforms, recommending where automation should expand and where human oversight must remain Benefits - Medical, dental, and vision insurance - Remote-flexible workforce - Wellness Programs - 401(k) program with employer match - Flexible paid time off - Generous Parental Leave - Donations for Doers - Pet insurance, legal and identity protection - Tuition reimbursement program

Related Job Pages

More Machine Learning Engineer Jobs

Full TimeRemoteTeam 501-1,000Since 2005H1B No Sponsor

• Define the technical vision and long-term roadmap for Feed Retrieval, aligning large-scale recommender-system investments with Reddit’s product, ecosystem, and business objectives. • Translate broad Feed Relevance goals into a focused team roadmap, making clear prioritization tradeoffs across model quality, inventory expansion, experimentation velocity, infrastructure cost, and operational reliability. • Coach and support the development of your team, constantly seeking opportunities to grow their skills and impact. • Oversee the design, development, and optimization of retrieval systems that source relevant, diverse, fresh, and high-quality candidates for personalized feed experiences. • Establish strong measurement, experimentation, and debugging practices so the team can understand retrieval quality, candidate coverage, source incrementality, and downstream impact. • Collaborate with ML platform, infrastructure, ranking, safety, and product teams to build scalable, low-latency retrieval systems that can support the next generation of AI-powered recommendations. • Maintain high standards for system performance, reliability, latency, cost efficiency, and responsible recommendation practices. • Work with cross-functional partners from across the company to identify key areas of opportunity, set expectations, and communicate your team’s work. • Partner with our incredible recruiting team to attract, interview, and hire diverse and talented machine learning engineers, growing a world-class team.

United States
$253.3K - $354.6K / year
Full TimeRemoteTeam 501-1,000Since 2005H1B No Sponsor

• Define the technical vision and long-term roadmap for Feed Ecosystems. • Coach and support the development of your team. • Work closely with product, design, data science, safety, community, ads, and platform partners. • Oversee the design, development, and optimization of ML systems. • Help define and operationalize signals for subjective and objective quality. • Collaborate with platform and infrastructure teams to build scalable AI-powered systems. • Maintain high standards for system performance, reliability, efficiency, and responsible AI practices. • Partner with our recruiting team to attract, interview, and hire diverse and talented machine learning engineers.

United States
$253.3K - $354.6K / year

Role Description Have you earned your OSCP, OSEP/OSED/OSWE, and gained offensive experience before and/or since then? Are you excited at the opportunity to contribute to the growth and education for the current and next generation of cybersecurity professionals? If the idea of building lab environments to provide hands-on experience for individuals to learn and upskill, then this might be the right role for you! Duties and Responsibilities - Content design and build - Researches and identifies topical, relevant attack vectors suitable for inclusion in OffSec labs, exams, and/or learning content. - Design and build VMs and multi-host environments from vectors across Linux and Windows, including Active Directory and Cloud attack paths. - Designs realistic scenarios and environment narratives that make each attack path plausible and discoverable through enumeration. - Ensures the range of vectors in each environment is appropriate to its stated difficulty level and learning objectives. - Maintains variety across the catalogue so that content remains distinct as it rotates through active use. - Quality, integrity, and reproducibility - Review labs for unintended solution paths and confirms each is solvable by the intended route within its expected time frame. - Validates that machines are deterministic and reproducible in an isolated environment, including reliable reset and revert behaviour. - Deliberate selection and use of software and OS versions, monitoring deployed components so that content behaves consistently over its lifetime. - Documents the steps required to complete each machine, with difficulty assessed at a level of detail that supports competency mapping and certification review. - Collaboration - Coordinates with the relevant team(s) to deploy, update, and retire content. - Coordinates with testing to ensure full coverage of both newly created and updated content. - Coordinates with the Lab team Manager and Content Architect on vector selection and exploit implementation within courses and learning paths. - Coordinates and makes recommendations to keep content additions consistent across products. - Regularly communicates content additions, changes, and retirements to relevant stakeholders. - Senior scope - Acts as a technical reviewer for lab concepts and builds produced by other Vulnerable Machine Engineers and community contributors, providing structured, actionable feedback. - Support and mentor other engineers on build standards, documentation quality, and content review practice. - Contributes across OffSec's lab and exam portfolio as priorities require, rather than to a single product. - Automation and tooling - Creates and maintains automation that makes lab creation timely, consistent, and repeatable. - Evaluates new and emerging technologies to make recommendations on their introduction into the lab estate. - Creates and maintains documentation for every lab, to a standard that allows any team member to rebuild it from the documentation alone, without assistance. Qualifications - OSCP minimum, OSCE³ preferred (or at least one OffSec 300-level cert required). - A Bachelor’s Degree in Systems Engineering, Computer Science, Information Systems, and/or Information Assurance from an accredited institution/related specialized field, or equivalent practical experience. - Demonstrable Active Directory and Cloud attack-path experience. - Proficiency with infrastructure-as-code and configuration management and version control. - Scripting proficiency (Python, PowerShell, Bash). - At least five or more years of experience as a Systems Engineer or in an Info-Sec related position, with experience reviewing or approving others' technical security content. - Experience with Penetration Testing. - Experience with Bug Bounty Hunting. - Experience as a Systems Administrator with a variety of Operating Systems: - MS Windows - MS Windows Based Server Systems - POSIX Server Systems - Linux and Mac Desktop Environments - Strong written and verbal communication skills with an ability to present technical ideas clearly to both technical and non-technical audiences. Work Location and Hours This role is a full-time salaried position. It is a fully remote position. Work hours for this position are flexible and will be performed from a home office. Direct Reports This position has no direct reports. However, the expectations of this role are to provide technical leadership. EEO OffSec provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.

Worldwide
Upstart logo

Senior Applied ML Engineer

Upstart

Upstart is an online lending platform dedicated to offering fixed-interest personal loans at fair rates using criteria that go beyond borrowers’ credit scores

• Design and build user-facing ML features that harness LLMs and generative AI to unlock new product capabilities • Partner with product, design, and ML research to prototype and deliver high-impact, ML-powered experiences • Own the technical architecture and implementation strategy for applied ML systems - balancing latency, observability, and iteration speed • Build scalable services and APIs that bring model outputs to users in trustworthy and intuitive ways • Collaborate across platform, infra, and legal/compliance teams to ensure ML deployments meet standards for safety, fairness, and performance • Establish and evangelize best practices for prompt design, model evaluation, and experimentation across the org

United States
$167.7K - $231.8K / year