Risk Labs Foundation logo
Risk Labs Foundation

Risk Labs is the foundation and core team behind UMA Protocol and its sister projects Across and Outcome.Finance.

Senior LLM Systems Engineer

Systems EngineerSystems EngineerFull TimeRemoteSeniorTeam 11-50H1B No SponsorCompany SiteLinkedIn

Location

Worldwide

Posted

1 day ago

Salary

$100K - $200K / year

Seniority

Senior

Bachelor Degree3 yrs expEnglishOraclePythonTypeScript

Job Description

Senior LLM Systems Engineer

Risk Labs Foundation

• We are hiring a Senior LLM Systems Engineer to own and improve the LLM-driven components of our oracle automation stack. • Focus on the accuracy, performance, resilience, and operational quality of the systems that use models to reason about wide ranging prediction market rules, evidence, and oracle outcomes. • Build the evaluations, observability, tooling, fallbacks, and feedback loops that make LLM behavior measurable and dependable in real-world conditions. • Improve prompts, model selection, tool usage, structured outputs, retrieval, and evaluation coverage so the system gets more decisions right over time. • Design validation, retries, fallbacks, uncertainty handling, and human review paths for ambiguous, adversarial, incomplete, or conflicting inputs. • Build datasets, regression tests, dashboards, traces, and review loops that make model quality visible and prevent repeated failures. • Help debug live issues, investigate regressions, improve runbooks, and reduce repeated operator friction.

Job Requirements

  • 3+ years of professional software engineering experience in Python, TypeScript, or similar production languages.
  • Hands-on experience building production systems that use LLMs, agents, retrieval, structured outputs, or model-powered workflows.
  • Experience designing evaluations, test datasets, regression checks, quality metrics, or manual review loops for AI systems.
  • Strong debugging ability across APIs, databases, queues, logs, model outputs, and external data sources.
  • Practical understanding of prompt engineering, tool calling, structured output validation, retrieval, and common LLM failure modes.
  • Ability to reason carefully about correctness in uncertain or adversarial environments.
  • High agency, strong ownership, and clear written communication.

Benefits

  • Pay packages include competitive salaries & meaningful long term equity participation
  • Salaries for this role range from $100-200k (USD)
  • Will pay in stablecoins or fiat
  • Philosophies for a culture that show we care: Take vacation when you need it, family care, training and development (just to name a few)
  • 100% remote, which means we encourage you to create the work environment that you thrive in
  • At least two team wide offsites a year

Related Categories

Related Job Pages

More Systems Engineer Jobs

ContractRemoteTeam 11-50H1B No Sponsor

• Design and deliver an intuitive, high-quality digital quote journey for HNW customers • Demonstrate early working outputs within weeks • Integrate with external and client backend services to enable downstream fulfillment processes • Establish a scalable delivery approach for iterative enhancement • Co-design and implement low-code business applications following OutSystems best practices and client architectural standards • Develop the frontend quote journey with intuitive user interface and seamless user experience • Implement screening logic against client "killer questions" • Build referral mechanisms for out-of-appetite applicants • Hands-on OutSystems application development across Traditional and Reactive Web platforms • Integrate with external services and client backend platforms: AddressCloud, Headless Drupal CMS, Policy Administration Platform (Merlin), Nexus API Integration Platform, Telephony Outbound Dialler • Design and develop REST APIs for seamless system communication • Ensure data flows correctly through all integrated systems • Develop and maintain unit tests and integration tests to ensure code quality • Create and update comprehensive technical documentation for all developed applications • Troubleshoot and resolve OutSystems-related issues • Ensure compliance with client architectural standards and best practices • Support Product Owners in backlog construction and refinement • Perform technical analysis and provide high-level effort estimations • Contribute technical insights and recommendations for continuous improvement • Collaborate closely with engineers, testers, and cross-functional team members • Review technical outputs and provide constructive feedback • Communicate effectively with all stakeholder levels, both verbally and in writing • Translate functional requirements into technical design and development tasks • Provide clear status updates and technical briefings to business and technical audiences • Ensure quality and timely delivery of all outputs

United Kingdom

MBSE Systems Engineer Staff

Lockheed Martin

Lockheed Martin is an international security company headquartered in Bethesda, Maryland. This company conducts research and designs, develops, and manufactures

Role Description Lockheed Martin is partnering with PG&E, Salesforce, and Wells Fargo to deliver EMBERPOINT™, an initiative designed to transform wildfire prevention, detection, and response across the United States. EMBERPOINT integrates artificial intelligence, advanced sensing technologies, real-time mission data, and cloud-based command-and-control capabilities to help detect fires earlier, improve operational coordination, and accelerate response and recovery efforts. To support this mission, Lockheed Martin Rotary and Mission Systems is seeking experienced Systems Engineers to join our Unified HMI team within the Digital Production Environment. This position supports a full-time REMOTE telework arrangement, allowing employees to perform their work schedule remotely outside of a Lockheed Martin designated office or job site. As a Systems Engineer supporting the Unified HMI platform, you will help design modern, mission-focused systems that fuse AI/ML insights, real-time operational data, and cloud-native infrastructure into a cohesive operational environment. You will create models, define requirements, and document architectures that support systems enabling the full Detection → Prediction → Response → Recovery workflow. Qualifications - Experienced in systems engineering, particularly in requirements generation, concept development, analysis, and simulation. - Proficient in interfacing with technical stakeholders and managing technical documentation and baselines. - Adept at preparing and delivering technical briefings and engineering products. Requirements - Bachelors degree from an accredited college in a related discipline, or equivalent experience/combined education. - Proficiency in Model Based Systems Engineering (MBSE) using tools such as Cameo, SysML, etc. - Experience using requirements tools (DOORS Next). - Comfortable using digital thread tools such as Jira, Confluence, JAZZ/ELM. Benefits - Medical, Dental, Vision, Life Insurance. - Short-Term Disability, Long-Term Disability. - 401(k) match. - Flexible Spending Accounts. - EAP, Education Assistance. - Parental Leave, Paid time off, and Holidays.

United States
$109.2K - $217.7K / year
Consumer Reports logo

Financial Systems Analyst

Consumer Reports

Consumer Reports was founded in 1936 with a vision to create a safer, healthier, and more transparent marketplace by empowering consumers with trustworthy infor

Title: Financial Systems Analyst Location: Yonkers, NY Job Description: WHO WE ARE Consumer Reports is an independent, nonprofit organization dedicated to a fair and just marketplace for all. CR is known for our rigorous testing and trusted ratings on thousands of products and services. We report extensively on consumer trends and challenges, and survey millions of people in the U.S. each year. We leverage our evidence-based approach to advocate for consumer rights, working with policymakers and companies to find solutions for safer products and fair practices. Our mission starts with you. We offer medical benefits that start on your first day as a CR employee that include behavioral health coverage, family planning and a generous 401K match. Learn more about how CR advocates on behalf of our employees. OVERVIEW CR’s Accounting/Finance team manages the organization's financial health and reporting. We’re hiring a Financial Systems Analyst to act as the "technical backbone" of the division, ensuring the integration, evolution, and maintenance of the financial ecosystem to support accurate data and strategic decision-making. In this position, you’ll report to the Manager, Financial Systems and will be focused on managing the end-to-end lifecycle of critical systems like Oracle E-Business Suite, and SAP Concur. The role is required to lead system upgrades, automate manual workflows to reduce "time-to-close," and serve as a technical partner to educate CR employees. This is a hybrid position with an expectation that you will be on-site at least one day per week. This role is not eligible for Sponsorship/Visa or relocation assistance. How You'll Make An Impact By ensuring the accuracy of financial data and the health of the financial ecosystem, this role enables CR to leverage its tools to their fullest potential. Modernizing financial operations and maintaining data integrity ensures the organization remains operationally excellent and strategically aligned with its long-term vision. On a regular basis you will focus on: - Providing technical support for Oracle Financials, Kofax Markview, Concur, and to the accounting department and other business users, including identifying opportunities for new and existing technologies. - Assisting with the monitoring and maintenance of systems supporting Oracle Financials, reviewing expense report exceptions, and aiding in upgrades and implementations of financial systems and their interfaces. - Troubleshoot complex technical problems and initiate resolution activities. - Writing computer programs, macros, and scripts to enhance and integrate technologies. - Participation in vendor meetings to assess new technologies and system upgrades. Developing and implementing strategic goals for the Finance division and performs other tasks as needed. ABOUT YOU You’ll Be Highly Rated If: - You have the education. Associate’s degree in Computer Science, Accounting, or a related field. - You have the experience. A minimum of two years of hands-on experience in the configuration or management of Oracle Financials E-Business Suite or similar systems. - You have familiarity with financial accounting and accounting workflows. - You know Excel. Strong proficiency in MS Excel, including experience with XLOOKUP, INDEX(MATCH), IFS, is essential. Editing VBA, is a plus. You'll Be One of Our Top Picks If: - You have an advanced degree. A Bachelor's degree is preferred for this role. - You know the systems and the environment. You have experience with Oracle E-Business Suite, Concur Travel & Expense and an understanding of Unix, Linux, and Windows environments. - You have experience in network infrastructure, integrating disparate systems, configuring Concur, and writing IBM Cognos Business Intelligence reports is advantageous. - You understand current technology and can resolve user problems. - You are collaborative and have the ability to work with others to achieve shared goals while valuing differences, offering support and help and sharing knowledge and expertise to help the team. FAIR PAY AND A JUST WORKPLACE At Consumer Reports, we are committed to fair, transparent pay and we strive to provide competitive, market-informed compensation informed by location, as well as the candidate’s particular combination of knowledge, skills, competencies, and experience. It is expected that most qualified candidates will fall near the middle of the posted salary range. We have three locations: Yonkers, NY, Washington, DC and Colchester, CT. We are registered to do business in and can only hire from the following states and federal district: Arizona, California, Connecticut, Illinois, Maryland, Massachusetts, Michigan, New Hampshire, New Jersey, New York, Texas, Vermont, Virginia and Washington, DC. Salary ranges - NY/California: $94K-$100K annually - DMV/Massachusetts: $90K-$95K annually - Colchester, CT and additional approved CR locations: $78,4K-$84K annually Consumer Reports is an equal opportunity employer and does not discriminate in employment on the basis of actual or perceived race, color, creed, religion, age, national origin, ancestry, citizenship status, sex or gender (including pregnancy, childbirth, related medical conditions or lactation), gender identity and expression (including transgender status), sexual orientation, marital status, military service or veteran status, protected medical condition as defined by applicable state or local law, disability, genetic information, or any other basis protected by applicable federal, state or local laws. Consumer Reports will provide you with any reasonable assistance or accommodation for any part of the application and hiring process.

New York
$94K - $100K / year
Golden Pet Brands logo

Sr. NetSuite Systems Developer

Golden Pet Brands

Golden Pet Brands is leading America’s pet health revolution by making longevity-focused nutrition simple for pet parents, and rewarding for pets, through product excellence, education, and storytelling.

Full TimeRemoteTeam 51-200Since 2026

Creating a world where pets live longer, healthier lives, from their very first meal. Learn more about our brands. We’re Looking For A(n): Sr. NetSuite Systems Developer Location: Remote or Hybrid (El Segundo, CA and/or Burbank/North Hollywood, CA) What You’ll Be Doing: The Sr. NetSuite Systems Developer serves as the senior technical owner of the company's existing NetSuite ERP environment, with primary focus on production support, minor enhancements, and workflow and script optimization. This role inherits an already-implemented NetSuite instance and is accountable for keeping it stable, performant, and aligned to evolving business needs — not for re-implementation. The Sr. Developer partners closely with the Business Systems Analyst, Salesforce and Deposco WMS support, and the iPaaS (Jitterbit) integration layer to ensure the order-to-cash, procure-to-pay, inventory, and financial flows operate reliably across the broader application stack. The role is expected to operate with significant autonomy, exercise sound technical judgment under change management discipline, and deliver durable fixes rather than tactical patches. - Serve as the senior technical owner of the existing NetSuite environment, providing Tier 2/3 production support for incidents, defects, and user-reported issues. - Triage NetSuite production incidents to root cause — including SuiteScript governance failures, scheduled job backlogs, workflow execution errors, and integration failures — and drive them to durable resolution. - Design, build, test, and deploy minor enhancements to SuiteScript 2.x (User Event, Client, Scheduled, Map/Reduce, RESTlet), SuiteFlow workflows, Saved Searches, custom records, and custom forms. - Optimize existing SuiteScripts and workflows for performance, governance unit consumption, and maintainability; refactor legacy 1.0 scripts to 2.x where appropriate. - Own day-to-day support of the NetSuite ↔ Deposco WMS integration, including order release, inventory sync, fulfillment confirmation, and exception handling. - Own day-to-day support of the Salesforce ↔ Jitterbit ↔ NetSuite integration, including order creation, customer sync, item sync, and refund/credit memo flows. - Monitor NetSuite scheduled jobs, Map/Reduce queues, and integration health; investigate alerts and degradation events, including post-upgrade performance issues. - Participate in NetSuite release readiness — review SuiteCloud release notes, identify impact to existing customizations, regression test in sandbox, and validate in production. - Coordinate sandbox refreshes, manage SuiteCloud Development Framework (SDF) projects, and promote changes from sandbox to production under documented change control. - Partner with the Business Systems Analyst to translate business requirements into technical designs, providing realistic level-of-effort estimates and surfacing risks early. - Author and maintain technical documentation, runbooks, and knowledge base articles covering customizations, integrations, recurring incident playbooks, and recovery procedures. - Identify recurring incidents and operational toil; design and implement permanent fixes, automation, or process improvements to reduce ticket volume. - Support NetSuite role and permission administration in partnership with the IT and security teams; uphold least-privilege and segregation-of-duties principles. - Contribute to the ERP separation roadmap and any subsidiary, instance, or carve-out work that touches NetSuite configuration, scripting, or integrations. - Provide technical input on third-party SuiteApps and bundles under evaluation; validate fit, risk, and integration impact before adoption. - Adhere to IT general controls (ITGC), SOX-adjacent change management, and security guidelines applicable to ERP environments. - Provide on-call or after-hours support for high-severity NetSuite incidents and pre-planned change windows as required. - Monitor ticket queue health, identify trends, and contribute to KPIs/SLAs for the Business Systems function. Qualifications (Note: We strongly encourage you to apply even if you don’t tick ALL of these boxes.): - Advanced NetSuite Development: Expert-level proficiency in SuiteScript 2.x, SuiteFlow, Saved Searches, SuiteAnalytics, and SDF; competent in SuiteScript 1.0 for legacy maintenance. - Production Support Discipline: Diagnose and resolve issues in a live ERP environment with calm judgment, clear communication, and durable fixes; differentiate between tactical workarounds and root-cause solutions. - Integration Troubleshooting: Trace issues across NetSuite, iPaaS (Jitterbit), Salesforce, and Deposco WMS; identify where a failure originates and which system owns the fix. - Change Management: Operate under documented sandbox-to-production promotion, release windows, and ITGC/SOX-adjacent controls; never deploy untested changes to production. - Performance Optimization: Identify and resolve SuiteScript governance issues, slow saved searches, long-running scheduled jobs, and inefficient workflows. - Documentation: Produce clear, accurate technical documentation, runbooks, and knowledge base articles that allow other team members to support and extend the system. - Escalation & Collaboration: Recognize when to escalate to NetSuite Support, integration vendors, or internal infrastructure teams; provide the context needed to expedite resolution. - Business Acumen: Understand the order-to-cash, procure-to-pay, and inventory flows well enough to assess business impact of technical issues and prioritize accordingly. - Communication: Translate technical issues into business impact for non-technical stakeholders; communicate concisely and precisely with technical peers. - Compliance & Security: Adhere to ITGC, SOX-adjacent change controls, and segregation-of-duties principles; model good security hygiene in role design and customization choices. - Ownership & Judgment: Operate with minimal supervision; exercise sound judgment on priority, scope, and risk; protect production stability. - Evidence-Based Decision Making: Ground technical recommendations and incident root cause in logs, data, and reproducible findings rather than speculation. - Required Education: High school diploma or GED equivalent - Preferred Education: Bachelor's degree in Information Technology, Computer Science, Information Systems, or related field (or equivalent work experience) - Required Experience: - 5+ years of progressive NetSuite development experience, including hands-on SuiteScript (1.0 and 2.x), SuiteFlow, Saved Searches, and SuiteAnalytics - Demonstrated experience supporting and enhancing an already-implemented NetSuite environment (production support model rather than greenfield implementation) - Strong working knowledge of NetSuite order-to-cash, procure-to-pay, inventory, and financial modules - Hands-on experience integrating NetSuite with a warehouse management system (Deposco, NetSuite WMS, HighJump, Manhattan, or similar) via APIs or middleware - Hands-on experience integrating NetSuite with Salesforce, including order, customer, and item synchronization - Working experience with an iPaaS platform (Jitterbit, Celigo, MuleSoft, Boomi, or similar) for NetSuite integrations - Experience designing, debugging, and optimizing SuiteScript 2.x — User Event, Client, Scheduled, Map/Reduce, and RESTlet script types - Demonstrated ability to triage and resolve production incidents in NetSuite, including script governance failures, scheduled job backlogs, and integration errors - Experience operating under change management discipline (release windows, sandbox-to-production promotion, documented testing) - Preferred Experience: - NetSuite SuiteFoundation, SuiteCloud Developer II, or Administrator certification - Direct experience with Deposco WMS and the NetSuite ↔ Deposco integration pattern - Direct experience with Jitterbit Harmony, including Salesforce ↔ NetSuite mappings - Experience in a multi-subsidiary, multi-brand NetSuite environment, including OneWorld - Experience supporting a NetSuite instance during an ERP separation, carve-out, or instance split - Experience with Shopify or other DTC storefronts feeding orders into NetSuite - Familiarity with NetSuite Advanced Revenue Management, subscription billing, or revenue recognition - Familiarity with ITGC, SOX, or similar control frameworks applied to ERP change management - Scripting outside SuiteScript (Python, PowerShell, or JavaScript/Node.js) for adjacent automation - Familiarity with ITIL or similar service management frameworks Golden Perks & Benefits: - Comprehensive healthcare coverage – We cover 100% of premiums for medical, dental, and vision plans for employee-only plans. - Annual bonus - We match up to 3.5% of your 401k contributions, ensuring your retirement savings grow alongside your career. - 6 paid sick and mental health days, an Employee Assistance Program, free weekly yoga and meditation classes, and additional mental health benefits, because we take a holistic approach to your well-being. - Invest in your future with our Employee Stock Ownership Plan, where company contributions fund your retirement plan with company stock. Learn more. - Reimbursements for a portion of personal cell phone and internet usage for eligible employees. - Professional development opportunities with mentorship, continuous learning programs, and performance-based pay increases, empowering you to progress and excel in your career. - Through our charity sponsorships, you can make a positive impact on communities worldwide. We’ve donated over $9 million to date. - Potential for quarterly KPI bonuses. The base salary range is $91,500-$122,000, plus bonus potential. The posted salary range in this job posting reflects data based on California's cost of labor analysis. This salary range is subject to change per state; please click here to find the pay differential in your state of residence if the role is remote. Salary is based on a wide range of factors that include relevant experience, knowledge, skills, other job-related qualifications, and geographical location. We will consider for employment all qualified applicants with criminal histories in a manner consistent with applicable law. A Note to Recruiting Agencies: At Golden Pet Brands, we handle all hiring through our internal Talent Acquisition team. Occasionally, we might use staffing or recruiting agencies to support our internal team in finding the right candidates. Agencies are not authorized to contact our hiring teams directly to present candidates. Our internal Talent Acquisition team is responsible for all candidate presentations to our hiring managers. Golden Pet Brands will need written approval and a signed agreement before submitting any candidate info for a specific job opening. Without this, we won’t pay any fees for placements. Thank you for your understanding. Candidate Safety: The safety of our candidates is our highest priority. When looking for a job, please be aware of cyber criminals, fake domains, and fraudulent job offers. Golden Pet Brands recruiters will only contact applicants from our official company e-mail domain, (NAME@GOLDENHIPPO.COM or NAME@GOLDENPETBRANDS.COM) or through our internal Applicant Tracking System, Greenhouse. In addition, Golden Pet Brands recruiters will never send you checks, or ask you to disclose personal financial details. If you receive any suspicious communications regarding an open position or a job offer, please contact Golden Pet Brands directly at safety@goldenhippo.com to verify its validity. Click here to learn more about general internet safety and our hiring practices.

California
$91.5K - $122K / year