Research institute investigating the trajectory of AI
Researcher, Evaluations
Location
United States
Posted
10 days ago
Salary
$115K - $200K / year
Seniority
Senior
Job Description
Researcher, Evaluations
Epoch AI
• Create and curate an evaluation suite. Find real-world tasks that serve as challenging tests for practical AI capabilities, and update the tasks over time as AI capabilities evolve. Devise rubrics for evaluating AI performance. - Evaluate AI systems. Regularly evaluate new, notable AI models and products on the task suite. Update tasks and rubrics to reflect the changing landscape of AI capabilities. - Communicate your research. Create public-facing reports, blog posts, and data visualizations with your observations. Ensure the evaluations feed into our other research topics and help keep our team informed. - Conduct data analysis. Analyze evaluation results and compare models across tasks. - Improve the process. You might automate parts of the workflow, and build out parts of the evaluation into standalone benchmarks.
Job Requirements
- Analytical thinking. **You conduct experiments with rigor and care, making sure that findings are well-supported by evidence.
- Grounded, skeptical mentality. **You form your own well-reasoned view of what an AI system can do, distinguishing practical capabilities from hype.
- Comfort with AI agents and tools.** You have experience working with AI agents in the course of your own work, and are comfortable delegating tasks.
- Familiarity with AI benchmarks and evaluations.** You follow AI capabilities at least casually and have opinions on what benchmarks do and don’t tell us.
- Research and data-analysis experience,** including enough comfort with light coding to analyze your own results.
- Strong written communication skills:** You can convey nuanced observations clearly and precisely.
Benefits
- Annual salary between **$115,000 – $200,000 USD**, depending on location and experience.
- Fully remote environment, including flexible work hours.
- Competitive global benefits program, including a comprehensive health insurance program—including supplemental benefits specific to a local country, as available and mandated by local law—and life insurance and a pension plan, if applicable in your country.
- Generous paid time off (PTO), including no specific annual limit, with 30 days PTO per year protected, unlimited personal and sick leave, and 4 months paid parental leave for permanent staff with at least 12 months of tenure (prorated parental leave if less than 12 months).
- A flexible and generous expense policy for you to spend on equipment and a large range of productivity tools or learning/development opportunities, including unlimited spending on AI tools, subject to regulations and manager approval.
- Paid work trips, including 3 staff retreats per year and relevant conferences.
- Access to our very well-equipped offices in Berkeley, California, including paid meals, snacks, gym, and more. All staff, independently of where they are based, have access to the office for at least 20 days each year.
Related Guides
Related Categories
Related Job Pages
More Researcher Jobs
• Transform raw institutional data from various campus systems into actionable reports and interactive dashboards. • Support federal and state compliance reporting (including IPEDS, institutional and program accreditations, and NYSED). • Oversee internal retention tracking, and provide data-driven insights to advance student success and institutional outcomes. • Assist in the governance, validation, and auditing of data inputs and outputs within enterprise systems (e.g., Slate CRM, Colleague ERP). • Provide regular data reports to support the operation of the College’s academic enterprise (e.g., academic standing, course outcomes). • Support the AVP for Institutional Effectiveness in the design, administration, analysis, and dissemination of institutional surveys.
• support zoning due diligence projects focused on commercial real estate transactions • conducting comprehensive research of property details, zoning classifications, and municipal ordinances • sending Freedom of Information Act (FOIA) requests to municipal jurisdictions throughout the country • collaborate with the Zoning Project Managers to provide accurate, timely information that informs critical site analysis • research property information available from assessor websites & publicly available GIS Maps • FOIA (Freedom of Information Act) submission to Local/County/State Agencies • request and obtain copies of municipal documents such as site plans, surveys, variances and conditional/special use permits • collaborate with Zoning Project Managers to ensure correct property details • coordinate with municipal departments to obtain information regarding outstanding code violations on file for the Subject Property • coordinate payments for copies of the aforementioned municipal documents/information requested via FOIA process • manage & track a list of follow-up tasks to ensure that all requested information/documents are received from the municipalities • stay current with industry best practices for property research • maintain comprehensive databases of municipal contacts
• Build and maintain systems for tracking ideas through development • Identify researchers and institutions we should know, build relationships with them, and keep the team informed about new opportunities • Find the right people and places for each story — and handle the outreach needed to make access happen • Stay across the latest research and discoveries — and flag developments that could become great Veritasium stories • Spot new stories, write pitches and present ideas to the team
Title Researcher
Long & Foster CompaniesBecause you don't just want to live in it, you want to love it. Long & Foster. For the love of home.
• Research real estate titles to produce a title commitment and/or title certificate • Update title abstracts to register property • Provide title evidence to examiners, attorneys or outside title company to facilitate real estate closings • Prioritize orders based on closing date • Acquire documents at district court, bankruptcy court and other appropriate sources • Research real estate records, title problems and legal descriptions • Transcribe/copy pertinent information into a format usable for processing, examining and closing personnel • Provide customer service and problem-solving to all other departments and answer all inquiries regarding title searches
