Epoch AI

• Manage finance operations such as reviewing payment requests and disbursing payments • Manage Mercury and credit card operations, ensuring compliance of expense policies • Track and support various sub-budgets like team budgets, restricted grant budgets etc. • Keep books accurate and current in QuickBooks Online • Coordinate with external accountants and invoice clients • Prepare for and coordinate the annual audit • Own financial SOPs and accounting policy, support internal controls

• Review benchmarks: Assess a new benchmark at least every two weeks. You’ll evaluate their methodology and what a 'good performance' would imply about AI capabilities. • Publish and maintain reports: Write up public-facing reports on new benchmarks and periodically update them as new versions release and models make progress. • Dig into data: Examine individual tasks within benchmarks in detail, using coding agents to assist while retaining your own critical oversight.

• Create and curate an evaluation suite. Find real-world tasks that serve as challenging tests for practical AI capabilities, and update the tasks over time as AI capabilities evolve. Devise rubrics for evaluating AI performance. - Evaluate AI systems. Regularly evaluate new, notable AI models and products on the task suite. Update tasks and rubrics to reflect the changing landscape of AI capabilities. - Communicate your research. Create public-facing reports, blog posts, and data visualizations with your observations. Ensure the evaluations feed into our other research topics and help keep our team informed. - Conduct data analysis. Analyze evaluation results and compare models across tasks. - Improve the process. You might automate parts of the workflow, and build out parts of the evaluation into standalone benchmarks.

• Implement benchmarks: Implement AI benchmarks within our evaluation infrastructure (primarily using the Inspect library) to expand the suite of capabilities we track. Develop our existing suite of benchmarks so we can quickly and painlessly evaluate new model releases. • Develop new benchmarks: Contribute to the development of brand new benchmarks. You will have the opportunity to pitch and prototype your own ideas in addition to helping out with existing projects. • Collaborate: Work closely with researchers, analysts, and other engineers at Epoch AI to ensure evaluation data and outputs are accurate, insightful, and effectively integrated into our research products and publications.

• Define and own strategy across our website and data products • Establish design processes and quality standards for the organization • Lead design projects end-to-end, from problem definition through delivery • Design responsive user interfaces, graphs, and data visualizations • Conduct user research to help us understand audience interests and how they use our research • Create design specifications, mock-ups, and prototypes while contributing to our design library • Manage and mentor another designer and part-time contributors

• Design responsive user interfaces, graphs, and visualizations. • Uncover design solutions to user and company needs. • Create design specifications. • Make mock-ups and prototypes to explore and communicate design concepts. • Ensure adherence to design systems and visual consistency. • Depending on your interests and skills, you could get involved in other aspects of our work, such as contributing to our design library and conducting user research and usability testing.

• Implement new designs and features for the website. • Fix issues with specific features of the website. • Coordinate with web development and data teams to support the Frontier Data Centers research project.

• We’re seeking Researchers and Senior Researchers to help us improve society’s understanding of AI capabilities and their trajectories. • Key activities will include helping define our future research directions and kick off new investigations into the most important trends in AI. • Conduct research on frontier AI datacenters, AI supply chains, polling, benchmark analysis, earnings growth of AI companies, and AI models. • Write data insights, reports, and articles to communicate your research. • Help create and maintain data sets related to AI.

• Carefully parsing assigned machine learning papers or benchmark leaderboards and identifying key information. • Accurately extracting and compiling these technical details. • Producing informed estimates of metrics about the physical infrastructure supporting AI based on limited information. • Gathering data on the inputs, impacts, and details of AI, as well as carrying out analysis of the gathered data. • Reporting on and discussing findings with your supervisor.

Over time, we expect benchmarking support to gradually take up a greater proportion of your time, but at the beginning we expect the division will look like this: Work closely with our benchmarking lead, Greg Burnham, to find contributors for new benchmarks: experts across a wide variety of domains—math, science, and various industries—who can help create challenging and meaningful measurements of AI capabilities. This involves rapidly developing familiarity with diverse fields. Coordinate with experts to ensure benchmark contributions meet our quality standards and to improve their experience as contractors. Organize events such as symposiums to bring together field experts and help create new AI benchmarks. Learn about and provide support with the dozens of tasks and details that help make new benchmarks great. Support the data team properly secure and license data for our data center tracking , and help ensure we remain compliant with local regulations. Other operations and external communications tasks as needed. For example, you might own various inboxes for external queries by our audience, responding or triaging with the team as necessary. You avidly follow AI developments, e.g. you know what GPQA Diamond is without googling and might speak passionately about why benchmark saturation is a challenge in AI. You are energized by complex challenges. You can own projects without a clear playbook, break new challenges down into manageable steps, and come up with creative solutions. You actively seek feedback and incorporate it quickly, and give clear, constructive feedback to others, even your supervisors. Instinctive organization and conscientiousness. You don't have to work hard to stay on top of details; it comes naturally. Two or more years of prior work experience in roles with many moving parts. Great written and verbal communication skills, and comfort with direct and empathetic communication. Comfort with and excitement about working in a startup-like, fast-moving environment. Time is our scarcest resource! You're motivated by Epoch’s work and our mission of improving humanity’s understanding of AI development. Annual salary between $75,000–$100,000 USD, depending on experience and location. Salaries are not restricted to USD, and contracts and payments are usually in local currencies. Conversions are based on one-year average exchange rates. Fully remote environment, including flexible work hours. Competitive global benefits program, including a comprehensive health insurance program—including supplemental benefits specific to a local country, as available and mandated by local law—and life insurance and a pension plan, if applicable in your country. Generous paid time off (PTO), including no specific limit on PTO with 30 days per year protected, unlimited personal and sick leave, and 4 months paid parental leave for permanent staff with at least 12 months of tenure (prorated parental leave if less than 12 months). A flexible and generous expense policy for you to spend on equipment and a large range of productivity tools or learning/development opportunities you might find valuable, subject to regulations and manager approval. Paid work trips, including 3 staff retreats per year and relevant conferences. Access to our very well-equipped offices in Berkeley, California, including paid meals, snacks, gym, and more. All staff, independently of where they are based, have access to the office for at least 20 days each year.