• Reporting to our Director of BI & Analytics, you will join our data team as a key internal consultant and technical expert. Whether you are a seasoned architect or a rising talent in the analytics space, your primary mission is the same: to move beyond simple reporting and transform complex data into compelling data stories. • This is a high-impact role where technical rigor meets design strategy. You will serve as the bridge between our data infrastructure and our stakeholders, owning the end-to-end data lifecycle. This includes architecting scalable data models and ELT pipelines, as well as partnering directly with business lines to build polished, intuitive Tableau dashboards. • If you are a versatile professional who enjoys both writing complex Python/SQL transformation logic and crafting data stories that answer the "so what?" for leadership, we encourage you to apply. • What you'll do: Visualization & Storytelling • Tableau Design: Design and build highly interactive dashboards that guide users through a clear narrative flow. You don’t just display data; you highlight trends, anomalies, and actionable insights. • UX/UI Strategy: Apply design best practices (layout, color theory, pre-attentive attributes) to ensure dashboards are intuitive, consistent, and reduce cognitive load. • Performance Tuning: Proactively monitor and optimize both SQL queries and Tableau workbooks to ensure fast load times and a seamless user experience. • Data Engineering & Architecture • Data Modeling: Own the design and optimization of dimensional data models (Star Schemas) in BigQuery to create a clean, accessible, and performant "semantic layer" for analytics. • Pipeline Development: Design, build, and maintain scalable ELT / ETL pipelines (using SQL, Python, and orchestration tools) to transform raw data into analytics-ready datasets. • Data Quality & Governance: Establish and advocate for data integrity by implementing automated testing, validation frameworks, and consistent metric definitions. • Strategy & Consultation • Internal Consulting: Act as a trusted advisor to stakeholders. Translate vague business questions into strict technical requirements and analytical stories that get to the root of the problem. • Mentorship: Foster a culture of data literacy by mentoring business users on dashboard interpretation and training junior analysts on SQL best practices. • Documentation: Maintain clear technical documentation for data lineages, metric definitions, and pipeline logic.

• Evaluation Strategy & Planning: Define and establish comprehensive evaluation strategies for new AI Agents. Prioritize the integrity and coverage of test data sets to reflect real-world usage and potential failure modes • LLM Output Integrity Assessment: Programmatically and manually evaluate the quality of LLM-generated content against predefined metrics (e.g., factual accuracy, contextual relevance, coherence, and safety standards) • Creating High-Fidelity Datasets: Design, curate, and generate diverse, high-quality test data sets, including challenging prompts and scenarios. Evaluate LLM outputs to proactively identify system biases, unsafe content, hallucinations, and critical edge cases • Automation of Evaluation Pipelines: Develop, implement, and maintain scalable automated evaluations to ensure efficient, continuous validation of agent behavior and prevent regressions with new features and model updates • Root Cause Analysis: Understand model behaviors and assist in the trace and root-cause analysis of identified defects or performance degradations • Reporting & Performance Metrics: Clearly document, track, and communicate performance metrics, validation results, and bug status to the broader development and product teams

• Define and establish comprehensive evaluation strategies for new AI Agents. Prioritize the integrity and coverage of test data sets to reflect real-world usage and potential failure modes • Programmatically and manually evaluate the quality of LLM-generated content against predefined metrics (e.g., factual accuracy, contextual relevance, coherence, and safety standards) • Design, curate, and generate diverse, high-quality test data sets, including challenging prompts and scenarios. Evaluate LLM outputs to proactively identify system biases, unsafe content, hallucinations, and critical edge cases • Develop, implement, and maintain scalable automated evaluations to ensure efficient, continuous validation of agent behavior and prevent regressions with new features and model updates • Understand model behaviors and assist in the trace and root-cause analysis of identified defects or performance degradations • Clearly document, track, and communicate performance metrics, validation results, and bug status to the broader development and product teams

• Design and implement efficient data pipelines using Python and AWS services (Lambda, S3, Glue, etc.) • Ensure data quality, reliability, and scalability across multiple sources and formats • Collaborate closely with data analysts, software engineers, and product teams to deliver actionable insights • Contribute to continuous improvement of our data infrastructure and best practices