Role Description CODEplus is currently seeking an AI/ML Engineer to support its operations in Atlanta, GA or Washington D.C., or Remote. Preference will be given to candidates located in Atlanta, GA or the D.C. Metro area. True innovation happens where machine learning meets cloud technology and real-world impact. As an AI/ML Engineer, you’ll join a collaborative team of technologists, data scientists, and stakeholders to tackle meaningful challenges using ML, Generative AI, and modern tools. You’ll contribute to building and scaling intelligent systems—from core ML models to chatbot and Retrieval-Augmented Generation (RAG) applications. With strong skills in Python, SQL, and cloud platforms like Azure, you’ll help deliver practical, forward-looking solutions in dynamic environments. Bring your technical expertise, curiosity, and customer-first mindset to help shape the future of intelligent systems across industries. Core Responsibilities: - Design, develop, and deploy scalable machine learning models and AI-driven solutions to address complex business and operational challenges. - Build and enhance Generative AI, LLM, and Retrieval-Augmented Generation (RAG) applications, including chatbot and conversational AI capabilities. - Develop and optimize data pipelines, feature engineering workflows, and large-scale data processing solutions using Python, SQL, and Spark. - Implement and support MLOps practices, including model training, deployment, monitoring, and lifecycle management using tools such as MLflow and Azure cloud services. - Collaborate with cross-functional teams and stakeholders to deliver customer-focused AI/ML solutions while staying current on emerging technologies and industry best practices. Qualifications - 3+ years of experience designing, developing, and deploying machine learning models. - 3+ years of experience with Generative AI, LLMs, or RAG applications. - 4+ years of hands-on experience with Python for ML and data engineering. - Experience with SQL for data manipulation and feature engineering. - Experience with big data tools such as Apache Spark. - Experience with Databricks and MLOps tools like MLflow. - Experience with cloud, preferably in Azure. - Ability to exhibit strong communication and customer-facing skills. - Ability to thrive both independently and in cross-functional teams. - Ability to problem solve and stay current with emerging ML trends. Nice If You Have - Experience with full-stack development or deploying end-to-end ML applications. - Experience with chatbot development or conversational AI. - Experience fine-tuning large models. - Experience with deploying ML solutions using MLOps pipelines. - Knowledge of Agile workflows and tools like JIRA.

• Build and deliver AI solutions: Take ownership of developing delightful, high-performance AI features to help users discover, organize, and optimize access to large datasets. • Rapid experimentation and iteration: Implement a highly iterative process where you quickly prototype, test, and validate with real users, including shipping and evolving LLM- or agent-powered workflows for the data engineering lifecycle. • Collaborate: Work with the rest of the team to shape AI-driven product features, including the integration of agentic components with internal tools like Slack and alerting systems while engaging with internal data teams for dogfooding. • Utilize AI tools effectively: Use AI and automation tools to enhance both product functionality and your own development workflows. • Effective communication: You’ll be working in a highly dynamic and collaborative environment, so we need someone who can communicate effectively and contribute across teams. • Ownership and impact: Take full ownership of the AI solutions you develop, ensuring they are not only innovative but also scalable, maintainable, and aligned with real user workflows.

• Architect and scale our GTM systems across sales, marketing, and product • Lead development of AI agents and workflows that automate key parts of the funnel • Orchestrate tools like Salesforce and HubSpot into a unified system • Turn product and user data into actionable growth signals • Build intelligent workflows (lead scoring, routing, outbound triggers, onboarding flows) • Define how AI is used across the entire customer lifecycle • Identify bottlenecks and design systems to remove them • Partner closely with founders on growth strategy, experiments, and priorities

Role Description Deltek is seeking a Principal Automation Engineer with deep expertise in AI-native test automation to help shape the quality engineering foundation of Deltek’s next-generation, AI-first ERP platform for project-based businesses. This is not a role for someone who automates feature regression. It is a role for someone who can harness AI tools to build intelligent automation frameworks that reason, adapt, and self-heal. You will be the automation architect behind Deltek’s in-house AI-native test automation platform combining Playwright with LLM-powered agents (Planner, Generator, Healer). You will extend, evolve, and industrialize this framework, integrating AI tools at every layer: test generation, self-healing selectors, LLM-as-a-Judge evaluation, and CI/CD-gated quality pipelines. If you are fluent in Playwright, agentic AI workflows, and modern test engineering — and want to build something genuinely new rather than maintain legacy frameworks — we invite you to join our team. ERP domain knowledge is a strong plus and will accelerate your impact. Responsibilities: - Architect and evolve the AI-native automation framework — extending Playwright-based agents with LLM-powered planning, test generation, and self-healing capabilities. - Use AI tools extensively (Claude, GitHub Copilot, LLM APIs) to design, generate, and augment automation suites — reducing human authoring effort while increasing scenario coverage. - Build and maintain Playwright agent pipelines for end-to-end workflow automation across Deltek’s Projects, Workforce Management, and Financials modules. - Integrate LLM-as-a-Judge (LLMaaJ) evaluation into the test pipeline to automatically score AI-generated outputs, detect hallucinations, and validate response quality against golden datasets. - Design and implement AI safety and correctness test cases: hallucination detection, bias testing, output guardrail validation, and behavioral consistency across edge cases. - Own the CI/CD automation pipeline (GitHub Actions / Azure DevOps) for AI-enabled releases — including regression gates, model-response validation, and automated quality dashboards in ReportPortal and Grafana. - Validate AI/ML outputs including prediction accuracy, recommendation relevance, natural-language responses, and inference API payloads. - Build and maintain golden datasets for AI drift detection, regression baselines, and LLM evaluation benchmarks. - Collaborate with Product Managers, AI/ML Engineers, and QE leads to define AI feature release quality gates and automation coverage targets. - Mentor QE team members on AI-assisted automation patterns, agentic testing concepts, and framework best practices. - Contribute to test strategy for data migration validation of schema fidelity and record correctness. Qualifications - BS/MS degree in Computer Science, Software Engineering, or a related field. - Relevant certifications in software quality, AI/ML, or cloud engineering are advantageous. - 8+ years of experience in test automation engineering, with at least 3+ years working with AI/LLM-based systems or agentic automation frameworks. - Proven hands-on experience with Playwright — including Playwright agents, fixtures, and API testing integration. - Demonstrated experience using AI tools (Claude API, OpenAI, GitHub Copilot, or equivalent) to accelerate test authoring, framework design, or output evaluation. - Track record of designing and implementing AI-based automation solutions — not just using automation tools, but building the frameworks others use. - Experience integrating automation into CI/CD pipelines (GitHub Actions, Jenkins, or Azure DevOps). - Experience with performance, scalability, or data-drift testing of AI features in production or pre-production ERP contexts. - ERP domain knowledge (Project Accounting, Financials, Payroll, Time & Expense) is a strong plus and will significantly accelerate onboarding and impact. Requirements - Familiarity with Ajera, Costpoint, Vantagepoint, or comparable project-based ERP systems. - Understanding of LLM fine-tuning, RAG pipelines, vector databases, and embeddings from a QA/validation perspective. - Experience building or working with self-healing automation frameworks or AIOps tooling. - Exposure to security testing for AI systems — prompt injection, output sanitization, guardrail bypass testing. - Familiarity with data privacy and compliance frameworks in AI-enabled enterprise software. Technical Qualifications - Deep, hands-on proficiency with Playwright — including agentic patterns, multi-step workflow automation, and integration with LLM backends. - Proficiency in TypeScript and/or Python for building automation frameworks, AI evaluation utilities, prompt-testing harnesses, and data-driven test pipelines. - Strong understanding of LLM/ML concepts from a QA perspective: prompt engineering, hallucination detection, output scoring, explainability validation, behavioral consistency testing. - Experience with REST and GraphQL API testing, including automated evaluation of LLM inference API payloads and AI-generated JSON responses. - Familiarity with ReportPortal, Grafana, or equivalent for test execution dashboards and quality metric visualization. - Strong SQL skills for data validation, training dataset verification, and ERP data pipeline testing. - Working knowledge of GitHub Actions and Azure DevOps (ADO/TFS) for CI/CD pipeline integration and issue tracking. - Good understanding of Agile/Scrum practices and AI model release cycles — shadow mode, A/B comparison, phased rollout validation. Soft Skills - Framework-builder mindset: thinks in systems, not scripts — builds what others use rather than executing what others built. - Strong communication skills: able to explain AI validation concepts clearly to engineers, product managers, and QE team members. - High ownership and self-direction: identifies automation gaps proactively and drives coverage improvements without waiting to be asked. - Collaborative and generous with knowledge: invests in mentoring team members and raising the team’s automation maturity. - Continuous learner: actively tracks the evolving AI tooling ecosystem and brings new techniques into the framework. - Able to manage multiple priorities in a fast-paced, distributed team environment. Benefits - Join Deltek and be at the forefront of how modern ERP quality engineering is done. - You will help build the AI-driven current automation framework into an industry-leading AI-native automation platform — combining Playwright agents, LLM-powered test generation, self-healing infrastructure, and intelligent quality gates.