Job Closed
This listing is no longer active.
Learning & productivity hacks for the world’s smartest humans.
AI Prompt Engineer
Location
New York
Posted
114 days ago
Salary
$40 - $100 / hour
Seniority
Junior
Job Description
AI Prompt Engineer
Brainscape
• Migrate and test existing bulk flashcard creation prompts in an updated AI environment with newer GPT models - and plan future migrations as OpenAI retires older models • Run test suites and manually review AI outputs for quality and correctness (fine-tune prompts) • Analyze real user data to identify failure patterns and inform prompt improvements • Streamline testing and evaluation workflows to make QA faster and more repeatable • Monitor production quality post-launch and detect regressions as underlying models shift • Build and maintain model evaluation datasets from real user inputs across all AI features • Write new test cases for edge cases, multilingual content, and messy real-world inputs • Document prompt changes, test results, and lessons learned • Work with the Content Team to apply flashcard authoring quality standards
Job Requirements
- 1+ years hands-on prompt engineering experience with LLMs / OpenAI API (systematic testing and iteration, not just casual ChatGPT usage)
- Familiarity with Cursor IDE or similar AI-assisted development tools (our work is primarily Python - Cursor experience is more important than raw Python skill)
- Some experience with Git version control and collaborating via shared repositories (we use GitLab)
- A habit of documenting what you tried, what worked, and why - you don't need a formal QA background, but you naturally keep track of your process
- Clear written communication skills
- Proactive attitude; ability to work independently and manage your own time
- BONUS: Experience building prompt evals, AI quality assurance, or using GPT to grade GPT outputs
- BONUS: Experience with regression testing for AI systems or detecting model drift
- BONUS: Background in education technology (EdTech) or content creation - especially microlearning, flashcards, or other concise Q&A formats
- BONUS: A degree in Computer Science, Information Science, or a similar field
Related Guides
Related Categories
Related Job Pages
More Artificial Intelligence Jobs
Would you like to be paid while staying active and teaching and promoting life skills through basketball and other sports? We offer a competitive hourly rate, training on the job, a flexible schedule, and incentive programs. The Position: We are seeking coaches for our basketball program through the Boys and Girls club and various Kipp Schools. Amazing Athletes was founded in 2003 and has grown into one of the nation's largest multi-sport enrichment programs with over 50,000 kids enrolled each year. Work Hours and Compensation - The starting pay is $40, with an opportunity to increase up to $45. - The schedule will span from Monday to Thursday, with games taking place on Saturdays, contingent upon the respective school. Additionally, a tournament is scheduled for April 26th for certain schools. - The practice sessions are scheduled to take place from approximately 4:00 PM to 5:30 PM, with some sessions potentially occurring on Saturday mornings, depending on the school requirements. Where You'll Be Working: Classes take place throughout South Los Angeles and Central Los Angeles. Candidates must be prepared to travel based on the locations.
• Own and operate production AI-enabled systems deployed in AWS • Keep systems reliable, secure, and running smoothly as usage scales • Improve data pipelines so AI-driven insights remain accurate and trustworthy • Partner with GTM / RevOps to support tools that power how we go to market • Enhance existing systems based on real user feedback • Collaborate with IT to build the operational infrastructure that powers our intelligent systems. • Forge connections between new, cutting-edge tools (including AI-driven solutions) and our existing ecosystem. • Monitor data pipelines and automated systems, guaranteeing the reliability of the insights we deliver. • Empower the rest of the business by providing clear, accurate answers to their most pressing questions.
• Own and drive engineering productivity lift targets, measured through cycle time, code velocity, and defect reduction • Lead AI-driven cost reduction initiatives across IT and support functions • Conduct a full audit and rationalization of Rackspace's AI licensing footprint, eliminating duplicative programs and spend • Report quarterly on quantified productivity gains with results tied directly to compensation milestones • Define and launch 3–5 standardized AI sales plays segmented by vertical and workload within the first 6 months • Translate Rackspace's private cloud and AI runtime capabilities into packaged go-to-market motions • Embed AI positioning into key partner relationships including VMware, Palantir, Uniphore, and Rubrik • Equip the Sales organization with clear AI messaging, competitive framing, and active deal support • Establish tracking infrastructure for AI-influenced pipeline and revenue attribution • Monitor model releases, inference economics, developer ecosystem shifts, and AI governance developments on a continuous basis • Translate market shifts into concrete strategy adjustments within 30–60 days of identification • Advise the CEO, marketing leadership, and Board on AI competitive positioning and narrative • Drive AI messaging alignment across earnings communications, analyst relations, and partner narratives • Deliver a formal competitive brief to the CEO and Board on a quarterly cadence • Lead a lean AI Labs function with a bias toward speed and commercial relevance • Incubate 1–2 focused, defensible IP assets per year aligned to GTM priorities • Apply rigorous ROI discipline — kill low-return experiments quickly and reallocate resources • Ensure Labs output directly supports customer-facing revenue strategy, not standalone research
• Formulate benchmarking methods for high-performance computing and AI tasks. • Perform and bring these methods to completion on large-scale GPU clusters. • Assess performance metrics to detect optimization opportunities and upgrade architecture. • Develop and maintain telemetry infrastructure to capture performance data. • Collaborate closely with hardware engineering, software development, and customer-facing teams to define performance requirements, fix bottlenecks, and validate configurations against real-world workloads. • Deploy and manage observability stacks including monitoring tools like Prometheus, visualization platforms such as Grafana, NVIDIA's DCGM, and custom telemetry solutions to provide actionable insights into cluster health, utilization, and performance trends. • Work directly with engineering and collaborate with internal partners to understand their performance requirements, conduct on-site benchmarking engagements, and deliver detailed analysis and recommendations for workload optimization. • Maintain extensive knowledge of industry-standard benchmarks in advanced computing and machine learning fields such as HPL, HPCG, MLPerf, and NCCL tests. • Contribute to developing new benchmarking methodologies for emerging workloads.



