How PromptLayer Enables Non-Technical Prompt Engineering at ParentLab

How ParentLab Builds Highly Personalized AI Interactions Using PromptLayer and Non-Technical Prompt Engineers

The following is a case study of how ParentLab uses PromptLayer.

ParentLab is on a mission to revolutionize parenting support through highly personalized AI interactions. By engaging parents in reflective conversations and providing timely, relevant resources, ParentLab aims to promote healthier family relationships and secure attachment. Achieving this level of personalization at scale requires a robust system for prompt management and collaboration between engineers and parenting specialists. That’s where PromptLayer comes in, enabling ParentLab’s content and engineering teams to efficiently collaborate on crafting the perfect AI persona.

PromptLayer has been a game-changer for us. It has allowed our content team to rapidly iterate on prompts, find the right tone, and address edge cases, all without burdening our engineers. This has been critical for creating an AI that truly connects with and supports our users.
John Gilmore, VP of Operations

The Product

At its core, ParentLab is an AI-powered app that guides parents to reflect on their children and parenting through an engaging conversational interface. The AI may start a daily check-in with a parent by saying, “I remember you mentioned the homework situation with Jimmy. Have you noticed any changes or improvements?” These personalized prompts encourage the parent to contemplate their specific challenges and successes.

Meet Bene, ParentLab’s Prompt Engineer

Bene Foley never thought she would be doing AI prompt engineering. With a background as a teacher for 16 years and in real estate for 4 years, her expertise in understanding people has been invaluable.

“There is a relationship between my background and prompt engineering. I want to get the AI to talk like a person would,” explains Bene. She sees her role as crafting the tone of the app’s AI voice and guiding it to respond appropriately across diverse scenarios. This requires extensive, iterative testing.

Prompt engineering is about finding then fixing the edgecases. In one instance, Bene noticed the AI trying to be excessively positive, even praising parents when they described potentially harmful actions. Finding this took a lot of iteration and fixing the prompt took even more.

Bene uses PromptLayer to rapidly iterate on prompts. She makes a change to a prompt in PromptLayer, loads up the ParentLab app that pulls down the latest prompt, tests a scenario, and then checks the PromptLayer logs. Despite having no prior experience with version control systems like git, Bene has found PromptLayer’s built-in version control intuitive, allowing her to track the evolution of prompts and triage real issues caused by prompt updates.

Building a Personalized AI Solution

John Gilmore has spearheaded ParentLab’s LLM strategy, and he recognized from day one that achieving highly personalized AI interactions would require a sophisticated prompt management system, but not necessarily a team of machine learning engineers.

The journey began by building a proof-of-concept “recipe database” in Notion of ParentLab’s prompts. John used no-code tools like Zapier to showcase to the engineers how prompts could be dynamically constructed while giving the product team visibility into the system. Unfortunately, like the Postgres database of prompts before it, the Notion database proved too clunky for the content team to efficiently iterate with.

Prompt Iteration and Deployment

Discovering PromptLayer was a breakthrough. PromptLayer’s CMS of prompt templates solved his problems perfectly. Content writers, like Bene, could edit a prompt visually and have it automatically propagate to the app without engineering handoffs. Prompts are decoupled from code, meaning releases are no longer blocked and prod/dev deployments could be handled visually through PromptLayer workspaces. The engineering team would pull prompts down programmatically at runtime.

Bene is able to edit a prompt visually through the PromptLayer prompt CMS and then load up the ParentLab app to try it out. Everything is logged, and versions can be deployed to staging or production. Learn more.

Iterating on prompts to find the right tone is not the job of engineers, who are generally not skilled at this task. Instead, PromptLayer empowers the content team to rapidly iterate & deploy through its visual CMS. This has enabled ParentLab to efficiently zero in on an AI voice that is both supportive and reflective.

Understanding User Behavior

As ParentLab grew, the next challenge was uncovering and addressing edge cases arising from the sheer diversity of human-AI interactions.

In the next phase of their AI development, ParentLab focused on understanding how users engage with the product and surfacing edge cases. The team realized that their extensive prompts, while effective, were becoming a form of technical debt (or as we call it “prompt debt”). John concluded that a more systematic approach was needed, transitioning to specialized prompts and AI models for specific tasks. PromptLayer’s CMS was perfect for this.

By analyzing interaction logs, the ParentLab team gained invaluable insights into user behavior and areas where the AI responses needed refinement. PromptLayer’s logging and analytics features have been critical for this process of continuous improvement.

Explaining the AI to Stakeholders

Bringing visibility to the behavior of the black box that is AI was a primary concern for the executives at ParentLab. PromptLayer meant engineers didn’t need to build this interface themselves, saving countless hours.

“PromptLayer has helped us bring visibility to our interaction with this alien intelligence that behaves differently from any technology you have used before,” John explains. Beyond addressing stakeholder worries, the logs and analytics have been instrumental for informing product decisions, identifying prompt edge cases, and understanding usage and costs as ParentLab scales.

PromptLayer’s analytics dashboard allows users to drill down into cost-per-user, cost-per-prompt, and much more. Learn more.

Most importantly, PromptLayer’s analytics dashboard allows stakeholders at ParentLab to track costs and prevent runaway user API spend.

Fine-Tuning and Next Steps

Looking ahead, ParentLab plans to explore fine-tuning models, price and speed optimizations, and further modularization of their prompt architecture. PromptLayer will continue serving as a critical infrastructure layer to enable ParentLab’s pioneering work in AI-human interaction.

PromptLayer’s Impact

Challenges faced by ParentLab:

• Build a personalized AI capable of interacting with sensitive user queries.
• Enable non-technical subject matter experts and content specialists to craft the AI’s tone.
• Surface analytics, logs, and controls to allow non-technical stakeholders to trust the AI feature.

Using PromptLayer, ParentLab achieved several key milestones:

• Finding the right supportive-yet-reflective tone for the AI required extensive prompt engineering, made possible by PromptLayer’s CMS for the content team to rapidly iterate.
• Bringing visibility to AI outputs helped address stakeholder concerns around safety and consistency.
• Aggregated interaction logs surfaced additional edge cases and unexpected user behaviors, informing product decisions and identifying areas where prompts needed further refinement.
• Gaining insights into usage and costs through PromptLayer’s analytics

Results

Deployed a new prompt 10x faster and without engineering assistance.

Over 400 engineering hours saved.

Ability to iterate >10x more leading to substantially better prompts.

In the first six months, 700 prompt revisions across 70 prompts.

Learn more about PromptLayer:

• Why use a prompt CMS? (Prompting best practices)
• Migrating Prompts to Open-Source Models with PromptLayer
• Building ChatGPT from scratch with PromptLayer
• Technical Documentation

PromptLayer is the most popular platform for prompt engineering, management, and evaluation. Teams use PromptLayer to build AI applications with domain knowledge.

Made in NYC 🗽 Sign up for free at www.promptlayer.com 🍰