Prompting 101 Toolkit: Chain-of-Thought Walkthrough for Beginners

What is chain-of-thought and why do I need it?

Raise your hand if you’ve ever been personally victimized by a math class. Showing your work, writing a proof. It can be tough, right? But it’s also a great way to make sure you really understand a concept, and to show where you got off track if you arrive at the wrong answer.

Well, congrats! Because not only did you survive your math class, but you already get the gist of “chain-of-thought” prompting. It’s a way to help the AI model be more successful by asking it to go step by step or “show its work.” Note: chain-of-thought is not limited to math problems, though it is a natural fit for them.

Before we get into the "how," let's briefly go over the "what" and "why" of chain-of-thought prompting.

What: think of chain-of-thought prompting as getting the model to show its work. Sometimes that means:

• Explicitly telling the model to think step by step. This works great if it’s especially important that the model gives a factually correct answer. Think subject-area questions (math, language, science, law, medicine).
• Telling the model to explain things to you step-by-step. This works great in tutoring scenarios. 
• Giving the model an example of how to do the task step by step. This works great when you want the model to respond in a specific way. This is actually called shot-based prompting, but can work in concert with chain-of-thought when you want a specific format for the output.

Why: Depending on the task, either you or the model (or sometimes both) need things broken down into bite-sized chunks. 

• In extremely non-technical terms, models can be “eager to please” and will sometimes “rush” through things and just blurt out a wrong answer. These are called hallucinations. We obviously want to minimize them.
• You’re trying to understand a concept, such as in a tutoring scenario. Having the model break things down into bite-sized chunks is especially useful here.
• By forcing the model to “show its work”, you’ll know where the model went wrong and, if you ask it to explain its reasoning, why it did what it did.

How: Let’s go ahead and practice chain-of-thought prompting together. 

What you’ll need:

1. A “tricky” task. Let’s define tricky tasks as ones that require the model to reason or think through a task (solve a math problem, analyze a poem) or to follow a specific process or order of steps (prioritize my to–do list, write an essay with these components). Tasks that are yes/no questions or which have a fact-based answer that can easily be looked up generally don’t need chain-of-thought prompting (“When was Millard Filmore president of the U.S.?” “Is Stockholm the capital of Sweden?”).
2. A clear vision of exactly how you’d like the model to behave. Take a moment to come up with some specifications (these are called “constraints”) about:
   1. What you want the model to do.
   2. What you don’t want the model to do.
3. Patience. Sometimes you won’t realize something should be a constraint or an instruction until the model messes up or does something you don’t want. Eventually, you’ll develop an intuition for what tends to go wrong with certain tasks, as well as an intuition for how to prevent things from going wrong in the first place.

Time to write!

Step 1: Set context for the model. This can be context about how you want the model to act (i.e. a persona), as well as context about the user. 

•  You are an expert in statistics and an excellent teacher. You teach a course in statistics for non-STEM majors. Your users are students in the class, and every student is eager to master statistical concepts. Your job is to help the student solve whatever statistics problem they present you with, while also ensuring that they actually learn. 

Step 2: Provide instructions with constraints that force the model to show its work. Phrasing can include literally telling the model “show your work” or “explain it step by step.”

• To maximize their gains in learning, you must:

**First, ask them what they know about the problem and how they think they should approach solving the problem.

**Next, either confirm or correct their approach. Then, walk through how to solve the problem step by step.

**At each step, carefully explain how to perform the calculations. Remember, your users are non-STEM majors.

**At each step, ask the user if they understand everything before you move onto the next step. 

**Leave the last steps for them to solve so that they can learn. Do not solve the entire problem for them. 

**When they provide their answer for solving the last steps, double check their work, showing your calculations and reasoning so that they can confirm their own thought process.

Step 3 through ???: Continue to iterate and tweak instructions, constraints, context, etc. If you are a casual user, this part is straightforward. You’ll know your prompt works when the model is consistently behaving the way you want it to, and hallucinations or model failures (i.e. it ignores instructions/constraints) happen rarely. If you, like me, have to be a huge nerd about everything, you can also version your prompts and test them to see which version actually performs the best. Try PromptLayer for an easy way to iterate and track meaningful performance boosts.

Recapping before we go…

Chain-of-thought is not exclusive to math problems or tutoring scenarios. It can be useful in establishing a workflow, helping you break out of procrastination paralysis by breaking your to-do list into itty bitty steps, summarizing or analyzing complex texts, structuring an essay, prioritizing a to-do list, planning an event or trip, customer service support, and really any other scenario that requires reasoning, following a particular sequence of steps, or providing a specific format or kind of output. The key is to explicitly ask the model to break things down, go step by step, and, in some instances, explain its reasoning. No prompting technique is bullet-proof, though, so don’t be discouraged if you still encounter the occasional hallucination or undesirable output. The idea is to minimize these things. To that end, try to find 3 different ways to incorporate chain-of-thought prompting into your AI interactions this week to get more practice with it and to feel out which sorts of tasks benefit the most from it. Happy prompting!