“Take a deep breath and work on this problem step-by-step.”
If you’re imagining this is an instruction to help you to work through “new math,” you’re close. According to researchers at Google’s DeepMind, this is the most effective way to prompt a large language model (LLM) working on a math problem.
In a paper from November 2023, researchers across multiple institutions discovered that including emotional stimuli in prompts leads to more success. The tested prompts used different psychological devices to intimidate, cajole, or encourage the model. These statements included comments like, “This is very important to my career” and, “Take pride in your work and give it your best. Your commitment to excellence sets you apart.” Testing across a range of different models showed a relative improvement in responses of 8%.
Another strategy for success: YELLING AT THE MODEL USING ALL CAPS. It’s also well established at this point that being polite in prompts leads to better results.
Why are these things working?
AI models reflect human beings’ feelings and preferences. We can see the evidence of this in the research and successful prompting, and it makes sense since models were trained on our content. We use caps to indicate important messages, so LLMs understand that. We prefer polite behavior, so LLMs respond to it better. We’re more successful when encouraged, so LLMs are too.
“We observed that impolite prompts often result in poor performance, but excessive flattery is not necessarily welcome,” reports the paper ‘Should We Respect LLMs?.’ “This finding reveals a deep connection between the behavior of LLMs and human social etiquette.”
This means there’s a limit to how much we can manipulate the LLMs with prompting sweet talk, but it also clearly indicates that we have our own limit. Most of us inherently feel put off when the AI responses are too supportive, leaning towards disingenuous.
When OpenAI refined its models last month based on user feedback, including “thumbs up/thumbs down” responses, the changes skewed “overly supportive” — and users noticed. The resulting “sycophantic interactions” were described as uncomfortable and unsettling.
Maybe it’s good news that most of the human race seems programmed to respond suspiciously to overly nice behavior.
What are the downsides?
Sam Altman, CEO of OpenAI, made a comment on X last month that using “please” and “thank you” in prompting has cost the company “tens of millions of dollars.” At first glance, that seems like a pretty big downside.
But in truth, all AI is tragically expensive. The costs to train large language models are astronomical not just in terms of money, but time and the environmental impact of electricity and water used by server farms. Ultimately, the “tens of millions” that go toward making our models more polite seem like a drop in the bucket. Altman himself called it money “well spent.”
After all, AI is for the benefit of humans, so it should use our human etiquette rules.
Of course, there’s also the fact that polite prompts tend to be better constructed prompts, meaning they’re more likely to be successful in general. So the rule of thumb for prompting your AI models is really quite simple: Write clear and polite instructions.
In some ways it’s comforting that the old adage still applies: “Treat others how you want to be treated.” Even if they’re a machine.
