Prompts must make clear what “shape” of answer they expect

A subtle challenge of prompt-writing is that a prompt must make clear “what it’s looking for.” For instance, this prompt has a missing segment… what’s supposed to go there?
Question: In the short run inputs are variable.
Answer: some
Cloze deletions tend to exacerbate this problem, since they have to use syntax structure to suggest the form of the answer. (see also Cloze deletion prompts seem to produce less understanding than question-answer pairs in spaced repetition memory systems)

Orion J Anderson:

An experience I had with a lot of the questions was that I would read your prompt and be uncertain what answer you were expecting. I would have a concept or quote from that segment in my mind, though. When I clicked to reveal the answer, I would see that the phrase that I had recalled was indeed the answer you had in mind, I simply hadn’t understood how or why it was the answer.

Soren Bjornstad points out that poorly written prompts require that you also memorize “what the prompt is asking.”

Related: Spaced repetition memory prompts should unambiguously produce a specific answer

Last updated 2023-07-13.