Reading comprehension questions are unpleasant

Reading comprehension is a big issue: People often struggle to remember details of prose text because they never processed them in the first place. Adjunct questions are one proposed solution (see Adjunct questions improve comprehension of related but untested content). But I notice that when texts include basic reading comprehension questions, I find them terribly aversive.

They’re usually banal, rarely connected with what’s interesting about the material. They often feel condescending; I often feel I’m trying to “guess the teacher’s password”, contra e.g. Expert response heuristic, after Issa Rice. In sum, they feel like homework.

Unfortunately, this is especially true when the question pertains to material which I failed to process. The question just won’t seem to make much sense. I can tell that it’s asking me to produce some particular answer, but I can’t tell what kind of thing it’s looking for (because I never encoded the requisite information). So the question sends me on a wild goose chase of sorts, in which I flip back through the chapter looking for some passage which pattern-matches to terms in the question. Then I realize that I hadn’t really read the passage at all, and I may see what the questions asking… but it’s an ungraceful way to get there.

Here’s a representative example from Cognition - Reisberg, ch 2:

TEST YOURSELF

• What is the difference between structural imaging of the brain and functional imaging? What techniques are used for each?
• What do we gain from combining different methods in studying the brain?
• What is meant by the phrase “localization of function”?

That second question, in particular, really reeks of failing Expert response heuristic, after Issa Rice. Which could be OK… except that I think the book wants “the right” answer. The others just feel banal.

Spaced repetition memory system review sessions often feel boring, too, but my stance towards these questions is slightly different: I’m looking at them in the context of “I’m doing memory review”—i.e. “I’m trying to make sure I remember this stuff.” I guess my emotional stance is roughly that it’s not supposed to feel interesting; it’s supposed to feel like retrieval (most of the time).

See also some very interesting comments in Lessons learned from applying the mnemonic medium to non-technical texts; e.g. “reminds me of school homework”; “answering prompts is a very cold, grey thing to do.” My instinct is that these impressions are especially tied to Mnemonic medium prompts are interleaved into the reading experience.

Some adjacent forms aren’t unpleasant

One thing I’m very confused about is that I don’t mind responding to the REPL prompts in Execute Program, at all. In fact, it’s engaging, interesting, fun. If I hadn’t understood something, I re-read the adjoining text. I don’t mind that Execute Program’s lessons unfurl their prose in response to reader interaction. I find this behavior somewhat more obnoxious with Mathigon and Brilliant, I think because their “chunk” sizes are larger.

Likewise, I don’t mind doing problem set questions in textbooks. In fact, I enjoy them (when they’re well written). I appreciate what they do for my understanding. And I don’t mind doing this sort of thing in video games, except in the most sluggish and unimaginative tutorials.

In my U Chicago Basic Program courses, I don’t mind being asked what roughly amount to comprehension questions in the course of our discussions. Quoting from 2023-06-30 Patreon letter - Reading comprehension and memory systems:

I think the difference is mostly about social context. A real person I respect is asking the question; they’re going to genuinely engage with my answer; other students might build on my answer or have interestingly different answers; the facilitator will connect our answers to subsequent questions; etc. It’s also partially about the framing of my activity. I’ve shown up to have a discussion about the book, so that’s what I’m going to do. I’m “discussing”, not “answering boring comprehension questions.”

Mathigon

Looking at the graphs course.

A condescending question: “Every day we are surrounded by countless connections and networks: roads and rail tracks, phone lines, the internet, electronic circuits and even molecular bonds. There are even social networks between friends and families. Can you think of any other examples?”

I’m asked (before being taught) for the terminology “vertex” and “edge”. I’m not allowed to continue until I give the right answer. This is obnoxious because I know the answer. Being asked to find the order and degree of a graph or vertex feels more reasonable—closer to “real” exercises, closer to Execute Program’s REPL.

Really feels like Programmed instruction, particularly when I’m told that “Graphs that consist of a single loop of vertices are called cycles. Then asked: All cycles have…??” (the same number of edges and vertices) I don’t like it. I’m not convinced of this yet. It seems to be true… but no proof is given or asked for. Feels like guessing the teacher’s answer.

In “Bridges of Königsberg,” there’s a nice interactive exercise with bridges.

I hate the reactions of the fox bot thing.

This just feels like guess and check:

First, we need to convert the city maps into graphs with edges and vertices. Every island or region of land is represented by ??? and every bridge connecting two regions is represented by a corresponding ???.

I notice that I only half understood the final conclusion of the Bridges section, but the questions didn’t reveal that.

This is a really nice interaction:
Screen Recording 2023-07-04 at 11.41.37 AM.mov

This question feels fine: The bipartite graph with two sets of size x and y is often written as Kx,y. It has ??? edges.

Likewise, these interactive widgets which let me play with a graph and figure out if I think it’s planar:

Coloring in the US is awfully tedious.

It appears that you have to try changing numbers whenever they appear in paragraphs, in order to move on. This is also obnoxious.

My broad summary: questions which ask me to apply what the prose has explained, or to make an interesting inference, generally feel pretty good. Particularly when there are associated manipulatives. Questions which ask me to repeat what a paragraph said, or to try to guess where the prose is going (Programmed instruction-style), feel obnoxious.

Brilliant

Looking at Practice Scientific Thinking | Brilliant

Gears: why am I answering this question?

I like that Brilliant has explanations available for if you find a question confusing.

These questions seem on the whole less successful than Mathigon’s. But they’re also less formal, less precise.

Feels like guessing the teacher’s password:

The “review and reflect” material rarely seems to involve either.

I find the scrolling behavior unpleasant: my eye doesn’t quite know where to go.

A problem in Practice Hypothesis Testing | Brilliant required that I use pen and paper, but I didn’t feel like doing it. It felt like it was “in my way.”

My broad impression is that Brilliant’s questions often aren’t sufficient to ensure comprehension and understanding. They’re less demanding than Mathigon’s, so when they’re bad, they’re less obnoxious.