Category Archives: L&B Blog

As I wrote last week, thinkers in edu-world often make strong claims at the expense of nuanced ones.

For example:

• “A growth mindset undergirds all learning” vs. “growth mindset is an obvious boondoggle.”
• “AI will transform education for the better” vs. “AI will make people dumber and schools worse.”
• “Be the sage on that stage!” vs “get off the stage to guide from the side!”

The list goes on (and gets angrier).

When researchers start digging into specifics, however, the daily experience of teaching and learning gets mightily complicated, and mighty fast.

All those strong claims start to look…well…too strong for their own good.

One extraordinary example of “digging into the specifics” can be found in Graham Nuthall’s The Hidden Lives of Learners. Nuthall put cameras and mics on students in New Zealand classrooms, and arrived at all sorts of astonishing conclusions.

Another recent study looks quite specifically — no, really specifically — at 4 teachers. The goal: to understand what part of their work helped students learn.

Here’s the story.

Time to Review

A group of scholars, led by Dr. Janet Metcalfe, wondered if students learned more from teachers’ responses to their mistakes than from teachers’ direct instruction. (You can learn more about the study here.)

A few important points merit attention right away.

First: the classroom sessions I’m about to describe are REVIEW sessions. The students have ALREADY learned the math covered in these lessons; the teachers are helping them review in preparation for a high stakes exam.

In other words: this study does not focus on initial instruction. It focuses on subsequent review.

Second: the students involved VOLUNTEERED to take part. They are, presumably, atypically motivated to learn math.

Keep these points in mind as you think about applying the ideas described below.

In this study, 4 teachers helped 175 8th grade students prepare for upcoming state math exams.

For half of the students, the teachers taught 8 lessons (“explicit instruction”) reviewing core math concepts that would be on that exam.

For the other half, the teachers responded to the misakes that students made on practice tests. That is: during 4 sessions, students took 45 minute math tests. And after each of those sessions, the teachers

“were instructed […] to focus on the students’ errors and to do whatever they deemed appropriate to ensure that the issues underlying the errors would not reoccur and that the students would learn from their errors.”

So, which review approach proved more helpful — the explicit instruction, or the learn-from-mistakes instruction? And, why?

An Envelope, and LOTS of Questions…

The answer to that first question — which kind of review proved most helpful? — is easy to answer.

Students in both groups learned math; they did better on the post-test than the pre-test.

The students in the “learn-from-mistakes” group learned more.

This straightforward finding leads to obvious questions. And — alas — those obvious questions are VERY tricky to answer.

For instance, “how much more did the students in the learn-from-mistakes group learn?” That’s a reasonable question. The answer takes some careful parsing.

Roughly speaking, students in the explicit instruction group increased their scores about 2% per hour of instruction.

For those in the learn-from-mistakes group, the answer depended on the teacher.

The least successful teacher helped students in this group improve 2% per hour of instruction. The most successful teacher helped students improve 5% per hour of instruction.

Of course, that last paragraph prompts another reasonable question: what was different about those two teachers? Why did one teacher benefit his/her students more than twice as much as their colleague?

Let the Sleuthing Commence…

The study’s authors spend a great deal of time — and crunch a great many equations — to answer that question.

For instance:

Maybe the teacher whose students learned more (let’s call her Teacher M) is just a better teacher than the one whose students learned less (Teacher L).

As the researchers point out, that explanation doesn’t make much sense. After all, in their explicit instruction sessions, both Teacher M and Teacher L helped their students equally.

(By the way: to simplify this blog post, I’m leaving out the two other teachers for now.)

Okay, maybe Teacher M did a better job of focusing on students’ mistakes, whereas Teacher L spent too much time focusing on questions that students got right.

Nope. This study includes quite an eye-watering graph to show that they both focused about the same on students’ mistakes.

As the researchers write: “all of the teachers taught to the errors of their students, and … the extent to which they did so did not predict student learning.”

So, what was the secret sauce?

The Perfect Combination

After a few more false leads, the study focuses on two moment-by-moment variables: the teachers’ focus, and the kind of interaction with the student.

Focus: did the teachers

“[dwell] upon how to solve the problem correctly,” or

“[delve] into the nature of the errors – why the students had made them, what the difficulty in the logic was, and/or how to recognize and circumvent such mistakes in the future”?

Kind of interaction: did the teachers explain/lecture, or did they discuss/interact?

With this pair of questions, at last, the study struck gold.

Teacher L — whose students learned relatively little — focused almost all her time on “how to solve the problem correctly.” While pursuing that goal, she divided her time equally between lecture and discussion.

Teacher M — whose students improved more quickly — spent almost all her time in discussion, with almost no time in lecture. While in this interactive mode, she divided her time more-or-less equally between solving problems and understanding the nature of the mistake.

This final insight allows us to make this claim:

Highly motivated 8th grade math students,

reviewing in preparation for a high-stakes exam,

learn less from explicit instruction and more from making and reviewing their mistakes,

as long as the teacher keeps those review sessions interactive,

and equally focused on “getting the answer right” and “understanding the nature of the mistake.’

Notice, by the way, all the nuance in this statement.

To emphasize just one point here: this study does NOT argue that “learning from mistakes” is better than “direct instruction” in all circumstances.

It argues that students learn more from mistakes when reviewing, as long as the teacher follows a very particular formula.

A Final Note

Heated battles in this field often get hung up on specific labels.

As I’ve written before, we do a LOT of arguing about benefits of “desirable difficulty” vs. “productive struggle” — an odd set of arguments, given that both phrases seem to mean the same thing.

This study was co-authored by (among other scholars) Robert Bjork — who helped coin the phrase “desirable difficulty.” For that reason, you might be surprised to learn that this study touts the benefits of “productive struggle.”

That is: the students took a test, they made mistakes, they wrestled with those mistakes, and they learned more. Their struggle (trying to understand what they did wrong) was productive (they improved on their test scores — and probably their understanding of math).

Of course, I could just as easily describe that process as “desirable difficulty.” The difficulties these students faced here — the test, the mistakes, the analysis — turned out to be beneficial — that is, “desirable.”

My own view is: don’t get hung up on the label. The question is: are the students both thinking harder and ultimately succeeding? If “yes” and “yes,” then this teaching approach will benefit students.

---

Metcalfe, J., Xu, J., Vuorre, M., Siegler, R., Wiliam, D., & Bjork, R. A. (2024). Learning from errors versus explicit instruction in preparation for a test that counts. British Journal of Educational Psychology.

Over on eX/Twitter, a debate has been raging — with all the subtlety and nuance of your typical Twitter debate. The opening salvo was something like:

“Despite what you’ve heard, all people learn the same way.”

You can imagine what happened next. (Free advice: look away.)

Despite all the Twitter mishegas, the underlying question is useful and important — so I’ll do my best to find the greys among the black-vs-white thinking.

Here goes.

Useful…

I suspect that this claim — “all people learn the same way” — got started as a rebuttal to various myths about “meaningful sub-categories of learners.” Alas, most of those proposed sub-categories turn out not to be true or useful.

• No, learning styles theory has not held up well.
• No, the theory of “multiple intelligences” has no useful teaching implications. (And Howard Gardner didn’t claim that it did.)
• No, “left-brain, right-brain” dichotomies don’t give us insights into teaching and learning.
• No, the Myers-Briggs Type Indicator doesn’t tell us how to manage classrooms or lesson plans. *
• My British friends tell me about some system to sort students according to different colored hats. (I do not think I’m making this up.)
• (I’ve written about these claims so many times that I’m not going to rehash the evidence here.)

Whenever anyone says “we can usefully divide students into THIS kind of learner and THAT kind of learner,” we should be highly suspicious and ask to see lots of research. (If you want to evaluate that research critically, I can recommend a good book.)

Well, the shortest rebuttal to this sort of claim is: “Those sub-categories don’t exist. ALL PEOPLE LEARN THE SAME WAY.”

Now, any time someone makes an absolute claim about teaching and learning in six words and seven syllables, you know that claim is oversimplified.

But you can understand the temptation to cut off all those untrue claims with a brusque rejoinder. That temptation pulses all the stronger because those untrue claims persist so stubbornly. (In 2025, schools of education are STILL teaching learning styles.)

…and (substantially) True

This claim (“all people…”) isn’t simply useful; it’s also largely accurate.

For example:

At the neuro-biological level — neurons, neurotransmitters, synapses, myelin, etc. — long-term memories form the same way for everyone.

As far as we know…

• men and women
• tall people and short people
• introverts and extroverts
• people who think cilantro tastes like soap, and the rest of us

… everyone forms new neural networks (that is: “learns”) the same way. (I should emphasize that our understanding of this neural process is still VERY basic. We’ve still got SO MUCH to learn.)

When we switch our analysis from neuroscience to psychology, the claim still holds up well.

For instance:

• Everyone uses working memory to combine new information from the environment with concepts and facts stored in long-term memory.
• Everyone depends on a complex of systems that we call “attention” to control the flow of all that information.
• Everyone responds simultaneously with emotion and cognition to any given set of circumstances. (These two systems overlap so much that distinguishing between them creates lots o’ challenges.)

And so forth.

Given all these similarities, cognitive science research really can offer up advice that applies to almost everyone in almost all circumstances.

Yes: we really must manage working memory load so that students can build concepts effectively.

Yes: retrieval practice helps almost all learners consolidate and transfer almost all school learning. (Yes, “retrieval-induced forgetting” is a concern, but can be managed if we strategize effecively.)

Yes: spacing and interleaving enhance learning in most circumstances.

And so on…

Given the broad usefulness and truth of the “we-all-learn-the-same” claim, I certainly understand why it’s tempting to make it — and to defend it.

Exceptions Matter

I’ve written that the claim is “broadly” useful and true; but I don’t think it’s ALWAYS true.

For example:

Students with diagnoseable learning differences really might learn differently.

For instance: dyslexic readers combine distinctive neural networks to get their reading done. Those readers almost certainly benefit from distinct teaching strategies. In other words: by any reasonable definition, they “learn differently.”

Another example:

All learning depends on prior knowledge.

That claim — which sounds like “all people learn the same way” — also suggests that people learn differently.

Let’s imagine that you know A LOT more about opera than I do. (This assumption is almost certainly true.) If you and I both attend an advanced lecture about an obscure opera — “Der Häusliche Krieg” —  your learning will function quite differently from mine. Because you’re an expert and I’m a novice, we will learn differently.

Lots of individual differences will bring teachers to this same point.

Because I teach English, I teach grammar — and MANY of my students simply hate grammar. Their prior experience tells them it’s boring, useless, and impossible to understand.

On the one hand, those enduring cognitive principles listed above (working memory, retrieval practice, etc.) do apply to them. But their emotional response to the content will in fact shape the way they go about learning it.

Core principles of learning apply, and my students’ prior experience means that their learning process might well be different.

Beyond Twitter Rage

Twitter generates lots of extreme debates because complex ideas can’t be boiled down into its trivializing format.

So it’s not surprising that a nuanced understanding of “individual differences within important, broad, and meaningful similarities” doesn’t work in Twitter-ville.

At the same time, I do think our discussions of learning should be able to manage — and to focus on — that nuance.

Our students will learn more when we recognize BOTH the broad cognitive principles that shape instruction, AND the individual variation that will be essential within those principles.

---

Back in 2019, Paul Kirschner wrote a blog post on this same point. His “digestive system” analogy is VERY helpful.

---

* A few years back, I emailed the MBTI people to ask for research supporting their claims. They did not send me any. They did, however, sign me up for their newsletter.

You come to Learning and the Brain conferences — and to this blog — because you want research-based insight into teaching and learning.

We sincerely hope that you get lots of those insights, and feel inspired by them.

At the same time, all sorts of work has to go on behind the scenes to make sure such advice has merit. Much of that work seems tedious, but all of it is important.

For instance: definitions.

When researchers explore a particular topic — say, “learning” — they have to measure something — say, “how much someone learned.”

To undertake that measurement, they rely on a definition of the thing to be measured — for example: “getting correct answers on a subsequent test = learning.”

Of course, skeptics might reject that definition: “tests don’t reveal learning. Only real world application reveals learning.”

Because these skeptics have a different definition, they need to measure in a different way. And, of course, they might come to a different conclusion about the value of the teaching practice being measured.

In other words:

If I define learning as “getting answers right on a test,” I might conclude that the Great Watson Teaching Method works.

If you define learning as “using new concepts spontaneously in the real world,” you might conclude that the Great Watson Teaching method is a bust.

The DEFINITION tells researchers what to MEASURE; it thereby guides our ultimate CONCLUSIONS.

A Case in Point

I recently read an article, by Hambrick, Macnamara, and Oswald, about deliberate practice.

Now, if you’ve spent time at a Learning and the Brain conference in the last decade, you’ve heard researcher K. Anders Ericsson and others present on this topic. It means, basically, “practicing with the specific intention of getting better.”

According to Ericsson and others, deliberate practice is THE key to developing expertise in almost any field: sports, music, chess, academics, professional life.

Notice, however, that I included the slippery word ‘basically’ in my definition two sentences ago. I wrote: “it means, basically, ‘practicing with the specific intention of getting better.’ ”

That “basically” means I’m giving a rough definition, not precise one.

But, for the reasons explained above, we shouldn’t use research to give advice without precise definitions.

As Hambrick, Macnamara, and Oswald detail, deliberate practice has a frustratingly flexible definition. For instance:

• Can students create their own deliberate practice regimens? Or do they need professionals/teachers to create them and give feedback?
• Does group/team practice count, or must deliberate practice be individual?

As the authors detail, the answers to those questions change over time.

Even more alarmingly, they seem to change depending on the context. In some cases, Ericsson and his research partners hold up studies as examples of deliberate practice, but say that Hambrick’s team should not include them in meta-analyses evaluating the effectiveness of deliberate practice.

(The back-n-forth here gets very technical.)

Although the specifics of this debate quickly turn mind-numbing, the debate itself points to a troubling conclusion: because we can’t define deliberate practice with much confidence, we should hesitate to make strong research claims about the benefits of deliberate practice.

Because — again — research depends on precise definitions.

Curiouser and Curiouser

The argument above reminded me of another study that I read several years ago. Because that study uses lots of niche-y technical language, I’m going to simplify it a fair bit. But its headlines were clear:

Project-based learning helps students learn; direct instruction does not.

Because the “constructivist” vs. “direct instruction” debate rages so passionately, I was intrigued to find a study making such a strong claim.

One of my first questions will sound familiar: “how, precisely, did the researchers define ‘project-based learning’ and ‘direct instruction.’ ”

This study started with these definitions:

Direct instruction: “lecturing with passive listening.”

Constructivism: “problem-solving opportunities … that provide meaning. Students learn by collaboratively solving authentic, real-life problems, developing explanations and communicating ideas.”

To confirm their hypothesis, the reseachers had one group of biology students (the “constructivism” group) do an experiment where they soaked chicken bones in vinegar to see how flexible the bones became.

The “direct instruction” students copied the names of 206 bones from the chalkboard into their notebooks.

After even this brief description, you might have some strong reactions to this study.

First: OF COURSE students don’t learn much from copying the names of 206 bones. Who seriously thinks that they do? No cognitive scientist I’ve ever met.

Second: no one — and I mean NO ONE — who champions direct instruction would accept the definition as “lecturing with passive listening.”

In other words: we might be excited (or alarmed) to discover research championing PBL over direct instruction. But we shouldn’t use this reseach to make decisions about that choice because it relies on obviously inaccurate definitions.

(If you’re interested in this example — or this study — I’ve written about it extensively in my book, The Goldilocks Map.)

In Brief:

It might seem nerdy to focus so stubbornly on research definitions. If we’re serious about following research-informed guidance for our teaching, we really must.

---

Hambrick, D. Z., Macnamara, B. N., & Oswald, F. L. (2020). Is the deliberate practice view defensible? A review of evidence and discussion of issues. Frontiers in Psychology, 11, 1134.

I recently attended a (non-Learning-and-the-Brain) conference, and saw a thoughtful presentation that included a discussion of teachers’ professional standing.

In this blog post, I want to …

1. summarize this speaker’s thoughtful argument,
2. explain my own reasons for doubting it, and
3. consider some weaknesses in my own argument.

Teachers as Pilots

The speaker started by summarizing a common claim about teachers’ professional independence:

“Because teachers are highly-trained professionals, we should have the same freedom for independent action and creativity that other professionals enjoy. Rather than scripting and constraining teachers, schools should allow them the leeway to think, act, and teach with meaningful independence.”

I should be clear, by the way, that this speaker’s summary is NOT a straw man. I know that people make (roughly) this argument because a) I’ve heard other people make it, and b) I’m going to make a similar argument in just a few paragraphs.

To interrogate this pro-independence argument, the speaker asks us to think about other highly esteemed professionals: doctors, airline pilots, and engineers.

In every case, these professionals work in highly constrained conditions. In fact, we would be rightly shocked were they to want to escape those constraints. Imagine:

• If a pilot were to say: “today I think it would be fun to go through this pre-flight check list in reverse order. And, heck, I think I’ll skip step #27; I’ve never understood what it was for!”
• If an ER doctor were to say: “I understand that you’re experiencing chest pains, and the protocols suggest several tests. But honestly: you look healthy to me. And, I’m feeling lucky today. So let’s assume it’s a touch of nerves and get you some nice chamomile tea.”

We would be horrified!

Pilots and doctors work within well-established constraints, and have a strict ethical obligation to follow them.

For this reason, society rightly condemns instances where these professionals go outside those constraints.

Another example: when engineers get sloppy and bridges fall down mid-construction, we feel both horror and surprise to learn that they didn’t follow professional codes that govern their work.

These examples — the speaker said — show that teachers’ demands for professional freedom are misplaced.

Tight constraints do not violate our professional standing; they embody our professional standing.

Pushing Back: Reason #1

Although I understand these arguments as far as they go, I disagree with the speaker’s conclusion. Let me see if I can persuade you.

I think that doctors, pilots, and engineers are not good analogues for teachers, because the systems on which doctors, pilots, and engineers operate are unified, coherent, and designed to function as a whole.

Here’s what I mean:

• An airplane’s ailerons and flaps have been scrupulously designed to act upon the wings in a specific way. So too the engines and the rudders. And, frankly, almost everything else about the airplane.

Because airplane parts have been structured to function together, we can have specific, precise, and absolute rules the operation of planes. When the flaps do this, the airflow over the wing changes in entirely predictable ways. The plane will, in these circumstances, always turn, or ascend, or land, or whatever.

Yes, special circumstances exist: turbulence, wind shear, or thunderstorms. But even these special circumstances call for predictable and consistent responses: responses that can be trained, and should be executed precisely.

• A bridge has been designed to balance the forces that act upon it and the materials used to build it. Steel does not wake up one day and decide to have the strength of aluminum. Gravity does not vary unpredictably from day to day or mood to mood.

Because bridges have been structured to function in a particular way, engineers can have specific, precise, and absolute rules about their construction. Engineers don’t insist on moment-by-moment freedom because the bridges they build have entirely predictable constraints.

If, however, you have worked in a classroom, you know that such absolute predictability – based on the unity of the system being operated – has almost nothing to do with a teacher’s daily function.

• Gravity doesn’t work differently before and after lunch, but students do.
• An airplane’s rudder doesn’t have a different response to the pilot’s input, but this student might have a very different response than that student to a teacher’s input.
• An EKG (I assume) shows a particular kind of result for a healthy heart and a distinct one for an unhealthy heart. A student’s test results might mean all sorts of things depending on all sorts of variables.
• By the way: all of these examples so far focus on one student at a time. They don’t begin to explore the infinite, often-unpredictable interactions among students…
• …or the differences among the topics that students learn…
• …or the cultures within which the students learn.

We shouldn’t treat a classroom system as a straightforwardly stimulus-response system (like an airplane, like a bridge) because classrooms include an unpredictable vortex of possibilities between stimulus and response.

The best way – in many cases, the ONLY way – to manage that vortex: give teachers professional leeway to act, decide, change, and improvise.

The Continuum of Professionalism

Let’s pause for a moment to consider other kinds of professionls — say, ARCHITECTS, or THERAPISTS.

We allow — in fact we expect — subsantial freedom and creativity and variety as they do their work.

Of course, these professionals work within some constraints, and follow a well-defined code of ethics.

But those modest contraints allow for great freedom because…

… this client wants her house to look like a Tudor castle, while that client wants his to look like Falling Water, or

… this client struggles with PTSD and doesn’t want to take meds, while that client is managing bipolar disorder.

In other words: some professions do permit — in fact, require — strict limitations. But not all professions do. And, in my view, ours doesn’t. We’re more like architects than engineers.

Pushing Back: Reason #2

I promised two reasons that I resist the call for doctor-like-narrow-constraints. Here’s the second.

The analogies provided in this case all focus on people dying. The plane crashed. The heart-attack patient perished in agony. The bridge crushed workers and passers by.

In these professions, constraints literally save lives.

Now, I (like all teachers) think that education is important, and can transform lives and societies for the better. Bad educational practices do have damaging results for individuals and communities.

But: no one ever died from a bad English class. (I know; I’ve taught bad English classes. My students didn’t learn much, but they survived.)

If, in fact, teachers should work within tight constraints — checklists, scripts, step-by-step codes — the argument in favor of that position should be persuasive without the threat of death to energize it.

I’m Right, but I Might be Wrong

I promised up top that I’d include the weaknesses in my argument. I see at least four.

One:

Obviously, novice teachers require lots of support, and should work within tighter constraints than experienced teachers.

And, some teachers aren’t very good at their jobs. We might reasonably decline to trust their professional judgments.

Also: some people LIKE scripted curricula. I don’t think we should take them away from people who want them.

Two:

Teachers shouldn’t be scripted or managed detail by detail, but we should operate within well-established cognitive science principles. For instance:

• Retrieval practice is, in almost all cases, better than simple review
• Working memory overload is, in ALL cases, a detriment to learning
• Myths like “learning styles” or “right/left brain learning” should not be a guide for instruction.

In other words: my call for independence isn’t absolute. We should know our subject and our craft — and then work flexibly and appropriately with those knowledge bases.

Three:

I suspect that a few specific disciplines might allow for precise scripting.

The teaching of reading or early math, for instance, might really benefit from doing EXACTLY step A and then EXACTLY step B, and so forth.

However:

Even in this case, I suspect we would need LOTS of different scripts:

… “typical” readers,

… students with dyslexia, or diagnosably low working memory,

… students with unusually low background knowledge,

… students whose home language isn’t the instructional language,

and so forth.

In other words: even a scriptable subject matter requires teacherly expertise and inventiveness in moving from script to script.

Four:

My own biases no doubt shape my argument.

I myself am a VERY independent person, and I have a penchant for holding teachers in great esteem.

For these reasons, I probably react more strongly than others do to the suggestion that teachers should be tightly constrained to meet our professional obligations.

In other words: I do have a logical argument in support of my position.

a) Flying an airplane is an inappropriate analogue for teaching a class;

b) Tight constraints are almost certainly impossible in the work we do.

But: that logical argument almost certainly gets much of its passion from a realm beyond logic.

In Sum

Ideas about pedagogy often rest on assumptions about teacher independence. For that reason, I’m glad that the speaker raised this point specifically, and made a strong argument to support one point of view.

I hope I’ve persuaded you that teachers — like architects — need informed independence (within some constraints) to do our work.

Even if not, I’m hopeful that thinking through these questions has proven helpful to you.

---

 

In this tweet threat, the invaluable Peps Mccrea gives an excellent example of a situation where teachers’ communal adherence to the same norms benefits students and schools.