Marshall Memo 652
A Weekly Round-up of Important Ideas and Research in K-12 Education
September 12, 2016
2. Are classroom observations accurate measures of teachers’ work?
3. A different way of thinking about differentiation
4. A professor changes his mind about cold-calling
5. Close reading of challenging texts in middle school
6. Good news about the rich-poor gap in kindergarten entry skills
7. On-the-spot assessment tools
8. Short items: The Kappan poll
“Money spent well is a good way to boost outcomes; money spent poorly is not. You don’t need an awful lot of social science research to prove that common-sense proposition: If you waste money you’re not going to see results.”
James Ryan, Harvard Graduate School of Education dean, quoted in “An F-Minus for
America’s Schools from a Fed-Up Judge” by Kate Zernike in The New York Times,
September 9, 2016, http://nyti.ms/2chPjXz, on a Connecticut judge’s decision
“We’ve decided as a society that it’s too expensive to modify the kid’s environment. So we have to modify the kid.”
A clinician quoted in ADHD Nation: Children, Doctors, Big Pharma, and the Making
of an American Epidemic by Alan Schwarz (Scribner, 2016)
“We can’t make children learn, but we can let them learn.”
Alison Gopnik in The Gardener and the Carpenter: What the New Science of Child
Development Tells Us About Relationships Between Parents and Children (Farrar,
Straus, & Giroux, 2016)
“I had never realized just how deeply race penetrated all of our actions, whether we are conscious of it or not.”
Phil Santos, a New York City principal, whose school is implementing restorative
justice, quoted in “Halls of Justice” by Susan Dominus in The New York Times
Magazine, September 11, 2016 (p. 58-63, 81, 83), http://nyti.ms/2cEewcC
“Students should only take tests that (1) are aligned to what they’re learning, (2) serve a useful purpose, and (3) are of high quality.”
Jonah Edelman in “Making Sense of the Opt-Out Movement” in Education Next, Fall
2016 (Vol. 16, #4, p. 59), http://bit.ly/2cDNaEg
In this Harvard Business Review article, Daniel Kahneman (Princeton University) and Andrew Rosenfield, Linnea Gandhi, and Tom Blaser (The Greatest Good Group) say that organizations expect employees to treat similar situations similarly, if not identically. “The problem,” they continue, “is that humans are unreliable decision makers; their judgments are strongly influenced by irrelevant factors, such as their current mood, the time since their last meal, and the weather.” The jargon for these random factors is noise.
Some jobs are noise-free – post office clerks are required to follow strict rules that limit subjective judgment and guarantee that identical cases will be handled in the same way. But in jobs where decisions often call for human judgment – law, medicine, education – noise is a huge issue. “Research has confirmed that in many tasks, experts’ decisions are highly variable,” say the authors: “valuing stocks, appraising real estate, sentencing criminals, evaluating job performance, auditing financial statements, and more. The unavoidable conclusion is that professionals often make decisions that deviate significantly from those of their peers, from their own prior decisions, and from rules that they themselves claim to follow… Where there is judgment, there is noise – and usually more of it than you think.” And indeed, in many organizations, noise is far above the level that leaders would consider tolerable – and yet they’re largely unaware of the problem. Why? Because people tend to be overconfident in their own judgment and the judgment of their colleagues.
Noise is not to be confused with bias, which can also produce errors in judgment. “The term ‘bias’ has entered the public consciousness to the extent that the words ‘error’ and ‘bias’ are often used interchangeably,” say the authors. But there’s a big difference. As they describe it, a bathroom scale that consistently shows that you weigh five pounds more than your actual weight is biased. A scale that displays different weights depending on where you put your feet is noisy, as is a scale that gives different readings when you weigh yourself twice in the same 10-minute period. [In a school, if a principal consistently gives harsher punishments to boys than girls for the same infractions, that is bias, but if she often gives harsher punishments to students just before lunchtime, that’s noise.]
How can organizations deal with this problem? One approach is creating a set of formal rules – an algorithm – using the data from a given situation to produce a prediction or a decision without the need for human judgment. [In schools, this might have been the thinking behind zero-tolerance student discipline policies and detailed teacher-evaluation rubrics accompanied by training to attain inter-rater reliability.] “In many situations, of course, algorithms will not be practical,” say the authors. “The application of a rule may not be feasible when inputs are idiosyncratic or hard to code in a consistent format. Algorithms are also less likely to be useful for judgments or decisions that involve multiple dimensions or depend on negotiation with another party.”
Another approach to dealing with noise is conducting a noise audit – asking several professionals to independently evaluate a particular scenario, identifying differences, diagnosing the source of the noise, and working toward more consistent performance. A noise audit works best when respected team members create a scenario that is realistic, the people involved buy into the process, and everyone is willing to accept unpleasant results and act on them.
“Improving teacher evaluation is one of the most pressing but also contested areas of educational policy,” say Julie Cohen (University of Virginia Curry School of Education) and Dan Goldhaber (American Institutes for Research) in this article in Educational Researcher. Much of the debate in recent years has been about value-added measures of teacher effectiveness, but these are available for only 20-30 percent of teachers and have been criticized for problematic statistical properties – specifically, misclassifying “effective” teachers as “ineffective” and vice versa.
Cohen and Goldhaber decided to focus instead on supervisors’ classroom visits, which have been a mainstay of teacher supervision and evaluation for decades. “Interestingly, given their prevalent use, we know surprisingly little about the statistical properties of classroom observations in consequential personnel decisions,” say the authors. “Little is known about the degree to which, and ways in which, teaching practices evolve in response to observation systems.” Cohen and Goldhaber point out that the 2012 Measures of Effective Teaching (MET) study drew its conclusions about teacher evaluation in a low-stakes research context, while other studies have shown that in the real world of schools, there’s rampant grade inflation, with virtually all teachers getting proficient or exemplary ratings.
The challenge, say the authors, is designing classroom observations that provide valid data on what’s happening day to day in classrooms, make meaningful distinctions among teachers, provide teachers with useful feedback, and support helpful, high-quality professional development. To accomplish these important goals, several challenges need to be addressed:
“Rethinking Differentiation – Using Teachers’ Time Most Effectively” by Kim Marshall in Phi Delta Kappan, September 2016 (Vol. 98, #1, p. 8-13), available for purchase at
http://pdk.sagepub.com/content/current
In this Chronicle of Higher Education article, David Gooblar (University of Iowa) says that cold-calling students in class has never been his cup of tea. “It just doesn’t fit with the kind of teacher I want to be,” he says. “I don’t want to be the authority figure in front of whom students cower. I get no pleasure from putting students on the spot, from scaring them into knowing the material.” He views his job as creating a classroom climate where students come to class prepared, are eager to take part, and gain from being active participants.
What began to change Gooblar’s mind was watching students sit on their hands, his questions hanging in the air unanswered, discussions dying on the vine. He was also influenced by research showing that over a semester, cold-calling actually increases students’ voluntary participation. “Cold-calling encourages students to prepare more and to participate more frequently,” said one researcher. “The more they prepare, and the more frequently they participate, the more comfortable they become when participating.”
That last point was the clincher for Gooblar. He began to see cold-calling not as an aggressive act producing student discomfort but as a warm invitation to contribute, a bridge to students enjoying his classes and learning more. “I need to remember,” he says: “The thing that often keeps me from calling on students – my concern that they might be uncomfortable speaking up in class – is actually a good reason to call on them. If we don’t encourage students to come out of their shells for fear of putting them on the spot, we may be doing them a disservice… You’re curious about their views and their understanding of the issues being discussed. What they think is important – both to their own learning and to that of their peers.”
Gooblar’s article sparked several responses, two of which raised concerns about cold-calling students who are highly introverted or have severe social anxiety. Both responses suggested accommodating those students, either by sharing questions in advance or refraining from cold-calling them in class.
In this article in AMLE Magazine, Doug Lemov, Colleen Driggs, and Erica Woolway of Uncommon Schools say that many middle-school teachers want to get students reading texts that will engage and motivate them – often contemporary young adult fiction. This is all well and good, say Lemov, Driggs, and Woolway, but “what students ‘like,’ or more precisely think they will like, is inherently limited. We can all name a handful of texts we read against our better teenage judgment – infallible though it seemed at the time – but which turned out to be transformative – instantly in many cases, years later in others.” So teachers need to make sure students use their finite and precious classroom hours to grapple with some complex texts of the sort that they’ll encounter in college and life. Here are five types:
• Archaic texts – The first sentence of Oliver Twist by Charles Dickens is 98 words long and includes unfamiliar turns of phrase like “to wit,” “inasmuch,” and “in this workhouse was born…” In earlier eras, whether in novels or documents like the Declaration of Independence, people used words in different ways, and students need to be able to unpack and comprehend such texts.
• Nonlinear time sequence – The narrative of Bigmama’s by Donald Crew switches back and forth between memories of a specific trip to the narrator’s grandparents’ house and recollections of visits made over several years. It’s challenging to keep all this straight, and Bigmama’s is ideal for helping young readers slow down and figure things out.
• Complexity of narrator – R.J. Palacio’s book Wonder uses six different narrators to tell the story of a boy with severe craniofacial disfigurement, and one of them uses idiosyncratic punctuation and no uppercase letters. “It’s a useful book, first and foremost as an object lesson in kindness and understanding,” say Lemov, Driggs, and Woolway. “But it’s also a starter kit for understanding books with complex and potentially confusing narration.”
• Complex plot and symbolism – Where the Mountain Meets the Moon by Grace Lin weaves fairy tales into the journey-of-discovery narrative, with characters telling other characters’ stories and characters reacting to fairy tales they hear, which shapes the plot. Mastering this kind of text helps students prepare for challenging narrative structures like those in William Faulkner’s novels.
• Resistant texts – The beginning of Slaughterhouse Five by Kurt Vonnegut uses a highly unconventional style to capture the difficulty of telling a story (the firebombing of Dresden in World War II) that cannot be told simply. “The elements create a thrilling narrative unbounded by traditional rules,” say Lemov, Driggs, and Woolway. “But confused readers – readers unaware that a text might deliberately try to disorient them, readers who have never struggled with that disorientation – may in fact be confused by the premise, not comprehend that they are not supposed to comprehend, and fail, perhaps even give up on the narrative.” Poetry doesn’t always conform to our “expectation of logic” – for example, “Jabberwocky” by Lewis Carroll. Close reading and unpacking of short passages of texts like these prepare students for the intense challenges of reading difficult material.
In this New York Times article, Sean Reardon (Stanford University), Jane Waldfogel Columbia University), and Daphna Bassok (University of Virginia) describe their research on the narrowing gap in the reading and math skills with which low-income and high-income children enter U.S. kindergarten classes. Drawing on two decades of data from the National Center for Educational Statistics, the authors found that between 1998 and 2010, the reading readiness gap closed by 16 percent and the math gap by 10 percent. The black-white and the Hispanic-white gaps also narrowed by about 15 percent. “The gaps that remain are still vast,” say Reardon, Waldfogel, and Bassok, noting that low-income children still enter kindergarten a full year behind their more-fortunate peers, “but even this modest improvement represents a sharp reversal of the trend over the preceding decades.” Better yet, the gaps closed because of rapid progress by low-income children, not declines in the readiness of high-income children, and the gains persisted at least through fourth grade.
These narrowing gaps are surprising because other indicators have been much less encouraging: income inequality grew by about 10 percent from 1998 to 2010, economic segregation increased by 20 percent, and racial gaps in education, employment, and health persist. What brought about the early reading and math gains? The authors believe several factors contributed:
• The availability of high-quality, publicly funded preschool programs – the percent of U.S. 4-year-olds enrolled in state-funded preschools has increased from 14 to 29 percent from 2000.
• The fact that more families are investing in books and other reading matter for children, as well as Internet access and computer games focused on reading and math skills.
• More parents are spending quality time with children, taking them to local libraries, and engaging in learning activities at home.
• Finally, say Reardon, Waldfogel, and Bassok, there is “the widespread diffusion of a single powerful idea: that the first few years of a child’s life are the most consequential for cognitive development. Less than a century ago… mainstream magazines routinely advised new mothers that intellectual stimulation of babies was harmful. Now we know better, the result of decades of scientific research about brain development, poverty, and the long-term effects of high-quality preschool programs.” Public information campaigns like Reach Out and Read, Too Small to Fail, Providence Reads, home-visiting programs, and preschools have spread these insights to more families. “Like a new medical innovation that is first adopted by the wealthy but then becomes commonplace, the emphasis on public and private investments in young children has helped turn a benefit for the rich into an equalizing force in society.”
The Kappan poll – The results of this year’s Phi Delta Kappan poll of U.S. attitudes on public education are available at www.pdkpoll.org. You can download a PDF of the report, link to poll archives, post comments, follow links to news coverage and national commentary, and read blogs by leading educators.
© Copyright 2016 Marshall Memo LLC
About the Marshall Memo
Mission and focus:
This weekly memo is designed to keep principals, teachers, superintendents, and others very well-informed on current research and effective practices in K-12 education. Kim Marshall, drawing on 45 years’ experience as a teacher, principal, central office administrator, and writer, lightens the load of busy educators by serving as their “designated reader.”
To produce the Marshall Memo, Kim subscribes to 64 carefully-chosen publications (see list to the right), sifts through more than a hundred articles each week, and selects 5-10 that have the greatest potential to improve teaching, leadership, and learning. He then writes a brief summary of each article, pulls out several striking quotes, provides e-links to full articles when available, and e-mails the Memo to subscribers every Monday evening (with occasional breaks; there are 50 issues a year).
Individual subscriptions are $50 for a year. Rates decline steeply for multiple readers within the same organization. See the website for these rates and how to pay by check, credit card, or purchase order.
Website:
If you go to http://www.marshallmemo.com you will find detailed information on:
• How to subscribe or renew
• A detailed rationale for the Marshall Memo
• Publications (with a count of articles from each)
• Article selection criteria
• Topics (with a count of articles from each)
• Headlines for all issues
• Reader opinions
• About Kim Marshall (including links to articles)
• A free sample issue
Subscribers have access to the Members’ Area of the website, which has:
• The current issue (in Word or PDF)
• All back issues and podcasts
• An archive of all articles to date, searchable
by topic, title, author, source, level, etc.
• A collection of “classic” articles from all issues
Core list of publications covered
Those read this week are underlined.
American Educational Research Journal
American Educator
American Journal of Education
AMLE Magazine
ASCA School Counselor
ASCD SmartBrief
Center for Performance Assessment Newsletter
District Administration
Ed. Magazine
Educational Evaluation and Policy Analysis
Educational Horizons
Educational Leadership
Elementary School Journal
Essential Teacher
Go Teach
Harvard Business Review
Harvard Educational Review
Independent School
Journal of Adolescent and Adult Literacy
Journal of Education for Students Placed At Risk (JESPAR)
Journal of Staff Development
Kappa Delta Pi Record
Knowledge Quest
Literacy Today
Middle School Journal
Peabody Journal of Education
Perspectives
Phi Delta Kappan
Principal
Principal Leadership
Principal’s Research Review
Reading Research Quarterly
Responsive Classroom Newsletter
Rethinking Schools
Review of Educational Research
School Administrator
School Library Journal
Teacher
Teaching Children Mathematics
Teaching Exceptional Children/Exceptional Children
The Atlantic
The Chronicle of Higher Education
The District Management Journal
The Journal of the Learning Sciences
The Language Educator
The Learning Principal/Learning System/Tools for Schools
The Reading Teacher
Theory Into Practice
Time Magazine
Wharton Leadership Digest