Great Resources

I am so impressed with these three websites, especially the first one. It is a Canadian organization that gives terrific ideas on how to teach, create, and share ideas on CT.

The Critical Thinking Consortium

Critical Thinking Consortium

Just today I received an email from them with this information about more resources at this URL (under Join heading):

tc2- Join

“If you are in a position to help implement critical thinking, here are three resources you might use to introduce the Tools for Thought to teachers during a short meeting. This online collection of ready to use lesson plans introduces students to a wide range of intellectual tools for effective thinking. “

There are lots of easy to use ideas for all levels, including lesson plans. What could be easier?

CriticalThinking.net (http://www.criticalthinking.net/howteach.html)

Written by one of the fathers of critical thinking, Robert Ennis provides “Twenty-One Strategies and Tactics for Teaching Critical Thinking.” Subheads include Underlying and Fundamental Strategies, Tactics, and Mid-Level Strategies. It’s a bit condensed, but it provides a great bird’s-eye view of what CT consists of and how to teach it.

Critical Thinking Pathways (Critical Thinking Pathways)

Edutopia, where this article appears, has a lot to recommend it on many issues educational, but this posting by blogger Todd Finley really sent me over the moon. It has everything from definitions of CT ideas for project-based learning to the formulation of questions. It’s a collection of resources (including Ennis’s from above) that you could spend a day looking over. Be sure to look at the other articles in the series.

 

 

Articles on Assessment of Critical Thinking 4

This is the last in this series of annotated bibs about articles on CT assessment, but I would like to revisit it soon. It is, after all, a huge topic and not at all settled. These last two items are by the same people, Richard Paul and Linda Elder. Please let me know if you find these postings helpful or would like me to add one of your choosing.

Paul, R. & Elder, L. (2007). Critical thinking competency standards: Standards, principles, performance indicators, and outcomes with a critical thinking master rubric. Tomales, CA: Foundation for Critical Thinking Press.

This book is from a series about critical thinking that the Foundation for Critical Thinking writes and publishes. All of Paul and Elder’s books are based on eights elements of reasoning, nine intellectual standards, and eight intellectual traits.

Their rubric for assessing critical thinking is a combination of these 25 competencies. The authors recommend assigning 0-4 points per standard in their grading rubric. Each standard lists several outcomes. For instance, Standard Four: Inferences and Interpretations lists this as one of the nine outcomes: “Students distinguish between assumptions and inferences; they uncover and accurately assess the assumptions underlying inferences,” (p. 20). It doesn’t give any instructions on how to go about assessing assumptions and inferences, just that you should do it.

There is a wealth of information in this and all their booklets on different aspects of critical thinking, but many are not easily navigated without some instruction and practice. They are also fairly dense and somewhat repetitive from book to book, probably so one can read any of them as a stand-alone document. The Foundation has made a name for itself by identifying and expanding the elements of critical thinking, although it can be fairly said that Richard Paul brought a sterling reputation to the Foundation from his years as a philosopher. Their booklets are more informational than pedagogically suited. Useful for those wanting to appreciate how complex critical thinking can be, the information can be tailored to address individual teaching needs, albeit with some difficulty.

 

Paul, R. & Elder, L. (2006). The international critical thinking reading & writing test. Tomales, CA: Foundation for Critical Thinking Press.

As with the other booklet annotated above, this one also draws upon seven “symbiotic skills” of disciplined thought: 1. Clarify purpose, 2. Formulate clear questions, 3. Distinguish accurate and relevant information from inaccurate and irrelevant information, 4. Reach logical inferences or conclusions, 5. Identify significant and deep concepts, 6. Distinguish justifiable from unjustifiable assumptions, and 7. Trace logical implications. Outcomes are provided.

The authors break down the processes of close reading and substantive writing into five levels: paraphrasing, explicating, analysis, evaluation, and role-playing. For each level, they provide very useful exercises as well as sample responses. They use fairly high level texts, e.g., Thoreau’s On Civil Disobedience and Fromm’s The Art of Loving. The method, however, is applicable to any text. This book is much more teacher-friendly because it goes carefully through the steps of close reading and substantive writing. [Much of the method is also covered in their booklet titled A Thinkers Guide to How to Read a Paragraph and Beyond: The Art of Close Reading (2003)]. Their method really shows how to put critical thinking into action and prevent students from zooming through readings with little understanding. It is the antidote to fast-food thinking, which is, of course, their point.

This booklet does not teach how to design or conduct the test but rather provides examples of how to examine a particular component of critical thinking. Grading, they caution, should be done only by someone who is familiar with the tenets of critical thinking if reliability is to be achieved.

While the efforts of the Foundation are admirable and an invaluable contribution to the field, I find their explanations dense and at times inaccessible, even to someone who has studied critical thinking a lot and even attended several of their conferences. For my money, critical thinking is already mysterious enough, and those who come to this and other booklets uninitiated in their canon may find them tough sledding but worth the effort for the trying.

 

Observation: The Earliest Critical Thinking Skill

This is another guest posting of a presentation given by Steve Coxon, Ph.D., assistant professor & director of Programs in Gifted Education at Maryville University in St. Louis, Missouri. He presented his talk at the 34th International Conference on Critical Thinking and Educational Reform sponsored by the Foundation for Critical Thinking (www.criticalthinking.org) in July this year. He has also generously offered to send his PowerPoint presentation that accompanied his talk. Below that is Dr. Coxon’s contact information and some websites pertinent to his talk.

Observation: The Earliest Critical Thinking Skill

Careful observation is fundamental to a fair-minded understanding of the world and is important to continue to develop in elementary students through increasingly challenging learning activities. In her book, Emergent Science, Johnston (2014, p. 8) refers to observation as the “first and most important scientific skill” and suggests it begins to develop in the womb as the fetus learns to recognize familiar voices. While preschool curricula often focus on improving observational skills in early childhood, researchers Eberback and Crowley (2009) note that elementary classrooms neglect further development in middle and late childhood. This is unfortunate as it takes increasingly challenging experiences to improve any skill and careful observation is needed for well-developed habits of mind including clarity, accuracy, precision, depth, and breadth. Moreover, strong observational skills underlie such critical thinking concepts as comparing and contrasting viewpoints, evaluating evidence, and noting the sometimes subtle differences between fact and opinion. This concurrent session will provide a background on the development of observational skills in early childhood, connections between observation and critical thinking, and challenging activities to improve observational skills in elementary school students.

 

-Graduate programs: http://www.maryville.edu/gifted
-Summer Science and Robotics: http://www.maryville.edu/robot
-Young Scholars: http://www.maryville.edu/youngscholars
Book Review Editor, Roeper Review
Follow me on Twitter @GiftedEdStLouis

Using Social Media as an Access Point to Analyze Our Thinking

Dear readers:

Shira Cohen-Goldberg delivered a presentation at the 34th International Conference on Critical Thinking and Educational Reform (July 2014) for teachers and elementary school students. Shira is a lead facilitator at the HILL for Literacy, Inc. Below is an abstract of her talk. Slides and handouts can be located at

http://hillforliteracy.org/resources/presentations (at the top of the page).

The Common Core State Standards establish that teaching practices and related tasks that help students cultivate their thinking are to be prioritized in classroom instruction. This presentation is designed to help participants explore novel ways to bring the core concepts of critical thinking to elementary and middle school students. By nature, digital information (including social networking sites and YouTube) is continuously accessible, abundant, and often requires careful analysis. Social media is a timely and appropriate means to help young students analyze new information and is an engaging point of access for all students. Cohen-Goldberg uses digital media to help participants explore the concepts of information/inferences/assumptions, touch on paraphrasing and explication, and deeply engage in the analysis level by exploring the logic of an internet persona’s thinking. She offers methods and tools to help students move their thinking beyond digital media and into textual analysis.

Critical Thinking in the Literature Classroom

Dear readers:

Hiner and Bird-Critical Thinking in the Literature Classroom is a PowerPoint that Dr. Amanda Hiner and Dr. John Bird presented at the 34th International Conference on Critical Thinking and Educational Reform (July 2014). Dr. Hiner has published extensively about this topic and has distilled many of her idea in this presentation, which contains many links and helpful resources. Please contact Dr. Hiner with questions and comments, or include them here and I will forward them

Amanda L. Hiner, PhD

Assistant Professor of English

Coordinator, Critical Reading, Thinking, and Writing

Winthrop University

241 Bancroft Hall

Rock Hill, SC 29733

803-323-2351

hinera@winthrop.edu

http://faculty.winthrop.edu/hinera

 

 

ARTICLES ON ASSESSMENT OF CRITICAL THINKING 3

Mazer, J., Hunt, S., Kuznekoff, J. (2007). Revising general education: Assessing a critical thinking instructional model in the basic communication course. The Journal of General Education, 56(3&4), 173-199.

In this article the authors describe their program and the courses where their critical thinking model is used. They also define critical thinking (as that is very important to do before embarking on any kind of assessment) and report on an experiment where they compare two CT instructional models for two Communications 110 courses, one current and one proposed. For assessment, they used two measurements. The first is a 17-item Likert scale for self assessment developed by Halpern (1996)1 and the second is a test specific to the content of the course. There were 10 multiple choice questions that measured students’ recognition of arguments, evidence, and fallacies.

This paper is useful for anyone wanting to see how CT is used in communication courses. Authors state that CT instruction is most effective when housed within a content course, such as a basic communication course, and applied to specific assignments” (p. 176). Moreover, there are many sources listed that address these important issues.

The research instrument used, as self-assessment tool which they call the CTSA, or the Critical Thinking Self Assessment, is

1Halpern, D. (1996). Thinking critically about critical thinking. Mahwah, N.J.: Erlbaum.

The authors have graciously granted me permission to publish the self-assessment instrument used in their study (with some modifications), as it is not included in the article.

Critical Thinking Self Assessment (CTSA) (printed with permission from Stephen Hunt)

 Directions: This questionnaire is designed to help you examine your own skills by asking you to describe how you interact with things you read and hear. Doing this accurately can help you know what skills you need to work on and what skills you have already developed. Your answers will not affect your grade in any way; so be honest with yourself. Think about times when you have seen or heard professionally-produced articles, stories, videos, books, speeches, or sermons that were designed to persuade you to believe something. Consider only those times when you paid attention. Using these recollections, and recollections about your own writing and speaking, please answer the following questions as honestly as you can. Please circle the appropriate response using the scale below (1 = never, 2 = rarely, 3 = sometimes, 4 = frequently, 5 = always)

Never Rarely Sometimes Frequently Always
1. When I read or hear items like those described above, I am able to get the point. 1 2 3 4 5
2. I am able to follow a fairly complex line of argument, so that I can tell which things are offered in support of which other things, and how it’s all supposed to fit together. 1 2 3 4 5
3. After reading or hearing someone’s line of argument on an issue, I can give an accurate, detailed summary of how the line of argument went. 1 2 3 4 5
4. I feel confident about deciding whether it is reasonable to believe a piece of evidence or a reason used in support of a conclusion. 1 2 3 4 5
5. I can tell when there are logical holes in the reasoning that is supposed to connect a conclusion and the reasons being used to support that conclusion. 1 2 3 4 5
6. I know how to tell the difference between a credible source and a garbage source of information or ideas. 1 2 3 4 5
7. I look for the hidden assumptions that are often present in an argument. 1 2 3 4 5
8. When I read reliable statistics that show two factors rise and fall together, I recognize that it doesn’t necessarily mean one caused the other. 1 2 3 4 5
9. When I evaluate someone else’s line of thinking, I consider their arguments rather than just deciding whether I agree with their conclusions. 1 2 3 4 5


Portfolio Assignment

Never Rarely Sometimes Frequently Always
10. I know how to go about deciding how strong an argument really is. 1 2 3 4 5
11. I am able to come up with acceptable reasons or evidence to support my conclusions when I write or give organized oral presentations. 1 2 3 4 5
12. When I write an essay or give a talk I try to respond carefully to possible significant objections to my positions. 1 2 3 4 5
13. I am able to construct an organized, logical argument that stays on topic. 1 2 3 4 5
14. When I present an argument for a position, other people can follow what I’m saying. 1 2 3 4 5
15. When there are good arguments for contrary views on a subject, I know how to evaluate them and come up with the best conclusion. 1 2 3 4 5
16. I am willing to take the time and make the effort to think through an argument carefully before deciding what I think about it. 1 2 3 4 5
17. I enjoy thinking through an issue and coming up with strong arguments about it. 1 2 3 4 5

In order to obtain a score, simply sum all 17 items. Your CTSA score ______.  Look back over your answers. Do you see any patterns of weaknesses or strengths? You can compare your score on the CTSA to other ISU students’ using the percentile chart below (the average beginning of semester CTSA score in a previous sample of ISU students was 64.11).

                  Percentiles                  Score

25                             60.00

50                             63.00

75                             69.00

ARTICLES ON ASSESSMENT OF CRITICAL THINKING 2

Here are two more articles that talk about assessment. By the way, these articles have been chosen because they mention assessment of critical thinking in their title. Their selection does imply any anything beyond that criterion.

Halpern, D. (2001). Assessing the effectiveness of critical thinking instruction. Best of the Journal of General Education: Featuring Articles from 1984–2000, 270-286.

Halpern provides a brief justification for why we teach critical thinking and how important it is. She lists 19 thinking skills that most courses draw from if they purport to enhance CT, e.g., reading with a high level of comprehension, using analogies, restructuring problems, planning a course of action.

Before launching into a survey of ways that CT is assessed, she cautions that any test must be sensitive enough to pick up subtle changes in students’ thinking abilities and in their disposition to think critically, e.g., a willingness to suspend judgment to consider information that would go against their thinking.

Halpern then outlines the various issues that surround a reliable assessment of critical thinking including who is tested and when, as well as which courses and activities contribute to critical thinking. She concludes that critical thinking can and should be taught in college, as its effects will benefit the students and businesses for years to come.

My question is why we wait until college to embark on teaching such a vital skill.

For further reading, see the 2013 issue of Inquiry: Critical Thinking Across the Disciplines, 28(3), with several articles critiquing the Halpern Critical Thinking Assessment (HCTA) and her reply.

Ikuenobe, P. (2001). Teaching and assessing critical thinking abilities as outcomes in an informal logic course. Teaching in Higher Education, 6(1), 19-32.

Many believe that informal logic and critical thinking intersect. Ikuenobe agrees but adds that although learning logical principles of reasoning is necessary for critical thinking, they are not sufficient because courses in informal logic do not clearly state outcomes. The article advances detailed abstract principles in informal logic that are also applicable to critical thinking, as well as the five developmental stages they are divided into for teaching, from the more passive skill of identifying to the more active ones of articulating, constructing, and evaluating argument structures.

Ikuenobe outlines in great detail what elements go into each of these stages and lists outcomes that can be turned into assessments, e.g., in Level 1, “They are able to identify statements, other kinds of sentences, arguments and …identify premises and conclusions in different texts” (p. 29). Level 5, however, involves the open-ended criterion of “applying general logical principles to the context of a subject matter” (p. 30).

While comprehensive, the article assumes intimate knowledge of informal logic and how to teach it, for instance, analyzing argument structure. The article is useful in helping us understand how informal logic and critical thinking might overlap. But the title might be a bit misleading in that it does not give any information on how to teach any of these skills. Perhaps most useful is the opportunity it provides those teaching critical thinking to see how their efforts jibe with the tenets of informal logic so they can build a few of those elements into their curriculum.

What Do You Know?: Assessing Critical Thinking

Here’s what I know for sure about assessing critical thinking: There are several ways to go about it. It sure is tricky though, mostly because critical thinking is not a skill that builds the same way we acquire vocabulary or learn to vault a high bar. It’s a protracted, nonlinear process that takes months, maybe even years, before significant progress can be measured.

I learned that the hard way. When I wrote my dissertation about critical thinking and persuasive writing, I hypothesized that persuasive writing would increase 4th graders’ ability to think critically. Among the data I collected were the results of a pre and post test on the Cornell Critical Thinking Test X that bookended a three-week unit on how to write persuasive essays. My results showed no significance. Duh! Critical thinking needs to be practiced–and practiced a lot over time and across domains–for it to become a habit of mind, a recurring method of inquiry, and a way of calibrating one’s ideas. So, no, three weeks in terms of learning to think critically is not enough time. Ah, live and learn.

So if it takes a long time to acquire those skills, when and how can we reliably measure them? To complicate things, critical thinking skills can differ from field to field. For instance, I may be a very good critical thinker when it comes to linguistic theory but a total bust when it comes to aviation or sociology. Some disagree, saying there are elements of critical thinking that carry over from one task to the next. (See Ennis, 1989, and Halpern, 1998, for thorough discussions about this still unsettled matter.) So should I just test their skill to think critically in math, or can I give them a generalized test that examines if they use those same skills in, say, English literature?

In the end, the question ends up being this: exactly how do we appraise students’ critical thinking and audit their progress? My next few postings will take up those issues with an annotated bibliography of several articles on the topic. (Hint: none of these methods includes multiple choice tests.) Feel free to suggest other articles for analysis.

Ennis, R. (1989). Critical thinking and subject specificity: clarification and needed research. Educational Researcher, 18(3), 4-10.

Halpern, D. (1998). Teaching critical thinking for transfer across domains. American Psychologist, 53(4), 449-455.

 

Reflecting on Reflections

If you think about it—and you should—you will agree that reflections are a mighty and indispensible component of critical thinking. That’s because humans don’t learn, or I should say don’t grow our understanding, in a linear manner. Unless we reflect on information presented to us, it becomes like the computer acronym GIGO—Garbage In Garbage Out, which is pretty accurate if you think educating someone is like shoveling coal into a furnace. Information needs to be processed, mulled over, considered, questioned. So reflections, written or oral, offer the brain the breathing space do those complicated maneuvers. Otherwise, listeners are just sophisticated transcribers of information. We’re back to the knowledge-banking concept that Freire talked about.

One useful feature of reflections is that the teacher can build them into any part of the class period. One instructor I observed had students take 5 minutes at the beginning of the class to reflect on what they remembered from the previous class or any questions they had about the material, class, field of study—anything. It serves to engage and focus their attention on that class and shove aside Facebook, arguments with parents, their date last night. Volunteers can share their musings, which offer a good springboard for discussion. Responses can be an ungraded and unobtrusive way to gauge students’ understanding and thinking. It’s also a good idea to just let the students have their own thoughts without voicing them. The same process can be repeated at the end of the class. You could even have them pose their own question that they try to find the answer to on their own. Great fodder for class the next day too.

Recently I had a stimulating discussion with a man who had seen my letter to the editor in the New York Times in response to Harvard’s plans to begin MOOCs, Massive Open Online Courses, which I believe to be bunk. (See Critical Thinking and MOOCs – A Marriage Made in Hell, posted on June 5, 2014.) We discussed lectures, a fact of life in many university classes. When I said that lectures could also be places for critical thinking, he asked how that could be. After all with a room of 300 people, he reasoned, you can’t just stop and ask students to reflect. Wrong! I said he was thinking of lectures solely as info-dumps, which I roundly condemn in any learning situation. Why couldn’t a lecturer stop, say, every 15 or 20 minutes, whatever seems to be a natural break, and ask audience members to turn to a neighbor and ask a question or request clarification of a hazy or troubling aspect of the lecture so far, or perhaps mention something that is at odds with their beliefs. Giving them guided questions is always good, such as, “What do you think is the strongest argument for immigration I’ve mentioned so far? Turn to your neighbor and see what s/he thinks.”

Reflections are the pause button allowing students to chew, swallow, and digest information instead of merely inhaling an invisible gas. Plus, learning how to think reflectively is a skill they can use for the rest of their lives. Now that’s what I call education.

Questions, continued, a.k.a. Making Pink

We think that getting an education is all about answering questions when it’s really all about asking them. That’s because questions help focus and sharpen our thinking. Answers just stop it cold. As Paul and Elder1 state, “Feeding students endless content to remember… is akin to repeatedly stepping on the brakes in a vehicle that is, unfortunately, already at rest.”

Questions, Paul and Elder continue, “determine where our thinking goes” (p. 62). Scientists would never get anywhere if all they did was collect facts. Instead, they ask questions such as “What other uses can we put this drug to?” “Why does Washington State have an inordinate number of cases of multiple sclerosis?” “How can we send a man to the moon?”

Interestingly, not all questions are created equal, especially in terms of critical thinking. Let’s take Bloom’s Taxonomy, that 60-year-old warhorse of a classification system that divides thinking into levels of cognitive difficulty. Recalling knowledge, a.k.a. remembering, as it is called by some, is at the bottom. Unfortunately, that’s where most of our education is stuck—stuffing facts down our students’ gullets. No question that facts are necessary. But what good are they if we don’t question their validity, hidden assumptions, and logic, or ultimately connect them to the big picture?

Here is the graph that shows the various levels Bloom outlines:

Blooms taxonomy1

I have attached a PowerPoint presentation in which I apply Bloom’s cognitive levels to Goldilocks and the Three Bears (with a great deal of help from http://courseweb.unt.edu/gmayes/documents/Blooms_Taxonomy.html).

For comparison, look in the PowerPoint presentation at the questions I developed for “The Global Child,” an essay (on tape) in National Public Radio’s excellent series, Raise the Issues. You can do this for any text you want. Better yet, have students do it on their own.

At the beginning of the PowerPoint, you will also see the slide below of paint cans with the question, How do I make pink? Behind the question is a story told by Indiana-based artist Marvin Bartel. He told me he teaches critical thinking by asking questions of elementary kids in his art classes. Really? I asked. Critical thinking in elementary school? In art class??? It made perfect sense after he explained it.

Paint cans

Consider this scenario. If a child comes to you and wants to know how to make pink, you have two choices. You can tell her how to do it, or you can say, What do you think? By choosing the second approach, the child is then free to hypothesize and experiment (i.e., analyze and create). Yes, she may get it wrong the first couple of times. But if you give her the information, she will have no opportunity to develop those higher level processes. If allowed to experiment, she becomes invested in the answer and begins to rely on herself for information. It’s so simple, and yet so powerful.

And speaking of questions, what kinds are you asking in your classes? I think you’ll find that the higher you climb on Bloom’s cognitive steps by asking questions that employ those cognitive skills, the more engaged students become.

Socrates was killed in 399 BC because those in power thought he was corrupting youth by encouraging them to question the status quo. So don’t think questions aren’t powerful. They are the very antidote we need against muddled thinking and seductive propaganda.

1Paul, R. & Elder, L. (2007). The thinker’s guide to the art of Socratic questioning. Dillon Beach, Calif.: Foundation for Critical Thinking.