February 16, 2010
Making Intellectual Development Reflexive
By Daniel Bernstein, University of Kansas
Any nuanced conversation about assessment in higher education recognizes many layers of performance and of measurement, ranging from the knowledge, skills, and understanding typical of any individual course to the general skills characteristic of a well-educated person such as critical thinking, communication, integrative analysis, and quantitative literacy.
I have found two frameworks for meta-evaluation very generative for how I go about teaching and measuring the impact of my teaching. One is articulated in the book Teaching for Understanding, edited by Martha Stone Wiske. Based on Project Zero in Boston, the authors argue that students demonstrate deep understanding when they remember ideas and material without re-memorizing and when they can use what they have learned in novel contexts. The chapter by David Perkins makes a compelling case for finding understanding in contextualized performance rather than in largely symbolic representations of relations among ideas and variables.
In 1998, I reframed my examinations from essays about conceptual systems and abstract representations to word problems in which students had to use those systems and representations to analyze and solve problems presented in vignettes of hypothetical situations. To my dismay, many students who were quite capable of reproducing my conceptual language were largely incapable of using the ideas on a real problem. I spent the next ten years learning how to teach differently so that students could use conceptual tools, as well as represent them formally.
A second big idea was a model of intellectual development initially put forward by William Perry, who was describing the informal ways of knowing of typical students. A substantial literature and a richly nuanced intellectual position have evolved from the original writing; see the Center for the Study of Intellectual Development for a comprehensive bibliography. I have a simplified variant on the theme, and it has proved useful in thinking about my teaching and my students' learning. Students typically arrive in college having largely learned that assignments have correct answers, known by teachers, and their task is to identify and correctly use that answer. After a few college courses, students recognize that there are often many answers and perspectives brought forth to illuminate an assignment or problem, and they fall into an uncritical relativism that functions as epistemological anarchy--all answers are in some way correct. When education works well, some students come to recognize that evidence, analysis, and argument can differentiate among alternatives, and some answers are more equal than others. In the Perry community, this is a form of commitment that embraces multiple perspectives and invites learners to the difficult task of critical evaluation of evidence and reasoned construction of their own positions.
In assessing my students' growth along these lines, I first followed the traditional line of using an "instrument" to get directly to the essence of students' thinking, and I was guided by the published work of Marcia Baxter Magolda. Lacking the resources to implement her full protocol of measuring tools, I worked with students to fashion a crude but reliable set of questions that ask students about their ways of knowing and their preferences for learning from authority or from discovery. After several years of teaching, I finally found a few methods that nudge students toward some commitment to critical evaluation. Now I am following Perkins' ideas in Teaching for Understanding by observing the development of critical evaluation and synthesis within course assignments such as term papers and research-based writing. No results as yet, but I derive some satisfaction from examining genuine samples of intellectual work rather than answers to my decontextualized prompts.
All this attention to students' thinking about knowledge has sensitized me to observing thinking among my faculty colleagues. We are masters of critical analysis, being well prepared and even eager to question and challenge the evidence, arguments, and conclusions of scholars in our own fields. Many of us, however, back away from a commitment to constructing a complex understanding when we read outside our primary field of scholarship. I may read and cite cultural anthropologists in my teaching, but I often present the work packaged exactly as I received it. I presume that I cannot bring sufficient knowledge and sophistication to challenge the conclusions I am offered, behaving in this regard very much like a beginning student. I notice this often among colleagues as well, as for example when I hear natural science or humanities professors cite Piaget in support of their teaching methods.
And coming full circle to assessment, I occasionally see familiar patterns in the way faculty members talk about measuring education. Some remain in the early undergraduate mode, citing studies that demonstrate the validity of this instrument or that. Without knowing how to ask questions about what observed predictions and associations demonstrate the utility of proxy measures, faculty simply accept the authority of testing specialists without questioning or understanding their methods or assumptions. Some more adventurous colleagues find multiple opinions among a range of testing researchers and claim that measurement cannot be done at all. A variant on that position is the notion that education is an ineffable thing that cannot in principle be measured, but both positions result in the same intellectual paralysis we observe in students trapped in uncritical relativism.
If Perry's original observations provide a lesson for assessment, it is that we need to commit ourselves to the harder task of learning enough about measurement that we can identify and/or construct the best possible ways to understand what our students are learning. There is a path through the unreliability of measures and the challenges of identifying appropriate intellectual work as a sample of complex cognition and affect, but it will take effort and openness to evidence and argument. We are very good at these skills within the confines of our own fields of study, and we need to learn to use those skills in new contexts in which we were not taught. That is the lesson to be taken from Teaching for Understanding; we enjoy seeing transferable learning in our students and we can revel in accomplishing it ourselves.
Is there an easy way to move our own community further in this intellectual development? Probably not, but perhaps we can start with the same interdisciplinary and integrative opportunities many call for in our teaching. One of Boyer's four scholarships is integration, and it is a less common form of scholarship than disciplinary discovery. Integration may also be undervalued. If we practice what we teach about integrative understanding, then perhaps we can break through our paralysis on measurement and begin to enjoy constructing representations of what our students know, can do, and understand.
|
