Home > Resources > Report

Report

Leadership for Learning

Report on a Listening

September 7 - 9, 2006

Over seventy invited participants—leaders of Teagle-funded projects, college presidents and administrators, foundation colleagues, Teagle board members and staff, and others—gathered on the Blue Ridge in North Carolina for a Listening on what needs to be done to bring student engagement and learning to a higher level. Any strategy would surely involve applying results that emerge from well-crafted assessment done under the guidance of faculty and administrative leaders. Beginning with this premise, the Foundation then wanted to know, "what else?"

Seven panel discussions were set up to help answer this question by articulating the issues at hand, and identifying some good work being done, as well as some places where improvement is needed. The panels were:

What is assessment teaching us about strengthening education in the liberal arts and sciences?

How can Teagle ad hoc collaboratives in assessment lead to more robust student learning?

Digression: Assessment imposed from above?

Knowledge transfer

Is there balm in Gilead?: Satisfaction, reward, recognition, or what?

Under the surface of all the above: Leadership for improvement of student learning

How can philanthropy help?

What follows next is a summary of recurring themes, important points, and possible steps forward that I heard from the discussions.

Recurring Themes

Assessment, Outcomes, and Student Learning

Assessment should, and indeed needs, to always be done in the service of teaching and learning. For assessment to be productive and useful, campuses need to complete the "feedback loop." Data need to be drawn from multiple points through multiple tools and methods. Once collected, data need to be analyzed and the results used (also, data have to "allow" for analysis, as well as be easy to use) by institutions in their strategic plan, and by faculty in their classrooms. If the "feedback loop" is not closed, then assessment may end up "running on its own track," that is, it may turn into assessment only done for the sake of assessment.

An alternative version of the "feedback loop" is the "feedback spiral" which insists that the assessment of student learning outcomes continually builds on the work done before, taking this work in a forward (or upward) direction. Campuses must repeatedly pursue these actions: hypothesize, collect, analyze, act, hypothesize, collect, analyze, act…

Many things are being done on the assessment front, locally on campus, and nationally across the higher education sector. The question, therefore, is not, "Is assessment being done?", but rather, "Are we—that is, those who believe that assessment can better student learning—telling the story well enough? A participant who has spent much of his career at research universities observed that the story told at the Listening is very much one of private liberal arts colleges, not necessarily one of public institutions. He argued that the good work being done needs to be communicated. Re-iterating the same theme, another participant commented that local, on-the-ground efforts to increase student learning need to be made transparent to external stakeholders, and consequently, that outcomes should not be assessed by standardized tests.

Why should assessment be done at the local level? Campuses identify outcomes that are important to them, and from that, define and measure them differently. At the same time, assessment needs to be done in context, especially when comparing the measurements of student learning goals at one institution with another. Comparisons should be done within an institution's peer group for the assessment to be realistic and genuine. Also, relevant standards of performance should be developed within the peer group.

When talking about assessment, one needs to make clear the level of assessment one is talking about: individual, classroom, project, program, department, institution, discipline, regional, national, international.

Perfection should not be the standard and it should not be the enemy of the good. For example, while there are a number of available assessment instruments that can effectively measure student learning outcomes, they are by no means perfect. Faculty and administrators should keep this in mind, but at the same time they should not be deterred from using these instruments because of their possible imperfections as they can still generate a lot of good and useful information about what and how students learn.

Many assessment reports focus too much on process and not enough on findings and results. The best results should be reported, and more importantly, framed in such a way that is productive, positive, and revealing.

Campuses should share with one another the nuts-and-bolts of their assessment efforts. Building such an information network is helpful so that the wheel does not need to be re-invented every time. One participant pointed out that state institutions, many of which have considerable experience with assessment, can teach private colleges and universities a lot about how assessment can be done.

Faculty, Student Learning, and (Not) Training to be a Teacher

Campuses must not forget that assessment is a means to an end and that end is better teaching and learning. Assessment should not be done for its own sake, nor merely to satisfy some external body, but to improve how students learn. In turn, faculty need to be encouraged—and ideally need to see—that teaching is part of their professional identity and that teaching entails more than just conveying knowledge to students. (Resistance to assessment stems in part from the belief of some faculty that because they have close relationships with their students, they don't need to assess learning outcomes. It can also stem from their belief that they lack the proper expertise to assess their students' learning.)

For assessment to have an impact on teaching and learning, campuses need to support a culture of assessment. Strong leadership, a financial commitment, and faculty incentives and development modules are some of the factors needed to build this culture. Such initiatives need to be institutionalized.

Faculty buy-in and intellectual engagement are key to embedding an assessment culture on campus. One participant suggested that assessment should be made an extension of what is happening at the faculty level, meaning that it should be relevant to what is happening in their classrooms. Another proposed that faculty should be asked, "What do you want to know about your students?" Yet another emphasized that there should be venues and occasions for campuses to reflect on assessment and what it can do for teaching and learning (in this way, assessment will be normalized and the campus will be socialized), that these reflections should be consequential so that ideas that emerge are seriously pursued, and that there should be continuous experimentation with the local experiences of a campus. What might help too is if faculty are encouraged to use assessment data in a variety of arenas, not just to enhance student learning. For example, when administrators consider which faculty projects to undertake, they should give more weight to those that come with assessment data. What must not be done, however, is to feed faculty "watered down" assessment just so that it becomes easier for them to swallow.

But what do faculty actually want? Faculty are diverse, and so to ask this question in the general, is difficult. One participant offered this insight: faculty members are caught between two opposing impulses, one from the "conservative right" and the other from the "sentimental left." Following the former, faculty want freedom, money, and power; following the latter, faculty have a desire for altruism, a desire to effect social change, and a desire to be a part of a genuine community that responses and engages the work they do. Faculty are also concerned with the needs of their private lives (their families, especially).

When building faculty (intellectual) engagement, what is more important? The initiatives that bring faculty together, or faculty's sense that they are part of a collective, intellectual project? Related to this point, one participant suggested that to hook faculty and to get them to bring about change on a campus, administrators will need to offer them "pleasure," characterized here as intellectual curiosity. A campus, however, fosters a number of structures that can impede the pleasure of intellectual curiosity, and that can drive away the excitement of teaching and learning. These include the tenure system, departmental organization, and money. Such structures are driving potential faculty away: many assistant professors do not get tenure, an increasing number of PhDs are not going into academia, and many graduate students drop out of their programs before getting their degrees.

Building engagement is important, and the introduction of "cool," new projects is something a campus can do to peak faculty interest. This is not an all-out good, however, and in fact can become problematic especially as faculty participate—or are asked to participate—in a number of these new projects. Responsibilities can add up to the point that faculty are spread too thin.

How does a campus move from building faculty engagement and ensuring faculty buy-in? How does it cultivate "faculty leadership"? One participant suggested a number of practical avenues: (1) Focus on "exemplary citizens," that is, those who take seriously the role of the "scholar-teacher;" (2) Focus on departments, these smaller and discrete communities that can be energized; (3) Focus on department chairs who already serve as mediators between faculty and the administration; (4) Be intentional about incentives provided for faculty as well as the ways that [faculty] behaviors are rewarded, and in particular, institute a systematic means of recognizing good teaching.

An institution can bend over backwards to get faculty buy-in, to ensure that faculty take student learning seriously, and to promote assessment as the means to improve how students learn. But try as they might, faculty were taught to prize research and scholarship in graduate school, not teaching and learning; the former is what will provide fame and influence, while the latter is the task that helps pay the bills. With this orientation, campuses surely have their work cut out for them.

Other Smart and Important Points

Taking student learning to a higher level is not a new thought, but there is an urgency to do so today, and with that, an urgency to assess student learning outcomes.

Does better teaching necessarily mean that it takes more time?

Is the public aware of the learning outcomes that institutions want their students to achieve? On an even more basic level, does the public care that colleges and universities are undertaking assessment measures of student learning?

Assessment is about social capital, that is, it is about transforming campuses and building relationships so that teaching and learning can be done at a higher level.

While it is tempting to construct a master narrative about assessment, it may make better sense—and it is certainly more useful—to consider assessment as a dialogic process, one that allows for self-correction, one that happens in incremental and iterative steps, one that is just as much about process—especially in the questions asked as they shape what data is collected—as it is about discovery.

Assessment can be most effective when it is woven into the culture of a campus, and especially when it is integrated with ongoing projects and research.

Assessment needs to take into account the fact that students come to college with varying agendas.

While anecdotes lack structure, they can, with some massaging, be turned into very useful information that can serve as a basis for some kinds of assessment.

Knowledge transfer of assessment and student learning needs to move in two directions: outward, to external environments and constituencies, and inward, among faculty and administrators such that a "community of practice" is developed.

As important as faculty buy-in is, a campus must also take seriously its administration's buy-in. One participant wondered whether faculty are in fact more ambivalent [towards assessment] than is generally thought. He questioned whether it is really administrators who hold a campus back as assessment and the improvement of student learning are costly ventures for which there is no room in and institution's budget.

Often, faculty's good work on teaching and learning raises the caliber of their institution, but such work does hardly anything for the individual faculty member as it is not professionally rewarded like scholarship and research.

If campuses talk about faculty "pleasure," then they need to also take seriously "student pleasure." Institutions need to ask: Where are students today? What motivates them? Where are they flocking?

There is a difference between "creativity" and "innovation." While innovation can push a campus forward, higher education is also in the business of conservation and preservation. Creativity is needed to balance this tension, to preserve traditions and habits of mind while at the same time keeping institutions current in changing contexts.

Mission statements have often become "impenetrable" and unhelpful to a campus. For mission statements to be meaningful and relevant, they have to be memorable to, and inspire commitment from, the campus community; they have to define what a campus does and be aligned with what happens in the classroom and with how a campus knows how it is doing.

Going Forward

What can a campus do? It can:

Focus faculty on teaching and learning by investing in a "center for liberal arts teaching and scholarship" that can provide campuses with a defined space to discuss pedagogy.

Include, in new faculty orientation, components that address student learning as a primary concern.

Revise the tenure process to include a component that takes seriously teaching and learning (if the institution does not already do so). When faculty are interviewed for teaching jobs, or when they come up for their tenure review, they should not only be asked to outline their pedagogical approaches, but also to describe what learning outcomes they want their students to develop in their courses, and in particular, to explain how they would know that these learning outcomes have been met.

Help cultivate departments as mini campus communities, and help them undertake a practice of continuous experimentation when it comes to student learning.

Work with graduate schools to ensure that they take seriously, from the very beginning, the importance of student learning. What may help here is a stronger connection between research universities and liberal arts colleges.

Mine education research more effectively to discover effective pedagogical models; often, there is no need to re-invent the wheel.

What can a foundation do? It can:

Fund projects that make undergraduate education a concern for professional societies, and will create a ripple effect through the discipline and departments on campuses.

Structure a Request for Proposals the nurtures new faculty as "scholar-teachers" and encourages them to be "exemplary citizens" on campus.

Fund smaller and less scientific (assessment) studies. Recognize that informal studies and small-step projects can also produce good information. One way to encourage local and continuous experimentation is to undertake a re-granting initiative which provides a campus (or another organization) with a grant that they can then use to run their own competitions for small-step projects.

Fund overhead. Recognize that there are real costs to doing assessment; recognize that creativity (in thinking about teaching and learning) costs money.

Fund consortia so that resources can be combined.

Fund longitudinal surveys of students so that student learning can be tracked over a long period of time.

Create a "speakers bureau" that sends qualified individuals from one campus to another to lend their expertise.

Create an information clearinghouse on assessment, pedagogy, and student learning on the internet.

Cheryl D. Ching
October 5, 2006