The first installment of the series featured Philip DiSalvio, dean of the College of Advancing & Professional Studies at the University of Massachusetts Boston, interviewing Carnegie Foundation ...]]>

In April, NEJHE launched its New Directions for Higher Education series to examine emerging issues, trends and ideas that have an impact on higher education policies, programs and practices.

The first installment of the series featured Philip DiSalvio, dean of the College of Advancing & Professional Studies at the University of Massachusetts Boston, interviewing Carnegie Foundation President Anthony Bryk about the future of the credit hour; the second featured DiSalvio's interview with Fastweb.com and FinAid.org Publisher Mark Kantrowitz about student debt; the third, DiSalvio’s interview with Lumina Foundation President and CEO Jamie P. Merisotis about Lumina’s commitment to enrolling and graduating more students from college; and most recently, his interview with American Council on Education (ACE) President Molly Corbett Broad about the efforts ACE is making to raise educational attainment in the U.S. and around the world.

In this installment of the series, DiSalvio speaks with Richard Arum, co-author (with Josipa Roksa) of Academically Adrift: Limited Learning on College Campuses. Arum and Roksa pose fundamental questions about whether undergraduates are really learning anything during their undergraduate experience. For a large proportion of students, Arum and Roksa's answer to that question is a definitive "no."

The context

Drawing upon survey responses, transcript data, and the Collegiate Learning Assessment (CLA) of more than 2,300 undergraduates at 24 institutions, Arum and Roksa deliver a stark message about the nature of undergraduate learning in the U.S.

Their analysis suggests that more than a third of American college seniors are no better at crucial types of writing and reasoning tasks than they were in their first semester of college. Furthermore, 45% of these students demonstrate no significant improvement in a range of skills—including critical thinking, complex reasoning, and writing—during their first two years of college.

Other findings show that, on average, students devote only slightly more than 12 hours per week to studying. Based on students' self-reports, Arum and Roksa suggest that such meager investments in studying might reflect meager demands placed on students in their courses in terms of reading and writing. Another finding of their work points to persistent racial and ethnic gaps in CLA scores—and those gaps widen, in the case of African-American students—over the course of four years of college

Some see their research results as a damning indictment of the American higher-education system. Conversely, their work has drawn its share of critics, who say their analysis falls short in its assessments of certain teaching and learning methods.

Nevertheless, the suggestion that four years of undergraduate classes make little difference in a student’s ability to synthesize knowledge and to put complex ideas on paper has serious implications for academic leaders and policymakers. Arum and Roksa may be delivering a wake-up call for those institutions failing to make undergraduate education a priority and falling short of properly preparing their undergraduates for 21st-century challenges.

The interview

DiSalvio: In your book Academically Adrift, Limited Learning on College Campuses, you conclude that a significant percentage of undergraduates are failing to develop the broad-based skills and knowledge they should be expected to master. How did you come to that conclusion?

Arum: The CLA, which we used to identify scores before and after a performance test, did not move up in a significant way for many students. The analysis showed that if the test was scaled from 0-100, 36% of the students didn’t go up even one point after four years.

What’s important to emphasize is that our conclusions do not rest solely on that instrument. We also surveyed students and asked them what they were doing in their course work and in their undergraduate education. It was those responses that we found even more powerful and troubling than the lack of growth demonstrated by the large numbers of students on the assessment.

Specifically, we asked them how many hours per week they studied and prepared for class. We found that on average, full-time college students in the U.S. studied only 12 to 13 hours per week and a third of that time was spent with their friends. Studying with their friends didn’t track at all with growth. In fact, it led to negative performance. Traditional studying alone is down today. Our analysis showed that 36% of these students studied alone five or fewer hours per week—less than one hour a day.

We also asked them about their reading and writing requirements in their class. Half of the students reported that in a typical semester, they did not have any class that asked them to write more than 20 pages over the course of the semester, and 32% said that they did not have a single class where they were asked to read on an average more than 40 pages per week.

This does not apply to all students, however. We found that some students grew on this indicator and that some were studying long hours. Some were taking rigorous and demanding courses. But what we identified in our data very clearly is that large numbers of students in U.S. higher education today are able to navigate through this system with very little demanded of them, and very little effort and very little learning, as measured by this CLA assessment.

It might be useful to say a bit about what the CLA performance test assesses. The CLA assesses critical thinking, complex reasoning and the ability to communicate in writing. These are general skills most individuals think all college graduates should develop. These skills are also what employers increasingly see as the core of what college graduates need to be successful in the workforce. These competencies are assessed by the CLA not through a standardized multiple-choice test, but rather students are given a task that might be asked of them by an employer in the future. Students are given a set of documents. They are asked to think critically about the documents. The reliability and validity of the information in the documents varies so they have to think critically about the usefulness of the data. They need to synthesize information across these various sources—a complex reasoning task. They then have to write a logical argument based on these documents using that information to provide evidence to support their points. This is, again, not a simple task but exactly the kind of task you’d expect college students to improve upon the longer they were in school. Unless of course, these students were either not studying at all nor being asked to study, and were not reading or writing. A pattern emerges where one might conclude that there is limited learning for all too many of these students.

DiSalvio: While not a direct rebuke to Academically Adrift, studies by the Council for Aid to Education offer a sunnier counter-narrative to your assertions about meager learning. Their studies suggest that college does have significant effects from freshman to graduating seniors. How do you account for that discrepancy between your research and the council’s research?

Arum:  In terms of the narrow measurement issues, our study followed several-thousand students from when they entered college to when they left college. They were given a test at two points in time, so that we could measure the actual growth that occurred. So our work, in a sense, is a descriptive report of what actually exists.

The Council for Aid to Education study that came out measures freshmen out of college and measures seniors out of college at the same point in time. That study tries to infer growth statistically through manipulation of the data. They need to statistically adjust their data because freshmen and seniors at a college or university look very different. Large numbers of students drop out of college. In many institutions, it’s 50% or more. So comparing a freshman class to the senior class is comparing apples and oranges.

Inferring growth by statistically adjusting for SAT is just coming up with an estimate based on these techniques. We’re not doing statistics at all on the descriptive findings I just noted. We simply said that this is where they started and this is where they finished. In fact, their statistical adjustments are inadequate and do not fully account for the attrition going on in colleges and universities. So they overestimate growth. That’s technically the difference.

Our work is often misinterpreted and misreported to say that students aren’t benefiting from college, that nothing is being learned and that college has no value. We have never said that. We have said that too many students are not applying themselves and not learning and moving up on this important indicator. This is not the only purpose that colleges and universities serve. And they have many functions outside this. Colleges and universities teach subject specific skills and contribute to larger socio-emotional development.

Yet our work is often mischaracterized as a condemnation of all college and all undergraduate education. In page after page, we highlight the important variation we see in the data and where some students are actually applying themselves and learning at quite reasonable rates. So again, some of the problems between our two studies are around this measurement issue.

There are significant numbers of students who grow from the undergraduate experience. We hope that no one misunderstands our work and thinks that we feel college is not a useful, critical, important thing for a young adult to embark upon. Even at the high cost of U.S. higher education, where students often go into significant debt to finance their education, we feel it is still a very sound investment for the vast majority of these students.

I think what happens often is when things become as public as the book has become, people take from it pieces that they like and use it for their own ends. The larger point we are making, is that colleges and universities should do a better job at dealing with these large numbers of students that clearly can be applying themselves and developing themselves more.

DiSalvio: You observed that the existing organizational cultures and practices of contemporary colleges and universities often fail to put a high priority on undergraduate learning. What do you think are the factors that contribute to that failure?

Arum:  There are multiple factors. One of the complications of identifying those factors, of course, is that U.S. higher education is quite diverse. Institutions such as UMass Boston may face a whole different set of issues around organizational culture than Boston College. Again, we have to be careful not to generalize, but at the same time we might be able to put forward some observations that may not apply everywhere but could apply to many schools.

A good place to start to address the question of factors is to observe how U.S. higher education has moved in the direction of a consumer-client satisfaction model. Higher education institutions have increasingly come to compete over individual students—to get them to apply and attend their colleges and universities—and then to keep them happy and satisfied while they are there.

In some ways, that’s a good thing. Organizations should be responsive to the clients that they’re servicing, but it has also led to all sorts of behaviors that are less than ideal. I have visited over 100 college and university campuses over the last few years and at almost every campus I visited, they would point out the new gym, the new student center and often the new dormitories being built. There is an arms race going on in U.S. higher education to provide greater amenities to 17-year-olds who go and visit campuses and make choices about where they should spend their next four years. While this is not a sensible way for the sector as a whole to be moving, it has moved very far in that direction.

This can be seen in the way instruction is often assessed on college campuses today, where too often the only measure of teaching quality that the institution considers is the course evaluation surveys that students fill out at the end of the semester. Those course evaluation surveys all too often are nothing but consumer-satisfaction surveys. “Did you like the class? Would you recommend it to a friend? Did you find it interesting?”

Many institutions are not asking questions about the academic content. “How many hours did you study? What kind of reading requirements did you have? What kind of feedback did you get on your paper?” So to the extent that institutions assess learning or teaching, it’s often again the consumer-client model, rather than a model that would promote academic rigor and excellence.

Another way to look at this issue would be to track expenditures. Over the last couple of decades, colleges and universities have increasingly invested in non-full-time faculty and student support services. The fastest growing sector of higher education is professional and quasi-professional staff in charge of student well-being. Now again, all things being equal, that may be well and good, except these are choices that the institution has to make about where investments are going to be made and where they are not going to be made. Increasingly, colleges and universities are making investments in non-academic functions through these amenities, through the student support services and again with a decline of full-time faculty.

So I think in broad strokes, those are some of the cultural factors that might be seen throughout higher education. This varies in many institutions and certainly not all are being judged exclusively on research and scholarship productivity, but many of the colleges and universities are busy, working to move up in their rankings. That again has little or nothing to do with undergraduate education and academic rigor. It has to do with what type of students you’re going to attract to the institution, the research and productivity of faculty. However it is not a measure of actual learning that’s occurring.

DiSalvio: In your research, you have found that learning in higher education is characterized by a persistent and a growing inequality. You maintain that there are significant differences in critical thinking, complex reasoning, writing skills when comparing groups of students from different family backgrounds and racial and ethnic groups. Given that there is a gap, what do you think has to be done to deal with that gap?

Arum:  The inequalities within higher education in terms of learning are a very important part of our book that is often ignored. In K-12 education, these inequalities have long been noted and we have a great deal of public policy discourse and public policy made to target and reduce them. For example, “No Child Left Behind,” with all its faults, brought increasing attention to assessing learning outcomes for not just average groups of students but for different racial groups of students, to make sure that all students, regardless of their backgrounds, were demonstrating growth and learning. In higher education, we haven’t had much of this debate at all. In fact, faculty often are in many institutions discouraged from working with students that are struggling because of the time commitments that are involved. They are implicitly encouraged to limit those investments and instead invest in the talented, and rewarding students in terms of their academic promise.

The struggling students are often diverted to remedial course work or remedial centers on campus. The effect of some of these programs and course work is unclear. I know there is a whole national debate emerging on the lack of effectiveness of remediation and remedial course work. Students are often put there, spin their wheels and take these courses repeatedly and their progress is compromised and suffers. We need to figure out how to do all the remediation in a smarter, more effective and efficient way. I think we need to pay greater attention to that area, just the way we do in K-12 education.

DiSalvio: You have said that the limited learning that exists on campuses qualifies as a significant problem and should be the subject of concern for policymakers and practitioners and parents and citizens. Yet you also say that limited learning on college campuses isn’t a crisis because the institutional actors implicated in the system are receiving the kind of organizational outcomes that they seek, so neither the institutions themselves nor the system as a whole seems to be in any way challenged or threatened. Why then should limited learning be a subject of concern?

Arum: In the book, we argued that we didn’t think it was a significant crisis because all the actors were generally happy. Institutions increasingly have been able to generate additional revenues through increasing tuition and other means. The students were generally happy, even those from whom not much was asked, because with their short-term orientation, it seemed like they had the best of both worlds. They are getting a college degree and it actually didn’t take much to do it. They had increasing time to devote to other pursuits, whether that was working additional hours to pay for their education or just engaging in leisure pursuits.

I think there’s evidence that actually both those things are going on. Students spend about five more hours today working for pay than they did in 1960. And there are another eight hours that are just unaccounted for in terms of their studying less, but not working more. So I think there is a lot of reason to think there has not been a crisis. Since that book came out in January 2011, increasingly there is a sense of crisis in the air. That is, I think, in part because of this unsustainable increase in the cost of higher education and that financial model coming up against some real limits.

The ability to increase tuition at twice the rate of inflation indefinitely and still have more students every year attend your institution, may now be hitting some natural limits. Debt load of students, the availability of additional funding for programs in Washington and economic circumstances are leaving large numbers of people to raise questions that during the time we wrote the book were not being asked. Additionally, what has changed is that technological disruption has accelerated. And so there is increased experimentation with alternative instructional delivery mechanisms, such as online education.

For those reasons, the sense of a crisis in higher education is becoming a more plausible argument, and in fact we are in a period of uncertainty when there appears to be some profound restructuring or reengineering happening.

Economically, today’s graduates have to compete in an increasingly globalized world. And so it’s not the case as it was maybe 40 or 50 years ago that their limited learning might not have mattered because they are still going to do better than the high school graduate who did not go to college. That still may be true to some degree, but they are competing today not just against the high school graduate but the software engineer in Ireland and the architectural design firm in Germany and the engineering companies in Asia.

The consequences of this limited learning for them and for the country as a whole are increasingly going to require addressing and remedying.

Limited learning in terms of critical thinking and complex reasoning and communication also has implications for the ability of our country in the future to have a functioning democratic system. The college graduates today are expected to be the civic leaders of tomorrow and yet the students that we’re tracking into the labor market were asked how often they read the newspaper in print or online and 36% of them said monthly or never. How can a functioning democratic system exist when the educated elite in the country are no longer reading news on a regular basis?

So, to think critically about the political rhetoric and to think systematically about the complex problems of the day are essential skills for the broader civic society. If we ignore this problem, I worry it will have much more profound implications than just our economy losing its competitive edge.

DiSalvio: What should higher education leaders and policymakers take away from your work and is it a call for action?

Arum:​ Hopefully it will be read as a call for action and that call for action will come from the leaders of higher education institutions. We feel strongly that the federal government, in a well-intentioned effort to address these problems, should not impose an accountability scheme on higher education in a centralized manner. We feel that would end up leading to all sorts of distortions and counterproductive changes as was seen in the "No Child Left Behind" program and the resultant imposed accountability frameworks imposed.

Because there is a real problem, demonstrated by consistent and overwhelming empirical data, we feel higher education has a responsibility to address this in a systematic way. The best place for that problem to be addressed, particularly in the U.S., with its decentralized higher education system and very large private sector, is at the institutional level.

Institutional leaders should take these issues seriously and must be willing to assess learning in their own institutions and to identify areas of concern to improve. We’ve been very clear to say that higher education institutions must answer three questions: 1) How are you measuring learning on your campus? 2) Where are the areas that need improvement? 3) What are you doing to improve those areas?

We should not need the federal government or a state government or even a board of trustees to tell us to do that. Every academic leader should be asking those questions because that is, in part, what defines academic leadership and what differentiates higher education institutions from an ordinary business.

DiSalvio: In your experience, how have higher education leaders generally responded to that call for action?

Arum: Higher education leaders overwhelmingly agree with the idea that there is a real problem. Some might express concern about the reputation of their own institution and say that the problem does not exist on their campus. But overwhelmingly, higher education leaders acknowledge that there is a problem because their institutional data confirms that the problem exists.

So, while it appears there is a widespread sense that there is indeed a problem that should be addressed, many feel they do not have the adequate tools to be able to do the work. There is not always a sense that the assessment tools available match up with their institution's needs.

While course evaluations are commonly used in higher education, I think different instruments might be used to track student’s objective learning growth. As those tools become available, our hope is these tools will be embraced and widely utilized by the vast majority of institutions.

Last month, I suggested we separate hype from reality—not so much to criticize distance learning, but to seek an even higher ideal. Much of what is thrust under the umbrella of distance learning isn’t conducted at much distance, isn’t well supported and limits opportunities for institution-wide collaboration and innovation. ...]]>

Last month, I suggested we separate hype from reality—not so much to criticize distance learning, but to seek an even higher ideal. Much of what is thrust under the umbrella of distance learning isn’t conducted at much distance, isn’t well supported and limits opportunities for institution-wide collaboration and innovation. Distance learning should be an exciting appeal, rather than just a pragmatic expediency—a positive good in itself, not a necessary evil. Settling for a low standard for online courses only validates the views of the skeptics, and justifies the doom-and-gloom malaise of those with pastoral memories of a higher learning that perhaps never was.

I offer a more aspirational definition of distance learning than simply deflecting class time to online activities: Reaching out beyond a region (nationally and even internationally) and providing a substantial investment in faculty and student support, an academic institution provides a full educational experience and learning community entirely online—worthy of the reputation and integrity of that institution.

Institutional resistance to online learning has been melting away during these recessionary times, as schools seek ways to address enrollment pressures without increasing faculty or classrooms. But the test for online learning should be based as much on learning efficacy as financial efficiency. Seeking comparability in learning outcomes should be the baseline standard. Even that understates the potential advantages that an online environment might create for faculty and students.

Faculty familiarity with technology should not be an advantage for some, but a generic function provided pervasively for the benefit of all. From our firsthand experience, we see examples of what a substantive and systematic distance education commitment can create. For example, instead of a faculty member just developing something that is used in one course, one professor, with significant support, developed a much more robust study of Boston’s Big Dig—with the idea that its use would transcend that one professor and that one course. The professor knew in advance that this component would have more utility and substance than just for his own immediate purposes. This investment in course development allows many different features of the Big Dig to be explored, and then potentially used as part of a “library” of tools by other faculty.

One program’s innovation can be applied to another: A Socratic technique for a law program worked beautifully for management case studies. Through a common platform and array of support services, courses can have a common look and feel, without the creative burden falling on the individual instructor.

With a collective commitment to distance learning and instructional technology, advances can be shared across an institution. Using “green screens” for faculty lectures, faculty can speak and illustrate at the same time. Well-constructed course materials that faculty devoted dozens of hours to developing can be archived in a media library for others to tap in future courses.

I have yet to meet a professor who hasn’t felt that teaching online makes for better in-class teaching. Opportunities abound to re-engineer the traditional classroom experience, to use technical tools to take some work out of the class setting, and to better appreciate that learning doesn’t best occur through one-way lecturing but through active student involvement—all powerful distance learning lessons that redound to the conventional classroom.

Student services can be pooled as well. A hotline for remote students can cover students’ various time zones and their weekend queries. Staff can provide webinars and discreetly monitor courses to check in with students at key points in the semester and provide quick responses to problems. Crisis intervention takes many forms; in one case, a staffer alertly caught a student posting bigoted offensive comments on a course discussion board. A central office can send discs and files to parts of the world with low bandwidth—for example, to soldiers deployed in the mountains of Afghanistan. When one Port-au-Prince student’s routine trips to Miami to take his exams were disrupted by the various Haitian crises, staff arranged for off-cycle proctored exams. Doctoral qualifying examinations have been administered in real time with students and faculty scattered in various locales. Deaf students experience distance learning through closed captioning. The opportunities to accommodate students’ needs are boundless and the examples inspirational.

Serendipitous community-building is perhaps the most exciting byproduct of a robust online environment. Rather than the typical ships passing in the night, part-time students get to know one another as they progress through a common curriculum, regardless of their busy lives and competing demands. And the potential for student diversity is far greater as distance learning expands the school’s sphere of influence beyond the limits of local homogeneity. We have built virtual space for students to network outside their course work. In one case, a course on the biology of food led some students to coauthor a class cookbook; in another, IT students designed their own career advising network; and another group created virtual “pizza and beer” gatherings across the U.S. to stay connected via Skype. Our students then tap formal university events and milestones as opportunities to congregate and finally meet one another in person.

Though concealed within the data, these are truly exciting times for recreating and redefining the learning process—through the roles faculty play, the opportunities to test new tools and techniques, the access and interaction of students across diverse locales and lifestyles, and the reach of institutions beyond their narrow borders. The future of distance learning is more about creating community than exploiting technology, more about enhancing education than enrollments—and even more about academic courage, leadership and innovation. The opportunities are endless, constrained only by our own imagination.

Jay A. Halfond is dean of Metropolitan College and Extended Education at Boston University.

G. K. Chesterton famously once said: “Christianity has not been tried and found wanting; it has been found difficult and not tried.” This, I believe, applies to distance learning as well. There is far too much self-congratulatory hyperbole about the growth and pervasiveness of online learning – which exaggerates reality and overlooks the true revolution ...]]>

G. K. Chesterton famously once said: “Christianity has not been tried and found wanting; it has been found difficult and not tried.” This, I believe, applies to distance learning as well. There is far too much self-congratulatory hyperbole about the growth and pervasiveness of online learning – which exaggerates reality and overlooks the true revolution occurring less visibly.

Much of what is shoved under the rubric of distance learning is not conducted at a distance, doesn’t attract new learners to these institutions, and isn’t terribly well supported by the schools boasting large online enrollments.

The most dramatic, frequently quoted, and perhaps embellished statistics come from the Sloan survey of more than 2,500 colleges and universities, who claim that more than one in four students took at least one course online. Almost five million students at any one time were learning online. This definition of online assumes at least 80% of the content occurs outside class time, thus measuring a shift from the classroom to the computer, but not necessarily a complete leap from a local to a virtual campus.

Should you opt to confront one of the wizards of Oz claiming large numbers of online enrollees, or giddy about the ubiquity of distance learning, here are a few questions guaranteed to unnerve them:

How many students do you actually have enrolled exclusively online? I have asked this question of leaders of some of the major institutions in the distance learning business. The typical answer is they do not know, nor even think to track this information, since online courses are simply a convenience and option offered to their existing student population.

How many students taking online courses actually live at a distance where they couldn’t ever commute to your campus? “Distance” learning is largely a misnomer: Few institutions attract students beyond their catchment area through online offerings, nor do they even try.

How many academic degree programs do you have fully committed to distance students in which the complete menu of required courses is guaranteed to be available online? Many institutions offer random web-based courses, scheduled sporadically, subject to faculty whims.

How do you provide faculty with assistance to develop their online courses? (By now, you are making the boaster squirm a bit, as you shift the questioning to support for distance learning.)

What is your staff size relative to the number of online courses offered? Do you employ instructional designers and media and graphics specialists to shadow the faculty, or do you just provide an online tutorial for faculty to work self-sufficiently with the learning management system? Distance learning has become a laissez faire crapshoot—a technological tool to be used whenever and however one chooses.

How does your institution establish and maintain academic standards, expectations and quality in the online arena? How do you know that students receive a comparable educational experience? Do you routinely monitor online courses to address problems that arise? Do you have a help desk specifically designed to support students at a distance? Have your institution’s resource operations—especially the library, bookstore and registrar—adjusted their hours, accessibility and services to accommodate remote students?

By now, you have almost guaranteed that you will not be invited back to future gatherings.  But, if you want to go in for the kill, begin to penetrate their understanding of the online student community, or lack thereof.

How well do online students get to know one another in their courses and programs? How do you promote affiliation, community, and peer learning? How do you connect these students to the mothership?  What are the demographic characteristics of your online students that differentiate them from those on campus? What applied research have you conducted to measure student satisfaction, comparable learning outcomes and impact on the academic culture? How have you taken the lessons learned online as opportunities to improve classroom instruction?

The idealistic goal should be to create an exciting and engaging learning environment where the remote student can have a holistic experience by learning from faculty, fellow students drawn from diverse backgrounds (geographically and otherwise), and course materials. Their institutions would be obsessed with attaining comparability with the conventional classroom and campus, if not a superior learning experience. And they would be investing significantly and collectively in making that happen.

Like Chesterton said of Christianity, most haven’t yet really attained the true potential of distance learning. American institutions have made great progress in creating technological tools that allow faculty, generally acting independently, to mitigate face time in favor of work online. Asynchronicity—not being the same place at the same time—is not a sufficient standard in itself to fulfill the promise of distance learning.

I offer a more idealistic and aspirational definition: Reaching out beyond a region (nationally and even internationally), and providing a substantial investment in faculty and student support, an academic institution provides a full educational experience and learning community entirely online—worthy of the reputation and integrity of that institution. How many would still boast that we have achieved that standard?

________________________________________________________________________

Jay A. Halfond is dean of Metropolitan College and Extended Education at Boston University. In a future one of these monthly columns, he will suggest what it might take to achieve the ideal of distance learning, and welcomes your comments and insights.