NEBHE convened more than 400 New England educators and opinion leaders in Boston in mid-October to discuss these new opportunities for students and challenges for traditional higher education institutions.

The speakers included EDUCAUSE President Diana Oblinger (below) who cited among signs of the newly connected world of open learning: digitized learning, student empowerment, peer-to-peer learning and an acknowledgment of student swirl, including “reverse transfer” from four-year colleges to community colleges and other kinds of institutions.

Oblinger noted that anyone can participate in the new open learning. Reminiscent in some ways of Wikipedia and fueled by in social innovations such as “crowdsourcing” and “do-it-yourself” instruction, the new models are rife with many of the edu-terms you’ve (over-)heard for years, but they are suddenly more cohesive and seem to have more momentum.

Models include Khan Academy and MOOCs (massive open online courses). They are fascinating modes of delivery with sophisticated analytics systems for learning assessment. (Still, for as along as the question of what students should learn goes unanswered, such issues about delivery should be noted with an asterisk.)

Oblinger also explained how groups such as Persistence Plus give at-risk students “nudges” via mobile devices to remind them to study for their exams, for example. She spoke about using technology for learning tools of the teaching trade through simSchool for pre- and in-service teachers, instructors and administrators to improve their knowledge and confidence.

Shocked at MIT

MIT professor of electrical engineering and computer science Anant Agarwal is the president of the nonprofit edX created by Harvard and MIT. Agarwal mocked how little has changed in higher ed over the past century. He showed slides of a recent MIT class, contrasted with one from a half-century earlier. “What do you notice? Whoop-de-do, we have colored seats … and one of the most spectacular inventions of all time in education has been sliding backboards,” he said. He then showed an edx class being offered to high school students in ... as the audience was surprised to learn ... Mongolia.

Agarwal contended that courses offered via edX are as rigorous as those offered on-campus. With no marketing, nearly 155,000 students from more than 160 countries registered for the inaugural Circuits and Electronics course; just over 7,000 wound up certified. The students were split evenly between traditional college-age (and a few high-school age) on one hand, and adult learners on the other.

Teaching 150,000-plus students required the same staff resources as teaching a 150-person class. Because of effective peer interaction, Agarwal predicted, fewer staff will be needed next time around. Students watch videos of about five to 10 minutes, not unlike those made famous by Agarwal's student Sal Khan, founder of Khan Academy. The videos are interwoven with short interactive exercises and online laboratories.

Agarwal noted that skeptics wondered why MIT and Harvard would give away the platform. The answer, he said, is that with open-source, you get "the whole community working together and improving the platform ... think of it as peer-to-peer software development."

When kids hit age 13, Agarwal added, they go digital and speak teenglish composed of grunts and silence. They don’t even answer the phone, he said, so “text them!” The students love instant feedback, said Agarwal, like the green check mark that is superimposed when they get something right.

An audience member asked about courses in areas such as the humanities that don’t lend themselves to the big green check mark. Agarwal noted that edX is exploring various assessments to grade open-form content and peer learning, but there’s a long way to go.

Another asked the difference between the 155,000 who started the program and the 7,000 or so who made it through. Agarwal said analytics show many of the students who started were not prepared, and the successful students simply spent more time doing the exercises.

Up from subprime

John Katzman said technology has been held out as a solution to higher education’s competitive challenges, but online learning began as the province of what some people would call “subprime educators"  ... and he showed logo of University of Phoenix.

The founder of Princeton Review, 2Tor and most recently, Noodle, Katzman noted that while the Internet began on college campuses, most tech-ed programs such as Blackboard flanked traditional campuses, rather than replacing them.

Noting that technology’s cost structure is higher at a small scale and lower at a large scale, Katzman extolled the collaboration long absent from the siloed and jealous higher education sector. He showed a slide with boxes labeling colleges as elite, middle, entry, two-year, four-year, MBA ... PhD., and suggested there'll be consolidation of institutions rewarding scale within each of those boxes, but not across them. If a college is a regional brand, rather than a global one, collaboration is especially crucial to get the benefits of scale.

Katzman contended that instructors say students are learning better and a larger percentage of online students walk for graduation than on-campus students. Colleges are also tracking how many students donate each year, reflecting a feeling of team, he said.

Another trend, he pointed out, is “edutourism.” Students from around the world, especially Asia, want to go to the U.S. for its reputation for academic freedom.

Reaching more students

Stanford University computer science professor Andrew Ng explained that the group he co-founded called Coursera uses technology to offer courses from top schools. He said his normal class reaches 400 students at Stanford; last year, when he put the course online, he reached 100,000 students.

Ng noted that the online learning is more interactive than the bricks-and-mortar classroom in terms of students answering questions. "When I ask a question in my classroom, usually half the class is still madly scribbling the last thing I said. About 10% are on zoned out on Facebook and there's one smartypants in the first row who blurts out the answer, and I feel really good that one student knew the answer and the class moves on with only one student having gotten in to attempt an answer. On the website, the video stops, and every student gets to attempt an answer."

He said a U.S. Department of Education study showed that online instruction and classroom instruction have comparable high quality, and a blend of the two is even better.

But if anyone can take a Princeton course online, he asked, why would they go to the campus. Ng conceded that the answer is the real value is not just the content, but rather the interaction with the professors and other equally bright students.

“Asking the students to watch the content at home allows them to come into the classroom and have more interactive discussions,” said Ng. “By marrying the idea of MOOCs and flipped classrooms, we’ve flipped many classrooms at many of our 33 partner campuses.”

At Coursera, Ng said, we think high-quality education is not a privilege for the elite, but a fundamental human right. Ng noted further that for many people, higher education is not a choice between Princeton online and the Princeton campus, but rather between online and nothing.

Branding and monetizing

Chris Vogel, who wrote a story on edX for Boston Magazine, asked if the new models cheapen a school’s brand? Katzman noted that colleges can dilute their brand by admitting students online whom they wouldn’t normally take or by giving students a bad experience, but, he said, scale actually correlates positively to reputation. Ng noted that Stanford’s brand has not been hurt, and Stanford faculty like the idea of reaching so many more students.

Vogel then asked a $64,000 question: how do you make money off the model? Ng said he often is asked: Why don’t you charge $5 for a course? “The most needy people in society not only don’t have $5 ... probably don’t have a credit card, he said. "But teaching online courses is an expensive enterprise; we need to bring revenue back to share with our university partners to cover our costs,” said Ng, adding: “Many of our partners have expressed interest in charging for a university-branded certificate with the course content being free.”

Coursera is also working on monetizing job placement. “If you do well in a Princeton class or a Stanford or Cal Tech class, that’s a strong sign that you’re a talented individual and companies would love to talk to you,” said Ng. “Being mindful of privacy, we’re piloting introduction between our top students and employers and charging employers for this.”

But, Vogel pressed, will employers appreciate certificates as much as degrees? Ng said yes. “There are many areas where having just one additional course that teaches you some latest technology can significantly boost someone’s income. Employers also take seriously the fact that these are Princeton, Cal Tech and Stanford classes, and it’s not easy to do well in them. Our demographic is people who are self-motivated and decided, for whatever reason, to spend their free time taking one of these ridiculously hard courses.”

I want Ghandi

Saul Kaplan, founder and “Chief Catalyst” at the Business Innovation Factory, facilitated a session called “Gandhian Innovation and Creating the $10,000 Degree.”

The first panelist was the first university president to earn approval from the New England Association of Schools and Colleges (NEASC), the regional accrediting agency, for a competency-based program, based not on credit hours but on competencies. Southern New Hampshire University's program will next be considered by the U.S. Department of Education.

“If the guys at Coursera and 2tor are working with USC or MIT on circuitry," said SNHU President Paul LeBlanc, "we’re talking about the 37 to 40 million Americans who have some credits but no degree and the 30 million who have no college credits at all."

LeBlanc said he is skeptical of the ability of established players being able to do disruptive game-changing innovation, except when programs with very high brands, built on exclusivity, release their brands. "If Podunk University does that same course with that delivery method, they’re not going to have 100 students showing up. If Stanford does it, who doesn’t want to have a Stanford course on their credentials?”

In an economically booming area of Texas that is home to 155,000 oil wells, the University of Texas of the Permian Basin has created a $10,000 college degree, even as other UT campuses raise tuition. President W. David Watts explained that the UTPB “Texas Science Scholars” offer the deal in the lowest-producing majors, such as chemistry.

Ed Klonoski, president, Charter Oak State College, compared the 40-year-old Charter Oak to a fish that had lungs—it proved  an advantage when the oceans dried up. We accepted credits from any regionally accredited institution and for portfolios and prior learning assessment. Now the idea is ripe. He told of a family that will earn seven degrees from Charter Oak for a total of $60,000. Klonoski called for a national common definition of competency-based learning, noting that he and his New England colleagues will be swamped by Coursera, 2tor and other national powerhouses.

Assessing assessment

Rosemarie Nassif, special advisor to the assistant secretary of the U.S. Department of Education quipped: “I’m from the federal government and I’m hear to help you.” Joking aside, her department, an occasional whipping boy for tech reformers, is indeed obsessed with meeting President Obama's goal to make the U.S. the world leader in college degrees by 2020. Meeting that goal could hinge on two major themes at the NEBHE conference: the role of IT and competency-based assessment.

Nassif noted that education can be assessed in new ways regardless of where the learning came from, including work and life experience. Such alternative assessment reveals more than transcripts can. It is time-independent allowing students to progress at their given pace, it increases affordability and allows for flexibility. Nassif called for forging widely accepted learning outcomes. She suggested higher ed could learn from the Common Core State Standards process being used in K-12 and involving industry and states.

Sally M. Johnstone, vice president for academic advancement at the Western Governors University (WGU), spoke of the online institution headquartered in Salt Lake City, Utah. When WGU was formed in 1996, the requirements were: serve workforce development, use technology to its fullest, and make it competency-based. WGU currently enrolls 36,000 students. The cost to student is $6,000 a year, “How long the students stay with the university is up to the student … we don’t count how long it takes a student, what we count is how well a student can demonstrate the skills and knowledge that have been defined for a bachelor’s or master’s degree,” said Johnstone. Today, WGU has about 600 full-time faculty, external councils comprising industry and academic representatives work on competencies in four schools: business, IT, health profession and teacher education … and committees create the courses to match the competencies.

Stinkin Badges?

Erin Knight, who leads the learning work at Mozilla, known for its mission to protect the open web and its open-source Firefox web browser, spoke of her "Open Badges" work supported by the MacArthur Foundation. The alternative credentialing system aims to allow the learner to control the credentials, moving away from seat time.

“The only things in the game right now are grades, transcripts and degrees, and there are only certain ways you can get those … there’s a bunch of learning that’s getting missed. The idea with badges is to have an alternative system that allows us to supplement the degree,” said Knight.

“Instead of having just a grade at the end of a course or a degree, we can recognize various competencies along the way,” said Knight. She said many of her peers in her master’s degree group were different kinds of learners who took different pathways, but the degree just presents them as all the same. Badges can capture a more comprehensive way to talk about their learning than just one-line naming degree.

Badges are not just images or digital stickers. Baked in is who issued the badge and when, a link to what they require, endorsements and links to urls of artifacts.

We want all the badges to work together, Knight said. Mozilla has built the plumbing on what should be in the badges—essentially digital resumes, which are evidence-based. “The learner is managing the collections and building identity and entrepreneurial side of things, and on the display side, there’s consumption for jobs and real results.”  Knight thinks employers will look at both badges and degrees, because the degrees don’t offer enough granular information. The narrative works particularly well in informal learning, out of school and on-the-job learning experiences, but colleges like Purdue and UC Davis are among those introducing badge systems for courses.

“A badge is just recognition of the learning experience," explained Knight. "Is there a way we can add more information to that badge that starts to get to the same results we lean on accrediting bodies to do now without requiring just a few top-down bodies to say, ‘Yes, this is OK, '” said Knight.

One session focused on“Flipped Instruction: The Interactive Classroom.” Julie Schell, senior educational research associate of the Mazur Group at Harvard University, told of the past and present of the flipped classroom idea. Schell quoted Bergmann and Sams: “Flipping the classroom is ... [a] mindset redirecting attention away from the teacher and putting attention on the learner and the learning.” One result is students spend class time on what we used to think of as “homework” and home-time viewing “lectures.” Schell explained the methods that inspired her blog Turn to Your Neighbor.

Click here for more on the conference ... And please watch here for additional videos ...

In 1999, the Vermont State Colleges (VSC)—comprising Castleton, Johnson and Lyndon state colleges, the Community College of Vermont and the Vermont Technical College—initiated a systemwide planning process that identified multiple strategic initiatives, including several designed to improve outcomes assessment and accountability. One initiative called for the establishment of common graduation standards for all students across the five colleges, at both the associate and bachelor’s levels. The board of trustees wanted to provide a “guarantee” to the public and employers that every graduate of the VSC could demonstrate essential skills for success after college.

The chancellor established a systemwide steering committee to oversee the graduation standards initiative. The committee included faculty representatives from each college and academic deans, and was co-chaired by the academic vice president of the system and the president of one of the four-year colleges. Faculty on the committee were expected to serve as liaisons to the faculty assemblies on each campus, to allow for broader faculty input and to facilitate endorsement of the committee’s plan. Likewise, reports were provided frequently for the state colleges’ Council of Presidents, the chancellor and the broader VSC community.

Areas of competency

The steering committee ultimately proposed six areas of competency: writing, quantitative reasoning, information literacy, oral communication, civic engagement, and critical thinking. Facing significant opposition to the entire initiative from a vocal group of faculty, the steering committee formed faculty-majority subcommittees to define the outcomes and propose assessment strategies for each standard. Several months into this process, civic engagement and critical thinking were permanently tabled as the subcommittees were sharply divided about the feasibility of valid assessment in those areas. This elevated the political challenges associated with assessing a limited set of skills rather than a broad set of learning outcomes such as those identified by the Association of American Colleges and Universities through Liberal Education and America’s Promise (LEAP).

Unexpectedly, it was easier to come to agreement about specific language for defining learning outcomes than about what to call the entire set of competencies. Faculty vehemently opposed the initial label of “minimum competencies,” on the grounds that it potentially conflated expectations for collegiate learning with those at the high school level. Faculty ultimately agreed to the term “graduation standards.”  Of course, this semantic shift did not mitigate the challenges associated with establishing appropriate performance levels for the standards, made politically charged given the VSC’s public access mission and that over 60% of students are the first in their families to attend college. Many expressed concerns about creating barriers to graduation. But by far, the most controversy centered on the assessment tool itself.

Fundamental methodological questions were debated. Would faculty design the assessments or would the VSC select commercially available instruments? Who would set the standards for passing? Would all students be assessed or would a sampling technique be employed? At what point in time would students be assessed? Ultimately the steering committee recommended a politically acceptable compromise—adoption of common statements of learning outcomes across the five colleges and agreement on a set of parameters for assessing the outcomes (including that every student would be assessed), while allowing each college to develop and implement campus-specific assessments for each standard. This plan satisfied the demands of the board of trustees and chancellor for common learning outcomes and a “guarantee” of minimum competency, and provided a mechanism for faculty buy-in at the campus level.

Implementation

The academic vice president in the system office worked closely with the college presidents and academic deans to ensure progress on the development of local assessments. The implementation timeline was staggered over a five-year period, beginning with the development of a writing assessment that met the requirements established by the steering committee. One college already had in place an institutional writing proficiency exam, and another had in place portfolio-based writing assessment. These models and others were shared among faculty and provided a foundation for the timely and relatively smooth implementation of writing assessments across the system.

The other three areas proved more difficult to implement. There was wide disagreement about the level at which students should demonstrate proficiency in quantitative reasoning, especially for students in STEM fields as opposed to those majoring in the humanities. There was disagreement about how to differentiate minimum competency in information literacy from what might be expected of high school graduates. Finally, there was ongoing confusion about how to differentiate expectations at the associate and bachelor’s levels. Concerns arose about the potential for wide variation across colleges in the performance levels being assessed, as well as in the overall quality of the assessments.

Several years into the implementation process, the academic vice president in the system office and academic deans at the colleges designed and implemented a process to regularly review the assessment methods and results at the colleges. In addition to annual monitoring of results across all assessments, one competency is evaluated comprehensively per year on a rotating basis. Faculty from across the colleges go together in a retreat format to reconsider the common learning outcomes, analyze local assessment methodology and results, and make recommendations to the presidents and chancellor for improving the process. This provided a mechanism for faculty to have a significant role in the ongoing improvement of the assessment system, while supporting the broader strategy of engaging faculty in assessment as part of the regular work of teaching and learning.

Given that writing was the first area to be implemented, it was also the first to be evaluated. As a result, revisions were made to the learning outcomes, as were recommendations for improving the reliability and validity of the local assessments. Writing faculty from across the system shared student writing samples and assessment rubrics, a process they found both useful and engaging, particularly given the opportunity for expanded colleagueship beyond the small departments in VSC colleges. Most recently, the assessment of information literacy was reviewed, which identified  areas of concern in the current approach, including the wide variability in expectations across departments within colleges.  Additionally, there was agreement that the standards and implementation are not rigorous enough in relation to intellectual property and the ethical use of information

Results and lessons learned

Given that all students would be assessed across all standards, the instruments developed by faculty at each college were, in theory, high-stakes. The VSC policy remains that no student can graduate without demonstrating competency in all four graduation standards. However, as assessments were implemented and have now been in place for several years, very few students fail to the pass the assessments in time to graduate. Students routinely require multiple attempts to pass (and benefit from a variety of academic supports in place to help them), but none of the colleges limited the number of times a student could attempt demonstrating competence. The de facto pass-rate, then, remains nearly 100%.

The perception of a high-stakes model may have brought about low standards (as did the original concept of “minimum” competencies). But the most consequential decision was to allow for the design of local assessments within a system-level model. This approach provided for substantial faculty ownership of the process but precluded any cross-college analysis or national benchmarking with similar institutions (although two colleges use a nationally normed online assessment of information literacy). Equally significant was the decision to measure competence at a single point in time rather than at multiple points in order to measure learning gains over time. While the notion of measuring the “value added” by a college degree is fraught with methodological problems related to isolating the effects of the institution (versus those resultant of maturation or experiences outside the institution), it has become the gold standard in outcomes assessment, particularly at a time when popular books such as Academically Adrift: Limited Learning on College Campuses (Arum and Roksa, 2011) have raised questions about the extent to which students learn anything at all in college. Further, measuring competence at a single point in time provides little insight into how students acquire skills and the extent to which particular curricular or pedagogical approaches impact learning gains.

To a large extent, the approach did not take advantage of the opportunity to aggregate and analyze system-level data to improve teaching and learning. Despite having a single administrative information system across the colleges, inadequate attention was paid to developing robust data-collection and analysis systems to support the graduation standards initiative. The strategy of early compromise was critical to ensuring faculty engagement in the assessment process, but it leaned too far in the direction of local autonomy. This manifests an inherent tension in higher education system leadership: supporting strong, unique colleges while maximizing the benefits of the system.

In other respects, the assessment approach did maximize the benefits of being a system. VSC policy remains that meeting the graduation requirements at one college also meets the graduation requirements at any other VSC college, despite the variation of assessment methodology. This benefits transfer students and encourages community college students to continue their studies in the VSC. Other benefits of the assessment model include systemwide awareness of national trends in assessment and accountability, faculty agreement on essential learning outcomes for all VSC graduates, and increased student awareness of performance expectations for college graduates.

Perhaps most valuable has been the annual systemwide retreat devoted to analyzing assessment methods and results in particular areas. In order for an assessment model to ultimately succeed as a means of improving learning outcomes, systemic processes must be in place at all levels to continually monitor, evaluate and strengthen the approach. The annual review process could potentially be enhanced through student involvement, reflecting the growing body of literature speaking to the potential benefits of engaging students in the study of teaching and learning. But by bringing together faculty from across colleges, systems have the opportunity to establish what the Carnegie Foundations calls “networked improvement communities,” which provide for highly structured, cross-functional, cross-institutional inquiry. Finally, the decision to focus on a limited set of outcomes, while for some creating the perception of diluting the greater purpose of a college education, provides the opportunity for in-depth analysis of how students learn a discrete set of skills commonly viewed as essential for success in and beyond college.

Carol Moore is the past president of Lyndon State College and currently works as a consultant. Karrin Wilks is the past senior vice president of the Vermont State Colleges and currently serves as university dean for undergraduate studies at the City University of New York.