UNIVERSITY PARK, Pa. — Among higher education institutions, a learning outcomes assessment (LOA) is an assessment process focusing on student learning at the culmination of an academic program. Penn State first implemented a University-wide LOA process during the 2016-17 academic year and, having forgone an LOA in 2020 due to the COVID pandemic, rounded out five years of the process in 2022.
Managed by the Office of Planning, Assessment, and Institutional Research (OPAIR), this critical component of high-quality educational processes helps to maintain the integrity of the academic programs offered to students. An assessment also can reveal various insights about an academic program, such as gaps or redundancies in the curriculum, curricular shifts, course sequencing issues, inappropriate performance expectations, misalignment between objectives and curriculum, and positive or negative student outcomes. The assessment cycle culminates on June 30 each year with the submission of program assessment reports.
At Penn State, LOAs focus on determining the extent to which students meet expectations in undergraduate, graduate, and for-credit certificate programs across the institution. This process also allows faculty to collaboratively ascertain the knowledge and skills that graduates should be able to demonstrate and then collect evidence of the extent to which students are meeting those learning expectations. Additionally, the Middle States Commission on Higher Education — the University’s accrediting institution — requires its member institutions to demonstrate continual student learning assessment at the program level as part of each reaffirmation of accreditation process.
In the beginning
The initial assessment process in 2016 began by asking programs to confirm existing or establish new program learning objectives (PLOs). The PLOs consist of short statements that describe the knowledge, skills, values and habits of mind that students should demonstrate upon graduation. Since summer 2017, the University has required programs to assess at least one PLO per year and to submit annual assessment reports documenting the prior year’s assessment findings, the resulting actions, and the following year’s assessment plan.
Historically, programs received templated paper forms to complete and submit to secure cloud storage. However, without a way to aggregate report data — short of manually entering every piece of a report into a spreadsheet — University-wide reporting was tedious and provided only a holistic view of the data.
In the following years, assessment questions were adjusted as the process evolved, and subsequent versions of the reporting form followed. However, additional forms caused confusion as various programs began using different versions, making it impossible to aggregate and evaluate responses properly.
OPAIR began searching for an assessment management system to replace its “paper” reporting model. Because OPAIR also heads strategic planning for the University, a single system was sought that would support strategic planning as well as assessment.
Moving forward
After a rigorous review of existing options, the search committee selected the Nuventive Improve assessment management platform. OPAIR and Nuventive staff spent a year customizing the program for Penn State, creating templates similar to previous assessment forms and using familiar language and the same required components from previous assessments.
Nuventive significantly decreased redundant data entry, eliminating the need to type in new information each year. It also provided a repository of assessment history and future plans that are easily transferred when program assessment leaders turn over. The platform allows various key metrics to be aggregated and monitored easily, further streamlining University efforts to evaluate the state of assessment institution wide. It also helps track individual reports’ progress, facilitating OPAIR’s ability to support programs where needed. A new, even more user-friendly Nuventive interface will be available later this year and will feature several user-requested refinements.
Today
A total of 701 programs were required to participate in the 2021-22 assessment process. Of those, 551 reports were completed, marking the highest proportion of submissions ever, at 79%, and helping to meet the goal of exceeding pre-pandemic assessment reporting in terms of submission rate and specific report quality measures.
The submission rate is improving year over year, and the quality of the assessments being conducted has improved dramatically over time. In 2021-22, 94% of the submitted assessment studies demonstrated strong alignment between the PLO and the method used to assess it, and 96% had an established performance target. Direct measures of student learning were the most common types of measures used (73% of all measures) and programs set action plans for 79% of all findings.
To read the recently published report, an executive summary infographic and the full report can be viewed on the OPAIR website.