UNIVERSITY PARK, Pa. — Two tools developed by Penn State IT’s Data Empowered Learning team are helping academic advisers and instructors proactively support students using data that was once scattered across multiple systems.
Elevate, designed for academic advisers, and Course Insights, for instructors, provide actionable information to those at Penn State who interact with students the most. Using these tools, academic advisers and instructors can easily see when a student on their roster isn’t engaging with coursework, allowing interventions to happen before a student gets too far behind.
The creation of Elevate and Course Insights from the Data Empowered Learning team began at the height of the COVID-19 pandemic, as staff and faculty raced to move the entire University to remote learning in the spring and summer of 2020.
“That’s where a lot of this originated, at the start of the pandemic,” said David Smith, executive director of the Division of Undergraduate Studies (DUS) and associate dean for advising. “We needed to be proactive in reaching out to students who might not have been able to make the transition to remote learning. And we’ve grown it from there significantly.”
DUS and the Data Empowered Learning team began working together to create a tool that would give academic advisers the information they needed and present it in a way that it was easy to see where intervention could be helpful. Later, the Data Empowered Learning team turned their attention to Course Insights for instructors, built right into Canvas. A central feature of both tools allows academic advisers and instructors to view the aggregated Canvas activity (but not specific actions) charted against a seven-day average for the entire class. This provides some insight into whether a student is engaging in the course at the same times and rates as their peers.
In Elevate, academic advisers can see their entire Starfish roster, with alerts for low or zero activity. They can click on an individual student and see a list of courses and in which course(s) the student may have activity alerts.
One of the other core features for Course Insights is the aggregation of demographics, class standing, cumulative GPA, and other data points that summarize the population of students enrolled in a course. Ben Hellar, manager of Data Empowered Learning, said his team focuses on providing data to the instructors and academic advisers who work daily with students to support their academic success.
“When we first started designing Course Insights, we talked to instructors about their data needs, looking to understand what questions they ask themselves about the courses they teach. What we discovered was that instructors had a need for basic demographic information to answer questions like, ‘What are the majors of students in my course?’ or ‘How many students in my course took the pre-requisite?’ Hellar said. “Many faculty send out manual surveys every semester, because there was no one singular tool or one easy way of automating this data collection.”
Hellar said his team was able to create answers for these questions and deliver the insights directly into Canvas where faculty administer their courses. In his own teaching as an adjunct for the College of IST, Hellar said he uses information from Course Insights to adjust his pedagogical approach. For example, if he has a lot of first-semester students, he knows that he’ll have to carefully explain things like accessing and using Canvas or acquiring textbooks. Another example he shared was from a World Campus course, where he saw in Course Insights that he had many adult learners and adjusted his student hours to make himself available at night when many were logging on to do their coursework.
Data for the front lines
The challenge for many academic advisers and instructors is the number of students on their rosters. Elevate and Course Insights help to visualize how students are doing through intuitive analytics that identify which students might need proactive outreach.
“How do we use data to drive action?” Smith asked. “And who are we expecting to take action? It’s the people on the front lines, the advisers, the instructors. We often think of data as critical to high-level decision making. And it is. But similarly, it’s really important to the people who are effecting change on a one-on-one basis with students.”
These tools, combined with efforts like Starfish progress surveys, have helped staff and faculty at Penn State support students while courses are still happening instead of reactively after a student has already failed or dropped a course. With insights from the course activity graphs, interventions can happen early in the semester before grades are readily available. Data collected by DUS showed that interventions are most helpful during the first five weeks of a semester when students have more time to get back on track.
Hellar and Smith stressed that low activity and engagement on Canvas does not necessarily mean the student is struggling. Some students are more prepared for specific courses or can easily grasp the material. Students with high Canvas activity may not actually be successful in the course, either.
“The insights received from Elevate are a signal to take action," Smith said. "Academic advisers need to bring their own knowledge and awareness to the situation so that they can interpret the data and provide appropriate outreach to their students. Elevate does not exist in a vacuum but stands as an additional insight to help our advisers provide differentiated care to their students."
Brian Petrosky, DUS adviser, said he looks at his roster in Elevate periodically to get a sense of who might be falling drastically behind. One of the features he likes the most, he said, was the notification if a student late-drops a course.
“One of the reasons that’s valuable is that it might be a critical entrance-to-major class that can’t be repeated,” Petrosky said. “In some circumstances, if a student acts quickly, the instructor will let the student back in if it’s a major error. Plus, if there’s multiple late drops [by a student], you should be worried.”
Sometimes an Elevate alert can uncover situations when a student is facing a major obstacle in their life outside the classroom.
“The other thing that’s nice,” Petrosky said, “is I can use Elevate to see which students on my roster have kind of 'zeroed out' in terms of meaningful class participation. In fall of 2020, I had a student that zeroed out. And, sure enough, he had COVID, and he was isolated and wasn’t doing anything.”
Petrosky said that previously he would have had to wait for a student who had “zeroed out” to reach out for help. Waiting until the end of the semester when a student has failed or dropped courses means there’s no chance of intervention.
Even if a student doesn't directly respond to progress survey concerns raised by an instructor or outreach by an academic adviser, Smith said sometimes they see an uptick in Canvas activity for that student, which indicates they have at least heeded the outreach and are working to improve.
While Elevate and Course Insights were designed with the same goal in mind — proactive outreach and intervention to students who may be struggling in their courses — academic advisers and instructors each have access to different information to protect student privacy and comply with national FERPA laws. Academic advisers, for example, can’t see individual grades posted to Canvas. Only the instructor of record has access to grades. Conversely, an instructor can’t see if a student is struggling in courses other than the one for which they are an instructor of record.
Data Empowered Learning Team earns award
The Data Empowered Learning team earned the platinum-level Learning Impact Award at the June 2023 1EdTech conference. 1EdTech is composed of 900 member institutions dedicated to accelerating the digital transformation of learning.
“We are honored that 1EdTech has recognized Course Insights as an impactful and innovative tool to support instruction and student success,” Hellar said in August. “The awards help validate that our approach of delivering learning analytics data to instructors has value not only to the institution but for the industry at large.”
Instructors interested in joining the ongoing Course Insights pilot should contact the Data Empowered Learning team at delteam@psu.edu for more information. Over 240 faculty from numerous departments, colleges, and Commonwealth Campuses are currently participating.
Starfish mid-semester progress survey open until March 20
Starfish mid-semester progress surveys are open now until Wednesday, March 20. During this period, all instructors and teaching assistants formally listed as instructors of record for full-semester undergraduate courses in LionPATH will receive an email prompt to share information about students’ course progress.
Progress surveys can be completed by clicking the emailed link or opening outstanding surveys within Starfish. Instructors and teaching assistants then check boxes corresponding to to-dos, flags and kudos for each student and, if they wish, provide written comments. A video on how to complete a progress survey is available on Kaltura. Academic advisers are encouraged to discuss concerns with students and to close the loop with instructors by letting them know the result of their efforts.