LeBow Program Assessment is the 2018 Winner of the Assessment and Pedagogy Award
Every year, the Office of Assessment, Accreditation and Effectiveness solicits nominations for an award for significant achievement in learning assessment and teaching pedagogy. This award is presented as part of the Faculty Recognition Awards and it is overseen by Stephen DiPietro, Vice Provost of Assessment Accreditation and Effectiveness. The award recognizes individuals and teams that have utilized assessment to improve a teaching and learning initiatives and, as a result, have significantly impacted curriculum design and the overall quality of teaching and learning at Drexel. “It is a way for our office and by extension the Provost to recognize quality work in this area,” DiPietro stated. Every year presents a challenge in selecting a winner as there are many excellent entries. 2018 would be no different. Out of the variety of quality proposals, the winner was “Integrating AOL into your curriculum improvements: The case from career learning at LeBow College of Business” submitted by Lisa DeLuca, Director, Deborah and Dominic Caruso Undergraduate Career Services Office and Teresa Harrison, Associate Dean of Academic Affairs and Programs.
The process for selecting a winner was changed this year to include the aid of the Faculty Assessment Fellows. Prior to this year, all of the submissions were judged by a group of staff/faculty led by DiPietro. Since the advent of the Assessment Fellows however, it seems best to utilize the combined talents of the Fellows to review an award for learning assessment and pedagogy. Diane DePew, Assistant Clinical Professor and Faculty Assessment Fellow, provided an insight into the process. DePew stated, “Well, we took a divide and conquer approach. We assigned two Fellows to read each submission and then present to the group if they thought that the submission should be a finalist for consideration.” Based upon the strengths and weaknesses, they chose the DeLuca/Harrison submission. “The focus was closing the loop. They didn’t just collect data for data’s sake. They collected it for a specific purpose and then then evaluated that data to determine actions.” DePew stated. “This led to interventions which led to more data which led to more evaluations and actions.”
Drexel’s LeBow College of Business is accredited by the Association to Advance Collegiate Schools of Business (AACSB). There are four learning outcomes for undergraduate business students across all four degree programs and career learning is one outcome assessed in all four degree programs. In 2007 LeBow determined that career learning would be assessed in UNIV 101 Part 2. Initially, all LeBow undergraduates took UNIV 101, The Drexel Experience (a course that is designed to help new students acclimate to Drexel) during their first two terms as freshman. In 2008, LeBow moved one term to senior year and reframed the course to be UNIV 101 Part 2, Career Management. Lisa DeLuca, Director of Undergraduate Career Services, was one of the co-founders of this course and continues to manage all aspects to date. In fall of 2014, the course was renamed UNIV B201, Career Management.
The idea is to utilize the student’s co-op experience and resume to translate into lifelong career skills and a “forever” resume. Much time is spent on interviewing skills, and other advanced career techniques and that is where the focus of this study is; specifically interviewing skills. “The original assignment was to develop a 10 min presentation on who they are and what they wanted to do. When we spoke to employers, we heard that the students needed more work on video interviewing, so we took one long video presentation and replaced it with 3 video interviews that would be peer reviewed.” DeLuca noted.
Peer review was the key as most students need to hear that I am doing great things, or I see that they are well behind others. Hearing and seeing that from other students is much more impactful then any instructor. That being said a group of 5-6 faculty and staff also provide feedback. The students upload their videos, get feedback and then apply the information in the next interview iteration. “Now getting to tell the story is changing based on the data and rubric based grading, so it is easier for them to make changes and for us to look at progress and achievement.” DeLuca added. LeBow College uses the Apprendet system to watch and use rubric for grading as well.
This process has led to other ideas and changes as well. One other change that we made while hearing employer feedback was a switch from portfolios that the students were developing, to LinkedIn. Employers were seldom looking at the portfolios at all. DeLuca and her team do not feel as if they are finished with improving the course. “The big change for this year that we are launching in the summer term (late June). We would like to look at the data on short term/long term goals. The students will write it ahead of time and then record the video. That worked so well with the other question, so we want to try it with this new question as well,” DeLuca added.
“We believe this career learning assessment program demonstrates a nice model of how a strong assessment program provides a key mechanism to improve the curriculum. The documentation of the learning objectives, used simultaneously with the student and instructor feedback, provided clear guide posts for the best ways to increase student learning within the course, in this case UNIV 201,” DeLuca said. When asked why she submitted, DeLuca stated, “Honestly, Teresa told me to. For years now, she has been telling me how well I am using assessment to which I say, “What do you mean? I am just looking at feedback and making changes and using the data.”
DeLuca and Harrison were recognized as part of the Faculty Recognition Awards. The ceremony/dinner highlights significant achievements/innovations within research/scholarly activity, community service and teaching. The night is presided over by the Nina Henderson Provost of Drexel University, M. Brian Blake. It was a wonderful evening of celebration amongst friends, family and colleagues. “At the awards dinner, being a staff person was a little different. Our goal was to always have the best experience for the students. I wondered if I belonged there, but it was interesting to see the other people who were doing great work,” DeLuca noted.
New for this year, the winners, along with first runners up, presented their projects at the 5th Annual Drexel Assessment Conference in September describing their successes, problems and constraints when implementing assessment approaches. DiPietro stated, “The panel showed how various strategies can be implemented at the same institution and that assessment is not a ‘one size fits all’ proposition.” The format of the session was to provide time for each of the three groups to present their process and findings with time for questions as well. The audience was composed of Drexel colleagues, but there were also representatives from other college/universities which was a nice way to showcase the talent within Drexel University. DePew: “The session went really well and the faculty that presented kept it brief and explained thoroughly. They all did a fine job of presenting and showed how hard it was to choose between the projects.”
The session was very well received and it will be repeated at the 2019 conference. DePew stated, “The session was facilitated by conscientious, collaborative people who were excited to share with others and learn from the other finalists.” DeLuca stated, “It was a great experience to be recognized and to present along with my colleagues.” Dr. Stream added, “It was an honor to present at the Assessment conference and share a component of our program’s assessment process with the attendees. We were able to share some of our experiences with a visitor who had questions about their program. It was valuable to hear the other departments that presented their assessment processes activities and outcomes.”
The two other finalists come from two other colleges within Drexel University. The first was from the Physician Assistant program (College of Nursing and Health Professions) and specifically, Patrick Auth, PhD, PA-C, Diana Smith, MHS, PA-C and Charles Stream, MPH, PA-C. This ongoing project involved critically analyzing the correlation between the program’s Didactic Comprehensive Examination, Summative Exam, the Physician Assistant Education Association’s Physician Assistant Clinical Knowledge Rating and Assessment Tool and the Physician Assistant National Certification Exam (PANCE) to develop the ability of students to self-assess their medical knowledge as well as to evaluate the program’s curriculum. Dr. Charles Stream stated, “As the students’ progress through both the didactic and clinical phases of their education, the comprehensive exam outcomes are shared with the individual student to identify their strengths and weaknesses to develop a study plan and identify learning resources to assist in strengthening their knowledge base in preparation for the next exam including the PANCE. The faculty annually review the Body System and Task components of the PANCE to identify any trending changes that may identify any curricular weaknesses and or strengths.”
The other finalist was represented by Dr. Jason Baxter who is the department head for Chemical Engineering (College of Engineering). Dr Baxter and his team laid out the process by which they used data, both quantitative and qualitative, to review their curriculum and make substantial changes. Using a variety of stakeholders including, students (current and alumni), faculty, staff and industry employers the curriculum was revamped to make it more current and applicable to the current market. Extensive work was done within the sequence and timing of skills as a map was developed to ensure that each essential skill was addressed multiple times in order to achieve mastery. The work that Chemical Engineering undertook is used as a model process for curriculum revision for the Program Alignment and Review process.
In the spirit of continuous improvement, The Faculty Assessment Fellows have suggested some changes for the process for 2019. In order to streamline the evaluation with the amount of diverse submissions, a more streamlined template has been proposed. DePew stated, “With so many wonderful submissions from so many different areas within the University including staff and faculty, this will make it easier to narrow down the field and select a winner.” For the presentation within the conference, the abstract will be augmented to maximize the audience from both inside and outside of Drexel University in order to augment the recognition that this work truly deserves.