General Framework
The annual General Education Assessment (GEA) Summit brings together a group of faculty for two days to assess assignments that are a regular part of general education course content. Each summit focuses on one of the learning goals of the general education program. Courses that address the intended learning goal are randomly selected to submit assignments. Instructors of these randomly selected courses are asked to select an assignment and submit work for that assignment from all students enrolled in that section. The intention is to have samples of student work from a representative sample of courses from a variety of departments. Typically, an equal number of lower-level (1000-2000) and upper-level (3000-4000) courses are selected for review.
The use of modified versions of the Association of American Colleges and Universities (AAC&U) VALUE (Valid Assessment of Learning in Undergraduate Education) rubrics (Rhodes, 2009) provide a common set of outcomes around the learning goals to which courses can align. The VALUE rubrics have been vetted nationally and have been used by hundreds of institutions over more than a decade for assessment of general education outcomes. AACU&U encourages campuses to modify the VALUE rubrics to best meet their needs. The rubrics developed for Appalachian may be a combination of criteria from several different VALUE rubrics that best fit the rationale created by Appalachian faculty for each of the learning goals.
Timeline
The General Education Assessment Summit takes place in May, usually in the break between the end of the spring semester and the beginning of the first summer session. Assignments are collected prior to the end of the spring semester. In addition, mini-summits may be held during winter break to test out and refine rubrics as needed.
May 2018 – Pilot Thinking Critically and Creatively Rubric and Summit Process
May 2019 – Assess Thinking Critically and Creatively Goal
Dec. 2019 – Pilot Local to Global Connections Rubric
May 2020 – Assess Local to Global Connections Goal
Dec. 2020 – Pilot Responsibilities of Community Membership Goal
May 2021 – Assess Responsibilities of Community Membership Goal
May 2022 – Assess Communicating Effectively Goal
May 2023 - Assess Thinking Critically and Creatively Goal
2023-2024 - Revise and test rubrics prior to the launch of General Education 3.0
Assignment Collection
The Director of General Education contacts instructors of courses that have been randomly selected to submit assignments at the beginning of the semester. This process is intended to require minimal effort on the part of faculty teaching the course. Instructors of the selected courses are presented with a copy of the rubric that will be used to assess assignments and asked to review their course assignments to identify one that they believe best fits the learning goal being assessed. The assignment should be a regular part of the coursework, which allows for more authentic assessment and eliminates concerns about student motivation to perform their best work.
After the identified assignment is completed by students in the course, instructors are asked to give a contact in Institutional Research, Assessment and Planning (IRAP) access to the assignment. The instructor can submit student work to IRAP in whatever format is easiest for them – hard copies that can be photocopied and returned, giving permissions to access assignments in AsULearn or APortfolio, sharing electronic copies in a Google Drive, etc. The instructor is also asked to share the instructions that students were given for the assignment.
IRAP removes any identifying information about the assignment, including student names, instructor names, and courses. All work is coded so that results can later be disaggregated by class level and other broad characteristics as requested. This assessment process is meant to provide information about the overall effectiveness of the general education program and is NOT used to report on the performance of individual students, faculty or course sections.
Review of Assignments
Ten faculty from a variety of departments that provide general education courses are recruited to participate in the summit. These faculty are paid a stipend for participating in the two-day event. There is also an additional faculty member, usually from the General Education Council, who serves as a third reader as needed.
The first morning of the summit is spent calibrating scores. As a group, participants carefully review and discuss the rubric. Then, all reviewers are given the same sample artifact to read and rate on each student learning outcome. Each rater then shares their scores and the reason they gave that score. The group discusses variability in scores. Then, the raters read, rate, and discuss another sample artifact. This process continues with multiple sample artifacts as time allows in an attempt to come to some agreement about what constitutes each level of performance.
The first afternoon and second morning of the summit involve faculty working independently to read and score assignments. Following a lunch break on the second day, the group reconvenes to discuss what they have learned and review initial results.
Rationale
As stated in its mission, “Appalachian’s General Education Program offers a rigorous liberal education that includes 44 semester hours of courses from across the university, plus two courses in the major program that address the General Education goals of critical and creative thinking; effective communication; making local-to-global connections; and understanding responsibilities of community membership.” The general education experience is a large part of a student’s education at Appalachian and the University wants to ensure that students are provided with a quality learning experience.
As with all other academic programs (majors) offered at Appalachian, assessment of General Education involves:
- Articulating what students should gain from the program
- Ensuring that the curriculum aligns with those expectations
- Providing evidence that students in the program are actually graduating with the skills and knowledge that the program claims
- Making improvements to the curriculum as needed to get students where faculty want them to be
- Documenting the process for accountability and future reference
The purpose of the General Education Assessment (GEA) Summit is to provide evidence that the general education program is teaching to the learning goals of the program, as well as identify strengths and weaknesses of specific dimensions of that goal. The faculty participating in the summit also provide feedback on the assessment process itself so that future improvements can be made.
The primary purpose of assessment of the general education program is to provide meaningful information to those responsible for the oversight (General Education Council) and delivery (faculty teaching general education courses) of the program. Assessment results are intended to be used to improve the general education program.
In addition, the University also needs to comply with accreditation standards of the Southern Association of Colleges and Schools Commission on Colleges (SACSCOC). As outlined in the Principles of Accreditation, Section 8.2: “The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of seeking improvement based on analysis of the results in the areas below . . . b. student learning outcomes for collegiate-level general education competencies of its undergraduate degree programs. (Student outcomes: general education).”
The most recent approach to general education assessment was assessment at the course-level as part of the course renewal process. The course-level assessment approach proved to be too cumbersome for faculty and for the general education program while not rendering information that would help improve the general education program as a whole. Many faculty struggled to write functional learning outcomes for the general education goals for their courses and identify appropriate ways to measure them. Incorporating general education assessment into the course renewal process proved to be a lot of work for little gain. With that in mind, several concerned parties at Appalachian came together to propose a new way to address general education assessment.
Involvement
The General Education Assessment Summit is a team effort by the General Education Council, General Education Program, Center for Excellend in Teaching and Learning for Student Success (CETLSS), and Institutional Research, Assessment and Planning (IRAP). IRAP takes the lead in planning the logistics of the summit each year.
Reporting
At the end of the GEA Summit, faculty participants provide feedback to be shared with general education stakeholders. The questions they help answer include:
- What are the areas of strengths related to the learning goal?
- What areas need improvement related to the learning goal?
- To what extent were all of the outcomes addressed in the artifacts?
- What recommendations would you make to the General Education Council regarding improving this learning goal?
- What recommendations would you make to your colleagues regarding improving this learning goal?
- What are your recommendations for improving the rubric and the assessment process?
IRAP consolidates the feedback and compiles statistics for a report. During the first meetings of the Fall semester for the General Education Council and the University Academic Assessment Council, members of IRAP and faculty participants in the GEA Summit present an overview of their findings. The report is also shared with colleagues in CETLSS to help identify professional development needs.
References
Rhodes, T. (2009). Assessing outcomes and improving achievement: Tips and tools for using the rubrics. Washington, DC: Association of American Colleges and Universities.
Southern Association of Colleges and Schools Commission on Colleges. (2018). Resource manual for the principles of accreditation: Foundations for quality enhancement (3rd ed.). Decatur, GA: Author.