(Page 49 of 50) - Balancing the Power: An Egalitarian Model for Course Evaluation and Assessment authored by Sloat, James.

Student Feedback: On the pilot course evaluation form, the students had the opportunity to
identify questions that were unclear to them. Relatively few students reported unclear questions.
The highest number of "unclear" responses (10-12 students) came on the department-specific and
section-specific questions. Of the common questions, the highest number of "unclear" responses
came on question 7 (Compared to similar level courses at W&J, the workload in this course
was: )(8 responses) and question 10 (I would recommend this course to other students who are
looking for a good course.) (7 responses). All of the rest of the questions had 4 or fewer students
noting them as "unclear." These low numbers provide reason to believe that students were able
to understand the meaning of the questions.

Students also had the opportunity to offer comments/suggestions about either the form or the
process. Students offered 151 comments/suggestions. 51 of the comments were either "Good
form" or "No improvements to suggest." 24 comments were that the form was too long or had
repetitive questions. 18 comments noted the absence of department- or section-specific
questions. 6 responses suggested that the form was too short or needed more questions. 4
responses noted that there were unclear questions. 3 responses suggested having a comment
space after each question. 3 responses expressed concern about being identified based on the
information from the Student section. The rest of the comments/suggestions came in as either
single or double responses.

As a supplement to the student written comments, I conducted a focus group with my students
about the instrument itself. The purpose of the focus group was to have the students state what
they understood each question to be asking--in order to make sure that they were answering the
question that we were asking. Students in the focus group identified ambiguity in questions 6, 7,
and 8. They also raised the question of "What is a 'good' course/instructor?," and they suggested
that we follow up on those questions (10, 22) with a "Why, or why not?"

Next Steps: The Course Evaluation Team will begin its work in the fall by reviewing the full
data from the pilot project and then working with Department Chairs, the Faculty Review
Committee, and the faculty at large to revise the course evaluation instrument in view of the pilot
project. We will also be doing some more sophisticated statistical analyses of the data in order to
identify any hidden pitfalls. Our target is for campus-wide implementation in the fall.

Over the summer, the Dean authorized the purchase of a new Scantron machine to assist in the
course evaluation process. The machine and software have arrived, and they will be ready for
use in the fall. This new equipment will make it possible for us to move to campus-wide
implementation without requiring manual data entry. Thanks to technological advances, we are
going to be able to capture both the bubble responses and the comments electronically. The
software will allow us to produce and distribute forms and reports electronically, so there will be
no need to photocopy forms. We will be able to reap most of the benefits of an online system
without incurring the additional logistical costs.

Once again, thank you for participating in this pilot project. Please let me know if you have any
additional comments, questions, or suggestions.

Jim