Teaching and Learning > DISCOURSE

Comparing Student Learning and Attitudes

Author: Renee Smith and Linda Palm

Journal Title: Discourse

ISSN: 2040-3674

ISSN-L: 1741-4164

Volume: 6

Number: 2

Start page: 205

End page: 225

Return to vol. 6 no. 2 index page

Outline

This study compared students in traditional and distance sections of PHIL 101: Introduction to Philosophy in terms of both objective learning outcomes and student attitudes in the form of students' anticipated and perceived learning and course satisfaction. It was hypothesized that while students in the two types of classes would not differ in their academic performance or in their anticipated learning and course satisfaction, students in the traditional classes would perceive that they had learned more and report being more satisfied with their learning experience than students in the distance classes. The results of the study indicated that the traditional and distance students did not differ significantly in their in performance on graded assignments, their expectations for learning at the start of the course, their perception of what they had learned at the end of the course, and their willingness to take another philosophy course. Distance students anticipated enjoying specific course activities more than traditional students, but at the end of the semester, reported enjoyment of these activities did not differ. Students in the traditional classes, however, rated the lectures as more interesting and rated the course, overall, as more enjoyable than students in the distance classes. The implications of these findings for philosophy instructors considering the use of the distance format are discussed.

Introduction

Distance courses provide a way for colleges and universities to increase enrollments without necessarily having to expand infrastructure or personnel (Howell, Williams, and Lindsay, 2003). There is a demand for such courses especially from non-traditional students whose work and family responsibilities prevent them from attending traditional courses and from students who do not live near a university (Dutton, Dutton, and Perry, 2000; Howell, Williams, and Lindsay, 2003; Jones, 2003, Minton and Willett, 2003). We have no doubt that in many cases distance courses fill a need and are an effective means of teaching and learning. If anything, distance courses provide access to education to those who might not otherwise have it so long as they can cross the socalled 'digital divide' (Latanich, Nonis, and Hudson, 2001; Minton and Willett, 2003) and could also be used to supplement more traditional university courses (Jones, 2003). Moreover, today's distance courses are not the simple correspondence courses of the past. Online courses can be packaged in multidimensional platforms such as Blackboard and WebCT and present material in a multitude of mediums.both synDiscourse: chronous and asynchronous.including PowerPoint presentations with attached audio overlays, online video lectures, learning modules, discussion boards, chat rooms, and white boards. It seems that as the technology has improved over the last decade, many of the objections faculty have had to online teaching have lost their force. A number of studies have shown, and it is the accepted view among 'teaching with technology' personnel, that there is 'no significant difference' in learning outcomes between students in traditional, 'on-ground' courses and those in distance courses (Russell, 1999; Shinaberger, 2006).

Criticism of distance education seems to center around a different divide.that between administration and faculty, specifically the divide between their goals (Feenberg, 1999). Administrations have economic goals for the university that sometimes seem to conflict with the teaching and learning goals of the faculty. The administration may have an idealized vision of increased enrollments at minimal cost to the university, but there have been lingering questions from the faculty about teaching distance courses: Should teaching be automated at all? Can real learning take place in a distance classroom? Is some essential component of learning lost in the distance format?

It would seem that the pros outweigh the cons when it comes to distance education, at least if one accepts the 'no significant difference' research, and new technologies continue to fill much of the gap that seemed so worrisome to faculty. In the field of philosophy, however, there seems to be a unique set of arguments opposed to teaching philosophy in a distance format.1 Despite any layperson's stereotype of the philosopher sage dispensing wisdom from a mountaintop or living as a reclusive bookworm, an essential component to doing philosophy is engaging in philosophical discourse. Nowhere does this seem more important than when one is developing one's philosophical skills. Socrates had his interlocutors, and one could argue that our students need their classmates and their professors to learn the basics of philosophy (for example, see Sadler, 2004; Lugenbehl, 2003; and Nehmias 2005). Only in real-time, face-to-face interchanges can one learn to raise questions, propose answers to those questions, give, object to and modify arguments, draw distinctions, and propose analyses. One could attempt this via email, on a discussion board or even in a chat room, but we think one would be hard-pressed to find a philosophy teacher who thinks that such discourse comes close to what can be achieved in the classroom.2

This conflict between the economic and social goals of the university and the methodology of doing and teaching philosophy was the impetus behind the present study. While one of the authors had a vested personal interest in teaching a distance course3, there seemed to be a legitimate question raised by these conflicting views of distance education: are there significant differences between a traditional 'on ground' philosophy course and a wholly distance course?

A number of studies have focused on distance courses and comparing distance and traditional courses and students. The aforementioned 'no significant difference' research tends to show that, objectively, distance students learn just as well as traditional students. However, the overwhelming trend right now seems to be to measure students' perceptions and attitudes concerning the course delivery methods themselves, namely, the course qua distance course. Several such studies measure students' perceptions of their 'learning environment' (Teh, 1999), their perceptions of the benefits of online learning (O'Malley and McCraw, 1999), their expectations concerning content delivery via online sources (Jurczyk, Kushner Benson, and Savery, 2004), their attitudes toward 'distinct facets of network-based instruction' (Federico, 2000), their anxiety about learning in a distance course (Katz and Yablon, 2002), and their satisfaction with things like 'course management,' 'support services,' and 'out-of-class communication with the instructor' (Biner, Dean, and Mellinger, 1994). There are also studies that examine students' perceptions of how they learn best (for example, Perez-Prado and Thirunarayanan, 2002). Moreover, while some researchers claim to have measured students' learning experiences, their research seems to report students' attitudes toward how they learned and how they used the technology rather than their perceptions that they learned (Perez-Prado and Thirunarayanan, 2002).

In the present study, we aimed to measure both actual and anticipated and perceived learning outcomes of students enrolled in traditional and distance sections of Introduction to Philosophy. Many philosophers are skeptical about teaching philosophy in a distance format since they believe some essential aspect of doing philosophy, something not necessarily reflected in students' performance on exams, is lost in that medium. This je ne sais quoi is a very difficult thing to measure. It occurred to us that what one thinks of one's learning experience insofar as that one has in fact learned something is just as important as how one actually performs in the class. We did not focus on students' experience of using a particular technology, but instead on their experience of learning and doing philosophy. We focused on students' expectations for the course at the beginning of the semester and their perceptions of having satisfied these expectations at the end of the semester. There is some evidence that children who perceive themselves to be good decision makers actually make better decisions (Grier and Firestone, 1998). It is plausible that students of philosophy who believe they learned something about what philosophy is and how it is done and believe they have improved their critical thinking skills will go on to use these acquired skills more effectively. Moreover, beyond the intrinsic value that might come from believing that one has been successful, this sense of accomplishment coupled with students enjoying the content and delivery of the course could lead to a second course in philosophy.an appealing prospect in a department like ours looking to increase enrollments. Thus the goals of this study were to compare (1) academic performance, (2) anticipated learning outcomes and course satisfaction, and (3) perceived learning outcomes and course satisfaction of students in traditional and distance formats of the course. Three hypotheses were evaluated:

  1. 1. Students in the traditional and distance course formats will not differ in their performance on graded assignments.
  2. 2. Students in the traditional and distance course formats will not differ in their anticipated learning outcomes and course satisfaction.
  3. 3. Students in the traditional course format course would perceive that they had learned better and be more satisfied with their learning experience than students in the distance format.

Very simply, we expected that the students who had the experience of coming to class, being involved in and observing classroom discussions, and hearing the lectures 'live' would feel that they got more out of the class than those students who took the class online. This would show that, while there may be certain benefits of distance education generally speaking, there are also certain drawbacks that may be discipline-specific.

Method

Participants

The participants in this study were 141 students who enrolled in Introduction to Philosophy during the fall semester of 2005 or the spring semester of 2006. The sample consisted of 56 men and 85 women. The mean age of the students was 20.77 (SD = 4.55). During the fall semester, 59 students enrolled in traditional sections and 27 students enrolled in the distance section of the course. During the spring semester, 27 students enrolled in the traditional section and 28 students enrolled in the distance section of the course.

Table 1 provides a comparison of demographic information, including age, class rank, gender, and race, that was collected via the university's Office of Institutional Research for the 86 students enrolled in the traditional format and the 55 students enrolled in the distance format of the course. Statistical analyses using the chi square (ƒÔ2) test for independence indicated no significant difference between the traditional and distance classes in terms of gender, class rank, and racial composition. However, a significant relationship was found between course format and age, with the traditional classes having a higher percentage of students under the age of 24 than the distance classes. The average age of the students in the traditional classes was 19.63 (SD = 1.65) while the average age in the distance classes was 22.55 (SD = 6.64).

TABLE 1 GOES HERE

Procedure

PHIL 101 Introduction to Philosophy satisfies a core (general education) requirement at Coastal Carolina University. The philosophy department offers 10-12 sections of the course during each of the 16-week fall and spring semesters, and enrollment in each section is usually capped at 27 students. Data for this study were gathered over two semesters. In the fall of 2005, Smith taught two sections of PHIL 101 in the traditional format (one of these was an accelerated 8-week course) and one section in the distance format. In the spring of 2006, she taught one section in the traditional format and one section in the distance format. The instructor teaches a traditional 'problem-based' course focusing on five areas in philosophy: logic, philosophy of religion, epistemology, philosophy of mind, and ethics.

Smith designed the traditional and distance versions of the course to be as similar as possible. Both formats used WebCT, a web-based course platform, and all students were required to have an internet connection to access course materials. The instructor demonstrated how to use WebCT on the first class meeting. Students in both formats used the same anthology4 and were provided with the instructor's lecture notes. Each of the thirty lectures, one for each class meeting, was available as a PowerPoint presentation with audio on WebCT. PowerPoint presentations were not used during class in the traditional sections, but the PowerPoint lectures were available online after the topic had been covered in class. Students also had WebCT access to ungraded 'selftests' corresponding to each day's lecture material. Smith held four 'onground' office hours and four 'virtual' office hours in a WebCT chat room each week. While students in both course formats corresponded with the instructor by email, students rarely attended office hours.

Students in the traditional sections of the course were required to attend class, and attendance was taken for approximately 50% of the class meetings. Five of the class meetings involved planned group discussion or group activities, five of the class meetings (one at the start of each of the five topics) were directed 'brainstorming' activities about the philosophical problem at hand, and one class meeting was devoted to watching a video.5 The other class meetings involved an 'interactive lecture' in which there was a planned lecture that included student participation. Students earned participation points based on attendance. Students in the distance sections of the course were required to meet in person only on the first day of class and at the end of the semester to watch the video6. The first class meeting was used to demonstrate WebCT and to go over the course materials and requirements. Whereas the traditional students earned participation points for class attendance, the distance students earned participation points for posting to the discussion board in WebCT. Five postings were required, one for each course topic.

Students in both course formats were given the same tests and written assignments. At the completion of each of the five course topics, an objective 25-question multiple-choice test was administered. A cumulative final exam containing 50 multiple-choice, true/false, and matching questions was given at the end of the semester. Testing for students in both the traditional and distance formats was conducted via WebCT. Additionally, two short (2-3 page) papers were assigned. Students in the traditional sections submitted these written assignments in class, while the distance students were permitted to email their papers to the instructor.

Students in both course formats were asked to complete an anonymous Starting Survey and Ending Survey during the first and last week of the semester, respectively. The surveys were administered on WebCT and each contained 53 questions. Responses to the following 16 questions on the Starting Survey and 17 questions on the Ending Survey were examined in this study. Responses to Questions 38-53 on the surveys were given on a 4-point Likert scale with 1 = strongly agree, 2 = agree, 3 = disagree, 4 = strongly disagree. Students were given a text box to give an answer for Question 6 on the Ending Survey.

Starting Survey Questions:

38. My only goal is to pass the class.
39. I anticipate satisfying the learning goals/objectives in this course.
40. I intend to get a high grade in this class.
41. I do not care what grade I earn as long as I learn something about philosophy.
42. I hope to earn a good grade and learn something about philosophy.
43. I hope to engage in philosophical thinking myself in this course.
44. I hope to engage in philosophical discussions in this course.
45. I expect that overall this class will be enjoyable.
46. I expect that the readings in this class will be interesting.
47. I expect that the lectures in this class will be interesting.
48. I expect that the videos in this class will be interesting.
49. I expect that the discussions (in class or on WebCT) will be interesting.
50. I will feel a sense of accomplishment upon completing this course.
51. My critical thinking skills will be improved upon completing this course.
52. I will have a better understanding of what philosophy is upon completing this course.
53. I will have a better understanding of how philosophy is done upon completing this course.

Ending Survey Questions:

6. Will you take another philosophy course in the future?
38. I am satisfied with the grade I earned in this class.
39. I satisfied the learning goals/objectives in this course.
40. The grade I earned in this course reflects the effort I put into the course.
41. Despite the grade I earned, I learned something about philosophy.
42. I am satisfied with the grade I earned and I learned something about philosophy.
43. I engaged in philosophical thinking myself in this course.
44. I engaged in philosophical discussions in this course.
45. Overall, this class was enjoyable
46. The readings in this class were interesting.
47. The lectures in this class were interesting.
48. The videos in this class were interesting.
49. The discussions (in class or on WebCT) were interesting
50. I feel a sense of accomplishment having completed this course.
51. My critical thinking skills have improved having completed this course.
52. I have a better understanding of what philosophy is having completed this course.
53. I have a better understanding of how philosophy is done having completed this course.

Results

Academic Performance

Students' scores were recorded for the following eight assignments: Tests 1-6, Paper 1, and Paper 2. Scores on each assignment were recorded as a percentage based on a 100-point scale. Table 2 summarizes students' performance on these assignments and reflects scores of students who completed at least one of the eight assignments. For each course format, the table shows the mean percentage score for each assignment, the standard deviation, and the number of students completing the assignment. The last row in the table shows students' mean performance on all completed assignments. The independent t test was used to compare the academic performance of students in the two course formats. No significant differences were found between the traditional and distance students in mean scores on the five topic tests or the two papers. However, a significant difference was found between the two course formats on the final exam. Traditional students scored 7.16% higher, on average, than the distance students on this exam. A comparison of students' mean performance on all completed assignments revealed no significant difference between the two course formats.

Survey Results

Starting Survey: Table 3 provides a comparison of responses of students in the traditional and distance classes to Questions 38-53 on the Starting Survey. The table shows the mean response, standard deviation, and number of students who responded to each question. The independent t test was used to compare mean responses of students in the two course formats. Mean responses to Questions 46, 48, and 49 were significantly higher for students in the traditional classes. That is, traditional students agreed less strongly than distance students that the course readings, videos, and discussions would be interesting. No other significant differences were found between responses of distance and traditional students on the remaining Starting Survey questions.

TABLES 2 3 and 4 GO HERE

Ending Survey: Table 4 compares the response of traditional and distance students to Question 6 on the Ending Survey which asked 'Will you take another philosophy course in the future?' Thirty-eight students in the traditional classes and 22 students in the distance classes responded to this question. The table shows the number and percentage of students in each course format who answered yes/maybe or no. A chi square test of independence indicated that there was no significant difference between the traditional and distance formats in the distribution of yes/maybe and no answers, x2 (1, n = 60) = 0.62, p > .05.

Table 5 provides a comparison of responses of students in the traditional and distance classes to Questions 38-53 on the Ending Survey. The table shows the mean response, standard deviation, and number of students who responded to each question. The independent t test was used to compare mean responses of students in the two course formats. Mean responses to Questions 45 and 47 were significantly lower for students in the traditional classes. That is, traditional students agreed more strongly than distance students that the class was overall enjoyable and that the lectures were interesting. No other significant differences were found between responses of distance and traditional students on the remaining Ending Survey questions.

TABLE 5 GOES HERE

Discussion

Three hypotheses were tested in the present study. The first hypothesis was that students in the two course formats would not differ in their academic performance. The findings comparing the scores of traditional and distance students on graded assignments, for the most part, were consistent with this hypothesis and lend support to the 'no significant difference' research (Russell, 1999; Shinaberger, 2006). The only difference found on the graded assignments was that the traditional students scored higher on the final exam than the distance students. This difference appears to have been the result of several extremely low final exam scores in one of the two distance sections of the course. The average final exam score for this class was 59% compared to an average of 76% in the other distance class. One possible explanation for this discrepancy may relate to the fact that students were allowed to drop their lowest score on any of the eight assignments. Given that the final exam was the last assignment, it may be that students not needing the final exam score to improve their overall grade did not prepare for, complete, or put much thought into the final. A cursory examination of the students' scores for this particular distance section revealed that of the thirteen students who took the final exam, only six students needed to do so. And, of those students not needing to take the final exam, or taking it to attempt to raise their overall average, seven received a lower score on the final exam than their average score going into the final. Several of these students may have 'given up' on the final once they realized it would not boost their overall grade. This explanation for the difference in mean final exam scores is supported by the finding of no significant difference between traditional and distance students' mean scores on all completed assignments or letter grades earned in the course.

The second hypothesis predicted that students in the two course formats would not differ in their anticipated learning outcomes and course satisfaction. Analysis of the Starting Survey data revealed few differences between traditional and distance students in terms of their course expectations. There was no significant difference in the sorts of grades they expected to earn, their desire to engage in philosophical thought and discussions, their anticipated sense of accomplishment, their critical thinking skills, and their knowledge of philosophy and philosophical methods that would be acquired by completing the course. However, there were significant differences in their expectations about how interesting various components of the course would be. While students in both the traditional and distance classes expressed the same expectations about the lectures being interesting, students in the traditional sections of the course expressed less agreement that the readings, videos, and discussions would be interesting. This difference could be attributed to responses of students who were repeating the class after previously failing the course. What may be more likely, however, is that the distance students, who were older, were more realistic about their role in the learning experience and more invested in getting something out of the class compared to those who enrolled in the traditional format.

The third hypothesis predicted that students in the traditional course format would perceive that they had learned better and be more satisfied with their learning experience than students in the distance format. As on the Starting Survey, there were very few differences in mean scores between students in the two course formats on the Ending Survey. Again, there was no significant difference between students' perceptions of their grades and satisfaction of course goals, their perception of having engaged in philosophical thought and discussions, their sense of accomplishment and their perceived understanding of philosophy and its methodology. Whereas the distance students had begun the semester agreeing more strongly than the traditional students that the readings, videos, and discussions would be interesting, at the end of the semester there was no significant difference between the two groups in how interesting students found these components of the course to be. There was a difference in the mean responses to how enjoyable, overall, they found the class to be and how interesting they thought the lectures were. The mean response of traditional students was solid within the 'agree' range whereas the distance students were leaning toward 'disagree' with regard to the overall enjoyableness of the course. And while the term 'lecture,' at least for the traditional students, is ambiguous between the PowerPoint lectures available online to both the traditional and distance students and the 'live' lectures given in class, we can assume that the traditional students found the in-class lectures more interesting than the distance students found the online PowerPoint lectures. This supports the idea that there is some essential component to learning philosophy that takes place in the traditional classroom that the instructor did not duplicate in the PowerPoint lectures prepared for her distance courses. Croy's (2004) comparison of student learning and attitudes between three sections of a deductive logic course, one traditional, one distance, and one a hybrid of the two formats suggests that the latter may capture the best of both worlds. Students in his hybrid course had the most favorable attitudes towards the instructor, the course, themselves, computers, and other students. Thus, even if there is no significant difference in academic performance between distance and traditional students, the face-to-face interaction between students and instructors may provide that missing ingredient essential to a positive learning experience.

If there is some aspect to learning philosophy that goes beyond what is measurable on objective exams, and we believe there is, this study did not specifically reveal it. Instead, for the most part, the data collected tend to support the 'no significant difference' research that, when coupled with arguments for distance education in general, seems to justify offering philosophy courses in a distance format. Since distance students were no less likely to believe they would take another philosophy course than the traditional students, a department wanting to increase enrollments should not rule out offering distance courses. In addition to seeking better, more revealing ways to measure student learning in philosophy courses in general, this study made it clear that it is important not only to help students satisfy the course goals, but to help them recognize that they have satisfied these goals. Moreover, we should continue to look for more engaging methods for teaching philosophy to distance students.

Finally, some general observations about teaching a distance course are in order. There are unique characteristics of teaching a distance course that should be kept in mind by those who wish to teach such courses and by those administrators who want to support their faculty who do so. Preparing a distance course is particularly time consuming as one could spend six months to a year planning and organizing such a course depending on how one intends to deliver the content of the course. Students benefit from and appreciate a detailed schedule and outline of the course materials and requirements to help them stay on track. It is easy to find oneself too available to students by constantly monitoring their progress online, sending reminders to complete assignments, and responding to email throughout the day, every day. Technology used to deliver the course content is sometimes complicated and is constantly changing which means that faculty must be trained in using the technology and have an accessible support staff. Since distance courses provide access to education to those who might not otherwise have it, distance courses play an important role in society. By extension, faculty who take on the challenge of teaching such courses make a unique contribution to student learning in today's society.

Acknowledgments

Funding for this research was provided by the Center for Effective Teaching and Learning Scholarship of Teaching and Learning Grant Program AY2005/6. We would like to thank Chris Mee and the Institutional Research department for providing the students' demographic data and Sara Brallier for consultations before and after the study was conducted.

Endnotes

  1. Analogous arguments can be made, mutatis mutandis, for other disciplines, for example, fine arts and theater.
  2. A notable exception is Ron Barnette who has been a champion of teaching philosophy in a distance format for the last decade. See Barnette 1998, 1999, 2000, 2003 and 2006.
  3. It gave her flexibility in teaching the semester her child was born in the absence of any university supported maternity leave.
  4. Rauhut, Nils and Smith, Renee, Readings on the Ultimate Questions: An Introduction to Philosophy (New York: Longman, 2005).
  5. Peaceable Kingdom, Tribe of Heart Ltd, (New York: Ithaca, 2004).
  6. Exceptions were made for students taking the course from out of state.

References

Return to vol. 6 no. 2 index page

This page was originally on the website of The Subject Centre for Philosophical and Religious Studies. It was transfered here following the closure of the Subject Centre at the end of 2011.

-
The British Association for the Study of Religions
The Religious Studies Project