Teaching and Learning > DISCOURSE

VLE Wiki as Philosophy Assessment

Author: Istvan Danka and Juha Saatsi

Journal Title: Discourse

ISSN: 2040-3674

ISSN-L: 1741-4164

Volume: 10

Number: 2

Start page: 147

End page: 157

Return to vol. 10 no. 2 index page

Introduction

This article reports a PRS Subject Centre funded mini-project that aimed to develop and evaluate collaborative online wikis as an assessment form in philosophy. Virtual Learning Environment (VLE) is a networked computer-based system that supports educational purposes. Wiki is a platform for creating and editing collaborative webpages. 'VLE wiki', hence, is a virtual educational environment for collaborative creation and editing of web pages. The aim of the project was to develop a framework for collaborative student work and its assessment based on a VLE wiki. The framework was developed in conjunction with two single-semester modules in philosophy of science. This article provides (i) the project's motivation and rationale; (ii) description of the wiki assessment employed in the pilot; (iii) the results achieved and problems solved; and (iv) the challenges that remain.

Motivation and rationale

Doing philosophy is often viewed as a somewhat solitary endeavour: sitting in the proverbial armchair reading, thinking or writing. At the same time philosophy is of course essentially argumentative, requiring at least two points of view juxtaposed one against another. Whilst experienced philosophers are able to develop and contrast alternative viewpoints by themselves, equipping undergraduate philosophy students with the required argumentative skills is an ever-present challenge. The use of a VLE offers various means to tackle this challenge. For example, one may consider tutorials and other face-to-face discussion groups that are part of the typical means of developing argumentative skills through peer interaction. Though undeniably integral to philosophy education, it is worth considering the virtues of offering students opportunities for corresponding interaction online using a VLE. For instance, less prepared students are less able to respond to arguments on the spot, and ditto for those averse to face-to-face social interaction. A VLE offers the possibility of 'instant but constant' responses in virtual discussion rooms: responses can be provided more or less immediately (as opposed to written feedback on an essay) but they are not ephemeral (as opposed to oral communication), and thus allow time for thinking and responding.

In order to provide incentives for students to participate in virtual 'discussion' and argumentation, one may want to employ VLE interaction as a form of assessment. Essay and exam writing are of course the traditional methods of assessment for philosophy undergraduates. Essay writing, in particular, is extremely entrenched because 'essay writing is what philosophers do'. Most philosophy students do not become professional philosophers, however, and many end up working in sectors where the ability to write extended essay-like arguments is not by default the most highly valued asset. Other forms of writing and efficient communication may be better developed by other forms of coursework, and it is worth exploring alternative forms of assessment to essay-writing, not only in order to offer variety, but also to provide a wider range of transferable skills. Equipping philosophy students with the broadest portfolio of transferable skills possible is another constant challenge. Again, a VLE offers various means to respond to this challenge.

Motivated by these kinds of broad considerations, we developed and evaluated VLE-based 'wiki' assessment as an alternative to essay writing. Students in small groups collaboratively authored an online wiki that was used both as a formative assessment during the course of the module and also as a summative assessment for the whole module. By enriching the variety of assessment methods and coursework, students are provided with a more enjoyable (or at least less monotonous) overall learning experience and a wider range of transferable skills. Collaborative wiki writing facilitates the acquisition of various transferable and non-cognitive skills not developed by traditional assessment methods: editing, reviewing, collaboration, IT-skills, organization, etc. It also develops a different kind of efficient communication from essay writing. Wikis can also provide novel means by which to provide both formative and summative feedback: students get formative peer feedback through collaboration, and summative feedback can take into account the quality of such peer collaboration. Wikis offer a contemporary blended-learning alternative to traditional assessment methods, taking full advantage of modern VLE facilities such as Blackboard.

Amongst other potential benefits of a wiki are the following: it introduces a genuine collaborative dimension ('learning community') that is difficult to achieve in philosophy; it allows a natural way to spread a written assessment over the module, forcing students to study the whole module equally thoroughly; it allows students to follow the progress of others' written work; and the module leader can teach and assess a different kind of effective communication that also personalizes student learning experience.

Scientific research on the educational applications of wikis is now an extended and growing field. There are, however, no accounts of VLE wiki techniques applied specifically to philosophy education. The pilot project aimed to provide some experimental ground for inves150 Istvan Danka and Juha Saatsi—VLE Wiki as a Philosophy Assessment tigating the applicability of the insights of the above-mentioned general sources.

The pilot

The project ran over two semesters in conjunction with two philosophy of science modules. It involved (i) designing a wiki assessment for one of the modules; (ii) testing it in practice and gathering feedback from the students through questionnaires and other means; (iii) improving the assessment method with the second module; before (iv) analysing the results and presenting them through a project webpage. In the pilot the wiki assessment had the following basic format: students, working in small groups of 5-7 on a total of 20 questions (spanning a whole module), collaborated by leaving comments on each others' initial answers so as to enable the author of each answer to improve it further.

What follows is a mix of details on the pilot implementation and related points of recommendation.

Keeping things simple

There are several ways to set up a broadly wiki-type collaborative assessment in philosophy. The pilot wiki endeavoured to keep things as simple as possible in order to make the rules of the game easy for the students to grasp. These rules are, in outline, as follows.

This is a very simple scheme. Indeed, it may seem so simple that it doesn't even satisfy the definition of 'wiki', since students do not engage in genuinely collaborative authoring of an answer. Whether a genuine wiki or not, the advantage of keeping things simple is that this makes the individual contributions clearly discernible. This is critical if a wiki is used as a form of summative assessment. (This is less of a concern if the assessment plays only a formative role in a module and the feedback provided is purely qualitative.) The more collaborative the writing process is, the harder it is to identify individual achievement. Hence, there are some tensions in the applicability of wiki technology to summative assessment. It may be possible to identify individual outputs—provided that the VLE platform at hand is powerful enough (e.g. Blackboard gives access to every edit of a wiki)—but the examiner's workload very quickly becomes unmanageable.

The cognitive/communicative value of wikis is the collaborative creation of shared knowledge, a sort of social interaction among the students. The more collaborative the authoring process is, the higher the efficiency in collectively producing high-quality answers. But an increase in collaboration decreases the efficiency of a wiki as an individual assessment. All the same, if collaboration increases the quality of writing and argumentation, these collaborative skills also must be assessed in order to evaluate a student's overall abilities.

Deadlines

The most important difference between traditional essay writing and wiki-style collaborative work is the possibility of writing comments on others' (initial) answers or essays. Comments appear immediately and in the pilot these could not be deleted or modified later by the students. Students then edit their wiki entry in light of the comments they have received. In order to ensure a smooth, timely process, some deadlines are required. In the exemplar set-up there is a deadline for each: (i) posting an initial version; (ii) posting comments to others; (iii) posting a final version. That is, a student revises his/her initial answer in response to a single round of comments from group peers. There could be more iterations—more than one round of comments—requiring more time and increasing the number of deadlines to keep track of. Even with a single round of comments, with four wiki entries per student, this amounts to 12 deadlines over the module, which already begins to detract from the enjoyableness of the exercise!

Marking scheme and encouraging collaboration

Regarding the collaborative dimension of the exercise, the biggest challenge was to ensure high enough student activity. As a carrot one can reward collaboration by a suitable marking scheme. In the exemplar a set percentage of the final mark was awarded for comments, with the additional clause that in order for a student to gain a mark above the pass threshold (40) for a given section of the module, at least one bona fide comment must be provided in that section. The percentage awarded for comments was 20% and 15%, respectively, for the two modules tested. Here 15% seemed to be high enough to act as an incentive. In as far as the final mark should reflect students' knowledge and understanding—as opposed to mere activity and effort—it seems better to keep the percentage relatively low. In the spirit of collaboration, each author needs to receive at least some comments. In order to encourage an equal distribution of comments, one can give small extra reward for the first comment on each entry.

In the modules piloted, the average number of comments remained modest. No serious debate or conversation occurred. On the other hand, student comments were focused and got to the point of criticising the form or content of the initial wiki entries, though they were sometimes either too general (e.g. 'The second paragraph could be clearer') or too fastidious (e.g. 'The world 'unnecessary' is repeated too many times'), and not appropriate for driving proper revision in content. In order to ensure the quality of peer feedback it was therefore extremely important to be very clear on what precisely a 'bona fide' comment consists in.

Wiki questions

Questions in the two pilot wikis were focused and asked for to-thepoint answers that could be given in 650 words or less. The idea was that all four wiki entries (per student), added together, would amount to approximately 2,500 words, which is on a par with a typical undergraduate essay. The incentive was to keep the overall workload manageable.

A tight word limit, accompanied by appropriate tutoring for this type of writing, would also help to develop a succinct writing style, and help students focus only on what is most essential.

The obvious downside of a tight word limit is that it does not allow much room for independent creativity and argument development. This could be viewed as a severe price to pay for a philosophy module, and for this reason a wiki assessment is best viewed as a way of complementing essay writing, certainly not replacing it. On the other hand, one may take the view that it is also important for students at this stage (level-2 undergraduates) to know and be able to succinctly express as many as possible of the key concepts, arguments, and positions pertinent to a given module. Arguably a wiki assessment where a student answers and contributes to numerous focused questions can serve this goal better than a single essay. In any case, much here depends on the exact wiki questions set and for more advanced students an encouragement to follow a debate for/against the main statement(s) of an initial paper could be a way to achieve more depth through collaborative work, for example.

If wiki assessment is used to complement an essay or an exam, one must (of course) minimize the overlap between these and the individual wiki questions. In the piloted modules this was achieved with wiki questions that zoomed in on specific topics, whilst using the exam questions tested for broader understanding drawing on several wiki questions from across the module.

Monitoring and feedback

The module leader can monitor students' progress through the various versions of their wiki entries. (S)he can also leave feedback, just as other group members do, by using the commenting facility (and perhaps somehow making clear which comments come from the module leader). Ideally, a well-functioning group would require only a modest amount of input from the module leader, as feedback is received from peers whilst writing a wiki entry and from the module leader on the finished entry through appropriately justified marking. In general, if a wiki is designed to spread the coursework across a module, students are likely to get a better sense of their progress throughout the module. Depending on the number of various deadlines involved, there can be some record-keeping in that regard, but Blackboard, for instance, provides an easy way to keep track of the actual editing and commenting times. It also allows the module leader to revert back to an earlier version, if needed.

Taking advantage of Blackboard VLE

More generally, the Blackboard VLE platform, with its out-of-the-box wiki facility, provides an efficient and adequate system for managing an exemplar-type wiki. There are a number of very useful features to this facility. For example, one can easily create student groups and multiple copies of the basic wiki structure, each copy accessible only to a single group. So one can divide students into small discussion groups that each work on a single essay, which can in turn be discussed by other students once the finished version is shared in a discussion room accessible to all students. One can also use unidentifiable usernames instead of real names for the sake of anonymity in order to help students argue more confidently. And one can create hyperlinked crossreferences between different wiki pages so that a group's final output becomes a coherently navigable, multilayered web-entity without unnecessary duplication. An inclusion of multimedia documents is also possible. How much use is made of such navigational features ultimately depends on the students, of course. It may be useful to keep things quite simple to begin with and emphasize the quality of the content over hypertext writing skills.

Blackboard allows the module leader to extract a wiki—with or without comments—in PDF form, for example. Marking and final feedback can be provided electronically with this facility. It may be wise to extract copies of the completed pieces of work in case a systemwide IT problem occurs, and certainly if one doesn't print out hardcopies for marking. Whilst Blackboard VLE in general functioned very well in the project, there are some expected limitations worth bearing in mind. For example, since only one person at a time can edit a page in the wiki, the overall structure should assign an individual webpage to each individual question.

Feedback and evaluation

The feedback gathered indicates very distinct appreciation of the attempt to enrich the array of assessment methods available. Here are some telling data (on the scale 1-5; 1=strongly disagree, 5=strongly agree, 3=uncertain):

'Wiki is a worthwhile assessment method in philosophy': 4.25
'I would recommend the adoption of wikis in other philosophy modules': 3.58
'Wikis introduce beneficial diversity to assessment methods in philosophy': 4.00
'The workload of the wiki has been appropriate': 4.17
'Wiki proves a good way of giving and receiving feedback': 4.33
'With a wiki one's workload is better spread over the semester': 4.27
'Wiki assessment develops different skills from essay writing': 3.91
'Wiki entries have covered the module's key topics well': 4.68

The course evaluation questionnaires for the two modules also said things like 'Wikis were a very good idea'. There were no distinct negative comments to the contrary. Students viewed both modules successful overall, with one summarised by the student representative as a 'resounding success'. The questionnaires also clearly indicate that wiki assessment can provide a welcome way of providing and receiving feedback. In light of the feedback and the students' general interest towards the wiki assessment, and its potential for further development, it is safe to say that the project was successful as a starting point for developing further this mode of blended learning in philosophy.

Discussion: advantages, disadvantages, and remaining challenges

Drawing on the student feedback and the module leader's personal reflections, we summarize here the perceived advantages, disadvantages, and remaining challenges of using a wiki of this type as a philosophy assessment.

Advantages

There are some clear advantages to a wiki in relation to essay writing. In particular, a wiki provides a good way of spreading the workload evenly over the course of a module. Students provide and receive feedback more continuously, both from the module leader and from student peers, thus teaching students to read their peers', and hopefully their own, essays more critically. Wiki assessment requires a student to have some understanding of even those key points, issues, and positions that are beyond the scope of their final essay; this allows students to be assessed on most of a module's key topics. Wiki also introduces variety to coursework, developing a wider range of skills and abilities. In particular, wikis develop efficient and 'prioritizing' writing skills, and also other transferable and non-cognitive skills that are not developed by essay writing (e.g. IT skills; reviewing, and editing, etc). Last but not least, wiki introduces a collaborative dimension ('learning community') to philosophy assessment—an aspect that has been largely missing in traditional forms of philosophy coursework.

Disadvantages

There are some clear disadvantages as well. For example, a wiki assessment (as implemented in the pilot) does not cater for the development of an in-depth analysis of a single philosophical issue or position. For the same reason, wiki does not allow a great deal of room for independent creativity and argument development. It is even possible that the collaborative dimension of the exercise allows a few outstanding students to be the source of most of the creativity exhibited. Neither does a wiki significantly develop independent research skills, or the ability to organise and structure a longer piece of written work. The number of written wiki assignments and the requirement to comment on others' work do not necessarily yield a particularly enjoyable assessment, due to the extended nature of the exercise and the number of deadlines involved. Also, a wiki can potentially be a relatively onerous method of assessment for the module leader, especially for larger cohorts.

Given these disadvantages, in our view wikis are only a way of supplementing essay writing (as has been already argued above).

Remaining challenges

The basic format of the piloted wiki assessment leaves much room for tweaking and further development. A key challenge remains regarding the collaborative dimension of a wiki: it is challenging to find a way of prompting genuine collaboration that produces an output that allows students to be individually assessed for their own contribution. The simple version of the assessment presented in the pilot exemplar only allows students to comment on each other's wiki entries; it does not invite or even allow collaborative editing of the entries themselves. This simple format satisfies the second desideratum, as each entry is ultimately solely written by a single student, but it only partially succeeds regarding the first desideratum. It is unclear whether this tension can be resolved from the present perspective. Collaborative work is to an extent incompatible with individual assessment, and in as far as the purpose of higher education is not exclusively to teach students but also evaluate their development as individuals, the room for any sort of collaborative work (including VLE Wiki) is restricted. This does and should not lead one to disregard the obvious advantages discussed above.

Return to vol. 10 no. 2 index page

This page was originally on the website of The Subject Centre for Philosophical and Religious Studies. It was transfered here following the closure of the Subject Centre at the end of 2011.

-
The British Association for the Study of Religions
The Religious Studies Project