Use of Virtual Science Park resource rooms to support group work in a learning environment.
This paper presents a detailed evaluation on the acceptability of a range of synchronous and asynchronous collaborative tools provided within the Virtual Science Park (VSP) for group work in a learning environment. In this study, the VSP was used to provide a web-based 'Resource Room' adopting the familiar 'folder' metaphor for structuring and linking resources, and a number of different user interfaces for interaction and sharing information. A list of criteria is established for the evaluation. By using scenario testing and structured questionnaires, qualitative feedback was collected from 43 Masters students. The findings reinforce and add to the concerns highlighted in other studies, in particular, the importance for shared awareness, privacy, conventions for interaction and the provision of an effective multimedia environment. More attention will be needed in these areas for effective use of these groupware tools.
INTRODUCTION
The aim of this paper is to explore the effective use of
a range of groupware tools provided in the Virtual
Science Park (VSP) for group work in a potentially
distributed learning environment. It examines the level
of support being offered to students in order to gain
access to tutors and for students to work together in
small groups outside the 'traditional' teaching
environment. The VSP has been developed at the
University of Leeds since 1995 [7,24]. It uses a range
of information and communications technologies (ICT)
to facilitate technology transfer from university to
industry. Only a subset of its functionality was used in
this study.
The motivation behind this paper was driven by two
areas of development. Firstly, there is the changing
role of advanced ICT in today's societies. Applications
such as electronic mail and the World-Wide-Web have
diffused into all walks of life. The concept of
electronic mail was first developed in the 1970s
ARPANET project. It took around a decade to become
a commercial product, mainly for big organisations,
and another decade to reach schools and homes in the
western world [17]. World-Wide-Web, on the other
hand, has a much shorter history. It was first invented
in 1990 by Berners-Lee as part of a CERN (European
Particle Research Centre) project and took no more
than 5 years to reach different communities.
Interestingly, real-time collaborative tools (such as
desk-top conferencing systems and joint authoring
tools) have received a rather mixed reception. A
number of research projects in the Computer
Supported Co-operative Work (CSCW) and Human-Computer
Interaction (HCI) communities have been
examining the systems and the human issues
[1,3,18,28,30]. It would be informative to find out to
what extent these research ideas have been adopted in
today's commonly-available commercial products.
The second driving force behind this study is the
increasing demand on universities to deliver flexible
and timely training at the tertiary level and beyond.
This comes in a range of different forms such as
distance learning, continuing professional education,
life-long learning and industry-led programmes. It is
likely that most of these students might not be campus-based
and the provision of training or support will be
on-demand. This opens up a range of new challenges
for university teachers - for examples, how to deliver
effective support 'on-demand' and how some of the
innovative forms of teaching, such as the use of group
work, could be conducted in this environment.
The rest of the paper begins by explaining the
background to the VSP and how the resource room
was setup which provided the basis for the trial. The
interaction required by co-operative learning is
outlined, and issues raised in other related CSCW/HCI
projects are discussed. The paper then reports on an
evaluation of how well the collaborative tools
(synchronous and asynchronous) were received in a
trial involving students and lecturers responsible for a
Masters-level module. The feedback was collected
using scenarios and questionnaires. This evaluation
was undertaken against a framework for systematic
analysis. The outcome will inform the designers and
potential users of these collaborative systems.
The Virtual Science Park has been created out of a
research and development programme in the
interdisciplinary Centre for Virtual Working Systems
at the University of Leeds. It is now being exploited
commercially via a majority-owned University
company, VWS Ltd. The Virtual Science Park
provides an on-line, Web-based environment for its
tenants to develop their businesses. Tenants use the
VSP for a variety of services. The most common are
information brokerage, on-line consultancy, support
for collaborative projects and, increasingly, graduate,
professional or executive education and training.
The VSP tool-suite is constantly evolving but includes,
in addition to the browser facilities inherent in the
rooms' metaphor, a range of knowledge-management
and collaborative working tools which allow tenants to
manage and exploit their knowledge resources. These
tools include structured directory services fed by an
underlying information model, wide exposure to search
engines, a document management system that allows
full control over the upload and retrieval of any type of
electronic multimedia content, bulletin boards, alert
services and integrated real-time collaborative working
tools. Additionally, there is a powerful search
capability across all information entities in the VSP
directory.
Drawing on experience gained through pilot projects in
work-based learning that involved partnership between
employees at the workplace, industrial mentors and
university tutors, the VWS team have refined the
provisions in the VSP to offer full support for
customer-orientated graduate, professional and
executive education and training. The service is
specifically designed to enable small groups of learners
to work in their own time and at their own
convenience, with help from industrial and academic
mentors, and to have access to a relevant set of
learning resources. The VSP is ideally suited to the
development of high-level skills to small groups. It is
not designed for the mass distribution of education to
large groups.
For the providers of educational services, whether
these be traditional academic tutors or company staff-
development trainers, the VSP offers a range of
services that allow the providers to manage their
resources, to dynamically grow them and to facilitate
re-use of material in different contexts. For the
consumers of educational services there is easy and
focused access to relevant resources, whether these are
information resources or domain experts, opportunities
for formal and informal group work and for social
interaction. The VSP offices with integrated
videoconferencing and collaborative working tools
support real-time supervision and tutorials, whilst
discussion forums support asynchronous discussions
within and between learners and tutors.
The VSP staff will help tenants to customise a
complete on-line learning environment which will
draw in relevant combinations of the VSP tool kit.
Additionally, because the learning environment is part
of the wider science park provision, there is access to
the full range of expertise and knowledge resources
publicly available in the VSP.
The VSP resources are now being used by several
major corporate clients for the career development of
their senior and middle management staff. They are
also being used by a consortium of 3 universities, 24
companies and the relevant trade association dedicated
to strategic visioning and scenario planning for senior
executives in small and medium-sized packaging and
associated industries. The VSP is also being used to
support an international partnership involving a British
university, a German university and a global IT
company to develop on-line MSc modules. The
systems have also been extensively and critically
evaluated by academic staff and a post-graduate group
of about 45 students in Leeds, the findings from which
are reported below.
A RESOURCE ROOM FOR GROUP LEARNING
In the context of a learning environment, a resource
room is a virtual place where students can access
electronic learning resources provided by the
lecturers/tutors, find peer and tutor support, as well as
a place to 'work' with other students. If desired, it can
also be used for students to deposit the relevant
material that they obtained or created. A resource room
uses the familiar 'folder' metaphor for structuring
resources (uploaded files, web links, and textual
notes). A student will gain access to the VSP using a
web browser and a password. Appropriate software
and hardware, such as Microsoft NetMeeting
collaborative tools and audio/video capture cards for
videoconferencing, are required for synchronous
interaction.
In the VSP, the managers of a resource room (usually
the lecturers/tutors) decide how to subdivide the
workspace, what functionality to include and how to
define the access controls within the area. For the
Masters module in this study, a set of folders was
provided containing various resources. These included:
. areas for working - where students would find
information about group projects and their private
study room for group work. Only members of the
group and the tutors could access a group study
room. Within each study room, students had a
discussion board for their own use and a contact
area for initiating Microsoft NetMeeting sessions
with the other group members. The group members
could decide how to structure their shared
workspace but they were not allowed to change the
access permission enabling tutors to monitor the
resource room use. Figure 1 shows a view of the
resource room in study area of group B. There are
folders for the group contacts, project work, and a
link to the groups' bulletin board.
. the ability to contact the tutor - where students can
either directly initiate a conferencing session with
the tutor/ email him/her or enter the tutor's virtual
office and use the facilities provided there (see
figure 2).
. learning materials - where students find lecture
slides and pointers to papers or web sites classified
according to subject areas. Figure 3 shows a view
of a few of the resources available in the 'Distance
Learning' folder.
The resources described above provided an
environment through which the use of asynchronous
facilities (such as email, discussion board and shared
document areas) and synchronous facilities available in
a typical desk-top conferencing system (such as chat
and shared applications, with or without audio/video
support) could be explored. A range of viewers was
also provided for viewing documents which were
produced using different tools.
This set up echoes the 'Common Information Space
(CIS)' concept articulated by Bannon and Bodker [2].
It also highlighted the 'dialectical nature' of CIS -
openness versus closure - by the way in which tutors
provide information in the subject folders which are
then open to the whole class. However, the facilities
and information in each group study room were only
available to the members of that group (and under the
prying eyes of the tutors).
It is also noted that the asynchronous environment in
the resource room is similar in some ways to
applications such as 'teamroom', based on
Lotus/Domino [9] and BSCW [5].

Figure

1. The resource room in study group B's area

Figure

2. Tutor's office - the colloborate link initiates a
NetMeeting session with the tutor

Figure

3. View of resources in the 'Distance Learning'
folder
ELEMENTS OF CO-OPERATIVE LEARNING
In order to appreciate the level of support needed from an
'ideal' system in a virtual learning environment, it is useful
to outline the characteristics of co-operative learning. It is
not within the scope of this paper to explore the philosophy
behind this kind of innovative teaching method, but the
discussion is focused on the interactive style it requires
amongst the students.
Johnson, Johnson and Smith [21] described co-operative
learning as "instruction that involves students working in
teams to accomplish a common goal". Co-operative
learning is not a synonym for students working in groups. It
must have the following elements in the learning exercise:
. the outcome depends on every member contributing to
the task;
. all students in a group are held accountable for doing
their share of the work and for mastery of all of the
material to be learned;
. it requires students to interact in terms of providing one
another with feedback, challenging each other's
conclusions and reasoning, and perhaps most
importantly, teaching and encouraging one another;
. students are encouraged and helped to develop and
practice trust-building, leadership, decision-making,
communication, and conflict management skills; and,
. members periodically assess what they are doing well
as a team, and identify changes they will make to
function more effectively in the future.
If we wish non campus-based students to be able to benefit
from this kind of teaching, the use of advanced ICT must
be explored.
Human Issues in Common Information Space
Groupware systems are people systems [9,11] and must
take into account the situated, frequently unstructured
nature of work. Studies had been conducted in the CSCW
and HCI research communities to gain a better
understanding of the subtle, taken-for-granted human
behaviour in order to inform and refine future generations
of groupware. The accumulated wisdom related to the use
of Common Information Space can be categorised in the
following way:
. the importance of shared awareness,
. the need to retain some privacy,
. the importance of having protocols/conventions for
interaction,
. the need to overcome the limitations imposed by the
monitor and keyboard.
It will be enlightening to see, in the evaluation, how far
these ideas have been taken up by today's widely available
products and if any of the concerns have been addressed.
Shared Awareness
Group awareness can be defined as "an understanding of
the activities of others, which provides a context for your
own activities" [8]. The hypothesis generally adopted in the
literature was that providing awareness is a good thing.
Representative projects include - Europarc RAVE media
spaces [12]; University of Toronto's Telepresence [15],
University of Calgary's Group Kit [29]. Gutwin, Greenberg
Roseman suggested a useful list of elements of
workspace awareness - identity, location, activity level,
actions, intentions, changes, objects, extents, abilities,
sphere of influence and expectations. Designers of
groupware systems are still struggling to develop a system
which can cater for all these elements. Various CSCW
projects have also been attempting to address some of the
above elements, although often in an application-specific,
limited, or ad-hoc manner [14].
In addition, the implementation of shared awareness is
often limited by technical factors such as bandwidth and
physical size of the input/output devices. The difficulty in
the implementation of telepointers (multiple cursors) was
reported by Greenberg and Roseman [13]: "Unfortunately,
modern window systems are tied to the notion of a single
cursor and application developers must go to great lengths
(and suffer performance penalties) to implement multiple
cursors".
It may be that, the burden is on us to find out what are the
essential elements without which the concept of distributed
group work will not be feasible; and what are the elements
that are merely 'nice-to-have'.
Privacy
Discussions concerning privacy in groupware usually
centres around security and legislative issues. Bellotti [4]
broadened the issues to include the individual's control
over how much information to share with others, and how
accessible they want it to be. In terms of Information
Access Control, the common mechanism is to implement
technical access control. Other novel ideas to provide social
control were also tried out such as 'ShareMon' which
provides auditory feedback when someone else is snooping
around your shared files over the network [6]. Interpersonal
Access Control makes more use of awareness and feedback
for social control. As there is no inherent reciprocity built
into computer mediated systems, it creates some challenges
to groupware developers. Projects in this area include
CAVECAT media space which provided four selectable
door states (open, ajar, closed and locked) to indicate
availability [25].
Interestingly, Bellotti suggested that having fixed private or
public spaces may well be creating obstacles to
collaboration as it prevents users from experimenting and
adapting to new uses.
Conventions for Interaction
In groupware, communications amongst people are
mediated by shared artifacts. The manipulation of these
artifacts requires coordination at both the human and the
application levels [31]. For example, in a real-time co-authoring
environment, how does one tell if a pause on the
shared application indicates whether the other party is
thinking about the wording of the next phrase or is waiting
for your contribution? Examples of research include IRIS, a
group editor environment [23] and the use of cursors for
gestures [16]. However, a study by Mark et al. [26] showed
that the effective use of conventions takes time to develop.
Beyond Monitor and Keyboard
In the early 90s, the integration of shared workspace and
interpersonal space into one workstation was a
breakthrough; for example, TeamWorkstation integrated
the video window with the text window [19]. Today's
commercial products resemble what TeamWorkstation
offered and also suffer from the same problems such as
having arbitrary seam between different 'spaces'. The lack
of real eye-contact is also a problem in desk-top video
conferencing systems. The ClearBoard metaphor was later
developed [20], which superimposed the interpersonal
space over the shared workspace using a huge screen and a
digitized pen. This overcame the two problems mentioned
but has not yet been seen as a commercial product.
New research work is also being carried out for co-located
and distributed team members under the auspice of
'cooperative buildings' [32]. Unfortunately, the systems are
still largely in research laboratories and may not be
available for commercial use for some time.
We would like to ascertain whether modern ICT can be
used to further enrich the learning of non campus-based
students by involving them in group work requiring a
certain amount of interactions. The first step towards this
aim is to find out the acceptability of the features provided
in the VSP Resource Room which is implemented as a
'Common Information Space' for student groups to
interact. It is important to note the difference between
acceptability and usability [27] with the later being a subset
of the former. The kind of feedback sought was related to
practical acceptability (e.g. usefulness, reliability,
compatibility, performance) rather than pure usability (e.g.
ease of use, learnability and such like).
It was decided to try the experiment with the 43 students in
a Masters module. Many students in the class have previous
work experience. The ratios of UK:EC:International
students are about 4:1:4. The characteristics of the students
matched closely with those who are likely to enrol on a
non-campus based course.
Two means of data collection were devised. Firstly, the
class was divided into groups of 4 or 5. They were asked to
use the VSP Resource Room for this module to test out two
standard scenarios:
1. A representative from each group was to clarify with
the lecturer, at real time, the content of a lecture slide
(which also included making an appointment in advance
for the meeting);
2. Three students from each group were to agree upon and
produce an outline of a group report. This included
sharing ideas at the initial phase through to allocating
responsibility to individuals in order to produce the
final report.
The students were also requested to test 2 to 3 additional
scenarios of their own choice. They were asked to
document the actual happenings while testing the scenarios.
The second set of feedback was obtained by a structured
questionnaire which asked each student to provide
comments on their preferences on the functions provided
by the Resource Room for group working as well as their
preferences for the user-interface, and also their reactions
to various aspects of the Group Study Room. It was made
clear to them that the questionnaires were not assessed and
would only be used for research. A total of 33 completed
questionnaires were received and analysed, the results of
which is reported below.
EVALUATION
The analysis of student responses was conducted at two
levels - human and IT system. The human behavioural
aspects include the individual's attitude and the group
dynamics within the common information space.
Evaluation at the system level includes the functionalities
and the user-interface adopted in the applications.
Use of the VWS Resource Room
In the questionnaire, students were asked to name two
functions of the VWS Resource Room which best support
group work and two functions which hinder group work.
Functions of the VWS Resource Room which best support
group work appear to be the ability to conduct shared work
asynchronously closely followed by the potential for real-time
collaborative work (Table 1). However, functions of
the VWS Resource Room associated with real-time
collaborative work (in particular, lack of control, poor
audio and inadequate feedback) were also most likely to
hinder the progress of the group (Table 2).

Table

1 Functions of the VWS Resource Room which
best support group work
Main Function (with comments) No.*
Asynchronous shared work
. deposit your own contribution for other
members of the group to add to/build upon
. makes the groups own working version of the
assignment document available to all members
of the group
. judge the progress of the group in generalReal-time group work
. work together remotely regardless of time and
location - late at night/from home
. collaborate within a short time span and
conclude on topics
. enhances collaboration and communication
between lecturers and studentsDiscussion board
. post messages for others in the group without
the need to meet
. keeps each other up-to-date with who is doing
what and what still remains to be doneShared storage area
. allows the group to post their work to one
central point
. locate together materials which are related to
each otherActivities related to own work
. create own work area
. manage documents and other resources*No. of times mentioned by students - they were asked to name
any two functions.
At the Human Level
Shared awareness
The main issues raised by students in relation to awareness
within their distributed working environment were centred
around poor feedback from the applications used. This
included lack of feedback about who was on-line; in many
instances, there was an assumption that everybody was
always logged on and had NetMeeting running in the
background which was clearly not the case. Furthermore,
there was no indication to show when other members of the
group had joined a NetMeeting session.
When using the Chat application students found that there
was no visual clue that someone was actively typing a
comment. This frequently led to confusion, for example, it
was often difficult to determine which members of the
group were still 'there' and whether they were still paying
attention.

Table

2 Functions of the VWS Resource Room which
hinder group work
Main Function (with comments) No. *
Real-time group work
. lack of control
. poor audio
. inadequate feedback on what's happening at
the other endActivities related to own work
. resource room is badly structured
. version update facility is not user-friendly
. end up with too many copies/drafts of
documentsAsynchronous shared work
. better version updating information required
. better notification facilities
. ordering on screen of resources submitted
. messy structure quickly propagates without
controlDiscussion board
. not very user-friendly
. poorly designed method of posting messages
. grant individuals the ability to manage their
own messagesShared storage area
. unaware of who else has access to the group
area
. unable to collaborate on earlier versions of
documents*No. of times mentioned by students - they were asked to name
any two functions.
Students sometimes experienced difficulties knowing what
method of communication to use. They felt it would be
useful if NetMeeting had some way of alerting the user on
the current method of communication being used or the
method of communication to be used to initiate a
collaborative working session. For example, at one stage a
lecturer was trying to have a dialogue with a student via the
Chat application, but as the student was frantically trying to
reset audio and microphone settings at home, the Chat
window was minimized on his PC. It would have been
useful for the student to have known which tool to set up in
the first place.
In another case, a student initiated communication with a
lecturer using the Chat tool. However, the lecturer, who had
just conducted an audio meeting with another student,
assumed that he would be using the same tool with the next
student; audio seemed to be his preferred method for real-time
communication with his students. Unfortunately, the
student did not have any audio equipment and the meeting
was delayed until he could set up a microphone and
speakers.
Privacy
The main issues raised by students in relation to privacy
within their distributed working environment were centred
around lecturers accessibility to their work and the
boundaries between other groups taking the module.
Students were asked whether they were aware that the
lecturers on the module could read the content of
everything in their 'private' group study room. Most
students were aware of this or suspected it (there was no
visual clue), and some 73% of students said that this did not
affect their use of the group study room. This is probably
because most people see the group study area as a 'public'
place anyway. The remaining 27% of students felt that the
process of writing their draft reports had been compromised
given that they often wanted to post comments which they
felt were only suitable for viewing by the other members of
the group.
Students found that the boundaries between groups were
tightly drawn and they were not able to interact with other
groups going the module who would also be using the VSP
Resource Rooms. This was an intentional design act by the
lecturers in order to discourage students from collaborating
with anybody other than the group members when testing
the scenarios. However, some students found the closeness
of the group study area was an inconvenience when they
wanted to collaborate or share experiences with someone
from another group.
For the lecturers of the module, a NetMeeting call was an
additional source of interruption on top of the telephone
and students knocking on the door. At one point, a lecturer
was having a meeting in her office while a couple of
students were trying to call her using NetMeeting. It was a
serious distraction and it took a few minutes to send those
calls away. Some students felt that the technology should
go some way in helping lecturers to be more contactable.
A few students had found that the VSP and the group study
rooms offered no completely private places (the contents of
the group study Rooms could be viewed by the lecturers),
and no open public spaces which might provide
opportunities for informal interactions (e.g. akin to the
coffee machine, the corridor, etc.
When students added a document to their group work area
they wanted to be able to explicitly grant or deny
permission to each member of the group. This would have
allowed the formation of sub-groups within the working
area under the direct control of the students.
Conventions for interaction
The main issues raised by students in relation to
conventions for interaction were largely concerned with
establishing general principles for control of the shared
workspace.
Guidelines for interaction were required during a
collaborative session in the following areas:
. taking control during the session;
. opening another application on top of the groups
collaborative work space;
. indicating the desire to relinquish control to another
. and, leaving or closing the session.
On starting NetMeeting, students experienced some
confusion over control as they did not expect that the first
member of the group to click the mouse would retain
control. The fact that when someone else was in control
ones own mouse stopped working made it very difficult to
break in and take control. Some groups resorted to
coordination of file sharing using the telephone, whilst
another group used the Chat board to co-ordinate
ownership of control. Groups found that if all members
wanted to add and amend the document at will then some
co-operation on screen control was essential. Some students
felt they spent too much time fighting for control in their
group and described their collaborative session as 'war-
like'.
The fact that any one member of the group has no control
over the actions of any other member of the group can
sometimes result in chaos. Students in one group were very
annoyed by the actions of one particular member who
insisted on opening non-shared windows over the shared
workspace. The result of this for the other participants was
that the shared window was blacked out. In another group,
the student in control during a NetMeeting session decided
to leave the meeting and shut down his application with the
result that the application on the other two machines also
disappeared. Unfortunately, in this instance, all work was
lost and had to be re-done. In such cases, students clearly
find it difficult to indicate their desire to leave a
collaborative session or are unaware of the consequences
for the other members if they do quit. On reflection, several
students felt their group should have agreed on a unique
for indicating the wish to leave a collaborative
session.
In terms of the ability of the VSP group study room to
create cohesion within a group, almost three-quarters of
students (74%) felt that the study room had united their
group and helped it work together effectively. However,
students considered this was only the case as long as all
members made proper and effective use of the available
features.
Beyond monitor and keyboard
The use of audio/video technology to augment text-based
communications was explored. From the study, there were
clear preferences for different communication media and
different levels of proficiency in using them. Some students
found responses from lecturers were rather brief when
Chat was used. When the communication medium was
switched to audio this was no longer a problem. Clearly
some people dislike too much typing. In other cases,
students found that they had to wait a long time for
messages to appear due to the slow typing speeds of some
members of their group. The setup procedure for audio
technology was not user-friendly. It took some trial-and-
error and sometimes the group had to resort to using a
telephone in order to get the settings right.
The lack of multicasting for audio/video means that only a
one-to-one connection is possible even in a multi-user
conferencing session. This made it unsuitable for group
work involving more than two people. This was obviously
a serious drawback with groups of at least 4 members.
Without an audio link, the users found it very annoying and
confusing to have to switch between different windows
even for a minor task, for example using Chat to
communicate intentions and back to one or more windows
to perform the task.
At the System Level
Functionality
Some of the most important functional requirements arising
from the evaluation are considered here.
For many students this was the first time they had used
NetMeeting for group work in a learning context. Their
reaction to the application was interesting; most students
liked the fun of using it and could see its potential, although
its limitations in addressing the human issues made it the
application that hindered group work the most. Some
students reported that they had only used it because they
were 'forced' to do so in order to adhere to the
requirements of the module. Others felt that the limitations
of NetMeeting had made their collaborative sessions more
of a recreational activity than an opportunity for serious
discussion and collaboration, and found the application was
suitable providing the level of detail required by the work
was not too great.
Most students liked the availability of private group study
rooms which allowed them to structure the space as they
liked and the group area also proved useful in helping them
keep track of the groups progress. Some students noticed
the need for an agreement by all members of the group to
use the room. They also remarked on the need for general
'housekeeping' tasks as in the shared document area it was
easy to end up with a collection of drafts and out-of-date
information. They found it was not easy to take a unilateral
decision to delete files. A temporary 'trash can' was
suggested. Also, the provision of log files which record
transactions such as file modifications and deletions for
each directory was suggested.
Although the group study area was favourably received, the
students did notice a deficiency in integrating their own
workspace (using their favourite tools) with the common
information space. The integration is currently not
seamless.
The provision of scheduling tools for project management
was also considered to be an important requirement.
User Interface
Students clearly preferred a user interface (UI) that is
intuitive and with which they have some degree of
familiarity (i.e. they have used the UI or components of it
before!). A number of metaphors which had been omitted
from the VSP Resource Rooms were requested by the
students including drag and drop functionality, ability to
bookmark pages, and ability to perform tasks across a
group of resources (e.g. deleting a group of files).
Some students could not descipher the meaning of a couple
of icons intuitively and required the help of tooltips. Only
later versions of the browser (i.e. Internet Explorer v4.0 and
above) had this feature.
During a synchronous session the desktop could become
very cluttered, for example, separate windows for chat,
shared applications, and a couple of others for searching
information. This problem was exacerbated for those
students using small monitors.
Some students liked the range of user interfaces provided
by the VSP, from the clickable image of a whole room (e.g.
the lecturer's personal office) to the purely text-based pages
used in the Resource Room. However, other students
clearly had a higher expectation of feeling they were in a
virtual space. Such students commented that a lot of pages
looked similar and were rather dull and uninspiring with
insufficient opportunities for interaction throughout the
VSP site.
Other
Finally, some groups found that the network connection
and its quality were not always reliable, especially when
using modem and an Internet Service Provider (ISP). Also,
some students requested training in the use of the VSP
Resource Rooms and the synchronous and asynchronous
facilities (students were only given a quick demonstration
of the VSP for the purpose of the course, as the rest should
be intuitive!).
Discussion
In terms of acceptability, most students in our evaluation
found that the asynchronous environment worked
reasonably well. However, the synchronous (real-time)
environment was still falling short of being ubiquitous and
was presenting obstacles for group working which requires
interaction in a multimedia space. HCI is, at present,
limited in its application in multimedia systems [22]. There
were some very good ideas in the research communities for
improving real-time collaborative environments (as
discussed in earlier sections), but currently it is still a tough
balancing act for designers of groupware applications: to
provide for awareness, for performance in speed, or for
cost-effectiveness. It was also interesting to have observed
the difference in users' perceived acceptability for
recreational purpose and for 'real-work'.
Nevertheless, interests in using modern ICT in supporting
non campus-based learning has escalated. New projects
continue to emerge such as those conducted in the
Knowledge Media Institute of the Open University in the
UK [10]. It is important that further developments are made
in the commercial products and services for overcoming
some of the limitations highlighted in this study. Firstly,
high-speed broadband network infrastructure is necessary
for implementing techniques for enhancing shared
awareness. A flexible environment is needed which either
enables users to define their privacy requirements or adapts
the access control according to the information captured by
the system. Conventions for interaction can evolve with use
(especially with the younger generation who is now rather
use to chatting on-line) but education on the proper use
should be provided. Virtual Reality might be the area which
will take the on-line learning environment to a different
level of experience.
CONCLUSION
Progress had been made in communications technology and
in understanding the notion of workspaces in recent years.
This paper presented a detailed evaluation on the use of a
virtual learning environment which was set up as a
'resource room' in the Virtual Science Park. The 43
Masters students in the trial provided qualitative feedback
on the use of some commonly available collaborative tools
on the market to inform designers of groupware systems
and their potential users. In general, the students found that
the asynchronous provision of the common information
space adequate and useful for co-ordinating tasks during
the process of group working. The findings also highlighted
some human issues which urgently needing attention before
collaborative tools can be effectively deployed. This is
especially true for synchronous tools, as there is still a lack
of attention on the provision of awareness, privacy,
conventions for interaction and seamless interactive
multimedia spaces which have been identified in the paper
as the important areas for addressing human issues. This
inadequacy is currently preventing their effective use in
performing real interactive group tasks.

ACKNOWLEDGEMENTS

We thank the Masters students (class 1998-99) in the
School of Computer Studies for their participation and co-operation
in the evaluation.



--R

Readings in Groupware and Computer-Supported Cooperative Work
Constructing Common Information Space.

What You Don't Know Can Hurt You: Privacy in Collaborative Computing.
Supporting collaborative information sharing with the World-Wide Web: The BSCW Shared Workspace System
'Kirk Here:' Using Genre Sounds to Monitor Background Activity.
Virtual Working Systems to Support R and D Groups.
Supporting awareness in a distributed work group.
Designing Groupware Applications: a Work-Centered Design Approach
The Knowledge Web - Learning and Collaborating on the Net
Workflow Technology in Computer Supported Co-operative Work
Realizing a
Groupware Toolkits for Synchronous Work.
Workspace Awareness in Real-Time Distributed Groupware: Framework
Communicating about communicating: Cross-disciplinary design of a media space interface
Implementing gesturing with cursors in Group Support Systems.
Groupware: its past and future.

TeamWorkstation: Towards a seamless shared workspace.
A seamless medium for shared drawing and conversation with eye- contact
Cooperative Learning: Increasing College Faculty Instructional Productivity
Towards Principles for the Design and Evaluation of Multimedia Systems.
Design issues for a distributed multi-user editor
The Virtual Science Park.
Experiences in the Use of a Media Space.
Supporting Groupware Conventions through Contextual Awareness.
Usability Engineering.

Building realtime groupware with GroupKit

Workspace Awareness for Distributed Teams.

--TR
TeamWorkStation: towards a seamless shared workspace
Experiences in the use of a media space
Realizing a video environment
Building real-time groupware with GroupKit, a groupware toolkit
Design issues and model for a distributed multi-user editor
MYAMPERSANDldquo;Kirk here:MYAMPERSANDrdquo;
Communicating about communicating
Readings in GroupWare and Computer-Supported Cooperative Work
Usability Engineering
Cooperative Buildings, Integrating Information, Organization, and Architecture
Workspae Awareness for Distributed Teams
What You Don''t Know Can Hurt You
Workspace Awareness in Real-Time Distributed Groupware
Towards Principles for the Design and Evaluation of Multimedia Systems

--CTR
John T. Langton , Timothy J. Hickey , Richard Alterman, Integrating tools and resources: a case study in building educational groupware for collaborative programming, Journal of Computing Sciences in Colleges, v.19 n.5, p.140-153, May 2004
Timothy J. Hickey , John Langton , Richard Alterman, Enhancing CS programming lab courses using collaborative editors, Journal of Computing Sciences in Colleges, v.20 n.3, p.157-167, February 2005
Christina Brodersen , Ole Sejer Iversen, eCell: spatial IT design for group collaboration in school environments, Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work, November 06-09, 2005, Sanibel Island, Florida, USA
evaluation;common information space;co-operative learning;collaborative tools;Virtual Science Park