text:
E-testing increases the time available for teaching. This is mainly because most questions are automatically marked by the system and the results are returned immediately to learners. The time spent in marking and returning learners' work is greatly reduced. Re-sits can often be organised on an individual basis, freeing the teacher or lecturer to concentrate on teaching the rest of the class.
E-testing systems have the ability to collate and instantly analyse learners' responses to online questions. This means that reports can be generated about the performance of a single learner, a class, or a number of classes.
E-assessment -v- paper assessment
Some learners with additional support needs find e-assessment preferable to traditional pencil and paper assessments. It should be possible for learners to modify the screen display to suit their particular needs.
Assistive technologies
Most e-assessment systems are capable of supporting assistive technologies commonly used, such as the screen reader Jaws.
Learners can use social software technologies, such as wikis, to collaboratively generate and edit content on a central website. This could include learners collaborating on research or to develop strategies for different tasks or to solve problems. Social networking tools, such as blogs and forums, allow learners to share views and information and to communicate easily and regularly. Some ideas on using these tools for assessment can be found in the Section Supporting assessment with Web 2.0.
Feedback through e-testing
In online testing, it is possible to analyse an individual learner's response to individual questions, and to use this to give feedback.
If a lot of the responses to a particular question from learners in the same teaching group show the same misunderstanding, this can be used to amend teaching. Also you'll be able to identify the learners who need help.
Instant feedback
Online testing which uses automated marking can be incorporated into the delivery of a Unit and can present feedback to learners immediately.
Feedback through e-portfolios and social software
Other e-assessment tools such as e-portfolios, blogs and wikis can be set up to notify teachers and lecturers when learners have created content which they may wish to review, or have submitted work for formal assessment. These systems allow you to create and deliver feedback to learners in a secure environment.
In conventional assessment, marking can vary between markers; however with e-testing, computers will mark the same responses in the same way every time. This also streamlines the marking load for teachers and lecturers.
Although free-text / extended response or essay questions can be handled by most assessment engines, they are not usually marked automatically. Instead learners type answers in text boxes and the assessment system stores the responses for scrutiny by a teacher or lecturer. Answers can be printed out if necessary for marking.
Web 2.0 describes the range of web services that collectively represent the second 'version' of the world wide web. These include collaborative and social networking facilities. These tools can be used to aid and measure learning and to find, capture, generate, store and present evidence for assessment.
'Assessment 2.0' is the use of Web 2.0 services to generate assessment evidence. Some of the ways in which Web 2.0 technologies can be used in the assessment process can be seen by using this link.
E-assessment can measure things that we have not been able to assess before. For example, by using social software (wikis and blogs) the contribution made by individuals to a group assessment task can be assessed with much more confidence.
Security will be improved though the gathering and storing of assessment data and results online.
Results can also be collated easily and provided to learners online or by e-mail.
In addition, e-assessment systems often offer secure, automatic transfer of assessment results directly into management information systems, saving time and reducing clerical errors.
There is a significant reduction in printing of paper assessments leading to a reduction in printing costs.
The associated administration involved in storing papers securely, as well as distributing papers and collecting responses is also eased. With robust e-assessment systems, the chances of a learner's responses going missing should be reduced.
There are many measures that can be taken to improve security during e-assessments. Some possible security breaches that can be prevented with the help of the software are:
Validity is about the effectiveness of an assessment to measure what it is supposed to measure. An important aspect of validity is the realism of the activities learners are asked to carry out; the more realistic the activity, the more valid the assessment. Social software and Web 2.0 approaches generally can improve realism by encouraging learners to use the technologies they use in their everyday lives, to capture and generate authentic items for assessment.
Authenticity relates to the ownership of the evidence; whether it is actually produced by the learner or someone else. A learner's e-mail messages, forum contributions and blog posts can provide an assessor with information on the learner's knowledge and skills, and so alert the assessor if the learner's work changes in quality. Technologies such as VoIP permit remote oral questioning of learners to verify that they understand the evidence they have submitted for assessment, which gives a good indication that the evidence is the learner's own work.
Although written in 2002, the paper Principles of Assessment published by the CAA Centre at the University of Luton defines validity and considers the different types of validity related to assessment.
Many online tools, such as social software and other Web 2.0 tools, can be used to aid validity and authentication.
The learning potential of Web 2.0 services is well known. This paper explores their assessment potential.
Although the use of technology can increase opportunities for plagiarism, it can also provide ways to detect plagiarism. Essays submitted electronically can be analysed using tools which will highlight inconsistencies between different sections of the essay. This means that if other peoples' work has been incorporated into the essay this can be detected automatically.
At the University of the West of Scotland, academics from a number of disciplines have implemented the use of Turnitin originality checking and plagiarism prevention software.
With e-assessment, e-verification (or remote verification) may sometimes be possible. Benefits of e-verification include:
From 2008-2010, SQA worked with centres and External Verifiers to pilot e-verification. This project was initiated because SQA was aware that centres and learners were increasingly using e-portfolios and other electronic means to manage assessment evidence and they wanted to explore how SQA could make use of e-verification as part of its quality enhancement process.
The report on this pilot is available by using the following link:
Piloting e-verification 2008-10
E-assessment, and electronic evidence in general, facilitates the online sharing of assessment evidence and resources and so facilitates professional discussion on standards and assessment decisions. This can benefit centres by supporting new models of internal verification – cross-centre; authority-wide; company-wide. It could also benefit awarding bodies by supporting new approaches to external verification – e-verification; regional verification; online central verification.
Online formative e-tests can be used for practice and to reinforce learning while preparing learners for the summative assessment. At the same time, the learners become familiar with the assessment engine.
Assessments that incorporate randomisation and selection from a substantial bank of questions can be used repeatedly by learners for formative assessment without losing the confidentiality of the questions for use in summative assessment.
When learners are working with e-portfolios, or in technology-supported environments (such as VLEs), the distinction between formative and summative assessment may be less clear than it is traditionally. Online learning environments provide access to learning resources, and communication tools and forums. Learners are able to work largely independently, or in collaboration with other learners, benefitting from online feedback on their formative work from peers, teachers/tutors and Assessors. This feedback is likely to become an integral part of the assessment process, encouraging learners to re-draft some of their formative work and promoting reflection and self assessment.
This means that formative work can evolve to become summative evidence, as learners work incrementally towards the point when they judge that their e-portfolios meet the assessment criteria. Once items are submitted for summative assessment they should be 'locked', preventing learners from making further amendments.
Use this link to find out how JISC shows that e-portfolios can be used to support assessment.
E-portfolios supporting assessment
Higher Education Academy
Use this link to find out what the Higher Education Academy says about the role of e-portfolios in formative and summative assessment.
The role of e-portfolios in formative and summative assessment
The PassIT Research Project
Use this link to read about the PassIT Research Project on formative and summative assessment.
E-assessment can have a positive impact on learning. By increasing the frequency of assessment, and improving feedback, we can improve learners' motivation. Learners who prefer to do coursework could be motivated by the increased level of feedback that online assessment can facilitate.
PassIT Research Project
You may find this link to the PassIT Research Project useful.
E-assessments can incorporate media-rich elements that include video, sound, animations, simulations and interactive graphics. These elements are designed to be user-friendly and encourage learner participation.
There is a vast range of assessment-creation software offering a huge variety of different functions, so it is difficult to find much that is common to all. Grouping such software, it is possible to organise them into two categories. Firstly, quiz authoring tools and, secondly, assessment engines.
Quiz creation tools create one-off quizzes, or groups of quizzes, often in html, which can then be used as stand-alone assessments in a browser environment, or through a learning delivery platform such as a VLE.
Assessment engines offer the functionality to author quizzes, but in turn will also offer an online environment in which to carry out the testing, or will integrate with a learning environment to deliver the same. The key feature of assessment engines is their ability to track, record and report on learners who carry out the assessments.
Online, on-screen tests are often made up of multiple choice and multiple response-type questions, but can also include other question types, such as sequencing and hotspots. Short answer questions, which allow learners to type in responses, could also be included.
The following is a list of the types of questions that you are likely to meet in an e-assessment.
JISC infoNet
JISC infoNet aims to be the UK's leading advisory service for managers in the post-compulsory education sector promoting the effective strategic planning, implementation and management of information and learning technology.
In one of their online infokits, 'Effective Use of VLEs: e-Assessment', JISC notes that there are three main types of objective questions which can be used which include students making a choice (single or multiple) inputting text (words, numbers or formulae) or manipulating information on the screen (marking relevant areas, moving items or drawing graphs or diagrams).
JISC's list of assessment question types can be found by using the following link:
E-portfolios are similar to paper portfolios but contain only digital items. Most are web-based so the learner is able to work online to create, store and arrange their items.
One of the key benefits of e-portfolios is that they encourage learners to take ownership of their work and to actively participate in the process of assembling their e-portfolios and using these to present their work. Although some candidates may need support in doing this initially, in the longer term it can lead to enhanced learner motivation and more rapid attainment of qualifications.
Another benefit of e-portfolios is that they allow the learner to include (or link to) evidence which spans subject boundaries, or experiences which take place outwith centres. This is important for interdisciplinary projects, and for qualifications where key pieces of evidence may come from employment or placement environments. The ability to add digital photographs, scanned images and video or audio files to e-portfolios can make these experiences even more authentic.
If used for assessment, the e-portfolio generally holds evidence for a particular unit or course. The learners, teachers/tutors and Verifiers can access the e-portfolio remotely and so the processes of review, assessment and verification are streamlined.
Social software includes communication tools for capturing, storing and presenting written, audio and video information, and also interactive tools which handle communication between users. Examples of social software include blogs and wikis.
A blog is a type of website, usually maintained by an individual who updates it with commentary and descriptions of events. Most blogs are interactive, allowing others to leave comments. It is this interactivity that distinguishes blogs from static websites. A typical blog combines text, images and links to other relevant web pages and media. Many blogs function as online diaries.
When used for assessment, blogs can be valuable for capturing learners' reflections and evidence such as learner personal statements; assessor observations; witness testimonies provided by others. All of this can help with authentication of evidence.
A wiki is a web site that allows users to add and update content on the site using their own web browser. This feature means that wikis tend to be created collaboratively by the site users.
This is valuable when a wiki is used for assessment, because it can capture the contributions of different members of a group who are working collaboratively on a document related to a project.
An Electronic Voting System (EVS) usually consists of a computer, a projector, one or more receivers and remote control handheld electronic transmitters, known as clickers. Learners are asked questions, shown as multiple-choice questions in a presentation, and they use the clickers to select their preferred options. The handsets transmit this information to the computer software which produces histograms or bar charts of the results.
When used for assessment, different types of questions can be used depending on the purpose of the assessment. Even the basic Yes/No or True/False question types can be used to test recall; to assess learners' understanding of issues or to test if they are able to recognise solutions or appropriate approaches for solving problems. Questions can also be broken down into smaller stages.
Games-based assessment uses computer gaming technology and techniques for assessment purposes. Games can be engaging for learners but can also be used to embed learning outcomes. When used for assessment, different types of games (eg quiz, action, strategy games, role-play) can be used to assess different learning outcomes such as memory/recall; behaviour; functional competence and so on.
Increasing interest is being shown in delivering assessment through mobile platforms, given that many learners own one or more devices offering the capacity to deliver assessment content. Such devices will include mobile phones, wireless enabled mp4 players, tablet pcs and others.
The key advantage of authoring for and delivering to such devices is the ability to provide learners with assessment in an appropriate context – such as a workplace – and on demand. In creating assessments for mobile devices there are two key areas to be mastered, the authoring of content appropriate for a mobile device, and the delivery of that content, either by pushing it out to learners or by providing a gateway for access.
The JISC e-Assessment Glossary includes over 700 items relating to the processes involved during the design, presentation and recording of candidates' e-responses to assessment stimuli. It has been produced for use by awarding bodies, teachers, lecturers, practitioners and administrators across all types of educational institution and across all sectors, as well as software developers and providers.
Although this glossary is not fully up-to-date (it doesn't include social software, electronic voting systems), it is still a very useful resource.
The starting point for selecting the types of e-assessment should be pedagogy - the learning and teaching approaches that are suitable for the subject. Consider which forms of assessment best support learning in conventional situations and how these approaches could be better designed or enhanced with technology. Most assessment approaches can be e-enabled.
Guidance on Pedagogy
Further guidance on pedagogy can be found in the 12 Principles of Good Assessment and Feedback which were produced by the Academic Policy Committee of The University of Strathclyde in Glasgow and agreed by Senate in 2008. These principles are intended to help academic staff who wish to promote student engagement and self-regulation in learning. They provide a guiding framework and they can be adapted to different disciplinary contexts.
12 Principles of Good Assessment & Feedback (The University of Strathclyde)
Objective questions, such as multiple choice questions (MCQs), would not be appropriate for all subjects or assessments. They are effective in assessing factual recall. It's more difficult to use them to assess higher-order skills such as creativity, communication or synthesis.
That's not to say it can't be done at all: learners' understanding and problem solving abilities could be assessed to some extent using these approaches but may be assessed better when the learners have to recall and enter information rather than when they are presented with a selection of possible responses.
What is important is that an e-test is comparable to a 'traditional' test, not that it is an on-screen copy of the pencil and paper test. The questions chosen should assess the same knowledge and/or skills with the same degree of difficulty. For example an e-test may use a 'matching' type of question which is not available in a traditional test environment.
Some software can be programmed to automatically mark short answer responses but complex qualitative or reflective responses would be captured and marked by human markers.
Some educational psychologists believe that different learners have different learning styles, such as visual, auditory or tactile approaches to learning, though in recent years learning style theories have also been challenged.
For more information on the history and theories of learning styles see http://en.wikipedia.org/wiki/Learning_styles
Using a variety of assessment and e-assessment approaches is likely to be the best way to suit the widest group of learners. The three categories of learning style most commonly recognized are:
It's important to bear in mind the purpose of the assessment.
Diagnostic Assessment is carried out before or after the learning activities take place to ascertain the level of a learner's (or group's) knowledge or skill.
Formative Assessment is carried out as part of teaching and learning. This is often known as assessment for learning because the information gathered by the assessment is used to adapt the teaching to suit the learner's needs.
Summative Assessment is carried out at the end of a learning programme or at specific points during it to judge or mark the learner's achievement. This can affect the level of award the learner achieves and/or allow progression to the next stage of learning.
JISC infoNet
JISC infoNet aims to be the UK's leading advisory service for managers in the post-compulsory education sector promoting the effective strategic planning, implementation and management of information and learning technology.
In one of their online infokits, 'Effective Use of VLEs: e-Assessment', JISC looks at the different types of assessment - diagnostic, formative and summative.
The following link will take you to JISC's Methods of Assessment:
Writing good quality assessments, which accurately assess learning, and which do not involve unnecessary distractions or offer unintentional clues, can be challenging. In particular, writing assessment items which test higher levels of learning requires time and thought.
You may find information in the following two SQA Academy Courses useful. If you don't have an SQA Academy account, you can login as a guest to these courses from the login page.
An Overview of Assessment Principles
Writing Objective Test Questions
Ensuring that assessment items are accurately and equally levelled requires thought and care. In particular it is possible to unintentionally introduce elements which affect the degree of difficulty of an assessment item. A simple example of this would be inappropriate distractors, which affect the difficulty of a question, or question stems with poor phrasing, which introduce ambiguity.
Bloom's Taxonomy
We tend to use Bloom's Taxonomy of educational objectives to focus on the skills and abilities we want to assess.
A revised Bloom's was published in 2001, this time focusing on learning objectives.
Andrew Churches has developed a Digital Taxonomy, suggesting verbs, activities and technologies, appropriate to the digital age, across the different areas of Bloom's. This Prezi by Joshua Coupal provides a summary of Bloom's and the Bloom's Digital Taxonomy.
You can find resources from Andrew Churches at Bloom's Digital Taxonomy.
There are two timing considerations with e-assessment.
The first relates to the demands e-assessment can make on learners who are required to use self-paced assessment. It may be possible for some learners to choose when to take the test but other learners will rely on the tutor to set the timing and pace of assessment.
Secondly, the issue of timing may be an institutional one. If assessment of learning requires large scale simultaneous testing – all candidates sitting the same test at the same time - then there are infrastructure and support requirements to be considered.
Also relevant is the section on Requirements – networks; computers; rooms in 'Preparation'; 'What other things do I need to think about?'.
Good quality feedback is an essential element of e-assessment, in particular where it is to be used formatively. This means feedback at the general level, but also for each individual response in objective tests. If the learner is to gain value from taking e-assessment tests formatively, then there must be feedback which explains not just that a particular choice is wrong, but also why it is wrong.
Adequate feedback, and opportunities to review and respond, have to be planned into the delivery. Timing of the programme overall should make allowance for students to absorb and respond to feedback, whether individually or in a classroom situation.
SOLAR is SQA's online assessment tool that provides summative and formative e-assessments. Assessments within SOLAR are made up of questions which are generated dynamically from an itembank. The summative assessments already available have been created and packaged by SQA, and quality assured by External Verifiers. The same bank of questions can be used to create formative assessments, and these can be delivered through the SOLAR e-assessment system in the same way as the summative e-assessments. These e-assessments cover qualifications at all levels from National Certificate to Higher National Diploma.
The summative e-assessments are pre-verified — so centres know that quality is assured. This provides real benefits for both learners and learning providers.
The formative e-assessments, which are quality assured and available to support many subjects, provide detailed feedback for learners. Learners can access the assessments 24 hours a day, 7 days a week using a browser and the SOLAR Open Assess portal.
Each delivery of an e-assessment to a learner is dynamically generated by the system, based on pre-specified rules.
The picture below is a link that will take you to a web page about SQA SOLAR assessments.
This link will allow you to try SOLAR formative assessments which include most of the question types mentioned in Suitability; What types of e-assessment could I use?; Types of assessment; E-testing.
SOLAR formative assessments
NAR is Scotland's National Assessment Resource for Curriculum for Excellence and is a joint development between Education Scotland, SQA and the Scottish Government.
It will support assessment in practice and will support practitioners in developing a shared understanding of standards and expectations for Curriculum for Excellence and how to apply these consistently.
This link will take you to the NAR website. You will need a Glow account or SQA credentials to access the National Assessment Resource.
If you do not have a Glow account or SQA credentials to access the National Assessment Resource, the following links will provide you with more information about the NAR.
NAR, Scotland's National Assessment Resource for Curriculum for Excellence
About NAR: Guide to the National Assessment Resource
About the National Assessment Resource: Video Guides
Video Bank: Finding Materials on NAR
Making use of e-assessments has a number of benefits for both the learner and lecturer and forms a critical part of the learning process.
Bank on Success, a project focusing on e-assessment, is currently being piloted at the JISC RSC Scotland South & West.
This link will take you to the JISC Regional Support Centre Scotland website where you can preview the assessments produced.
These resources were developed for Moodle 1.9 and the last version of Blackboard but you should still be able to view them. If there was interest, JISC may consider updating them.
As they were developed as part of Bank on Success they were not quality assured; they represent resources developed by practitioners as part of that project.
Funded by the Skills Funding Agency and its predecessors to support learners and organisations in the post-16 sector, the NLN materials represent one of the most substantial and wide-ranging collections of e-learning materials in the UK.
The NLN Materials are small, flexible 'bite-sized' episodes of learning. They are not whole courses but are designed to support a wide range of subject and topic areas. The materials adopt a range of approaches involving some knowledge acquisition, a chance to practice and absorb, and some form of assessment to check that the learning has been understood and the learning outcome(s) achieved.
The materials give learners things to do and a chance to think for themselves. They are designed to be fun. Learners can make mistakes but can try again. There is plenty of in-built help if they get stuck and opportunities to go back and revise. Many of the materials include games or quizzes. Accessibility options are available on many of the materials to suit learners' individual needs. The materials are also designed to appeal to the different learning styles of students.
This link will take you to the NLN website where you can access the materials.
Scotland's Colleges has a range of Learning Resources which include online formative assessment items, across a range of subject areas at SCQF levels 1-8. These assessment items were designed to be incorporated into the range of VLEs in place in colleges in Scotland.
The benefits include:
E-testing is a rapidly growing area of e-assessment involving the delivery of examinations and assessments on screen, either using local systems or web-based systems.
Guidance on e-testing delivery is available by using this link.
SQA is encouraging centres to provide learners with opportunities to use e-portfolios when taking SQA qualifications. E-portfolios offer benefits for learners, centres and SQA. They provide a secure environment where learners can store, organise and share their learning and assessment materials, in a range of media formats. E-portfolios allow tutors to access learners' work, to assess and provide feedback. For SQA, e-portfolios offer a secure and flexible way to view assessment evidence and internal verification materials.
SQA has published guidance on using e-portfolios. This resource will give you advice about good practice, and highlights some of the things you'll need to take into account when you're supporting learners as they create e-portfolios for SQA qualifications. The guidance also provides examples of e-portfolio forms and templates which could be useful for demonstrating good practice in using e-portfolios for assessment.
From 2008-2010, SQA worked with centres and External Verifiers to pilot e-verification. This project was initiated because SQA was aware that centres and learners were increasingly using e-portfolios and other electronic means to manage assessment evidence and they wanted to explore how SQA could make use of e-verification as part of its quality enhancement process.
The report on this pilot is available by using the following link:
Piloting e-verification 2008-10
DeskSpace allows candidates to record their course evidence online, keeping everything together and accessible to teachers/tutors and assessors, even when they are at more than one centre. Work can be uploaded and submitted online and the user friendly interface helps candidates structure their learning through the use of online templates.
The DeskSpace e-portfolio is available to all SQA Skills for Work and Baccalaureate candidates and is currently available to centres free of charge until June 2012.
This link will take you to the SQA webpage: Skills for Work ePortfolios (DeskSpace).
SQA's Endorsement Service
SQA offers an endorsement service for organisations which provide resources to support the delivery of SQA qualifications and awards. This service may also be applied to e-portfolios. The Endorsement team evaluates the e-portfolio against both the endorsement criteria and SQA's requirements for e-portfolios to ensure that it is robust and fit for purpose. You can find out about SQA's Endorsement Service by using this link.
There are various types of e-portfolio systems ranging from a secure file storage function within a stand-alone system, to more sophisticated web-based products, or they may be part of a VLE. Centres may also decide to develop their own e-portfolio systems.
Guidance on e-portfolio use is available by using this link.
JISC is the UK's expert on information and digital technologies for education and research.
Two resources from JISC offering guidance on large-scale e-portfolio implementation are available by using this link.
The e-Portfolio Implementation Toolkit has been developed from the JISC-funded e-Portfolio Implementation (ePI) study of successful institution-wide practice in the UK, Australia and New Zealand. The toolkit contains over one hundred exemplars of e-portfolio use in a wide range of further and higher education contexts and eighteen case studies of approaches to wide-scale implementation. These form the basis of the Toolkit guidance that addresses the challenges, issues and stages in wide-scale e-portfolio implementation and presents an implementation model to assist your institution or programme in embedding and sustaining e-portfolio use.
Also available are five new video case studies, Stories of e-portfolio implementation, illustrating implementation strategies taken by five of the case study institutions. These rich-media resources provide additional personal insights into the challenges and benefits each institution has encountered. The videos are embedded into the Toolkit and can be downloaded or played by using this link. Transcripts are available for users of screen readers.
Both resources are made freely available, providing practitioners and managers in further and higher education with a unique insight into current UK and overseas e-portfolio practice. Feedback, thoughts and comments are welcome on these resources, and the facility to do so provided in the Toolkit itself. The resources can be used in conjunction with Effective Practice with e-Portfolios (JISC 2008) and the JISC e-Portfolios infoKit to provide a comprehensive package of support and guidance from JISC on e-portfolios in further and higher education.
CLASS (Collaborative Learning Assessed by Social Software) is a tool for learners studying for the Project-based National Course (PBNC) in Health and Safety in Care Settings. It offers them a blog and a wiki to record group activities and generate a group project.
The graphic below is a link that will take you to the CLASS homepage.
JISC infoNet
JISC infoNet aims to be the UK's leading advisory service for managers in the post-compulsory education sector promoting the effective strategic planning, implementation and management of information and learning technology.
JISC has produced an online Infokit Guide to Social Software.
This infokit explains more about social software, illustrated with real-life examples, and outlining potential uses for learning and teaching, research and business and community engagement. It also aims to highlight the key issues which may need to be considered when implementing such technologies, and to provide pointers for successful use.
Travel and Tourism at Cardonald College
The link below will take you to the transcript of a Case Study on using a wiki to support research, planning and assessment in Travel and Tourism.
Podcast Transcript – Cardonald College
Education Scotland and game-based learning
The vibrant and dynamic world of the computer game and its impacts on learning and teaching is an area that Education Scotland is committed to exploring, promoting and developing.
You can find out what has been happening in classrooms around Scotland by using the following link: Education Scotland and game-based learning
Mobile and wireless technologies can provide flexible and timely access to learning resources, instantaneous communication, portability, active learning experiences and the empowerment and engagement of learners, particularly those in dispersed communities. Yet, despite the almost ubiquitous ownership of mobile phones, the adoption of mobile technologies in post-16 teaching practice is still in its infancy.
This link takes you to a case study on using mobile phones for formative assessment.
This link takes you to a case study on using tablet PCs for e-assessment.
SQA Deskspace
The Deskspace Mobile approach has now been trialled and tested and is available for download from the Add Ons section in your Deskspace Tools menu. The application is a simplified version of the e-portfolio with ability to send pictures and record evidence straight into your e-portfolio account. We hope this will help overcome difficulty in accessing suitable ICT equipment and time available to upload material to the e-portfolio. There are three formats available to suit a range of mobile phones.
Deskspace Mobile - Basic: A basic WAP enabled interface which is visible on all web enabled phones including an upload function. Deskspace Mobile - PDA: This includes the functionality above but is designed to operate on Windows Mobile PDAs and Smartphones.
Pebblepad iPhone app: This is an approved app by Apple. However, this version is currently only available under the Pebblepad branding and costs £1.79 to download, but may be customised for Deskspace at a later stage.
The graphic below is a link that will take you to the DeskSpace homepage.
Guidance on the use of electronic voting systems technology
If using these systems for summative assessment it would be advisable to seek awarding body prior-verification of the bank of questions you will use.
Electronic voting systems resources
You may find the following resources about electronic voting systems useful.
JISC is the UK's expert on information and digital technologies for education and research.
Electronic voting analysis and feedback for all
Case Study on Using Moodle easyVoter Student Response System for formative assessment
SQA offers a Prior verification service to centres who devise their own instruments of assessment or who wish to significantly change items from the National Assessment Bank to suit their particular local needs.
This link will take you to SQA's Prior Verification Service.
SQA is the national accreditation and awarding body in Scotland.
You may find information in the following links about how appropriate e-assessment is for various subjects and assessments:
SQA Academy
SQA Academy has been developed to extend and enhance the training and development opportunities available to all those who contribute to the assessment processes that underpin the qualifications of the Scottish Qualifications Authority.
The SQA Academy website is divided into a number of distinct "webspaces", each of which fulfils a different purpose. The majority of these webspaces are only available to registered users. However, some spaces are open to all, without the need for registration.
Professional Development Workshops
SQA's Professional Development Workshops are subject-based. SQA is currently piloting online workshops. They hope that by introducing online workshops they are able to reach more teachers as the cost will be much less. The online workshops are delivered on the SQA Academy website (www.sqaacademy.org.uk).
SQA's programme of Professional Development Workshops can be accessed by using the following link:
Professional Development Workshops
JISC is the UK's expert on information and digital technologies for education and research.
JISC has a set of e-Assessment resources in the JISC Design Studio.
You may find information in the following link about how appropriate e-assessment is for various subjects and assessments.
Assessment and Feedback programme
JISC projects funded under the Assessment and Feedback programme have just commenced and will be disseminating key findings over the next 3 years.
Scotland's Colleges exists to support, represent and promote the Scottish college sector.
Scotland's Colleges maintain Subject Networks which provide opportunities for debate, discussion, collaboration and sharing information.
These Subject Networks can be accessed by using the following link:
Education Scotland is the national body which is responsible for supporting quality and improvement in learning and teaching from early years to adult and community learning.
The e-Assessment Toolkit exists to provide clear, current, practical information to e-assessment practitioners, or those thinking of adopting e-assessment.
The toolkit offers useful tools and resources, helping users to develop and share their knowledge of e-assessment for effective implementation.
This link will take you to the e-Assessment Toolkit.
The e-Assessment Toolkit is hosted on the website of the e-Assessment Association (eAA).
The e-Assessment Association is for assessment/learning professionals or educators wanting to know more about the use of technology for exams, tests and assessments, and who want to influence the way electronic assessment develops.
This link will take you to the website of the e-Assessment Association.
Where e-assessments are created and used for summative assessment, they will be subject to validation and verification checks by the relevant external body.
The validation of e-assessments will focus on the form and content of the assessment questions, to check on the validity of the assessment for its intended purpose. Some awarding bodies will offer a prior-verification service for this.
The verification process will focus on the delivery and support mechanisms for the e-assessments, in order to ensure both reliability and practicality. Some awarding bodies will carry out checks on this at the prior-verification stage while others will consider it as part of their standard verification process.
In verification of e-tests which have already been delivered, it should be possible to view details of when a learner took an assessment, how long it took, and from which IP address it was taken. It should also be possible to re-create the assessment exactly as it was taken by the learner, including any randomly generated parameters and ordering used. Records of all responses to each question should be kept on the server, including the full text of free-text responses and any feedback or result given to the learner.
Where centres create their own assessment instruments and marking schemes, for use in summative assessment of SQA qualifications, SQA encourages them to seek prior verification of the assessments and the associated marking schemes. Prior verification is a valuable service which helps avoid issues arising with centre-devised assessment materials at external verification. It is currently a free service and if you are intending to create e-assessment items for summative assessment purposes you can also use this service.
If you intend to create a bank of assessment items, to allow learners to take online, automatically-marked summative tests for SQA qualifications, we would encourage you to tell us about that in advance to allow the bank of items and the assessment approach to be discussed.
SQA offers a Prior verification service to centres who devise their own instruments of assessment or who wish to significantly change items from the National Assessment Bank to suit their particular local needs.
This link will take you to SQA's Prior Verification Service.
The SOLAR summative e-assessments are pre-verified — so centres know that quality is assured. This provides real benefits for both learners and learning providers.
SQA has produced several publications on internal quality enhancement.
Although it was published in 2001 and uses the term 'Internal Moderation' instead of the current term 'Internal Verification', Guide to Internal Moderation for SQA Centres still contains a wealth of relevant information.
Specifically written for Ofqual qualifications, Internal Verification: A Guide for Centres Offering Ofqual Regulated Qualifications, is a brief guide on good practice in internal verification which is relevant in many other areas.
Scotland's Colleges
Scotland's Colleges maintain Subject Networks which provide opportunities for debate, discussion, collaboration and sharing information.
These Subject Networks can be accessed by using the following link:
SQA's Professional Development Workshops are subject-based. SQA is currently piloting online workshops. They hope that by introducing online workshops they are able to reach more teachers as the cost will be much less. The online workshops are delivered on the SQA Academy website (www.sqaacademy.org.uk).
SQA's programme of Professional Development Workshops can be accessed by using the following link:
SQA has adapted requirements from E-assessment: Guide to Effective Practice for e-portfolios and for e-testing.
Standards exist that govern the delivery of online testing – BS ISO/IEC 23988:2007 Information technology – and the following information in this section is drawn from those requirements and is intended to relate more to timed, supervised e-assessments.
Technical capacity & infrastructure of institution
There are different types of e-assessment, some more demanding on centres' infrastructure than others. It may be that not every institution is capable of applying and supporting the infrastructure needed for high stakes assessment using e-testing software, unless the institution can guarantee the levels of IT support and backup facilities required.
Similarly e-testing and other e-assessment approaches that require all candidates to undertake an assessment at the same time will make heavy demands on the infrastructure of an institution – computers, rooms, networks, invigilators - and may not be the right choice of strategy for every centre.
However e-portfolios, and other approaches to e-assessment which allow candidates to present their evidence for assessment when they are ready, should be possible for most centres if the centres are willing to invest in the necessary technology, staff development and learner support.
Student workstations
The supplier of the assessment engine will specify the minimum requirements for the memory and speed of computers being used by students for e-assessments, as well as the operating system needed. You should check the screen resolution and colour depth, as well as the keyboard and mouse for functionality, of all computers that will be used by students. No student should be put at any disadvantage, or given an unfair advantage, by the specification or the set-up of the equipment that they are using. Sufficient workstations are required to allow one networked computer per student, with some spare capacity. We suggest that one additional workstation for every ten students sitting the assessment would be sufficient to cater for most unforeseen difficulties.
A typical workstation needed for e-assessment would include:
It is advisable if possible to mirror the server in a separate location so that e-assessments can still go ahead if the primary server breaks down.
Servers should be backed-up daily, and it is particularly important to back up students' responses immediately after the assessment has taken place.
JISC RSC E-Responsibility Online Resource
JISC Regional Support Centres have brought together the best resources for raising awareness and planning e-safety which will support a range of roles. This toolkit has been put together as a national resource designed to be of use to develop strategy.
RSC UK - National e-Safety toolkit
BS ISO/IEC 23988:2007 Information technology - A code of practice for the use of information technology (IT) in the delivery of assessments - recommends that no one should spend longer than 1½ hours working continuously at a computer. This should not pose any real problem where NAB or college assessments are involved, although it is as well to be aware that the time any student spends on the computer may include:
The infrastructure requirements for large scale delivery of summative e-assessments, particularly high-stakes assessments, are considerable, and before an institution commits to such a delivery, adequate thought must be given to contingency planning. This can cover a host of contingencies – server problems, network problems, backup and data recovery issues, desktop issues, personal profile problems, logging problems, interference from other network services, and so forth.
When running an e-assessment session It is advisable to have spare workstations in case equipment fails (approximately one spare workstation per ten students). Students must be able to restart the assessment on another computer without loss of working time, and if possible to access any responses they have already made. In the event of system failure or network outage, responses should be saved so that students can restart from where they left off when the system has been restored. Timing should be reset accordingly.
In the event of fire alarms or other emergencies, it should be possible to vacate the room with browsers still open. When they return to the assessment, students should be given the full time for completion. Details of all emergencies and the implications for the assessment should be logged.
Invigilation is the supervision of an examination for the purpose of maintaining a fair and consistent testing environment. Even if a centre has a room designed for e-assessment, it may still be necessary to invigilate summative e-assessment sessions, and more than one member of staff may be needed to ensure the assessment is securely monitored and technical support is at hand.
The publication e-Assessment: Guide to effective practice includes information on the roles and responsibilities of those involved in e-assessment, including for invigilators monitoring e-testing.
Key areas of responsibility could include:
Some subject specialists feel the balance of time spent in staff development on e-assessment should be weighted in favour of pedagogy over software, because the skills needed to use e-assessment systems are relatively easy to develop, compared with developing skills and experience in the pedagogy of assessment writing.
Both staff and students need adequate preparation to use e-assessment. For students, this will largely focus on the need to familiarise themselves with the software interface, although if e-assessment is to be used within the context of a virtual learning environment, the initial induction may have to be more substantial and ongoing. This will also be the case with deploying software such as e-portfolios.
For staff, however, a similar process of familiarisation will be necessary, but in conjunction with further training in deployment, tracking and reporting processes associated with e-assessment. An entirely different and more complex kind of induction will be necessary where staff will be required to create e-assessments. The focus here will be as much on pedagogy as on technology.
While members of a Senior Management Team may find a briefing on e-assessment adequate, teaching staff who will be using the technologies are likely to need training and support. Support staff in libraries, and in administration and technical areas, may need specialised training in how to support those using the systems.
Technical assistance is needed both before and during e-assessment sessions at a number of levels. Maintenance of computer labs, networks and servers becomes much more critical when they are being used for an invigilated assessment. Also when running e-assessment sessions and when using practice e-assessments, it may be necessary to involve the librarian or other support staff. It is advisable that:
Key areas of responsibility within e-testing and e-portfolios contexts will vary. Responsibility for e-assessment may be taken forward by one individual or indeed by different people depending on the resources available.
Guidance on roles and responsibilities in e-assessment is available by using this link.
Although e-assessment benefits many candidates, the use of an e-assessment option can create an added demand on the technical skills of learners. In some circumstances, this can create an unnecessary barrier to the assessment of knowledge and learning. This will vary from individual to individual and from group to group, and with different software interfaces.
In general, requiring learners with low levels of IT skills and confidence to use an e-assessment interface, particularly one which is not user-friendly, may be inappropriate. One strategy to help remove this barrier is to use the interface initially for formative assessment, to build learner confidence and familiarity.
Learners who are required to use an e-assessment system will need to familiarise themselves with the software interface. If e-assessment is to be used within the context of a virtual learning environment, and/or e-portfolio, the induction may have to be more substantial and ongoing.
In order to ensure that the technology doesn't affect the outcome of any assessments, students must be familiar with the system, and comfortable with using it, before they use it for summative assessment. They should practise using the system, trying out the navigational aspects of the software and most of the question types - questions with a similar style, level of difficulty and method of feedback to those to be used in the actual assessment.
Students should be given Usernames and Passwords as required to carry out the practice assessments. Practice assessments can be conducted in the centre, either in timetabled subject time, for example during personal development time (PDT), or outwith timetabled time, such as in a library during a break. Alternatively, it may be possible for students to carry out practice assessments at home, if you feel this is appropriate, and the software supports it.
It's important that learners are made aware, at an early stage in the use of e-testing, that the marks provided in a summative online test will be automatically assigned by the computer. You may need to modify these marks and, once this has happened, the results can be returned to the learners. You should also make learners aware that, as happens with paper-based assessments, the results will be provisional at this stage, and could be subject to internal and external verification.
Some e-assessment is self-paced, particularly in the area of formative assessment. Where it is not, the planning of e-assessment, especially planning the timing and pacing of the e-assessment delivery, must be considered.
Largely, consideration of timing issues for e-assessment will be similar to considering timing for conventional assessment delivery, but with the caveat that learners may need to be given earlier opportunities to familiarise themselves with the software interface. Pacing can be a more particular consideration for certain assessment software packages, ie those which allow you to set time constraints on the assessments at the global or individual question level.
Although online assessment, particularly objective test type questioning, may appear to be less time consuming than conventional deliveries, learners need to be given adequate time to read and check questions and answers, and it is a common fault to underestimate the amount of time needed to complete such assessments.
Research has shown that learners taking assessments on computer require on average ten percent more time to complete the assessments compared to traditional methods. You should allow for this when you are planning the assessment.
JISC TechDis is a leading UK advisory service on technologies for inclusion.
They explore and promote inclusive practices, resources and advice for learning and teaching in UK higher education, further education and skills sector, and independent and specialist colleges.
The following link will take you to an overview of the resources and publications from JISC Techdis complete with signposting to their location on the new JISC Techdis website.
A Guide to JISC Techdis Resources
TechDis Tuesdays
TechDis Tuesdays are online fortnightly workshops organised by JISC where they:
For more information about TechDis Tuesdays, including topics for future sessions and to access recordings of previous sessions, visit the TechDis Tuesday page of the JISC website.
SQA has worked with CALL (Communication, Access, Literacy and Learning) Scotland to design digital question papers for learners who've been identified as having reading and/or writing difficulties, including learners who may sometimes require human readers or scribes. The digital papers are supplied in PDF format on CD. Learners can read the question paper 'on screen', using speech technology, where appropriate. Where the format of the question paper is a question/answer booklet, learners can use the computer to write their responses, or can 'speak' their responses.
Digital question papers from previous years are available for download from SQA by using this link.
CALL Scotland provides services and resources to assist with communication difficulties, particularly difficulties experienced in learning contexts. CALL Scotland can provide advice and training on creating digital question papers for use as prelims or for assessment generally.
Scotlands Colleges
Copies of the Profound and Complex Needs Newsletter are available in Latest Publications at Scotland's Colleges website by using the following link:
Some e-assessment activities will require supervision, such as taking an on-screen, closed-book test in which the learner is not allowed to access any additional resources. Other e-assessment activities may not need supervision; for example, creating a report for uploading to an e-portfolio for assessment.
See the following section on Authentication.
Authentication involves introducing and maintaining systems to establish the identities of learners and the ownership of the evidence they submit for assessment. Centres are responsible for ensuring the authenticity of this evidence.
Authentication is linked to assessors' knowledge of learners and their work; if assessors are familiar with a learner's level and pattern of work they should be able to spot differences or anything unusual. Assessors will often observe activities being carried out and know that evidence produced is learners' own work. However if evidence is produced without supervision, especially using resources acquired online, assessors may need to confirm that the evidence was generated by the learner, to ensure malpractice doesn't occur.
Centres should have authentication strategies incorporating combinations of tools and approaches such as:
When running e-assessment sessions designed to be held in controlled conditions, the following procedures are recommended:
Internal verification is the process used by centres to ensure that internal assessment has been carried out consistently within the centre and that national standards have been applied.
This involves checking assessment evidence and other assessment resources. Internal verification is checked as part of the External verification process.
If you use an online internal verification process to sample the assessment resources and record internal verification activity, this will allow more flexible approaches to external verification, such as E-verification.
Education Scotland
You may find the information on the Education Scotland website about Quality assurance and moderation useful.
Curriculum for Excellence
The following publication in the Curriculum for Excellence Building the Curriculum 5 - A Framework for Assessment series contains useful information about quality assurance and moderation.
Understanding, Applying and Sharing Standards in Assessment for Curriculum for Excellence: Quality Assurance and Moderation
External verification is the process used by SQA to ensure that internal assessment within and across centres has been carried out consistently and that national standards have been applied. It involves checking assessment evidence and other assessment resources, which can be in paper or electronic form; and also checking centres' Internal verification processes.
External verification is carried out by SQA External Verifiers by visit, by post or through central verification. Where a centre is assessing learner performance using electronic media, such as e-assessment, External Verifiers could verify either by visiting the centre or, in some circumstances, by accessing the electronic assessment resources remotely. This is sometimes called E-verification.
As with external verification of traditional assessments, verification of e-assessment may also require the centre to provide information about the assessment event, such as the location; conditions and arrangements; authentication of learners and their evidence; results and reporting.
SQA has produced a Guide to External Verification, which can be accessed by using the following link:
External Verification: A Guide for Centres
External Verifiers can in some circumstances verify by accessing a centre's electronic assessment systems and resources remotely. This is sometimes called e-verification. With a centre's agreement, External Verifiers could gain access to its systems online, from the External Verifier's home, awarding body premises or another mutually agreed location, for the purposes of carrying out e-verification.
Centres should have documented procedures detailing access arrangements for External Verifiers, including access arrangements to allow e-verification, where the centre has agreed to this. If access to the centre's system is not possible remotely, traditional external verification will be used.
SQA has produced guidelines for FE staff and external moderators on e-moderation. SQA e-Moderation: guidelines for FE staff and external moderators (2004)
They have also published Case Studies on the moderation of e-assessments. Moderation of e-assessment : Case studies (2006)
Verification of an e-assessment involves the inspection of various electronic documents, and possibly also paper documents. If the e-assessment is an online, automatically-marked test, these would include:
An external verifier carrying out an e-verification may need access to other documentation covering: the assessment event(s) - information relating to the location and conditions and arrangements used; the authentication of learners and the learners' evidence; results and reporting. To be valid as evidence for external verification, these documents should be date and time stamped.
How centres organise and deliver e-assessment will vary considerably, and the planning processes for this delivery must take into account existing protocols for conventional assessment. These may largely be capable of being applied as they stand to e-assessment delivery, but it is likely that new procedures will also need to be created – for example the processes for appeal of or review of e-assessment decisions.
Your centre or authority may already have an e-assessment policy. Once you've checked this, the first things to do are to analyse your reasons for introducing e-assessment, consider the curricular areas to be targeted, and decide whether your centre has the necessary organisational and technical infrastructure. You should explain your reasons to the staff who will be involved in e-assessment. Students, and (if it is necessary) their parents/guardians, should also be told why e-assessment is being introduced, and how it links with the other course activities.
Your centre or authority may already have an authentication/security policy. Ideally this should be a documented procedure on how the centre staff will check learners' authentication and on the strategies to be used to ensure the authenticity of learners' evidence.
You will find further information in the Authentication section of this web resource.
The four UK qualifications regulators, SQA, QCA, DELLS and CCEA liaise on an ongoing basis on the issue of e-assessment. Some of the UK's awarding bodies also contribute to these discussions.
In 2007 the regulatory bodies published Regulatory Principles for e-Assessment to guide awarding bodies and centres on the use of e-assessment, with the intention of encouraging innovative practice and enhancing the quality assurance of e-assessment.
SQA does not dictate the e-assessment systems centres use. However, as with traditional assessment approaches, we do require that the assessments are kept secure; that the storage of assessment materials is robust; that access to the assessments is restricted; that a system for authentication of candidates and their work is in place. If the external verification process determines that these requirements have not been met then the e-assessments may not be accepted.
SQA has drawn on Regulatory Principles for e-Assessment and other publications to produce criteria designed to advise SQA centres and SQA External Verifiers about the standards expected of e-assessment systems which are used to deliver SQA qualifications SQA Requirements for e-assessment.
SQA's requirements for e-portfolios provide guidance to ensure centres' e-portfolios are robust and fit for purpose.
The e-Assessment Toolkit was created on the basis of E-assessment: Guide to effective practice, a publication which was developed by the UK regulatory bodies.
This link will take you to the e-Assessment Toolkit.
Centres are advised to check with awarding bodies for guidance on their e-assessment policies.
JISC, Scotland's Colleges and the e-Assessment Association may also be able to advise.
By law, centres offering assessment for qualifications should take all reasonable steps not to discriminate against any student. If an e-assessment is not felt to be appropriate, an alternative or equivalent assessment, such as a paper version, should be provided.
You may find this paper from Education Scotland (formerly Learning & Teaching Scotland) to be of interest.
Focusing on inclusion and the Education (Additional Support for Learning) (Scotland) Act 2004
E-assessment systems should be compatible with different assistive technologies - screen readers, speech recognition software and touch screens, for example. More time should be allocated for students using such technologies, if it is needed.
For assessments where sound input or output is needed, students will have to be provided with headphones or be located in a separate room.
Images used in e-assessments should be provided with a textual equivalent. Students should be able to adjust the text size. Most systems allow the use of style sheets, enabling students to customise the appearance of the pages on the screen. Shortcut (access) keys should be made available as an alternative to the use of a mouse.
SQA's Assessment Arrangements
SQA has a section of its website detailing its Assessment Arrangements. This can be accessed by using the following link:
JISC TechDis
JISC TechDis has some resources relating to assessment policy. These can be accessed by using the following link:
You may find the following resources about accessibility and inclusion useful.
Interoperability is the term used to describe the ability of two or more systems to exchange and use information. A typical example might be where one e-portfolio system allows a learner to transfer some or all of the contents of his or her e-portfolio to another e-portfolio system, perhaps when moving on from education to employment. Technical standards are being developed to facilitate this process.
JISC CETIS provides advice on educational technology and interoperability for the Higher and Further Education sectors.
You may find this JISC CETIS paper to be of interest:
The Future of Interoperability and Standards in Education – System and Process (2010)
International Organization for Standardization
Standards exist that govern the delivery of online testing – BS ISO/IEC 23988:2007 Information technology
World Wide Web Consortium
The World Wide Web Consortium (W3C) is an international community that develops open standards to ensure the long-term growth of the Web. W3C is working to make the Web accessible to all users (despite differences in culture, education, ability, resources, and physical limitations).
There is a W3C Community Group that focuses on e-learning. Participants discuss new and existing technologies for e-learning and M-learning. The group also talks about the reach, social change and impact of e-learning. E-learning: Evolving technologies and growing reach
The report by John McClelland C.B.E. on his Review of ICT Infrastructure in the Public Sector in Scotland was published in June 2011.
The Scottish Government response to this Review includes a mention of the engagement that will be needed with all parts of the public sector and with the ICT industry to deliver the vision described in the McClelland report, in a section entitled 'Governance'.
Amongst their achievements, the Scottish Government note that they have delivered two major services to the education sector: Glow (the national learning intranet for Schools) and Interconnect 2.0 (the supporting high speed broadband network).
The Scottish Government has produced an Infrastructure Action Plan which outlines its commitment to a world-class, future proofed infrastructure that will deliver digital connectivity across the whole of Scotland by 2020.
Following a consultation exercise, SQA has re-launched its Vision and Strategy for E-assessment. First published in March 2005, this document aims to set out clearly SQA's priorities for this area of work and the likely implications for SQA and its various stakeholder groups.
SOLAR e-assessments
SOLAR is the SQA's online assessment tool that provides summative and formative e-assessments. It's based on banks of quality-assured question items. These e-assessments cover qualifications at all levels from National Certificate to Higher National Diploma.
The summative e-assessments are pre-verified — so centres know that quality is assured. This provides real benefits for both learners and learning providers.
The formative e-assessments are quality assured, and provide detailed feedback for learners. Learners can access the assessments 24 hours a day, 7 days a week using a browser and the Open Assess portal.
Each delivery of an e-assessment to a learner is dynamically generated by the system, based on pre-specified rules.
SOLAR Training
SQA has produced a range of interactive web-based training and support materials to enable centres to access and use the SOLAR e-assessment platform. The materials cover all of the main features of SOLAR administration and delivery.
To access these materials visit the SOLAR website and select Training Materials from the left hand menu. The materials are designed to inform on the key functions and activities within SOLAR.
If you require help with an area that is not listed, you can contact the SOLAR support team on firstname.lastname@example.com or call 0303 333 0330.
Information for New Centres
If your centre is an SQA approved centre but you are unsure if your centre has access to SOLAR, please visit the SOLAR website and check the List of approved centres trained to see if your centre already has a centre administrator for SOLAR. These administrators will be able to provide training and arrange for colleagues to access SOLARimmediately. (Some centres may have more than one person already trained).
If you think your centre does not yet have access to SOLAR, you can familiarise yourself with the SOLAR training materials and the functions of the e-assessment system before completing the training form provided. This should be submitted by email to email@example.com . The SOLAR team will then arrange access for your centre and email your login details.
To try a SOLAR formative assessment use this link.
If you intend to offer e-assessment for SQA qualifications using an online testing system, your centre will be responsible for ensuring that the system meets the necessary standards in terms of how assessments are developed, scheduled and delivered to learners and how results are managed, internally quality assured and reported.
SQA uses quality criteria for this, which we adapted from the Regulatory Principles for e-Assessment, devised by the UK Qualifications Regulators. SQA's version of the principles are designed to provide centres, technology providers and SQA External Verifiers with guidance on the standards required of non-SQA e-assessment systems which are used to deliver SQA qualifications.
You can download the criteria here - SQA Requirements for e-assessment (683 KB)
You may be able to find information on other online testing systems on the JISC and Scotland's Colleges websites.
VLEs typically offer both the capacity to delivery e-assessment content, created through stand-alone tools, and to create and track their own assessment items. A key feature in the VLE delivery of e-assessment is the ability to track, record and report on learners' assessment activity. Some VLEs may also be able to integrate assessment activity with learning – for example returning students to a piece of online learning if they do not achieve the hurdle of an end assessment.
If you intend to use VLEs for online testing see Preparation: What other things do I need to think about? Quality enhancement: Verification of online tests.
JISC infoNet
JISC infoNet aims to be the UK's leading advisory service for managers in the post-compulsory education sector promoting the effective strategic planning, implementation and management of information and learning technology.
They have developed an infokit on the Effective Use of VLEs.
Since many types of learning require an ongoing, portfolio approach to assessment it should be possible to build e-assessment opportunities using e-portfolio software. This technology is continuing to develop, and there are many types of portfolio software, which fulfil a wide range of differing functions.
When used for assessment, key things which portfolio software attempts to do are:
SQA does not recommend any one e-portfolio system to its centres.
There are many different approaches and we will work with whichever system centres choose to use:
SQA has developed e-portfolio requirements to provide guidance on choosing a system.
SQA has also published guidance on using e-portfolios. This resource will give you advice about good practice, and highlights some of the things you'll need to take into account when you're supporting learners as they create e-portfolios for SQA qualifications. The guidance also provides examples of e-portfolio forms and templates (51 KB) which could be useful for demonstrating good practice in using e-portfolios for assessment.
SQA DeskSpace allows candidates to record their course evidence online, keeping everything together and accessible to teachers/tutors and assessors, even when they are at more than one centre. Work can be uploaded and submitted online and the user friendly interface helps candidates structure their learning through the use of online templates.
The DeskSpace e-portfolio is available to all SQA Skills for Work and Baccalaureate candidates and is available to centres free of charge until June 2012.
SQA offers an endorsement service for organisations which provide resources to support the delivery of SQA qualifications and awards. This service also applies to e-portfolios.
The e-portfolio is evaluated against endorsement criteria and SQA's requirements for e-portfolios to ensure it is robust and fit for purpose.
You can find about the e-portfolios endorsed by SQA by using this link.
Two resources from JISC offering guidance on large-scale e-portfolio implementation are available by using this link.
The e-Portfolio Implementation Toolkit has been developed from the JISC- funded e-Portfolio Implementation (ePI) study of successful institution-wide practice in the UK, Australia and New Zealand. The toolkit contains over one hundred exemplars of e-portfolio use in a wide range of further and higher education contexts and eighteen case studies of approaches to wide-scale implementation. These form the basis of the Toolkit guidance that addresses the challenges, issues and stages in wide-scale e-portfolio implementation and presents an implementation model to assist your institution or programme in embedding and sustaining e-portfolio use.
Also available are five new video case studies, Stories of e-portfolio implementation, illustrating implementation strategies taken by five of the case study institutions. These rich-media resources provide additional personal insights into the challenges and benefits each institution has encountered. The videos are embedded into the Toolkit and can be downloaded or played by using this link. Transcripts are available for users of screen readers.
Both resources are made freely available, providing practitioners and managers in further and higher education with a unique insight into current UK and overseas e-portfolio practice. Feedback, thoughts and comments are welcome on these resources, and the facility to do so provided in the Toolkit itself. The resources can be used in conjunction with Effective Practice with e-Portfolios (JISC 2008) and the JISC e-Portfolios infoKit to provide a comprehensive package of support and guidance from JISC on e-portfolios in further and higher education.
Open source e-portfolios
Open source e-portfolio products are free for anyone to use, however centres would have to host and manage their own versions of these products.
One example is Mahara, an open-source e-portfolio that can be integrated into Moodle, allowing learners to transfer content from their learning environment, such as forum posts and assignments, to their e-portfolio. Mahara authenticates with Moodle, so learners signed into Moodle can access their e-portfolio without logging in to Mahara. Mahara has a range of partners who can supply hosting, training and development. There's also a Mahara User Guide.
In June 2011, Cumbernauld College won an award for the Immersive Use of Mahara for Creative Industry Courses.
If you're thinking about using a VLE, wiki or blog as an e-portfolio, consider developing an e-portfolio 'approach' with these. This means replicating the structures, templates and security features, generally found in e-portfolio products, to organise and present assessment evidence.
Adopting e-portfolio approaches means:
All of this should be available and easily located within, or linked to, the VLE, wiki or blog.
Web 2.0 technologies include social networking sites, blogs, wikis and video sharing sites and tend to focus on information sharing, user-centered design and collaboration.
Clearly there is a great deal of potential for using Web 2.0 technologies for assessment – for example to assess collaborative-type activities and group tasks, and for peer assessment.
You may find this Assessment 2.0 paper from SQA to be of interest.
Assessing an individual's contribution to a collaborative project can be difficult, however social software can help with this. In 2007/2008 SQA piloted the assessment of group work using a wiki and a blog. Learners were asked to work with two or three others to plan tasks, carry out research and then jointly develop a product. Each learner kept a record of the activities they completed as an individual, and shared this with the group through the blog. These records formed part of the learner's assessment evidence and helped them plan their evaluation reports.
CLASS (Collaborative Learning Assessed by Social Software) is a tool for learners studying for the Project-based National Course (PBNC) in Health and Safety in Care Settings. It offers them a blog and a wiki to record group activities and generate a group project.
The learners' contributions to the 'product' were stored in the wiki, which, by capturing the entries and amendments made by all members of the group, created an authenticated record of every learner's contribution to the project. Tutors were able to keep track of each individual's progress; the wiki showed progress with the project, at all times, while the blog provided information on ongoing activities (research etc) and a means of communication for the tutor. A secure login contributed to authenticating the learners' work. SQA Markers then marked the group project for each learner, using evidence from the learner's entries to the blog and the wiki.
The Report into the pilot study:
The CLASS Project:
The group wiki and blog software provided to students and teachers of the PBNC in Health and Safety in Care Settings is currently available for any other qualification. The eAssessment and Learning team is keen to extend the use of CLASS and will facilitate its adoption by any interested teams. If you are interested in learning more, please contact Veronica Harris or Tel: 0776 855 4004.
JISC has published a briefing paper aimed at policy makers, senior managers and practitioners who are interested in an overview of game-based learning and how it may be used to support effective learning and teaching practice.
Although this paper concentrates on game-based learning, it does contain information relevant to game-based assessment.
You may find the following links about mobile learning and assessment useful.
The Deskspace Mobile approach has now been trialled and tested and is available for download from the Add Ons section in the Deskspace Tools menu. The application is a simplified version of the e-portfolio with ability to send pictures and record evidence straight into your e-portfolio account. We hope this will help overcome difficulty in accessing suitable ICT equipment and time available to upload material to the e-portfolio. There are three formats available to suit a range of mobile phones.
Deskspace Mobile - Basic: A basic WAP enabled interface which is visible on all web enabled phones including an upload function.
Deskspace Mobile - PDA: This includes the functionality above but is designed to operate on Windows Mobile PDAs and Smartphones.
Pebblepad iPhone app: This is an approved app by Apple. However, this version is currently only available under the Pebblepad branding and costs £1.79 to download, but may be customised for Deskspace at a later stage.
'Assessment refers to measuring learner performance (either before or after a teaching intervention, or both). Thus, assessment can be part of an evaluation, but assessment and evaluation aren't synonymous.'
Educause - Measuring Success: Evaluation Strategies for Distance Education.
The article above suggests that, in order to evaluate the success of your (e-assessment) programme, you may first need to define what you mean by 'success'. Can success be measured in terms of higher uptake, for example because you're able to reach a wider group of learners using online assessment? Is it related to higher retention rates, better learner results or higher learner satisfaction? If e-assessment is to work effectively, meeting the needs of all users, including teaching professionals and support staff, as well as learners, each of these factors will probably be relevant. Therefore you may need a comprehensive and carefully planned evaluation plan.
Your evaluation plan could include regular surveys to capture information on learner results, staff views and technical and procedural issues. Gathering this information should allow you to monitor how well e-assessment does or doesn't contribute to supporting your centre's objectives, and so could also inform future plans.
You may be able to use your centre VLE to survey learners and centre staff, or one of the wide range of online survey tools available for creating questionnaires, delivering them and analyzing the responses.
Another potential evaluation tool is the 'Interview', which can be used instead of, or to supplement, a questionnaire. Whichever approach you use, you'll need to decide on the areas you wish to evaluate. The Educause article suggests that most areas of interest fall under inputs (resources, personnel etc), outcomes (performance, attitudes etc) or implementation issues. Subcomponents of the implementation issues, for example, on which to focus evaluative questions, could include:
This resource from Oklahoma State University (2002) offers advice on designing questions for both questionnaires and interviews -
As mentioned elsewhere, good e-testing systems will have Teacher Reporting facilities to allow you to monitor learner attainment in tests taken across different groups of learners and over time, as well as identifying which areas of the tests present problems for learners. These systems should be able to generate reports about the performance of individual learners, a class, or a number of classes.
Your centre will be involved at some level in coordinating, or participating in, internal and external quality assurance processes, designed to improving standards in the centre, and in education generally. These processes may require monitoring, tracking or benchmarking of learners' progress, and could take place, for example, through:
If learners are using e-assessment, you may be able to gather data from the recommendations, findings or comments resulting from these quality processes which could contribute to your evaluation of the e-assessment provision.
If you have good examples of e-test questions, or e-assessments which can be delivered via e-portfolios or social software which you think may be valuable for others to use, please contact Campbell White at SQA.
Use the following link to find out how you can contribute to Curriculum for Excellence resources in the NAR, for example by:
Contribute to Curriculum for Excellence resources in the NAR.
If you would like to contribute resources to JISC, the e-Assessment Association or Scotland's Colleges, which could in turn be linked to this e-assessment resource, please contact these organisations directly by using the links below.


text:
Aam Aadmi Party (AAP) leader Yogendra Yadav is set to shift base from Delhi to Gurgaon, the seat he will contest from in the coming Lok Sabha polls.
The AAP stragetist, who made a mark analysing elections, is likely to make the move to the NCR town in Haryana shortly. He is the AAP's in-charge of Haryana and has already started making regular visits to Gurgaon.
The AAP has decided to contest all 10 Lok Sabha seats in Haryana.
Yadav, who is away in Mumbai for an outreach programme of his party for industry captains, could not be reached for comment.
A close aide of the leader said, "No date has been fixed. Also, it is not yet decided whether he will move there alone or with family."
An Ahir, Yadav is a native of Saharanvas in neighbouring Rewari district which forms part of the Gurgaon Lok Sabha constituency.
In all likelihood, Yadav will take on Rao Inderjit Singh (BJP), a Rewari royal. Singh is also from the Ahir community.

subset: commoncrawl


text:
Product Features
*Available for US Shipment Only.
The Amoeba grip provides the most cushioning offered of all the SL grips thanks to nearly 75% of raised surface area contacting your hands. The HydraTak material is extremely durable, and becomes sticky when wet. Combine them to give you the newest handle innovation around.
Key Features of the Straight Line Amoeba-Tak Wakeboard Handle:
- Dip Molded End Cap: PVC dipped endcap, lightweight and pad printed with Straightline branding
- 4mm Dyneema Leader: Lightweight, Dyneema leader line, no stretch
- PVC Dyneema Tubing: PVC tubing, that covers Dyneema leader line to resist abrasion
- HydraTak Molded Oval Grip: Oval grip with EVA backed HydraTak grip material (exclusive to Straightline) hot- pressed with SL Amoeba grip patterns
- T6-6061 Aluminum Bar: No flex, no bend, lightweight bar extrusion with built in floats
All Gear by: Straight Line


text:
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]
st: Re: simultaneous equations with fixed effects and robust standard errors
Kit Baum <firstname.lastname@example.com>
st: Re: simultaneous equations with fixed effects and robust standard errors
Wed, 13 Jun 2007 06:38:43 -0400
No, your syntax is not correct. All endogenous variables and excluded instruments must appear in one parenthesized expression in ivreg, ivregress, xtivreg, ivreg2, xtivreg2:
xtivreg2 maturity (leverage my_binary_var=size profit) dividends year86-year05, fe robust
If you want to put all 6 binary variables into the same equation, you will obviously need more instruments. The equation as written above is exactly identified, and you cannot do tests of overidentifying restrictions.
Re the justification for this approach: if as I suggested you searched the Statalist archives, you would find
You do not need to use probit or logit to generate consistent estimates of the parameters of the equation as written above.
Kit Baum, Boston College Economics and DIW Berlin
An Introduction to Modern Econometrics Using Stata:
On Jun 13, 2007, at 2:33 AM, Duy Hinh Khieu wrote:
Thank you - Kit Baum - very much for your comments. I am not really
sure, however, of what you mean by standard IV estimators in the
context of my model, which is mixed with one continuous endog var
and 6 binary endog variables. If I use the xtivreg2 as follows, can
you tell me if I translate your comments correctly into STATA code:
xtivreg2 maturity (leverage=size profit) (my_binary_var=size profit ) dividends year86-year05, fe robust endog( leverage my_binary_var)
I would need to do the bracket part of the code that contains my_binary_var six times because I have six binary endog variables and include them all in the above xtivreg2?
The reason why I need to use probit or logit is not because I am concerned about a consistent estimator, but because I need to do a two-stage estimates and the first stage involves a binary variable. Am I right?
If you have a source that states that the first stage estimation does not need to involve a probit or logit, I'd really appreciate your giving me that reference. All I have is Maddala (1983), pages 243-245.
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
|© Copyright 1996–2017 StataCorp LLC | Terms of use | Privacy | Contact us | What's new | Site index


text:
Zack Snyderĺs ôMan of Steelö might be a miracle, and I know Iĺm treading on the thin ice of hyperbole with that statement. The director, who graduated to feature films with his successful ôDawn of the Deadö remake nearly a decade ago, has often been a whipping post for critics (including this one) who failed to differentiate between the subjects of his films (including, but not limited to, zombies, mutants, omnipotent beings and scantily-clad warrior women) and their subversive themes, often commenting on, if not decimating altogether, the cultural values being approximated. 2011ĺs ôSucker Punchö was the zenith of Snyderĺs auto-critique, employing the sexism of a fanboy culture as a pro-feminist tool in the action movie mold, so brazenly biting the proverbial hand that feeds that the generally reviled attitude toward it should itself count as high praise.
While genuine arguments might yet be had on the merits of Snyderĺs earlier films (ô300,ö in particular, suffers from some of the source materialĺs unchecked fascism), anyone who fails to recognize his artistry after ôMan of Steelö is either blind or unrepentantly biased. His intuitive understanding of genre in form and content made him ideal for this reboot, which, under the guidance of producer Christopher Nolan, hews towards the realism employed in his recent Batman films, but remains firmly indebted to Snyderĺs distinct flair, here able to roam freely within the Superman universe (one that gets a few changes that might upset purists), best exemplified in the filmĺs extended opening on Krypton. At once a greatest hits of science fiction influences (the production designs of ôThe Matrix,ö ôAlien,ö and ôBlade Runner,ö amongst others, are all echoed) and an incredible act of world-building, the alien homeworld of Clark Kent is both creepy and beautiful, and the apocalyptic destruction of the planet is but one of many indelible images to follow.
Where ôMan of Steelö proves miraculous is in its balance of so many seemingly incompatible elements. Viewers have compared the filmĺs style to that of both Michael Bay (citywide destruction, a cacophony of chaos, rapid editing) and Terrence Malick (ôartsyö compositions, fuzzy foregrounds against focused backgrounds, etc.), but the reality is more than just a fusion of two otherwise opposing styles. Snyderĺs approach eschews camp and lends Superman a genuinely fantastic, spiritual heft (the Christ pose is brief but potent in a stunning aerial shot, and the number 33 is relevant herein, too), one that grounds the ruthless destruction to come and elevates it above mere popcorn spectacle. Closer in tone to Steven Spielbergĺs ôWar of the Worldsö than this summerĺs ôStar Trek Into Darkness,ö ôMan of Steelö earns its portentousness, which is not to say the film is without its fun (a favorite gag: an early test of Supermanĺs power of flight misfiring and destroying the top of a mountain).
The cast, then, rises to the occasion, selling these larger-than-life characters with an approach that suggests behaving more than simply acting. The surrounding circumstances donĺt allow much time for romance but the immediate chemistry between Henry Cavill (Clark Kent/Superman) and Lois Lane (Amy Adams) is palpable. As the tragic antagonist General Zod, Michael Shannon lends significant complexity to suggest the good intentions of his motivations, and brings his expert lunacy without disengaging from the filmĺs rough-hewn realism. Arguably best, though, is Kevin Costner as Clarkĺs earthly father, Jonathan Kent, evoking the classic nobility of Gary Cooper or John Wayne as he guides his unlikely son through a tumultuous adolescence. There isnĺt a weak link herein, and ôMan of Steelö rests high on the short list of great superhero films.
Robert Humanick is a contributing writer for Slant Magazine.