text:
4.3 Definitions The purpose of this section of the paper is not to settle upon specific definitions but rather to elucidate the different emphases associated with the varying use of basic terms, with a view to exploring the prospects for shared understandings of policy intent and possible areas of common ground. Regrettably, there is little consistency, let alone consensus, over the meaning of even the most commonly used terms. Sometimes, the same term is used in relation to different factors, or different terms are used to describe similar factors; e.g. 'outputs' and 'outcomes' can be used interchangeably. At other times, different meanings derive from different contexts; 'competence' and 'employability' have different meanings in the European context than in the English or Australian contexts (Brockmann et al., 2008). And at times, definitions are constructed with a hint of sophistry and with a view to directing change according to the preferences of the defining agency; e.g. outcomes may be defined as 'direct' measures of achieved learning and contrasted with 'proxy' measures of graduate success such as employment and income consequences (Nusche, 2008), even though the direct measures are themselves proxies, such that we get 'primary' and 'secondary' proxies: "Outcomes of higher education are not limited to learning outcomes. Students can benefit from their HEI experience in many different ways, such as better social status, higher employment rates, civic engagement, opportunities to pursue further studies, or simply leading a more fulfilled life (Ewell, 2005). While such outcomes are related to learning, they should not be confused with the actual mastery of knowledge, abilities, and skills that result from students' engagement in HEI learning experiences (Ewell, 2005). Such long- The ambiguous use of term social and economic benefits of the HEI experience can serve basic terms relating to as secondary proxies for learning outcomes, but they are not direct the accountability for outcomes of learning" (Nusche, 2008). quality agenda in higher In a similar vein, Shavelson, one of the developers of the Collegiate Learning Assessment instrument, focuses on 'direct' rather than education sets off alarm 'indirect' measures of learning, because the former relate to "actual bells, but curiously they learning as a relatively permanent change in observed behavior are not resonating. over a period of time" (Shavelson, 2010). Some may regard quibbling over definitions as indulgent. But clarity of policy intent requires clarity of definition. Ambiguity in the use of terms may reflect complexity but it may also permit permissiveness and licence authoritarianism. The ambiguous use of basic terms relating to the accountability for quality agenda in higher education sets off alarm bells, but curiously they are not resonating. One wonders why. 4.3.1 Inputs, processes, outputs and outcomes In a background paper for the OECD's AHELO project, Nusche (2008) offers the following definitions: Inputs are the financial, human and material resources used, such as funding and endowments, faculty and administration, buildings and equipment. Processes (or Activities) are actions taken or work performed through which inputs are mobilized to produce specific outputs. Examples of higher education activities include curriculum design and teaching. THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 126 Outputs are anything that an institution or system produces, e.g. articles published, classes taught, educational material distributed, and degrees awarded. Nusche does not include graduates in the category of outputs, although she does include 'degrees awarded' (by implication the people to whom they are awarded). If one regards graduates as outputs then outcomes may be seen as the benefits that graduates obtain from their achievement, whether employment, income, and wellbeing, as well as the contributions that graduates make to society. The question of how to classify graduates in an educational 'system' is complicated by the fact that students are inputs and co-producers, learning is an interactive experience, and graduates are people who, because they learn, cannot be neatly or normatively defined. Nusche treats outcomes separately, distinguishing between intent and actuality: "Outcomes describe what the student actually achieves as opposed to what the institution intends to teach" (Allan, 1996). She goes further to suggest that "Inputs, activities and outputs have little intrinsic value in terms of student learning. They are only the intermediate steps that may or may not lead to outcomes or benefits" (Nusche, 2008). But what accounts for learning in this view, or doesn't that matter? If learning is understood as an independent variable why bother with teaching? There is a basic flaw in the logic for understanding education. Nusche (2008) appears to confuse differences between actuality and intent, which result from inappropriate or ineffective processes, with differences between cause and effect. Given Nusche is setting the scene for testing direct learning outcomes across different contexts, it may be understandable that she narrows the scope through definitions in order to focus on student achievement. Thus, borrowing from the pioneers of the controversial model of 'outcomes-based education', and focusing on the summative rather than the formative purposes of assessment, Nusche takes a behaviourist approach, echoing Shavelson (2010): "In behavioural terms, learning outcomes have been defined as something that can be observed, demonstrated and measured" (Nusche, 2008): "Outcomes are clear, observable demonstrations of student learning that occur after a significant set of learning experiences… Typically, these demonstrations or performances reflect three things: (1) what the student knows; (2) what the student can actually do with what he or she knows; and (3) the student's confidence and motivation in carrying out the demonstration. A well-defined outcome will have clearly defined content or concepts and be demonstrated through a well-defined process beginning with a directive or request such as 'explain', 'organize', or 'produce'" (Spady & Marshall, 1994). By a different logic, one that acknowledges the complex interactions involved in learning but looks to a measurable end effect, a similarly reductionist approach has been adopted by AUQA in its 2009 discussion paper on measuring and monitoring academic standards. Here the argument is analogous to the cement mixer whose inner working are not readily observable, but the strength of the mix can be tested once poured: "A large number of important variables influence how well students achieve. These include: student backgrounds; students' knowledge and skills on entry to a course; the design of individual courses and degree programs; how much effort students make; institutional resourcing levels for teaching; and the quality of teaching. Gathering data about and evaluating these types of input and process variables is a very valuable exercise, particularly for each institution's own continuous improvement, but limiting the scope of quality assurance procedures strictly to these cannot substitute for a direct focus on achievement itself. Primarily, this is because the various inputs and processes interact in complex ways, and are not deterministic. An explicit focus on academic achievement, however, examines the net learning effect of all the variables operating together. It thus serves two purposes. It allows the attained level of achievement to be assessed and recorded (as grades on student transcripts, for instance), and it allows evaluation of how well the teaching and learning system is working" (Woodhouse & Stella, 2009). THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 127 These approaches of Nusche (2008) and Woodhouse & Stella (2009) can be seen to share a positivist view which (falsely) represents social reality as existing objectively and independently of those whose action and work actually produces the conditions observed (Horkheimer, 1937). Additionally, they reduce the notion of learning to 'academic achievement'. The AUQA approach is particularly narrow, with its focus on cognitive achievement. Nusche takes a wider taxonomical approach, including cognitive and non-cognitive learning outcomes (see Box 35). Of particular note is her exposition of the possibilities for assessing generic skills independently of knowledge and learning contexts. Importantly, domain knowledge and domain-specific skills are not readily transferable. Box 35. Cognitive and Non-cognitive learning outcomes Cognitive outcomes Knowledge outcomes General content knowledge refers to the knowledge of a certain core curriculum whose content is considered essential learning. Domain-specific, or subject-specific, knowledge outcomes refer to acquired knowledge in a particular field, such as biology or literature. Assessments focusing on domain-specific knowledge outcomes are particularly useful to compare learning quality in a particular field across different institutions. Skills outcomes Cognitive skills are based on complex processes of thinking, such as verbal and quantitative reasoning, information processing, comprehension, analytic operations, critical thinking, problem-solving and evaluation of new ideas. There is some disagreement as to whether such thinking processes are generic (following general patterns) as opposed to being field-specific. Assessments aiming to compare learning outcomes across different courses often focus on generic skills outcomes. Generic skills. The common characteristic of all generic skills outcomes is that they transcend disciplines. They are transferable between different subject areas and contextual situations. Such skills are not directly tied to particular courses. They relate to any and all disciplines and they allow students to be operational in a number of new contextual situations (Pascarella and Terenzini, 2005). Generic skills outcomes can be assessed using tests that are based on application rather than on knowledge, thus focusing on students. ability to solve intellectual problems. Usually, students are asked to provide constructed answers that also give evidence of writing skills. Focusing on outcomes in terms of skills may allow comparing how well programmes and institutions with diverging missions and ways of teaching achieve to develop certain common skill dimensions in students. Yet, there are some doubts as to whether such outcomes can really be connected to the university experience. Domain-specific skills are the thinking patterns used within a broad disciplinary domain, such as natural sciences or humanities. They are stated in terms of methods of enquiry, ways of evaluating evidence, and patterns of procedure necessary to confront new contextual situations in specific fields of study. They involve an understanding of how, why, and when certain knowledge applies. Domain-specific skills are not entirely transferable throughout subject areas. Non-cognitive outcomes Non-cognitive development refers to changes in beliefs or the development of certain values. Psychosocial development includes aspects of self-development such as identity development and self-esteem, as well as relational developments such as students' relationships with people, institutions and conditions. Relational outcomes include interpersonal and intercultural skills, as well as autonomy and Attitudes and values. Nusche, 2008. THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 128 Outcomes-based education (OBE) OBE has many variants (e.g. mastery learning, performance-based education) but generally refers to a student-centred learning philosophy that focuses on measuring student performance (outcomes), in contrast with traditional education, which focuses on the resources (inputs) available to the student. OBE does not specify or require any particular style of teaching or learning. Instead, it requires that students demonstrate that they have learned the required skills and content. In practice, such as in secondary schools in Western Australia, OBE promotes curricula and assessment based on constructivist methods and discourages approaches based on direct instruction methods and preferencing of classic texts. However, the terminology can be used in a less extensive and prescriptive way. For example, the University of Western Australia has distilled its approach in the following way: "A Student Learning Outcomes approach focuses on student learning by: 1. Using learning outcome statements to make explicit what the student is expected to be able to know, understand or do; 2. Providing learning activities which will help the student to reach these outcomes; 3. Assessing the extent to which the student meets these outcomes through the use of explicit assessment criteria" (Centre for the Advancement of Teaching and Learning, UWA, 2009). OBE is directed at improving student achievement and focuses, therefore, on formative assessment. Tensions arise when the approach is adopted for purposes of external accountability with its focus on summative assessment. As the OECD has noted, from the perspective of tertiary education systems as a whole, both the purposes of accountability and improvement are essential; "the difficulty lies in combining them in the design of a quality assurance framework and its implementation" (Santiago et al., 2008). A starting point in reconciling the dual purposes is to recognise that learning outcomes are more than test scores, and that the choice of proxy measures matter: "Accountability must be inferred from observing outcomes in any system where all actions cannot be observed directly. To do this 'inferencing' the performance measure is an indicator of the desired behavior, not the behavior itself. In business, there is a clear outcome measure (revenue or stock price) to guide business decisions and actions. You can't manage a business if you can't measure its outcome. In education, outcomes are many and debated. The outcome indicator—most often a multiple-choice achievement test, is but a proxy for the desired outcome. When this indicator becomes an end in itself, and it does in education, well-intentioned accountability may very well distort the system it was intended to improve" (Shavelson, 2009). Thus one can only find empty the approach of Nusche (2008) and Woodhouse & Stella (2009) in seeking to gauge the effectiveness of education only by reference to summative measures. 4.3.2 Quality, quality assurance, quality enhancement and quality evaluation Quality is a subjective view of the properties that distinguish an object. Harvey & Green (1993) identified five sets of meanings …mission-related criteria attaching to quality in higher education (see Box 36). Of particular for quality remain note is that the fitness-for-purpose criterion turns from a demand- powerful in a student side customer requirement to a supply-side provider mission. The key inference is that mission-related criteria for quality remain demand driven system. powerful in a student demand driven system. This point contrasts with the ill-considered view that fitness for purpose is a less relevant criterion in a diverse student driven system. THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 129 Box 36. Definitions of quality in higher education "The exceptional view [of quality] sees quality as something special. Traditionally, quality refers to something distinctive and élitist, and, in educational terms is linked to notions of excellence, of 'high quality' unattainable by most. Quality as perfection sees quality as a consistent or flawless outcome. In a sense it 'democratises' the notion of quality and if consistency can be achieved then quality can be attained by all. Quality as fitness for purpose sees quality in terms of fulfilling a customer's requirements, needs or desires. Theoretically, the customer specifies requirements. In education, fitness for purpose is usually based on the ability of an institution to fulfil its mission or a programme of study to fulfil its aims. Quality as value for money sees quality in terms of return on investment. If the same outcome can be achieved at a lower cost, or a better outcome can be achieved at the same cost, then the 'customer' has a quality product or service. The growing tendency for governments to require accountability from higher education reflects a value-for-money approach. Increasingly students require value-for-money for the increasing cost to them of higher education. Quality as transformation is a classic notion of quality that sees it in terms of change from one state to another. In educational terms, transformation refers to the enhancement and empowerment of students or the development of new knowledge." Harvey, 1995. The following definitions of academic quality, quality assurance in higher education, and quality enhancement, are taken from the UK's Quality Assurance Agency for Higher Education (QAA, 2006): Academic quality Academic quality is a way of describing how well the learning opportunities available to students help them to achieve their award. It is about making sure that appropriate and effective teaching, support, assessment and learning opportunities are provided for them. Quality assurance (QA) Quality assurance refers to a range of review procedures designed to safeguard academic standards and promote learning opportunities for students of acceptable quality. There are various interpretations of what exactly constitutes acceptable quality: e.g., an institution's provision should be "fit for purpose"; should make effective use of resources; should offer its stakeholders value for money; etc…but it is increasingly agreed that it is important to promote improvement of quality, not just to ensure that quality is maintained. This shifts the emphasis from quality assurance to quality enhancement. Quality enhancement (QE) Quality enhancement is taking deliberate steps to bring about continual improvement in the effectiveness of the learning experience of students. These are useful working definitions, and the policy intention to emphasise enhancement is compelling. Educational quality evaluation A rounded approach to the evaluation of higher education quality has been advanced by Scott (2008) in a research and analysis brief prepared for the 2008 review of Australian higher education (Scott, 2008). He defines quality with reference to judgements which can be made about the design, support, delivery, and impact of a program. Judgements of quality can be about: THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 130 1. the relevance and desirability (fitness-of-purpose), feasibility, and fitness-for-purpose of a learning program's design; 2. the support and infrastructure put in place to enable its delivery; 3. the implementation of the program, e.g. evidence that that the planned course and its support systems are being put into practice in the way intended and to the satisfaction of both the students and teaching staff involved; 4. the impact of the program, e.g. evidence of high quality performance on valid, reliably marked assessment items; positive performance on proxy measures of impact including employability, graduate salaries, employer satisfaction with graduates, successful further study, etc. Scott's approach generates the range of information necessary for making balanced judgements. It locates 'impact' (effectiveness and benefit) in the context of program purpose. It contrasts with the view that impact can be meaningfully assessed without reference to the purpose and context of learning. 4.3.3 Qualifications and Qualifications Frameworks Qualification A broad descriptive definition of a qualification is offered by the OECD: "A qualification is achieved when a competent body determines that an individual has learned knowledge, skills and/or wider competences to specified standards. The standard of learning is confirmed by means of an assessment process or the successful completion of a course of study. Learning and assessment for a qualification can take place during a programme of study and/or workplace experience. A qualification confers official recognition of value in the labour market and in further education and training. A qualification can be a legal entitlement to practise a trade" (OECD, 2007). A narrower description is offered by Tuck (2007): "A qualification is a package of standards or units judged to be worthy of formal recognition in a certificate": 'Standards' in this context = "a set of information about outcomes of learning against which learners' performance can be judged in an assessment process". 'Units' in this context = "A coherent set of standards which form a short, unified program of learning". A deeper understanding of the role of qualifications is indicated by Keating (2008): "Qualifications have been designed to discriminate. They concentrate upon individuals and they testify to knowledge, skills, attributes and experiences that are not shared by all. They do have social attributes. However, the collective attributes are essentially communal where qualifications play the role of gatekeeper for entry into occupations or alumni". Qualifications thereby function as passports for learner mobility in labour markets and contexts for further learning. Qualifications Framework Considerable diversity in qualifications frameworks is reflected in the OECD's definition. It allows for a range of practices, and does not suggest that one form of practice is better or worse than another: "An instrument for the development and classification of qualifications according to a set of criteria for levels of learning achieved. This set of criteria may be implicit in the qualifications descriptors themselves or made explicit in the form of a set of level descriptors. The scope of frameworks may be THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 131 comprehensive of all learning achievement and pathways, or may be confined to a particular sector, for example initial education, adult education and training or an occupational area. Some frameworks may have more design elements and a tighter structure than others; some may have a legal basis whereas others represent a consensus of views of social partners. All qualifications frameworks, however, establish a basis for improving the quality, accessibility, linkages and public or labour market recognition of qualifications within a country and internationally" (OECD, 2006). This matter is discussed at 4.4 below. 4.3.4 Standards Of the thirty or so dictionary meanings of a 'standard', the following may be pertinent to the current discussion: anything taken by general consent as a basis of comparison; serving as a basis of value, comparison or judgement; an approved model for imitation; a measure to which others conform or by which the accuracy or quality of others is judged; a grade or level of achievement; a level of quality which is regarded as normal, …'standard' can adequate or acceptable; degree of excellence required for a connote 'normal' (i.e. particular purpose; a document specifying (inter)nationally agreed properties for manufactured goods etc. undistinguished), Thus 'standard' can connote 'normal' (i.e. undistinguished), 'acceptable' (i.e. fit for 'acceptable' (i.e. fit for purpose), 'model' (i.e. worthy of imitation) or, purpose), 'model' (i.e. more neutrally, an agreed set of properties to be used for making worthy of imitation) comparisons. In its neutral sense, a standard is a criterion, and a set of standards comprise criteria or benchmarks for making or, more neutrally, an comparative judgements, such as in assessing performance. Higher agreed set of properties education standards, then, can be defined simply as 'criteria for the to be used for making assessment of capacity and performance'. However, much depends on who sets the standards, the criteria they select and the levels at comparisons. which they set them (e.g. whether they are 'minimum acceptable standards' or 'threshold standards, or 'typical standards' or 'high standards' or 'aspirational standards'). Standards setting is contested ground, and the most contested area is that of academic standards. Academic standards can include curriculum standards, learning resource standards, pedagogical standards, assessment standards, and achievement standards. These different standards need to be integrated within an institutional context and purpose. If they are treated separately they can conflict: "It needs to be acknowledged that there is an important tension between pedagogical standards and achievement standards. The highest standards of pedagogy hold that the level of expected student academic achievement should be matched to the background and current level of knowledge of the particular students. Expecting an inappropriately high level of academic achievement for a group of students would not be regarded as good teaching practice and would not be judged as meeting a high standard of pedagogy. Thus, if one focused not on student academic achievement but on teaching as the focus of academic standards one would make very different assessment of academic quality" (Dearn, 2009). In Britain, the focus of higher education quality assurance is on standards of student achievement (learning): "Academic standards are a way of describing the level of achievement that a student has to reach to gain an academic award (for example, a degree)" (QAA, 2006). Key questions, which are under present debate, include: who should set them, in what contexts, at what levels, and to what extent should they be common? THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 132 According to one view, standards are purpose-related, and can only be meaningfully set with reference to the nature and purpose of educational provision: standards are "criteria established by an educational institution to determine levels of student achievement" (education.com). This view reflects the necessary integration of student achievement standards and pedagogical standards at the institutional or program level (Dearn, 2009). According to another view, academic achievement standards are necessarily based in disciplinary contexts and are essentially dynamic, and while they may be set externally to an educational institution they can only be determined by academic communities: "We use 'standards' to refer to the nature and levels of learning outcomes that students are expected to demonstrate in their university studies. This places the onus for setting and monitoring standards squarely with academics and academic communities within fields of study and disciplines. Standards are neither absolute nor timeless; standards are continually being re-defined and created as knowledge grows in existing fields and as new fields emerge" (James, McInnis & Devlin, 2002). Van Damme (2003) even goes so far as to suggest that there can be no fixed standards, since quality depends on its relationship to the internal purposes of a program or the external expectations of consumers and stakeholders (cited in Hämäläinen, 2003). Yet another view sees academic achievement standards as fixed, once they have been pre-set by academics and other stakeholders: "An academic achievement standard is: • an agreed specification or other criterion, • used as a rule, guideline or definition, • of a level of performance or achievement. This definition has two key features. First, a standard refers to a level that is preset and fixed. After that, it remains stable under use unless there are good reasons for resetting it. In higher education this would mean that the standards are not reset for each cohort of students, or for each assessment task. An academic standard is therefore a big-picture concept that stands somewhat apart from particular assessment tasks and student responses. Second, agreement on the specification must be by authority, custom, or consensus, as standards are not private matters dependent on individuals but collegial understandings …it is not the officials shared among academics and other stakeholders" (Woodhouse & but the athletes who Stella, 2009). achieve the heights of Are these different views reconcilable? Pre-set and 'fixed' standards may be applicable in relation to learning generic skills, which as performance and set the discussed above, are regarded as being knowledge and context standards of excellence. independent. They can be seen to be fixed in that they express criteria that need to be satisfied by an individual learner in order to 'pass' a course, irrespective of the performance of other students in a class at a particular time. The concept of 'standard' as a pre-determined and fixed basis against which the capacity and performance of institutions, programs or graduates can be judged is useful in appreciating the difference between vernacular claims to 'slippage in standards' and demonstrable differences between institutional or individual performances with reference to set standards. But it is a complacent view. Performance may slip from time to time, relative to standards set previously, but new standards can be set by superior performance. To use a sports analogy—high-jumping—a standard will fall only if an official deliberately lowers the bar for some reason. Normally competitor performances keep the bar rising. Importantly, it is not the officials but the athletes who achieve the heights of performance and set the standards of excellence. THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 133 A standard set as the basis for a national higher education system can be only the minimal acceptable quality permitted; it is the provider qualifying criterion, the foundation on which institutions can perform at thehigher standards they set for themselves. The adjectives 'same' and 'common' can be applied validly with regard to this pre-set and fixed standard because it is prescribed for all as a minimum. An institution cannot be licensed if it cannot meet the prescribed conditions and continue to perform at least at the defined level of acceptability. However, performance above the pre-set standard is not expected to be the same for all, because some will excel more than others (have a higher degree of quality) and in different ways for different purposes (exhibit different quality characteristics). In the case of individuals, as well as having different prior attainment and background circumstances, students have different purposes, some keen to pursue a special interests, some curious to taste the unfamiliar, some "developmental" and others "instrumental" in their orientation to learning (Brown, 2007). In the case of institutions, as well as having differences in physical and other circumstances, and differences in talent, universities (as one category of institutional types within which there is much diversity) have varying missions, some focused primarily on the preparation of graduates for professional employment, others focused more intensively on knowledge breakthroughs, perhaps with an interest in the development of rounded graduates. For courses leading to entry to professional occupations, there may well be common areas for learning, and even common expectations of graduate capabilities. Similarities may be evident in the curriculum of cognate fields across different institutions. But common and similar coverage does not equate to sameness of provision, as there can be different orientations and methods chosen by different providers. If we focus on 'standards-based education' as a derivative of criterion-referenced learning ('mastery learning') and assessment, standards can be understood as references which guide curriculum objectives, the design and organisation of learning experiences, and related forms of assessment. Standards-based education (see Box 37) is an outcome of the failed 'outcomes-based education' approach abandoned in the US in the 1990s and in Australia in the 2000s. It involves clear, measurable standards for all students and usually involves • the creation of curriculum frameworks which outline specific knowledge or skills which students must acquire, • an emphasis on criterion-referenced assessments which are aligned to the frameworks, and • the imposition of some high-stakes tests, such as graduation examinations requiring a high standard of performance to receive a diploma (http://en.academic.ru). Box 37. Standards-based education (SBE) in Colorado Standards-based education in Colorado is defined as an ongoing teaching/learning cycle that ensures all students learn and can demonstrate proficiency in their district's adopted content standards and associated benchmark concepts and skills. This teaching/learning cycle frequently measures student achievement through a variety of formats and assessments and ensures multiple opportunities for students to learn until they reach a proficient or advanced level of performance. Regardless of content, course, level, identified outcomes or revisions in standards, this teaching/learning cycle remains constant. A. Standards in all academic disciplines or content areas, along with benchmark information, concepts and skills, are identified and adopted at the district level. B. Essential benchmark information, concepts and skills expected for all students are identified and described. (These may also be called essential learnings, learning targets, power standards, objectives or grade-level expectations.) C. Essential benchmarks are articulated and aligned within and among grade levels and across the district to ensure there are no gaps or unnecessary overlaps in those expected learnings. THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 134 D. Adopted curricula provide a scope and a sequence of essential benchmarks (sometimes called curriculum objectives or targets) that engage students in learning standards in all content areas. E. Curriculum guides (frameworks), maps, pacing guides and other curricular tools are produced at the district level to assist teachers to plan effective instruction that focuses on essential benchmark knowledge, concepts and skills. F. Descriptions of proficiency are created to describe the types and levels of performance expected for all essential benchmarks in all content areas and grade levels. G. Examples of proficient student work are created and distributed to teachers to provide models of learning and performance expectations for all essential benchmarks. H. Adopted or purchased instructional programs and materials are intentionally articulated and aligned with standards-based curricula. I. Standards and benchmarks are communicated effectively to students and parents. Students understand and can describe proficient performance for those concepts and skills. Benson, 2008. Externally-developed statements of standards can inform institutional decisions about curriculum design, teaching and assessment but they cannot determine them entirely. In criterion-referenced education, standards have to be integrated in the context of learning to fit the needs and abilities of learners. Similarly, the results of collegial discussion in the academy on expectations of learning outcomes in particular disciplines "Collegial processes of (e.g. Tuning, Subject Benchmark Statements, ALTC Benchmarks for teaching and learning quality assurance), may serve as helpful debate about academic references for program design but they can be no more than standards do not references: necessarily lead to totally "Collegial processes of debate about academic standards do not common understandings necessarily lead to totally common understandings about what about what the minimum the minimum or base expectations are; nor should they. They often quite validly lead to differences which result in innovation and or base expectations are; progression for curriculum, assessment and value adding diversity of nor should they." graduate outcomes" (ATN, 2009). On balance, externally-developed standards, beyond the threshold of acceptability for operational licensing, have a limited role, primarily as references against which internal decisions can be made about educational objectives, curriculum design and assessment: "Quality evaluation should not be exclusively focused on assessing institutions within a standardised and externally defined framework, but should see the capacity of institutions to stand out through innovation and individual and institutional creativity" (Teixeira, 2010). So what is meant by "outcomes and standards-based arrangements" (Bradley et al., 2008) in respect of higher education in Australia? A working model of standards-based arrangements can be found in relation to the National Code relating to the provision of education services for international students. Providers must be registered on the Commonwealth Register of Institutions and Courses for Overseas Students (CRICOS) as a condition of their students being able to get a visa to study in Australia. CRICOS-registered providers must comply with 15 standards that ensure their quality of education and professionalism is of a sufficiently high standard to enrol international students. These education providers must demonstrate their compliance with the standards at the point of CRICOS registration and throughout their CRICOS registration period. THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 135 Each Standard in Part D is linked to the National Code 2007 Explanatory Guide. The 15 standards cover the following aspects of delivery of education to international students: Standard 1 Marketing Information and Practices Standard 2 Student Engagement Before Enrolment Standard 3 Formalisation of Enrolment Standard 4 Education Agents Standard 5 Younger Overseas Students Standard 6 Student Support Services Standard 7 Transfer Between Registered Providers Standard 8 Complaints and Appeals Standard 9 Completion Within Expected Duration Standard 10 Monitoring Course Progress Standard 11 Monitoring Attendance Standard 12 Course Credit Standard 13 Deferment, Suspension or Cancellation of Study During Enrolment Standard 14 Staff Capability, Educational Resources and Premises Standard 15 Changes to Registered Providers' Ownership or Management By way of illustration, Standard 14 ensures providers have suitable staff, educational resources and premises to educate overseas students. The provision of staff and services are to accord with existing quality assurance frameworks that apply to the course or, where none exist, providers must have appropriate policies and procedures of their own. Key requirements • The staff of registered providers are suitably qualified or experienced in relation to the functions they perform for students. • The educational resources of registered providers support the appropriate delivery of courses to students. • The suitability of staffing, educational resources and provider The heavy use of premises will be determined in accordance with applicable qualifiers such as quality assurance frameworks. "sufficient", "appropriate", • If no quality framework applies to staffing resources, providers must have, and use, documented policies and processes for: "reasonably available" recruitment, induction, performance assessment and ongoing in the statement of development of staff who recruit or work with overseas standards renders the students. process vulnerable • If no quality framework applies to education resources, providers must have adequate resources to deliver the to inconsistent registered course to the students enrolled. judgements and reduces • The provider must notify the designated authority and procedural fairness. enrolled students of any intention to relocate premises at least 20 working days before the relocation. Is the National Code model what we can expect from TEQSA? As noted above (see 3.5.2), the Australian Government's "Higher Education Standards Framework" comprises "provider registration standards", "provider category standards", "qualifications standards", "information standards", "teaching and learning standards" and "research standards". THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 136 Provider registration standards can be expected to take the form of a document specifying properties that a provider must be able to demonstrate as a condition of obtaining a license to operate. The first draft of provider registration standards in 2009 specified 89 requirements under 9 categories (see Box 38). A problem with the draft, apart from its excessive requirements and the extensive reporting they demand, is that whereas some requirements are readily observable, many of them require interpretation, e.g. under 'management': "the provider maintains an internal culture of respect and trust, including respect for all employees, for students, for Indigenous Australians, for multiculturalism and pluralism and for learning". In what sense is that a standard? Whatever it is it is plainly inoperable, not least because a provider cannot know what it takes to comply. The heavy use of qualifiers such as "sufficient", "appropriate", "reasonably available" in the statement of standards renders the process vulnerable to inconsistent judgements and reduces procedural fairness. Box 38. First Draft Higher Education Provider Registration Standards and Requirements 1. Legal status and standing: The higher education [provider is reputable and is legally accountable for the higher education it offers. 2. Financial viability and safeguards: The provider has sufficient financial resources and financial management capacity to sustain the operation of the provider's higher education awards at an acceptable standard of quality, including the provider's awards offered through partnerships with other institutions within Australia or overseas. 3. Primacy of academic quality and integrity: The provider maintains academic quality and integrity. 4. Governance: The provider is well-governed in respect of its higher education activities. 5. Management: The provider is well-managed in respect of its higher education activities. 6. Responsibilities to students: The provider defines and meets its responsibilities to students, including the provision of information, support and equitable treatment. 7. Human resources and professional development: The provider engages and retains sufficient appropriately qualified and skilled personnel to ensure effective student learning and ensures its personnel are able to professionally develop their skills and knowledge. 8. Physical resources and infrastructure: The provider makes available sufficient physical and electronic resources and infrastructure to ensure the achievement of its higher education activities, including achievement by students of expected learning outcomes. 9. Standards for programs: The provider maintains appropriate academic standards in its higher education programs. Source: DEEWR, 2010. There are two projects being funded by the ALTC relating to teaching and learning standards. One is the 'Benchmarks for teaching and learning quality assurance' exercise discussed at 3.5.5 above. The other is the 'Teaching Standards Framework', outlined at 3.5.6 above, the design of which is based on a template developed by Macquarie University. Macquarie University/ALTC Teaching Standards Framework project Macquarie University has developed a teaching standards framework based on the view that "effective learning requires teaching built on: • A university culture that is focused on enhancing the quality of student learning in professional, intellectual, social and ethical terms; THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 137 • Universities that are socially dynamic and student-centred (in both administration and teaching), with policies and practices that enhance their social inclusiveness and enrich university study as a total human experience; • Governance that is transparent, accountable and responsive to student, community and government priorities; • Policies and practices which facilitate excellence in learning and teaching outcomes through clear academic planning, explicit appointment criteria and career development practices; • Appropriate resourcing; • Teachers who are familiar with the latest developments in their disciplines; establish clear learning and teaching strategies and outcomes; are familiar with innovative thinking on learning and teaching, and are accessible and responsive to students, colleagues and the community" Macquarie University, 2010a). Macquarie defines teaching standards as "the criteria by which we assess the quality of learning and teaching performance and outcomes" (Macquarie University, 2010b). Its institutional level teaching standards framework considers 'culture', 'governance' and 'practices' along levels of achievement in relation to the criteria: "In general terms, 'No' indicates a failure to address the criterion; at 'No But' there is some manifest acknowledgement of the criterion and some intention of meeting it, but so far there has been no substantial progress towards that goal; at 'Yes, But', there has been an active attempt to meet the criterion, but without significant innovation or initiative; at 'Yes', institutions will be actively re-thinking what they do in light of the criterion, and innovating accordingly. There is provision to exceed 'Yes', where an institution will be pioneering new methods of learning and teaching that will contribute to a re-definition of the criterion" (Macquarie University, 2010b). By way of illustration under 'practices', in relation to the criterion "University funding models recognise and reward good teaching", the following levels are described for self assessment purposes: 'No' Funding models do not recognise teaching excellence. 'No, But' Funding models recognise the importance of teaching excellence but do not provide adequate funding due to competing priorities 'Yes, But' The University allocates resources to support teaching excellence through its funding models, but they are targeted narrowly due to competing priorities 'Yes' The University funding model allocates appropriate resources to support teaching excellence across the institution. Why should Macquarie's template be replicated across other universities? While it may be useful for performance improvement purposes for individual institutions voluntarily to benchmark their policies and practices, it is not self-evident that such an approach should be part of a national standards framework and as part of a regulatory mechanism. Indeed, standard practices in this area are inappropriate, as each institution should gear its teaching strategies to meet its particular objectives in relation to its students. Hence, it is curious that the project is being funded for the purpose of developing a framework which "would then be validated as a tool which could be used by government agencies such as TEQSA and for inter-institutional benchmarking" (Macquarie University, 2010a). The ATN Academic Indicators Another guide to the possible evolution of "standards-based arrangements" for academic quality assurance is the suite of indicators being developed by the Australian Technology Network (ATN) group of (formerly capital-city polytechnics) universities. The ATN commissioned ACER (a major vendor of THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 138 testing instruments) to develop a draft set of 'academic standards', The clear inference is and the ensuing report offered a model which might "further distinguish ATN institutions as a consolidated network, and provide that something along a foundation for network-wide and evidence-based planning, the lines of the ATN practice and review" (Coates, 2007a). The initiative is of some note academic standards because of the explicit reference to it in the Bradley report (Bradley being of the ATN stable), with the enjoinder that the ATN model framework ought to should be replicated: be applied universally "Work is already under way in the sector to start articulating across the Australian academic standards in a more sophisticated way. For example, the Australian Technology Network group of universities has higher education commenced a project on academic standards which could sector, and quickly. be used to benchmark across institutions. While this is an important initiative, what is needed is more rapid and systematic implementation of a coherent national framework that applies to all higher education providers" (Bradley et al., 2008). The clear inference is that something along the lines of the ATN academic standards framework ought to be applied universally across the Australian higher education sector, and quickly. But why? And why the rush? And why model the Australian higher education sector on the aspirations of the ATN? It is one thing for a group of institutions to seek to differentiate themselves through a particular model of reporting on their capacity and performance, but it is quite another thing, indeed self-defeating as well as ingratiating, to impose that group's model on everyone else. Importantly, the proposed approach reflects a lack of regard for diversity and a complete lack of understanding of what drives innovation and quality in higher education. The Go8, for instance, would not wish to be limited by the horizons of the ATN. The Teaching and Learning Academic Standards Framework for the University of South Australia is at Attachment C. Is this indicative of the operational model to be imposed on all institutions? Why should it be assumed that its particular approach has merit? Or does it reflect the notion of a standard as merely 'acceptably normal'? Indeed its blunt approach to knowledge is well short of cutting edge. In Go8 universities, academic staff are appraised against disciplinary leaders internationally, and learning is informed by discovery well in advance of what appears in textbooks. But the University of South Australia is apparently satisfied with a much lower standard: "The University encourages academic staff to contribute to their discipline and be in touch with current research and scholarship, integrating into their teaching the knowledge and understanding they and others create through scholarly activity, including the creation of text books and other teaching resources" (University of South Australia, 2009). In any event, the ATN model is a long way from being operational, as indicated in the recommendations of the commissioned report (Coates, 2007a): 1. ATN institutions should adopt a consistent definition of 'academic standards as being 'levels of performance on key academic indicators of educational quality'. 2. ATN institutions should endorse the proposed ATN Academic Standards Model, which consists of a high-level indicator framework, a suite of measures to support these indicators, an approach for gathering data on each of these measures and a series of standards for indentifying performance. 3. ATN institutions should produce a succinct plain language summary that provides information to relevant stakeholder groups on the specification, measurement, monitoring and enhancement of academic standards. This could be prepared by individual institutions, or across the ATN as a whole. THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 139 4. ATN institutions should implement the ATN academic Standards model. This would involve operationalising the model, mapping data elements against defined measures and indicators, managing and analysing data, developing performance measures and reports, and establishing routines for benchmarking and improvement. 5. ATN institutions should develop their capacity to measure and hence assure general graduate capabilities including work readiness. To provide a foundation, a comparable set of graduate capabilities should be defined and embedded into learning and teaching. Assessments should be developed to measure graduates' capability, which may include routine assessments, feedback from employers, or an objective test. 6. ATN institutions should undertake a systematic and multifaceted review of student assessment and reporting. Such a review could develop ATN capacity to: monitor student input standards, produce validated assessment tasks, develop moderation processes to ensure the equivalence of learning standards, develop comparable curriculum standards, develop common reporting metrics, develop transparent statements of attainment and conduct routine analyses of student performance data. 7. ATN institutions should develop a systematic approach to monitoring and enhancing industry involvement in learning. Institutions might: highlight the important role that employers, industries and working professionals play in ensuring the quality of higher education; enhance the formative input provided by industry into educational design, delivery, assessment and review; strengthen or build relationships with professional bodies; and obtain more systematic forms of feedback from graduate employers. 8. ATN institutions should further develop their approach to documenting and developing educational resources. They should design and implement a systematic approach to the production of teaching portfolios and initiate the development of course portfolios. Of particular note is the set of actions at recommendation 6 above, including equivalent learning standards, comparable It is one thing for a curriculum standards and common reporting metrics. It is one group of five like- thing for a group of five like-minded institutions to develop minded institutions to comparable approaches but quite another to require all institutions to comply with a single model. develop comparable Indeed it is inconceivable that a government in a contemporary approaches but quite democracy would contemplate such a latter-day Stalinist model. another to require all The ATN Academic Standards Model involves sets of measures to institutions to comply support three types of performance indicators: outcomes (see Box 39); process and context (see Box 40); and inputs (see Box with a single model. 41). The development of data for comparable reporting and benchmarking on these measures can be seen as a significant improvement agenda for the ATN. But what has this to do with the role of a national regulator? As noted at 2.5.3 above, wide adoption of common templates tends to replicate sameness and reduce diversity. THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 140 Box 39. Measures to support outcome indicators Level Indicator Measures Student Graduation Completion rates Time to completion Graduate destinations Labour-force participation rates Further study participation rates Satisfaction Graduate satisfaction data Completion rates Student satisfaction data Student retention rates Learning outcomes Validated assessment results Student success rates Student engagement data Numeracy and literacy data Further study participation rates Graduate qualities Employer satisfaction data Graduate attribute assessments Labour-force participation rates Data on generic skills Work readiness Capstone program participation rates Data on work readiness Data on employability skills Teacher Teaching experience Number of teaching awards Teaching staff experience Teaching quality data Teaching resources Teaching resource satisfaction data ICT resource satisfaction data Library satisfaction data Production of teaching resources Provider Institutional growth Number of partnerships and networks Teaching and learning income Institutional reputation Placement in institutional rankings Course demand data Number of teaching awards International student exchange rates International staff exchange rates International student numbers Community Size of alumni programs Data on community engagement engagement Employer satisfaction data programs Equity group access and quality data Service learning participation rates Box 40. Measures to support process and context indicators Level Indicator Measures Student Student engagement Student engagement data Retention rates Completion rates Retention and progress Retention rates Retention programs Progress rates Learner support services Teacher Teaching processes Teaching quality data Sessional staff support programs Staff/student rations Teacher review processes Staff mentoring programs Curriculum management processes Staff development programs Course management Scheduling and timetabling management Arrangements for course coordination Industry involvement in course design Course approval processes Course viability and relevance Staff teaching load Course development processes Provider Academic governance Education policies Management policies Academic Education plans and systems Learner support programs management Management plans and systems Systems for managing student experience Academic culture Staff support services Education support programs Diversity of academic staff Plagiarism rates Staff development Staff development participation data Teaching development grants International staff exchange rates Academic staff promotion rates Quality systems Monitoring processes Staff mentoring programs Enhancement activities Academic appeals processes Examination procedures THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 141 Box 41. Measures to support input indicators Level Indicator Measures Student Entry levels Literacy and numeracy data Course demand and selectivity Academic literacy Entry pathways Credit transfer arrangements Demand from qualified regional students Student selection processes Diversity of entrance pathways Advanced standing arrangements Transfer and articulation arrangements Extent of financial supports Student diversity Incoming student characteristics Equity group acess and participation Number of exchange students Student exchange supports International student numbers Teacher Staff characteristics Academic staff in senior positions Academic/administrative staff rations Staff teaching qualifications Sessional teaching staff numbers Academic staff with doctorates Teaching staff experience Staff international experience University University enculturation programs Retention programs enculturation Educational resources Teaching resources Library resources and services Teaching development grants Learning innovation programs Course development Financial status of courses Curriculum relevance Course accreditation processes Course review processes Course development processes Industry involvement in course design Course approval processes Teaching development grants Course coordination arrangements Support systems ICT resources and supports Equity student support programs Staff mentoring programs Student support services Sessional staff support programs Disability support services Staff development programs Induction programs Provider Institutional Investment in learning infrastructure Community outreach programs characteristics Institutional ranking Institutional reputation Course demend and selectivity Alumni programs Presentation at conferences Partnership and network arrangements International student numbers Institutional rankings Institutional resources Learning infrastructure Library resources and services Partnerships and networks Teaching and learning income Educational deveopment programs Teaching development grants Teaching staff experience Industry engagement Course accreditation processes Alumni programs Course relevance Labour-force participation rates Service learning programs Course-integrated careers advice Industry involvement in course design Industry partnerships and networks 4.3.5 Comparability or consistency? There are now very wide differences in the input factors to higher education, including students and teachers whose interactions are the critical determinants of learning, and it would be unreasonable to expect flattening of those differences in the characteristics of graduates. "Any agreement to have a uniform system-wide set of standards for student academic achievement raises the issue of whose standards. It is unlikely that any institution would wish to lower its standards of student academic achievement which immediately raises the issue of the implications of imposing the same unrealistically high levels of academic achievement on all students in the sector in terms of equity and social inclusion" (Dearn, 2009). THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 142 The greater diversity of the student mix, provider types and modes of teaching and learning requires more sophistication rather than more simplicity in the representation of the characteristics and contributions of higher education: "At a time when only a very small proportion of the population went to university, and the student population was broadly equivalent in "It makes little sense to terms of background and ability—and when degree courses were seek comparability of considerably more uniform in terms of their nature and intended outcomes, and indeed it outcomes than they are now—it was undoubtedly a reasonable expectation that the outcomes of degree courses should be broadly would actually be wrong comparable, and that there should be mechanisms available to to do so." police this (hence, external examiners). Today, the environment is radically different. Nearly half of the young population now participate in higher education, the range of ability of those students is very wide, and the purpose, nature and intended outcomes of programmes all vary considerably. It makes little sense to seek comparability of outcomes, and indeed it would actually be wrong to do so. Given the extraordinarily high previous educational attainment of students attending, say, Oxford or Cambridge, the substantially greater resources devoted to them, the greater intensity of study that they undergo, and other factors, it would in fact be a surprise if the outcomes of students from those universities were no higher than those of students from other universities who have far lower prior attainment, resources devoted to them, and so on. But, self-evident as this might seem, there are actually no instruments available to demonstrate it." (Brown, 2010a). As noted at 3.2 above, the question of comparability or consistency of degree standards has been raised in Britain through the House of Commons, motivated primarily by a desire to remove discrimination against graduates of less prestigious institutions and to inform students of the worth of their degrees. A similar debate is in progress in the US (see Box 42), inspired by similar concerns and a need to improve the information available to employers. Box 42. Making Degrees Easier to Interpret "Suppose an employer advertises an entry-level position that requires advanced statistical knowledge. The employer narrows down the applicant pool to three finalists for the position: an Ivy League graduate, a graduate from a small public college, and a graduate from a for-profit university. All the candidates have bachelor's degrees in statistics and all have roughly the same GPA's, previous work experiences, and pleasant demeanors. How can the employer possibly distinguish the values of the three finalists' degrees? There is essentially no method to determine which of the three graduates have the knowledge and skills that match the advertised position. Grades and academic standards often vary so much by institution, department, and instructor that transcripts are written off as arbitrary and meaningless by those making hiring decisions. Outside fields with licensure exams like accounting and nursing, employers often hire workers based on connections, intuition, and the sometimes-misleading reputations of applicants' alma maters. This system doesn't allow labor markets to function efficiently. And it's far from meritocratic for college graduates, especially the talented ones who attended less-selective schools and are disproportionately likely to be first-generation, low-income, or students of color. To rectify this broken hiring system, academia and industry should form stronger partnerships to better determine which skills and knowledge students in various fields need to master. Some types of common and field-based assessments are needed to help employers match their jobs to graduates with complementary skills, even if the assessments are entirely voluntary for students. The traditional college transcript is simply too impenetrable for anyone outside—or inside—academia to comprehend." Hinton, F. (2010). THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 143 By what means could qualitative differences in student learning One option for be demonstrated amid great diversity? implementing a more It has been suggested that the very quest for consistency in higher education standards is quixotic and fails to appreciate the diversity customised approach is and dynamism of the field. A more customised approach is seen to to develop the 'diploma be appropriate, where a higher education institution puts forward supplement' as a fuller the objectives, learning opportunities and assessment strategies for its programs, reflecting its mission and validated by the relevant record of the learning field and professional communities (see Box 43). One option for experiences of students. implementing a more customised approach is to develop the 'diploma supplement' as a fuller record of the learning experiences of students. Box 43. Comparability and consistency in British Higher Education "There is no mechanism to ensure consistent and meaningful comparability among institutions and subjects, and no mechanism I can envisage that could make it so. National examinations, which some have suggested, or individual degree standards overseen by a body such as QAA, would create a vast industry and an attendant bureaucracy and its inevitable failure would make the annual row over GCSEs and A Levels look very tame indeed. It would be much simpler to stop using these out of date classifications designed to meet the needs of another century, and provide individually focused information which actually tells the user something about the student and what he or she has learned. The 'one size fits all' scheme we now use is a travesty of fairness and consistency. We seem in this country to have no capacity to think beyond monolithic hierarchies and, in trying to shoehorn very different purposes, clienteles, structures and people into a single narrow boot marked 'The only acceptable HE standards for the UK', we perhaps reduce our opportunities to innovate, develop and recognise a much more useful set of standards based on the particular characteristics of the students and programmes being offered. Provided the standards are clearly stated and readily available, validated by the relevant subject and professional community as useful, valuable and appropriate, and form the basis for the assessment of students, then the variations between subjects and institutions should become a reason for celebration, not the sort of angst about irreconcilable differences." Williams, P. (2010). Even the search for threshold standards is seen to be a formidable challenge in a sector which continues to diversify: "I'd like to refer to what I've called Brown's Paradox (but I don't claim originality for it) which is that, as the system expands, the pressures of comparability increase but, by the same token, the ability to ensure it reduces. Indeed the major changes that have taken place over the last decade have produced an incredibly heterogeneous sector with far more types and structure of degree than in the past. And this looks set to continue. They make such threshold standards increasingly impossible to implement, at the same time as creating a situation which makes their absence felt, and I think that is the nub of the problem" (Brown, 2010b) Similarly, in the US there is a troubled view about the penchant of governments to seek simple comparisons of higher education outcomes based on scores on standardised tests, and the damage that approach can do to diversity: "Using common measures and standards to compare institutions that serve markedly different student populations (e.g., a highly selective, residential liberal arts college compared to an open-access THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 144 community college with predominantly part-time students, or a comprehensive public university serving a heterogeneous mix of students) results in lowered expectations for some types of institutions and unreasonable demands for others. If similar measures are used but "acceptable standards" are allowed to vary, an inherent message is conveyed that one type of mission is inherently superior to the other. The diversity of the US higher education landscape is often cited as one of its key strengths. Homogenous approaches to quality assessment and accountability work against that strength and create perverse incentives that undermine important societal goals" (Borden, 2010). The challenge of comparability is complicated by the range of expectations for it, and the associated confusion of policy intent: "Comparability means that the standards of learning aimed at and achieved by students in any two programmes leading to the same or a cognate award are genuinely equivalent. So it could mean, for example, that all students in one institution obtaining a bachelors degree in any subject are achieving the same standard, all students from several institutions obtaining a bachelors degree in any subject are achieving the same standard, and it could mean all students from several institutions obtaining a bachelors degree in the same subject are achieving the same standard. It could also refer to common standards in all elements of a programme, options as well as core, and it could mean common standards over time in different cohorts of a programme" (Brown, 2010a). In principle, consistency of degree standards would require commonality in each of following conditions: • within all the components of a degree program (including options) within an institution; • in the degree program followed over several years; • in the standards aimed at and achieved in similar programs in the same subject in different institutions; • in the standards aimed at and achieved in different subjects both within an institution and across the sector (Brown, 2010a). To provide valid and reliable information about the comparative "I believe that any real quality of programs and awards it would be necessary that: comparability now • the programs would have to be comparable in terms of aims, is infeasible, at least structure, content, learning outcomes, delivery and support; without a national • similarly, the awards would have to involve comparable assessment methods, criteria and outcomes (marks or grades); curriculum and • the assessment judgements would have to be valid, reliable national examiners and consistent; and answerable to a national • students pursuing the programs (and/or interested in pursuing standards agency." the programs) would have to have comparable starting attainments, aspirations, motivations and learning objectives (Brown, 2007). These conditions are neither likely nor desirable in a diverse and responsive system. Not only is the feasibility of consistency ('strong comparability' in British usage) dependent on a Napoleonic approach, of a national curriculum delivered regimentally, but it could also produce perverse outcomes: "…is strong comparability really desirable? Should a demonstrable persistently significant lack of comparability mean some exam boards, departments or even possibly institutions giving larger numbers of highly rated awards and others fewer? Would some courses have to teach less or to a THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 145 lower standard and vice versa? Should there be changes in resourcing levels and policies in ambitions, criteria, etc? A combination of some or all of these might put certain programmes, departments or even, dare I say, institutions, out of business. Who would decide these things assuming we were to get that far? I believe that any real comparability now is infeasible, at least without a national curriculum and national examiners answerable to a national standards agency" (Brown, 2010b). Curiously, in the British context, 'comparability' has come to have the peculiar meaning of 'same', 'common', 'consistent' and 'equivalent'. Additionally, the terms are applied interchangeably to standards and performance. Such confusing use of terms is unhelpful for international discourse. It would better to distinguish between key terms, and to be clear about the policy purposes attached to each. Various definitions of the concepts being used in policy discussions, including for 'learning outcomes', have been explored. A set of working definitions for the wandering adjectives is offered in Box 44, with the underlined phrase being the preferred meaning for each adjective. Box 44. Working definitions of key qualifiers Same identical; uniform; unvarying; Common typical; occurring often; shared by many; of the most familiar type; Similar alike; resembling the same kind ; Equivalent equal in value, or importance or utility; of commensurable worth; Consistent not contradictory; constant to same principles; compatible; Comparable capable of being compared; enabling estimated similarity or dissimilarity Sources: Australian Oxford and Macquarie dictionaries. These adjectives may be qualifiers for either standards or performances, but they have very different implications according to what is being qualified. For instance, consistency is not sameness. Rather, it is constant adherence to a set of principles, on the part of a particular higher education provider. Thus, consistency cannot be norm referenced. Equivalence is about social value and recognition, despite difference. Comparable differs from same and common, in that it relates to dissimilarities as well as similarities. These are not trivial nuances. They go to the heart of appreciating what is worthwhile and what can be demeaned by lack of that appreciation. They expose as vacuous any notion of consistent standards across a national system of higher education. 4.3.6 Fitness for purpose, fitness of purpose, and a standards-based approach "fitness for purpose" is a definition of quality that allows institutions to define their purpose in their mission and objectives, so "quality" is demonstrated by achieving these. This definition allows variability in institutions, rather than forcing them to be clones of one another" (Woodhouse, 1999). "Fitness for purpose approaches explicitly acknowledge diverse institutional missions and the differences in what they achieve. Standards-based approaches emphasise what institutions should have in common, especially in terms of the nature and level of learning outcomes that students are expected to demonstrate in their university studies" (James, McInnis & Devlin, 2002). The concept of quality as fitness for purpose differs from other notions of quality in fundamental ways, for it is based on the premise that if something does the job for which it is designed, then it is a quality product or service. That is, every product or service has the potential to fit its purpose and thus be a quality product or service: THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 146 "The ultimate measure of perfection, 'zero defects', may be excellent as a definition of quality but runs the fatal risk of being perfectly useless. If the product does not fit its purpose then its perfection is irrelevant" (Harvey & Green, 1993). As one of the five definitions of quality identified by Harvey and Green (1993), fitness for purpose is the most deceptive, "for it raises the issue of whose purpose and how is fitness assessed?" Fitness for purpose offers two alternative priorities for specifying purpose. The first puts the onus on the customer, while the second locates it on the provider: "Fitness for purpose sees quality as fulfilling a customer's requirements, needs or desires. Theoretically, the customer specifies requirements. In education, fitness for purpose is usually based on the ability of an institution to fulfil its mission or a programme of study to fulfil its aims" (Harvey & Green, 1993). Harvey & Green elaborate on the extent to which fitness for purpose is customer-specified, in the sense that a customer has requirements that become the specifications for the product, and the outcome meets those requirements: "Thus a quality product is one that conforms to customer determined specifications. This approach provides a model for determining what the specification for a quality product or service should be. It is also developmental as it recognises that purposes may change over times thus requiring constant re-evaluation of the appropriateness of the specification" (Harvey & Green, 1993). However, they note that customer specification is an idealisation, and that in practice, customers rarely specify their individual …customers rarely requirements. In the general production of goods and services in specify their individual mass markets, providers anticipate and assess what the customer requirements. is prepared to buy. In education there is the added complication of multiple customers and consumers who may not know what they want: "First, the notion of 'customer' is itself a tricky, indeed contentious, concept in education. Is the customer the service user (the students) or those who pay for the service (the government, the employers, parents)? Second, the customer, the student for example, is not always able, nor necessarily in a position to, specify what is required. Fitness for purpose, therefore, leaves open the question of who should define quality in education and how it should be assessed" (Harvey & Green, 1993). So with some circularity, 'fitness for purpose' in education moves from being driven by student requirement to being driven by institutional mission. The important corollary is that quality is a function of how well an educational institution fulfils its mission: "The tricky issue of determining who are the customers of higher education and what their requirements are can be avoided, to some extent, by returning the emphasis to the institution. Quality can be then be defined in terms of the institution fulfilling stated objectives or mission" (Harvey & Green, 1993). However, there remains another problem. Defining quality only in terms of fitness for purpose has no referent other than what an institution claims to stand for: "a major weakness of the fitness for purpose concept is that it may seem to imply that "anything goes" in higher education so long as a purpose can be formulated for it" (Campbell and Rozsnyai, 2002). This tension can be addressed by locating fitness for purpose in the context of shared understandings (see Box 45). In this understanding of the complexities, fitness for purpose approaches to quality assurance can be complemented by references to external expectations, such as in the form of criteria for employability and indicative standards. The issue, as always, is the balance between similarity and dissimilarity of expectations, and the degree of discretion that providers are allowed in serving different needs as best they can. THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 147 Box 45. Fitness for purpose and fitness of purpose "Among the various criteria used in judging quality, we find the terms 'fitness for purpose' and 'fitness of purpose'. The former, often used in quality assurance activities, means determining whether the academic strategies are suitable for achieving the declared aims of a programme. The latter means determining whether the aims of the programme are suitable or not. In the Tuning view, to develop true quality, 'fitness for purpose' has meaning only when the fitness of purpose itself is thoroughly established and demonstrated. As a consequence Tuning holds that quality in programme design and delivery means guaranteeing both "fitness for purpose" (i.e. suitability for achieving the declared aims of each programme ), and "fitness of purpose" (i.e. suitability of the aims of each programme: these should meet the expectations of students, academic staff, employers and the broader ones foreseen in the Bologna Process). Guaranteeing "fitness of purpose' requires a strong connection with research and academic standards as well as a consideration of employability which is only implicit in the "fitness for purpose" definition". Source: Quality enhancement at programme level: The Tuning approach. Tuning Educational Structures in Europe. http://www.tuning.unideusto.org/tuningeu/index.php?option=content&task=view&id=176. Thus we return yet again to the basic question; whose standards? The major policy issues arising from this question are: Who should set standards for higher education? Should externally-set standards serve as references or guidelines for higher education institutions to use, inter alia, in setting their own standards? Or should the institutions focus on ways and means of meeting the externally-set standards? It may be argued that external standards leave institutions free to determine the ways and means of achieving desired outcomes. That is, the setting of standards as criteria for the assessment of effectiveness does not necessarily mean standardisation of what is taught and how it is taught, nor does it diminish institutional autonomy in respect of curriculum and pedagogy. However, the setting of academic standards is the fundamental expression of what a university stands for. To take away from a university the function of setting its educational goals is to deprive it of its reason for being. The university has its own standards of excellence to live up to. It also needs to be responsive to the expectations of others. In a plural system the university's own expectations and those of the community it serves may not always align with standards set by a national regulator. THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 148
Pages to are hidden for
"Definitions"Please download to view full document


text:
How to: Make a Colorful Bottle Cap Tray
I'm sure there are some of you out there with a great bottle cap stash, or could easily accumulate caps after a party of some kind. So, what can you do with so many caps? How about your own custom serving tray? Continue Reading

subset: commoncrawl


text:
I've obtained a copy of the presentation that was made to the FTC about debt settlement companies that are trying to evade the FTC telemarketing sales rules which in turn lead to consumer harm. It was interesting to see so many images from this site used since I had nothing to do with the presentation, but hey, I have reported ... Read More »
Credit Answers Hides Behind DMI Mailer to Consumers
Thanks to a great tipster (send in your tips here) I received a mailer that was sent to consumers. The envelope looks official and comes from: DMI 6200 Tennyson Parkway Suite 200 Plano, TX 75024 The letter inside is the first mention of Credit Answers in the small print on the bottom. It is also the first place we learn ... Read More »
Debt Settlement Company Elite Financial Services Agrees to Repay Missouri Consumers
Attorney General Koster yesterday announced that the Massachusetts-based debt settlement firm Elite Financial Services, Inc., has agreed to refund all fees paid by Missouri consumers and cease operating in Missouri. Koster said Elite Financial Services charged consumers a fee and promised to reduce their unsecured debt, such as credit card debt, but failed to do so in many instances. He ... Read More »
Legal Helpers Debt Resolution Email in Search of Marketers
A tipster (send in your tips here) forwarded me this email, image below, that Legal Helpers Debt Resolution is forwarding in search of companies in the debt relief world that want to "Make Money Next Week." It is certainly an interesting approach for a law firm to take and it did strike me that they are pushing the conversion to ... Read More »
Premier Debt Resolvers Deceptive Mailer – Government Economic Stimulus Act
This mailer comes from Premier Debt Resolvers at 615 Degraw St, Brooklyn, NY 11217. 877-845-6161. The two issues that jumped out immediately to me were the use of a government act that has nothing to do with debt settlement and the use of the equal housing logo on the bottom right. I also have a problem with the mailer making ... Read More »
CareOne Jumps on Bandwagon with Credit Counseling Request for FTC to Take Action Against Rogue Debt Settlement Companies
A pretty smart move by CareOne. See press release below. CareOne Expands On Push to Close Debt Settlement Loopholes Columbia, Md. (December 16, 2010) – CareOne Services Inc. is adding its support to an effort launched December 15th by 11 nonprofit debt relief providers seeking more aggressive enforcement of the Federal Trade Commission's recently amended Telemarketing Sales Rules. However, the ... Read More »
Minnesota Sues TaxMasters Claiming Fraud and Deception
Minnesotta Attorney General Lori Swanson today file suit against TaxMasters, a tax debt relief company. The suit allegedly states that TaxMasters used deceptive and misleading advertisements and commercials to attract consumers into large up-front fees for debt relief services, paying upwards of $8,000. The suit is said to claim the TaxMasters salespeople encouraged consumers to pay the steep fees on ... Read More »
Debt Free League Launches FreeBankruptcyTest.com With Attack on Credit Counseling
Apparently the lads over at Debt Free League, who I've written about before, have launched a new website with a big press release. The site is FreeBankruptcyTest.com and claims "The company's site, FreeBankrutcytest.com helps the public to freely compare debt relief choices, including bankruptcy, debt negotiation, debt consolidation, tax settlement, and student loan consolidation." Just one problem, it does not ... Read More »
US Credit Card Hardship Benefits Mailer – Bogus and Deceptive
Another fantabulous tipster (send in your tips here) sent in the following scanned copy of yet another government looking self-mailer that sells debt relief services. This one is for the 2010 US Credit Card Hardship Benefits program. As you can see it purports to be an official looking government form with form number 10-PNH3. It states that the pending hardship ... Read More »
Credit Counseling Agencies Challenge FTC to Take Agressive Action Against Debt Settlement Companies for Violating Telemarketing Sales Rules
The following letter was sent to the Federal Trade Commission: December 13, 2010 Federal Trade Commission Office of the Secretary Room H-135 (Annex T) 600 Pennsylvania Ave, NW Washington, DC 20580 Re: Debt Settlement Company Evasion of Amendments to TSR Dear Commissioners, We are writing to speak out against the deceptive tactics employed by a growing number of debt settlement ... Read More »
Credit Counseling Groups in the Crosshairs With Pay For Performance History
This comes straight from a tipster (send in your tips here) who directed me to the site of Cotchett, Pitre & McCarthy. I reached out to CPM for additional information but as of the time of publication I had not heard back. Cotchett, Pitre & McCarthy is continuing its investigation into fraudulent "debt counseling" and debt collections in the subprime ... Read More »
Goooood Mooooorning….Potential Affiliates. The Attorney Network.
An awesome tipster (send in your tips here) sent this to me recently. What's your take on this not being fee splitting or landing people into trouble? I was intrigued by the statement: 3rd Party Debt Buyer takes the chance by re-writing the clients loan without even having to have the credit to qualify (Basically giving the client the chance ... Read More »
Maryland Tries for a Fourth Time to Pass Debt Settlement Regulation
Maryland legislators for the fourth year in a row will consider regulating debt settlements, and new recommendations in a report out this month may be the charm. In Maryland, legislators have tried three times to regulate the industry. Sen. Thomas "Mac" Middleton, chairman of the Finance Committee, says his committee requested a study before the next session begins in January. ... Read More »
Offering Any Debt Relief Services in Florida? Pay Attention. Telemarketer Arrested.
Florida Agriculture and Consumer Services Commissioner Charles Bronson has announced the arrest of a Lee County man for falsifying information on a state document that telemarketers are required to submit. 38 year old Jose A. Negron was charged with two 3rd degree felonies for falsification of a telephone salesperson application. The department regulates telemarketers and state law requires them to ... Read More »
Tipster Wants to Know About Trinity Affiliate Management
I got a question from a tipster (send in your tips here) asking me about debtsettlementprocessingnow.com. I had never visited the site before [quicktime]http://getoutofdebt.org/wp-content/uploads/TrinityAffiliateManagement-20101213.mov[/quicktime] As of the last public listing of domain ownership, 9-17-2009, the domain was registered to: Trinity Debt Group 1180 Lincoln Ave Suite 1 Holbrook, NY 11741 A Google search of the address above now lists it ... Read More »


text:
Economic Returns
Market-oriented assessments of educational output, with attention to how salary effects vary by area of study and by institutional quality, have been explored in the economics literature.26 Some studies attempt to assess evidence on the relationship between college cost and quality in terms of student and institutional performance. Beyond wage outcomes, indicators have reflected number of graduates who find a job within a given period after graduation; surveys of alumni satisfaction with their education; surveys of local business communities' satisfaction with the university's role in providing skilled workers; percentage of students taking classes that require advanced work; and number of graduates going on to receive advanced degrees.
Despite work in this area, many tough issues remain even if the goal is to estimate only the economic returns to education. How can wage data best be used for such calculations, and what is most important: first job, salary five years out, or discounted lifetime earnings?27 Furthermore, intergenerational and business cycle effects and changing labor market conditions cause relative wages to be in constant flux. Perhaps most importantly, student characteristics, demographic heterogeneity, accessibility and opportunities, and other factors affecting earnings must be controlled for in these kinds of economic studies. The reason full quality adjustment of the output measure is still a futuristic idea is that much research is still needed to make headway on these issues. The literature certainly offers evidence of the effects of these variables, but using precise coefficients in a productivity measure requires a higher level of confidence than can be gleaned from this research.
Student Learning
Beyond measures of credits and degrees produced, and their associated wage effects, is the goal of measuring the value added of student learning.28 The motivation to measure learning is that the number of degrees or credit hours completed is not, by itself, a complete indicator of what higher education produces. That is, earning a baccalaureate degree without acquiring the knowledge, skills, and competencies required to function effectively in the labor market and in society is a hollow accomplishment. Indicators are thus needed of the quality of the degree represented by, for example, the amount of learning that has taken
26See, for example, Pascarella and Terenzini (2005), Shavelson (2010), and Zhang (2005).
27Estimating lifetime earnings would introduce long lags in the assessments as some evidence suggests that the most quantitatively significant wage effects do not take effect until 8 to 10 years after undergraduate degree.
28Not only is learning difficult to measure in its own right, it would be challenging to avoid double counting with wage effects (assuming those who learn most do best in the job market).


text:
SHE'S putting the 'kid' back into 'Kidman'.
Oscar-winner Nicole Kidman will be getting her hands on the Grouchier kind of Oscar in the new season of Sesame Street.
The Aussie superstar will join a slew of big-name Hollywood stars, including Robin Williams, Sofia Vergara, Seth Rogen, Craig Ferguson, Jay Sean and Bruno Mars, who've signed on to hang with Bert and Ernie.
The Sun/Richard Termine


text:
Building Better Futures...
Thank you for your interest in working at Martin Community College. To be considered for employment, all Martin Community College applicants are required complete the State of North Carolina Application for Employment.
Martin Community College is seeking to attract qualified applicants for a Basic Skills Part-Time Instructor at Bertie Correctional Institution. The selected candidate(s) will substitute for basic Skills classes at various times as needed.
Must have a Bachelor's degree from an accredited college or university with major course work in education with 1 or more years of experience in an instructional setting as a teacher.
Must have the ability and knowledge to provide basic skills instruction to students in reading, writing, science, social studies and math. Must possess skills necessary to make appropriate student assignments for a multi-level classroom, administer assessments and tests and maintain accurate student and program records.
Responsibilities include: collecting, analyzing, and reporting data regarding student success initiatives, including but not limited to, Completion by Design, and Achieving the Dream; developing, coordinating, and implementing institutional evaluations; assisting with the program and administrative assessment process; and other projects. This is a part-time, grant funded position.
Preferred Qualifications: Bachelor's degree in social sciences or education. Experience with quantitative and qualitative research, statistical software (SAS, SPSS), developing and analyzing surveys, and experience within the community college system.
Responsibilities include preparing for and teaching classes (traditional and online); managing dental labs; and supervising clinical instruction. Required qualifications include National Board Dental Assisting Certification; baccalaureate degree (or currently enrolled in a baccalaureate program); coursework/training in educational theory and methodology; CPR certification; and work experience in a dentist's office. Community college teaching experience is preferred.
Required qualifications include an associate degree in Heating, Ventilation, and Air Conditioning from a regionally accredited college. Preferred qualifications include a bachelor's degree in HVAC or a related field; community college teaching experience; one year of industry experience; CFC certification; and a Heating Contractor Group 3 licensure. Teaching assignment is at the Bertie Correctional Institute in Windsor, NC. Teaching assignments may be interrupted and/or cancelled at times due to unscheduled prison activities, and all assignments must be made up to fulfill terms of the contract.
Requirements are a bachelor's degree with major coursework in Business Administration, Accounting, Information Systems, or related field; management supervisory experience; knowledge of state, federal, and local regulations and policies related to admission of students and, veterans, and academic records; strong computer skills; and excellent oral, written, and interpersonal skills. Community college Registrar experience is preferred.
The position is responsible for maintaining the College's BLET accreditation; teaching, supervising, and evaluating instructors; coordinating program activities, including advising and recordkeeping.
Required Qualifications are a Bachelor's degree in criminal justice from an accredited university or college; NC Criminal Justice Instructor certification; one year experience as law enforcement officer; and teaching experience, preferably in a post-secondary setting. Master's degree preferred.
Review of applications will begin immediately, but position is open until filled. Salary is commensurate with education and experience.
This is a full-time position that is available immediately. Qualifications include a minimum of an Associate Degree in Information Technology or related area and experience in computer system and software installation and maintenance, application support, web programming, security and server support, and windows desktop and/or server related operating systems. Physical requirements include ability to lift a minimum of 50 lbs, climb ladders, kneel, and bend.
Desired qualifications include HTML, CSS, experience with PHP and a bachelor's degree in Information Technology or related area. Salary is commensurate with credentials and experience.
Responsibilities include providing technical assistance that includes, but not limited to, installing, maintaining, analyzing, troubleshooting hardware and software related problems repairing desktops, servers, computer peripherals, running cable, programming, and security for web-based resources. A flexible work schedule is required at times. Review of applications will begin immediately, but position is open until filled. Salary is commensurate with education and experience.
Educational requirements include an associate degree in Fire Protection Technology, Emergency Services, or related field; with additional education in one or more areas of emergency services from an accredited college or university and certified as a Level II EMS instructor or Level II Paramedics Instructor. Experience requirements include four years of increasingly responsible experience in academic and instructional programming and two years of post-secondary teaching and/or counseling experience. Bachelor or Master's degree preferred.
Review of applications will begin immediately, but position is open until filled. Salary is commensurate with education and experience.
Air Conditioning, Heating, and Refrigeration
Automotive Mechanic
Email questions to email@example.com
|HVAC (Heating, Ventilation, and Air Conditioning)
|Small Engine Repair
|Auto Mechanics
|Medication Aide
|Med Tech
|Pharmacy Tech
E-mail questions to:
or call 252-792-1521, ext. 248 or ext. 216