Quality Assurance for Learner Support

 Overview

These materials support a discussion on the topic of the terminology of quality assurance, especially as it applies to open and distance learning.

Source materials for this topic

Barnett, R. Improving higher education: total quality care. Buckingham: Society for Research in Higher Education and Open University Press, 1992.

Deming, W.E. Out of the crisis. Cambridge: mit, 1986.

Haughey, M. Can Quality management help us cope with change? In A. Tait (ed.), Quality Assurance in open and distance learning: European and international perspectives, pp. 117–25. Cambridge: Open University, 1995.

Juran, J. Quality control handbook. New York: McGraw-Hill, 1979.

Lentell, H. Quality: is it always a move to better things? In D. Sewart, ed., One world, many voices: quality in open and distance learning, vol. 2, pp. 121–24. Birmingham: Open University, 1995.

Mintzberg, H. The structuring of organisations. Englewood Cliffs, New Jersey: Prentice Hall, 1979.

Robinson, B. Assuring quality in open and distance learning. In F. Lockwood, ed., Materials production in open and distance learning, pp. 185–94. London: Paul Chapman, 1994.

Sallis, E. Total quality management in education. London: Kogan Page, 1993.

Tait, A. Systems, values, and dissent: Quality assurance for open and distance learning. In A. Tait (ed.), Quality assurance in open and distance learning: European and international perspectives, pp. 241–51. Cambridge: Open University, 1995.

Tait, A. Introduction: international perspectives on quality assurance in open and distance learning, the importance of context. In A. Tait, ed., Perspectives on distance education: quality assurance in higher education, selected case studies, pp. 1–8. Vancouver: The Commonwealth of Learning, 1997.

West-Burnham, J. Managing quality in schools. Harlow: Longman, 1992.

 Quality in open and distance learning

What do we mean by ‘quality’?

Discussion: Begin this discussion by distributing pieces of paper, one to each participant, and asking participants to write down — in letters large enough for the group to see —their definition of quality in an educational setting. When participants have finished, ask one of them to collect the sheets and pin them up in front of the group. Draw out the features common to them all, point out the differences, and ask whether there are features to be added. The features mentioned might include the following:

•      chosen standards or criteria;

•      the relative nature of quality;

•      services as well as products;

•      perceptions as well as measured outcomes; and

•      relevance.

Additional points to be made might include the following:

•      Everyone agrees on the desirability of quality.

•      There is less agreement, however, on what quality is.

•      This is because ‘quality’ does not exist in isolation from its context of use.

•      Also, judgements differ according to whose views are being sought; for example,

• there is an amalgam of different meanings under the label quality;

• different stakeholders have different perspectives on quality; and

• different functional areas within a single organisation have different views.

•      Priorities will vary according to

• who is making the assessment; and

• for what purposes the assessment is being made.

Discussion: Ask participants to provide examples of these points about quality from their own experience.

Quality assurance or quality control?

Following is a preview of some terminology that will be dealt with in greater detail in this topic:

•      Quality is a characteristic of the products and services an organisation offers.

•      Quality assurance is a process directed toward achieving that characteristic. It is the set of activities that an organisation undertakes to ensure that standards are specified and reached consistently for a product or service.

•      Quality control operates retrospectively, ‘inspecting out’ or discarding faulty products that fail to conform to a predetermined standard.

•      Quality control and quality assurance, together with the assessment of quality systems — the monitoring, evaluation, and audit of procedures — are overlapping functions in regulating how an organisation or venture works.

All of these tasks have a role in quality management approaches, the best known of which is total quality management.

In summary:

•      quality assurance involves pro-active measures taken to avoid faults;

•      quality control involves re-active measures taken to remove faults;

•      quality assurance plus quality control plus continuous monitoring and evaluation equal total quality management.

Discussion: Do any of these terms or distinctions cause participants difficulties? Examples are always useful in clarifying such terms. Ask participants to provide examples from their own experience as you preview the list.

 Quality assurance in learning institutions 

Why the concern with ‘quality assurance’?

While ‘quality assurance’ may be a recently applied term in the educational context, there is nothing new about educational organisations’ undertaking systematic review and inspection of products and services to ensure their quality.

Discussion: Take this opportunity to solicit examples from participants of the ways in which the processes of review and inspection have been used in their contexts to ensure quality of educational products and services. In addition, almost all the case studies that are included with this kit contain examples of processes aimed at improving quality.

More recent use of and emphasis on the label quality assurance can be attributed to factors such as the following:

•      governments’ interest in return on public investment in education relative to other areas of expenditure;

•      the assertion that education and training is essential to economic recovery, growth, and competitiveness;

•      the assertion that the institutions responsible for education in the recent past have failed in their mission to meet demand because of ivory tower or anti-business attitudes; and

•      insistence that education costs should be reduced and educational organisations made more accountable.

Discussion: Does this list of external factors fit with your participants’ experience? Do they have other factors to add?

The industrial model of quality assurance

Quality assurance has its origins in the manufacturing industry and the military. Initially these ‘quality systems’ emphasised:

•      quality control, those measures taken to remove faults at the end of the production process;

•      the setting in place of systems to obtain better data about discrepancies between proposed and actual performances;

•      the necessity of statistical processes to control non-compliance;

•      using budgets as incentives and penalties to encourage units to tighten up procedures and reduce errors; and

•      using market analyses and client satisfaction surveys to help keep clients interested in the products and services.

Discussion: Ask participants for examples from their own experience of applying these kinds of procedures to improve the quality of products and services. Also suggest they look for examples in the case studies which are included with this kit.

Total quality management

Over the last fifty years, industry has moved increasingly from quality control —measures taken to remove faults at the end of the production line — to a more pro-active process called quality assurance — measures taken to avoid faults. Even these measures were insufficient to enable them to reduce error costs in order to compete in increasingly global markets. Out of this need for more effective ways to increase and assure quality of products and services developed the system known as Total Quality Management, or tqm.

The relationship among these processes is represented in the following ‘equation’.

The Total Quality Management Equation

Quality Control

+

Quality Assurance

+

Continuous monitoring and evaluation

=

Total Quality Management

West-Burnham (1992) outlines the basic points of total quality management as the following.

Basic Points of Total Quality Management

Element	Purpose and Scope
Focus	Internal and external customers
Definition	Meeting customer requirements
Scope	Every aspect of the organisation
Responsibility	Everyone
Standard	Right the first time (fitness of purpose)
Method	Prevention not detection
Measurement	Zero defects
Culture	Continuous improvement

Applicability to education

Discussion: Begin this discussion by asking participants for their thoughts on whether this industrial approach to quality assurance is appropriate for education. Where does the approach fit? Conversely, what problems arise?

Is this industrial approach to quality assurance appropriate to educational institutions? The points of contention centre on two features of the approach:

•      organisational mission; and

•      terminology.

Organisational mission

The organisational mission to be adopted by institutions following a quality assurance or total quality management approach is summed up by Sallis (1993:84) in the following statement. Sallis writes that in a total quality management environment there should be

... a single command for each process — the key processes, whether they are curriculum, pastoral, or administrative need to be charted and organised so that each process is brought under a single chain of command.

The processes in which the providers of education are engaged, however, are teaching and learning, that is, fostering the creation and sharing of knowledge. They may not lend themselves to such a tightly defined organisational mission.

Terminology

Given the nature of the ‘business’ in which educational institutions are engaged, debate also centres on the terminology characteristic of total quality management in particular. Examples include:

•      fitness of purpose;

•      the product;

•      customers and learners; and

•      services.

Fitness of purpose

The term fitness of purpose can usefully force us to ask questions about our ends, for example, about the nature of our audience or the style of our teaching.

Purposes in an educational institution are varied, and in some cases conflict. For example, our job as educators is to facilitate our learners’ learning. At the same time, however, we are expected to enforce certain educational standards of performance, which our learners may fail to meet. No business faces such a conflict.

Oversimplified notions drawn from the business sector and uncritically applied in educational contexts ignore the sometimes contradictory demands of various stakeholders, including:

•        learners;

•        academic and professional interest groups;

•        research funders and practitioners;

•        governments;

•        employers;

•        society at large; and

•        future generations.

Discussion: Ask participants for examples from their own experience of ways in which the interests of these various stakeholders can contradict each other.

The product

The aims of the educational process are to bring about changes in learners’

•      knowledge;

•      skills; and

•      attitudes.

Upon successful completion of the process set out by the educational organisation, the learner may be awarded a credential of some kind.

These outcomes — changes in knowledge, skills, and attitudes and awards of credentials — may be called ‘products’ but they are considerably more complex than are the products of a manufacturing process.

Customers and learners

In quality assurance, all actors within and outside an organisation are customers, providing a service to others.

Unlike businesses, in higher education institutions we have to fail ‘customers’ (learners) from time to time, acting in accordance with other stakeholders such as professional bodies, academic peers, and prospective employers.

Thus elements of formal education, which is not based on the purchase of a service, remain in the relationship.

Services

The ‘services’ provided by educational organisations are as varied and complex as their ‘products’.

Services in support of learning include

•      provision of information to prospective applicants;

•      pre-enrolment counselling and advising;

•      screening of applicants;

•      enrolment and registration;

•      teaching;

•      supporting learning by means of tutoring, counselling, advising, materials provision, libraries, and learning technologies;

•      assessing learners’ performance; and

•      post-course advice and counselling.

Of these services, screening of applicants and assessing their performance in particular set off education provision from other kinds of services.

Discussion: Do your participants agree with these points? What would they add, by way of agreement or disagreement?

 Development of a quality culture 

The importance of organisational cultures

Quality initiatives will inevitably flounder if the organisation implementing them does not take into account its organisational cultures. This requires a recognition that education organisations are not ‘people-incidental’ systems. Rather, they exist critically as the creation of their staff and learners.

It is often assumed that all employees, regardless of their terms and conditions of employment, share the same vision. Rather, education organisations tend to possess a plurality of cultures that significantly influence their operations. Most education organisations can be characterised by at least the following cultures (drawn from Mintzberg 1979 as presented in Lentell 1995):

•      faculties and schools: the ‘professional bureaucracy’; and

•      administration and operations: the ‘machine bureaucracy’.

The ‘machine’ bureaucracy

The administration and operations group is found in all units of the organisation. It includes:

•      secretarial and clerical staff;

•      technicians;

•      warehouse workers;

•      printers; and

•      administrators.

This group is concerned with making the organisational ‘machine’ work on a daily basis. Typically workers in this sector have specialised and segmented knowledge of the business. This means that the power and the ability to see the enterprise as a whole tends to lie with the top managers. This bureaucracy tends to favour rules and regulations. Decision-making tends to flow in a chain of authority. Clerks and other main grade employees tend not to expect to be consulted and have limited discretion over work patterns.

The ‘professional’ bureaucracy

The ‘professional’ bureaucracy comprises the academic staff, who are primarily involved in developing, maintaining, and teaching courses. In having to rely on trained professionals, the organisation has to surrender some power to them. The academic and teaching staff tend to have considerable autonomy. In theory authority is horizontal, that is, between peers. But in practice there are considerable differences in status and authority based on factors such as

•      seniority;

•      academic standing within the wider academic community; and

•      the kind of contract one has, for example, full-time or part-time, term or continuing.

Professionals tend to work independently. Power over the carrying out of work rests with the individual professional, subject only to the collective control and regulation of colleagues. This regulation is based on shared professional values. Their work is normally too complex to be prescribed.

Creating a unified quality culture

A ‘quality culture’ in educational terms is one that puts the interests of the learner and the facilitation of learning at the centre of its activities at every level, constantly striving to improve the effectiveness and efficiency of these activities in every way possible.

The variations in working conditions, tasks and roles, and levels of commitment among employees shape their attitudes to their work and their vision of the organisation. For example, professional staff are more likely than support or administrative staff to be critical of attempts at ‘shared vision’ and more protective of their autonomy.

Tait (1997) suggests that creating a commitment to quality assurance among all employees, whether support or professional, requires managers to recognise the following:

•      the importance of a mechanism of self-evaluation in relation to meeting quality standards, so that professional staff and academic staff in particular feel they own the system;

•      the connected importance of peer review and site visits by external experts accepted as unbiased specialists in the field;

•      the importance of reporting in such a way as to facilitate development and improvement rather than judging or ranking; and

•      relationships between outcomes of a quality review system and funding, which should not be direct and rigid since this will lead to a ‘compliance culture’ rather than a real interest in quality assurance.

Discussion: Ask participants to provide examples from their own experience of variations in organisational culture and how they deal with them in implementing quality assurance strategies.

Checklist for a quality assurance programme

To review, quality assurance focuses attention on operational processes and systems. It has three main elements:

•      You set standards for a product or service.

•      You organise the production or delivery of a product or service so that the standards are consistently met.

•      You thereby create confidence in the client or recipient that what is promised is what will happen.

To implement these procedures, it is helpful to ask the questions in the following quality assurance checklist. (These are taken from a workshop developed for iec by Bernadette Robinson and subsequently published in Robinson 1994:187–88.)

Quality Assurance Checklist

Quality policy and plan

q      Has your organisation developed a policy on quality with which all staff         are familiar?

q      Has this policy been translated into a practical plan?

Specification of standards

q      Are specified and clearly defined standards in place?

q      Have they been communicated to all concerned?

q      Are they specified for key activities?

q      Are they achievable?

q      Are they reasonable?

q      Are they measurable?

Identifying critical functions

q      Have the critical functions for achieving the standards been identified?

q      Have they taken the learner as the starting point?

q      Have the procedures to achieve them been analysed?

Documentation

q      Are the procedures to be followed clearly documented?

q      Are they explicit?

q      Do they represent fact or fiction?

q      Are they consistent in different documents?

q      Are they concentrated on essential procedures?

q      Are they in a readable and user-friendly form?

q      Do all those who need them have access to copies?

Staff involvement

q      Have all staff been involved in the development of quality assurance         systems?

q      Have their suggestions been built in?

q      Has enough time been given to this process?

Monitoring

q      Are there systematic monitoring mechanisms for critical functions?

q      Do they check whether standards are being met and procedures         followed?

q      How do you know?

q      Are the findings disseminated?

q      Are they harnessed to appropriate action?

q      Do they result in improved performance or a review of practice, or a          reappraisal of standards?

q      Do they provide effective feedback loops between providers of         products and services and learners or clients?

Training

q      Is there adequate provision of training and staff development?

q      Is this linked to the achievement of standards?

q      Are there effective mechanisms for assessing training needs?

q      Are these reviewed regularly?

q      Are there resources allocated to meet them?

Costs

q      Is there a strategy for monitoring the costs of implementing and maintaining quality assurance activities?

q      Does this strategy take account of human and financial costs?

q      Are the costs greater than the benefits?

q      Is there a review process to find out?

 Monitoring learner achievement 

Learner achievement and quality assurance

Learner achievement is another aspect of quality in open and distance learning, and monitoring that achievement is correspondingly a tool in quality assurance. We can assess the quality of the processes (the learning experiences) and the quality of the products (the graduates).

Why assess?

Assessment in open and distance learning may have any of three main purposes:

•      formative assessment: to give learners feedback on their progress so that they know how well they are doing and can, if necessary, change the way they are tackling the course;

•      summative assessment: to provide the basis for marks that may contribute to the learner’s eventual certification; and

•      as part of the overall evaluation process: to help the open and distance learning institution to monitor the effectiveness of its courses.

Who should assess?

Assessment may be carried out by any of a number of people, including:

•      the learner him or herself: generally called self-assessment;

•      other learners: called peer assessment;

•     the learner’s tutor: often through tutor-marked assignments that are built into the course;

•      examinations: an examiner or assessor, as may sometimes be the case with summative assessment; and

•      course evaluations: someone else does the assessment, perhaps a researcher evaluating the course.

When to assess

In deciding at which times during your course assessment is appropriate, here are some points to bear in mind:

•      Early in the course, learners may not have learned anything significant enough for testing.

•      On the other hand, an early assignment provides an opportunity for early interaction and feedback and thereby builds the relationship between learner and tutor.

•      Relate assessment to major sections of content.

•      Spread assessment evenly to spread the load and generate regular feedback.

•      Keep in mind the turnaround time and capacity of your tutors.

•      If an assignment is prescribed very late in the course, learners are unlikely to receive feedback before any end-of-course examinations.

Assessment methods

There are a number of possible methods of assessment, each appropriate for testing certain kinds of aims and objectives:

•      diagnosing learning needs: early on in a course, assessment can help learners decide which parts of the course they need most, and may form the basis of a learning contract;

•      checking progress: self-assessment questions during or at the end of study units enable learners to check how they are getting on and provide immediate reinforcement of learning;

•      increasing motivation: reinforcement helps to keep learners going;

•      providing feedback: tutor comments on tutor-marked assignments ensure the learner knows what to do next;

•      encouraging a deep approach to learning: particular types of assessment such as questions that call for reflection, analysis, or application; projects; and practical assignments can help learners improve their approach to learning;

•      facilitating contact between learner and tutor: tutor-marked assignments are often the main point of contact between a learner and his or her tutor, and are therefore an invaluable way of reducing learner isolation; and

•      increasing learner control: giving learners the means to assess their own progress can increase their control over their own learning.

 Practice exercises 

What is quality assurance?

Instructions: Ask participants to take 5 to 10 minutes to try to develop individually their own definition of quality assurance in an educational setting. Then ask participants to share their definition with the other participants in the workshop (depending on numbers of participants, perhaps share definitions first within a small group and then have each group report to the group as a whole the definition on which they have agreed).

Timeframe: Between half an hour and an hour, depending on size of group and extent of discussion required.

Materials required: Flip chart paper and pens if reporting back to large group is required.

What is a quality assurance system?

Instructions: Ask participants to answer the question, ‘How would the structure of your own organisation have to change in order to become committed as a whole to quality assurance?’ As a facilitating device, ask them

•      to pair themselves off;

•      to each draw, for and with their partners, a chart of their organisation as it exists at the moment;

•      to indicate on it using a different colour of marker the kind of activities that are necessary at each level of position to foster commitment to quality assurance;

•      to post these charts on a wall, for description and discussion.

Timeframe: Approximately three-quarters of an hour.

Materials required: Flip chart paper and a variety of coloured markers.