CHAPTER ONE
INTRODUCTION TO EVALUATION

THE MPUMALANGA MASTER OF MANAGEMENT

The Mpumalanga Master of Management is one of the programmes of the Graduate School of Public and Development Management (P&DM), which was established in its own facilities, as part of the Faculty of Management in 1993.

The vision of P&DM is:
Leadership through learning for
- democratic governance
- social transformation and
- participatory development in an African context. (P&DM.(1996). Strategic Plan 1996/97 Draft: November 1996. Johannesburg: P&DM)

Its four strategic thrusts are:
- Academic and Specialist Programmes
- Staff Development – Academic Staff
- Institutional Development
- Strategic Positioning.

It is important also to see the Mpumalanga Master of Management Programme in its context in the Mpumalanga Management Programme (MMP) as a whole. Although described by the programme manager as the most important project, the Master of Management is only one of several projects in the MMP.

The objective of the MMP is not to develop a satellite campus but to extend its existing academic programmes to Mpumalanga via distance education for a five year period so as to provide management capacity building, training and technical assistance in the region during a period when indigenous institutions are establishing and building more permanent capacities.(de Coning, C. (1998). Annual progress and Status Report of the Mpumalanga Management Programme (MMP) June 97 to May 98. Nelspruit: MMP.)

The Mpumalanga Management Programme (MMP) was initiated and established under the auspices of the Mpumalanga Provincial Government (MPG) and the Graduate School of Public and Development Management (P&DM), following initial discussions especially between Mark Swilling, Jeets Hargovan, and Matthews Phosa, during 1996. The initiative also has the funding support of the private sector, notably Gencor, now Billiton, and Liberty Life who have respectively approved funding for the programme and the appointment of a Liberty Life Chair in Development. The establishment of the MMP initiative follows a significant demand for public and development management training in the local setting.

Following the debate on an envisaged University in Mpumalanga, the MMP has been established to support both the university initiative in the short to medium term as well as to respond to immediate demands for training in the Province. The MMP presents both the Master of Management as well as specific certificate courses. In addition, the MMP became involved in other initiatives such as a partnership with the University of Alberta (Canada) to present a project leaders’ course (funded by CIDA) and the Maputo Corridor and will also develop an analytical capacity to support the province with its technical assistance endeavours. The MMP with all its sub-projects went into operation on 1 June 1997, with appointment of a full time senior academic based in Nelspruit and the setting up of local facilities.

The Mpumalanga Master of Management Programme itself started in July 1997.

The main teaching and learning strategies in the programme are lecture sessions (some delivered via PictureTel videoconferencing technology), course packs, learning groups, and assessment by means of individual and group assignments and tests, and formal examinations at the end of each block. These strategies are complemented by locally based tutoring (compulsory courses, electives and research reports), workplace observation from the second year and self-managed learning groups.

There are four compulsory courses:
1. Paradigms for Public and Development Management;
2. Organization and Management;
3. Information-Based Policy Analysis and Decision Making; and
4. Resources Management.

Students are also required to choose three electives from a list of 33 electives. Numbers of students opting for particular electives determine which of these will be offered in any given year.

In the last six months of the course, students are expected to complete a research report in which they apply analytical and applied research techniques to the topic chosen from their area of specialization. Students are assigned advisors who supervise and guide the formulation, preparation and completion of the research report.

The concentration of the evaluation is on the third compulsory course, but in the interviews of staff and students, comments on the programme as a whole are elicited.

This evaluation concentrates on the effectiveness of the distance education practices in the part-time Mpumalanga Master of Management degree delivered over two years from 8 July 1997. The P&DM Master of Management has been offered at Parktown for a number of years. Because of delays in negotiations, MMP started six months later than at first intended. The effect of this is that a group of 9 students had to start the Masters Programme in Parktown, and then in July 1997 transferred to Mpumalanga. These students are referred to as the second year group, and the group that began on 8 July 1997 as the first year students. There are currently 27 first year and 9 second year students involved in the Mpumalanga Management Programme.

The major question addressed by the evaluation is:
What are the key lessons of experience regarding distance education practices, modes of delivery, and areas of improvement when compared to similar initiatives?

A subsidiary but important question is:
Are the Mpumalanga students getting a comparable learning experience to the students in Parktown?

EVALUATION PROCESS

DATA COLLECTION

Data for the evaluation was collected in the following ways:

1. Review of research and existing South African practice in the use of conferencing technologies (see Appendix One).

2. Review of documentation about the programmme (publicity documents, previous evaluations, as well policy and planning documents).

3. Interviews with Mpumalanga Management Programme staff:
- Prof. Christo de Coning as programme manager of the MMP (31/07/98)
- Cathy Churchill as teaching and learning coordinator of the MMP (19/08/98)
- Enoch Leputu as technical assistant for the MMP (25/08/98).

4. Interviews with staff involved in teaching on the programme:
- Mike Muller (25/08/98)
- Kirsten Harrison (08/08/98)
- Petal Thring (08/10/98)
- Christo de Coning (19/08/98).

5. Telephone interview with Mark Swilling as being involved with the initial conceptualization of the MMP (4/10/98).

6. Interview with a class representative (Jasper Ngobeni) of the Mpumalanga MM1 group (08/08/98).

7. Focus group interviews with the Mpumalanga MM1 student group (18/08/98, 19/08/98).

8. Interview with Linda Mwale, class representative of the Mpumalanga MM1 Group (19/08/98).

9. Questionnaires submitted by Learning Group leaders of MM1 and MM2 students (at intervals during September 1998).

10. Observation of lecture sessions:

- 28/07/98: Mike Muller (Probability Modeling) – PictureTel (Nelspruit & Parktown classes)
- 08/08/98: Kirsten Harrison (participatory rural appraisal) – face-to-face (Nelspruit)
- 18/08/98: Mike Muller (Linear Regression) – PictureTel (Nelspruit class only)
- 19/08/98: Christo de Coning (Intro to Public Policy) – face-to-face (Nelspruit)
- 25/08/98: Mike Muller (Statistical Forecasting) – PictureTel (Nelspruit & Parktown)
- Videocassette recording: Mark Swilling (Systems Theory) – PictureTel (Nelspruit class).(See Appendix Four for copies of the observation reports).

11. Review of course packs for the third compulsory course – Information-based Policy Analysis and Decision-making.

12. Review of assignment design and a sample of the individual and group assignments submitted by MM1 students for the course ‘Information Gathering for the Policy Process’.

13. Review of the MM1 and MM2 student profiles and assessment results.

A diagrammatic representation of the evaluation follows.

The effectiveness of the teaching and learning practices, as well as the appropriateness of the blend of methods to serve the needs of the students at a distance in Mpumalanga, are evaluated in terms of:
- access and equity;
- interactivity; and
- support.

Although slightly differently defined in the various chapters, broadly speaking access and equity concern: how accessible the programme is to the students (one of the key issues in distance learning is opening access); and how equitable their learning experiences are when compared with the Parktown group. Interactivity includes the extent to which programme and instructional design processes have exploited technological interactivity (such as use of technology-enhanced opportunities for networking provided by e-mail); educational interactivity (such as use of collaborative learning approaches); and social interactivity (such as informal contact and support between teachers and facilitators). Support is vital in any programme making use of distance methods where the isolation of learners is a key factor hindering learning. Where PictureTel is used, the role of the facilitator in Mpumalanga is important in providing support to students. Staff members also need support in order to use the technology effectively.

The assessment practices in the programme are evaluated in terms of their effectiveness in helping students achieve the purpose and objectives of the programme (an access and equity issue), as well as in terms of the kind of support that is provided to students to meet the requirements of the assessment. Issues of quality assurance are also looked at.

The distance education practices in this programme are compared with others mainly in Appendix One – An Educational Perspective on the Use of Conferencing Technologies. In this appendix, national and international research on the use of conferencing technologies is reviewed, and criteria are drawn out. Case studies of the use of conferencing technologies at three South African universities are included.

CONCLUDING REMARKS

It was clear from the documentation as well as from interviews, that there is a resistance to distance education, as it is perceived to be a strategy for mass education, rather than for intensive postgraduate programmes. At the same time it was regarded as essential for the programme to be offered at a centre other than Parktown – for maximum strategic impact, as well as for the sake of the Mpumalanga students, who would be able to study while remaining in close contact with their workplace. Therefore, the programme uses certain distance education methods (such as videoconferencing technology), but every effort is made to retain the ‘customized personal relationships’ of a face-to-face programme.

We will argue that lessons from distance education experience can be applied to increase the quality of the teaching and learning in this programme. Good distance methods are designed to increase the individual attention to students at a distance, but if effort is expended mainly on replicating face-to-face methods with distance students, quality often suffers.