Management Benefits and Costs
Cost Effectiveness

Basic English Course Evaluation
Mauritius College of the Air
Distance Education Division
I Jheengut

Context:

The aim of this evaluation is to:

  1. determine the relevance of the course materials and suitability of the delivery methods.
  2. decide on the future of the Basic English course

Source:
Collected from MCE a SAIDE country visit to Mauritius in 1999

Copyright:
Reproduced with permission.

Title

TABLE OF CONTENTS

1. Background

1.1 Objectives of the Basic English Course
1.2 The Course Team
1.3 Course Outline
1.4 Activities and Self-Marked Assignments
1.5 multiple-choice Questions
1.6 The Tutor-Marked Assignments (TMAS)
1.7 Administrative Arrangements for the implementation of the course
1.7.1 Face to Face Sessions
1.8 Assessment

2. Monitoring

3. Summative Evaluation

4. Findings

(i) Views of the tutors
(ii) Views of the Employer
(iii) Evaluation Questionnaires
(iv) Examination Results

Conclusion

Annexures

 
1. Background

The Mauritius Telecom (MT) requested the Mauritius College of the Air (MCA) to develop a Basic English Course for its manual grade employees.

These employees:

  • had formal schooling only up to standard VI
  • lacked writing and reading skills

They needed training in written and some spoken English in order to be able to cope with the situations that cropped up in their workaday life. The MCA therefore proposed a flexible learning Basic English Course to meet that educational need. The duration of the course was 24 weeks.

1.1 Objectives of the Basic English Course

The target group had uniformly studied up to Std VI at primary level. While they had all passed the CPE, the level reached by each individual had not been gauged. The core objectives of the Basic English Course were as follows:

  • to raise the level of the target group's written English and to some extent spoken English.
  • to develop the group’s problem-solving skills in English
  • to develop skills of connected prose writing
  • to enable the students to cope with daily situations involving usage of English

1.2 The Course Team

A team of seven persons worked on the project:

  • Mr I Jheengut, Senior Lecturer, MCA
  • Mrs S. Gokool-Ranidoo, Lecturer, MCA
  • Miss T. Beeharry, Lecturer, MCA
  • Mr Karl Abraham, part time course writer with a wide experience in the teaching of English for special purposes, having already taught at English bridging course to a group of MT workers.
  • Mr Clency Sooben - course advisor, Associate Professor at the MIE. His considerable pedagogical experience helped in an area where an informed vision about what is required is a basic necessity.
  • Mrs Laleeta Chummun - Tutor for running the face to face sessions. She is presently teaching English in a Secondary school.
  • Mrs Pratima Jugoo - Tutor for running the face to face sessions. She is one of our tutors for the A-Level distance education courses.

1.3 Course Outline

The course was developed at MCA under the leadership of Mrs S Gokool-Ramdoo. It consisted of 4 modules. The modules included exercises pertaining to:

a) Comprehension
b) Visual stimulus and
c) Creative writing
d) Problem solving exercises

The course material was deemed to be self-sufficient and appropriate for the independent learner

1.4 Activities and Self-Marked Assignments

A lot of emphasis was laid on active learning. Activities were included to reinforce the above. These activities which acted as an aid to learning involved mainly thinking and writing.

At a distance, one way of stimulating this mental activity was through questions

The questions of the comprehension and visual stimulus exercises were meant to help keep the learners on the alert: they directed attention to whatever was going on in the passages.

Some questions (e.g. the creative writing ones) were designed to get the student draw on his own experience. These were not meant to test memory but to encourage pause and reflect.

Some exercises were simple in form and required considerable thought e.g. filling of gaps in sentences (Cloze test)

1.5 Multiple-choice question

Such exercises were considered suitable for adults whose understanding was considerable but whose experience of writing connected prose was limited.

Activities of this sort provided a constant feedback for the independent learner on his progress. Immediate feedback to the questions was provided at the end of each chapter.

1.6 The Tutor-Marked Assignments (TMAS)

Tutor- marked assignments were also included in the course. These had 2 functions:

  • they opened up the possibility of a genuine interaction between the student and the tutor who was responsible for guiding him through the course and who came closest
  • they provided feedback on the progress of individual students and about the suitability of the course material to the course writers and producers.

They were more demanding than the Self-Marked Assessments (SMAs). The TMAs were worked out by the learners and submitted to the tutors according to a calendar which specified target dates. After marking the tutors return the assignments to MCA for recording purposes before being despatched to the learners.

1.7 Administrative arrangements for the implementation of the course.

The course was started on a pilot basis for 50 learners

1. 7. 1 Face to Face sessions

At a meeting with the above team, it was decided that the course tutors would meet the students on a fortnightly basis for face-to-face sessions to discuss learners’ problems, introduce new ideas/concepts. There were twelve sessions, each session was of 2hr duration. Two tutors ran the sessions, each with a group of 25 learners.

1.8 Assessment

The assessment was through:

  • tutor-marked assessments and
  • a final examination

A written examination was held at the end of the course, on 30 September 1997, The weightings were as follows:

  • 80% for the examination
  • 20% for TMAs

A combined score of 40% qualified the learners for the award of the certificate. The TMAs and the final examination were designed to test how far the above objectives have been reached. Results were proclaimed in November 1997

2.0 Monitoring and Evaluation

During the developmental stages, contacts with the prospective learners and the MT ensured that their needs were being catered for. A sample of the material was protested with a group of students. In the light of this, relevant modifications were made. The course was then run on a pilot basis for a group of 50 students, from May - September 1997.

The course delivery was continuously monitored closely by MCA staff. Progress of the learners was followed and remedial action taken when necessary.

3.0 Summative Evaluation

An evaluation exercise was carried out at the end of the course to determine the:

  • Suitability and relevance of the course
  • Suitability of the delivery method
  • impact of the course

The following form important aspects of this exercise:

(i) views of the tutors were sought through dialogues and through a written report regarding:

  • course suitability of the delivery method
  • the duration
  • the assessment methods
  • the attitude of the students
  • their briefing/ training prior to tutoring on the course,
  • the support they got from the MCA and the MT

(ii) views of the employer, MT, were sought through dialogue and through a written report regarding:

  • course content, whether they could cope with it,
  • method of delivery
  • tutoring
  • new knowledge and skills acquired as a result of the following course
  • future of the course.

A sample of the questionnaire is at annexe I

(iii) views of the students through a group meeting and through individual questionnaires regarding the:

  • course content, whether they could cope with it
  • method of delivery,
  • tutoring,
  • new knowledge and skills acquired as a result of following the course,
  • future of the course.

A sample of the questionnaire is at annexe I

(iv) results of the TMAs and the final written examination.

4. Findings

(i) Views of the tutors

  1. The views of the tutors have been very positive. They have found the Basic English appropriate for the target audience and the target audience and that it was properly designed to improve the literacy level of the learners. This course was principally designed to improve reading and writing skills. However, one relevant recommendations was to examine the possibility of introducing more oral activities to improve oral skills.
  2. The duration of the course was reasonable although one tutor has expressed the view that the time scale was limited.
  3. The learners were selected by the Mauritius Telecom. Most of the learners were enthusiastic and attended the face to face sessions regularly and they took part in the activities. They showed a positive attitude towards the course as well as to the tutors. They were eager to complete the course and start another one of a higher level.
  4. The tutor felt that the assessment methods were well devised. However, performance could have been if the face to face sessions were held in the morning.
  5. They were satisfied with their initial briefing/ training.

(ii) View of the Employer

The Mauritius Telecom has rated the course materials as being appropriate and relevant to the daily work of the learners. It also acknowledges that the delivery method was the most cost-effective one. However, MT felt that more frequent face to face sessions could have been optional for the learners

MT attributed the nine initial drop-outs to a decrease in the level of motivation. Learners were enrolled on the basis of a survey carried out

(iii) Evaluation Questionnaire

Being given the academic level of the course, a very simple questionnaire was used. The questionnaires were sent to the forty-one learners. Thirty-eight completed questionnaires were returned showing a response of 92.7%. The questionnaire used was straight-forward and easy to fill in. It consisted of nine items.

  1. 90% of the learners found the course books easy to read and understand
  2. They unanimously a-reed that the face to face sessions were helpful and the tutors were friendly, and supportive.
  3. 85% of the learners could work through the questions at the end of the unit by themselves.
  4. 90% of them felt that their English has improved since
  5. 85% of them can now write letters and short reports in English.
  6. 88% of them can now follow instructions in English
  7. All of them have expressed their wish to follow a more advanced course in English.

(Detailed findings annexe 1a)

(iv) Evaluation Results

Out of the forty-one learners, forty were present for the written examination and one was absent. It is to be noted that nine learners abandoned the course at the start. Using the combined assessment based on continuous assessments and the final written examination, thirty five learners have completed the course successfully. Five learners failed to achieve a combined score of 40%. These figures indicate a success rate of :

35/40 x 100 = 87.5%

CONCLUSION

The employer was satisfied with the course The course was found to be relevant although, MT has not formally assessed the impact of this course on the daily work of the learners, it has been observed that some of them can hold simple conversations in English.

The learners have expressed their satisfaction regarding the course and method of delivery. They had expressed their wish to be able to enrol for a more advanced course in English. They were very enthusiastic and they felt that their level of English had improved.

The tutors had expressed their satisfaction regarding the course content relevance and method of delivery. They have suggested that more oral work could be included to improve the speaking skills of the learners. They have also signalled the benefit of "more face to face sessions". The tutors feel that their own training/briefing session was well done.

The examination results showed a success rate of 87 5%, exclusive those who dropped out after one week - a very satisfactory performance. The learners showed a high degree of commitment to complete the course successfully as evidenced by attendance.

 Monitoring reports indicate that the tutors have shown a high degree of commitment and professionalism in their work. This has contributed to the success of this course. There was also commitment and valuable collaboration of senior staff of MT to make this course successful.

Recommendations for the future

The Suggestion to include more face to face is probably justified on account of the academic level of the learners. This can be examined for other similar groups of learners for this course. However, this would mean more resources and higher costs. The inclusion of more oral work is also a valid suggestion for the same reason. The remark from MT regarding adequate advance preparation is very useful. For successful distance learning, the learners must be prepared mentally to undergo the learning experience.

This course was designed and developed, with the MT context in mind. References are to the MT work environment. If we are to offer this course to other group of learners, some adaptations will be required

ANNEX 1

BASIC ENGLISH COURSE EVALUATION QUESTIONNAIRE

  1. Did you find the course book easy to read and understand?
  2. Were the face to face sessions with your tutor helpful?
  3. Were your tutor friendly and supportive?
  4. Were you able to work through the questions at the end of each unit by yourself?
  5. Do you feel the course has helped you to improve your English?
  6. Can you write letters and short report in English?
  7. Can you now follow instructions in English?
  8. Would you like to follow a more advanced course in English?

Any other comments which would help improving the Basic English Course.
____________________________________________________________
____________________________________________________________
____________________________________________________________

BASIC ENGLISH COURSE EVALUATION QUESTIONNAIRE

  1. Did you find the course book easy to read and understand? TOTAL
    2. YES IIII IIII IIII IIII IIII IIII IIII II 37
    NO I               1
  2. Were the face to face sessions with your tutor helpful?
    3. YES IIII IIII IIII IIII IIII IIII IIII III 38
    NO                  
  3. Were your tutors friendly and supportive?
    4. YES IIII IIII IIII IIII III IIII IIII III 38
    NO                  
  4. Were you able to work through the questions at the end of each unit by yourself?
    5. YES IIII IIII IIII IIII IIII IIII IIII   35
    NO II               2
  5. Do you feel the course has helped you to improve your English?
    6. YES IIII IIII IIII IIII IIII IIII IIII II 37
    NO I               1
  6. Can you write letters and short report in English?
    7. YES IIII IIII IIII IIII IIII IIII IIII 35
    NO II             2
  7. Can you now follow instructions in English?
    8. YES IIII IIII IIII IIII IIII IIII IIII I 36
    NO I               1
  8. Would you like to follow a more advanced course in English?
    9. YES IIII IIII IIII IIII IIII IIII IIII III 38
    NO                  

 

Cost-Effectiveness • Benefits and Costs • Management • Return Home