Papers Presented at the 1st National NADEOSA Conference
Held 11-13 August 1999
rd_bds.gif (1931 bytes)

Author:
Dr Liz (ESG) Greyling Bureau for University Education Rand Afrikaans University

Title:
How To Determine The Instructional Quality Of Online Courses: A Case Study
Sub-theme: Courseware Design and Development & Technology

Abstract:
In answer to the powerful argument of why South African higher education institutions should continue to pay the high costs of traditional instructional methods when new technologies can make education available in better and less costly ways, the Rand Afrikaans University (RAU), like many other South African universities, has introduced a number of online courses at undergraduate as well as postgraduate levels. These courses are offered in addition to reduced contact and distance courses in all faculties. The primary task of the Bureau for University Education (BUE) is teaching development. This being the case, the BUE is involved in assessing the teaching and learning in all study programmes and courses offered by the university. Since the introduction of online courses, the BUE was therefore also faced with assessing such courses to determine if they were instructionally sound and if they met institutional, faculty, departmental, lecturers’ and students’ needs. This paper deals specifically with a case study on the assessment outcomes of two Engineering courses. It aims to highlight the assessment procedure and compare specific outcomes with trends emerging from research done for the context of the paper.

About the Author:
Liz Greyling, a Senior Researcher at the Bureau for University Education, RAU, is currently responsible for the assessment of Web-based, reduced contact and distance courses; teaching-learning development; editing; and lecturing Hons and M-students in applied Linguistics. She holds a BA Languages and Senior Teaching Diploma (STD) from the University of Stellenbosch, and a BA (Hons), B.Ed (Hons), M.Ed (Educational Management) and D.Ed (Tertiary and Adult Education) from the RAU. Special fields of interest: Developing and assessing study material for technology-based distance education.

rd_bds.gif (1931 bytes)

INTRODUCTION

Students and lecturers in South African higher education institutions are increasingly being exposed to the virtual world’s potential as a teaching, learning and research resource. It is widely believed that virtual education will have major advantages for and will fit into the policy framework for South African higher education. Technologies spearhead serious institutional reform because they create real change, especially in the area of course content and teaching. However, their successful use depends on the quality of the teaching staff and the excellence of the teaching and learning programme (Wydeman, 1996:10; Privateer, 1999:62). Across all sectors of the higher education landscape, institutions continue to struggle with key aspects of IT planning and infrastructure: developing a strategic and financial plan for it, planning curriculum integration, and providing adequate user support. According to Ehrman (1999:27) and Green (1997:1), the cost of putting a course online is affordable but requires advanced faculty skills. Whether this be true or not, universities worldwide, as well as in South Africa, have timeously embarked upon introducing online courses. Such courses will cater for larger and more diverse student populations, thus affording an increasing number of students access to higher education.

The BUE of the RAU assessed the online courses offered at the university. The assessment outcomes of two Engineering courses, namely Electromagnetics 3A and Electrotechnics 2A, will be discussed in this paper. Both courses are structured in the form of a combined teaching-learning programme, consisting of 80% contact and 20% computer-based learning, thus combining working with the learning content with e-mail responses. The computer-based part of the course therefore amounts to 20% of the students’ total formal assessment marks.

ASSESSMENT RATIONALE

The main objective of this assessment was to determine the usefulness of the computer component of the courses. The lecturers were responsible for formulating the critical and specific outcomes of their courses, ensuring that learning takes place and for designing adequate assessment strategies. They needed to formulate learning outcomes that would be consistent with how teaching technology could be integrated throughout the course curricula, preparing students for what they would need to demonstrate on completion of the courses. What was needed, was re-engineering of what was being done and re-inventing of what was produced. Both lecturers agreed that student competence with technology had become essential for the courses (compare Ehrman, 1999:26/27; Privateer, 1999:63-66).

ASSESSMENT PROCEDURE

At the request of both lecturers, discussions were held to determine the format of a proposed questionnaire. BUE staff provided a clear understanding of the aim of the proposed questionnaire, namely to obtain important information on how the Engineering student experienced the computer-based part of the course. It was also explained that

honest, anonymous student opinions would be of great value for course development in the department and that feedback should be viewed as perceptions of students, being the "clients" doing the course. Subsequently, a data bank of 50 questions was compiled and structured in three sections:

Responses anticipated for Section C of the questionnaire included answers to the following questions (compare Ehrman, 1999:29):

The lecturers were advised to select no more than 25 questions, to alter where they deemed necessary or add questions they wanted to include. The two lecturers use different teaching strategies and styles and therefore selected different (though still 25) questions. Electromagnetics 3A is strongly based on a computer study guide, while Electrotechnics 2A is strongly based on practical exercises. Both lecturers use WebCT software. Questionnaires were handed out to students during their usual contact session. A total number of 72 students completed the questionnaires.

ASSESSMENT OUTCOMES

The assessment produced a mix of good and bad results. Outcomes were reported in writing, and subsequently discussed with both lecturers (Van Tonder, 1999).

Electromagnetics 3A

A total number of 34 students completed the questionnaire.

Section A: The lecturer selected 18 questions from the databank of 39.

Outcomes

Only half the respondents (50%) agreed with the following statements regarding the design of the programme (compare figure A):

Q1: Sufficient training was provided for using the computer programme.

Q2: The self-assessment questions in the programme helps students to understand the learning content better.

Q11: The volume of learning content is manageable.

Q14: The students gain basic computer competency through using the programme.

Section B: The lecturer selected 5 questions from the databank of 8 questions.

Outcomes

The students rated the statements from very important to definitely unimportant, as follows (compare figure B):

Q19: Only half the students (50%) rated the study guide as important to prepare for contact sessions.

Q20: Students did not regard the study guide as important when preparing for practicals.

Q21: Students regard the computer-based study guide as important when preparing for tests.

Q23: Students feel that because of the reduced contact time, the study guide is very important.

Section C: Students completed all 3 open-ended questions from the databank.

Outcomes

Q24: A total number of 8 (23,5%) students indicated that they had printed a hard copy of the computer-based study guide, while 26 (76,5%) did not print a copy of the study guide.

Commentary was as follows:

Q25: A total number of 17 (50%) students commented as follows on the course in general:

Electrotechnics 2A

A total number of 38 students completed the questionnaire.

Section A: The lecturer selected 17 questions from the databank of 39 questions. The lecturer altered Q2 to include tests and added a question on the linking of class and computer work (Q18).

Outcomes

The students did not agree with the following statements regarding the design of the programme (compare figure C):

Q3: Sufficient computer access is available.

Q14: Students can control their learning pace (with reference to the completion of assignments).

Section B: The lecturer selected 6 questions from the databank of 8 questions.

Outcomes

The students rated the statements on the use of the computer-based study guide from very important to definitely unimportant, as follows (compare figure D):

Q19: Students rate the study guide as important to prepare for contact sessions.

Q20: Students rate the study guide as very important when preparing for tests.

Q24: Students feel that because of the reduced contact time, the study guide is very important.

Section C: Students completed all 3 open-ended questions from the databank.

Outcomes

Q25: A total number of 23 (60,5%) students indicated that they made use of the allocated computer time in the computer centre, while 15 (39,5%) did not use the allocated time. Commentary was as follows (compare figure E):

Q26: A total number of 22 (57,9%) students indicated that they had printed a copy of the computer-based study guide, while 16 (42,1%) did not print a copy of the study guide. Commentary was as follows:

Q27: Students commented as follows on the course in general:

TRENDS EMERGING FROM RESEARCH COMPARED TO SPECIFIC OUTCOMES OF THE ASSESSMENT

When studying the outcomes of this assessment, useful trends to consider when planning to assess online courses, emerged. These trends echo research findings that were studied for the context of this paper (compare Green, 1998:1; Van Ryneveld, 1998:49/50; Ehrman, 1999:26-29); Privateer, 1999:61-75). Computer-based courses:

CONCLUSION

The computer extends the traditional face-to-face classroom to include a virtual classroom. Real education is to excite students’ interest and Virtual Reality has the possibility to foster better learning outcomes (Van Ryneveld, 1998:54; also compare Brown, 1998:7 and Ehrman, 1999:29). In addition, interactive classrooms also offer ‘telepresence’ - the feeling of being present in a classroom with a teacher and other students where there is considerable opportunity for human intellectual and social interaction during the teaching and learning process (Hegarty, Phelan & Kilbride, 1998:3). I would like to conclude with two quotations:

"There are ‘... three areas where computers are more skilled than human minds: [they] are evaluations of computations, storing massive amounts of data, and remembering things without forgetting’" (Privateer, 1999:76).

"Technology - with its unpredictable changes and vast expenditures - has made flying blind much more dangerous than it used to be. The good news is that [some] institutions are learning to see" (Ehrman, 1999:29).

REFERENCES

BROWN, S. 1998. Use the Internet to teach! Who? Me? Pro Technida, 15(2): 5-15.

EHRMAN, S.C. 1999. Asking the hard questions about technology use and education. Change, 31(2): 25-29.

GREEN, K.C. 1998. Colleges struggle with IT Planning. The Campus Computing Project: 1998 National survey of information technology in higher education [http://www.compuscomputing.net.ericir.syr.edu/Projects].

HEESE, M. 1998. Developing an interactive multimedia academic reading skills course for naive computer users. Progressio, 20(2): 50-64.

HEGARTY, M., PHELAN, A. & KILBRIDE, L. 1998. Classrooms for Distance Teaching and Learning: A Blueprint [http://www.kuleuven.ac.be/upers].

PRIVATEER, P.M. 1999. Academic technology and the future of higher education. The Journal of Higher Education, 70(1): 60-79.

VAN RYNEVELD, L. 1998. Virtual reality in the adult learning environment. Vital, 12(1): 47-54.

VAN TONDER, A.H.D. 1999: Kursusterugvoer: Internetkursus in Elektromagnetika 3A en Elektrotegniek 2A (Departement Elektriese en Elektroniese Ingenieurs-wese). Randse Afrikaanse Universiteit: Buro vir Universiteitsonderwys.

WYDEMAN, J.L. 1996. Evaluation of a real-time broadcasting model for distance education. A NETECH report on a pilot project. Pretoria: HSRC.

 

rd_bds.gif (1931 bytes)

NADEOSA HomepageIst NADEOSA Conference Papers Index  |
Global Distance Education Network : South African Resources

rd_bds.gif (1931 bytes)
Send comments on the web site to the web designer