Papers Presented at the 2nd National NADEOSA Conference
Held 21-22 August 2000

Author:
JF van Koller

Title:
Online instruction at Technikon SA: An evaluation of a course in online instruction offered to TSA academic staff members

Introduction

The changing educational economic and environment where the use of high-tech technologies has already become a normal practice, is increasingly forcing institutions to introduce technology-based forms of instruction.

It is becoming clear that information technology is increasingly playing a prominent role in the educational process. In distance education in particular technology has already become more than merely a means by which instruction should take place. Technology is increasingly becoming the most important method by which distance education is conducted as indicated by Hughes et al (1997:5). Hughes et al also indicated that competence in the use of instructional technology, has become one of the core competencies DEP should be equipped with.

Maier, Barnett, Warren & Brunner (1996: 1 - 14) discussed the following reasons for the increasing importance of and need for the use of technology in the teaching/learning process:

• Pressure from changes in the student composition (e.g. enormous increases in student numbers, changing entry requirements, intakes of more mature students, overseas students, part-time students, and so forth).

• Pressure from new public expectations from higher education, e.g. that higher education institutions should offer types of training and qualification that are relevant for future employment prospects.

• Pressure from the impact of technology on the production and dissemination of knowledge itself (e.g the enormous multiplication in knowledge based in all discipline/subject fields increasingly require more efficient methods to manage and disseminate that knowledge, and technology is an important way of doing so).

• Developments in educational research which suggest more effective and efficient approaches to teaching and learning of which technology-based instruction forms a core component.

The USA government has been investing huge sums of money in the large-scale implementation of technology-based forms of instruction. There is for example currently a project underway to connect each classroom to the Internet.

Various distance education institutions have already made great advances in the field of technology-based/online forms of instruction. Many traditionally residential institutions in South Africa, such as RAU, Stellenbosch, UP and others are increasingly implementing distance education modes of instruction at their institutions. The main means of instruction they seem to prefer, is technology-based instruction, and in particular online instruction.

Research has already shown numerous advantages of online instruction. Some of the most common advantages are as follows:

• It helps to take the distance out of distance learning.
• Greater interaction between facilitator and learner is made possible.
• Faster delivery of course contents.

In view of the above scenario, TSA realised that it was of crucial importance to equip their academic staff with the required instructional technology skills in order to implement technology-based instruction effectively. That is the reason why the TELES Dweb Course, which will be discussed in the remainder of this paper, was introduced.

• To evaluate the effectiveness of the course.
• To determine whether the course resulted in the acquiring of necessary skills among our DEPs.
• To determine whether there were any major problems or obstacles in the offering of the course.

• A questionnaire/evaluation form was prepared.
• Two separate evaluations of the course were conducted.
• The findings of the two evaluation projects were compared and certain conclusions reached.

This course was aimed at training and equipping educators and trainers in the skills of online/web-based instruction.

1.1 The main contents of the course (syllabus)

The course consists of the following eight modules:

• Module 1: Your Online Classroom, The Technology and Network Learning
• Module 2: Instructional Design and Network Pedagogies
• Module 3: Web Multimedia Attributes and Their Use for Learning
• Module 4: Teaching and Moderation in the Online Classroom
• Module 5: Assessment and Evaluation of Online Students
• Module 6: Developing the Course: Producing and Assembling Course Components
• Module 7: Class Presentation of Instructional Design Templates Projects
• Module 8: Presentation of Instructional Design Templates, Formative and Summative Evaluation, Conclusion

1.2 Composition of the online campus

The online campus consists of the following main sections or components:

• The conference section: This is the main and most important section of the campus since most of the discussions, exercises and activities take place here. Participants are required to read the first message of the instructor which provides an introductory discussion of the lesson/module and the tasks or assignments for the week. Participants must also read and comment on the messages of other participants. The conference section also provides a facility for the submission of assignments.

• The workspace section: This section is mainly used for the activities and exercises that require a lot of practice and experimentation. Participants can use this section to practice and overcome any difficulties that they might have. There is also a submission box for the submission of assignments and tasks in addition to a similar facility in the conference section.

• The information section: This section provides participants with basic information on the campus, the course(s) offered and where to find different sources of information.

• The courses section: This section provides information on: the different courses offered; the syllabus of each course; the grades each participant has obtained at every stage of the course and a section where participants could evaluate the past week’s session and activities.

• The library: The library provides different references and also allows access to different sources in a direct fashion (e.g. by including web-addresses or direct links to certain sources). The library also allows direct access to different search engines on the worldwide web.

• The U-Chat section: This section is used mainly for real time (synchronous) communications or chats. Participants are required to utilize this facility at a fixed time when everyone can participate at the same time.

• The café section: This section allows for the initial introduction of each participant in the course. It also allows opportunities for informal chats and requests for assistance. However, the café communications are also asynchronous.

• The administration section: This section is used for the setting of participants’ own profiles on aspects such as password, e.mail address, language preference and so forth. Participants could change any of the above aspects if the need arises. They can also use this section to send feedback on the campus structure to the system developers.

• The gallery section: This section contains some examples of multimedia and other images and can be visited by participants for interest sake.

• Every Monday morning, Lucio Teles, who is the main instructor/facilitator, presents a discussion on the week’s lesson or module.
• The messages on CD should be read in conjunctions with Teles’ discussion.
• His discussion includes the tasks or activities for the week. There are both individual and group tasks.
• Participants are required to discuss Teles’ message individually and also to comment on the messages of other participants.
• Individual tasks consist mainly of reading exercises and the consultation of references and responses to such references.
• Responses are posted either in the conference section or the workspace section.
• Participants are divided into different groups early in the course and they have to work jointly on specified tasks or activities.
• A high emphasis is placed on collaborative learning.
• At the end of the week each participant must submit an evaluation of the week’s module and activities and what they have learnt.

Student assessment is based on the following criteria:

• Time spent on the campus
• Participation in the form of messages and comments
• The effective use of multi-media in messages
• Quality of responses
• Speed of responses
• Group-work

I sent out an assessment form to each of the 13 participants in the course. However, I only received back 9 completed forms.

Questions:

1. Did you learn, anything new from the Teles D-web Course?
2. What were in your view the major shortcomings of this course?
3. What were in your view the major strengths of this course?
4. Would you be able to design and develops an online course on the basis of the skills you acquired from the course if the necessary infrastructure is available? Motivate your answer.
5. Would you recommend other staff members to attend the course? Motivate your answer.
6. Are there any recommendations you would like to make regarding the improvement of the course?

Analysis of questions (NB: 9 = 100%):

Below is an analysis of the responses to each question:

Question 4: Would you be able to design and develops an online course on the basis of the skills you acquired from the course if the necessary infrastructure is available? Motivate your answer.

4. Findings

The main findings I reached on the basis of the responses to each question, were as follows:

• 89% of the respondents felt that they have definitely learnt something from the course.
• 11% of the respondents were apparently not totally convinced that they have learnt something.

Question 2:

The common problems participants seemed to have experienced, were the following:

• Insufficient pre-course or in-course training in the uses of media, multi-media and software packages such as Frontpage.
• Technical (software, hardware and internet-related) obstacles which caused problems in the gaining of easy access to the Teles Campus and certain additional sources.
• The late installation of all the required software and hardware in computers:
• Time constraints due to workload at TSA and the amount of time required by the course.

The remaining problems were of an individual nature.

Question 3:

It seemed as if participants regarded the following aspects as the major strengths of the course:

• Instruction in the design of an online course and how to offer an online course.
• Good feedback from facilitators.
• Interaction with other participants (national and international).

• It seemed as if all of the respondents (100%) felt that they would be able to design and develops an online course on the basis of the skills they acquired from the course if the necessary infrastructure was available.

• It seemed as if 89% of the respondents would recommend the Dweb course to other staff members at TSA provided that certain suggestions/recommendations were implemented.
• It seemed as if 11% of the respondents would not recommend the Dweb course if the aim was to instruct staff on the more technical aspects of an online course.

It seemed as if the main recommendations respondents would like to make in order to improve the Teles course, were the following:

• Lecturers should get an orientation to the course and the different uses of multi-media and html.
• The course should set clear requisites for enrolment (i.e. skills for multimedia, frontpage, and so forth).
• Participants should be given more time (from work) to participate in the course.

• Total enrolments: 25 - 100%
• Number of staff who completed the course successfully: 11 - 44%
• Number of staff who did not complete the course: 14 - 56%

I sent out an evaluation form to each of the 25 participants in the course. However, I only received back 12 completed forms (48%). The evaluation form was divided into two sections:

• The A Section had to be completed by those who completed the course and the B Section by those who did not complete the course.
• I received back an equal number of responses from both groups (6 each). The analysis will therefore reflect the two groups and 6 will represent 100% in both cases.

Below was a record of the findings obtained:

3. Findings

Section A: Only those who completed the course had to complete thIs section. (NB: 6 = 100%):

Question 4: Would you be able to design and develop an online course on the basis of the skills you acquired from the course if the necessary infrastructure was available? Motivate your answer.

Question 5: Would you recommend other staff members to attend the course? Motivate your answer.

Question 6: Were there any recommendations you would like to make regarding the improvement of the course?

1: Why did you not complete the course? Please specify the reason for not completing the course and clearly indicate the main reason(s).

2: Would you like to re-register for the course? Please motivate your answer.

Specific responses

Would you like to re-register for the course? Please motivate your answer.

4. Findings

The main findings I reached on the basis of the responses to each question, were as follows:

• It seemed as if 83% of the respondents felt that they have definitely learnt something from the course.
• It seemed as if 17% of the respondents were apparently not totally convinced that they have learnt something.

Question 2:

It seemed as if the respondents regarded the following aspects as the three main shortcomings of the course (arranged in order of importance).

• Technical/internet-related problems seem to be the single most common problem.
• The fact that the course did not provide sufficient information up-front.
• Time constraints.
• The remaining problems were of an individual nature.

Question 3:

It seemed as if participants regard the following aspects as the main strengths of the course (in order of importance):

• Learning about technology for online instruction, the various sources of information available and the various online activities that could be engaged in.
• The fact that the course provided a good overview of the design and presentation of online/web-based courses.

(Other strengths were indicated individually.)

• It seemed as if the majority of respondents (67%) felt that they would be able to design and offer new online courses, provided that certain conditions were met.
• It seemed as if 17% of the respondents felt that the course did not equip them with adequate skills to design and offer an online course.
• It seemed as if 17% of the respondents were not totally sure whether they would be able to design and offer online courses as an outflow of the Teles Dweb course.

• It seemed as if the majority of respondents (83%) would recommend other staff members at TSA to enroll for the course.
• It seemed as if 17% of the respondents were not sure whether they would recommend other staff members to enroll for the Teles Dweb course.

• It seemed as if there were no particular recommendations that were emphasized by the respondents.
• It seemed as if most of the recommendations were of an individual nature and varied.

It seemed as if the main reasons why certain staff members dropped out of the course were as follows (in order of importance):

• The time-consuming nature of the course (67%).
• Workload and work priorities (50%).
• Lack of knowledge of Internet and the use of online technologies (33%).
• Lack of individual (technical) support at TSA (33%).

Question 2

• It seemed as if the majority of respondents who dropped out of the course (83%) would be willing to re-register for the course and try to complete it.
• It seemed as if 17% of the respondents have strong reservations about re-registering for the course.

There seemed to be a number of similarities between the findings of the first and second report on the Teles Dweb course. Below is an illustration of the main similarities:

5.1 Question 1

• In both reports, the majority of respondents who completed the course successfully seemed to feel that they learnt something new from the course.

5.2 Question 2

Two of the main shortcomings of the course identified in the first report were repeated in the findings of the second report. Those were the following:

• Technical and internet-related problems.
• Time constraints (workload versus time required by course).

5.3 Question 3

• In both reports, the presentation of an overview of the design and offering of online courses, was indicated as one of the main strengths of the course.

5.4 Question 4

• In both sets of findings the majority of respondents indicated that they would be able to design and offer online courses if the necessary infrastructure and technical support were available.

5.5 Question 5

• In both sets of findings the majority of respondents indicated that they would be willing to recommend the course to other staff members at TSA.

5.6 Question 6

The findings in question 6 seem to contain the only difference between the two reports. These differences were as follows:

• In the first report there seemed to be certain recommendations that were clearly emphasized by different respondents, for example, a need for orientation of new participants; a clear statement of the requisites for the course; the provision of more time (from work activities) for participation in the course, and so forth.
• In the second report, although some of the above recommendations were repeated, there seemed to be no clear emphasis on particular recommendations by the respondents.

5.7 Section B

There was no section B in the first report and therefore no comparison was possible.

The following conclusions were reached on basis of the above comparisons:

• The course was regarded as worthwhile by the majority of staff members.
• Staff members who completed the course successfully did acquire valuable online instruction skills from the course.
• If the necessary IT infrastructure was available, staff members who completed the course successfully would be able to embark on the offering of online courses.
• The main problems involved were not so much with the content or quality of the course, but more of a technical nature, e.g.: access to the internet; insufficient bandwidth; hardware and software problems.

Although the percentage responses received in report 2 were much lower than in report 1, the similarities in the two reports increase the validity of both reports. The similarities allow for a more convincing generalization of the findings of both reports upon the rest of the staff members who were enrolled for the course, but who did not complete the evaluation form.

References

Hughes C, Hewson L & Nightingale P, 1997: Developing new roles and skills (In: Yetton P, 1997: Managing the introduction of technology in the delivery and adminwastration of higher education, Fuyitsu Centre, University of New South Wales: Evaluation and Investigation Program (EIP).

Maier P, Barnett L, Warren A & Brunner D, 1996: Using technology in teaching and learning. London, Stirling (USA): Kogan Page.

|  NADEOSA Homepage |  2nd NADEOSA Conference Papers Index  |
Global Distance Education Network - South African resources  |

Send comments on the web site to the web designer