Papers Presented at the 2nd National NADEOSA Conference
Held 21-22 August 2000

Author:
Wendy Dilys Coetzee, Faculty of Science, Viata University

Title:
A hierarchy of design principles for writing instructional texts in science - Enhancing teaching and learning through research

Introduction

The importance of using reliable design principles to ensure the accessibility of instructional texts for speakers of other languages has been investigated in this study. The qualitative part of an earlier enquiry has been utilized for the quantitative aspect discussed here which focusses on the diagnostic testing of course material in the biological sciences. The research has been driven by the belief that there is a need to develop well-grounded design principles for use as guidelines by the writers of course material in science. It seems possible, and helpful to writers, to group design principles into clusters and to arrange their relative importance in a hierarchy of significance according to the results obtained.

An earlier investigation began by ascertaining the problems which distance education students said they had with some of their course material, specifically in their first year course in the biological sciences. Information was obtained through a series of interviews with them, conducted first on an individual basis, then in pairs and finally in small groups. Running parallel to the interviews with students, the researcher also sought the views of the writers of the course material under consideration and the views of colleagues writing similar courses. Their attitudes to writing were ascertained, as well as their perceptions of the students’ difficulties with their courses.

The diagnostic test was administered in 1999 to first year BSc students on three of Vista University’s contact campuses. The contact students selected for the main diagnostic test consisted of 101 volunteers, split between three campuses, namely in Port Elizabeth, Soweto, near Johannesburg and Mamelodi, near Pretoria. Each of the three classes was divided, more or less evenly, into two groups for the purposes of the test. The groups totaled 52 and 49 students respectively. The material being tested was taken from a diploma course in the biological sciences offered by the Vista University Distance Education Campus for in-service teachers.

A non-parametric test, the Mann-Whitney U-test, to compare two unpaired groups with each other, was used for statistical testing.With respect to levels of statistical significance, the standard 5% (p £ 0.05), the most widely recognised level for testing of this nature, was taken as indicating significance. Differences that generated a level of 1% (p £ 0.01) are described in this study as ‘highly significant’, whilst those that generated a level of 10% (p £ 0.10) are described as indicating a ‘tendency towards significance’.

Discussion of test results

The questions were divided into four parts according to the extract upon which the four sets of six questions had been based. Each question was linked to the respective design principle or principles which it sought to test. Raw scores were calculated for each campus for all four options given for each of the 24 questions posed. In Appendix 1 the scores for the group using the original version are followed by those obtained for the group using the revised version of each extract. Total numbers and percentages are provided in each case for the answers which were regarded as the most appropriate and were therefore marked as being right.

Many of the results indicated the importance of a particular design principle to writers. Overall 75% of the features tested for showed a positive swing in results, when restructured. Furthermore there was a highly significant difference in the results of six of the design principle clusters into which the questions were grouped, suggesting that the diagnostic testing of course material, if well planned, and carefully executed, is a recommended procedure to follow.

Discussion of test results by design principle

It was deemed useful to group the questions according to clusters of related design principles. These clusters of design principles were then considered in a more global fashion and arranged hierarchically, according to the degree of significance of the results obtained. Given below are two key questions which I sought to address:

• What are the main types of design principles to have emerged and which of these appear to be more important than others?

• Are there any clear patterns which have emerged upon which recommendations for making use of certain design principles can be based?

Nine clusters of design principles were identified from the research undertaken, which were grouped under the following headings:

• the use of access devices

• uniformity of description

• accuracy of description

• definition of words with more than one meaning

• helping the reader to make the desired inferences

• explaining cultural and historical allusions

• maximising syntactic clarity

• the judicious use of ellipsis

• making strong links between text and diagrams.

Each category of design principles will be dealt with in turn.

The use of access devices

The statistical analysis of this cluster of questions, which tested the use of access devices, revealed that there is a highly significant difference between the performance of the students using the revised extracts in this cluster over those using the original versions. The access devices included here are: the appropriate use of headings and sub-headings, numbering and other listing devices, such as dashes.

It is advisable for writers to provide appropriate headings for the subjects they write about which accurately reflect the topic under discussion. Equally, the use of sub-headings within passages is recommended to help readers logically organise the information provided. Sometimes these sub-headings can be numbered, to good effect. Listing devices, should not be used to link unrelated points, however.

Uniformity of description

The cluster Uniformity of description, tested three related design principles. The statistical analysis revealed that there was no significant difference between the performance of the students using the revised extracts in this cluster over those using the original versions. The difference between the two sets of scores was only slight. Even so, certain lessons can also be learned.

Accuracy of description

In contrast with the last cluster, there was a highly significant difference between the performance of the students using the revised extracts in the cluster accuracy of description over those using the original versions. It would seem essential for writers of scientific texts to pay special attention to the accurate wording of their texts in order to avoid confusion and lack of comprehension on the part of their readers. Writers should exercise great care in providing accurate descriptions of the functioning of processes or entities, since failure to do so could result in faulty interpretations.

Defining words with more than one meaning

The difference between the scores of the students in the cluster: defining words with more than one meaning who used the original extracts and those using the revised versions was found to be highly significant. Where words have more than one meaning, the diagnostic test results suggest that definitions be provided for speakers of other languages, to avoid their making wrong interpretations. One question tested a word with two general meanings. Another one looked at a word with completely different general and scientific meanings.

Helping readers to make the desired inferences

Two design principles fall under the umbrella category of ensuring that readers make the required inferences. These are: clearly defining cause and effect relationships to avoid the reader’s having to make unreasonable inferential leaps and helping readers to make the correct inferences by providing sufficient textual information.

There was a highly significant difference between the performance of the students using the revised extracts in this cluster over those using the original versions. Writers of scientific text should clearly delineate cause and effect relationships. Equally, it is essential to provide sufficient textual information to ensure that readers make appropriate inferences and draw the required conclusions. 

Explaining cultural and historical allusions

The design principle tested under this heading was that cultural and historical allusions, particularly those involving traditional beliefs and superstitions, should be fully explained where the readership is known to be or suspected of being culturally diverse. There was a highly significant difference between the performance of the students using the revised extract over those working from the original version. There was only one question in this cluster.

Evidently, it is essential for those writing, who make historical or cultural allusions and whose target groups are mainly from other cultural groups, to explain such references to their readers.

Maximizing syntactic clarity

Under the category Maximising syntactic clarity fall two design principles: that there should be logical ordering of sentences within paragraphs of a passage to facilitate the explanation of stages in the development of an entity or process; and that straightforward rather than complex syntactic structures should be employed to describe scientific entities or processes. The statistical analysis for this cluster did not, overall, reveal a significant difference between the performance of the students using the revised extracts over those working from the original versions although individual language oddities were identified which writers would do well to pay attention to.

In its original version, one sentence read: ‘A sclerotium, followed by a 17 word embedded clause..is formed’. This sentence is unnecessarily complex. Writers should avoid such complex constructions. This cluster highlighted the importance of logical ordering, by evaluating an improvement of this feature in the revised version of the extract on viruses. Since the virion forms part of the extracellular phase of viral reproduction, it is misleading to include mention of it in the paragraph discussing the intracellular phase of the development of a virus.

The judicious use of ellipsis

Two questions tested the need to be circumspect in the use of ellipsis when writing for readers whose mother tongue is other than English; in other words to make judicious use of ellipsis. There was a highly significant difference between the performance of the students using the revised extracts in this cluster over those who were given the original version.

The first question in this cluster tested the design principle which stipulated that the use of ellipsis should be avoided when the result involves no mention of the main topic under discussion. In the extract, that dealing with the tobacco mosaic virus, the subject referred to by quantity and quality is never mentioned at all. This can be problematic for readers.

The next question tested the use of ellipsis when there is a risk that the reader will misinterpret words and phrases which are not sufficiently qualified. In this question the word chew could be equally interpreted as chewing gum or chewing tobacco, since the referent is not stated. On the other hand no referent is required for the word smoke in this context, since this is a high frequency word and not likely to be misinterpreted if no referent is given.

There was a highly significant difference between the performance of the scores of those using the original and restructured texts.The results obtained for the two questions which tested the use of ellipsis suggested that this language device should be used judiciously by writers when writing for speakers of languages other than English.

Making strong links between text and diagrams

The two design principles considered under the heading Making strong links between text and diagrams covered the essential link which should exist between a text and its accompanying diagram. Firstly writers should have included in diagrams only those features which are referred to in the text. Secondly diagrams should be selected which clearly facilitate the recognition of features referred to in the text.

Although it could be argued that these are two sound design principles which writers of instructional texts would do well to observe, the results were not statistically significant. More investigation needs to be done on the efficacy of scientific diagrams, a study in its own right. 

Conclusion

Overall 75% of the features tested for showed a positive swing in results, when restructured. Ten out of the 24 questions posed had significantly different findings between the groups working from the original and the revised versions of the four texts in question. In two further cases there were tendencies towards significant difference. With regard to the clusters of design principles, six out of the nine clusters indicated a highly significant difference between the scores of students using the revised extracts over the original versions. For the other three clusters there was no significant difference between the two sets of scores. These clusters of design principles have been arranged hierarchically, according to the degree of significance of the results obtained.

The design principle clusters tested have been arranged in rank order, according to importance:

1. Give accurate descriptions of features or processes

2. Use access devices, such as headings, bullets and lists

3. Define, in context, words with more than one meaning

4. Explain less obvious cultural and historical allusions

5. Make judicious use of ellipsis, to avoid ambiguity.

6. Help readers to make the required inferences

7. Maximise syntactic clarity, by avoiding complex sentences.

8. Draw strong links between text and diagrams.

9. Employ uniformity of description, within a text and between text and diagram(s).

It is recommended that the periodic diagnostic testing of course material against reliable design principles, similar to the method described above, is the route to follow. Evaluation of this kind will enable course material writers to become ever more aware of the difficulties which might be facing some of their students, and to write more clearly with this grounded knowledge in mind. 

APPENDIX 1

Results of main diagnostic test as percentage totals of the three campuses

* Questions marked with an asterisk indicate those which yielded a tendency towards significance with regard to the scores of the group using the revised version over those using the original one, at the 10% level (p£ 0.10).

** Questions marked with a second asterisk indicate those which yielded a significant difference between the scores of the group using the revised version over those using the original one, at the 5% level (p£ 0.05).

*** Questions marked with a third asterisk indicate those which yielded a highly significant difference between the scores of the group using the revised version over those using the original one, at the 1% level (p£ 0.01).

Questions marked with a # indicate those which yielded a reverse significant difference between the scores of the group using the revised version over those using the original one, at the 5% level (p£ 0.01).

Bibliography

Baumann, J. (1986) ‘Effect of rewritten content textbook passages on middle grade students' comprehension of main ideas: Making the inconsiderate considerate. ‘Journal of Reading Behavior,, Vol. XVIII, No. I, pp 1-21.

Blau, E.K. (1982) ‘The Effect of Syntax on Readability for ESL Students in Puerto Rico. TESOL Quarterly, Vol16, No.2, pp. 517-527.

Carrell, P. L. and Eisterhold, J.C., (1983) 'Schema theory and ESL reading pedagogy’, TESOL Quarterly Vol. 17, pp.553-573.

Carrell, P. L., Devine, J and Eskey, D E. (1988) Interactive Approaches to Second Language Reading, Cambridge, Cambridge University Press.

Cook, G. (1989) Discourse, Oxford, Oxford University Press.

Cooper, P. A. (1995) 'In Search of Sufficient Vocabulary: Testing the Vocabulary Levels of Undergraduate Students, South African Journal of Linguistics, Supplement 26, pp 25-37.

Cunningham, J.W. (1972) and Moore, D. W, ‘The contribution of understanding academic vocabulary to answering comprehension questions’ Journal of Reading Behaviour, Vol.25, pp.171-180.

Freebody, P. and Anderson, R. C., ‘Effects of vocabulary difficulty, text cohesion, and schema availability on reading comprehension.’ Reading Research Quarterly Spring 1983 Vol. XVIII/3, pp.277-305.

Grabe, W. (1991) 'Current developments in second language reading research', TESOL Quarterly Vol. 25, pp 375-406.

Klare, G.R. (1981) Practical aspects of readability, Textual Communication Research Group Paper, Milton Keynes, Open University.

Klare, G.R. (1985) How to Write Readable English, London, Hutchinson.

Leighton, M (1992) ‘The Cognitive Reading Level of First Year Vista University Students and Vista Study Manual Readability', PhD Thesis, Pretoria, Vista University.

Lesgold, A.M. and Perfetti, C.A., (1981) Interactive processes in reading, New Jersey, Lawrence Erlbaum Associates.

Loman NL & Mayer RE (1983) ‘Signalling techniques that increase the understandability of expository prose’ Journal of Educational Psychology. Vol.75, No.3, pp.402-412.

Macdonald, C.A. (1990) English language skills evaluation, Pretoria, Human Sciences Research Council.

Mobley, M. (1986) 'Evaluating curriculum materials, Unit 2', Readability, York, Longman.

Moe & Irwin (1986) "Cohesion, Coherence and Comprehension’ in J W Irwin: Understanding and Teaching Cohesion Comprehension, 1986: 3-8

Perfetti, C.A., (1985) Reading Ability, New York, Oxford University Press.

Riley, J. (1984) 'The Problems of Drafting Distance Education Materials', British Journal of Educational Technology, Vol. 15, No.3, pp 192-204.

Riley, J. (1984) The Problems of Revising Drafts of Distance Educational Materials, British Journal of Educational Technology, Vol. 15, No.3, pp 205-226.

Riley, J. (1984) 'Drafting Behaviours in the Production of Distance Education Materials, British Journal of Educational Technology, vol. 15, no.3, pp 226-238.

Rowntree, D. (1992) Exploring open and distance learning, London, Kogan Page.

Rowntree, D. (1992) Teaching through self-instruction, How to develop open learning materials, London, Kogan Page.

Rowntree, D. (1994) Preparing materials for open, distance and flexible learning: an action guide for teachers and trainers, London, Kogan Page.

Silberstein, S. (1987) 'Let's take another look at reading: twenty five years of reading instruction' English Teaching Forum, Vol. 25, pp.28-35.

Smith, F. (1971) Understanding reading: A psycholinguistic analysis of reading and learning to read, New York, Holt, Rinehard and Winston.

Spiridakis , J.H., and Standal, T.C., (1987) ‘Signals in expository prose: Effects on reading comprehension’ Reading Research Quarterly. Vol.22, pp.285-298.

Stotksy, S., Types of Lexical Cohesion.

Wilhite , S.C., (1986) ‘The relationship of headings, questions, and locus of control to multiple-choice test performance’, Journal of Reading Behavior, Volume XVIII, No.1.

Yano, Y Long, M.H., and Ross, S. (1994) ‘The effects of simplified and elaborated texts on foreign language reading, comprehension’, Language Learning , Vol 44, No.2, pp. 189—219

|  NADEOSA Homepage |  2nd NADEOSA Conference Papers Index  |
Global Distance Education Network - South African resources  |

Send comments on the web site to the web designer