SURVEY OF INSTRUCTIONAL TECHNOLOGY RESEARCH

or

IF INSTRUCTIONAL TECHNOLOGY

EFFECTIVENESS WERE A CRIME,

WOULD THERE BE ENOUGH EVIDENCE TO CONVICT IT?

"The big story in the field of information technology for education and training is not what marvellous new technology we educators now have at our fingertips. The big story is the slow take up of this technology and the challenge this poses for educational managers."

--John Mitchell

© 1997, 1998, 2000, 2001 Michael Szabo All Rights Reserved

TABLE of Contents

INTRODUCTION

1.0 COMPUTER ASSISTED INSTRUCTION

2.0 COMPUTER MANAGED INSTRUCTION

3.0 STATIC VISUAL DISPLAYS

4.0 AUDIO

5.0 COLOR/SCREEN DESIGN

6.0 ANIMATION

7.0 MULTIPLE-CHANNEL LEARNING

8.0 NAVIGATION & HYPERMEDIA

9.0 INSTRUCTIONAL TELEVISION

10.0 INTERACTIVITY

11.0 CRITICISMS OF MEDIA RESEARCH

12. HAS THE JURY REACHED A VERDICT?

13.0 RECOMMENDATIONS

MAJOR REVIEW SOURCES

REFERENCES

ADDITIONAL READINGS

GLOSSARY

INTRODUCTION

The Information Society and Instructional Technology

It is clear that most societies are well into the industrial society and contemplating the change to information societies. What kind of educational system will get us through the transition from preparing graduates from industrial society to preparation for information society?

The role played by instructional technology, that branch of the information system which deals with providing education and training, is not clear. This presents an outstanding opportunity for educators and trainers to help shape that future.

This paper considers what we can learn from the research on instructional technology. As the title suggests, if it were a crime for instructional technology to bring about better learning, would there be enough (research) evidence to convict it?

The Components of Instructional Technology

The definition of instructional technology (IT) according to the Association for Educational Communications and Technology (AECT) is "the theory and practice of design, development, utilization, management and evaluation of processes and resources for learning." (Seels & Richey, 1994, p. 1). There are many ways to expand this definition. For example, the author includes Computer Based Instruction, Computer Mediated Communication, and Instructional Systems Design in an expanded definition.

The systematic implementation of complete IT learning systems has not yet occurred so we begin to inquire into this question by examining research on the separate components of IT; computer based instruction, static visual displays, dynamic visual displays, audio and screen design, including color. Another consideration is how the interactions between different media affect learning, e.g., video with and without audio. This general topic has historically been referred to as multichannel instruction.

Computer Assisted Instruction (CAI)

CBI employs computer technology to assist the instructor to instruct or the guide the learning program of individual students. Its main components consist of CAI and CMI, each of which may employ a variety of media. Computer Assisted Instruction employs instruction modes of tutorial, review and practice and simulation.

Computer Managed Instruction (CMI)

Computer Managed Instruction includes diagnostic assessment and prescriptive study assignments. Because of the leading role of developing and using CMI in Alberta, the topic will be accorded its own special separate section.

Static Visual Displays (SVD)

These are often called visuals or graphics. Their function is to provide a non-text based representation of some object, process, concept or skill to be learned, although they are often accompanied by text. They do not move through space or time and vary in the amount of detail and realism they contain, from a simple line drawing created with a draw/paint program to a photograph.

Audio

Audio broadcasts information or data in a format which can be heard and may include instruction which is redundant with text or unique. It may also include warning sounds and sounds which are used to determine the state of things, for example a badly-tuned automobile engine.

Screen Design/Color

Screen design refers to how information is spatially organized for presentation to the learner. Some separate screen design issues are the use and placements of fonts, color and numerous design issues such as balance, borders, and so forth. Color has been the object of numerous studies dating back to the middle of the century.

Animation (Dynamic Visual Displays)

Animations are visual images which represent motion through space or time. Like SVDs, they may vary in the amount of detail and realism they contain, from a simple line drawings through to video. They are usually created using special authoring software, such as Authorware, Director. Alternatively, they may be video sequences which have been captured in analog or digital format. The set of motion visuals that includes animation and video has been called dynamic visual displays (Park & Hopkins, 1993).

Multi-Channel Learning

Interactive multimedia enables simultaneous delivery of instruction via several senses or channels. It might appear that the same information, delivered through multiple channels simultaneously (e.g., visual and audio) would enhance learning in a measurable way. Alternatively, could information from one channel interfere with information from another channel and decrease learning? What what if information delivered by one channel is in conflict with or irrelevant to information delivered via another channel? The research and theory dealing with these and other topics is referred to as Multi-Channel research.

Navigation and Hypermedia

Navigation refers to the process of acquiring information from a rich multi-media data base which has no obvious organizational pattern. The World Wide Web is an example of the latter. It is intuitive and attractive to believe that navigation as a learning system will result in significantly better learning than highly structured learning. To date, there is little research in this area. The conclusion which seemS to be emerging is that the effectiveness of navigation cannot be assumed present for all learning situations. While research on navigation is quite new and as yet limited, it is expected to increase dramatically in the next few years.

Instructional Television

Instructional television refers to the use of televised media, whether broadcast, analog tape or videodisc or digital imagery which is employed in an instructional setting to supplement or supplant other forms of instruction. As with other forms of media, ITV originally required massive production facilities, personnel and budgets. With the advent of the transistor, microprocessors and chips, the equipment for capture and editing has become much less expensive and has migrated to the desktop. The basics of good production have remained the same, although styles have been driven by the popular media, styles such as the short sound bite, rapid flashes of imagery, and special graphics effects. There is extensive research on ITV, some of which will be summarized in this paper.

Interactivity

Historically, IT was heavily employed for transmission of information about content from a knowledgeable source to the student (information transmission). As such, it is not capable of the high levels of interaction possible between an instructor and learner. There are many definitions of interaction but they generally require that two things be able to carry out activities which elicit a response from one another. Perhaps the highest level of interactivity is a series of activities which result in the student learning or processing information at a cognitive level which is higher than rote memorization. This definition stands in stark contrast to the current and loosely used terms which generally means any control over the environment by the learner.

The Research Dilemma

In this survey, I chose to examine the individual components, some of which have been researched extensively while others have had little attention from researchers. This is largely because few comprehensive studies of IT have been done because it is so complex and contains so many components. The extensive research on color and computer based instruction probably is accounted for by the fact that these areas were of great interest to educational psychologists who also have a strong history of conducting empirical research. Screen design, on the other hand, is the domain of artists whose history and tradition is not grounded in empirical research.

Does research make a difference? It depends on an individual's 'mental model' of how the world of instructional technology works (Szabo, 1997). There seem to be three different orientations toward the development of interactive multimedia. The logical approach examines the current literature and practice, makes careful plans, and executes the design. Research is important in this approach. The empirical approach seeks to create quickly but test extensively and repeatedly, using the information gained from each pilot to refine and improve the product. Finally, there is the artistic approach, in which the artistic appearance and qualities of the product are considered paramount. Prior to the availability of desktop IT, the artistic approach required great skill and added expense to create mediated lessons. Although they are more popular than text-based lessons, there is little research to substantiate the importance of IT as a whole. Different components of IT seem to have considerably different research-demonstrated results. Research results are not as crucial to designers with the last two orientations as to the first.

Organization of the Module

This module seeks to summarize a portion of the current research literature on the components of interactive multimedia in order to allow us to proceed with assurance or caution, as the situation warrants. It is not an exhaustive consideration but a valuable starting point for further inquiry.

Return to Table of Contents

1.0 COMPUTER ASSISTED INSTRUCTION

Objective: Summarize the research findings with respect to achievement, learning efficiency, and attitude and describe the key research studies in the field of computer assisted instruction (CAI).

Next to color, computer based instruction has been the most widely researched element of IT. Although the ability of CAI to function as an IT system appeared only recently, the first IT system became available in the mid-1960s as the IBM 1500 System, and the first CAI appeared around 1959.

Early Research Limitations

In spite of the IT capabilities of some of the early CAI systems, such as the IBM 1500 and PLATO, most of the comparative research was conducted using CAI which was text-based. That situation has changed dramatically with the advent of the multimedia-enabled computer and the research needs to be updated. Conversely, most of the research done on aspects of IT has been done without the computer.

Much of this research has focussed on the effectiveness of CAI which is demonstrated through improved test scores (Balajthy, 1987; Williams & Brown, 1990). Effectiveness has also been measured through "heightened affective responses, or better attitudes, reduced learning time, higher course completion rates, an increased retention duration, and finally cost" (Williams & Brown, 1990, p. 214). Generally the effectiveness of CAI has been determined by comparing CAI with traditional classroom instruction (Clark, 1985).

Meta-Analytic Studies

To date, the most comprehensive compilation of studies on the effectiveness of CAI has emanated from the University of Michigan. The Kuliks and their colleagues have conducted several meta-analyses. These studies have revealed that students learned more and liked the instruction more when it was delivered via computer. Their meta-analyses also revealed that instructional time was reduced (Kulik, Bangert, & Williams, 1983; Kulik, Kulik, & Bangert-Drowns, 1984; Kulik, Kulik, & Cohen, 1980). This was the case for elementary, junior high, senior high school, and college students. Upon further analysis, instigated by Clark's (1985) critique, Kulik, Kulik, & Bangert-Drowns (1985), revealed that the effects of CAI were larger in published studies, in studies where a different teacher was used in the experimental group than in the control group, and in studies of shorter duration.

A more recent meta-analysis of studies concerning the effects of CAI on cognitive outcomes. In this meta-analysis of 31 studies Liao (1992) concluded that ". . . CAI is a mildly effective approach for teaching students cognitive skills in the classroom setting" (p. 377). But it was also noted that there was no evidence to indicate that CAI is more effective or even as effective as other instructional strategies.

Research on CAI has typically compared achievement, learning efficiency and student opinion in CAI and conventional instruction environments across an astounding range of subjects and age groupings. Several meta-analytic studies (Bangert-Drowns, Kulik, & Kulik, 1985; Kulik, Kulik, & Cohen, 1980; Kulik, Kulik, & Shwalb, 1986; Niemiec, Samson, Weinstein, & Walberg, 1987) have concluded that when CBI is used, achievement is moderately higher than or equal to that obtained from conventional instruction. When efficiency is the criterion, CAI is significantly better than conventional instruction, in the range of 10-35% savings of instructional time. Finally, student opinions generally favor learning under CAI than conventional instruction.

Component Research

The results are so powerful, relative to other instructional interventions, that some have called for a halt on the comparisons with conventional instruction; they instead urge investigations in which the components or attributes of CAI are investigated to determine optimum learning conditions. For example, in the context of CAI, the role of interaction (Worthington & Szabo, 1995), animation (Szabo & Poohkay, in press; Szabo & Schlender, 1996), audio (Rehaag & Szabo, 1995) and testing (Szabo, Poon, and Ally, 1997) has been researched in recent years.

CAI research is not without its critics. Clark (1985) has criticized the methodology of all media research, including CAI. However valid his criticisms may be, one cannot help but be impressed with the results. (Worthington & Szabo, 1995), animation (Szabo & Poohkay, in press; Szabo & Schlender, 1996), and audio (Rehaag & Szabo, 1995) has been researched in recent years.

Return to Table of Contents

2.0 COMPUTER MANAGED INSTRUCTION

The University of Alberta has a long history of the use of computers as instructional devices. It began in 1968 with the introduction of an IBM 1500 computer assisted instruction system and progressed though the installation of a CDC PLATO system which was subsequently removed in 1987. Management of instruction has always been a key element of any computer related research done at the University of Alberta whether as a part of large computer assisted instruction courses, or as courses specifically designed to utilize computer managed instruction. This paper presents a context for the study of computer managed instruction and reviews a number of the key studies in computer managed instruction at the University of Alberta. With the reorientation toward microcomputer based instructional systems, it seems propitious to summarize the research in CMI to this point in time. (Keywords: computer managed instruction, attitudes, achievement, costs)

Can researchers and teachers devise teaching-learning conditions that will enable the majority of students under group instruction to attain levels of achievement that can at present be reached only under good tutoring conditions? (Bloom, 1984, p. 5). A CBI system that replaces part or all of a teacher's instructional role will receive little if any long-term use. (Baker, 1981, p. 23)

The University of Alberta (U of A) has a long history of involvement in computer managed instruction (CMI). In 1968, the U of A received one of the first IBM 1500 computer systems. The IBM 1500 was designed as a complete multimedia computer assisted instruction (CAI) system, with a high resolution (by 1968 standards) graphics display screen, light pen, random access film strip projector and audio record/playback units at each student station. It also utilized a specially designed CAI authoring language, Coursewriter II. While many other designers of CAI in the 1960s and 1970s concentrated on building stand-alone modules to supplement teaching the orientation of authors at the University of Alberta was towards the development of complete courses where the computer performed all the mainline instruction, testing and record keeping. The philosophy of these courses was that the instructor should be involved only when the computer diagnosed that an individual student required remedial instruction or skill-based evaluation that the computer could not perform. This freed instructors to act as tutors, concentrating their efforts where they could do the most good.

As courses were developed during the early 1970s, authors began to realize that, while Coursewriter II provided good instructional capabilities, there was no predefined ability to manage the testing and routing of students through the course. Each author needed to build such CMI capabilities into the CAI code.

One of the first people to investigate the possibility of separating the models of instruction, including testing and prescription, from the actual content of the instruction was Romaniuk (1970). His early work on the Versatile Authoring Language for Teachers (VAULT) system was later identified by Fred Morrison as one of the inspirations of the TICCIT System.

Some of the earliest work in developing a CMI capability for the IBM 1500 was undertaken at the U.S. Army Signal Corps, Fort Monmouth, NJ (IBM, 1968). The source code was made available to the IBM 1500 community and was subsequently modified by staff at the Division of Educational Research Services (DERS) at the U of A to include a number of different CMI capabilities. Petruk (1975, 1978) used these modified CMI features as the basis for two sequential projects under the joint sponsorship of Alberta Advanced Education and Manpower and Canada Manpower and Immigration to use CAI to teach electrical apprentices at the Northern Alberta Institute of Technology. The instructional model used by Petruk (1978, p. 5) represented an advanced use of CMI within a CAI environment.

At the same time, the National Research Council of Canada contracted with staff at DERS to develop a set of preliminary specifications for an instructional support system for NATAL-74 (the NATional Authoring Language). A great deal of the document (particularly chapter 5) is oriented towards the provision of CMI capabilities within NATAL-74 (Hunka et al., 1978).

By the time the IBM 1500 was returned to IBM in 1980, a number of courses had been designed and implemented at the University of Alberta in which the primary instruction was by computer. Hunka (1978) describes five of these courses in which the average student time to complete the course varied from 15 to 75 terminal hours. While all of these courses were considered to be CAI, they each contained a major CMI component.

In 1980 the U of A installed a CDC PLATO system. Among the many powerful features of this system is an integration of CMI with CAI. Because of the powerful software design and mainframe capacity, a prescription to an existing CAI lesson could be carried out immediately, subject to the control of instructor and student. Courses in textiles, anatomy, physical education, nursing education, and a variety of other CMI applications were developed using this system, and many were transported to other PLATO systems.

In 1987, the PLATO system was decommissioned at the University, presumably due to budget constraints. University president Paul Davenport stated in the Edmonton Journal on 29 April 1990,

The Role of CMI in Promoting Individualized Instruction

Learning is significantly more effective when instruction can be tailored to the unique needs of each learner. For example, Bloom (1984) found that students who are individually tutored learn twice as much as (score 2 standard deviations above) students instructed in the conventional manner.

Learning is also significantly more efficient when instruction can be tailored to the unique needs of each learner. Studies of student-paced instruction (vs. instructor-paced instruction) show that students learn the same amount in 20 to 50% less time than when they are instructed in the conventional manner (e.g., Kulik, Kulik, & Cohen, 1980). Self-pacing is only one form of adaptation to the many individual differences exhibited in a typical classroom.

Research on learning severely challenges our conventional wisdom about instruction and learning, namely, that aptitude is fixed and places a limit on what our students can learn. This research has given rise to the testable hypothesis that given enough time and optimal learning conditions, most students "do have the potential to reach this high level of learning" (Bloom, 1984, p. 4). Carroll (1963) argued that aptitude could be redefined. "The amount of time that a student needs to learn a given task under optimal learning conditions, is, in the author's opinion, a reflection of some basic characteristic or characteristics of the student that may be called 'aptitude'" (p. 72). The deeper implications of the time dimension of learner aptitude has profound implications for the way we instruct.

Tennyson and Park (1984) made a significant contribution to the field of instructional theory and practice. The purpose of their research was to review four models of increasing sophistication which use CMI to link instructional procedures to learning outcomes. The underlying theme of their work is that appropriate models which provide more real-time monitoring and guidance (advisement) of individual student progress (1) have been shown to be effective and (2) can be administered by computer through CMI. It also represents a strong and more direct linkage between instruction and assessment.

What specific conditions of the learning process promote better learning? Bloom (1984) identified the six (after tutoring) most important alterable variables in terms of their effect on student achievement. These are reinforcement, corrective feedback, cues and explanations, classroom participation, study time on task, and improved reading/study skills. Let's now turn to the role of CMI in addressing these individual, alterable instructional variables.

Computer managed instruction is a highly specialized application of the computer to assist in the process of managing an individualized instructional plan for students. It consists of two required functions. First, the computer tests the student to identify his/her strengths and weaknesses. The result is a diagnosis which indicates which objectives the student has and has not mastered. A unique dimension added by CMI is that the diagnosis is individually tailored for each student's pattern of performance. Based on the diagnosis, each student receives a computer-generated prescription, that is an individual course of study designed to help the student meet the specific unmastered objectives. The second function of CMI is, therefore, prescriptive. CMI is not simply computerized testing or item-banking as these do not provide individually-tailored diagnostic and prescriptive functions.

Competency-based learning/mastery imply a repeated cycle of instruction and testing until competencies have been met. The iterative nature of the computer offers tremendous potential to accommodate this cycle through its ability to administer exams with little or no instructor intervention and to clerically process the data and present the performance results in a summary form to the instructor.

Hansen (1970) listed five major functions of CMI: (a) providing diagnostic evaluations with learning prescriptions, (b) counselling students about adaptive learning strategies and career development, (c) developing an optimal scheduling system to match students and learning resources, (d) maintaining an appropriate student instructional record system, and (e) limited use of CAI for drill and practice.

While the technology used to deliver CMI has changed dramatically since the 1960s, the structure and function remain the same. Two outstanding sources for information on the structure and functions of CMI are found in Baker (1981) and Allen (1972). Figure 1 represents the core structure of CMI. Note that CMI can be and is often used in conjunction with existing learning materials. Thus, an instructor can incorporate CMI

without substantially restructuring his/her ongoing instruction. In addition, CMI makes mastery learning possible in that retests to determine when mastery has been obtained are much easier to manage (administer, score, document, and report) than are conventional classroom tests.

People learn best when they can be made to actively and consciously process the information they encounter when they encounter it. Conventional instruction discourages the individual from processing information other than by trying to copy it to short-term memory, which has been shown to be most ineffective (some would say counterproductive) to learning. The computer is unique among media in that it can interact with a student in a processing way (Martin & Szabo, 1990) and stimulate deep cognitive processing (Anderson, et al, 1975).

Most of the specific conditions (alterable variables) identified by Bloom (1984) can be enhanced with the use of CMI. Individualized reinforcement can be provided for each student, item, and objective through CMI. Similarly, CMI can provide corrective feedback precisely where and when it is needed, to the individual student. Cues and explanations can be built into the CMI application and presented only to those whose performance indicates additional help is needed. Student classroom participation is interpreted to mean the active cognitive processing of information by each individual student, irrespective of level of participation. In a CMI exercise, the student must be actively processing information at a significantly more intense rate than during conventional instruction. During conventional instruction, there are many distractions which often cause the student to miss what is going on. During CMI, the student's attention is directed to the task at hand (assuming the ability level of the student is reasonably matched with the difficulty level of the CMI items). There is less 'down time.' We now turn to see whether research supports these statements.

Research Studies

This section of the paper briefly describes a number of studies of the use of CMI. Studies undertaken at the University of Alberta or in which U of A equipment or personnel were involved are emphasized. A few landmark studies undertaken in other places are included to place the U of A studies in context. For each study, we have briefly described the study, then discussed the findings and outcomes. A later section will amalgamate the findings of the studies.

Franklin and Marasco (1977) examined interactive computer based testing in post-secondary science courses. They identified (and exploded) five myths: (a) forcing students to use a computer presents an unwarranted additional obstacle to their attempt to attain their educational goals, (b) computer based testing is crude and inflexible, (c) the use of (d) interactive computer based testing (ICBT) is too expensive, and (e) ICBT has serious problems with security and cheating. They also identified several advantages: ICBT (a) provides a broad variety of sophisticated tactics, (b) is amenable to virtually any educational strategy, (c) permits a more thorough integration of testing and teaching, (d) provides immediate and individualized feedback, (e) requires and promotes explicit pedagogical decisions, (f) allows excellent pedagogues to widen their audience and increase their impact, (g) has evolutionary potential, (h) has extensive record-keeping and clerical capabilities, and (i) is an economical approach to serious educational problems.

Several large-scale projects on individualizing instruction in the 1960s and 1970s found the computer to be valuable in managing instruction: Program for Learning in Accordance with Needs (Flanagan, 1969); Individually Prescribed Instruction (Cooley & Glaser, 1969); Instructional Management System (Silberman, 1968); and Computer Assisted Study Management System or CAISMS (Anderson et al., 1975).

Anderson and colleagues (1975) developed a computer assisted study management system (CAISMS) for use in a University of Illinois economics course. A well-controlled experimental research study was conducted among students in a large undergraduate course

to compare students who used CAISMS with those who didn't. Students in the treatment group read study assignments and immediately completed on-line tests to assess their learning. Successful learning allowed a student to proceed through the syllabus, while unsuccessful learning required continued study of material already covered with retest opportunities. The treatment was developed in accordance with deep cognitive processing theory.

Results from this study indicated that students using CAISMS scored significantly higher on all course exams. One student's anecdotal comment is worth noting: "I really think CAISMS helped me a lot in this course, if not for anything else, it made me study regularly and be sure I got the right things out of what I read" (p. 8). Attitudes toward the course and method, as measured in the Anderson et al. study were significantly higher for the CAISMS students. Attitude scores for students were improved through the use of CAISMS for large but not for small classes. CAISMS had no overall effect on course completion rate.

Tennyson (1981) addressed problems of learner control associated with CMI. He was concerned with research indicating that students may be incapable of making and carrying out "decisions of content element selection and personal selection" (p. 426). Specifically, students whose instruction is self-paced may terminate their study prematurely (before they have mastered the objectives) and may not assume proper responsibility for their own learning. The Minnesota Adaptive Instructional System (MAIS) was devised to indicate to students the discrepancy between actual and mastery scores, provide study assignments based on discrepancies, and continually update the student. The MAIS system was replicated in several different content areas using 12th grade students. The criteria consisted of several examination scores covering the content of the lessons.

The results of the Tennyson series of studies are notable for their direction and their explanation. Two groups received identical CMI instruction. One group received advice from the CMI system via MAIS, while the other did not. It was found that achievement scores were significantly higher for the groups which received advice to help them to decide when to terminate instruction and which content to study next when compared with those who made the decisions in the absence of any external advice. The comparison was made between students who could terminate their study at any time and those who received performance feedback to assist them to decide whether they were ready to discontinue instruction. It was concluded that the guidance function provided by the management system enabled high school students to make better judgments about their own learning and thus improved scores. The implication drawn was "students can successfully participate in the management of their learning when provided with their own individual diagnostic and prescriptive information" (p. 430).

In the Tennyson (1981) study, students receiving advice spent 30% more time on task than students without such advice. It must be remembered here that the comparison was not with conventional instruction time but with students in self-paced instruction where they could terminate anytime.

The Nursing Studies

Szabo and Estes (1978) reported on the comparative costs of developing computer based instruction materials. Data were taken from logs of a project in which six university bachelor degree nursing courses were converted to a CMI format on the IBM 1500 System for distance delivery using a mobile van. Exterior and interior views of the IBM 1500 Mobile Vans are shown in Figures 3 and 4.

Figure 4. Interview view of PSU Mobile Computer Assisted Instruction Van

A comparison was made between developing computer managed instruction materials and developing computer assisted instruction materials. The CMI development was based upon the use of existing instructional resources; none were developed. On the other hand, the CAI materials were developed from scratch. It was found that an hour of CMI student contact time was developed in an average of 39.7 developmental hours. In the same project, an hour of CAI was developed in an average of 161.5 developmental hours. These data suggest a developmental ratio of 4 to 1, favoring CMI over CAI.

Kot, Skillen and Wales (1986) conducted an evaluation of a PLATO-based CMI course in nursing at the University of Alberta. Forty-six students at the post-RN baccalaureate level were randomly assigned to either a conventional lecture/discussion treatment or to a CMI treatment for the theory portion of the course. After the course had been delivered on campus for several terms, it was delivered in a distant town using microcomputers, modems, and public telephone networks.

Achievement scores, as measured by midterm, final, and practical exams, was a dependent variable in the Kot, Skillen and Wales (1986) study. Achievement scores and learning time for the two groups were equivalent, indicating equal amounts of learning. The Kot, Skillen and Wales (1986) study examined instructor time and concluded faculty "spent twice as much time per student on instructional activities for the control group" (p. iii). This time savings worked out to about 35 hours per instructor per course for this particular study. When students kept student time logs, it was concluded that they spent equivalent amounts of study time in the CMI and the conventional instruction classes.

Day and Payne (1987) compared the use of lectures with CMI to deliver basic nursing content in the first year of a baccalaureate in nursing program. This study used the same CMI system and course materials as were used in the Kot, Skillen and Wales study. The major difference was that the Day and Payne study used first year "generic B.Sc." students as subjects while the Kot, Skillen and Wales study used post-RN students as subjects.

The Day and Payne (1987) study confirmed the Kot, Skillen and Wales (1986) results in that no significant difference in achievement was observed between the two groups. The study found that students learned equal amounts based on course examinations and clinical sessions.

In the Day and Payne study, there was an indication of less positive attitude among the CMI group, although no statistical analysis was presented. If indeed lower attitude scores were present, several alternative plausible hypotheses would need to be confirmed, for instance (a) the application was well designed (some right answers were counted wrong, no feedback given for wrong answers); (b) it was not used with the appropriate target audience (the course was designed for experienced nurses returning for academic upgrading—the sample was first year baccalaureate nurses); and (c) the majority of students had recently completed a high quality computerized testing program in anatomy and may have made unobserved comparisons between it and the nursing application.

Boblin and Gibson (1986) developed a CMI component for an anatomy and physiology course for diploma nursing students at the University of Alberta Hospitals School of Nursing. This component was developed using the PLATO Learning Management (PLM) program. The CMI component consisted of a module component and an exam component. The module component consisted of 21 modules which corresponded to the the subject areas taught. These modules were optional and were "designed to assist students in acquiring the course content and preparing for the exams" (p. 127). The exam unit consisted of six compulsory unit tests. Due to cost, a "picture book" was developed for student use rather than developing on-line graphics.

Boblin and Gibson (1986) found that students in the class using CMI achieved higher marks "for all of the unit tests, for the average of the unit tests and for the final course mark" when compared to the previous three (non-CMI) classes. An interesting problem which resulted from the use of CMI was identified:

Boblin and Gibson reported that students were more comfortable with computers after the course; instructors were not, and instructors reported fewer testing/retesting clerical duties and consequently more time for instruction (time estimates not identified).

Boblin and Gibson (1986) state that "Records on the PLATO system show that students spent an average of 2.5 hours/week on the system" (p. 95). Further, while specific time was scheduled into student timetables for access to PLATO, only of students actually used PLATO most often during scheduled time. 95% of students indicated that they "...liked and took advantage of being allowed access to the computers at times other than those scheduled" (pp. 94-95). During the time of their study, PLATO was available 22 hours a day, 7 days a week, with up time in excess of 99.7%. Boblin and Gibson (1986) made an extensive effort to keep track of time and costs. Tables 1 and 2 report the manpower requirements and the total cost of their development of 21 modules and 6 exams. Discounting costs for hardware which was used for developmental purposes only, the cost to design the program and run it on a mainframe system for one year was under $30,000.

The Alberta Vocational Centre Studies

The Alberta Vocational Centre/Edmonton (now Alberta Vocational College) has had a continuing involvement with CBI. This originated because of the initial involvement of AVC/Edmonton in the Canadian validation of the Basic Skills curriculum delivered on the PLATO system. Fahy (1985) reports on the development of four CMI projects where PLATO PLM was piloted in existing adult basic education programs. Each of the four projects in the study was specifically designed to facilitate student self-direction and self-pacing in order to see whether current theory in adult education could be put into practice in an existing adult education program. In terms of CMI development time, other data are helpful.

Fahy (personal correspondence) kept a log of developing a CMI application using the PLATO PLM system and existing instructional materials. His data suggests that an existing course can be developed in CMI format in 5 minutes per test item. Extrapolated, a course of 1200 test items requires a development time of 100 hours. The U of A PLATO Instructional Systems Group commonly used the figure of 20 minutes per test item when bidding commercial CMI development projects for PLM. While this is four times the estimate of Fahy, it must be remembered that Fahy was using existing instructional material in an internal development, while a commercial developer must be extremely conservative when estimating time and resources.

With the success of the use of CMI with the adult basic education program, the AVC/Edmonton investigated the feasibility of becoming more involved in CBI. An external contract was let to undertake a study to "investigate and report on the feasibility of implementing computer assistance in operating and managing student record keeping and testing activities as a means of attaining program objectives. The study is to consider both administrative and educational management practices and needs" (Montgomerie, 1985, pp.85-87). The resulting study made six recommendations:

Alberta Vocational Centre/Edmonton incorporated CBI in the Nursing Assistant Program. Fahy (1987) reported on this implementation: All but one recommendation of the Montgomerie report (1985) were met fully and one recommendation met partially in this PLATO-based CML implementation project. The unimplemented recommendation, reiterated in this study, was for an institutional instructional design group (with project management responsibilities).

Fahy (1987) gave a pre- and post-questionnaire to students . He reports that

Fahy went on to report that, in interviews, students' major complaints about the PLATO system involved accessibility (number of terminals, hours of operation, etc.), privacy (dividers between stations, and "staff should not watch students over-the-shoulder while they are testing"), and lack of noise insulation (p. 14).

Fahy (1985, p. 16) found that "...those [instructors] who actually experienced individualization in the projects found that computer management is capable of addressing many instructor reservations about, and of providing a relatively painless introduction to, individualization." Students were very positive about the use of CMI, reporting that they felt

Instructors reported fewer testing/retesting clerical duties and consequently more time for instruction (Fahy, 1987). Fahy (1987, p. 21) compared the average cost per student using PLATO CMI with more traditional testing. The costs (in 1987 $) are summarized in the Table 3.

The Police Studies

Szabo (1987) evaluated a project in which a major city police force converted its training section to CBI. The system made heavy use of PLATO-PLM; it also used CAI modules and short videotaped demonstrations. Thirty-two modules of instruction were converted from conventional instruction to CBI delivery over a 3 year period and the CBI modules were phased in as they were completed and validated. The evaluation study involved approximately 1100 trainees using the materials over a period of one year. As an experimental study was not possible, conclusions were drawn based on comparative data gathered during the six month period prior to the introduction of specific modules.

Table 3. Costs of PLATO-Based CMI and Other Delivery Modes

The Szabo (1987) study examined indicators of both paper and pencil exam achievement as well as selected on-the-job performance measures. In those measures available, trainees scored significantly higher if they were trained using CBI as compared with conventional (previous) training. Included in the data were such measures as scores on promotional exams, use of canine unit, number of arrests made using the canine unit, and fewer costly clerical errors on critical arrest warrants. The police department study resulted in the re-assignment of six full-time instructors to their regular duties after the CBI modules were fully operational. This resulted in significant cost savings, as the instruction normally required one full year of each instructor. In the police study, the CBI instruction resulted in an average reduction in student completion time of 25%. The salary savings from this reduced time plus other considerations led to the conclusion that the CBI system resulted in a cost savings of $350,000 over a five-year period.

Moisey (1988) undertook to investigate the effects of testing frequency and feedback upon achievement and retention in a series of CAI modules in Canadian law. A sample of 171 volunteer police officers in a large urban force were divided into eight treatment groups. Four variations of testing were provided: after each module, after two modules, after four modules or after eight modules. Every subject received the same number of test questions. Two types of feedback were provided: immediately following each item, or after a 24 hour delay. All groups had significantly higher achievement and retention (10 week delay) scores than pretest scores. No significant differences were found for test frequency or feedback on the pretest, achievement test or retention test. More frequently tested groups had significantly higher module test scores than less frequently tested groups. More frequently tested groups spent significantly less time studying CAI modules than did less frequently tested groups and groups which received delayed feedback spent significantly more time on tests than groups which received immediate feedback. Moisey summarized her results by stating that

Computer Based Testing Studies

While we are careful to distinguish that computer based testing (CBT) is not CMI, there is a transferability of much of the work in CBT to CMI. Carbonaro (1988) undertook an extensive review of the capabilities of computerized test item banking programs and found fifty five individual features. He then surveyed 350 high school teachers in Alberta as to whether a computerized test item banking system would be useful to them and which features they would like to have. This study provides a comprehensive synthesis of computerized test item banking features.

Rumbolt (1989) undertook a study of the use of CBT in an apprenticeship training program. The study compared two systems of administration of the same test — one in which on-line testing using Quizmaster (a microcomputer CBI program) was used, and the other in which paper and pencil tests (using the same questions) were generated using the Computer Based Training System (a mainframe CMI system). A quasi-experimental design was used in which the two experimental groups "flip-flopped" between on-line and pencil and paper tests. A third (control) group was given only pencil and paper tests.

Rumbolt found no significant difference in the level of achievement between those doing on-line testing and those doing paper and pencil tests. Further, he found no significant difference in the attitude towards computers or in test anxiety between those doing on-line testing and those doing paper and pencil tests.

Rumbolt was particularly interested in actual cheating, or opportunity to cheat. No actual cheating was observed, and based upon responses to questions about cheating, Rumbolt states that "the on-line testing method was perceived as the system by which students were less likely to be able to cheat" (1989, p. 75).

Conclusion

An examination of these research studies shows that different studies concentrated on different aspects of CMI. The following areas were addressed by one or more of the studies: achievement, attitudes, cheating, instructor and student time, development time and costs, and delivery costs.

The research on pure CMI is thin. Much of the research on CBI includes components of CMI, so it is difficult to isolate the specific contribution made by CMI to the effectiveness of the instruction, attitudes towards computers, etc. Although there are single studies which do not confirm support for CMI, any scientist bases conclusions not on single studies but on the overall effect indicated by several similar studies. The very strong impression gained from a review of the studies reported here is that CMI has shown to be efficient, effective and relatively easy to develop compared with other forms of mediated instruction.

Experience indicates that CMI can be implemented by instructors, working alone or in teams. A team may or may not include computer analysts, depending upon the transparency and user interface of the particular CMI system being used. While there are advantages and disadvantages to working in teams, the former tend to outweigh the latter. A comment often made within developmental teams is that one of the major benefits is the logical re-thinking of a course or curriculum required when implementing CMI.

CMI has been shown to be useful in the context of existing courses (e.g., Anderson, et al, 1975; Boblin & Gibson, 1986) as well as in brand new courses (Szabo & Estes, 1978). It has also been used as mainline instruction or as supplementary instruction, (the latter required or optional). CMI has been successfully applied over a wide range of content areas, ranging from nursing to special education to pilot training.

Although not systematically reported here, there is a rapid proliferation of microcomputer based CMI applications and programs as people begin to apply the microcomputer to the management of student learning. The only bit of advice we would offer here is for aspiring developers to carefully examine the features of the most outstanding mainframe-based CMI systems and avoid reinvention of the CMI wheel.

One might criticize CMI for being costly to develop and deliver. However, such conclusions need to be evaluated in the light of several factors. In spite of years of formal instruction using conventional methods, there is not a clear picture of what it costs to create that conventional instruction.

In developing CMI, the heavy effort is at the beginning — in the design and development phases (which can be condensed if a good pool of items/objectives/ learning resources is available). Once the system is in place, it can be delivered repeatedly with minimal instructor involvement in clerical matters of test administration, etc.

The cost of computer hardware and software to handle CMI has steadily declined and by all indications will continue to decline.

There are costs which are incalculable but are much more important than hardware and software costs. One of these is the tremendous price a country pays in lack of basic literacy, a problem which can be partially addressed through individualized instruction. In addition, what costs are paid by educational institutions which are unable to lead in instructional technology and, in many cases, don't even keep up to provide a good 'mirror' of society and its march to technology?

Regardless of what we as instructors would like to believe about our work, many of our students will do anything to avoid our lectures; most of which do not contribute as much to learning as we instructors would like to believe. The data suggests that CMI can play a useful and productive role while satisfying student urges.

Return to Table of Contents

3.0 STATIC VISUAL DISPLAYS

SVDs in instruction is defined as any representation of an object, concept or process, as perceived through the eye, which does not rely on the use of text or numbers. Historically these were referred to as graphics but that term covers more than this focussed topic. Extensive research on SVDs occurred during the 1960's and 80's and has diminished in recent years.

Definition

SVDs can represent relationships in space and structure when text material is abstract or difficult to comprehend or express verbally. They can also provide relevant detail when it is needed. SVDs may also provide an organizing structure the student can use to visualize some process or sequence which otherwise may have to be constructed from text. Also, SVDs may provide particular advantages to students who lack skills, abilities or prior knowledge of the topic (Wetzel, Radtke, & Stern, 1994).

Research Findings

It was found that SVDs increased the amount learned by adults (Alesandrini, 1985) and by children (Pressley, 1977). Alesandrini and Rigney (1981) found SVDs to be an effective review strategy compared with verbal strategies. In a study which examined student attitude toward SVD-based learning, Rigney and Lutz (1976) concluded that using SVDs as analogies in CBI resulted in high levels of satisfaction with the learning experience.

The most definitive work on SVDs has been done in a series of controlled studies by Dwyer. This work shows that visuals which emphasize critical details which are relevant to the learning outcomes are most effective. Conversely, addition of visual realism to SVDs does not increase learning. Indeed added unnecessary detail can add to learning time without increasing achievement when time for learning is constrained, such as in a fixed classroom (Dwyer, 1967, 1968, 1970, 1981, 1987). Finally, simplification of visuals could reduce lesson development costs significantly, although this action may conflict with developers who use the artistic approach to instructional design.

Dwyer (1970) showed that simple line drawing SVDs tend to be superior to photographs or other more realistic drawings. The key seems to be the relevance of the cues to the learning task. For example, using a photograph of a car engine to teach about the location of the carburetor might be appropriate in terms of relevant cues, whereas the same photograph would be inappropriate to teach about the structure and function of the carb itself.

Joseph and Dwyer (1982) concluded that the integration of realistic and abstract SVDs may reduce achievement differences between students of different ability levels and appears to interact with the concreteness or abstractness of the topic to be learned.

Levie and Lentz (1982) conducted a meta-analysis of 46 comparisons between text which did or did not involve SVDs. Their findings confirmed those noted above of Dwyer and added that SVDs did not transfer their ability to improve learning to non-illustrated text. They developed a list of guidelines which are presented later in this paper.

Willows (1978) was concerned about potential interference between the messages provided by text and SVDs. However with CBI, one can control the presentation to avoid such conflicts.

Rigney and Lutz (1976) elaborated effects of SVDs in instruction by urging students to form mental images as they studied the material. Increased learning resulted from this combination. These results are supported by the research of Canelos (1979) who showed that training in visual imagery techniques resulted in better learning on a highly visual, real-world memory task.

Levin, Anglin & Carney (1987) identified 5 different purposes of SVDs in text learning; decoration, representation, organization, interpretation, and transformation. Winn (1990) cautioned that the use of diagrams, charts, and graphs may not be self-explanatory; that is they depend on whether the learner can process the information given, understands the conventions used, is asked to inspect them, and whether there is supporting text.

Finally, the use of SVDs in the process of testing shows that students who receive the same SVDs as part of their testing and instruction score higher on some portions of achievement than students whose testing is limited to verbal formats (Szabo, DeMelo, & Dwyer, 1981; Szabo, Poon, & Ally, 1997).

Return to Table of Contents

4.0 AUDIO

The literature review by Nocente (1996) is most instructive on the use of audio in learning. The technical aspects of audio have changed over the years. Figure 2 below summarizes the characteristics of storage requirements, search time, editing time, ability to edit and long term stability of the medium of the three modes of audio playback. As digital compression algorithms improve, greater storage capacity will result, but at the expense of reduced audio quality. The technical development is ongoing in digital storage and minimal in the tape and videodisc modes.

Figure 2. A Comparison of Selected Technical Aspects of Audio Playback Modes.

Audio may be used to provide information which is redundant, contradictory, enhanced or unrelated to other input. The research findings supporting achievement gains via audio are weak or non-existent. Barton & Dwyer (1987) concluded that students with higher verbal skills do not profit significantly from the use of audio. Barron & Kysilka (1992) found no significant effects for immediate recall between text-based and text plus redundant audio. The latter group however required more time to complete the instruction.

The appearance of desktop audio has stimulated several studies in which audio has been incorporated into computer based instruction lessons. Rehaag & Szabo (1995) and Barron & Atkins (1994) tested redundant audio in a tenth grade mathematics course and university students in a computers in instruction course, respectively. Both studies reported no significant differences for achievement or student attitude. The Rehaag study however found that higher ability students in the CBI with audio group took more overall time to learn than the controls, but took less time to complete built-in practice questions. Lower ability students in the CBI plus audio group were generally more positive about the learning experience.

LaRose, et al. (1998) found that students in traditional lecture were compared with students who learned via pre recorded audio in WWW (audiographic telecourse), detailed course outlines and related courses pages on the WWW. Test scores between two groups were equivalent. Student attitude ratings across the two groups were equivalent.

Can it be that audio limits the ability of students to learn at their own, individual rate of learning and thus defeats self-pacing? We know that the learning rate in a typical classroom varies by as much as 5 to 1. This issue is fairly crucial in the ability of a CBI program to allow adaptation to individual learning rates.

Audio and Human-Computer Interaction

Human-computer interaction has focused primarily on the visual and has almost completely neglected the use of audio (Aarntzen, 1993; Buxton, 1987; Buxton, 1989; Edwards, 1989; Gaver, 1989). Jaspers (1994a) conducted an extensive review of the literature spanning 30 years and only found 80 items regarding audio in film and educational media. The reasons immediately apparent for this neglect are:

Recently synthesized speech has been improved substantially and has become far more intelligible. Compression and decompression techniques have drastically reduced the amount of space digitized audio will occupy and storage space is becoming less and less of a with the advent of CD-ROM. With the technical problems almost resolved, research has begun in both the area of speech and non-speech audio.

Non-Speech and Speech Audio

Buxton (1989) places these audio messages into three categories: (1) alarms and warning systems are signals that interrupt an ongoing task, (2) status and monitoring indicators provide information about an ongoing task such as keys clicking while typing, and (3) encoded messages and data are used to present quantitative data as in the case with sound graphs. Buxton suggests that certain non-speech sounds convey critical information, effectively communicating higher level messages.

The move toward the graphical user interface has put more emphasis on visual ability to operate a microcomputer (Edwards, 1989). Buxton (1989) and Blattner, Sumikawa, and Greenberg (1989) suggest non-speech audio can play an important role in providing cues when the visual channel is overloaded. With this rationale, Blattner et al. (1989) have developed "earcons" to reduce the cognitive overload of visual displays. These earcons range from being a simple pitch representing a common entity such a key click every time a character is deleted to more complex combinations of audio elements representing the creation of a file.

Gaver (1989) believes that "sound and vision are complementary modes of information" (p. 70) which convey different information. Along with the reasons already mentioned for using audio, Gaver believes that "Auditory information can be redundant with visual information, so the strengths of each mode can be exploited" (p. 72). Gaver used everyday sounds to convey a message and hence reduced cognitive overload. For example, when a file was being copied the visual cue was accompanied with the sound of a liquid being poured into a container. The audio cue would allow the user to focus on something else until the pouring was completed.

CAI speech can be produced in three ways on the computer: digitized, synthetic, or encoded (Parham,1988). In digitized speech, the spoken words or taped words can be digitized and played back directly by a computer. The quality of the speech depends on the amount of compression and the sampling rate. If speech is limited to a set of selected words, a less memory intensive technique can be used to produce high quality speech.

Synthesized speech is used when the words cannot possibly be prespecified. In synthesized speech, conversion tables and algorithms are used to convert text to phonemes (a set of sounds) then to digital data using rules for word pronunciation and their exceptions, rhythm and intonation, and rules about the duration of sound (Kaplan & Lerner, 1985; O'Malley, 1990). The more rules used to refine the speech and the larger the conversion tables, the more memory is required.

Pisoni, Nusbaum & Greene (1985) found that their subjects quickly became accustomed to the synthetic speech but it required more cognitive effort to encode than natural speech. Greater demands were placed on short term memory which then effected retention. They concluded that synthetic speech imposes greater capacity demands on the human information processing system which in turn may interfere with other cognitive processes. An instructional designer would avoid synthesized speech when presenting new information but could implement it in material that was intended for review purposes.

The audio feature must be used to meet certain objectives. In certain simulations, sounds that would actually be heard in the real life situation can bring more reality into the courseware. For example, the development of radar maintenance training for the Canadian Forces used audio to help the trainee determine which diagnostic unit required repair. The radar diagnostic unit has a particular humming sound when functioning properly, but the sound changes distinctively when a fault has occurred.

With the use of computers now becoming prevalent in schools, the advantages of audiocassettes, including the possibility for stopping, going backwards, and repeating a section can be magnified through CAI. Computer capabilities such as student tracking and increased interactivity make Computer Assisted Language Learning (CALL) desirable courseware for schools (Haddad, 1989).

There is some evidence that when reading from a computer screen, reading rate is lower (Yeaman, 1987). The more experience individuals have with computer text reading, the less likely reading rate will be affected (Feeley & Wepner, 1986). Fatigue may be another factor in screen reading and seems to occur more quickly in screen reading than in conventional reading (Hathaway, 1984). An audio component that reads the text to the student may help to alleviate fatigue. From an instructional design perspective, would the reasons cited above justify the expense of integrating audio into a CAI program?

Learning Style and Preference for Modality of Instruction

Instructional designers make reference to learner characteristics and accommodating learning style (Gagne, 1987; Lepper & Chabay, 1985; Reigeluth, 1983). Gagne (1987) says, "Development of rationally sound instructional procedures must take into account learner characteristics such as innate capacities, experiential maturity, and current knowledge states" (p. 5). Salomon and Gardner (1986) say:

The problem with learner characteristics is the area of learning style or what Gagne refers to as "innate capacity." Because learner style is not easily determined nor is an instructional prescription easily made, there is a continuing debate in the literature over learning style and its subset sensory modality preference.

Dunn et al. developed numerous instruments between 1977 and 1984 to measure learning style (Dunn, 1988). A section of Dunn's inventory is used to measure modality preferences, an area of modality preference that seems to have created the greatest debate even before Dunn's research commenced. In a study conducted at SAIT, Shewchuk (1997) found no correlation between various aspects of Dunn’s learning styles and achievement in a computerized testing course. The learning styles tested included visual, verbal, and preference for time of day or formal vs informal learning environment.

Daniel and Tacker (1974) hypothesized that memory would be enhanced if the student's modality preference was matched for stimulus presentation. Students demonstrated higher recall if taught in their preferred modality (auditory, visual, none) and performed worse if taught through their unpreferred modality. In another study, researchers found that there were no significant differences between the groups (Ringler & Lenore 1971). Students taught in their preferred modality did as well as those taught in their unpreferred modality. Kampwirth and Bates (1980) reviewed 22 studies on modality preferences and found that only two contained positive results. Likewise, in their review of the literature, Tarver and Dawson (1978) found that although modality preference instruction has intuitive appeal, there really is very little evidence to substantiate its use.

Along with the inconclusive results no one seems to be able to adequately explain why teaching in a student's preferred modality facilitates learning. Because of this, Snider (1990) cautions against providing specific treatment for a learning style. Rather, information should be provided in a variety of modalities because the tests used cannot be relied on to characterize an individual into a particular learning style.

Summary

The educational technology researchers discussed above and others (Edwards, 1989; Buxton, 1989) believe audio needs to be integrated into CAI partly because it is available and it is a natural means of communication for humans. However, instructional designers under time and financial constraints need more justification than this to integrate audio into CAI. They need to be able to rationalize the integration in terms of cost benefits. It is apparent after reviewing the literature, that the controversy over modality based research is far from resolved and seems to have a resurgence every decade.

Improvements in computer technology have made audio integration more feasible than in the past. This presents a new challenge to instructional designers for they must determine if audio integration leads to learning gains and is cost beneficial. Some uses of audio are clearly evident. The visually impaired need an audio interface. Training in any area that requires an audio component, such as music and language development, can now be accommodated in CAI. Lower ability readers benefit from a read-a-long format and from the potential interactivity available from the computer if its capabilities are taken advantage of.

From an instructional point of view there has to be a clear benefit to the learner before it can be integrated. Several benefits may be possible but require further research to substantiate the claims made. It has become apparent that as students get into their mid teens they can process information equally well in either visual or auditory modes; however, lower ability students do seem to continue preferring one modality.

Possibly both modality preference and redundancy have a role to play in meeting the learner’s needs but so many other variables are involved. The content may affect how the learner will benefit from receiving information in one or two modes. Since there are so many possibilities, it is appropriate to let the learner decide if audio is needed. Students may turn on the audio because it is their preferred modality or they may turn it on because they want the redundancy.

Return to Table of Contents

5.0 COLOR/SCREEN DESIGN

Most instructors would agree that color used in instruction (1) increases learning, (2) makes the instructional environment more appealing and therefore increases attention and motivation to learn, and (3) does not distract from the learning tasks at hand Unfortunately, they are not completely correct.

Color

Extensive research on using color to increase the level of reality in an instructional situation shows that learning is not generally enhanced by color (Wise, 1982; Dwyer, 1967, 1968, 1970). The exception to these finding occurs when the instructional objectives require the learning of color elements (Dwyer, 1987). Examples include learning color-coded electrical components, flowers, and optical spectroscopy. Apparently when color elements are not required to meet the objectives, they provide neutral cues to the learner. The worst case is that color provides cues which distracts the learner from the desired elements (Dwyer, 1970).

Although there is evidence that color-cueing can increase the speed at which persons search through lists, learning does not seem to be affected. Dwyer (1981) discovered situations where the presence of color interfered with learning of the required tasks, perhaps due to irrelevant cues provided or a drawing of attention to the color and less attendance to the task to be learned.

A study by Livingston (1991) found that the use of color interferes with learning while England (1984) and Reilly & Roach (1986) cited by Kanuka (1997) concluded that too many colors will reduce the legibilty of a presentation. It was suggested that a maximum of four colors be used in screen displays (Yang & Moore, 1996), although Rambally & Rambally (1987) allow that up to 11 colors is acceptable in screen displays.

Dolsky (1993) studied adult preferences for color and color combinations in instruction. From the 70 different display combinations, she found that combinations of vivid colors such as red, purple, black, blue and magenta, were preferred over less vivid colors such as green, yellow and orange. The color combinations least preferred were green and yellow and green and orange.

Instructors will often cite the increased motivation from color along with the need to have colorful presentations to hold students' attention in today's color-rich world. Commercial publishers will not consider any monochromatic instructional materials. The disparity between effectiveness and perceived effectiveness is nowhere as great as it is in the realm of color.

Someone once said that the best thing about microcomputers is that you can customize them to your unique needs and specifications. When it comes to delivery on other micros, the worst thing about microcomputers is that they can be customized to someone's unique needs and specifications, including colors! The effects seen on one machine (the developer's machine) can change dramatically when seen on a student machine.

The use of color may also place additional demands of an efficiency nature on instructional material which can interact with other factors in a negative way. For example, heavy use of color can degrade the speed of execution of lessons on stand alone micros. When run in a network environment the degradation can be significant. This raises the related issue of unleashing full creative power in lessons at the risk of them not being able to run acceptably in older micros.

Screen Design

Screen design is the marriage of art and technical design skills. While there have been extensive writings and studies about screen design, no previous studies were located that examined the effect of screen design on achievement. Szabo and Kanuka (1998) conducted a study on the effects of screen design on achievement, instructional time, and rate of completion. Using four accepted screen design principles, two CAI lessons on the topic of writing a term paper were created. One lesson used the design principles well, the other violated them. Students who received the poorly designed lesson a) achieved the same amount on a paper and pencil test, b) took significantly more time to complete the lesson and c) were less likely to complete the lesson compared with those who received the well designed lesson. It was suggested that when students study, they often go into an ‘automatic processing mode’, in which they accept things at face value. Well designed lessons encourage automatic processing. When something in the instructional presentation is new, different, or unusual, the automatic processing mode is interrupted. In this case, the interruption may have had the effect of slowing the learning rate or decreasing motivation to continue.

Return to Table of Contents

6.0 ANIMATION

Animation does not have the rich research history that is associated with color and graphics. This may be due to the fact that until recently, creation of animation required mainframe computers with complex software and high skill levels. Today, basic animations can be created using micros and relatively inexpensive and easy to use software.

Film Research

A precursor to animation research may be found in early research on movies. The two major reviews of research on film (Hoban and Ormer, 1950; Reid & MacLennan, 1967) confirmed no patterns of significant difference and set the stage to investigate specific variables within films which might contribute to effectiveness. In an early study using movies to provide animation, motion was shown to be superior to slides or sequential photos in presenting time- and motion-based concepts (Wells, 1973). The reverse was true for concepts involving the presentation of spatial entities.

Computer-Delivered Animation

Baek and Layne (1988) compared learning conditions of text only, text plus graphics, and text plus animation. The adults in the study scored higher in the animation condition than either text or graphics. The animation condition also resulted in less study time, suggesting that animation results in more efficient learning. In another study with adult learners, Mayton (1991) found increased scores in the animation condition immediately after study; the effects persisted and were measurable one week later. Szabo & Poohkay (1996) administered three versions of a math tutorial on how to construct triangles. One version used computer animation, another version used the same visuals but in static mode, and the third version used only text. Achievement was significantly greater among the three treatments with the animation version resulting in the highest scores and text in the lowest. The results followed the same pattern when students' opinion toward the method of instruction was assessed.

Rieber and Boyce (1990) compared animation-based instruction with carefully designed verbal presentations which used highly imaginative examples and illustrations. The results with an adult sample indicated no significant difference in the amount learned but the animation group required less time to retrieve the information they learned.

Of 27 experimental studies that investigated animation in the context of computer-based instruction, 15 showed full or partial significant effects with animation (Alesandrini, 1982; Alesandrini & Rigney, 1981; Baek & Layne, 1988; Carpenter & Just, 1992; Collins, Adams, & Pew, 1978; Crosby & Stelovsky, 1995; Kaiser, Proffitt, & Anderson, 1985; Mayton, 1991; Park, 1998; Park, & Gittelman, 1992; Rieber, 1989; Rieber & Boyce, 1990; Reiber, Boyce, & Assad, 1990; Rigney & Lutz, 1976; Szabo & Poohkay, 1996). Twelve showed no significant differences (Caraballo, 1985; Caraballo-Rios, 1985; Doll, 1986; King, 1975; McCloskey & Kohl, 1983; Moore, Nawrocki, & Simutis, 1979; Park, 1998; Peters & Daiker, 1982; Reed, 1985; Reiber, 1990; Reiber & Hannafin, 1988; Szabo & Schlender, 1996). It might be instructive to analyze the two groups of studies to determine their commonalities and differences.

Any widespread belief in the superiority of animated over non-animated instruction within the context of computer-based instruction is at odds with the research findings. Park (1998) observed "Visual displays, whether animated or static, facilitate learning only when their attributes are congruent with the specific learning requirements of the given task." And "more meaningful learning is fostered when visual displays are presented with adequate narratives explaining their instructional roles in the given content." (p. 38).

Return to Table of Contents

7.0 MULTIPLE-CHANNEL LEARNING

Single Channel Learning Theory

Recently microcomputers have become capable of displaying an increasing array of multimedia elements. This change makes it easier to transmit information through various senses (visual, auditory, etc.) and gives reason to re-open the discussion on learning from multiple senses, or channels. For a long time it was assumed that information which was presented using several simultaneous channels would result in increased learning (Schramm, 1973).

Broadbent's theory of one channel processing (Hsia, 1968a) states that humans are only capable of processing information through one channel at a time and that it is not possible to process two simultaneously. To do so would only result in audio and visual stimuli arriving at the central nervous system simultaneously, causing the information to jam, leading to poorer retention of material (Broadbent, 1958).

A single-channel theory was developed in which multiple inputs are assumed to form a single channel input into the central nervous system (Travers, 1964). This theory suggests multiple channel input should not present any learning advantages. The single and multiple channel theories were developed with non-visual modalities in mind. Consideration of SVD and DVD led to extensions of the theory.

Multiple Channel Learning Theory

Some research on multi-channel learning questions whether learning is enhanced or inhibited by being limited to single channels (Jaspers, 1991). Paivio (1986) proposed a dual-coding theory in which pictorial and linguistic information are processed differently. One implication is that different forms of modalities, such as text-and visually-based would strengthen learning using either alone. Nugent (1982) reported increased achievement when a combination of images and audio were used but no achievement increases when audio and text were combined.

This notion has been supported in other theory, including Bruner's famous tactile-iconic-symbolic forms of instruction and Piaget's notion of the growth of intellectual ability in just the reverse order as presented by Bruner. We have a great deal to learn before we come to powerful understandings about multi-channel learning. The rapid growth of the ability of the desktop computer to handle the technical aspects of IT has outpaced our research into how learning takes place under different combinations of media.

Another group of researchers are of the opinion that redundancy of information will lead to increased retention. Hartman (1961a) concluded that print alone should be used by literate students for complicated material and that audio is beneficial when the material is simple or the subjects are illiterate. Hartman’s results indicated that greater learning occurred when both audio and print were used simultaneously instead of alone.

Baggett and Ehrenfeucht (1983) found that when college students were presented with both visual and verbal/auditory information there was no competition for resources. They researchers concluded that encoding information in one medium does not hinder the other. Extraction of information from the sequential presentation was not increased even though the amount of time spent on instruction was doubled. The students in this study would also process equally well if the information was read rather than listened to.

Hsia (1968a) contends that the optimal level of redundancy can only be ascertained when environmental differences and individual information processing capabilities are taken into account. If this is the case, learners should be given the control to use one or two channels when they feel it is best.

In one of his first studies, Hsia (1968a) found that as the rate of presentation increased, the subject's performance deteriorated. The subjects seemed to make better use of one modality once pressured by the density or rate of presentation of information. In another study, Hsia (1969) compared high intelligence subjects to low intelligence subjects in the presentation of audio, visual, and redundant audio-visual information. The use of audio-visual channels seemed to benefit low intelligence subjects more in the recall of information and in the reduction of equivocation.

Strang (1973) prepared an individualized instructional unit on automotive repair. One group received pictorial/print, another group had pictorials/audio, and a third group had pictorial/print/audio. This third group asked for instructor assistance less than the other groups and was able to complete the unit in less time. Good readers could ignore the audio and move ahead quickly. Poorer readers benefited from the redundancy by listening to the audio and reading along. In a public setting, students preferred to receive information via multiple forms of media if something unknown were to be revealed, but the choice of audio per se was quite low (Wu & Martin, 1997).

Not all research supports the multiple channel theory. There are also studies that indicate there is no significant difference in retention (Holliday, 1971; Severin, 1967; Wu & Dwyer, 1990, Szabo & Rehaag, 1995). These researchers do not support Broadbent’s theory nor do they accept the benefits reported for visual-audio redundancy. In an effort to determine the effects of redundancy on learning, Holliday (1971) presented students with print, print/audio, and just audio. There were no significant differences on the print-based test. Researchers such as Beck and Hartman would probably contend that if the test had been delivered using the same modality as in the presentation, significant gains in retention would have been made.

Several researchers contend that some individuals process information better through the auditory channel rather than the visual channel (Carbo, 1978; Dunn, 1988; Dunn, 1990; Barbe, Swassing, & Milone, 1979). Another group believes that the simultaneous visual and auditory presentation of content will increase retention (Hartman, 1961a; Hsia, 1968b; Hsia, 1969; Nasser & McEwan, 1976). In terms of CAI presentation, there is some question about the readability of the screen (Muter, Latremouill, Treurniet, & Beam, 1982; Reinking, 1989).

Severin (1967) found that redundant use of print and audio did not increase scores and suggested this is because the added channels are not actually providing additional cues, except for poor readers since these individuals are likely to benefit from the redundancy.

The visual/auditory presentation of content has been suggested for journaling systems (Gresham, Ferguson, and Allen, 1988). Journaling systems are walk-throughs of computer applications. For example, a program such as Spectator captures screen changes (that is, mouse movements, pulling down menus, etc.), and then replays them. This visual display results in an animation that can be accompanied with print or audio. The audio could be listened to as a process is being demonstrated rather than have the learner attend to both animation and text simultaneously. Fleming and Levie's (1978) emphasize the following:

With animation occurring on the screen, audio seems to be a sensible addition to the CAI.

Grecsei and Girard (1990) have developed a computer interface that gives the student the impression that the speaker on the video is speaking to him/her using a text-to-speech synthesizer and a videodisk with hundreds of animated mouth movements. Various facial expressions were captured and stored on videodisk. After text was entered on the computer each word was translated into its phonetic form. Then each phoneme was matched to a corresponding facial image on the videodisk. The speech and videodisk frames were synchronized and played back to the user giving the impression of a person speaking. From an instructional designer’s perspective this would initially be an expensive venture to take on unless future benefits could be proven.

Return to Table of Contents

8.0 NAVIGATION & HYPERMEDIA

The advent of the Internet and World Wide Web has ushered in a new era which is characterized by the possibility that one’s desktop can be electronically connected to EVERY OTHER DESKTOP IN THE WORLD. Among the many implications of this rapidly growing phenomenon, which is being hailed by many enthusiasts as the next panacea for all our educational ills, is the process of searching for and finding the information one seeks in this network, which has been expanded into multimedia through World Wide Web browsers. Navigation may be defined as a search through a complex and abstract database using some systematic plan, regardless how the database is organized in order to locate data or information. This definition excludes the process of validating the results of the search.

The word navigation conjures up in one’s mind the vision of a beautiful vessel, churning through the seas, skippered by an individual who has experience, training, all the latest navigation aids, and a clear plan. The reality of navigation on the net at the present stage is more like two guys in a dinghy with no knowledge of the sea, no experience, no provisions who decide to sail to Hawaii from Vancouver! Therein lies a major problem--how to navigate the net and find the desired and needed information, and do so effectively and efficiently. As one speaker once said, you could learn to deliver babies in a hospital through the internet, but without guidance and direction, you might spend the majority of your medical school training finding the necessary information.

The problem of navigation is a classic one of the different perceptions of underlying structure between the learner and designer of the material. Thus it can be hypothesized that the greater the disparity between the mental model of the underlying structure held by the developer and recipient, the more difficult navigation will be. Consider these questions:

•  
• How can the user navigate the system without help?
•  
• What are the important individual differences among users navigating through the program?
•  
• How many and what kind of navigational aids should be available to the user?
•  
• Which forms of tips are best suited as navigational aids?
•  
• How structured should the case study be?
•  
• How should this structure be presented to the user?

Achievement

In spite of claims regarding the potential benefits of using hypermedia in education and controversy about the relationship between media and learning, research results comparing the effects of hypermedia and traditional instruction are in conflict. For example, significant gains of hypermedia for traditional instruction are reported by Bain, Houghton, Sah & Carroll (1992), Barnes (1994), Chen (1993), Delclos & Hartman (1993), Gretes & Green (1994), Lui & Reed (1995), Overbaugh (1995), Smith, Jones, & Waugh (1986), and Toro (1995).

Due to the recent arrival of hypermedia, few studies of hypermedia on educational outcomes appeared before 1993 (Liao, 1998). Two recent meta-analyses of effects of hypermedia on learning across 35 and 43 research studies, respectively, concluded that CAI and text influenced learning the most, followed by hypermedia, traditional instruction and videotape instruction. (Liao, 1998, 1999). The 1998 study, which covered research published between 1986 and 1997 reported a modest but significant effect size of 0.48 or a percentile of 68 on achievement.

Dillon & Gabbard (1998) conducted an extensive review of quantitative studies " of learner comprehension compared across hypermedia and other media, effects on learning outcome offered by increased learner control in hypermedia environments, and the individual differences that exist in learner responses to hypermedia. It is concluded that, to date, the benefits of hypermedia in education are limited to learning tasks reliant on repeated manipulation and searching of information and are differentially distributed across learners depending on their ability and preferred learning style" (p.322).

Balcezak, et al. (1998) evaluated web-based risk management and medical-legal curriculum for graduate medical education in which a course in risk management and medical-legal issues was taught with additional material on the web. Students who browsed the web material scored significantly higher on the achievement test compared with those who completed the course without the web material.

Radhakrishnan & Bailey (199 ) concluded that students in web based instruction learned significantly more course content in web-based instruction than in conventional instruction.

Lanza and Roselli (1991) compared learning a programming language using structured and hypertext-based systems. Both groups performed equally well, but the hypertext group exhibited greater variability in achievement, prompting the conclusion that hypertext might be better suited to more able students. Tripp and Roby (1990) compared students learning Japanese words in an unstructured hypertext environment and in an environment with orienting devices, that is, advance organizers and visual metaphors. Students using both advance organizers and visual metaphors scored less well. The authors concluded that the two types of orienting devices used in this study activated conflicting mental models of the Japanese lexicon.

Bughouse is a program on the influence of insects on human culture. Students can access the program through a Browse mode (visual metaphor of a Victorian farmhouse and grounds), an Index mode in which the subjects are arranged alphabetically, and a Guide mode, in which an on-line advisor recognizes patterns in the student’s search and recommends new areas for exploration. Using the criterion of relevant items found, Gay, Trumbull, and Mazur (1991) determined that the Guide and Browse modes were more effective than the Index mode.

Campbell (1989) created a hyperdocument (digital document with hyperlinks to other resources) with video that explored the United States during the 1930’s. Students could browse the data or act as an investigative reporter given a story to write. Rather than a visual metaphor, the index card metaphor was used. One unique aspect of this project was the collaboration between a librarian, an English teacher, and a commercial hyperdocument developer.

Studies finding no significant differences were reported by Azevedo, Shaw & Bret (1995), Barker (1988), D’Alessandro et al. (1993), Hess (1994), Kinzie, Strauss, & Foss (1993), Leonard (1992), McCoy (1994), Rojewski, Gilbert, & Hoy (1994), Sheldon (1995), and Tabar (1991).

Johnson, et al. (2000) compared learner satisfaction and learning outcomes in online and face-to-face learning environments. Nineteen grad students learned ISD via face-to-face (F2F) instruction and 19 learned ISD via distance (no F2F instruction). The F2F group produced fewer projects but of equivalent quality compared with non-F2F. Jones (1999) compared an all web-based class to a traditional class, a replication of an earlier study compared university students in a conventional classroom with those taught the same course by web based instruction. The replication found no significant difference in achievement whereas the earlier study reported 20% higher scores for web based instruction. White (1997) compared students who had completed a course via Internet or conventional instruction and concluded there was no significant difference in learning across the two groups.

In a study which employed a problem-based learning environment, Nicaise & Crane (1999) worked with 12 graduate students to served as multimedia designers to assemble information, make connections and conclusions and demonstrate understanding by creating an extensive multimedia, web-based book. Students were highly satisfied with the course and claimed to have learned much about design, theory and technology. Results from in-class probes and examination of final artifacts, however, indicated that most students had but a modest understanding of the concepts under study and one third held astonishing misconceptions.

Day, et al. (1998) found students who completed a writing course on the web exhibited higher achievement scores than those in conventional instruction in the area of communications writings.

Disorientation

As the Web is so new, much of our research knowledge about navigation comes from the literature on searching bibliographic databases using various technologies, including on-line, local, and CD-ROM based sources (Protherore & Wilson, 1994). There are differences among search techniques which may have a bearing on the outcomes of research. For example bibliographic data bases are organized hierarchically while Web databases are organized and can be searched by document title, URL, content or some combination. There are differences in the way they index, search and display results (Courtoise, Baer, & Stark, 1995).

Most navigation studies examined end-user success at the college or university level while using simple or advanced search strategies for bibliographic databases (Johnson, 1997). Although Boolean logic is a dominant and effective search strategy, most end-users do not successfully use it in their search strategies (e.g., Mischo & Lee, 1987; Neuman, 1995). Nor do they select appropriate search terms in their search strategies.

At the college level, Puttapithakporn (1990) found that students made errors when they combined too many search concepts and used parentheses improperly within those operators. In general, however, most end-users performed searches using only one or two keywords and made little use of Boolean logic (Ensor, 1992). College students did little advance planning of search strategies (48% in Faries, 1992). Barbuto and Cevallos (1991) found that few people used a thesaurus for repeat searches, and they did not understand the difference between descriptor and keyword searching. In general, Boolean logic is not used effectively.

Beasley & Waugh (1995) studied cognitive mapping architectures and hypermedia disorientation and found hierarchical maps produced least perceived disorientation, hotwords the most. The browsing device accounted for about 7% of the disorientation reported by users.

Chen, Horney, & Anderson-Inman (1995) used cluster analysis to analyze students' hypertext processing patterns. Their conclusion is that students have a group of identifiable processing patterns into which most fall. Processing is not usually idiosyncratic to individuals.

Felix, Graf, & Krueger (1991) studied user interfaces for public information systems. They report that searching at a public kiosk results in shorter completion time and fewer errors for most people, but this was reversed for those with higher education and computer experience

Hill & Hannafin (1997) asked teachers to report on their metacognitive knowledge, orientation, self-efficacy, technical knowledge of the system, and prior knowledge of subject and found (1) students used a wide variety of search strategies, (2) those with high metacognitive knowledge reported higher ability to reflect on search processes and refine their actions and were better oriented to the system, (3) those with low system knowledge used more primitive search strategies, (4) those with more extensive subject knowledge used more advanced search strategies, (5) high perceived disorientation limited effectiveness of browsing and searching. It was concluded that moving to WWW based instruction, teachers need to help learners construct functional mental models of the system and to support learners.

Koved & Schneiderman (1986) compared embedded menus with explicit menus. The former result in greater user satisfaction, fewer screens viewed and more questions answered correctly.

Lai, Y-R & Waugh (1995) looked at effects of three different hypertextual menu designs on various information searching activities and concluded that structure and reference links can affect learning and attitude toward instructional programs.

Marchionini & Schneiderman (1988) compared finding facts vs. browsing knowledge in hypertext systems. They discovered that information searchers prefer using keyword searches or index mechanisms when search questions are clear, but when search questions are vague, searchers resort to browsing or exploring.

Schuerman & Peck (1991) examined pull-down menus, menu design, and usage patterns in computer-assisted instruction. Their main conclusion was that design of a menu system influences learners' usage patterns.

Ventura (1991), studying hypertext usage in a public museum, found touch screen users had slightly shorter sessions and examined less of the program in a museum environment than those using a mouse.

Learner Control of Sequence

One feature of hypertext systems is that students are offered greater control over their search behaviors compared with classroom instruction. Hobbs (1987) examined effects of content sequencing and mode of presentation on recall in learning statistics. He found that presentation mode has no effect on recall, the quantity of recall varies with the amount of time spent learning, and high structure material results in better recall. Gray (1987, 1988) studied sequence control and its effect on learning using menu screens. Compared with students who were constrained to a linear sequence, the students who could move freely within the program performed better on a comprehension level test but the same on a retention test, and had less positive attitudes toward the software. They also performed better on one of the two long term tasks. She also found that student performance was better if the menu system was organized in ways that were meaningful to the students.

Singanayok & Hooper (1998) examined effects of cooperative learning and learner control on students' achievement, option selections and attitude. Sixth graders studied ecological concepts via CAI under (learner or program) control; individual or cooperative learning; and high or low achievement. Effects were assessed on achievement and attitude. The cooperative group scored higher than individual groupings on immediate and delayed achievement and attitude. Interaction between achievement and control showed low achievers retained more under program control and high achievers retained more under learner control. Cooperative learning with learner control scored higher than learner control working individually with the tutorial and checking their own learning.

Attitude & Preferences

Ahmad (1999) studied effectiveness of web-based learning environments in business education. 192 business undergraduates enrolled in an introductory IT course were taught by either web based instruction or 'traditional' instruction. Subjects in WBI reported less satisfied with the learning environment.

In a study of preferences, Chun & Plass (1995) observed that students chose picture and movie links equally over text links and rated picture and movie links equally and better than text links. A study of learning styles revealed no relationships.

Summary & Summary (1998) surveyed students whose instructors used and required web extensively vs. those who did not in freshman economics courses. Student attitude toward the course was higher for higher use of WBI.

Mischo and Lee (1987) and Ankeny (1991) found a large discrepancy between end-users’ perception of success and documented (actual) success. End-users who are trained are generally more satisfied with their search results than those who are not trained (Jackson-Brown & Pershing, 1993).

Students from a community college and a virtual high school system were interviewed as to their preferences for web-based, distance learning formats. A major factor in the choice is that it allows students to control their learning environment, while those who chose conventional F2F formats have a higher need and desire for interaction with instructors and peers (Roblyer, 1999).

Hypermedia vs. Books

In 1988, Vickers and Gaines created a learning system devoted to performance skills of athletes using both books and hypermedia. They concluded that both can be combined into an effective learning strategy with the low-cost and portable book as a study aid and the hypermedia system as a supportive learning environment. Both books and hypermedia can be used to present structured material in a non-sequential format, and may at times constitute an ideal combination. Haring & Fry (1979) studied effects of pictures on childrens' comprehension of written text and found evidence that suggest both structure and reference links can affect learning and attitude toward program.

Dobson and McCracken (1997) compared four instructional conditions (face to face expository, books, books directed by an expert teacher and WWW) in terms of time spent at the source and useful outcomes. They found that in the contact with expert teachers, 55% of the time spent resulted in 70% of the results while students spent 30% of their time on the Web which accounted for only 5% of the results. This leads to the hypothesis that Web based learning, compared with the expert, is significantly less effective and less efficient.

Some of the research is focussed on errors made by end-users. These include lack of general search strategies, failure to pre-plan search procedures, selecting inappropriate search terms and not applying Boolean logic (Johnson, 1997). In a study with high school students, training in keyword and Boolean search strategies had no effect on search success. Johnson (1997) suggested that sophisticated search strategies now used on the Web incorporate many advanced searching features which compensated for lack of Boolean search strategies. At the college level, Puttapithakporn (1990) found that students who did use Boolean operators experienced errors when they combined too many search concepts and used parentheses improperly within those operators. In general, however, most end-users perform searches using only one or two keywords and made little use of Boolean (Ensor, 1992). College students did little advance planning of search strategies (48% in Faries, 1992) and few took advantage of the end-user’s workshops for training (Chen, 1992). Barbuto & Cervallos (1991) found that few repeat searchers used a thesaurus and did not understand the difference between descriptor and keyword searching. In general the effective use of Boolean logic is not overwhelming.

End users who are trained are generally more satisfied with their search results than those not trained (Jackson-Brown & Pershing, 1993). Perceived success rates are low with 40% seeming to crop up in several studies (Johnson, 1997). There seems to be a large discrepancy between end-useres’ perception of success and documented (actual) success (Mischo & Lee,1987; Ankeny, 1991).

Computer Mediated Communication

CMC refers to the ability of computers and computer networks to facilitate communication among students, faculty, scholars and of course multimedia resources in machine readable format. While many new to the field believe that CMC is a relatively new phenomenon associated with the Internet, the truth is that computers were used for CMC for decades. One of the most extensive uses of CMC for educational purposes was the complex set of features available in the mainframe version of the PLATO system (Szabo, 1994).

CMC is alternatively seen as a useful adjunct or threat to conventional F2F instruction. It has led to research and investigations regarding the extent to which CMC can be successfully and profitably integrated into learning environments. One such line of research examines the analysis of dialogues which take place in CMC, using such tools as e-mail and web conferencing. The assumption is that discussion by students enhances learning in behaviorist and cognitive science models of how people learn.

Scardamalia, et al. (1999) have developed systems for learning which capitalize on a wide range of CMC. Most complex learning environments, such as computer simulations, are completed by individuals at learning stations, followed by a group debriefing, often with a F2F instructor. An area of particular relevance is the use of technology to enable dialogue within inquiry learning, such as when students engage in conversation to express viewpoints, organize and present data and information to support/refute conclusion, attempt to persuade others, and try to understand others; and to be able to do so asynchronously. Word-processing and multimedia composition tools, coupled with projection and network transmission tools, allow for representation of ideas. Other tools, such as the shared structured database of Computer Supported Intentional Learning Environments (CSILE) allow students to add to and annotate in a variety of ways, using text and pictures. Inquiry communication also involves visual displays, artifacts, and video messaging and supports dialogue for co-located students, as exemplified in a learning environment called Boardwalk.

Bonk, et al. (1998) compared interactions when groups communicated either synchronously or asynchronously and found the latter engaged in more serious and lengthy communication than the former. More specifically, the asynchronous group had more instance of counter assertions and alternative arguments, general feedback, acknowledgements and suggestions, and praise or positive approval of another or of the task. The results may be confounded with selection as Ss were not randomly assigned to treatments.

Howell-Richardson & Mellar (1996) developed and tested an exploratory system of an eclectic mixture of learning strategies used as a framework to analyze computer comferencing transcripts.

In an online moderated forum (with no subject matter expert) in continuing professional education, the vast majority of messages fell into the early stage of knowledge construction (sharing & comparing, the lowest of 5 levels of categorization) as revealed by transcript analysis. Participants agreed the greatest value was to share and network, not create new knowledge. It was suggested that perhaps the text-only format limited communication by ruling out body language and visuals; and that transcript analysis may not be able to capture knowledge construction (Kanuka & Anderson, 1998).

In an experiment studying critical thinking in a teacher-guided (F2F) and computer-supported (CS) group learning environment, significant differences were found between CS and F2F. CS brought in more outside material, linked ideas to solutions and contributed less. However the level of CS critical thinking was higher, and had a higher ratio of important statements. F2F on the other hand was slightly better at generating new ideas. It raises many questions, including does CS encourage convergent, in depth thinking while F2F facilitates more and divergent interaction? (Newman, et al., 1996).

Rourke, et al. (2000) concluded not all postings are significant to the creation and maintenance of social or cognitive development but they do provide a window to the cognitive and emotional state of the participant. Research on analysis of transcripts is problematic. They also contend that CMC supports a high level of responsive, intelligent interaction between and among faculty and students while overcoming time and geographical barriers. They use a term called social presence, which is the ability of learners and instructors to project themselves into a community of learners using CMC.

Studies of patterns of message and message exchange suggest that communication patterns are more democratic and group oriented than is found in classrooms (Levin, et al., 1990). On the other hand, Siegel et al. (1986) found that CMC groups, compared with F2F groups, took longer in the decision-making process and interacted less. In addition, CMC group members tended to behave more as equals and avoided the social inequality and unequal participation of F2F groups.

Work of Cheng, et al. (1991) does not hold promise for those who hope CMC improves learning efficiency or effectiveness. They found CMC groups scored lower on achievement and attitude than F2F groups, although time on task was the same. Completion rates tended to be higher for F2F groups, except for one particular CMC site. On the other hand, CMC in military training resulted in somewhat higher scores, higher dropout rates, and delivery costs about 48% less than equivalent conventional courses (Phelps, et al., 1991). Costs were calculated after the initial conversion and startup costs are recovered.

Studies by Romiszowski & Jost (1989) qualified two major problems experienced in CMC; a loss of 'sense of structure' on the discussion (by participants and CMC organizers) and loss of control over what would actually being discussed (by CMC organizers). By modifying the software environment, both problems were greatly reduced.

Weiser & Brown (1998) have cautioned about the limitations of CMC when they observe that communicating with the world through the Internet is like viewing it through binoculars: while some things are magnified and clarified, many key elements of the environment which provide rich cues to meaning are shut out. They describe some current and future state of the art digital technologies which are being developed to reduce this bias.

There is an abundance of descriptive studies about the benefits of CMC. For example, Linn et al. (1999) conducted case studies to investigate how effective instructional innovations could be created using the Internet. They describe how science education partnerships were formed between educational researchers, natural scientists and pedagogical experts. Their conclusion was that the scaffolded knowledge integration framework gave a head start on effective designs.

Importance of Navigation & Hypermedia

Predictors of the future point to the day when the majority of work will involve rapid and accurate access to information located in diverse places. On-Line mainframe databases have been superceded by thousands of server-based databases as evolving information resources. All the WWW search engines deliver a high proportion of irrelevant information when a search moved slightly beyond simple (Venditto, 1996). Or as Harris (1996, p. 36) stated that our students are becoming "Information Age hunters and gathers in cyberspace, sharing news of the richest locations by exchanging addresses and URLs with members of your virtual clans. Yet it is here, at the point of information access, that many current knowledge creation efforts falter."

At the present time, it appears the use of hypermedia in education and learning on achievement and attitude has mixed results, and further studies need to be conducted to determine what causes success in one case and failure in another. The definite possibility exists that learning efficiency (time to master a given set of objectives) may be severely reduced by such things as the sheer amount of poorly catalogued and sometimes inaccurate information, coupled with disorientation resulting from disparate interfaces. Implementing CMC into instruction tends to be problematic and research has yet to show any impact on achievement.

The state of research on learning from the WWW is in its infancy. The hype surrounding the WWW and the ease of posting home pages to a server have enticed many users who have accepted things without a critical eye or attitude. As Windschitl (1998, p. 32) observed, "...if we are not being critical and creative as researchers in pursuit of knowledge about Web-based learning, we are abdicating our responsibilities.

Return to Table of Contents

9.0 INSTRUCTIONAL TELEVISION

Hundreds of studies, reviews of studies and reviews of reviews have been written since 1953 about educational television (ITV). A most comprehensive review of these can be found in Wetzel, Radtke, & Stern (1994).

Areas of investigation include effects of ITV on student achievement, attitudes and different techniques used within ITV. These studies usually fall into one of the three genres of video: live action, animation, and talking heads (Collins, Neville, & Bielaczyc, 1999).

Social and educational impacts were of great concern, hence the research. The educational establishment demanded proof of the effectiveness or at least the lack of harm attributable to ITV. It should not be worse than conventional forms of instruction.

ITV was originally viewed as a replacement for the instructor, hence the predominant mode or usage was in the form of a televised lecture, the so-called ‘talking head’. Lest we forget, there were significant numbers of studies done on film, prior to the advent of television. Here is a historical reminder.

Student Achievement

Chu and Schramm (1967, 1975) conducted a classic systematic study which examined 393 and 207 studies that included 393 and 421 separate comparisons, respectively. Campeau (1967) reviewed 58 studies of film and ITV. The main conclusion seems to be that ITV has

has no overwhelming effect, either positive or negative on learning in general. When the data were analyzed by grade level (elementary, secondary, college and adult), similar percentages were found.

One must question the scientific credibility of these studies before drawing solid conclusions from the above. Stickell (Wetzel, Radtke, & Stern, 1994) examined 250 comparisons of televised and classroom instruction based on whether they met scientific acceptability criteria. Only ten of the studies were acceptable and none found significant differences.

Studies which yield simple counts of how many studies were significant are of limited value. Of more value is a technique which estimates the magnitude of the difference, in terms of standardized measures. This technique is called meta-analysis and was used by Cohen, et al (1981) on 74 studies of a variety of visual teaching strategies. In the 65 studies which examined achievement, Cohen found an average effect size of 0.15, which is equivalent to a percentile gain of 6 points, i.e., from the 50th percentile to the 56th percentile. By comparison, the effect size of 0.15 is small when compared to meta-analytic studies in computer based instruction, which are 2 to 3 times as large, and interactive video studies which are 3 to 4 times as large.

After the Chu and Schramm studies, researchers turned their attention to evaluating the effects of the newer media. Note that the research on ITV may not be generalizable to interactive computers with synthesized video.

At the macroscopic level, a series of four experiments investigated effects of selected filmic coding elements (zooming in and out, alternating static pictures of the whole and close ups of details, and static clips only) on covert mental skills of young children. Salomon (1994) concluded that filmic coding elements influenced singling out details, visualization, changing points of view, rotations in space and identification of embedded figures. Untested in these studies were stability over time and transfer value, and whether using video "symbol systems is necessarily the best educational method to cultivate a skill" (Salomon, 1994, p. 156).

Fletcher (1990). Summarized IVD instruction in military settings. Compared with no instruction, he found IVDI increased achievement from 50 percentile to 92 percentile. Compared with conventional instruction IVDI in military training (24 studies) increased achievement to 65%; in higher education (14 studies) increased achievement to 75%; increased achievement in both knowledge and performance measures, to 64%. The more 'interactivity' built into IVDI, the more effective the resulting instruction. IVDI was more effective than computer-assisted instruction without videodisc interaction. IVDI may increase time on task and did not effect longer term retention.

The effects of IVI on attitude were studied by Nicholas & Toporski (1993) and Schaffer and Hannafin (1996). The former found IVI improved attitude toward learning and toward instructional content, however this may be attributable to visuals and (limited) learner control over the program. The latter concluded IVI improved attitude toward learning and toward instructional content, although this may be attributable to visuals and (limited) learner control over the program.

Fletcher (1990) included cost analysis and concluded that IVDI was less costly and more cost effective than conventional instruction]

A series of studies was conducted on language learning and IVI. Supinski (1999) found the addition of cooperative learning to interactive video learning in German language did not improve the achievement of college freshmen. Chang & Smith (1991) and Cuevas (1993) similarly found the addition of cooperative learning strategies to IVI did not improve learner's achievement in Spanish language acquisition.

Gale (1989). Investigated interactivity and found increased learning of Spanish occurred when video was interactive rather than linear.

Verano (1989) reported increased learning of Spanish when video was interactive rather than linear. Of three treatments, linear only, segmented with embedded questions and segmented with feedback and help functions, the latter two were equivalent and better than linear only with respect to achievement.

With the synthesizing of video into computers with their interactive capabilities, the research on ITV may not be generalizable.

Student Attitude

Many studies which examined achievement effects also compared student attitude toward instructional method, content, and desire to learn more. There is also the belief that attitude and achievement are correlated, and to increase one is the increase the other and vice versa. There is no evidence that the correlation is causative, however. Attitudes of students and instructors have been investigated.

A study by Jamison et al. (Wetzel, Radtke, & Stern, 1994) concluded that initial attitude of instructors and students are negative toward ITV and they tend to lessen over time. As institutions gain more experience, the majority of students have neutral or favorable attitudes toward ITV. Keep in mind the context of these studies was to use ITV in the talking head mode, which limited interaction, feedback and many of the instructional strategies often used by live instructors.

Chu & Schramm (1967, 1975) found attitude to be more favorable in elementary grades than in higher level grades or university. Administration was more favorable than instructors, possibly from an administrative and cost-reduction expectancy basis. Students taking the course from home tended to be more positive than students taking the same course in the regular classroom and off-campus students valued the experience for the convenience of not having to travel to school. Chu and Schramm observed that liking ITV was not always correlated with learning from it, a point noted earlier.

The National Technological University, based in Ft. Collins, Colorado, is in the business of offering masters degrees in engineering to students throughout North America and Asia, chiefly by re-broadcasting ITV lessons delivered at a host of member schools of engineering. NTU claims to be one of the largest degree granting institutions in the world in this area.

Dubin & Hedley (Wetzel, Radtke, & Stern, 1994) reviewed college level attitude and found that exposure increased attitude in a positive direction. Students were more likely to voluntarily choose ITV when the alternative was a large lecture class. One-half of the instructors were favorable toward ITV courses, one quarter were neutral, and about 20% were unfavorable. All instructors tended to be least favorable toward their children attending university ITV courses.

Instructional Methods

Many of the techniques used in ITV were not unique to ITV and Chu and Schramm (1967) observed that learning seems to be more affected by what is delivered than by the delivery system. This position was amplified in the now-famous critique of media research by Clark (1983). When used effectively, ITV was integrated into an instructional system which seemed to employ instructors to do what they do best and ITV to do what it does best.

Some of the techniques and issues that have been considered include:

repetition of viewings, using relevant introductions by instructors, encouraging student participation, overt and covert student responding, knowledge of results, inserted questions, rest pauses, use of ITV to introduce or summarize an instructor-led lesson, pacing of the presentation, distraction of note-taking, viewing angles, competing noise, size of viewing group, color vs. black and white, screen size and motion.

Return to Table of Contents

10.0 INTERACTIVITY

The argument that true knowledge can come only through questioning and dialogue is as old as Plato and Socratic dialogue. This has led to the belief in North America that face-to-face instruction is better than distance learning and is to be preferred. This idea has profound influences on acceptance of media, particularly that which deals with distance delivery. This section will briefly look at new instructional designs, some influence of artificial intelligence, adaptive courseware, motivation and individual differences, computer conferencing and simulations.

Definition

Many definitions of interactivity have been posed. An expanded version of the definition by Daniel and Marquis (1990) for distance education (use of instructional technology implied), interaction takes place when the student is in two way contact with another person or persons or intelligent agent in such a way as to elicit reactions and responses which are specific to the student’s own requests or contributions.

Lundin (1989) identified 6 levels of interaction when telecommunications systems are used for distance delivery.

Level 1: ‘reaction’ as a form of interaction with prepared audio (radio) and video (television) broadcast. This is a voluntary, usually passive and, therefore an ineffective and often unproductive kind of interaction.

Level 2: ‘parallel participation in which the program shows activities and asks listeners or viewers to carry out the same activities. for example, ‘Play School" and yoga lessons on television.

Level 3: ‘limited interaction’ in which the participant has choices regarding the exploration of a fixed data base. For example, viewdata (Viatel, Telidon) is claimed to be interactive in this way, as are most data bases and programmed learning.

Level 4: ‘responses’ requested as a form of interaction built into the program software. For example, a 30 minute audio or videotape can be produced in such a way as to keep a student involved for up to a week or two to study by requesting certain activities and investigations to be carried out, then returning to the tape, and so on.

Level 5: ‘simulated’ interaction in which the program acts as a catalyst for local, real, live interaction among participants.

Level 6: ‘live’ transactional interaction at a distance (in real time)- i.e., interaction by which participants can, by comments and questions, contribute to the creation of the unique content or data base which becomes the product of the program or event. This interaction can be both synchronous (e.g., audio and video teleconferencing) or asynchronous (e.g., computer conferencing).

Design for Interaction

In recent years there has been a movement away from designing instructional materials for specific populations to achieve precise, predetermined objectives toward creation of knowledge bases which can be used under computer control for a multiplicity of purposes (Romiszowski, 1996). The paradox is that while this demands greater levels of interaction between student and system, creation of hypermedia systems focus on facilitating access to information, not causing learning to occur. Speed of access and ability to conduct dialogues over the internet are currently strong inhibitors to dialogue in a IT environment. Conversational tutorial and dialogue needs to be added, particularly for the adult student.

Most courseware designers find it very difficult to conceive and implement meaningful interactions, probably due to years of experience in stand-up or paper-based instruction. This prevents adequate interaction and thwarts the power of the computer. For example, rather than the fixed review and practice approach so often used, one could supplement it with feedback, discrimination learning, and performance-based advancement or termination Alessi & Trollip, 1991).

Those who study artificial intelligence applications to learning understand the centrality of dialogue and the need to dissect and analyse interaction and emphasize how it can be accomplished. The human tutor is still more effective when it comes to learning (Bloom, 1984). One view of interactivity is creating a model of each learner’s knowledge as it develops in the mind of the student and probing the model with extended questions or observations to encourage evaluating alternatives and formulating responses (Elsom-Cook. 1996). Martin and Szabo (1990) described the results of incorporating dialogue into chemistry lessons (in an environment which did not employ artificial intelligence techniques). Cox (1996) discussed programs which build models that can be interrogated by students while the building proceeds. Interrogation programs allow the student to pose complex questions, in natural language format, to the computer with the expectation of valuable answers from the computer. Laurillard (1978) described a system which interrogates a student, matches him or her to a selection profile and extracts a unique set of learning resources for that student from a database (Nieveen & van den Akker, 1996). This approach is similar to a non-AI system called computer managed instruction discussed earlier.

Some (e.g., Salomon, 1984) have studied media and motivation to learn, arguing that greater motivation results in more attention to the task. Students tend to like media which are easier to learn with, although they most often result in poorer learning. This might be related to the perception that video for example is for entertainment and when video is brought into the classroom, it is for entertainment and concentration is not necessary. The beliefs about the attributes of different media exhibit large individual and cultural differences and these may change over a short period of time (Clark & Sugrue, 1986).

Computer conferencing and e-mail certainly involve interactions. It can result in growing personal knowledge of other conferees to the level found only in fact-to-face situations (Walther, 1993). And computer conferencing means that interactions between students and instructors is no longer dependent upon either time or place. The biggest inhibitor seems to be across time zones.

Laurillard (1978) examined the interactions which occur when students use computer simulations for learning. He observed checking knowledge and understanding, experimenting, visualizing, reasoning and interpreting types of interactions among students.

Worthington and Szabo (1995) provided modest evidence that CBI which incorporates interactivity (the ability to vary musical components during ear training) resulted in better learning. IT by its nature is primarily an information transmission phenomenon, with the exception of the computer which can, when properly prepared, provide for a two way interaction with the student. There is little research which isolates interactivity in IT instruction but some clues may be gained from examination of the combination of video and computer based instruction, commonly called interative video.

Schwier & Misanchuk (1988) studied the relationship between interaction, perceived need for training, and the learning outomes of effectiveness and efficiency. They found that learning was more effective and efficient for learners with high perceived need of training whose instruction required structured overt responses. Learners with low perceived need of training learned equally well whether overt or covert responses were required. Of course the inferred distinction is that covert or overt translated into interactivity as a cognitive and meaningful (as opposed to rote) activity as practiced by the learner.

Fletcher (1990) reviewed a series of studies which yielded 47 comparisons of learning from interactive video (computer-based) or non- or limited-interaction (non-computer-based) approaches to instruction, such as lecture, text, videotape, or on the job training. These studies showed an average increase in achievement for interactive video studies of about .50 standard deviations, which is somewhat higher than achievement increases from CBI alone. McNeil & Nelson (1991) conducted a meta-analysis and found a similary (.53) effect for interactive video. The studies Fletcher examined learning time revealed a 30% savings in time in the interactive video training. Once again this is similar to the findings of the CBI efficiency studies.

Wetzel, Radtke, and Stern (1994) observed from this study that higher levels of interactivity were associated with higher levels of achievement. Studies of retention show no difference between method of instruction.

McNeil & Nelson (1991) noted that achievement effects were larger when interactive video was used to supplement instruction as opposed to its use in replacing conventional forms of instruction. They, along with Fletcher (1990) found that instruction where the program controlled sequence, review and practice had higher achievement scores than instruction where the learner controlled these elements.

Perhaps the extent to which interactive video incorporates valid instructional practices determines effectiveness. Several studies reviewed by Wetzel, Radtke, and Stern (1994) suggest positive effects from using embedded questions, feedback and branching to reviews. Furthermore, combinations of lesson organization, sequencing, individual diagnosis of progress, and remedial branching seem to have greater effects on achievement than the use of these elements alone.

Two final notes are in order. All the major learning theories incorporate some form of meaningful interaction to be taking place among learners, instructors, and the environment. And the majority of users of the term interactivity rarely provide operational definitions as to what they mean, resulting in some calling pressing the space bar an interaction, while others require deep cognitive processing to occur before it is called interaction.

Return to Table of Contents

11.0 CRITICISMS OF MEDIA RESEARCH

Before we ask the jury to begin deliberation, we must consider what the prominent critics of media research have to say. Computer Assisted Instruction (CAI) has been a popular topic in the educational technology field since it first appeared on the scene in the late 1950’s. It has been one of the most heavily researched technologies ever since computers became affordable to the school system and general public. Much of this research has focussed on the effectiveness of CAI which is demonstrated through improved test scores (Kulik, Kulik, & Cohen, 1980; Williams & Brown, 1990). Effectiveness has also been measured through "heightened affective responses, or better attitudes, reduced learning time, higher course completion rates, an increased retention duration, and finally cost" (Williams & Brown, 1990, p. 214). Generally the effectiveness of CAI has been determined by comparing CAI with traditional classroom instruction (Clark, 1985).

To date, the most comprehensive compilation of studies on the effectiveness of CAI has emanated from the University of Michigan. The Kuliks and their colleagues have conducted several meta-analyses. These studies have revealed that students learned more and liked the instruction more when it was delivered via computer. Their meta-analyses also revealed that instructional time was reduced (Kulik, Bangert, & Williams, 1983; Kulik, Kulik, & Bangert-Drowns, 1984; Kulik, Kulik, & Cohen, 1980). This was the case for elementary, junior high, senior high school, and college students. Upon further analysis, instigated by Clark's (1985) critique, Kulik, Kulik, & Bangert-Drowns (1985), revealed that the effects of CAI were larger in published studies than in unpublished studies (e.g., theses), in studies where a different teacher was used in the experimental group than in the control group, and in studies of shorter duration. CAI was being compared to conventional instruction in these meta-analytic studies.

A more recent meta-analysis of studies concerning the effects of CAI on cognitive outcomes. In this meta-analysis of 31 studies Liao (1992) concluded that ". . . CAI is a mildly effective approach for teaching students cognitive skills in the classroom setting" (p. 377). But it was also noted that there was no evidence to indicate that CAI is more effective or even as effective as other instructional strategies.

Critical Analysis of Media Research Studies

The comparative paradigm studies and meta-analyses have received strong criticism. In particular, educational psychologists, such as Gavriel Salomon, Howard Gardner, and Richard Clark, present persuasive arguments against this type of paradigm. The bulk of CAI effectiveness research is plagued by many uncontrolled variables. Materials prepared for a CAI lesson can undergo hundreds of hours of instructional design whereas the teacher preparing for the same lesson may only spend several hours (Clark, 1985). If equal amounts of preparation time were devoted to paper-based, self-paced materials, we probably would not notice any differences from a typical CAI lesson. In addition, these comparative paradigm and meta-analyses experiments looked at individual instruction in a self-paced mode and compared it to group instruction with group pacing (Clark, 1985). Salomon and Gardner (1986) say that the same naive approach taken in television research has been repeated in CAI research. That is, "When everything else is indeed kept constant, save the medium, not much of an effect can be observed" (Salomon & Gardner, 1986, p. 14). There are far more important factors in determining achievement outcome than the object used to deliver the instruction.

Clark (1983), Salomon and Gardner (1986), and Torkelson (1986) contend that media based research should look at the attributes of the media. It would be far more productive now to "...study attributes of media and their influence on the way that information is processed in learning" (Clark, 1983, p. 451). Willis (1991) suggests that research in CAI effectiveness forces each delivery method to be constant in every aspect except the medium. This prevents any discovery of what is different about the computer’s presentation of information and indeed reduces the power of the computer to the level of that which it is being compared against. Williams and Brown (1990) concur and suggest now is the time to investigate the special features of media such as audio and visual coordination. They suggest further research needs to look at ". . . how computer-related technologies can provide the optimum conditions for learning and which components of instruction foster more effective learning for which types of learners" (p. 222). Consider the example where selected elements of different media might be sufficient to facilitate learning of students who lack the skill being modelled, e.g., using zooming techniques to teach visualization to students who have low visual skills. Alternatively consider the effect of learner expectations on the amount of effort they put into learning. If learners expect media in the classroom to entertain them, they might not put in sufficient energy to learn effectively. This information would be valuable to instructional designers who need to make media decisions. There is no question that typical CAI can be as effective as other means of instruction (Liao, 1992). What needs to be investigated are its specific attributes to determine which are more successful in delivering instruction.

Finally, it should be pointed out that while the causal effect of media on achievement, learning time or attitude may be subject to interpretation, there are other areas of media which need to be considered. These include cost, distribution, adequacy of different media to convey different symbol systems, equity of access and so forth. Many of these issues are discussed in A (1995) book on technology and distance learning.

A somewhat different look at the shortcomings of research on interactive multimedia research was provided by Szabo (2000)

Return to Table of Contents

12. HAS THE JURY REACHED A VERDICT?

Before reaching any verdicts, we must look at the quality of the evidence. With the exception of the research on color, most IM research is not detailed, comprehensive or systematic. This limits our ability to draw solid and meaningful conclusions from them. Secondly, the early stages of adoption of any innovation, including IM, are often characterized as play, probing, and developing an understanding of the innovation, as opposed to wise applications. How much faith can we put in the research that derives from these early stages?

Achievement

Does it improve learning? The evidence suggests that the use of IM to carry out specific instructional strategies which are known to promote learning does indeed improve achievement. Blind application of some form of media is not likely to affect achievement.

Does it increase learning efficiency? The answer appears to be a resounding yes, particularly in the area of computer based instruction, where self-pacing is employed. Savings of 10-40% in learning time have been reported in the literature. It is not clear that gains in learning efficiency are due to IM or to self-pacing, which has also shown to increase speed and therefore efficiency of learning. Regardless, we won’t be able to capitalize upon such efficiencies as long as education is funded for providing ‘seat’ time and the classroom, lock-step single location mode of instruction is superseded.

Attitude

Does it increase attitude? Again, there is the suggestion that the wise and correct use of IM does increase attitude toward instruction and content. This is not, however an automatic by product of the use of IM. In fact, the new user of IM can make certain mistakes based on trying to transfer years of conventional instruction into an IM format. For example, clear and accurate directions need to be provided when the instructor is not always present or at hand to compensate for lack of clarity. Poor directions may reduce student attitude toward the instruction, regardless of the media or technology used.

Access

Perhaps we should also consider two additional questions. Does IM increase access to learning? The answer here is an unqualified yes, given the availability of CD-ROM delivery and more recent World Wide Web delivery tools. While the results of such new technologies are far from demonstrated, or even adequately researched, growing pressure from funding agencies and the working adults who need access to life long learning will assure continued and increasing emphasis on access and equity.

Access is the minimization of barriers to instruction which arise due to geographical and/or time dislocations between source and receiver. Access has not been studied directly as a function of specific components of interactive multimedia or hypermedia but are more a function of progress in the availability, cost and speed of telecommunications systems. Furthermore, the criteria are nominal–either more learners have access or not. Informed practice has much more to tell us here than research. We can learn about access from stories.

We have come a long way since the 1960s when access to multimedia instruction was provided by a series of CBI systems which delivered instruction to educators by loading the computer and terminals (which was what they were called in those days) into specially designed mobile vans which used the ‘concrete network’ of highways (Figures 3 and 4). This system delivered university education to almost 11,000 adults between 1972 and 1977 (Dimmick, 1977).

Insert Figures 3 and 4 about here

A major system, which used POTS, delivered instruction and training to millions of students and trainees from its inception in the mid 1960s to its transformation in the early 1990s. The PLATO system expanded access beyond the conventional classroom to homes, institutes, centres, schools and other places where adults learned. These are but two examples of the many research and development systems created to deliver education at a distance.

A recent study examined the effectiveness of distance education in the United States based on 70 studies (Phipps & Merisotis, 1999). The major conclusions were that achievement compares favorably across a variety of age and content levels. User satisfaction is higher than in traditional instruction regardless of the technology used. Shortcomings of the studies used were identified.

Powell, Conway & Rossy (1990) identified characteristics they claim are associated with success in a first distance education course using computer mediated learning. The list includes self rating of high persistence on new projects, married, female, viewing consequences of failure as serious, rate chances of succeeding higher, can function independently of others, don’t find it important to discuss work with peers, high levels of literacy, rate themselves higher on time manageemnt and rate formal and informal learning high in relation to their goals.

Leigh (1999) concluded that access to the Internet (and by extension to Internet-based instruction) is correlated with ethnic, racial and socio-economic status of students. Low SES students are likely to have limited access to computers and those have low power and slow speeds which are capable only of accessing text-based transmissions.

What we currently have is much more pervasive in terms of connectivity, bandwidth, ease of use and cost; the Internet and its partner, the World Wide Web. These tools are increasingly used by many open learning institutions, such as the Open University of the UK with 60,000 students on line (Daniel, 2000), Athabasca University in Alberta, National Technological University and the newly formed United States Open University (Daniel, 2000), to name just a few.

It is clear that technology has advanced our ability to substantially increase access to education at all levels. The speed of the uptake has been labeled as slow as described in general by Race (2000). "In our capitalistic society, which rewards innovation and enterprise, technologies continue to be invented at a prolific rate. But because ours is a democratic capitalism, there usually must be a political or market consensus before we, the people, adopt a fundamentally new way of doing things."

The most reasonable conclusion seems to be that if IM is used to achieve specific goals through the enhancement of instructional strategies which are known to be (independently) effective, we will achieve our goals. Several examples of many where IM has a role to play might be helpful.

•  
• diagnostic and prescriptive assessment to make sure students have the necessary and specific prerequisite skills, knowledge and/or attitude
•  
• increase the quality and quantity of interaction and communication between instructor and students and among students
•  
• permit students to choose their path or rate of speed through the instruction with tracking and guidance provided by IM
•  
• establish and enforce mastery conditions which can be managed by IM
•  
• allow self-pacing to accommodate the 5 to 1 learning rate variance found in typical groups of students
•  
• increase access through modular instruction which can be delivered via DD-ROM or the WWW with monitoring and tutoring by an instructor
•  
• use DVDs to visualize motions and processes which change in space and/or time.

Return to Table of Contents

13.0 RECOMMENDATIONS

Recommendations for the Use of Computer Assisted Instruction

Recommendations for the Use of Computer Managed Instruction

Recommendations for the Use of Static Visual Displays

Levie & Lentz (1982, p. 65) provided several recommendations serve as useful guidelines for the use of Static Visual Displays in IT instruction.

Recommendations for the Use of Audio

Recommendations for the Use of Color/Screen Design

•  
• Limit the palette per screen to what the eye can actually keep track of at one glance, usually about 6 colors.
•  
• Design first for monochrome displays, then add color
•  
• Use colors selectively to manipulate attention, to highlight text or graphics to make them conspicuous.
•  
• Electronically generated colors take on different properties in relation to one another.
•  
• Color coding, if used, should be used consistently.
•  
• Make sure color combinations are compatible by avoiding saturated complementary colors such as blue/orange, red/green, violet/yellow.
•  
• Gray is a versatile color. Use it in inactive screen areas and backgrounds to enhance other colors.
•  
• Use high color contrast for character/background pairs. Incorporate shape as well as color when possible to make the screen usable for those with color-deficient vision.
•  
• Avoid using blue for text, use it as a background color to enhance depth perception.
•  
• Use color coding to link logically related data and differentiate between required and optional data, highlight errors and separate prompts, commands and other elements in the interface.
•  
• Use color changes to indicate status changes.
•  
• Be aware that colors change with the lighting, according to their surroundings, and as a function of individual learner variations in color genes.

Recommendations for the Use of Animations

Recommendations from Multiple-Channel Learning Theory

Recommendations for the Use of Navigation

Recommendations for the Use of Instructional Television

Recommendations for the Use of Interactivity

2.  
3. Some media permit passivity (ITV) and others inherently require interactivity (CAI, CMI).

Return to Table of Contents

MAJOR REVIEW SOURCES

Return to Table of Contents

REFERENCES

Return to Table of Contents

ADDITIONAL READINGS

Return to Table of Contents

GLOSSARY

Anchored Instruction: a technique of situating instruction in a variety of real-life settings [often simulated] to aid reflection, transfer, and higher level problem solving.

Aptitude-Treatment Interaction: differential interaction between learner aptitudes and instructional treatments.

Audiovisual Aids: instructional materials or media that rely on both hearing and vision for their effects, but loosely used to describe virtually all instructional materials and media other than conventional printed materials.

Audiovisual Technologies: ways to produce or deliver materials by using mechanical or electronic machines to present auditory or visual messages.

Authoring: using an authoring language or system to design and develop instruction.

Authoring Language: a computer language which is specifically designed for developing computer-assisted instruction [and which] requires [the user to have] some knowledge of computer programming.

Authoring System: a computer program which is designed for computer-assisted instruction development. Procedures are pre-defined and require little or no programming knowledge on the part of the user.

Behavioral Psychology: the school of psychology which holds that all behavior of an organism can be explained in terms of stimulus-response bonds.

Boolean logic: the use of AND, OR, and NOT operators and punctuation makrs to combine, include or exclude terms in a multiple term search statement.

Browser: a software package which allows users to examine documents placed on the World Wide Web, independent of platform (Mac, Windows, Unix) being used. Navigator™ and Internet Explorer™ are two current popular browsers at the time of this writing.

Certification: official endorsement of professional competence.

Code of Ethics: principles intended to aid members of the field individually and collectively in maintaining a high level of professional conduct.

Cognitive Psychology: a branch of psychology devoted to the study of how individuals acquire, process, and use information.

Competency: knowledge, skills, or attitudes which the student can demonstrate at a pre-determined level.

Computer-Based Technologies: ways to produce or deliver materials using microprocessor-based resources.

Conceptual Models: models that define, explain, and describe relationships among variables . . . a product of a synthesis of the related research and knowledge base. They can take various forms; they can be narrative descriptions, or taxonomies for example, or mathematical formulations, or visualizations.

Conditions of Learning: the external and internal circumstances that affect learning.

Conditions of Learning (external): specific and unique events that facilitate learning (Gagne and Driscoll, 1988, p.83), especially those which pertain to the stimuli that are external to the learner such as the timing, sequence and organization of the presentation.

Conditions of Learning (internal): specific and unique events that facilitate learning (Gagne' and Driscoll, 1988, p.83), especially those which pertain to the states of mind that the learner brings to the learning task; in other words, they are previously learned capabilities of the individual learner.

Confirmation Evaluation: the process of determining whether learners have maintained their level of competence and materials remain effective. It occurs continuously after a period following formative and summative evaluation.

Constructivism: a school of psychology which holds that learning occurs because personal knowledge is constructed by an active and self-regulated learner who solves problems by deriving meaning from experience and the context in which that experience takes place.

Cost-Effectiveness: a technique for jointly considering the costs and outcomes of something in order to make a decision.

Criterion-Referenced Measurement: techniques for determining learner mastery of pre-specified content.

Database: a collection of documents indexed in a variety of possible ways and accessible by an electronic search engine.

Decoded search: identification of main concepts for searching by extracting keywords directly from the query.

Delivery System: the method (a combination of media and support systems) by which distribution of instructional materials is organized and employed to present instructional information to a learner.

Delivery System Management: involves planning, monitoring and controlling "the method by which distribution of materials is organized." It is "a combination of medium and method of usage that is employed to present instructional information to a learner."

Design: the process of specifying conditions for learning; also a domain in the field of Instructional Technology.

Development: the process of translating the design specifications into physical form; also a domain in the field of Instructional Technology. Developmental Research The systematic study of designing, developing and evaluating instructional programs, processes and products that must meet the criteria of internal consistency and effectiveness. See also Evaluation Research.

Diffusion of Innovations: the process of communicating through planned strategies for the purpose of gaining adoption.

Dissemination: deliberately and systematically making others aware of a development by circulating information.

Distance Education: any instructional situation in which the learner is physically distant from the point of origination, characterized by limited access to teacher and other learners.

Distance Learning: see Distance Education.

Dynamic Visual Displays: visual images that are perceived as moving through time or space.

Educational Technology: see Instructional Technology.

Educational Television is the use of the medium of televised images, using any form of television media, to support or supplant classroom instruction regardless of location in time or geography.

Effectiveness: the extent to which the intervention accomplishes the purpose or achieves the ends desired.

Efficiency: economical pursuit of ends through use of resources. Elaboration Provide[s] detailed information that links a new concept with relevant prior knowledge. Elaborations can use either deductive [expository] or inductive [experiential] processes.

Electronic Performance Support System (EPSS): a combination of hardware and software components which provides an 'infobase,' expert system, job aids and tools and other elements to support performance of tasks.

Encoded search: composing a search statement based on a written topical query requiring a user to create an integrated search statement using Boolean logic.

Evaluation: the process of determining the adequacy of instruction and learning; also a domain in the field of Instructional Technology. Evaluation Research that gathers data for decision making in order to prove, improve, expand, or discontinue a project, program or project.

Expert System: a computer program, assembled by a team of content experts and programmers, that teaches a learner how to solve complex tasks by applying the appropriate knowledge from the content area.

Formative Evaluation: gathering information on the adequacy of an instructional products or programs and using this information as a basis for further development.

Formative: experimentation Research which uses a small scale trial and error approach to study a variable in a real life context.

Front End Analysis: accomplishment of the early stages of the design process, such as analysis of needs, goals, and objectives, and organizing the course units.

Functional Job Analysis: a technique for determining the complete array of tasks performed in a job by grouping in terms of data, people and things and then identifying the associated level of difficulty and the amount of instruction needed.

Functions: the Field Tasks and roles performed by professionals in the field.

Implementation: using instructional materials or strategies in a real (not simulated) setting.

Inductive Learning: a teaching[learning] strategy that proceeds as follows: immersion in a real or contrived problematic situation, development of hypotheses, testing of hypotheses, arrival at conclusion (the main point). Also known as the discovery method.

Information Management: involves planning, monitoring and controlling the storage, transfer or processing of information in order to provide resources for learning.

Installation: using an instructional material, strategy or program in a permanent or semi-permanent fashion usually by embedding it in the curriculum.

Institutionalization: the continuing, routine use of an instructional innovation in the structure and culture of an organization.

Instruction: intervening in order to facilitate learning.

Instructional Technology: the theory and practice of design, development, utilization, management and evaluation of processes and resources for learning.

Instructional or Educational Television: the use of the medium of televised images, using any form of television media, to support or supplant classroom instruction regardless of location in time or geography.

Instructional Strategies: specifications for selecting and sequencing events and activities within a lesson.

Instructional System: the total 'package' of materials, tests, student guides, and teacher guides that is needed to reach the goals for any instructional unit, course, or curriculum, along with all supporting activities and processes required to operate the system as it was designed to be operated.

Instructional Systems Design (ISD): an organized procedure for developing instructional materials or programs which includes the steps of analyzing (defining what is to be learned), designing (specifying how the learning should occur), developing (authoring or producing the material), implementing (using the materials or strategies in context), and evaluating (determining the adequacy of instruction).

Integrated Learning System (ILS): a set of interrelated computer-based lessons organized to match the curriculum of a school or training agency.

Integrated Technologies: ways to produce and deliver materials which encompass several forms of media under the control of a computer.

Internet: a series of local hard drives, accessible and connected via telecommunications hardware and software, giving everyone access to everyone else’s desktop computer.

Intranet: a series of local hard drives, accessible and connected via telecommunications hardware and software, giving everyone within a limited group access to everyone else’s desktop computer. For example, many organizations set up within-company internets which are accessible only to a restricted group, such as employees.

ISD See Instructional Systems Design.

Iterative: steps are repeated and revisions made as new information is revealed at a later step.

Keyword/Boolean logic knowledge: knowledge of boolean and keyword database search strategies to construct search statements.

Keyword selection: The process of identifying appropriate search terms that best summarize the content of a document in order to effectively classify that document.

Knowledge Diffusion: the effective transmission of knowledge from those involved in research and development to those who have use for such knowledge.

Learner Characteristics: those facets of the learner's experiential background that impact the effectiveness of a learning process.

Learning: a relatively permanent change in a person's knowledge or behavior for attitudes] due to experience .

Macro-Design: a reference to using the ISD process. Also used to refer to the development of large units of instruction, such as programs and curricula.

Management: involves processes for controlling Instructional Technology practice including planning, organizing, coordinating and supervising.

Mastery Learning: a systematic approach to instruction based on students performing to a pre-specified criterion level on a given unit of instruction before moving to the next unit of instruction. See Criterion-Referenced Measurement.

Materials Evaluation (instructional products): evaluations that assess the merit or worth of content-related physical items, including books, curricular guides, films, tapes, and other tangible instructional products.

Media Utilization: is the systematic use of resources for learning. Message A pattern of signs (words, pictures, gestures) produced for the purpose of modifying the psychomotor, cognitive, or affective behavior of one or more persons.

Micro-Design: the design of instructional strategies. Also used to describe the development of small units of instruction, such as lessons and modules.

Motivation: the magnitude and direction of behavior. .. the choices people make as to what experiences or goals they will approach or avoid, and the degree of effort they will exert in that respect.

Motivation Design: [planning instructional interventions that] are interesting, meaningful and appropriately challenging [by specifying strategies likely to lead to] interest, relevance, expectancy and satisfaction.

Multimedia: a collection of materials in several different media or a single work designed to be presented through the integrated use of more than one medium.

Needs Assessment: a systematic process for determining goals, identifying discrepancies between goals and the status quo, and establishing priorities for action.

Norm-Referenced Measurement: techniques for ranking of learners on some evaluation of instruction.

Objectives-Oriented: instruction Teaching sequences which are designed so that learners will achieve pre-defined goals and learn pre-defined content.

Organizational Development (OD): a complex educational strategy intended to change the beliefs, attitudes, values, and structure of organizations so that they can better adapt to new technologies, markets, and challenges, and the dizzying rate of change itself.

Performance Technology: is the process of selection, analysis, design, development, implementation, and evaluation of programs to most cost-effectively influence human behavior and accomplishment.

Phenomenological Research: investigation based upon an epistemological perspective that advocates studying human behavior only from the subjects' point of reference. Objective knowledge is consequently rejected, and data from individual cases are meaningful in their own right without being grouped with similar observations.

Policies and Regulations. the rules and actions of society (or its surrogates) that control the diffusion and use of instructional technology.

Positivism (Logical): a philosophy asserting the primacy of observation in assessing the truth of statements of fact and holding that metaphysical and subjective arguments not based on observable data are meaningless.

Post-Modernism: a way of thinking which celebrates the multiple, the temporal, and the complex over the modern search for the universal, the stable, and the simple. Other synonyms for postmodern include breakup, irony, and violent juxtaposition.

Practice: theoretical and experiential knowledge to the solution of problems.

Print Technologies: are ways to produce or deliver materials, such as books and static visual materials, primarily through mechanical or photographic printing processes.

Problem Analysis: involves determining the nature and parameters of the problem by using information-gathering and decision-making strategies.

Procedural Models: models which describe how to perform a task. [they are] prescriptive, and can serve as guides to the solution of specific problems.

Process: a series of operations or activities directed toward a particular result.

Processing: consists of changing some aspect of information to make it more suitable for some purpose.

Program Evaluation: assessing educational activities which provide services on a continuing basis and often involve curricular offerings.

Programmed Instruction: a method of presenting instructional material printed in small bits or frames, each of which includes an item of information (prompt), an incomplete sentence to be completed or a question to be answered (response), and the correct answer (reinforcement).

Program: a set of instructions describing actions for a computer to perform in order to accomplish some task; while conforming to the rules and conventions of a particular programming language.

Programming Planning: a program for a computer or telecommunications medium.

Project Evaluation: evaluation that assesses activities that are funded for a defined period of time to perform a specific task.

Project Management: involves planning, monitoring and controlling instructional design and development projects.

Provider: someone who is attempting to convince others to use an innovation.

Qualitative Research: an approach to scientific inquiry which typically uses non-experimental methods, such as ethnography or case history, to study important variables that are not easily manipulated or controlled and which emphasizes using multiple methods for collecting, recording and analyzing data rather than statistical analysis.

Quantitative Research: an approach to scientific inquiry that typically manipulates independent variables in controlled conditions using experimental designs that incorporate statistical methods of data analysis.

Regulations: see Policies and Regulations.

Research: scholarly or scientific investigation or inquiry.

Resources: sources of support for learning, including support systems and instructional materials and environments.

Resource Management: involves planning, monitoring, and controlling resource support systems and services.

Screen Design: planning images on computer screens, both text and visual, that adhere to principles of message design and aesthetics. See Text Design.

Search engine: software component of a database that determines how the database processes an incoming search statement as to accessing documents in that database.

Search satisfaction: opinion of the searcher about the value of their individual search strategy as measured by their perceived success and their feelings generated by their search experience.

Search success: relative accuracy in searching WWW to find topic related documents.

Situated Learning: an instructional strategy requiring that] students work on authentic tasks whose execution takes place in a "real world" setting.

Specifications: explicit and detailed statements about design requirements.

Statement construction: creation of statements to search databases to locate specific information.

Static Visual Displays: visual images which are perceived as a still picture or representation.

Structured Writing: properties, i. e. blocks, of the text, especially organization and structure, that permit the structure of the subject matter and the document to be perceived by the reader.

Systematic: using processes or step-by-step procedures that allow one to create systems composed of interrelated, interworking elements that together constitute a whole.

Systemic Design: concurrent and creative consideration of the many aspects of a situation which can affect the learning process.

Summative Evaluation: involves gathering information on adequacy and

using this information to make decisions about utilization.

Task Analysis: a process used to determine how a task is performed and to identify the attributes that affect performance.

Technology: systematized practical knowledge which improves productivity.

Theory: concepts, constructs, principles and propositions that contribute to the body of knowledge.

Text Design: applying principles for sequencing, structuring, designing, and laying-out the printed page, whether that text is reproduced on paper or on a computer screen, in order to more effectively present written discourse.

Uniform Resource Locator (URL): the address of any document found on the Internet, including protocol, server identity, path to the file and name of file to be accessed.

User: someone who is a potential adopter of the innovation.

Utilization: is the act of using processes and resources for learning.

Visual Communication: using visual symbols to express ideas or convey meaning.

Visual Language:non-verbal languages such as sign language, body language, pictographic languages or the elements of visual communication such as shots and composition.

Visual Learning:learning from visuals or research on designing visuals for instruction.

Visual Literacy:the ability to understand and use images, including the ability to think, learn and express oneself in terms of images.

Visual Thinking: organizing mental images around shapes, lines, colors, textures and compositions.

World Wide Web: the multimedia-enabled version of the internet (see Internet).

Return to Table of Contents