ENHANCING DISTANCE LEARNING THROUGH RESEARCH ON MULTIMEDIA AND HYPERMEDIA: A REVIEW OF EFFECTIVENESS, EFFICIENCY, ACCESS AND ATTITUDE

Michael Szabo, Ph.D.

Professor of Educational Psychology and Technology

University of Alberta

Edmonton, AB Canada

Tel 780-492-3667 x225

Fax 780-492-3179

Mike.szabo@ualberta.ca

www.quasar.ualberta.ca/DRMIKE

SUB THEME: Enhancing Teaching and Learning Through Research

Abstract................................................................................................................ 2

Introduction.......................................................................................................... 2

Definitions....................................................................................................... 2

Research Criteria.............................................................................................. 3

Research and Educators of Adults.................................................................... 3

Research Results.................................................................................................. 4

Access.............................................................................................................. 4

Computer Based Instruction (CBI).................................................................. 5

Static Visual Displays (SVD)........................................................................... 6

Dynamic Visual Displays (DVD).................................................................... 7

Audio............................................................................................................... 9

Color.............................................................................................................. 10

Screen Design................................................................................................ 10

Hypermedia.................................................................................................... 11

Conclusions....................................................................................................... 14

References.......................................................................................................... 15

Tables & Figures............................................................................................... 25

Biodata............................................................................................................... 26

Abstract

Extensive research has been conducted on multimedia and the body of research on hypermedia is growing. This research is of value to educators of adults to assist in decision-making in the area of instructional technology applications. This paper examines research on selected components of instructional multimedia (IM) and hypermedia (IH): computer based instruction, static and dynamic visual displays, audio, color, screen design and hypermedia. The four criteria examined include achievement, learning efficiency, attitude and access to instruction. The general finding seems to be that instructional multimedia and hypermedia result in (1) equal or moderately greater achievement, (2) significantly reduced learning time, (2) significantly increased access to instruction, and (4) equal or modestly more positive attitude toward instruction, when compared with conventional instruction. There is significant variation in quantity and quality of research across these topics, ranging from extensive and definitive in the area of static visual displays to modest quantities in instructional hypermedia. The latter is likely related to the recent appearance on the scene of hypermedia. Finally, some cautionary observations are made about the research currently available.

Introduction

What can we learn from the research on IM and IH? The tradition of hard and social sciences argue for the use of research results to enable us to make informed and valid decision. This paper is an extension of that tradition into the realm of instructional technology. Of course, we must be judicious in our selection and interpretation, since data can be used as a drunk person uses a light post—more for support than for illumination. Basic research tells us about how the world works while applied research tells us how to make things work. Research results in these two areas can often be counterintuitive, as for instance in the case of colour and animation as instructional elements, where what most instructors believe is not supported by the research. This paper is based upon a series of on-going reviews of research by the author (Szabo, 1994; Szabo & Poohkay, 1994; Szabo, 1998)

Definitions

Instructional multimedia (IM) is defined as the presentation of information for learning through the use of one or more media, such as fonts, static visual displays, dynamic visual displays, audio, computer based instruction and manipulables). Hypermedia is formed by the two different components of hypertext and multimedia, resulting in an imprecise definition (Burton, Moore & Holmes, 1995). Gayeski (1993, p. 5) defined hypermedia as “a class of software programs which consist of networks of related text, graphics, audio files, and/or video clips through which users navigate using icons or search strategies.” Schweir & Misanchuk (1993, p. 324) defined interactive multimedia as “an instructional program which includes a variety of integrated sources in the instruction. The program is intentionally designed in segments, and viewer responses to structured opportunities influence the sequence, size, content, and shape of the program.” Instructional hypermedia (IH) refers to a set of logical links between two or more pieces of instructional information which are displayed in some multimedia format. While the basic format of both IM and IH are digital, much of the underlying research in the former was conducted in non-digital format.

The term convergence has been used to describe the fact that all conventional forms of multimedia may now be delivered through the computer, thanks to the digitization of multimedia. This welcome phenomenon causes some problems for the researcher who is attempting to isolate and identify the contribution to educational outcomes of various multimedia elements. With this limitation in mind, the following research criteria have been identified.

Research Criteria

There are numerous criteria that may be of interest in an assessment of research on IM and IH. This paper focuses upon four broad criteria; achievement, efficiency, access and attitude. Achievement refers to the amount of learning that can be objectively assessed through standard measures of achievement, whether they be nationally standardized or locally developed. Efficiency is a measure of the amount of time it takes to learn or master a given set of materials; greater efficiency is associated with less time and vice versa. Access to instruction means the ability to participate in formal learning in spite of barriers of geographical or time separation between the student and instructor or other source of information. Attitude reflects a lasting predisposition toward something. In the case of research studies on IM and HM, the most commonly investigated categories are attitude toward the method of instruction and toward the content.

Research and Educators of Adults

These four criteria are of great interest to adult learners. Adult education literature and informed practice tell us that adults want to achieve rather than to simply ‘put in their time’. Given the responsibilities of work, family, home and community which most adult students manage, efficient learning rates high on their list. This point was brought home through exaggeration in a recent Scott Adams cartoon in which Dilbert’s boss announces he has just signed up for an executive MBA program which can be completed in one hour! He then declares he needs to leave as he is already one half hour late.

A major barrier to adult education has been and continues to be access. For many, instruction is not available due to geographical and/or time dislocations between the source of information and the student. Access is increasingly the focus of attention due to the training and upgrading requirements of adults who find themselves in a global competition, increasingly service-oriented careers (the post-industrial society, Bell, 1999), and an average of 5 career changes in a lifetime. Promoting attitude and positive feelings about instruction appears to be increasingly important among those who view the student as client who can and will vote with his or her feet (Daniel, 2000).

Research which seeks causal relationships typically employs independent variables which have been objectively operationalized. In the case of IM, studies which incorporate all elements of multimedia mentioned above are nonexistent. The approach taken in this paper is to examine the research results into the individual components of computer based instruction, static visual displays (SVDs), dynamic visual displays (DVDs), audio, color and screen design. Hypertext in research is relatively new and has few useful operationally defined components. This review generally focuses upon those aspects which do not involve computer-mediated communications directed by humans toward humans.

Research Results

Access

The first topic discussed is access, the minimization of barriers to instruction which arise due to geographical and/or time dislocations between source and receiver. Access has not been studied directly as a function of specific components of IM or IH but are more a function of progress in the availability, cost and speed of telecommunications systems. Furthermore, the criteria are nominal—either more learners have access or not. Informed practice has much more to tell us here than research. We can learn about access from stories.

An early example was provided by a series of CBI systems which delivered instruction to educators by loading the computer and terminals (which was what they were called in those days) into specially designed mobile vans which used the ‘concrete network’ of highways (Figures 1 and 2). This system delivered university education to almost 11,000 adults between 1972 and 1977 (Dimmick, 1977).

Insert Figures 1 and 2 about here

A major system which used POTS delivered instruction and training to millions of students and trainees from its inception in the mid 1960 to its demise in the early 1990s. The PLATO system expanded access beyond the conventional classroom to homes, institutes, centres, schools and other places where adults learned. These are but two examples of the many research and development systems created to deliver education at a distance.

A recent study examined the effectiveness of distance education in the United States based on 70 studies (Phipps & Merisotis, 1999). The major conclusions were that achievement compares favorably across a variety of age and content levels. User satisfaction is higher than in traditional instruction regardless of the technology used. Shortcomings of the studies used were identified.

Powell, Conway & Rossy (1990) identified characteristics they claim are associated with success in a first distance education course using computer mediated learning. The list includes self rating of high persistence on new projects, married, female, viewing consequences of failure as serious, rate chances of succeeding higher, can function independently of others, don’t find it important to discuss work with peers, high levels of literacy, rate themselves higher on time management and rate formal and informal learning high in relation to their goals.

Leigh (1999) concluded that access to the Internet (and by extension to Internet-based instruction) is correlated with ethnic, racial and socio-economic status of students. Low SES students are likely to have limited access to computers and those have low power and slow speeds which are capable only of accessing text-based transmissions.

What we currently have is much more pervasive in terms of connectivity, bandwidth, ease of use and cost; the Internet and its partner, the World Wide Web. These tools are increasingly used by many open learning institutions, such as the Open University of the UK with 60,000 students on line (Daniel, 2000), Athabasca University in Alberta, National Technological University and the newly formed United States Open University (Daniel, 2000), to name just a few.

It is clear that technology has advanced our ability to substantially increase access to education at all levels. The speed of the uptake has been labeled as slow as described in general by Race (2000). “In our capitalistic society, which rewards innovation and enterprise, technologies continue to be invented at a prolific rate. But because ours is a democratic capitalism, there usually must be a political or market consensus before we, the people, adopt a fundamentally new way of doing things.”

Computer Based Instruction (CBI)

For purposes of this paper, CBI is defined as the use of the computer to provide all aspects of instruction which a competent tutor could provide, including tutorials, questioning, intelligent answer judging, feedback contingent on answers, simulation, games and diagnostic and prescriptive testing.

The four major components of CBI include computer assisted instruction, computer managed instruction, computer managed communication and instructional systems design. It should be pointed out that not all elements of this definition have been operationalized in the majority of the studies cited.

Research on CBI has typically compared achievement, learning efficiency, and student attitudes in CBI and conventional instructional environments across an astounding range of subjects and age groupings. Studies that yield simple counts of how many studies were significant are of limited value. Of more value is a technique, called meta-analysis, that estimates the magnitude of the difference, in terms of standardized measures. To date, the most comprehensive compilation of studies on the effectiveness of CBI has emanated from the meta-analyses conducted by the Kuliks and their colleagues at the University of Michigan. These studies reveal that students learn more and like the instruction more when it is delivered via computer. When efficiency is the criterion, CBI is significantly better than conventional instruction, in the range of 10–35% savings of instructional time. (Kulik, Bangert, & Williams, 1983; Kulik, Kulik, & Bangert-Drowns, 1984; Kulik, Kulik, & Cohen, 1980). This is the case for elementary, junior high, senior high, and college students.

Several meta-analytic studies (Bangert-Drowns, Kulik, & Kulik, 1985; Kulik, Kulik, & Cohen, 1980; Kulik, Kulik, & Shwalb, 1986; Niemiec, Samson, Weinstein, & Walberg, 1987) conclude that when computer-based instruction is used, achievement is moderately higher than or equal to that obtained from conventional instruction. Further analysis, instigated by Clark’s (1985) critique, revealed that the effects of CBI were larger in published studies, in studies in which a different teacher was used in the experimental group than in the control group, and in studies of shorter duration (Kulik, Kulik, & Bangert-Drowns,1985).

In a more recent meta-analysis of 31 studies concerning the effects of CBI on cognitive outcomes, Liao (1992) concludes that “CBI is a mildly effective approach for teaching students cognitive skills in the classroom setting” (p. 377). But this study also notes that there is no evidence to indicate that CBI is more effective or even as effective as other instructional strategies.

The results of studies comparing CBI to other instructional interventions are so powerful that some have called for a halt to them. Instead, they urge investigators to examine the components or attributes of CBI to determine optimum learning conditions. For example, in the context of CBI, the roles of interaction (Worthington & Szabo, 1995), animation (Szabo & Poohkay, 1996; Szabo & Schlender, 1996), audio (Szabo & Rehaag, 1995), and testing (Szabo, Poon, and Ally, 1997) have been studied in recent years.

In a study comparing WBI with CI, Radhakrishnan & Bailey (1998) found that students in web based instruction learned significantly more course content than in conventional instruction. While students in WBI spent more time studying the subject, regression analysis showed students in the conventional instruction improved their grades more per hour of study.

Balcezak, Lynch, Jackson, Richter, Jaffe, & Cadman, (1998) reported on a course in risk management and medical-legal issues was taught with additional material on the web. Students who browsed the web material scored significantly higher on the achievement test compared with those who completed the course without the web material.

Studies showing no difference in achievement between CBI and conventional instruction were reported by Day, Raven & Neman (1998), Hajizainuddi (1999), Jones (1998), LaRose, Gregg & Eastin (1998), Summary & Summary (1998) and White (1998).

CBI research is not without its critics. Clark (1985) has criticized the methodology of all media research, including CBI, and argued for the separation of or distinction between method and media. However valid his criticisms may be, one cannot help but be impressed with the results evidenced by most studies in this area. In reviewing Thinker Tools and the Jasper Woodbury Series, Kozma (1994) concluded with an argument for a deeper, more integral relationship between method and media.

Static Visual Displays (SVD)

Static visual displays (SVDs) are defined for the purpose of this discussion as any representations of objects, concepts, or processes that do not rely on the use of text or numbers and are perceived through the eye. Historically, SVDs were referred to as graphics, but this is a much broader term. Research on SVDs was extensive during the 1960s and 1980s, but has diminished in recent years, probably due to the definitiveness of the findings.

Research has revealed that SVDs increase the amount learned by adults (Alesandrini, 1985) and by children (Pressley, 1977). Alesandrini and Rigney (1981) found that the use of SVDs was an effective review strategy, in comparison with verbal strategies. In a study that examined student attitude toward SVD-based learning, Rigney and Lutz (1976) concluded that using SVDs as analogies in CBI resulted in high levels of satisfaction with the learning experience.

The most definitive work on SVDs has been a series of controlled studies by Dwyer. They show that visuals emphasizing critical details that are relevant to the learning outcomes are most effective. Conversely, the addition of visual realism to SVDs does not increase learning. Indeed, unnecessary detail can add to learning time without increasing achievement when learning time is limited, as in a conventional classroom (Dwyer, 1967, 1968, 1970, 1981, 1987).

The simplification of visuals can reduce lesson development costs significantly, although developers who use the artistic approach to instructional design may object to this. Dwyer (1970) showed that simple line drawing SVDs tend to be superior to photographs or more realistic drawings. The deciding factor seems to be the relevance of the visual clues to the learning task. For example, using a photograph of a car engine to show where the carburetor is located might be appropriate, while the use of the same photograph would be insufficient to teach students about the structure and function of the carburetor itself.

Joseph and Dwyer (1982) concluded that the integration of realistic and abstract SVDs may reduce achievement differences between students of different ability levels. Different results are likely when concrete or abstract visuals are used with instruction that is concrete or abstract. Levie and Lentz (1982) conducted a meta-analysis of 46 comparisons of text with and without SVDs. Their findings confirmed those of Dwyer. They also found that the improvements in learning accomplished through the use of SVDs were not transferred when students were tested on nonillustrated text. Levie and Lentz developed guidelines that are presented later in this module.

Willows (1978) was concerned about potential interference between the messages provided by text and by SVDs. However, with CBI one can control the presentation to minimize such conflicts.

Finally, students who are shown the same SVDs during instruction and testing score higher on some portions of achievement than students whose testing is limited to verbal formats (Szabo, DeMelo, & Dwyer, 1981; Szabo, Poon, & Ally, 1997).

Dynamic Visual Displays (DVD)

Animation. Two common types of DVDs are animation and video. We start with animation. Baek and Layne (1988) compared learning using text only, text plus graphics, and text plus animation. The adults in the study scored highest when the animation mode was used. Learning that involved animation required less study time, suggesting that animation results in more efficient learning. In another study with adult learners, Mayton (1991) found that the scores of those whose lessons included animation were higher than those of other learners immediately after study; the effects persisted and were measurable one week later.

Szabo and Poohkay (1996) administered three versions of a math tutorial on how to construct triangles. One version used computer animation, another version used the same visuals in static mode, and the third version used only text. Achievement was significantly greater for the animated version, with text alone ranking lowest. Students’ attitude toward the three modes of instruction followed the same pattern.

Rieber and Boyce (1990) compared animation-based instruction with carefully designed verbal presentations that used highly imaginative examples and illustrations. The results with an adult sample indicated no significant difference in the amount learned, but the animation group required less time to retrieve the information they learned.

Of 27 experimental studies that investigated animation in the context of computer-based instruction, 15 showed full or partial significant effects with animation (Alesandrini, 1982; Alesandrini & Rigney, 1981; Baek & Layne, 1988; Carpenter & Just, 1992; Collins, Adams, & Pew, 1978; Crosby &
Stelovsky, 1995; Kaiser, Proffitt, & Anderson, 1985; Mayton, 1991; Park, 1998; Park, & Gittelman, 1992; Rieber, 1989; Rieber & Boyce, 1990; Reiber, Boyce, & Assad, 1990; Rigney & Lutz, 1976; Szabo & Poohkay, 1996). Twelve showed no significant differences (Caraballo, 1985; Caraballo-Rios, 1985; Doll, 1986; King, 1975; McCloskey & Kohl, 1983; Moore, Nawrocki, & Simutis, 1979; Park, 1998; Peters & Daiker, 1982; Reed, 1985; Reiber, 1990; Reiber & Hannafin, 1988; Szabo & Schlender, 1996). It might be instructive to analyze the two groups of studies to determine their commonalties and differences.

Any widespread belief in the superiority of animated over nonanimated instruction within the context of computer-based instruction is at odds with the research findings. Park (1998) observed "Visual displays, whether animated or static, facilitate learning only when their attributes are congruent with the specific learning requirements of the given task." And "more meaningful learning is fostered when visual displays are presented with adequate narratives explaining their instructional roles in the given content." (p. 38). Now we turn to video.

Video. Chu and Schramm (1967, 1975) conducted a classic systematic study that examined 393 and 207 studies that included 393 and 421 separate comparisons, respectively (Table 1). Campeau (1967) reviewed 58 studies of film and ITV. The main conclusion seems to be that ITV has no overwhelming effect, either positive or negative, on learning in general. When the data were analyzed by grade level (elementary, secondary, college, and adult), similar percentages were found.

Insert Table 1 about here

Stickell (Wetzel, Radtke, & Stern, 1994) examined 250 comparisons of televised and classroom instruction based on whether they met scientific acceptability criteria. Only ten of the studies were acceptable and none found significant differences. One must question the scientific credibility of these studies before drawing solid conclusions from the above.

Meta-analysis was used by Cohen, Ebeling, and Kulik (1981) on 74 studies of a variety of visual teaching strategies. In the 65 studies that examined achievement, Cohen found an average effect size of 0.15, which is equivalent to a gain of 6 points, i.e., from the 50th percentile to the 56th percentile. By comparison, the effect size of 0.15 is small when compared to meta-analytic studies on computer-based instruction, which are 2 to 3 times as large. After the Chu and Schramm studies, researchers turned their attention to evaluating the effects of the newer media. Note that the research on ITV may not be generalizable to interactive computers with synthesized video.

At the macroscopic level, a series of four experiments investigated effects of selected filmic coding elements (zooming in and out, alternating static pictures of the whole and close ups of details, and static clips only) on covert mental skills of young children. Salomon (1994) concluded that filmic coding elements influenced singling out details, visualization, changing points of view, rotations in space and identification of embedded figures. Untested in these studies were stability over time and transfer value, and whether using video "symbol systems is necessarily the best educational method to cultivate a skill" (Salomon, 1994, p. 156).

Fletcher (1990). Summarized IVD instruction in military settings. Compared with no instruction, he found IVDI increased achievement from 50 percentile to 92 percentile. Compared with conventional instruction IVDI in military training (24 studies) increased achievement to 65%; in higher education (14 studies) increased achievement to 75%; increased achievement in both knowledge and performance measures, to 64%. The more 'interactivity' built into IVDI, the more effective the resulting instruction. IVDI was more effective than computer-assisted instruction without videodisc interaction. IVDI may increase time on task and did not effect longer term retention.

The effects of IVI on attitude were studied by Nicholas & Toporski (1993) and Schaffer and Hannafin (1996). The former found IVI improved attitude toward learning and toward instructional content, however this may be attributable to visuals and (limited) learner control over the program. The latter concluded IVI improved attitude toward learning and toward instructional content, although this may be attributable to visuals and (limited) learner control over the program.

Fletcher (1990) included cost analysis and concluded that IVDI was less costly and more cost effective than conventional instruction]

A series of studies was conducted on language learning and IVI. Supinski (1999) found the addition of cooperative learning to interactive video learning in German language did not improve the achievement of college freshmen. Chang & Smith (1991) and Cuevas (1993) similarly found the addition of cooperative learning strategies to IVI did not improve learner's achievement in Spanish language acquisition.

Gale (1989) investigated interactivity and found increased learning of Spanish occurred when video was interactive rather than linear.

Verano (1989) reported increased learning of Spanish when video was interactive rather than linear. Of three treatments, linear only, segmented with embedded questions and segmented with feedback and help functions, the latter two were equivalent and better than linear only with respect to achievement.

Audio

Audio may be used to provide information that is redundant, contradictory, enhanced by, or unrelated to other input. Barton and Dwyer (1987) conclude that students with higher verbal skills do not profit significantly from the use of audio. Barron and Kysilka (1992) found no significantly different effects on immediate recall between users of text and text plus redundant audio. Learners in the latter group, however, required more time to complete the lesson.

The appearance of desktop audio has stimulated several studies in which audio has been incorporated into computer based instruction. Szabo and Rehaag (1995) and Barron and Atkins (1994) tested redundant audio in a tenth-grade mathematics course and a university course on computers, respectively. Both studies reported no significant differences in achievement or student attitude. The Rehaag & Szabo study, however, found that more able students using CBI with audio took more time to learn than the control group, but less time to complete built-in practice questions. Less able students in the group using CBI plus audio were generally more positive about the learning experience.

In terms of media preferences, Wu & Martin (1997) concluded that in a public setting, the choice of audio was quite low. All three forms of media were preferred when something unknown was to be revealed.

Can it be that audio limits the ability of students to learn at their own individual rate and thus defeats self-pacing? We know that the learning rate in a typical classroom varies by as much as 5 to 1 (that is, the fastest learners typically master an objective in 20% of the time taken by the slowest learners). Whether or not an audio instruction program can adapt to individual learning rates is a crucial issue.

Research on human-computer interaction has focused primarily on visual elements and has almost completely neglected the use of audio (Aarntzen, 1993; Buxton, 1987; Buxton, 1989; Edwards, 1989; Gaver, 1989). Jaspers (1991) conducted an extensive review of the literature spanning 30 years and found only 80 items related to audio in film and educational media.

Recently, synthesized speech has been improved substantially and has become far more intelligible. Compression and decompression techniques have drastically reduced the amount of space digitized audio occupies and storage space is becoming less of a concern with the advent of CD-ROMs. With the technical problems almost resolved, research has begun in the areas of both speech and non-speech audio.

Color

Most instructors would agree that using colour in instruction (1) increases learning, (2) makes the instructional environment more appealing and therefore increases attention and motivation to learn, and (3) does not distract from the learning tasks at hand. Unfortunately, they are not completely correct.

Extensive research on using colour to increase the level of reality in an instructional situation shows that learning is not generally enhanced by colour (Dwyer, 1967, 1968, 1970; Wise, 1982). The exception to this finding occurs when the instructional objectives require the learning of colour elements (Dwyer, 1987). Examples include learning colour-coded electrical components, flowers, and optical spectroscopy. Apparently when colour elements are not required to meet the objectives, they provide neutral cues to the learner. The worst case is that the use of colour distracts learners from focusing on the content of instruction (Dwyer, 1970).

Although there is evidence that colour-cueing can increase the speed at which people search through lists, learning does not seem to be affected. Dwyer (1981) discovered situations in which the presence of colour interfered with learning required tasks, perhaps because irrelevant cues were provided or learners’ attention was drawn to the colour rather than to the task to be learned.

Livingston (1991) found that the use of colour interferes with learning, while England (1984) and Reilly and Roach (1986) conclude that the use of too many colours reduces the legibility of a presentation. Yang and Moore (1996) suggest that a maximum of four colours be used in screen displays, although this is not a hard and fast rule because different contexts exert different demands on the material.

Screen Design

Screen design is the marriage of art and technical design skills. While there has been extensive written opinion about screen design, almost no studies have objectively examined the effect of screen design on achievement. Szabo & Kanuka (1998) studied the effects of screen design on achievement, instructional time, and rate of completion. Using four accepted screen design principles, they created two CAI lessons on the topic of writing a term paper. Students who received the poorly designed lesson achieved the same amount on a paper and pencil test, but took significantly more time to complete the lesson and were less likely to complete it. An explanation was offered that when students study, they often go into an “automatic processing mode,” in which they accept things at face value. Well designed lessons encourage automatic processing. When something in the instructional presentation is new, different, or unusual, the automatic processing mode is interrupted. In this case, the interruption may have had the effect of slowing the learning rate or decreasing motivation to continue.

Hypermedia

Due to the recent arrival of hypermedia, few studies of hypermedia on educational outcomes appeared before 1993 (Liao, 1998). Two recent meta-analyses of effects of hypermedia on learning across 35 and 43 research studies, respectively, concluded that CAI and text influenced learning the most, followed by hypermedia, traditional instruction and videotape instruction. (Liao, 1998, 1999). The 1998 study, which covered research published between 1986 and 1997 reported a modest but significant effect size of 0.48 or a percentile of 68 on achievement.

Navigation. Predictors of the future point to the day when the majority of work will require rapid and accurate access to information located in diverse places. On-line mainframe databases have been superseded by thousands of server-based databases as evolving information resources. All the WWW search engines deliver a high proportion of irrelevant information when a search moves slightly beyond simple (Venditto, 1996). Harris (1996) states that our students are becoming “Information Age hunters and gathers in cyberspace, sharing news of the richest locations by exchanging addresses and URLs with members of your virtual clans. Yet it is here, at the point of information access, that many current knowledge creation efforts falter” (p. 36).

Research on learning from the WWW is in its infancy. The hype surrounding the WWW and the ease of posting home pages to a server have enticed many users who accept things without a critical eye or attitude. As Windschitl (1998) observes, “If we are not being critical and creative as researchers in pursuit of knowledge about Web-based learning, we are abdicating our responsibilities” (p. 32).This is affected by a classic problem, in that the learner and the designer of the instructional material often have different perceptions of the underlying structure of the medium. Consider these questions:

· •How can users navigate the system without help?

· •What are the important individual differences among learners navigating through the program?

· •How many and what kind of navigational aids should be available to them?

· •Which forms of tips are most appropriate as navigational aids?

· •How structured should case studies be?

· •How should this structure be presented to the user?

As the WWW is so new, much of our research knowledge about navigation comes from the literature on searching bibliographic databases using various technologies, including on-line, local, and CD-ROM-based sources (Protherore & Wilson, 1994). There are differences that may have a bearing on the outcomes of research. For example, bibliographic databases are organized hierarchically, while Web databases can be searched by document title, URL, content, or some combination of these elements. There are differences in the way they index, search, and display results (Courtois, Baer, & Stark, 1995).

Most studies of navigation examine end-user success at the college or university level while using simple or advanced search strategies for bibliographic databases (Johnson, 1997). Although Boolean logic is a dominant and effective search strategy, most people do not use it successfully in their search strategies (Mischo & Lee, 1987; Neuman, 1995), nor do they select appropriate search terms in their search strategies.

Beasley & Waugh (1995) studied cognitive mapping architectures and hypermedia disorientation and found hierarchical maps produced least perceived disorientation, hotwords the most. The browsing device accounted for about 7% of the disorientation reported by users.

Chen, Horney, & Anderson-Inman (1995) used cluster analysis to analyze students' hypertext processing patterns. Their conclusion is that students have a group of identifiable processing patterns into which most fall. Processing is not usually idiosyncratic to individuals.

Felix, Graf, & Krueger (1991) studied user interfaces for public information systems. They report that searching at a public kiosk results in shorter completion time and fewer errors for most people, but this was reversed for those with higher education and computer experience

Hill & Hannafin (1997) asked teachers to report on their metacognitive knowledge, orientation, self-efficacy, technical knowledge of the system, and prior knowledge of subject and found (1) students used a wide variety of search strategies, (2) those with high metacognitive knowledge reported higher ability to reflect on search processes and refine their actions and were better oriented to the system, (3) those with low system knowledge used more primitive search strategies, (4) those with more extensive subject knowledge used more advanced search strategies, (5) high perceived disorientation limited effectiveness of browsing and searching. It was concluded that moving to www based instruction, teachers need to help learners construct functional mental models of the system and to support learners.

Koved & Schneiderman (1986) compared embedded menus with explicit menus. The former result in greater user satisfaction, fewer screens viewed and more questions answered correctly.

Lai, Y-R & Waugh (1995) looked at effects of three different hypertextual menu designs on various information searching activities and concluded that structure and reference links can affect learning and attitude toward instructional programs.

Marchionini & Schneiderman (1988) compared finding facts vs. browsing knowledge in hypertext systems. They discovered that information searchers prefer using keyword searches or index mechanisms when search questions are clear, but when search questions are vague, searchers resort to browsing or exploring.

Schuerman & Peck (1991) examined pull-down menus, menu design, and usage patterns in computer-assisted instruction. Their main conclusion was that design of a menu system influences learners' usage patterns.

Ventura (1991), studying hypertext usage in a public museum, found touch screen users had slightly shorter sessions and examined less of the program in a museum environment than those using a mouse.

Learner Control of Sequence. Hobbs (1987) examined effects of content sequencing and mode of presentation on recall in learning statistics. He found that presentation mode has no effect on recall, the quantity of recall varies with the amount of time spent learning, and highly structure material promotes better recall. Gray (1987, 1988) studied sequence control and its effect on learning using menu screens. Compared with students who were constrained to a linear sequence, students who could move freely within the program performed better on a comprehension-level test but the same on a retention test, and had less positive attitudes toward the software. They also performed better on one of the two long-term tasks. She also found that student performance was better if the menu system was organized in ways that were meaningful to the students. However, rarely if ever are students asked what organization is meaningful to them, and what is meaningful to the instructor is not always so for the student.

Singanayok & Hooper (1998) examined effects of cooperative learning and learner control on students' achievement, option selections and attitude. 6^th graders studied ecological concepts via CAI under (learner or program) control; individual or cooperative learning; and high or low achievement. Effects were assessed on achievement and attitude. The cooperative group scored higher than individual groupings on immediate and delayed achievement and attitude. Interaction between achievement and control showed low achievers retained more under program control and high achievers retained more under learner control. Cooperative learning with learner control scored higher than learner control working individually with the tutorial and checking their own learning.

Achievement. In spite of claims regarding the potential benefits of using hypermedia in education and controversy about the relationship between media and learning, research results comparing the effects of hypermedia and traditional instruction are in conflict. For example, significant gains of hypermedia over traditional instruction are reported by Bain, Houghton, Sah & Carroll (1992), Barnes (1994), Chen (1993), Delclos & Hartman (1993), Gretes & Green (1994), Lui & Reed (1995), Overbaugh (1995), Smith, Jones, & Waugh (1986), and Toro (1995). Studies finding no significant differences were reported by Azevedo, Shaw & Bret (1995), Barker (1988), D’Alessandro et al. (1993), Hess (1994), Kinzie, Strauss, & Foss (1993), Leonard (1992), McCoy (1994), Rojewski, Gilbert, & Hoy (1994), Sheldon (1995), and Tabar (1991).

Lanza and Roselli (1991) compared learning a programming language using structured and hypertext-based systems. Both groups performed equally well, but the hypertext group exhibited greater variability in achievement, prompting the conclusion that hypertext might be better suited to more able students. Tripp and Roby (1990) compared students learning Japanese words in an unstructured hypertext environment and in an environment with orienting devices, that is, advance organizers and visual metaphors. Students using both advance organizers and visual metaphors scored less well. The authors concluded that the two types of orienting devices used in this study activated conflicting mental models of the Japanese lexicon.

Bughouse is a program on the influence of insects on human culture. Students can access the program through a Browse mode (visual metaphor of a Victorian farmhouse and grounds), an Index mode in which the subjects are arranged alphabetically, and a Guide mode, in which an on-line advisor recognizes patterns in the student’s search and recommends new areas for exploration. Using the criterion of relevant items found, Gay, Trumbull, and Mazur (1991) determined that the Guide and Browse modes were more effective than the Index mode.

Hypermedia vs. Books. Dobson and McCracken (1997) compared four instructional conditions (face-to-face exposition, books, books with study directed by an expert teacher, and WWW) in terms of the time spent at the source and useful outcomes. They found that in the contact with expert teachers, 55% of the time spent resulted in 70% of the results, while students spent 30% of their time on the Web, which accounted for only 5% of the results. One might conclude that Web-based learning is significantly less effective and less efficient than learning directed by an expert teacher.

Some of the research is focused on errors made by end-users. These include lack of general search strategies, failure to preplan search procedures, selection of inappropriate search terms, and failure to apply Boolean logic. In a study of high school students, training in keyword and Boolean search strategies had no effect on search success (Johnson ,1997). The author suggested that the sophisticated search strategies now used on the Web incorporate many advanced search features that compensate for the lack of Boolean search strategies, although users’ perceived success rates remain low, about 40%.

At the college level, Puttapithakporn (1990) found that students made errors when they combined too many search concepts and used parentheses improperly within those operators. In general, however, most end-users performed searches using only one or two keywords and made little use of Boolean logic (Ensor, 1992). College students did little advance planning of search strategies (48% in Faries, 1992). Barbuto and Cevallos (1991) found that few people used a thesaurus for repeat searches, and they did not understand the difference between descriptor and keyword searching. In general, Boolean logic is not used effectively.

Mischo and Lee (1987) and Ankeny (1991) found a large discrepancy between end-users’ perception of success and documented (actual) success. End-users who are trained are generally more satisfied with their search results than those who are not trained (Jackson-Brown & Pershing, 1993).

Attitude. Ahmad (1999) studied effectiveness of web-based learning environments in business education. 192 business undergraduates enrolled in an introductory IT course were taught by either web based instruction or 'traditional' instruction. Subjects in WBI reported less satisfied with the learning environment.

In a study of preferences, Chun & Plass (1995) observed that students [Chose picture and movie links equally over text links and rated picture and movie links equally and better than text links. A study of learning styles revealed no relationships.

Conclusions

It should be noted some research reviewed has a variety of weaknesses. First, most research is nonsystematic, piecemeal and contains many areas of weakness. It’s like the late comedian Fred Allen’s description of television: It’s called a medium because its rarely well done. Second, research on instructional technology is often confounded with other variables and issues (Clark, 1985). More specifically, the use of technology requires other issues, such as attention to instructional quality, to be addressed. Perhaps it is these which account for the positive results noted from several studies. Third, race-horse comparisons between technology- and conventional-based instruction provide us with little new information. To make the comparison fair, one or more approach is usually hobbled, thus removing its major source of impact. More and more there are recommendations to examine the components which may be responsible for effects. Perhaps the greatest caveat is that when making choices and arriving at decisions in the realm of instructional technology (as well as most forms of instruction), the weight of comments by a colleague often are taken in place of any research results.

At the present time, it appears the use of hypermedia in education and learning on achievement and attitude has mixed results, and further studies need to be conducted to determine what causes success in one case and failure in another. The definite possibility exists that learning efficiency (time to master a given set of objectives) may be severely reduced by such things as the sheer amount of poorly catalogued and sometimes inaccurate information, coupled with disorientation resulting from disparate interfaces. Implementing CMC into instruction tends to be problematic and research has yet to show any impact on achievement.

The state of research on learning from the WWW is in its infancy. The hype surrounding the WWW and the ease of posting home pages to a server have enticed many users who have accepted things without a critical eye or attitude. As Windschitl (1998, p. 32) observed, “...if we are not being critical and creative as researchers in pursuit of knowledge about Web-based learning, we are abdicating our responsibilities.

Unfortunately, these generally positive attributes of digital multimedia instruction are generally not valued in conventional institutions which rely heavily on the resident, face-to-face models of instruction. There is little if any reward to the instructor for increased learning or attitude; saving student learning time is not valued because we do not place proper economic value on student time; and increased access comes at the expense of reduced face-to-face instruction, the holy grail of conventional education.

The situation is reversed in the case of adult learning. Increased achievement and more opportunity to provide performance feedback through digital technology are valued by adult learners. They are also appreciative of the opportunity to learn at rates which are comfortable to them, which often translates to less learning time than conventional instruction. Finally, adults appreciate and are willing to pay for the ability to study where and when they choose, freed of most specific timelines imposed by conventional instruction. As the world of digital computing merges with multimedia and can be transported around the world in a moment, the words of a current CISCO Systems TV commercial come to mind—“Someday all training will be in the Internet.”

References

Aarntzen, D. (1993). Audio in courseware: Design knowledge issues. Educational and Training Technology International, 30(4), 354–356.

Ahmad, R. M. (1999). Effectiveness of web-based learning environments in business education: Focusing on basic skills training for information technology. Unpublished doctoral dissertation. Louisiana State University.
Alesandrini, K. L. (1985). The instructional graphics checklist: A look at the design of graphics in courseware. Paper presented at the annual convention of the Association for Educational Communications and Technology. (ERIC Document Reproduction Service No. ED 256 303)

Alesandrini, K. L., & Rigney, J. (1981). Pictorial presentation and review strategies in science learning. Journal of Research in Science Teaching, 18, 465–474.

Ankeny, M. L. (1991). Evaluating end-user services: Success or satisfaction? Journal of Academic Librarianship, 16, 352–356.

Azevedo, R., Shaw, S. G., & Bret, P. M. (1995, April). The effectiveness of computer-based hypermedia teaching modules for radiology students. A paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. (ERIC No. ED 385 187)

Baek, Y., & Layne, B. (1988). Color, graphics and animation in a computer-assisted learning tutorial lesson. Journal of Computer-Based Instruction, 15, 131–135.

Bain, A., Houghton, S., Sah, F. B., & Carroll, A. (1992). An evaluation of the application of interactive video for teaching social probelem solving to early adolescents. Journal of Computer-Based Instruction, 19, 92-99.

Balcezak, T. J., Lynch, P., Jackson, S., Richter, J., Jaffe, C. C., & Cadman, E. C. (1998). A web-based risk management and medical-legal curriculum for graduate medical education. Journal of Biocommunication, 25, 2-5.

Bangert-Drowns, R. L., Kulik, J. A., & Kulik, C. C. (1985). Effectiveness of computer-based education in secondary schools. Journal of Computer-Based Instruction, 12, 59–68.

Barbuto, D. M., & Cevallos, E. E. (1991). End-user searching: Program review and future prospects. Research Quarterly, 31, 214–227.

Barker, S. P. (1988). Comparison of effectiveness of interactive videodisc versus lecture-demonstration instruction. Physical Therapy, 68, 699-703.

Barnes, W. G. W. (1994, April). Constructing knowledge from an ill-structured domain: Testing a multimedia Hamlet. A paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. (ERIC No. ED 372 743)

Barron, A. E. (1991). The effectiveness of digital audio in computer-based training. Unpublished doctoral dissertation, University of Central Florida, Tampa Bay.

Barron, A. E., & Atkins, D. (1994). Audio instruction in multimedia education: Is textual redundancy important? Journal of Educational Multimedia and Hypermedia, 3, 295–306.

Barron, A. E., & Kysilka, M. L. (1992, April). The effectiveness of audio in computer-based education. Paper presented at the World Conference on Multimedia and Hypermedia, Boston.

Barton, E. A., & Dwyer, F. M. (1987). The effect of audio redundancy on learners’ ability to profit from printed-verbal/visualization instruction. International Journal of Instructional Media, 14, 131–135.

Beasley, R., & Waugh, M. L. (1995). Cognitive mapping architectures and hypermedia disorientation: An empirical study. Journal of Educational Multimedia and Hypermedia, 4, pp. 239-256.

Bell, D. (1999). The coming of the post-industrial society. NY, NY: Basic Books.

Buxton, W. (1987). The audio channel. In R. M. Baecker & W. A. S. Buxton (Eds.), Readings in human-computer interaction (pp. 47–58). Los Altos, CA: Morgan Kaufman.

Buxton, W. (1989). Introduction to this special issue on nonspeech audio. Human-Computer Interaction, 4, 1–9.

Campeau, P. L. (1967). Selective review of literature on audiovisual media of instruction. In L. J. Briggs, P. L. Campeau, R. M. Gagné, & M. A. May (Eds.), Instructional media: A procedure for the design of multi-media instruction, a critical review of research, and suggestions for future research (pp. 99–142). Pittsburgh, PA: American Institute for Research.

Caraballo, J. N. (1985). The effect of various display modes in computer-based instruction and language background upon achievement of selected educational objectives. Unpublished doctoral dissertation, Pennsylvania State University, State College.

Caraballo-Rios, A. L. (1985). An experimental study to investigate the effects of computer animation on the understanding and retention of selected levels of learning outcomes. Unpublished doctoral dissertation, Pennsylvania State University, State College.

Carpenter, P. A., & Just, M. A. (1992). Understanding mechanical systems through computer animation and kinematic imagery (Report No. ONR-1). Arlington, VA: Office of Naval Research.

Chang, K. R., & Smith, W. M. (1991). Cooperative learning and Call/IVD in beginning Spanish: An experiment. The Modern Language Journal, 75, 205-211.

Chen, A. (1993). The use of interactive multimedia systems to improve the learning of classroom practices. Education Research and Perspective, 20, 24-32.

Chen, D-T., Horney, M. A., & Anderson-Inman, L. (1995). The standard states technique: A utilization of cluster analysis in analyzing students' hypertext processing pattern. Journal of Educational Multimedia and Hypermedia, 4, 53-73.

Chu, G. C., & Schramm, W. (1967). Learning from television: What the research says. Washington, DC: National Association of Educational Broadcasters; Stanford, CA: Institute for Communication Research.

Chu, G. C., & Schramm, W. (1975). Learning from television: What the research says. (ERIC Document Reproduction Services No. ED 914 900)

Chun, D. M., & Plass, J-L. (1995). Project Cyberbuch: A hypermedia approach to computer-assisted language learning. Journal of Educational Multimedia and Hypermedia, 4, 85-116.

Clark, R. E. (1985). Evidence for confounding in computer-based instruction studies: Analyzing the meta-analyses. Educational Communication and Technology Journal, 33(4), 249–262.

Cohen, P. A., Ebeling, B. J., & Kulik, J. A. (1981). A meta-analysis of outcome studies of visual-based instruction. Educational Communication and Technology Journal, 29, 26–36.

Collins, A., Adams, M., & Pew, R. (1978). Effectiveness of an interactive map display in tutoring geography. Journal of Educational Psychology, 70, 1–7.

Courtois, M. P., Baer. W. M., & Stark, M. (1995). Cool tools for searching the Web: A performance evaluation. Online, 6, 14–32.

Crosby, M. E., & Stelovsky, J. (1995). From multimedia instruction to multimedia evaluation. Journal of Educational Multimedia and Hypermedia, 4, pp. 147-162.

Cuevas, G. (1993). The relationship of computer assisted language learning environments and cognitive style to achievement in English as a second language. Unpublished doctoral dissertation, University of Miami, Miami, FL.

D’Alessandro, M. P., Galvin, J. R., Erkonen, W. E., Albanese, M. A., Michaelsen, V. E., Huntley, J. S., McBurney, R. M., & Easley, G. (1993). The instructional effectiveness of a radiology multimedia textbook (HyperLung) versus a standard lecture. Investigative Radiology, 28, 643-648.

Daniel, J., Sir. (2000, January 5). Implications of Virtual Universities for the University of Alberta. A colloquia presented the University of Alberta, Edmonton, AB. Available at www.open.ac.uk/vcs-speeches

Day, T., Raven, M. R., & Newman, M. E. (1998). The effects of world wide web instruction and traditional instruction and learning styles on achievement and changes in student attitude in a technical writing in agricommunication course. Journal of Agricultural Education, 39, 65-75.

Delclos, V. R., & Hartman, A. (1993). The impact of an interactive multimedia system on the quality of learning in educational psychology: An exploratory study. Journal of research on Computing in Education, 2, 83-93.

Dimmick, F. D. Computer Assisted Instruction at the Pennsylvania State University 1964-1977: A Program History. Unpublished Research Report. University Park, PA: College of Education.

Dobson, M., & McCracken, J. (1997). Problem-based learning as a means to evaluate multimedia courseware in science and technology in society. Proceedings of the ED-MEDIA 97 World Conference on Multimedia and Hypermedia. Charlottesville, VA: American Association for Computers in Education.

Doll, L. S. (1986). The instructional effectiveness of the microcomputer with an elderly population. Unpublished doctoral dissertation, Georgia State University, Valdosta.

Dwyer, F. M. (1967). Adapting visual illustrations for effective learning. Harvard Educational Review, 37, 250–263.

Dwyer, F. M. (1968). Effect of visual stimuli on varied learning objectives. Perceptual and Motor Skills, 27, 1067–1070.

Dwyer, F. M. (1970). Exploratory studies in the effectiveness of visual illustrations. Audio-Visual Communication Review, 18, 235–247.

Dwyer, F. M. (1981). Strategies for improving visual learning. State College, PA: Learning Services.

Dwyer, F. M. (1987). Enhancing visualized instruction: Recommendations for the practitioners. State College, PA: Learning Services.

Edwards, A. D. (1989). Soundtrack: An auditory interface for blind users. Human-Computer Interaction, 4, 45–66.

Ensor, P. (1992). Knowledge levels of users and nonusers of keyword/Boolean searching on an online public access catalog. Research Quarterly, 32, 60–74.

Faries, C. (1992). Users’ reactions to CD-ROM: The Penn State experience. College and Research Libraries, 53, 139–149.

Felix, D., Graf, W., & Krueger, H. (1991). User interfaces for public information systems. In H. J. Bullinger, (Ed.), Human aspects of computing. New York: Elsevier.

Fletcher, J. D. (1990). Effectiveness and cost of interactive videodisc in defense training an education. IDA Paper P-2372. Alexandria, VA: Institute for Defense Analyses.

Gale. L. E. (1989). Macario, Montevideodisc, and interactive D game: Developing interactive video for language instruction. In W. M. Smith (Ed. ) Modern technology in foreign language education: Applications and projects. ACTFL Review of Foreign Language Education, Vol. 19. Lincolnwood, IL: National Textbook Company.

Gaver, W. W. (1989). The SonicFinder: An interface that uses auditory icons. Human-Computer Interaction, 4, 67–94.

Gay, G., Trumbull, D., and Mazur, J. (1991). Designing and testing navigational strategies and guidance tools for a hypermedia program. Educational Computing Research, 7, 189–202.

Gretes, J. A., & Green, M. (1994). The effect of interactive CD-ROM/digitized audio courseware on reading among low-literate adults. Computers in the Schools, 11, 27-42.

Gray, S. H. (1987). The effect of sequence control on computer-assisted learning. Journal of Computer-Based Instruction, 14, 54–56.

Gray, S. H. (1988). Sequence control menus and CAI: A follow-up study. Journal of Computer-Based Instruction, 15, 57–60.

Hajizainuddi, A. M. (1999). A study of learning styles and hypermedia's organizational structures in a web-based instructional program designed for trainee teachers at the International Islamic University Malaysia. Unpublished doctoral dissertation, International Islamic University Malaysia.

Harris, J. (1996). Mining the Internet. Learning and Leading with Technology, pp. 36–39.

Hess, G. J., Jr. (1994, June). Strategies for integrating computer-based training in college music theory courses. A paper presented at the ED-MEDIA 94 World Conference on Educational Multimedia and Hypermedia. Vancouver, British Columbia (ERIC No. ED 388 256).

Hill, J. R., & Hannafin, M. J. (1997). Cognitive strategies and learning from the world wide web. Educational Technology Research and Development, 45, 37-64.

Hobbs, D. J. (1987). Effects of content sequencing and presentation mode of teaching materials on learning outcomes. Programmed Learning and Educational Technology, 23, 292–299.

Jackson-Brown, G., & Pershing, G. (1993). Comparisons of graduate and undergraduate end-users of ERIC and PscychLIT on CD-ROM. (ERIC Document Reproduction Service No. ED 372 738)

Jaspers., F. (1991). The relationship of sound-image. International Journal of Instructional Media, 18, 161–174.

Johnson, B. (1997). Effects of instruction on search success and satisfaction on the World Wide Web. Unpublished master’s thesis, University of Alberta, Edmonton.

Jones, E. R. (1998). A comparison of an all web-based class to a traditional class. ED432268.

Joseph, J. H., & Dwyer, F. M. (1982). The instructional effectiveness of integrating abstract and realistic visualization. Paper presented at the annual meeting of the Association of Educational Communications and Technology, Research and Theory Division, Dallas, TX.

Kaiser, M. K., Proffitt, D. R., & Anderson, K. (1985). Judgments of natural and anomalous trajectories in the presence and absence of motion. Journal of Experimental Psychology: Learning, Memory and Cognition, 11, 795–803.

King, W. A. (1975). A comparison of three combinations of text and graphics for concept learning (Technical Report No. SDC 269-7–19). San Diego, CA: Navy Personnel Research and Development Center.

Kinzie, M. B., Strauss, R., & Foss, J. (1993, January). The effects of an interactive dissection simulation on the performance and achievemetn of high school biology students. Proceedings of Selected Research and Development Presentations at the Convention of the Association for Educational Communications and Technology Research and Technology Division. Washington, DC: Association for Educational Communications and Technology. (ERIC No. ED 362 173)

Koved, L., & Schneiderman, B. (1986). Embedded menus: Selecting items in context. Communications of the ACM, 29, 312-318.

Kozma, R. B. (1994). Will media influence learning? Refraining the debate. Journal of Educational Technology Research and Development, 42, 7-19.

Kulik, C., Kulik, J. A., & Bangert-Drowns, R. (1984). Effects of computer-based instruction on elementary school pupils. (ERIC Document Reproduction Service No. ED 244616)

Kulik, C., Kulik, J. A., & Bangert-Drowns, R. (1985). Confounding in educational computing research. Journal of Educational Computing Research, 1(3), 129–139.

Kulik, C., Kulik, J. A., & Shwalb, B. (1986). The effectiveness of computer-based adult education: A meta-analysis. Journal of Educational Computing Research, 2, 235–252.

Kulik, J. A., Bangert, R., & Williams, G. (1983). Effects of computer-based teaching on secondary school learners. Journal of Educational Psychology, 75(1), 19–26.

Kulik, J. A., Kulik, C. C., & Cohen, P. A. (1980). Effectiveness of computer-based college teaching: A meta-analysis of findings. Review of Educational Research, 50, 525–544.

Lai, Y-R., & Waugh, M. L. (1995). Effects of three different hypertextual menu designs on various information searching activities. Journal of Educational Multimedia and Hypermedia, 4, 25-52.

LaRose, R., Gregg, J., & Eastin, M. (1998). Audiographic telecourses for the web: An experiment. Journal of Computer-Mediated Communication (On-line), 4 Available: http://www.ascusc.org/jcmc/vol14/issue2/larose.html.

Lanza, A., & Roselli, T. (1991). Effects of the hypertextual approach versus the structured approach on learners’ achievement. Journal of Computer-Based Instruction, 18, 48–50.

Leigh, P. R., (1999). Electronic connections and equal opportuniites: An analysis of telecommunications distributions in public schools. Journal of Research on Computing in Education, 32, 108-127.

Leonard, W. H. (1992). A comparison of student performance following instruction by interactive videodisc versus conventional laboratory. Journal of Research in Science Teaching, 29, 93-102.

Levie, W. H., & Lentz, R. (1982). Effects of text illustrations: A review of research. Educational Communication and Technology Journal, 30, 195–232.

Liao, Y. K. (1992). Effects of computer-assisted instruction on cognitive outcomes: A meta-analysis. Journal of Research on Computing in Education, 24(3), 367–380.

Liao, Y. K. (1998). Effects of hypermedia versus traditional instruction on students’ achievement: A meta-analysis. Journal of Research on Computing in Education, 30, 341-361.

Liao, Y. K. (1999). Effects of hypermedia on student’s achievement: A meta-analysis. Journal of Educational Multimedia and Hypermedia, 8, 255-277.

Liu, M., & Reed, W. M. (1995). The effects of hypermedia assisted instruction on second language learning. Journal of Educational Computing Research, 12, 159-175.

Livingston, L. A. (1991). The effect of color on performance in an instructional gaming environment. Journal of Research on Computing in Education, 24, 246–253.

Marchionini, G., & Schneiderman, B. (1988). Finding facts vs. browsing knowledge in hypertext systems. IEEE Computer, 21, 70-70.

Mayton, G. B. (1991). Learning dynamic processes from animated visuals in microcomputer-based instruction. Proceedings of the Annual Meeting of the Association for Educational Communications and Technology. (ED334999, 13 pages)

McCloskey, M., & Kohl, D. (1983). Naive physics: The curvilinear impetus principle and its role in interactions with moving objects. Journal of Experimental Psychology: Learning, Memory and Cognition, 9, 146–156.

McCoy, L. P. (1994). Decisions in inferential statistics with HyperdCard: Design and field test. Computers in the Schools, 10, 69-77.

Mischo, W. H., & Lee, J. (1987). End-user searching of bibliographic databases. Annual Review of Information Science and Technology, 22, 227–263.

Moore, M., Nawrocki, L., & Simutis, Z. (1979). The instructional effectiveness of three levels of graphics displays for computer-assisted instruction (Technical Report No. ARI-TP 359). Arlington, VA: Army Research Institute for the Behavioral and Social Sciences. (ERIC Document Reproduction Service No. ED 178–057).

Neuman, D. (1995). High school learners’ use of databases: Results of a national Delphi study. Journal of the American Society for Information Science, 46, 284–298.

Nicholas, M. A., & Toporski, N. (1993). Developing 'The Critic's Corner' computer assisted language learning for upper-level Russian Students. Foreign Language Annals, 26, 469-478.

Niemiec, R., Samson, G., Weinstein, T., & Walberg, H. J. (1987, Winter). The effects of instructional technology in elementary schools: A quantitative synthesis. Journal of Research on Computing in Education, 20, 85–103.

Overbaught, R. (1995). The efficacy of interactiv video for teaching basic classroom management skills to preservice teachers. Computers in Human Behavior, 11, 511-527.

Park, O-K. (1998). Visual displays and contextual presentations in computer-based instruction. Educational Technology Research and Development, 46, 37-50.

Park, O., & Gittelman, S. (1992). Selective use of animation and feedback in computer-based instruction. Educational Technology Research and Development, 40, 27-38.

Peters, H. J., & Daiker, K. C. (1982). Graphics and animation as instructional tools. Pipeline, 7, 11–13.

Phipps, R., & Merisotis, J. (1999, April). What's the difference: A review of contemporary research on the effectiveness of distance learning in higher education. A report for the Institute for Higher Education Policy, Washington, D.C.

Powell, R., Conway, C., & Rossy, L. (1990). Effects of student predisposing characteristics on student success. Journal of Distance Education, 5, 20-37.

Pressley M. (1977). Imagery and children’s learning: Putting the picture in developmental perspective. Review of Educational Research, 47, 585–662.

Protherore, N., & Wilson, E. (1994). The Internet handbook for school users. (ERIC Document Reproduction Service No. ED 375 821).

Puttapithakporn, S. (1990). Interface design and user problems and errors: A case study of novice searchers. RQ, 30, 195–204.

Race, T. (2000, January). Technology Sprints, but Users Set Their Own Pace. New York Times, January 1, 2000. Available at www.nytimes.com/specials/010100mil-tech-leap.html

Radhakrishnan, S., & Bailey, J. E. (1998). Web-based educational media: Issues and empirical test of learning. ED499543/

Reed, S. K. (1985). Effects of computer graphics on improving estimates to algebra word problems. Journal of Educational Psychology, 77, 285–298.

Rehaag, D., & Szabo, M. (1995, February). An experiment on effects of redundant audio in computer-based instruction on achievement, attitude and learning time in 10th grade math. Paper presented at the annual meeting of the Association for Educational Communications and Technology, Anaheim, CA.

Reilly, S. S., & Roach, J. W. (1986). Designing human/computer interfaces: A comparison of human factors and graphics arts principles. Educational Technology, 16, 36–40.

Rieber, L., & Boyce, M. (1990). The effects of computer animation on adult learning and retrieval tasks. Journal of Computer-Based Instruction, 17, 46–52.

Reiber, L., Boyce, M. J., & Assad, C. (1990). The effects of computer animation on adult learning and retrieval tasks. Journal of Computer-Based Instruction, 17, 46-52.

Rieber, L., & Hannafin, M. J. (1988). The effects of textual and animated orienting activities and practice on learning from computer-based instruction. Computers in the Schools, 5, 77–89.

Rigney, J., & Lutz, K. (1976). Effects of graphic analogies of concepts in chemistry on learning and attitude. Journal of Educational Psychology, 68, 305–311.

Rojewski, J. W., Gilbert, J. P., & Hoy, C. A. (1994). Effects of a hypertext computer program on academic performance in a special education course for nonmajors. Teacher Education and Special Education, 17, 249-259.

Salomon, G. (1994). Interaction of media, cognition, and learning. Hillsdale, NJ: Lawrence Erlbaum Associates.

Schaffer, L., & Hannafin, M. J. (1996). The effects of progressively enriched interaction on learning from interactive video. Educational Communications and Technology Journal, 34, 89-96.

Schuerman, R. L., & Peck, K. L. (1991). Pull-down menus, menu design, and usage patterns in computer-assisted instruction. Journal of Computer-Based Instruction, 18, 93-98.

Sheldon, E. J. (1995). CAI/multimedia approach to farm tractor and machinery safety certification. Unpublished doctoral dissertation, Purdue University, West Lafayette, IN.

Singanayok, C., & Hooper, S. (1998). The effects of cooperative learning and learner control on students' achievement, option selections and attitude. Educational Technology Research and Development, 46, 17-36.

Smith, S. G., Jones, L. L., & Waugh, M. L. (1986). Production and evaluation of interactive videodisc lessons in laboratory instruction. Journal of Computer-Based Instruction, 13, 117-121.

Summary, R., & Summary, L. (1998). The effectiveness of the world wide web as an instructional tool. ED431395. [Available] at; http://www.mtsu.edu/~itconf/proceed98/ressumary.hmtl

Supinski, S. B., (1999). An experimental comparison of strategies using cooperative learning and interactive video: Lessons for the interactive classroom. Journal of Interactive Learning Research, 10, 123-148.

Szabo, M. (1994, November). Research-based guidelines for the components of interactive multimedia. Organizer and presider for a forum presented at the 1994 Canadian Multimedia Conference. Calgary.

Szabo, M. (1998). Survey of educational technology research. ETPD Series. Organizational Development Services. Edmonton, AB: Northern Alberta Institute of Technology.

Szabo, M., DeMelo, H. T., & Dwyer, F. M. (1981). Visual testing: Visual literacy’s second dimension. Educational Communications and Technology Journal, 29, 177–187.

Szabo, M., & Kanuka, H. (1998). Effects of violating screen design principles of balance, unity and focus on recall learning, study time, and completion rates. Journal of Educational Multimedia and Hypermedia, 8, 23-42.

Szabo, M. & Poohkay, B. (June 1994). So what if it's in color and moves? A critique of multimedia. A talk presented at the annual meeting of the World Conference on Educational Multimedia and Hypermedia. Vancouver, BC. 85.

Szabo, M., & Poohkay, B. (1996). An experimental study of animation, mathematics achievement, and attitude toward computer assisted instruction. Journal of Research on Computers in Education, 28, 390–402.

Szabo, M., Poon, R., & Ally, M. (1997). The role of visual testing when learning from multimedia. Paper presented at the meeting of the World Conference on Multimedia and Hypermedia, Calgary, AB.

Szabo, M., & Schlender, R. S. (1996, June). An experimental comparison of effects of dynamic and static visual displays in computer-based instruction on declarative and procedural knowledge of selected object-oriented authoring skills. Paper presented at the World Conference on Multimedia and Hypermedia, Boston.

Tabar, C. R. (1991). Computer-assisted interactive video instruction: An alternative to lecture method for nutrition education in baccalaureate nursing. Unpublished doctoral dissertation, University of Cincinnati, Cincinnati, OH: Dissertation Abstracts International, 51, 2258A.

Toro, M. A. (1995). The effects of HyperCard authoring on computer-related attitudes and Spanish language acquisition. Computers in Human Behavior, 11, 633-647.

Tripp, S. D., & Roby, W. (1990). Orientation and disorientation in a hypertext lexicon. Journal of Computer-Based Instruction, 17, 120–124.

Venditto, G. (1996). Search engine showdown. Internet World, pp. 79–86.

Verano, M. (1989). USAFA interactive study in Spanish. In W. M. Smith (Ed. ) Modern technology in foreign language education: Applications and projects. ACTFL Review of Foreign Language Education, Vol. 19. Lincolnwood, IL: National Textbook Company.

Ventura, A. (1991). Finnegans hyperwake - The clicky way to waking. Hypertext 91 Proceedings. New York: Association for Computing Machinery.

Wetzel, C. D., Radtke, P. H., & Stern, H. W. (1994). Instructional effectiveness of video media. Hillsdale, NJ: Lawrence Erlbaum Associates.

White, S. E. (1997). The effectiveness of web-based instruction: A case study. ED430261.

Willows, D. (1978). A picture is not always worth a thousand words: Pictures as distractors in reading. Journal of Educational Psychology, 70, 255–262.

Windschitl, M. (1998). The WWW and classroom research: What path should we take? Educational Researcher, 27, 28–32.

Wise, R. E. (1982). The differential employment of cognitive skills as a function of increasing iconic stimulus complexity. Paper presented at the annual meeting of the Association of Educational Communications and Technology, Research and Theory Division, Dallas, TX. (ERIC Document Reproduction Service No. ED 223206)

Worthington, T., & Szabo, M. (1995, February). Interactivity in computer-based aural skills instruction: A research study. Paper presented at the annual meeting of the Association for Educational Communications Technology, Anaheim, CA.

Wu, M. H-F., & Martin, C. D. (1997). An exploratory study of user media preferences in a public setting. Journal of Educational Multimedia and Hypermedia, 6, pp. 2-22.

Yang, C-S., & Moore, D. M. (1996). Designing hypermedia systems for instructors. Journal of Educational Technology Systems, 1, 3–30.

Tables & Figures

Significance of Achievement	Chu and Schramm, 1967	Campeau, 1967
No significant difference	308 (73%)	42 (72%)
ITV instruction superior	63 (15%)	10 (17%)
Conventional instruction superior	50 (12%)	6 (10%)

Table 1 Comparison of Educational Television and Conventional Instruction

Figure 1. Exterior view of PSU Mobile Computer Assisted Instruction Van

Figure 2. Interview view of PSU Mobile Computer Assisted Instruction Van

Biodata

Michael Szabo, Ph.D. is Professor of Educational Psychology and Division of Technology in Education at the University of Alberta, Edmonton, Alberta Canada. He developed the graduate program in Instructional Technology and conducts research and development into several aspects of effects of instructional technology on learning and the process of instructional technology as innovation and change. Prior to this appointment, Professor Szabo was Manager of Instructional Systems Development (PLATO) at the University of Alberta from 1981 to 1987.

From 1969 to 1981, Professor Szabo was on the faculty of Education at The Pennsylvania State University and also served in Directorship positions in the Computer Assisted Instruction Lab (IBM 1500), the Instructional Support System and the Center for Educational Research and Evaluation.

Professor Szabo has consulted and been involved in creating interactive multimedia programs for a variety of government and commercial agencies around the world, most recently in Asia. He has published more than 80 research articles, 244 papers at conferences and several book chapters. He is currently developing Web Based Instruction courses and regularly consults on multimedia, computer based instruction and web-based instruction. In 1985, he founded Interactive Solutions, Inc. a training company which consults on creating interactive multimedia solutions for education, government and commercial training.