Conversion of a EDIT 571 'Introduction to Instructional Technology and Communication' from Conventional F2F to Web-Based Instruction
Professor Michael Szabo, Ph.D., Project Director <mike.szabo@ualberta.ca>
and
Annette Fuchs, M.Ed. Assistant Project Director
University of Alberta
Edmonton, AB Canada
15 October 1998
The Department of Educational Psychology of the University of Alberta offers a graduate (EDIT 571) and undergraduate (EDPY 488) course to introduce students to instructional technology and communication. These courses were traditionally offered once a year, limited to 20 on-campus students, delivered in lecture-test format, required the scheduled use of a classroom and computer lab, and did not model the use of instructional technology in the classroom.
LEE funding enabled the conversion of these courses to alternative delivery using Web-Based Instruction whereby students are now able to complete the course without lecture and at their own individual pace, time and location. The WBI courses accommodate for individual differences in learner background knowledge and provide for more active learning vs. the passivity of lectures. They also serves to provide one model of WBI which students can add to their repertoire of 15-16 years of classroom-based instruction. The courses are now offered twice a year to unlimited enrolment and can easily be offered on an open entry basis. They are delivered on Macintosh and Windows platforms and modern internet communications which are accessible from home, work, or the university. To date, the courses are being offered for the third time and the response of students has been quite positive.
The first deliverable consists of two complete three credit courses which are available anywhere in the world at any time of the night or day. The second deliverable is a Web-Based management and communication system consisting of e-mail, learning contract, marks administration, conferencing, and module evaluation form.
The course consists of nineteen modules, of which each student chooses both 1) 10 to complete and 2) the due date for each module, with certain limitations applied. An example of a limitation is the student with no experience with WBI tools of e-mail, ftp, conferencing, etc. is required to complete Module I, Internet Communication at the very beginning of the course. The course runs off an existing Faculty of Education Unix-Server and the management system runs off an NT-Server, also provided by the Faculty.
The management system was written to gather, store and report student progress data from an Access database using active server pages coded to transfer data from client to server and vice versa. The elements of the management system are a Learning Contract, a Marks Module, and a Module Evaluation form. Communication is managed using off-the-shelf communication tools, such as Eudora™ for e-mail and WebBoard™ for web conferencing. Students using a different e-mail package may continue to use their own software.
Project Planning and Implementation
Professor Michael Szabo is the Project Director and designed the system, oversaw its development, and created most of the Modules. Annette Fuchs, Assistant Director, developed modules and the WWW interface and collaborated with reviewers and other developers. Brian Johnson, a graduate student in the Instructional Technology M.Ed. designed several modules and assisted in the validation of others. Bradley Clark, another graduate student participated in the validation of modules, development of scoring keys, and creation of active serve pages for data transfer between client and server.
A standard system of development was followed, including design, development, validation, and implementation. These stages were completed between April, 1997 and August, 1997. A needs analysis was not conducted as this project involved rebuilding an existing course in a new delivery format as opposed to creating a new course from scratch.
A list of topics and objectives for the modules of the course was developed based on the experience of the Project Director. This list was circulated to members of the IT faculty and several graduate students for review and comments, which were incorporated into the topic list
A format for the modules was developed to provide a modest amount of instruction and development. Each module contains a title and humorous quote, and is divided into the following sections: Purpose, Objectives (Core and Enrichment), Prerequisites, Preassessment, Activities (Core & Enrichment), Postassessment, Resources, Glossary and Module Evaluation form. See the web site below for a sample Module. The complete course structure, including modules and management elements can be seen at http://www.quasar.ualberta.ca/edmedia/ETCOMM/ithome.html/
Core and Enrichment Activities
The heart of the course consists of a series of activities in each module. The activities are required (core) and optional (enrichment). There is a one-to-one correspondence between objectives and activities. Students are required to complete all core activities and choose one enrichment activity in order to satisfy a module.
Assignments are submitted by e-mail or file transfer or by uploading to the central gpu server. Activities use a mixture of individual or group work, posting to a conference board, reading and reflecting, or creating a product.
The original intent was to build activities around resources which were freely available on the web. As such, significant amounts of time were spent identifying, evaluating and discarding WWW resources. We concluded that we would have to access additional resources of a print and text nature to provide the best activities possible. The current configuration uses a combination of learning resources. However, the main goal, that of creating learning activities without having to write new content was met.
Next we created and located supplementary resources to support the activities, including such things as pre-post tests, purpose, glossary and supplementary resources. Since numerous lessons used Authorware for computer assisted and computer managed instruction, considerable time was spent in identifying useful lessons and streaming them for use on the Web site. During the project, version 4 of Authorware was released and this made it much easier to stream Authorware lessons. Another supplementary resource is a master list of all the resources required, by module. This list was later used to calculate and submit copyright royalties for materials photocopied under the CANCOPY agreement with the University. Finally, answer keys for each of the activities were created. An additional resource is the personalization of the course in which we post photographs and e-mail addresses of all students, subject to their approval.
As modules were produced, they were validated by staff and graduate students who are familiar with the current educational system. By validation, we mean the careful review and identification of errors or factors which would make for a better module. Validators were also asked to provide time estimates for each activity so that we might judge the appropriate length of the course we were revising in terms of a 3 credit course.
The modules and related material were incorporated into a web site which we designed from the ground up. The major tool used was Front Page (version 1 for Mac and 97/98 for Windows). This was enhanced with limited use of HTML coding. A graphics artist was seconded to create the graphics for the web site. It should be noted that the IT program has identified 5 courses for WBI along with a consistent set of graphics which use the theme of a wave, whether it be water or snow. The theme for EDIT 571 is "Ride a Wave" as can be seen from the web site and the title page of this report.
We experimented with several commercial conferencing systems before settling upon O'Reilly's WebBoard. The module evaluation form, completed at the end of each module was created using forms and ASP coding to capture the data to an Access database.
A programmer was hired to write the code to create a learning contract and marks program. The learning contract allowed the student to register and choose those modules to complete and select the due dates for each module. The software uses ASPs and Java Script to capture data from the client, transfer it to the NT server and deposit it into a database structure. When students want to review their contract, they requested it using their student id number and the request is translated into a SQL search. The search results are formulated into HTML and the desired information is displayed to the student.
The Marks program was similar except that the instructor enters student marks into the marks program as the activities are graded. These marks are then available to the student in a similar fashion to the learning contract.
This entire package was assembled and tested as a system during July and August of 1997. Links were checked, editorial review of the site was completed and the course was ready to deliver.
The course was delivered in the Fall Term of 1997 to 35 students (20 in EDIT 571 and 15 in EDPY 488). During this time, extensive data was collected on all aspects of the functioning of the course, including formal and informal evaluations. During the final week of the term, students were invited to provide feedback through written means and through a focus group led by an expert in evaluation studies
The courses were offered for the second time during an abbreviated 6 week Spring Session in 1997. At this time, several students from northern Alberta signed on and completed the course at a distance, giving us feedback on the efficacy of distance delivery with this approach. The courses were offered again during the Fall Term of 1998 to a mixture of on and off-campus students.
There were several notable accomplishments that resulted from this course.
First, it has been demonstrated that WBI courses can indeed be delivered anytime, anywhere. This was of course, never in doubt in the mind of the Project Director, who has been involved in distance delivery with computer based instruction since 1970. However, this experience should serve to convince others of the viability of the approach.
It is clear that because the courses can now be delivered anytime and anywhere, access has been increased. The course was designed from the ground up to accommodate both on- and off-campus learners. Expansion to include significant numbers of students is subject to advertising the options and a re-thinking of administrative policy. We can say with confidence that this course, with a cadre of grading assistants, could be easily scaled up to handle hundreds of students and on an open admission basis. As an aside, students enrolled in the fall 1997 class were polled regarding their access to the necessary technology to complete the course at a distance. All 35 students had access to their own computer and only one student did not have a modem. Most students (65%) reported completing the course using computers in their homes.
Increased Effectiveness and Efficiency of Learning
The key questions to ask are, does this method of delivery increase achievement, efficiency and access to a course of instruction and by implication, to other similar courses? Relative to achievement, no data-driven studies of a comparative nature, with control group etc., were done with this course. Subjectively, it can be concluded that because students were more actively involved in their learning (completing the learning activities vs. listening to lectures), the quality of learning was increased. As Richard Clarke has argued in his criticisms of media research, any increases might be attributed to the extra effort which went into course design, enhanced principles of learning or a host of other factors, rather than the technology.
With respect to efficiency, that is the amount of time needed to complete the course requirements, the anecdotal evidence points to two students who completed the course in nine weeks, as opposed to the conventional 13 plus weeks. One of those students held down a full time job which required that he travel to Ireland or Toronto for 10 days out of every month. Only one student took an incomplete and had not completed the course 13 months later. Only two students had to drop the course and only one took extra time to finish.
Our records suggest that it required four-person months, spread over 5 calendar months, to redesign the course for WBI delivery format. This amounts to 1.2 modules produced per week. Another one-person month was used to complete the course revisions which arose out of the pilot testing. These data must be qualified, however, in that the two people who designed and developed the course consisted of a veteran with 30+ years of experience in the field and a highly capable person with a Masters in the field and several years of development experience. A team of developers without this level of experience would likely take significantly longer to do a similar amount of work, perhaps as much as much as 6 person-months. And in all liklihood, the list of needed revisions would require considerably more than one month of attention.
It was estimated that running the course for the first time, including grading, communicating, fixing bugs and upgrading materials took approximately 11 hours per week. While there are no hard and fast data for conventional courses, using the rule of two hours outside preparation for each hour of class yields a figure of 9 hours per week. Thus the operation of this WBI course appears somewhat more labor intensive. However, the next time the course is delivered, the new and conventional delivery times should begin to balance out.
The Transformation of Learning
The conversion of this course is necessary but not sufficient to transform the learning program of a research university. The M.Ed. in Instructional Technology can now offer all its 5 core courses via distance delivery as well as through the conventional on-campus mode of instruction.
There are many other significant factors which must be addressed, factors which are considered in the TIES project, for example (http://www.quasar.ualberta.ca/edmedia/TIES/Ties_home.html). Several students enrolled in Athabasca University's MDE program have also taken this course as an approved option, which raises the question of cooperation among institutions. Since inter-institutional cooperation was not a goal of this project, one should not be surprised that it didn't occur (along with several other possible but incidental outcomes). However, such cooperation did occur informally and provides a basis for discussion at higher levels to ascertain the nature of such cooperation.
Once the course began to be delivered, there were four forms of evaluation which took place. First was word of mouth, then module evaluation for each module, paper based survey and focus group session.
During the term, Professor Szabo agreed to be in a computer lab during regularly scheduled course time, three hours every Thursday evening. Professor Szabo made it clear that this was a pilot of the course and students are encouraged to make known as soon as possible any concerns they had. Special emphasis was placed on the course workload with respect to a three credit course.
At the end of each module, an evaluation form was embedded which gathered information about the students' reaction to the learning experience provided by that module. Data were collected and captured in a database on the NT server.
During the last week of the course, students were asked to attend an evening session which would be devoted to 1) providing feedback about the course and 2) consuming copious quantities of pizza and soda. The evaluation was conducted by Dr. Stanley Varnhagen, evaluation specialist with Academic Technologies for Learning (also LEE funded). Professor Szabo was out of town during this evaluation.
During the same meeting just described, Dr. Varnhagen also conducted a focus group evaluation with the students.
At the end of the Spring Term 1998 course offering, Professor Szabo conducted a course evaluation with several students from the course.
A general summary of the findings from the evaluations with students is presented here.
The findings were carefully screened and a list of 21 updates and changes was created. These revisions were made during the months of January through April, 1998.
Means of Disseminating Results
Within the Instructional Technology area, the project and others like it are well-known due to the close collaboration among the developers of the program. The course was demonstrated to members of the university administration and UCAT group at a planning retreat in December 1997. The Final Report will be distributed to the members of the Department of Educational Psychology and Dean of the Faculty of Education, and will be placed on the Faculty of Education Web site. A workshop on the project was presented to the larger university community as part of the Showcase of the Academic Technologies for Learning group. A workshop on the course was presented to participants of the Council for Distance Education and Athabasca University MDE program in May, 1998. A description will be submitted for presentation to at least two major WBI-oriented conferences. During international training worskshops in Thailand and Singapore in April 1998, Professor Szabo used the course format and web site as a model and example for others to use in creating their WBI courses. The course was also used as a model during the training of TIES Leadership Task Forces in February, 1998. Numerous graduate students in the Instructional Technology program participated in the design, development and evaluation of the course and will take this knowledge back to their local schools.
The course structure was used to re-develop EDIT 568 from a conventional F2F format to WBI. The course was piloted with 26 students (several off-campus) during the Fall Term of 1998. As part of another LEE grant, LEE-II, the course model will be used to re-develop EDIT 572 for WBI delivery. Once all the courses in the M.Ed. program in IT are deliverable through WBI, plans will be developed to serve more students around the Province and across Canada.
LEE/571 |
UNIVERSITY |
|||
BUDGETED |
ACTUAL |
BUDGETED |
ACTUAL |
|
Content Experts, Authoring |
6000 |
15486 |
6000 |
8000 |
Consultants/Secretary |
2000 |
|||
Graduate Student Support |
12570 |
5895 |
2999 |
|
Project Management |
2000 |
1775 |
||
Instructional Design |
||||
Internet Specialist/Tools |
1800 |
|||
Graphics Interface Design |
300 |
|||
Audio Visual Production |
||||
Staff Development |
||||
Meeting Expenses |
||||
Equipment |
14485 |
12750 |
2800 |
|
Software |
1354 |
|||
Copyright Clearance |
378 |
|||
Library Costs |
||||
Miscellaneous |
430 |
285 |
2400 |
|
Overhead @ 15% |
3000 |
|||
Pilot |
||||
Evaluation |
500 |
|||
Dissemination |
||||
Other |
||||
SUBTOTALS |
35485 |
35485 |
6285 |
25952 |
The Project Director would like to acknowledge the many people who assisted in this project to make it a success. The list includes the students themselves, technical support people of the Faculty of Education, and ATL graphics artists.
Annette Fuchs contributed her expertise and WWW development talents in almost every aspect of the project. Much of the innovation and creativity expressed in the course are directly related to Annette's influence.
Special recognition must go to two other people who made enormous contributions to the program during their time as graduate students in the Instructional Technology program. Bradley Clarke, a teacher in the Leduc County Schools provide excellent advice, judgement, and programming skills as well as a high level of diligence throughout his entire involvement in the project. Brian Johnson, a teacher in the Red Deer schools provided great insights into providing a course with a nice balance of practical application and learning theory. And everything came in on time and on budget so the Project Director didn't have to yell once! Okay, maybe once.