LEARNING OF BUSINESS STATISTICS
This study involved the development of multimedia movies
covering the core concepts in an introductory business
statistics course. The movies were designed to aid students
in content mastery in an online course setting. Such tools
are critical to online learning, particularly for courses
requiring the acquisition of precision skills such as
mathematics, engineering, computer programming, and the
like. The movies incorporated the findings from multimedia
instructional research regarding the most effective features
in such tools. While the movies have been used in two
online sections of the course, two of the movies were given
more rigorous testing using a randomized design.
Results indicated that students using the multimedia
approach scored no differently from students receiving a
traditional instructor led intervention. Further repetitions
of the experiment will provide strength to this finding.
BACKGROUND AND PURPOSE
For the past decade, web-based instruction has been
widely accepted and practiced in American universities with
the hope to increase university enrollment, promote access
to higher education, and allow greater flexibility in
instruction (Waschull, 2001; Ridley et al., 1997). However,
many educators are still skeptical and concerned about the
quality and effectiveness of web-based instruction as
compared to traditional face-to-face classrooms. Inman and
Kerwin (1999) found that although most instructors who are
currently teaching web-based courses are willing to teach
the courses again, 50% of them reported that the quality of
the web-based instruction, comparing to a traditional format,
was of lower quality.
A considerable body of research has been done to
investigate the effectiveness of web-based instruction and
has yielded mixed results. Many studies found no
differences between web-based and traditional classroom
courses on student performances. Russell (1999) reported
that more than 300 studies investigating traditional classes
and distance courses found no difference in performance of
students. His reports included correspondence courses,
online courses, and telecourses. However, other researchers
reported contradictory results (Waschull, 2001; Wang &
Newlin, 2000; Ridley, 1998). For example, Mottarella et al
(2004) examined students’ course grades and achievement
scores on ACAT (the Area Concentration Achievement Test
in Psychology) and found that students in web-based
courses had significantly lower course grades than
classroom-based or web-enhanced (blended/hybrid) courses.
Furthermore, students in both web-based and web-enhanced
courses scored significantly lower in standardized
achievement test (ACAT).
While most of the studies strived to investigate
performance related differences between traditional and
web-based courses, the design of those web-based or webenhanced
courses are rarely addressed. Are the web-based
courses designed with sound cognitive learning theories?
Are the cognitive load imposed on students reasonable? Are
there any multimedia components built into the course? If
so, is the design of multimedia components based on
multimedia learning principles proven to be effective? We
believe that in order to make a valid comparison between
traditional and web-based/web-enhanced courses, we need
to first address the design of web-based/web-enhanced
course. This approach will help us avoid comparing a welldesigned
traditional course to a poorly-designed webbased/
web-enhanced course.
The design of the web-enhanced course under
investigation in this study is informed by a wide range of
literature on learning and cognition theories.
COGNITIVE LEARNING THEORY
Cognitive learning focuses directly on human cognitive
processes, “considering how people perceive, interpret,
remember, and otherwise think about the environmental
events they experience” (Ormrod, 1999 p. 145). Cognitive
information processing theory provides a framework for
understanding how human learn and think. It believes that
memory plays an important role in learning.
There are three stages of memory during information
processing: sensory memory, short-term memory, and longDevelopments
in Business Simulation and Experiential Learning, Volume 34, 2007
52
term memory (Ormrod, 1999). During the first stage of
processing, the stimuli are registered in the memory system.
This sensory memory briefly holds information until the
stimulus is recognized or lost. When people attend to and
recognize the stimulus, the information is then sent to the
short-term memory. The short-term memory can hold the
information longer than sensory memory through
elaboration and rehearsal. Finally, once information is
encoded from short-term memory (activate and link to prior
knowledge), it is stored in the long-term memory in the
form of schemata, which is a complex, organized, and
permanent body of knowledge. Thus, facilitating schema
acquisition and construction should be a primary goal for
instruction and learning.
COGNITIVE LOAD THEORY
Cognitive Load Theory (CLT) has evlolved ffrom
information processing theory and has been used to
investigate some conditions that need to be considered to
construct schemas. Cognitive load refers to the total amount
of mental activity imposed on working memory at an
instance in time (Cooper, 1998). There are three types of
cognitive load: intrinsic cognitive load, extraneous cognitive
load, and germane cognitive load. Intrinsic cognitive load is
associated with the nature of the instructional materials and
is normally irreducible. Extraneous cognitive load refers to
the demand imposed by the manner in which materials is
presented and the activities required of the learner (Sweller,
van Merrienboer, & Paas, 1998). It’s also called ineffective
cognitive load, which usually caused by poorly design
instructional materials. Germane cognitive load refers to the
memory resources used to engage learners in conscious
cognitive processing that is directly related to schema
construction.
Thus, the primary goal of CLT is to support the design
of instructional procedures and activities that effectively
manage cognitive load to enhance learning. In other words,
reduced extraneous cognitive load and increase germane
cognitive load. Some of the recommendations proposed by
Sweller (1999) include:
• Change problem solving methods: Avoid meansends
approaches and use goal-free problems or
worked examples.
• Integrating multiple resources to eliminate the need
for learners to have to mentally integrate that
information.
• Reduce repetitive and redundant information.
• Use auditory and visual information under
conditions where both sources of information are
essential (i.e. non-redundant) to understanding.
COGNITIVE APPRENTICESHIP
Cognitive apprenticeship theory takes another approach
and attempts to “enculturate students into authentic practice
through activity and social interaction” (Brown et al., p.37).
Brown (1989) proposed the cognitive apprenticeship as an
implementation of situated cognition. It is based on the
traditional apprenticeship where apprentices initiated into a
profession through modeling, coaching, and fading. For
example, when teaching a new topic, the mentor would
make his/her thinking explicit through modeling the
cognitive activity. Then, modeling is followed by coaching
where the mentor provides support and guidance in the
learning process. Mentor support is gradually reduced as
students grasp the concepts.
In addition to the traditional apprenticeship
components, cognitive apprenticeship also embraces
collaboration, reflection, and articulation (Brown et al.,
1989). Collaboration supports group discussion and
problem-solving, which allows learners to understand
multiple aspects and roles. Reflection enables learners to
compare their own performance with other resulting in
reflection and deep thinking. And articulation encourages
learners to make their thinking explicit, which promotes
reflection and comparison.
Therefore, it is important to provide students
opportunities for modeling, reflection, articulation, and
collaboration during their learning process.
MULTIMEDIA LEARNING THEORY
Mayer (1997) reviewed 24 research studies on
multimedia learning and proposed a generative cognitive
theory of multimedia learning based on dual coding theory,
cognitive load theory, model of working memory (Baddeley,
1992), generative theory (Wittrock, 1989), and Mayer’s
(1996) SOI model of meaningful learning.
Mayer’s multimedia learning theory proposed that the
learner possesses a visual information processing system
and a verbal information processing system (Mayer, 1997).
These two systems, then, can be used to explain learner’s
cognitive process in multimedia learning. Mayer and
Moreno (2002) argued that learners engage in three
important cognitive processes in multimedia learning. The
first cognitive process is Selecting. Verbal and visual
information serve as signals that help learners select
relevant information. The second cognitive process is
Organizing. Both visual and verbal information serves as
organizers that help learners build cause-and-effect relations
among pieces of visual information and among pieces of
verbal information. The final cognitive process is
Integrating. Learners build connections between
corresponding parts in the visual representation and verbal
representation. This cognitive model is illustrated in Figure
1.
Derived from this cognitive model, Mayer and Moreno
(2002) further recommended five principles on the design of
multimedia learning.
1. The multiple presentation principle: It is better to
use two modes of representation (visual and verbal)
rather than one.
Developments in Business Simulation and Experiential Learning, Volume 34, 2007
53
2. The contiguity principle: Students learn more
deeply if they do not have to hold the entire
animation in working memory until the narration is
presented, or vice versa. Thus, corresponding
words and pictures need to be presented
contiguously.
3. The split-attention principle: Words are better
presented as auditory narration rather than onscreen
text.
4. The individual differences principle: Multimedia
effect, contiguity effects, and split-attention effects
depend on individual differences. Learners who
lack prior knowledge tend to benefit more from the
above principles (Mayer & Sims, 1994).
5. The coherence principle: Students learn better if
they do not have to process extraneous words and
sounds in verbal working memory or extra pictures
in visual working memory. This principle is also
referred as redundancy effect in cognitive load
theory (Sweller et al., 1998).
The literature review considered thus far leads us to
believe that in order to promote meaningful and effective
learning, the following elements should be considered and
built into the design of multimedia learning modules in our
web-enhanced statistics course:
• Contiguous presentation of visual and verbal mode
of instructional materials (multiple
presentation/contiguity principle, cognitive load
theory).
• Audio narration of instructional materials (splitattention
principle).
• Elimination of repetitive and redundant
information (coherence principle, cognitive load
theory).
• Opportunities for modeling and reflection
(cognitive apprenticeship).
• Presentation of worked examples (cognitive load
theory).
The current study focused on the development of
multimedia tools that could facilitate learning without
benefit of a classroom teacher while at the same time going
beyond a basic textbook. The aim was the development of
materials that would aid students in online statistics classes
in their knowledge mastery.
METHOD
To aid students’ mastery of the material online beyond the
mere provision of written material, the researchers created
multimedia FLASH movies covering the content of the
course. Thus, the movies covered:
• Graphing
• Measures of location and dispersion
• Probability distributions
• Hypothesis testing
• T and z tests
• ANOVA
• Correlation
• Univariate and multivariate regression
The movies were developed based upon research findings
regarding the most effective features of multimedia learning
tools, using Mayer’s principles:
• The movies incorporated the multi-presentation
principle, using both visual graphics and verbal
mechanisms to convey information.
• This multi-presentation ensured that the auditory and
visual offerings occurred in sync with the material
addressed, that is, it abided by the contiguity principle.
• The words were presented auditorily rather than as onscreen
text, in accordance with the split-attention
principle.
• Although students were given a physical copy of the
text of the movie, they were told it was only for
reference; the movie was the learning intervention and
contained no redundant information.
• Scores on pretests indicated that students had no prior
knowledge of the topics and the topics were complex
(Analysis of Variance, correlation, and univariate
regression), thus allowing for the individual differences
principle to manifest.
Figure 1. A cognitive model of multimedia learning (from Mayer, 1999)
Developments in Business Simulation and Experiential Learning, Volume 34, 2007
54
• Interspersed in the movies were opportunities for
students to test their knowledge using a multiple choice
format. If students chose an incorrect answer, a
message explaining why the answer was incorrect and
redirecting the students’ thought processes was
provided. This aligns with Mayer’s cognitive
apprenticeship principle.
• Worked examples were provided both within the movie
itself and within subsequent practice opportunities.
The movies incorporated other features such as
• Guided learning mechanisms or cues (Huk, Steinke,
and Floto, 2003). These included procedures such as
moving arrows, flashing results, circled points of
emphasis, and others.
• Navigation tools which included the ability to stop the
movie and repeat sections or to jump to specific
sections (Koohan & Plessis, 2004),
• Attention influencing instructional strategies such as
movement, variation in presentation and color and
positioning (Farley & Grant, 2001).
HYPOTHESIS
The hypothesis was that students watching the
multimedia movies would perform no differently from
students participating in an instructor-led class. Of course, a
hypothesis such as this is the inverse of the usual scientific
testing criterion where the effort is to discover if the
performance of the treatment group is better than that of the
control group. Failure to reject the null hypothesis does not
mean that the null hypothesis is true. Thus, such a result
does not have the same power as that in the alpha error
testing approach and, as a result, is more limited in its
ability to confirm the value of the treatment. However,
repeated testing with similar results would provide strong
evidence in favor of the null hypothesis. This study
represents a first step toward the accumulation of that
evidence.
EXPERIMENT
Students enrolled in the Introduction to Business
Statistics class in the adult college at St. Edward’s
University during the 2004-2005 school year participated in
the study. For the most part, adults enroll at St. Edward’s
because they value the classroom experience over online
learning. Thus, the study was biased against a favorable
performance from watching movies versus having a
traditional instructor-led class.
Forty-two students were randomly assigned to
treatment and control groups. Both student groups first
completed a pretest over the content material, analysis of
variance. Then those in the treatment groups viewed the
movie in a computer lab, completed homework, and took a
posttest. Those in the control group attended the class in a
traditional instructor led setting, completed homework
questions together with the instructor, and then took the
posttest. The next week, students swapped places such that
those who had watched the ANOVA movie the first week
received classroom instruction the second week covering
regression and the converse regarding students in the control
group the first week.
Upon completion of the posttest, students viewing the
movies were asked to rate the movies along several
dimensions.
RESULTS
Table 1 provides descriptive statistics for each group:
(See tTable 1 below)
While in both experiments, students receiving in-class
instruction scored better than did those watching the movies,
was that performance significantly better? Followup t-tests
of means indicated that students watching the movies,
whether ANOVA or Regression, performed no differently
from those in the traditional classroom setting (ANOVA
movie: t=1.42, 40 d.f.,p=.161;Regression movie:t=2.02,
df=40,p=.149).
DISCUSSION AND CONCLUSIONS
These results indicate that, although the control groups
performed better than the treatment groups, the differences
in performance were not statistically significant. This
indicates that the provision of multimedia tools covering
course content can assist students in mastering material
related to an introductory statistics course, enabling mastery
comparable to an in-class teacher-led intervention. The
results are particularly significant given the population of
students in the study. These students are willing to spend a
Table 1: Descriptive Statistics
ANOVA Median Mean Standard Deviation N
Treatment 87.7% 60.4% 29.6% 21
Control 60.0% 74.1% 32.4% 21
Correlation &
Regression
Mean Standard Deviation N
Treatment 80% 63.3% 30.7% 21
Control 80% 76.9% 29.0% 21
Developments in Business Simulation and Experiential Learning, Volume 34, 2007
55
significant amount of scheduled time in class and tuition
money to receive in-class instruction, believing that this
type of instruction is better for them. That they nevertheless
performed at a level similar to those in the in-class
environment reflects the effectiveness of the multimedia
instructional design. Further, the material covered in the
experiment was some of the most complex material covered
in the introductory statistics course. The students’
performance again attests to the effectiveness of the
instructional materials in facilitating content mastery. The
development of such materials is particularly important for
students enrolled in online technical courses whose only
references may be textbooks or other written materials.
In addition to the above test results, students watching
the movies were administered a questionnaire asking for
their perceptions of the effectiveness of the movies and
specific features of them. The results are given in Table 2.
(See Table 2 above)
These results indicate that the majority of students
believed most of the features to be effective aids in learning.
Surprisingly, 95% of the students found the audio script to
be of significant value. This runs counter to the redundancy
principle. However, it is probable that for such complex
material, students may find written materials helpful for
reference. It should also be noted that the written script did
not appear on the multimedia page, causing the split
attention effect. It was only ancillary.
Future repetitions of this experiment can strengthen the
findings of this study. Future studies may also benefit from
comparing the use of multimedia tools with the use of
textbook materials only. This latter is especially valuable
because of the cost of creating, revising, and continually
updating multimedia tools. Finally, studies of online
learning effectiveness should incorporate detailed attention
to the materials provided the online students. Differentiated
instructional mechanisms may be needed to optimize
learning online from that occurring in the traditional
classroom settings, particularly for certain types of course
content.
REFERENCES
Baddeley, A. (1992). “Working memory”. Science, 255,
556-559.
Brown, J. S., Collins, A., & Duguid, P. (1989). “Situated
cognition and the culture of learning”. Educational
Researcher, 18, 32-42.
Cooper, G. (1998). “Research into cognitive load theory and
instructional design at UNSW”. Retrieved December,
2006, from
http://projects.ict.usc.edu/itw/materials/clark/UNSW.ht
m
Farley, F.H. & Grant, A. (2001). “Arousal and cognition:
memory for color versus black and white multimedia
presentation”, The Journal of Psychology, 94, 147-150.
Huk, T., Steinke, M, Floto, C. (2003). „The educational
value of cues in computer animations and its
dependence on individual learner abilities”.
Proceedings of Ed-Media 2003, 2658-2661.
Inman, E. & Kerwin, M. (1999). “Instructor and student
attitudes toward distance learning”. Community College
Journal of Research & Practice, 23, 581-592.
Koohan, A and Plessis, J (2004). “Architecting usability
properties in the E-learning instructional design
process”. International Journal on E-Learning, July-
September 2004, 38-44.
Mayer, R. E. (1996). “Learning strategies for making sense
out of expository text: The SOI model for guiding three
cognitive processes in knowledge construction”.
Educational Psychology Review, 8, 357-371.
Mayer, R. E. (1997). “Multimedia learning: are we asking
the right questions?” Educational Psychologist, 32, 1-
19.
Mayer, R. E., & Sims, V. K. (1994). For whom is a picture
worth a thousand words? Extension of a dual-coding
theory of multimedia learning. Journal of Educational
Psychology, 86, 389-401.
Mayer, R. E., & Moreno, R. (2002). “A cognitive theory of
multimedia learning: Implications for design
principles”. Retrieve October 26, 2003, from
http://www.unm.edu/~moreno/PDFS/chi.pdf
Table 2: Student ratings on effectiveness of instructional media.
Feature Effective Neutral Ineffective
Navigation controls 72% 11% 17%
Audio 95% 5% 0
Graphics 77% 17% 6%
Worked examples 67% 17% 11%
Interactive Q&A 44% 22% 11%
Speed of movement through content 50% 33% 17%
Use of arrows, blinkers to call attention
to content
94% 6% 0%
Clarity of content presentation 77% 11% 11%
Provision of audio script to accompany
module
95% 5% 0%
Overall quality of module 67% 28% 6%
.376.
keep it up!!!!!!
TumugonBurahin