Blended Learning in Engineering Education

This paper reports the findings of a two year pilot study introducing blended learning in a school of engineering at a major Australian university. The school has been developing a blended learning Masters course since 2005. Blended learning denotes the blend of face-to-face with web-supported learning, promising to offer globally disparate students educational access.  In the pilot project two interactive e-learning lecture presentations covering a fundamental engineering concept were developed and delivered via the University’s learning management system, followed by a face-to-face workshop. A short assessment was included to check students’ understanding of the topic, followed by a question in the unit exam.

The purpose of the study was gain a greater understanding of how students learn and access their learning in a blended environment. The pilot sought to evaluate, on a small scale, the development of course materials in the larger Masters project. The results showed that blended learning offers opportunities for deep and persistent learning in an online environment when well designed and developed, but also raised a number of questions, which are further explored in the paper.

Keywords
Interactive learning environments; authoring tools and methods; post-secondary education; applications in subject areas; pedagogical issues

Access, quality and learning in a Pilot of Blended Learning in Engineering Education

Karin Oerlemans and Eric F. May

Introduction: Aims of the Pilot Study
There is a view that the internet provides an almost limitless arena for knowledge, and that the development for the twenty-first century is a “brave new era of borderless learning” (Brown, 2000, p. 348) through the provision of online learning and the establishment of e-Universities. Although it is now some years on from when that view was expressed, the School of Mechanical Engineering at a research intensive University, has been engaged in a large teaching and learning project to develop a Master of Engineering to be delivered via a blend of learning modalities, combinations of face-to-face, printed materials, and technology-based or online learning. This paper reports the findings of a small pilot study, undertaken as part of the larger project, as second year undergraduate engineering students were introduced to the greater flexibility of an advanced mode of technology enhanced learning to test and evaluate the blended learning delivery to be used for the Masters programme. Data was collected over two years, and the results evaluated in order to better understand the process adopted and how students accessed online learning.

The aim of the pilot was to test the proposed blended learning delivery in an environment where immediate student feedback could be sought and would allow for an iterative process of design, evaluation and revision as necessary throughout the project (Akker, Gravemeijer, McKenny, & Nieveen, 2006).  One of the guiding questions for the study was to understand how students would access the materials provided through an examination of the time and frequency of their access (Morris, Finnegan, & Wu, 2005), and whether the engagement with the online resources would encourage improved understanding for the students measured through performance in their assessments (Hrastinski, 2009). The study was run utilising a second year engineering unit, Process Engineering, usually delivered face-to-face. The technical content of the pilot study was ‘Heating values and enthalpy’, which is a fundamental concept that second year engineering students can have great difficulty in understanding. It is an essential concept that, when not well understood, has repercussions in third and fourth year chemical and process engineering. Previously, students had covered the material in three lectures and one workshop. This was then followed by a question in the end of semester exam. Anecdotal comments from a lecturer of the students, once they had reached third year level, suggested that students had failed to grasp the material in their second year, and he always had to re-teach the material in the third year. An examination of the students’ exam results from previous years, confirmed this perception.

The rest of this paper explains the teaching and learning issues to be resolved when embarking into blended learning, the methodology employed for the conduct of the pilot study, before discussing the outcomes of the pilot and their implications for the ongoing development of the Masters course.

Background: Online Teaching and Learning Issues
Blended learning, a term used to denote the blend of face-to-face with web-supported learning, is said to offer student improved educational access throughout the world.  The promise of blended learning is to bring ‘high-touch’ education to students globally situated, drawing on the strengths of both face-to-face as well as the flexibility of online learning (Graham, 2006). A blend of learning modalities, employing such technologies as learning management systems, video conferencing, mobile devices as well as the more traditional face-to-face delivery can be utilised to ensure frequent opportunities for students to participate and be ‘in touch’ with their tutors and fellow students. Blended learning not only provides for these extended modalities, but has also been successfully trialled with a number of different educational learning styles, including individual, paired and collaborative group work, through the use of chat, discussion groups, and mobile devices (see some of the chapters in Bonk & Graham, 2006b). It offers opportunities for deep and persistent learning when well designed and developed.

The blended learning format combines the teaching and learning approaches of face-to-face with the established fields of distance and electronic learning (e-learning), where distance learning is distinguished from conventional face-to-face provision of education for students who cannot physically attend a campus, and delivers materials through the medium of the post office. E-learning encompasses the online or web-based provision of the same course materials, including amongst other things recorded lectures, course readings and assessments through, for example, a learning management system (LMS). The strengths of these modalities of learning are the availability of learning opportunities to students wherever they are situated, at a time and place convenient to them (Graham, 2006). Blended learning offers opportunities to enhance student engagement, personalised learning, inclusion, and the use of productive time for those taking advantage of this approach (Laurillard, 2007). Yet, the literature on technology enhanced learning has also identified a number of potential problems and weaknesses.

Some of the inherent weaknesses in e-learning and distance learning include the often poor design (especially of many e-learning courses, which are often content driven and just the online placement of a face-to-face unit), the relative isolation of many of the students and their lack of immediate interaction with teachers/lecturer and other students, and the assumption of autonomy of the student and the attendant responsibility for taking the initiative and self-discipline to listen to lectures, engage with the course materials and complete the set assignments (Oh, 2003). One of the most commonly cited issues is the apparent practice of placing existing face-to-face material on the web and calling them e-learning, yet they are more often a reading experience rather than a learning experience (Blass & Davis, 2003), or ‘content-oriented’ rather than ‘learning-oriented’ (Gonzalez, 2008). A major criticism has in fact been the lack of engagement students have or are encouraged to have with the materials (Oh, 2003). A fundamental construct of learning is the development of an interactive learning environment that will engage students in their learning. There has been a growing interest in the development of Vygotsky’s constructivist principles of teaching and learning, especially in online education (Altalib, 2002). To achieve the desired learning outcomes in this framework, students must actively engage with the subject, its content and each other, as well as interact with a learning facilitator. As Hrastinski (2009) has recently argued, “If we want to enhance online learning, we need to enhance online learner participation” (p. 78). Students are required to construct their own meanings in an environment that is principally student-centred and encourages flexibility, accessibility and an understanding of the different learning preferences (individual, paired, or collaborative) of the students. It has therefore been imperative that the instructional designers in the larger Masters project  “help create environments for learning, which in part are populated with predetermined learning resources, yet contain within them tasks, activities and spaces with the potential to bring forth new types of learning interactions, learning resources and outcomes not pre-ordained by traditional instructional design approaches” (Segrave & Holt, 2003, p. 10).

Blass and Davis (2003) suggest a number of criteria that should be met to produce a quality online learning experience. These include ensuring that the online learning experience is both appropriate to the subject and sustainable over time, constructing the material to achieve the learning objectives stated, and building it around pedagogy rather than just the functionality of the technology, creating appropriate spaces for both student-student interactions and for student-faculty interactions, and finally ensuring that both the student learning and the online learning product are effectively assessed. Oh (2003) lists similar criteria for effective online course construction, but adds that the whole learning environment must match the learner needs. He makes the point that an important player in e-learning is the student, and that part of the focus of the construction of the learning material must be on their learning needs and demands.  Liaw, Huang and Chen (2007) would add that to positively affect students’ attitudes the e-learning should consist of a self paced learning environment and should include a form of multi-media instruction. The pilot study sought to exploit these suggestions to enable quality course development, focussing where possible on teaching and learning and not only the technology, but taking advantage of the multimedia possibilities available as explained in the rest of the paper.

In late 2005 a decision was made by the University to offer a blended learning Masters course to mobile and distant employees of the oil and gas industry. There was seen to be a need for courses of this type, particularly in an era where more and more students search globally for education that suits their particular needs, lifestyle demands and learning styles.  The resulting Master of Engineering was designed using a blended learning format to enable students, who are unable to participate in traditional classroom-based learning, to further their education, no matter where they are physically located. Students enrolled in the program are generally global people working in the oil and gas industry, more often away from home, than near the bricks and mortar of university buildings, which would otherwise enable them to participate in lifelong learning opportunities. The new Masters course was developed with a balanced view that recognised that the students are working adults in an industry with great mobility, complexity and variety (Kasworm & Hemmingsen, 2007). It is also an industry that has seen tremendous technological development in its relatively short history, and hence many of the current and potential lecturers are in fact industry experts working in the field. Therefore, the definition of blended learning for the purposes of this work is extended to include the concept of mobile learning, which recognises the lecturers’ mobility as well as the nomadic nature of the student population. This mobility should be recognised as including both the real and virtual, the flow of people, images and information, that crosses societal borders “in new temporal-spatial patterns” networked across the globe (Urry, 2000, p. 186). It recognises the unpredictability, immediacy and speed of movement of people in an industry where its product, the discovery of new explorations, and viability of old fields, dominate the popular media. But it should also be recognised that the mobility of students even in other areas of higher education show dramatic increases (see Shaeffer, 2006, for figures pertaining to the international mobility of students across OECD countries).

Methodology for the conduct of the Pilot Study
To ensure that the new blended learning Masters course would work in the way anticipated, exploiting the possibilities of the technology, focussing on the pedagogy, whilst creating spaces for collaboration, a small pilot project was run, using an educational design research approach (Akker, Gravemeijer, McKenny et al., 2006), with second year undergraduate engineering students, enrolled in a face-to-face unit.  The methodology chosen for the pilot study was educational design research, which would allow for the designing of the resources in the real world of university teaching and learning, encourage an iterative cycle to allow for revision over time, focus on understanding both the process of production and the final product, and allow the resources to be evaluated for its practicality by the students in a very real context (Akker, Gravemeijer, McKenney, & Nieveen, 2006), that would also draw on an understanding of student learning and participation and access in online environments. The focus of the pilot project was on producing a quality product that was geared towards providing an appropriate balance so that students had the opportunity to both learn the materials and to provide them with a successful learning experience. Yet, the provision of the online experience was not about getting caught in the ‘hype cycle’, where e-learning is necessarily seen as the best, creating unrealistic assumptions of what the technology will do in the learning environment (Clegg, Hudson, & Steel, 2003). Instead, in the pilot study, as in the larger project, the aim was to design an interactive set of resources as part of the overall program, which would encourage students to discuss the materials and their learning with others, as online learning is best accomplished “when learners participate and collaborate” (Hrastinski, 2009, p. 79). Therefore the students were given opportunities to both rehearse the materials and engage with them in a number of different settings. Technology enhanced learning should be seen as complementary to, not instead of, the traditional model of education.

For the pilot study two engineering lectures that covered the topic ‘heating values’ were re-developed and delivered using the blended learning format that was intended to be used for the online Masters program. The lecture materials were re-developed for web delivery using the interactive e-learning authoring tools Articulate Presenter™ and Engage™, and uploaded on to WebCT. WebCT is the University’s LMS, and all students have access, and must access it to retrieve and submit assignments and course notices. Both previously lectured materials were condensed into one online presentation, covering the entire concept, which would be presented via web delivery. Articulate and Engage e-Learning authoring tools, utilising PowerPoint as the base program, allow the instructional designer to move beyond mere podcasting and encourage the students to actively engage with the material presented. The flash-based lecture presentation developed for the project included the use of animated annotations throughout that highlighted important points, multi-level navigation giving students ample opportunities for reviewing and rehearsing, and learning interactions that encouraged students to drill down into key elements of the equations presented. The students then participated in a face-to-face workshop to ensure they covered the topic adequately and to answer any questions they may still have had. In the online Masters the face-to-face tutorials would be replaced by either online discussion forums or with synchronous tutorials in a virtual classroom using Adobe Connect.

The online lecture was followed by a summative assessment task, used to allow students to check their understanding of the topic, but also so that the lecturer could check their understanding of the material delivered. This was followed by a second flash-based lecture presentation, the solution, which went through the assessment task and gave the students an opportunity to check their answers.  The solution presentation was made available to the students prior to the final exam. The final test of their understanding was included as a question on the end of unit exam. The advantage of providing the material in these multiple ways was to ensure that students had extended opportunities in an interactive setting for learning the material. After overcoming some initial hurdles the process was repeated the following year with a new group of second year students. Educational design research does not require the replication of the instructional innovating in precisely the same way, but allows for the adjustment of process and an innovation based on the ongoing evaluation of the results (Gravemeijer & Cobb, 2006). This proved desirable, in the current circumstances as will be discussed at the end of the paper. The students’ learning for each year was evaluated and their results on the exam question compared with the results for the same exam question from the year previous to the running of the trial.

One of the advantages of offering the materials via the university’s LMS was the opportunity to track students’ usage and compare this with the results.  Articulate™ can be uploaded as a SCORM (Sharable Content Object Reference Model) learning module to allow tracking of student access, the number of slides viewed and also record the length of time the students accessed the learning resources. This gave some interesting results, which will be discussed below. In the year prior to the pilot (2005) 46 students completed the exam. In the first year of the pilot (2006) 53 students completed the exam. In the second year (2007) 84 students completed the exam. To give a better comparison of the three years, all three sets of exams were given to a new marker to ensure a fair comparison from year to year, utilising an exam marking key prepared by the lecturer. Exams had all identifying material removed from them, including the year they were administered. Results were then compared and evaluated in contrast to the time students spent in accessing the online materials. Finally, in the initial year of the pilot students were given a WebCT questionnaire to provide feedback and on their experiences and give their judgement on the quality of the online course resources (Walker, 2006). The rest of this paper presents the results and evaluation of the process and explains how the information gathered was used to feed back into the larger project. However, the pilot also raised some interesting questions, around the assumptions made when introducing online education to students. These are briefly discussed at the end of the paper and suggestions made for further research.

Results
The outcomes of the pilot study showed that students had a variety of different access patterns and there was an improvement in student learning as indicated by their exam results discussed in this section. Descriptive statistics were used to analyse the data, including a one-way analysis of variance (ANOVA) to evaluate the effectiveness of the blended learning format in relationship to exam results. The survey feedback results were explored for common themes to gain a sense of students’ experiences of the format. As the numbers in the study were small, undertaken with different groups and at different times, the findings of the pilot are suggestive rather than conclusive. Furthermore, there were a number of technical problems with the online resources, which may have affected the outcomes, and these will be discussed at the end of the paper.

One of the main difficulties with using Articulate™ lectures in WebCT and the recording of the data was that the SCORM communication between WebCT and the lecture did not function in the way anticipated. Problems arose when students used internet browsers other then Internet Explorer, or used Mac PCs. The lecture and solution were about 40 minutes long each, and the SCORM sequencing should have recorded whether students completed the lecture or not after viewing all the slides, how many slides they viewed, and for how long they accessed the materials. This did not happen in quite the manner anticipated and students’ attempts were listed as incomplete, even though they were recorded as having accessed all slides and times recorded were often for longer than 40 minutes. For this reason, where students had viewed all the materials and had sat through the materials for longer than 40 minutes, even though the SCORM results listed them as incomplete, they were counted as having completed the material.

Student access to the resources
In Year 1 (2006) 53 students enrolled in the course and sat the exam. In Year 2 (2007) 85 students were enrolled in the course, though only 83 completed the exam. Access by the students, shown in Table 1, over the two years varied from a total of 1504 minutes (approximately 25 hours for the lecture in 2007) to not at all, although typically students accessed the material about 51 minutes in Year 1 and 48 minutes in Year 2. There were five students (or 9% of the cohort) in Year 1 and 12 (14%) in Year 2, who did not at any time access the lecture or the solution to the assessment task once it had been posted on WebCT.

Table 1: Measures of students’ access to lecture and solution measured in minutes

When comparing students level of access of the lecture materials with their results of the assessment task, Table 2 shows that those who completed the lecture presentation performed better than those who did not or who failed to access the materials altogether. (Six students in Year 2 failed to submit the assessment task, although all those in Year 1 submitted theirs. Students who failed to submit the assessment were excluded from the table.) The table shows that as students’ level of access increased (no access, not completed, and completed viewing the lecture) so their results improved, with 30% of students who completely viewed the lecture scoring at 80% or higher for the assessment. This is compared with the almost 6% of students in the same year who failed the assignment after not accessing the online materials. This is corroborated by the findings of Morris et al. (2005) in their study of the relationships between online participation and achievement; they found that frequency and duration of participation were an indicator of levels of achievement. Whilst this may seem an obvious outcome it is the comparison between the exam results in consecutive years, including the year prior to the intervention (see table 4), which shows the more marked effect of student access to the online materials, as discussed below.

Table 2: Percentage of students’ level of access to lecture and assessment task result

The exam results
Students’ levels of access of the online materials were compared with the results of their exams. Table 3 shows the percentage of students’ level of access to the lecture and the solution and their exam results. (All students enrolled in the course in Year 1 sat the exam, but two missed the exam in Year 2. Students who failed to sit the exam were excluded from the table.) One of those who missed the exam, failed to submit their assessment, and did not access either the lecture or the solution. The other student, who missed the exam, submitted and did well in their assessment (scoring ≥ 20), and also viewed both solution and lecture. In Year 1 all those who failed to access the lecture or solution failed the relevant question in the exam. In one instance, when completing the assessment task, there was one student who did not access the lecture, yet gained a high (>20) assessment result. This same student failed to access the solution when it was posted, and subsequently failed the exam question.

Table 3: Percentage of students’ level of access to lecture and solution and exam results (Year 1, Year 2)

Table 3 shows that for the two years of the pilot students who accessed the online materials did better than those who did not or who failed to complete viewing either the lecture or the solution. Table 2 and 3 show a general trend that increased access led to improved results. However, due to the failure of the SCORM communication between the online resources and WebCT, giving incomplete results, it is not possible to say if this is a significant correlation; however, as discussed next, a comparison between exam results over three years demonstrates the effectiveness of the online materials and the blended learning delivery.

To examine the overall effectiveness of the use of the online materials for student learning, the exam results over the two years after the introduction of the resources were compared with the results from the year prior to the study (2005), when students only attended face-to-face lectures and workshops. Table 4 shows that the average final result for the exam question displayed a marked increase from 2005 to 2006, and this improvement continued for 2007. A one-way ANOVA between the groups yielded a significant effect, F(2,140)= 7.767, p<0.05. Post Hoc analysis using Tukey’s HSD test indicated that 2005 scores were significantly lower than 2006 and 2007 scores. There is no indication of why there was an increase in the means for the exam result from 2006 to 2007; further investigation would be required to find out why this occurred, though the Post Hoc analysis did not show this increase to be significant. The results from the comparison between 2005, and 2006 and 2007, would indicate that the delivery of the materials through blended learning translated into significantly better exam results.

Table 4: Students exam results comparison of means, 2005-2007

The student feedback
In the first year of the pilot, students were also asked to give their feedback on the resources as part of the ongoing evaluation of the process and the product (Akker, Gravemeijer, McKenny et al., 2006). The WebCT questionnaire option was used to gather the feedback, the completion of which was optional.  Sixty two percent of students enrolled chose to complete the questionnaire. The comments were collated and similarities and differences were identified through thematic analysis (Creswell, 2005). Specific themes included issues with the technology, students’ perceptions of the process, and how useful they believed the online resources to be for their learning (Oh, 2003). Students would have the best understanding of the technology issues of the resources ‘in use’ and so their comments provided valuable insights in these early stages of the design process (Walker, 2006).

One significant technological issue was a problem with the quality of the sound recording, as one student commented, “the sound quality is terrible” and students sometimes found it difficult to listen to and wanted to see this improved. Other issues identified included the use on Mac OS platform. Some students were concerned that the online materials would eventually replace all the face-to-face lectures, as one student said, “I feel that live lectures should not be replaced by online lectures and this format should be chosen only when live lectures are impossible to deliver”.

However, most students appreciating the flexibility in the program, in particular the ability to repeat information was a significant positive, as a student commented, “the ability to go back and recover specific points was really quite nice” and another student commented, “Good use of technology as you could revisit any troublesome areas or see how values were attained”. Although another student suggested that if this was used again a discussion board would be useful so that “any questions we have can be asked and answered and shared”. Overall though the reception was good and most students said they enjoyed the online experience and wished to see it utilised further, as this indicative comment illustrates, “definitely a great way of doing it and I would love to see more of that clear concise learning”. The feedback from the questionnaire indicated that the concept of online learning was acceptable to the undergraduate students, “it’s a great process” (student comment), providing it does not replace the normal face-to-face, but become part of a hybrid or blended form of education (Young, 2002), as one student remarked, “we still need face to face contact for all our queries”.

The results from the quantitative and qualitative data show that the blended learning delivery gave students increased access to the material to be learned, particularly in giving them the ability to revisit difficult areas, and led to significantly better exam results.  It is certain that the flexibility of a blended approach is what students enjoyed most, “wonderful when people cannot attend lectures” (student comment) and “certainly more useful than iLectures [podcasts of lectures] where trawling through for a particular piece of information is not only time consuming but not always productive” (student comment).  But the flexibility needs to be measured against the obvious problem that some students will either procrastinate, or not take advantage of accessing all or just some of the materials, as happened with a small number of the students in the study (see also Graham, 2006).

Discussion
The use of blended learning has gained much acceptance in the corporate world, where it has been identified as one of the top ten trends to emerge in the delivery of knowledge and company in-house training (Bonk & Graham, 2006a). However, in higher education it has often been ignored as a way of teaching students, underutilised or badly constructed (Blass & Davis, 2003; Young, 2002), although this is changing rapidly.  The pilot project, drawing on educational design research methodology, proved an excellent method for testing an innovative strategy whilst at the same time ensuring students learned a fundamental concept, essential to student’s success in the later years of their degree. The research team, lecturers and instructional designers working together, were able to learn much along the way, about the process, the product, and the way that students accessed the material, to inform the ongoing development of the online Masters program (Gravemeijer & Cobb, 2006). The blended learning format, combining interactive online materials and followed by face-to-face follow up, can be seen as providing opportunities for deep and persistent learning, as highlighted by the improved exam results over the three years. However, there were a number of limitations in the pilot, discussed below, which will need to be addressed to give greater rigour to the research, for example, the SCORM recordings of time of access was inaccurate, and the quality of the materials produced was poor – yet the process and findings of the study did give some insights as to how blended learning would work better in the online Masters program.

The main issue for the pilot study, particularly in the initial year, was one of quality of the materials. Blass and Davis (2003) suggest that an online program should be more than just face-to-face materials moved online, the qualities of the materials should be of a high level, the learning experience must be appropriate to the subject, and the materials constructed to achieve the learning objectives.  Similarly, Liaw (2008) found that system quality and quality multimedia would enhance learners’ positive attitudes to e-learning.

The Articulate™ lecture presentations for the pilot were redesigned from their original format; however, despite best efforts the sound quality of the narration was not high; and it was difficult to keep the audio ‘clean’ from external noise. As the students for the Masters programme are globally dispersed, coming from a variety of nationalities (including non-English speaking backgrounds), a lack of clarity in the audio presented a significant problem. However, as a result of the pilot, a way around the issue was explored. It was decided to create scripts for each of the lectures, and utilising the services of a professional voice talent, the materials for the second year were recorded in a studio, creating a better quality product. This adds to the cost of the programme, but does ensure that a high quality is maintained, making the learning materials easier to access for all students. One problem which emerged in Year 2 with the delivery of technical content by a non-technical voice artist was the omission of vocal emphasis by the voice artist on technically important key words or concepts. However, this problem was overcome for the online Masters learning materials by careful and prescriptive scripting, and coaching by the lecturers for correct pronunciation and inflection.

Further quality of materials in the Masters program has been ensured by having all materials developed by industry experts and academics with the help of instructional designers, who ensure that the units progress beyond the simplistic attempts to move face-to-face courses online and develop courses offering an integrated learning experience that provide learning activities to bring about the stated learning objectives (Segrave & Holt, 2003). Gonzalez (2008), building on previous research, makes the distinction between three conceptions of teaching, on a continuum from content-centred to learning-centred, that includes an ‘intermediate’ approach, which draws on both. The Masters program is a professional graduate degree, and by drawing on industry professionals as well as university academics, utilises the design capabilities of instructional designers, ensuring that the programme is both content and learning centred, based in theory and research, to best meet students immediate and future needs (Kasworm & Hemmingsen, 2007).

The online Masters students are different from the undergraduate cohort; they have prior professional involvement, are older and have high motivation to succeed. Yet, as a very mobile workforce, whose lives are more complex, there is a need to recognise they are differentiated learners, who are being “challenged in more intense and disruptive ways related to their life and their work”, and who may need more to support their “psychological, sociological and cultural conditions” (Kasworm & Hemmingsen, 2007, pp. 446-447). In the online Masters programme, the need to ensure student engagement with the materials, and with each other, for successful learning, is paramount, but difficult considering their dispersion. To simulate the role of the face-to-face tutorial, online group discussion forums, tutorials held in a virtual classroom using audio, visual and whiteboard (through Adobe Connect), and chat rooms are being utilised as ways of enabling student participation and interaction with each other and their tutors.

As a result of the lack of access by some of the students in the pilot, the decision was also made to employ tutors to act as course facilitators, whose role is to assist and encourage students to use the various technologies as a tool, to communicate with each other, and to engage with the learning tasks and activities (Segrave & Holt, 2003). Tutors in the program are trained to become ‘intelligent agents’ sensitive to the limitations of the technology, and encouraging synchronous, as well as asynchronous group collaboration amongst students, mobile and geographically dispersed across the globe (Hill & Roldan, 2005). As a result of the pilot, and the obvious lack of access by some students, the tutors in the online Masters program are required to be rigorous and active in the follow up of students’ participation, ‘high touch’ not just in the use of a blend of technologies, but also in the frequency of contact between tutor and student. Follow up can take more time, and again has implications for course costs, and hence implications for the further development of online learning in higher education settings, but may need to be considered if it results in the increased access and improvement in student learning.

Finally, the lack of access by some students in the pilot, despite the improved results of the cohorts overall, has also led to a further restructuring in the online Masters program. It was obvious that a number of students relished the opportunity to rehearse the material and took the extra time available to ensure their understanding of the material covered. The initial structure of the blended learning Masters program was based on the course design of the face-to-face Masters, short intensive units, designed to capture a market that often worked ‘month-on, month-off’ and so had considerable down time at their disposal. However, insights gained from the pilot study showed that this would not suit the blended learning format, that online access takes longer, and so a more traditional trimester model was developed. This would give students and tutors greater opportunities for communication and working through the course materials, whilst also giving students greater flexibility, thereby balancing students’ needs, with the increasing costs associated with intensive personalised learning.

Conclusion
Although the pilot study is complete it did raise a number of other questions for consideration. One of the core problems, which partially prompted the pilot study in this undergraduate unit, was that many students seemed to have failed to grasp the material in their second year, requiring the lecturer to re-teach the material in the third year. To fully evaluate the extent of the learning, some of the questions raised, and which should be examined, is how much of the learning did students retain? Was the learning deep learning, were students able to engage with the materials enough, that they were able to retain the information, and were able to apply that knowledge in the following year? Whilst it can be inferred from the improved exam results that greater understanding of the material was encouraged through the ability to access and revisit materials, it was beyond the study to evaluate the retention of knowledge over time.

As regards the online Masters programme, one area hinted at but not explored in this paper is the cost of running the program. Laurillard (2007) in her paper on comparing the teaching time costs of technology enhanced learning, gives an excellent review on the current literature and various models employed for modelling the cost to higher education institutions. There is no doubt that blended learning methodologies cost more than traditional methods of face-to-face delivery, and it is something to be considered both in the ongoing development of the Masters and for other institutions wishing to offer quality courses in the online environment.

The final task will be to evaluate the course, its offerings, and the quality and effectiveness of the teaching and learning strategies adopted. Blass and Davis (2003) and Oh (2003) amongst many others, raise questions for evaluation which will need to be considered. A question to be asked will be to see whether enough has been done to ensure that the students are in fact fully engaged. What changes will need to be made to the course as a whole, as well as unit content and delivery as the program continues and further learning about the online environment takes place? But perhaps the most important question is one that moves beyond the program and needs to be raised concerning the rapid moves to online learning, which is the question of access. There is an assumption that technology enhanced learning increases levels of productivity and flexibility; that as institutions of higher education make it available that there will be general uptake by students. But as the findings from the pilot project demonstrate, access by students is not guaranteed, not all students will readily move into the online environment, or access materials made available there. It would appear that students take their behaviours from their face-to-face classrooms into the online space (see also Cole, 2009). The underlying assumption that availability equals access, equals learning is challenged by this research, and raises questions for teachers about student engagement, and especially for students in online learning.

Acknowledgements:
The authors would like to acknowledge the support and help of Professor Bob Hurle, who lend his voice and materials to the project, and to Professor Joan Abbott-Chapman who read and gave extensive feedback on earlier drafts of this paper.

References
Akker, J. v. d., Gravemeijer, K., McKenney, S., & Nieveen, N. (2006). Introducing educational design research. In J. v. d. Akker, K.

Gravemeijer, S. McKenney & N. Nieveen (Eds.),Educational Design Research (pp. 3-13). London: Routledge.

Akker, J. v. d., Gravemeijer, K., McKenny, S., & Nieveen, N. (Eds.). (2006). Educational Design Research. London: Routledge.

Altalib, H. (2002). The use of mobile-wireless technology for education (No. Eric Document Reproduction Service No. 475 184): George Mason University o. Document Number)

Blass, E., & Davis, A. (2003). Building on solid foundations: Establishing criteria for e-learning development. Journal of Further and Higher Education, 27(3), 227-245.

Bonk, C., J., & Graham, R. C. (2006a). Preface – About ths book. In C. Bonk, J. & R. C. Graham (Eds.), The handbook of blended learning: Global perspectives, local designs (pp. i-iii). San Francisco: Pfeiffer.

>Bonk, C., J., & Graham, R. C. (Eds.). (2006b). The handbook of blended learning. San Francisco: Pfeiffer.

Brown, L. (2000). Lifelong learning: Ideas and achievements at the threshold of the twenty-first century. Compare, 30(3), 341-351.

Clegg, S., Hudson, A., & Steel, J. (2003). The emperor’s new clothes: Globalisation and e-learning in higher education. British Journal of Sociology of Education, 24(1), 39-53.

Cole, M. (2009). Using Wiki technology to support student engagement: Lessons from the trenches. Computers & Education, 52, 141-146.

Creswell, J. W. (2005). Educational research: planning, conducting, and evaluating  quantitative and qualitative research. Upper Saddle River, N.J.: Pearson/Merrill.

Gonzalez, C. (2008). Conceptions of, and approaches to, teaching online: a study of lecturers teaching postgraduate distance courses. Higher Education (Online First)   Retrieved 1 December, 2008, from http://www.springerlink.com/content/q21u130781713782/fulltext.html

Graham, R. C. (2006). Blended learning systems: Definitions, current trends, and future directions. In C. Bonk, J. & R. C. Graham (Eds.), The handbook of blended learning (pp. 3-19). San Francisco: Pfeiffer.

Gravemeijer, K., & Cobb, P. (2006). Design research from a learning design perspective. In J. v. d. Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational design Research(pp. 17-51). London: Routledge.

Hrastinski, S. (2009). A theory of online learning as online participation. Computers & Education, 52(2009), 78-82.

Kasworm, C., & Hemmingsen, L. (2007). Preparing professionals for lifelong learning: Comparative examination of master’s education programs. Higher Education, 54(3), 449-468.

Laurillard, D. (2007). Modelling benefits-oriented costs for technology enhanced learning. Higher Education, 54(1), 21-39.

Liaw, S.-S. (2008). Investigating students’ perceived satisfaction, behavioural intention, and effectiveness of e-learning: A case study of the Blackboard system. Computers & Education, 51(2), 864-873.

Liaw, S.-S., Huang, H.-M., & Chen, G.-D. (2007). Surveying instructor and learner attitudes toward e-learning. computers & Education, 49(4), 1066-1080.

Morris, L. V., Finnegan, C., & Wu, S.-S. (2005). Tracking student behaviour, persistence, and achievement in online courses. Internet and Higher Education, 8(3), 221-231.

Oh, C. H. (2003). Information communication technology and the new university: A view on eLearning. The Annals of the American Academy, 585, 134-153.

Segrave, S., & Holt, D. (2003). Contemporary learning environments: Designing e-learning for education in the professions. Distance Education, 24(1), 7-24.

Shaeffer, S. (2006, July  5-7). Diversity in Asia and the Pacific: Impact on the internationalisation of higher education. Paper presented at the Australian Universities Quality Forum, Perth.

Urry, J. (2000). Mobile sociology. British Journal of Sociology, 51(1), 185-203.

Walker, D. (2006). Towards productive design studies. In J. v. d. Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational design research (pp. 8-13). London: Routledge.

Young, J. R. (2002). ‘Hybrid’ teaching seeks to end the divide between traditional and online instruction. The Chronicle of Higher Education, 48 (28), 33-34.

© 2008 Karin Oerlemans and Eric F. May