Quality Criteria for Designing Learning Objects for English Language Teachers*
Ana Bertha Tiscareño Ramí­rez1 & Marí­a Soledad Ramí­rez Montoya2
Colegio de Postgraduados: Campus Córdoba, Tecnológico de Monterrey
Contact:  tiscareno@colpos.mx, solramirez@itesm.mx
* This is a refereed article
This is an open-access article distributed under the terms of a CC BY-NC-SA 4.0 license
Abstract: The objective of this research was to investigate the quality criteria to be met to design a Learning Object (LO); it was intended to contribute to the professional development of English language teachers. The study was of an exploratory orientation, and the LO was evaluated and validated by thirteen experts: five in pedagogy, two in computer technology, one in media evaluation, three in graphic design and two English teacher trainers. Furthermore, English language teachers were acknowledged as potential users and eight professionals were asked to use and assess the LO. The data was gathered using a questionnaire that was auto-administered (SurveyMonkey) using a Likert type scale. The findings, which incorporate aspects of technology, pedagogy and design or format, revealed the diverse aspects that determined the quality of the LO.

Keywords: Learning Object, quality criteria, professional development


Resumen: El objetivo de este trabajo fue investigar los criterios de calidad que se deben satisfacer para diseñar un Objeto de Aprendizaje (OA), el OA fue diseñado con el propósito de contribuir a la formación del docente del inglés. La investigación fue de tipo exploratorio con validación de trece expertos en cinco campos o disciplinas: cinco en pedagogí­a, dos en tecnologí­a, una en evaluación de medios, tres en diseño gráfico y dos en formación de maestros(as) del inglés. Además, se identificaron a los docentes del inglés como posibles usuarios del OA por lo que se les pidió a ocho profesionales que usaran y evaluaran el OA. Los datos fueron recopilados a través de un cuestionario, éste fue auto-administrado (SurveyMonkey), usando una escala tipo Likert. El análisis de la información, la cual incorpora diversos aspectos de tecnologí­a, pedagogí­a y diseño, revelo los diversos aspectos que determinaron la calidad del OA.


Introduction

Currently, development in different domains has led to a type of society that is characterized by a number of facts: endless innovations of Information and Communication Technologies (ICT); emphasis on producing and using knowledge to generate human development; and new social contexts. These contexts include different ways of communicating and interacting, the inclusion of other means of education (blended and online learning), and the appreciation of values such as diversity. These changes have created new settings and conditions that constitute what is known as the ‘knowledge society’. The UNESCO Report, Towards Knowledge Societies, states that the concept of knowledge society not only refers to technological development but also “encompasses much broader social, ethical and political dimensions” (Bindé, 2005, p. 17).

Naturally, development, along with Internet and mass media driven social dynamics or interactions, especially among young people, have created a school environment that generates and requires a series of competencies and innovative methodologies from teachers. Among the approaches being implemented are Learning Objects (LOs); Wiley (2000), one of the specialists in the topic, defines LO as “any digital resource that can be reused to support learning” (p. 6). A LO is a self-contained item that is designed from a pedagogical approach. The contents usually include a diagnostic test, learning objectives, learning activities, evaluation mechanisms, and feedback. It is a digital entity that is accessible with any electronic device that has an Internet connection.

As digital resources, learning objects constitute a feasible or alternative mode for teacher training or development; this is due to the fact that they can accommodate to different modalities: self-instruction, blended learning, e-learning or online instruction. It is necessary to emphasize that LOs assist e-learning; Cabero (2006) mentions that one of the distinctive features of online instruction is the support provided by learning objects. This type of instruction relates to the third phase of Computer Assisted Language Learning (CALL), which Warschauer (1996) identifies as the integrative phase. He mentions that some of the characteristics in the integrative phase are that resources are linked and learners can navigate their own path simply by pointing and clicking a mouse. The LO includes links to other learning objects, so users have the possibility of doing additional or supplementary activities which in turn permit them to be more in control of their learning.

Components of a Learning Object

Diverse authors (Ossandon and Castillo, 2005; Ramírez, 2006; Santacruz, Aedo, and Delgado, 2004) acknowledge that there are two main components of a LO: the technological and the pedagogical parts. To promote the learning process, the pedagogical aspect of an object needs to be sustained by learning principle theories. Wiley (2000) explains that “instructional design theory, or instructional strategies and criteria for their application must play a large role in the application of learning objects if they are to succeed in facilitating learning” (p. 9). Moreover, Ramirez (2006) agrees with Santacruz et al. (2004) when they point out that a LO should include learning objectives, relevant content, and an evaluation mechanism. Concerning the technological part, Wiley (2000) proposes that the object should be accessible, usable and reusable. Figures 1 and 2 illustrate components of the learning object designed for improving the pedagogical knowledge of L2 English teachers in Mexico.

Figure 1. General information of the LO

Figure 2. Evaluation

Since LOs are available online and their applications are diverse, learners can access them any number of times and they have become a popular means of instruction. A LO should provide a meaningful learning experience and the web provides a wealth of materials. Thus, some questions to deeper explore objects are: What are the characteristics that distinguish LOs? and How do we differentiate a LO from other digital material? It is basic to identify them from two points of view: as users or as designers of this type of instructional materials.

In view that the trend to create and use LOs is increasing, and in order to understand what determines the quality of learning objects as a means of teacher training or professional development, a LO was designed by Tiscareño (2010). The object was intended as a medium of reference for English language teachers. It is important to point out that, as in this particular case, learning objects can become Open Educational Resources (OER), which are “teaching, learning and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use or re-purposing by others” (Atkins, Brown, and Hammond, 2007, p.4). Furthermore, learning objects can be linked to other open educational resources such as web pages.

Research Context

This research study was part of a macro study called “Evaluar para mejorar: Sistema de evaluación educativa para escuelas de bajo logro académico”1. In the third and last phase of the project, the objective was to train public elementary school teachers using a set of learning objects. Valenzuela and Ramirez (2010) indicate that the teacher is an essential factor for the academic performance of the students; these authors suggest that the teacher is responsible for promoting quality in education. Furthermore, skilled teachers address relevant contents and facilitate the learning process. Regarding this, Richards (2001, p. 209) points out “it is teachers themselves who ultimately determine the success of a program.” Additionally, the idea is that teachers not only develop competences to deal with the educational problems of today, but also be capable of working on problems which teachers and students might face in the future

In Mexico, the mechanisms or strategies of professional development for teachers that have been most frequently used are: distance education courses and subject oriented workshops (Aguerrondo, 2003). However, Aguerrondo suggests that these schemes of training have not been entirely successful due to issues such as budget, institutional policies, and teachers’ professional interests. Specifically, the professional development of English teacher has been a continuous concern in Mexico. Richards (2009) mentions that the Ministry of Education of Mexico (La Secretaria de Educación Pública) acknowledged the need of raising professional standards; therefore, the Ministry of Education provided training courses for 150,000 teachers working in the public sector. The courses contributed to teachers’ professional development and were aimed to the attainment of new qualifications.

As a means of training, it is important to highlight that LOs can reach a wide number of professionals; moreover, they are a suitable option of professional development for teachers who live in geographically remote places. Wiley (2002) points out that contrary to traditional instructional media which can only exist in one place at a time, LOs can be accessed from any place by any number of people. LOs break with barriers of time and distance. Furthermore, these objects cater to teachers’ needs for training. Richards (2003) categorizes the teachers as the following: untrained, trained, novice and experienced. The skills of these groups of teachers often reflect their level of training. LOs can be used and reused in diverse learning settings and modalities and favor a personalized approach.

This type of instructional technology appears to have numerous advantages. Wiley (2000) refers to LOs as instructional components that can be broken into smaller parts or combined with other digital resources to facilitate learning. Furthermore, Santacruz, Aedo and Delgado (2004) highlight that LOs are reusable which decrease the time and cost of content assembly. These authors also state that objects are delivered with a high level of individualization; this allows the curricular personalization according to interest and individual needs. However, to thrive and facilitate the learning process, Wiley (2002) claims that as any instructional technology, learning objects “must participate in a principled partnership with instructional design theory if they are to succeed in facilitating learning” (p. 29).

Research Question and Design

With this background, the research question that guided this study was aimed at defining the quality criteria of LOs for language teacher training (or even for L2 instruction) purposes. Therefore, the question is:

What are the most important quality criteria for designing an effective learning object for language teacher training?

This question will provide the designers of LOs a framework of aspects to keep in mind as they construct their own learning objects.

To find a reliable answer to this question and with the support of a professional in pedagogy and specialists in computer programming, a learning object was designed and produced:

(http://www.ruv.itesm.mx/convenio/tabasco/oas/ming/homedoc.htm). The object addresses some issues about teaching English and can be used as part of a course, a ‘diplomado’ or other self-training for in-service English teachers. However, a main feature of LOs is their flexibility and reusability, so even though this was the main target audience, others can use the LO, or even just take certain pieces of the LO, and adapt them for their own purposes.

The contents of the LO describe a few principles of the communicative approach such as learning to communicate for a variety of purposes and functions. Furthermore, aspects of how to vary the use of language in relation to the context and the role were also addressed. Richards and Rodgers (2001) propose that to be effective an approach should seek to engage students through its content materials; thus, it was referred that learning situations should be meaningful and provide students with the opportunity to communicate, to get the message across and to negotiate meaning. These are strategies that align with the main purpose of in-service language teaching training.

Methodology

The study was of an exploratory orientation and the object was evaluated and validated by thirteen experts in different disciplines: five in pedagogy, two in technology, one in media evaluation, three in graphic design and two teacher trainers in English as a Foreign Language. These professionals interacted with the LO, answered a questionnaire in their area, and validated the criteria in their respective domain or area of expertise. Furthermore, eight English language teachers were asked to use it and then each of them answered an electronic questionnaire that included thirty nine questions (see Appendix 1). The teachers, five female and three male, are professionals who have worked at different school levels: elementary, secondary, high school and university in different states in Mexico. The majority of them teach at the university level in different places that include both private and public systems. Their ages as well as their levels of experience as language teachers are diverse. For instance, the youngest teacher has taught for four years; the range of teaching experience for six of the professionals is from fourteen to twenty years. The most experienced has taught for over twenty-five years. In total there were 21 (n=21) participants (13 experts and 8 teachers) for this research.

The instrument used to gather the information was an electronic questionnaire (self-administered). The format used a multiple choice four-point Likert scale (completely agree, agree, disagree, and totally disagree) using SurveyMonkey. There were common questions for experts and teachers; the idea was to analyze the same indicator from different approaches and perspectives. However, there were also very specific and unique questions for each group of participants. These questions were oriented to their area of specialty. For instance a question for the experts in technology was: Do the LO comply with international standards? Open questions were included at the end of the questionnaire and they were the means to obtain additional comments, perceptions and suggestions about the learning object. It is necessary to point out that in the open questions some participants did not have any additional comments. To quantify the responses, each option of response in the Likert scale was given a numerical value. The scale has four options. Completely agree and agree reveal a positive evaluation towards the items; these were given 4 and 3 numerical values respectively. The options that represent a negative assessment, disagree and totally disagree, were assigned 2 and 1 correspondingly. The categories of the research study and the indicators used to evaluate the LO are represented in Table 1.

Table 1. Categories

Results

Pedagogical Issues

The results suggest that the relevance of the LOs was validated by both groups of experts and English teachers. All the participants in the study endorsed the opinion that it is important to be competent in the English language in today’s context. These findings agree with Richards and Renandya (2002), who assert that “English is the language of the globalization, the international communication, the trade and business, and the media among others” (p. 2).

The scope of the LO is greatly determined by the contents included and the strategies that favor the learning. The participants validated that there are elements of how training competences can be developed. However, not all the professionals agreed that the contents facilitate the development of these skills.

The analysis of the pedagogical aspect revealed that while the teachers completely agreed that the learning objectives were clearly defined and appropriate for the didactic purposes, the experts only agreed with the clearness of the definition because two of the eight experts considered that these were not appropriate for the didactic purposes. It is important to point out that the objectives of the LO were aimed at reviewing and reflecting on some aspects that are involved in the teaching process. Other issues encompassed in the LO were diverse factors that influence second language learning: age, motivation, previous language experience and learning styles among others (Oxford, 2002). Furthermore, the LO refers that when learning another language, students apply cognitive, metacognitive, and social strategies. For instance, in the cognitive domain, learners comprehend, use and produce language; the metacognitive aspect refers to the control students have over the learning process; and the social strategies promote language learning through the interaction with others.

The LO was designed with the aim of considering the previously mentioned factors and strategies; thus, teachers can reflect on the factors and promote the implementation of the strategies to develop linguistic competences.On the other hand, the assessment of the evaluation techniques raised different opinions. The evaluation mechanisms were considered appropriate. In this regard, 53% of the experts completely agreed and 47% agreed that the tests explore previous knowledge of the users, and also reveal acquired knowledge through the interaction with the LO. Additionally, each test included a feedback section where correct answers and short explanations were provided to reinforce knowledge and to clarify any misinterpretation of the content.

The indicator of motivation was perceived as following: 1) seven of the participants completely agreed that the object was highly motivating; 2) three of the experts classified it as motivating; 3) two of the participants disagreed with this. The results are presented in Table 2. Moreover, the participants agreed that the multimedia elements in the LO generate motivation. Cedillo, Peralta, Reyes, Romero and Toledo (2010) found that to the use of resources supported by the Information and Communication Technologies generates interest and motivates both teachers and students.

Table 2. Pedagogical indicators

Technological Issues

Concerning the technological aspect, the usability of the LO was evaluated on navigation arrangements, information layout, clearness, precision and pertinence of the contents. The participants validated that the object is usable. This property of the LO propitiates that the time invested in the learning experience translates in interaction with the contents and not in dealing with technological aspects. When the user can navigate the object easily (usability), little time is wasted due to technical inconveniences or arrangement of contents.

Regarding accessibility, the correlation of the answers, where the mean was 3.7, indicated that the learning object is easy to access. Furthermore, 60% of the participants completely agreed that the design of the controls to manipulate the LO was appropriate and the rest agreed with the statement. Likewise, it is considered that the LO can run on mobile devices; moreover, it includes multimedia elements and the educational resources do not take up too much bandwidth. That means that the LO can be easily accessed from different locations, with diverse devices and by several users simultaneously. These characteristics permit a convenient or personalized use of the LO. The analyzed information is presented in Figure 3.

Figure 3. Accessibility

Table 3 presents the results of reusability. In this particular case, the experts validated that the LO can be used in different learning settings (mean 3.6 mean); furthermore, 80% of the evaluators agreed that it can be understood by teachers with different levels of training. In order to assure interoperability, it was necessary that the LO fulfilled international standards and the two experts in technology agreed that it complies with these (mean 3.0). Furthermore, they also acknowledged that the metadata are visible (3.5). However, one of the two experts disagreed with the fact that the incorporated metadata are sufficient. The reusability indicators were evaluated within the positive range.

Table 3. Reusability

Graphics and Discourse

Regarding graphics and discourse, the participants labeled the arrangement of the elements as pertinent. The font and typography were also endorsed by the research population (mean 3.4). The different aspects of semantics were validated as well. The information gathered from open questions came up with some suggestions. For instance, concerning the contents, the teachers advised that activities should be devised to foster meaningful learning and knowledge construction. Additionally, the formats of the evaluations should be diverse and these ought to propitiate the application of the knowledge and not only information retrieval.

Conclusions

The Learning Object presented an alternative of training and it was not necessary to attend a face-to-face lesson. It is believed that the object provided, for some teachers, an innovative experience of professional development. As a result the teacher-users identified what learning objects mean and the potential of their use. It is hypothesized that now that the participant-teachers acknowledge LOs, they will search in Learning Management Systems or in the repositories of Open Educational Resources to adopt, adapt, use and reuse these didactic materials.

It can be concluded that to meet the criteria for design, the contents of the LO ought to be pertinent and relevant for the subject matter. Furthermore, Wiley (2000) suggests that when using digital materials for learning purposes, the pedagogical aspect is often left out, thus it is necessary that: 1) LOs are designed including clear and attainable learning objectives; 2) they generate motivation through the use of multimedia elements and intellectual challenges; 3) the LO tests should be meaningful and be presented to in formats that promote knowledge transfer and not only the memorization of contents.

Concerning the technological aspect, the LOs should be easy to access. They should be user friendly to navigate and reusable in different learning contexts. They should also not take too much bandwidth so that they can run smoothly. For the layout and discourse, the LOs should have an attractive format and the written discourse should be clear.

Learning takes place in different contexts and modalities. Schools and teachers are increasingly using virtual platforms or learning management systems to improve education. Consequently it is important to identify the aspects that influence learning on an online environment. LOs are one of the bases of online instruction thus to identify the quality criteria permits to design relevant objects.

 

References

Aguerrondo, I. (2003). Formación docente: Desafíos de la política educativa.    Cuadernos de Discusión SEP. Retrieved February 17, 2011 from: http://ses2.sep.gob.mx/dg/dgespe/cuader/cuad8/cud_08.pdf

Atkins, D., Brown, J., & Hammond, A. (2007). A Review of Open Educational Resources (OER) movement: Achievements, challenges and new opportunities. Retrieved   October 1, 2011 from: http://www.hewlett.org/uploads/files/Hewlett_OER_report.pdf

Bindé, Jérôme. (2005). Towards Knowledge Societies:UNESCO World Report. Retrieved October 28, 2011, from: http://www.unesco.org/new/en/communication-and-information/resources/publications-and-communication-materials/publications/full-list/towards-knowledge-societies-unesco-world-report/

Cabero, J. (2006). Bases pedagógicas del e-learning. Revista de Universidad y Sociedad del Conocimiento RUSC, 3 (1), 1-10. Retrieved September 22, 2011 from:http://www.uoc.edu/rusc/3/1/dt/esp/cabero.html

Cedillo, M., Peralta, M., Reyes, P., Romero, D. & Toledo, M. (2010). Aplicación de recursos educativos abiertos (REAS) en cinco prácticas con niños mexicanos de 6 a 12 años de edad. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 8 (1), 119-15. Retrieved February, 23, 2011, from: http://www.ruv.itesm.mx/convenio/catedra/recursos/material/re_06.pdf

Ossandon, Y. & Castillo, P. (2006). Propuesta para el diseño de objetos de aprendizaje. Revista de la Facultad de Ingeniería, Universidad Tarapacá, 14 (1), 36-48. Retrieved January 6, 2012 from: http://www.scielo.cl/scielo.php?script=sci_pdf&pid=S0718-13372006000100005&lng=es&nrm=iso&tlng=es

Oxford, R. (2002). Language learning strategies in a nutshell: Update and ESL suggestions. In Richards, J. & Renandya, W. (Eds.) Methodology in Language Teaching: An Anthology of Current Practice (pp. 124-132). New York: Cambridge University Press.

Ramírez, M. S. (2006). El objeto del objeto de aprendizaje: Experiencia de Colaboración Institucional y Multidisciplinar [objeto de aprendizaje]. Cátedra de investigación de innovación en tecnología y educación. Retrieved January 8, 2012 from: URL:http://www.ruv.itesm.mx/cursos/maestria/proyectos/oa/homedoc.htm

Richards, J. C. (2001). Curriculum Development in Language Teaching. Cambridge:          Cambridge University Press.

Richards, J. C. (2003) Program factors in effective foreign and second language teaching. Recovered, May 11, 2012 from: http://www.professorjackrichards.com/pdfs/program-factors-effective-teaching.pdf

 Richards, J. (2009). The changing face of TESOL. Jack Richards’ Plenary Address at the 2009 TESOL Convention in Denver USA. Retrieved September 7, 2011 from: http://www.professorjackrichards.com/articles/

 Richards, J. & Renandya, W. (Eds.) (2002). Methodology in Language Teaching: An Anthology of Current Practice. Cambridge: Cambridge University Press.

Richards, J. & Rodgers, T. (2001). Approaches and Methods in Language Teaching. Cambridge: Cambridge University Press.

Santacruz-Valencia, L., Aedo, I., & Delgado, C. (2004). Objetos de Aprendizaje: Tendencia en la Web Semántica. Retrieved January 5, 2012 from: http://www.rediris.es/difusion/publicaciones/boletin/66-67/ponencia18.pdf .

Tiscareño, A. B.(2010). Competencia para Manejar el Idioma Inglés [objeto de aprendizaje]. disponible en el sitio Web: http://www.ruv.itesm.mx/convenio/tabasco/oas/ming/homedoc.htm. Disponible en el repositorio abierto de la cátedra de investigación de innovación en tecnología y educación del Tecnológico de Monterrey en: http://catedra.ruv.itesm.mx//handle/987654321/101.

Valenzuela, J. R. & Ramírez, M. S. (2010). Trans–formando a los profesores: desarrollo de competencias para una sociedad basada en conocimiento mediante objetos de aprendizaje abiertos. Ponencia presentada en el XI Encuentro Internacional Virtual Educa,Santo Domingo, República Dominicana.

Warschauer M. (1996). Computer Assisted Language Learning: An Introduction.In Fotos S. (Ed.) Multimedia Language Teaching. Tokyo: Logos International.

Wiley, D. (2000). Connecting learning objects to instructional design theory: a definition, a metaphor, and a taxonomy. The Instructional Use of Learning Objects: Online Version 2000. retrieved January 18, 2011 from http://www.reusability.org/read/.

 


1 This research was sponsored by El Fondo Mixto CONACYT-Tabasco, 2009-2011.


Contact us

mextesoljournal@gmail.com
We Are Social On

Log In »
MEXTESOL A.C.

MEXTESOL Journal, vol 37, núm. 1, 2013, es una Publicación cuadrimestral editada por la Asociación Mexicana de Maestros de Inglés, MEXTESOL, A.C., Versalles 15, Int. 301, Col. Juárez, Delegación Cuauhtémoc, C.P. 06600 Mexico, D.F., Mexico, Tel. (55) 55 66 87 49, journal@mextesol.org.mx. Editor responsable: M. Martha Lengeling. Reserva de Derechos al uso Exclusivo No. 04-2015-092112295900-203, ISSN: 2395-9908, ambos otorgados por el Instituto Nacional de Derecho del Autor. Responsable de la última actualización de este número: Asociación Mexicana de Maestros de Inglés, MEXTESOL, A.C. JoAnn Miller, Versalles 15, Int. 301, Col. Juárez, Delegación Cuauhtémoc, C.P. 06600 Mexico, D.F., Mexico. Fecha de última modificación: 31/08/2015. Las opiniones expresadas por los autores no necesariamente reflejan la postura del editor de la publicación. Se autoriza la reproducción total o parcial de los textos aquí­ publicados siempre y cuando se cite la fuente completa y la dirección electrónica de la publicación.

License

MEXTESOL Journal applies the Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license to everything we publish.