Previous
Next

4. Quality

4.9. Quality criteria for university use of OER

Use of OER can have a negative effect on resulting knowledge among students (if good-quality resources are not recommended to them or they are not led to recognize them). Research carried out in the USA has pointed to the fact that although students function in the digital environment, they have problems recognizing high-quality information. For example, more than 80% of them believe that advertising messages are official news reports. They also have problems with evaluating messages on Twitter and other messages (Wineburg et al., 2016). This can be influenced by the amount of freely available information of varying quality and insufficient competencies in assessing it. Results of research among Czech students of environmental fields of study have shown that a significant number of them is not able to define criteria of quality; those who did list criteria most often gave answers that could be categorized as “Using citations”, “Credibility (verifiability)” or “Well arranged” (see the figure below).  

Figure. Graph depicting individual categories of quality criteria, which students adhered to for judging OER quality. Number of respondents: 233 (Petiška, 2018a). 

Yaari et al. (2011) researched which attributes of quality are relevant for Wikipedia users and found that they often cited things like the amount of information, satisfaction with content and external links. On the contrary, attributes like the number of edits and number of unique editors were contradicting – “both a small number of edits / editors and a large number of edits / editors were listed as attributes of high-quality articles.” Wikipedia (in English) has a sophisticatedly designed system of quality assessment in terms of users, who evaluate individual articles both in terms of various aspects of quality (Wikipedia Contributors, 2019b) and in terms of their fulfillment within the individual thematic categories that the article belongs to. On English-language Wikipedia, articles are then evaluated according to the quality of content as it has been evaluated by users (Figure below). 

Figure. Evaluation of articles on English-language Wikipedia according to quality – articles marked FA (featured articles) are seen as being of the highest quality, whereas “stubs” are of the lowest (Wikipedia Contributors, 2019b)

OER quality is one of the greatest barriers in applying these resources to study curricula. Quality evaluation in the open environment of the internet has its own specific qualities as compared to classic study texts and traditional genres. Dlouhá et al., 2015 suggest creating codes that would help evaluate OER in the form of indicators. Nonetheless, establishing quality criteria for OER is problematic in light of the fact that there is no agreement in scientific literature in quality criteria for online resources or a definition of information quality (Alkhattabi, Neagu & Cullen, 2010). This is due to the fact that the assessment of quality of given information is individual and always dependent on the context and purpose this information fulfills (i.e. fitness for use). Therefore, OER assessment and quality criteria are dependent on the target group and specific purpose (e.g. a specific master’s thesis or term paper) the resource serves. 

Primarily, there is a difference between the quality criteria that can be used for the given OER as an overall system (i.e. technical environment and method of operation) and criteria that evaluate content. A resource (in the sense of storage) that possesses marks of quality in a technical sense can also contain many poor-quality sources of information (e.g. poorly written articles). We will deal with the system environment of these resources and, by using them, provide specific recommendations for their creation and use.

To a certain degree, a significant number of approaches in the OER quality evaluation makes use of the community of their users, which in a certain sense makes them into the co-creators of these resources (Clements et al., 2015). For digital educational resources supported by public budgets, copyright requirements (i.e. placement under an open license) are defined as one of the primary criteria of quality (National Institute for Education, 2016). An important sign of the quality of resources that can be used for university education is the possibility of peer review (i.e. the checking of a given article by an expert) and whether this review is labelled; other important signs are tagging the date of creation and the updating of the resource (Dlouhá et al., 2015). 

The so-called peer-review process can be defined as a basic aspect of content quality evaluation for academic use. This process also creates a basis for assessing the quality of scientific work. It is generally defined as the evaluation of given work by experts in the field. In the environment of scientific work, expertise is quite well-defined, e.g. based on the authorship of scientific publications, in which both the quality of the journals in which these works are published and their perception by the scientific community (e.g. frequency of citation) are assessed. Based upon this, it is fairly easy to recognize expertise in the given field. With OER, the situation is different: contrary to scientific publications, which are highly specialized, these resources can be seen rather as textbooks. They are tertiary resources, and the criterion of quality is thus the way in which secondary sources are used. 

OER often deal with complex and interdisciplinary topics. As their complexity grows, it becomes harder to find an expert that is able to evaluate it as a whole. This applies especially to resources that are intended for the general public (i.e. Wikipedia type); in addition, it is difficult to determine whom and what educational purposes the given resource is intended to serve. The same article that can be sufficient for basic or secondary school students may not be the same for a university student. If OER use a system of quality evaluation, they therefore often rely on users, whom they see to a certain degree as “peers”, i.e. equal to one another (i.e. author and assessor). By doing so, they strengthen the element of their openness. However, users are not experts in the true sense of the word, and therefore this is not a classic peer review process that could be compared to scientific publications. It is rather the evaluation of the majority of users, which does not tell of the quality of the given resource in general, but only on how those who have assessed it perceive the resource’s quality. OER use secondary sources that should guarantee content quality. Thus, this concerns avoiding their use outside the original context or their misinterpretation.

LITERATURE:

 

Previous
Next