OER creation

Site: MOOC Charles University
Course: How to correctly create and use open educational resources, which focuses on wikisystems.
Book: OER creation
Printed by: Guest user
Date: Saturday, 18 May 2024, 1:14 PM

1. Open educational resources based on Mediawiki software or Wikipedia?

Before starting to create an open educational resource in Mediawiki software (OER MWS) or Wikipedia, it is helpful to define: 

  • The target group that I want to create the OER for

  • The purpose that the resource should serve

Once I’ve defined this, I can decide which resource will be more suitable to create.

I’ll choose to create a Wikipedia page if I don’t have a clearly defined target group. The purpose of Wikipedia is to be an “encyclopedia for all”.

Generally speaking, Wikipedia is suitable as a general introduction into a topic, i.e. as an encyclopedic summary for which it has been designed.

I create OER MWS when I’ve got a specialized topic, especially when this topic’s ambition is to cover a spectrum of knowledge that can be used, for example, during an examination; in this situation, they can be used to a certain degree in place of paper study materials (see the Czech WikiSkripta project by created by medical faculties).

If the designed OER MWS doesn’t have a distinctive purpose and specific group that it is been designed for, there is a danger that it will become a copy of Wikipedia in a degree of varying quality. Rather than creating an entry in your own OER MWS that is similar to Wikipedia, it is better to take part in its creation directly on Wikipedia, as it has a better-functioning community and it is therefore more probable that the article will be cared for and updated. This can be more beneficial especially if the long-term sustainability of the project is not ensured from a financial and personnel perspective.

Inspiration from Wikipedia:

When creating an OER MWS, it is good to take inspiration from Wikipedia. Wikipedia is used far more than OER MWS, and therefore it is useful to become acquainted with their function while creating alternative OER MWS. For example, its rules for dispute resolution and creating / changing content are some of the best ever created for these types of resources. For a basic understanding of this issue, interested parties can use the Five pillars on English-language Wikipedia.

Interconnections: 

Wikipedia (and other Wikimedia Commons projects) can also serve as a suitable complement to creating your own OER WMS and vice versa – see the image below. In creating your own resources, texts from Wikipedia and media from Wikimedia Commons can be used. And, if the OER WMS we are creating are under an open license, it is possible to copy parts from them and paste them into Wikipedia with a link to the original resource (thus also increasing visitor frequency of alternative OER WMS projects at the same time). In creating OER WMS, it is also possible to involve students – see e.g. the Wikipedia education program (Students write Wikipedia).

 

 

Diagram depicting possible relationships between OER WMS, Wikipedia and other sources

 

2. Genre

When creating OER WMS, it is advisable to decide on a genre that you’re planning to create. As was mentioned previously, these resources can to a certain degree represent a new type of genre if we make use of the possibilities they provide. It is possible not to take advantage of these advantages and simply “transform” printed study materials into digital form, but in this case it is not necessary to create this type of resource. OER WMS are suitable for resources which, for example, function like encyclopedias (Wikipedia); in regard to university materials, they can be used for creating various resources based on printed study materials (in the Czech environment, for example, Wikiskripta).

 

3. Content

Content is fundamental when creating OERs. In order for a resource to fully fulfill the purpose it was created for, it is first necessary to ask the following key questions:

For whom is the given resource being created?” or

Who are its users intended to be?” 

These questions, although they might seem trivial, are decisive to the whole subsequent path of the OER. Topics can be processed in order to accommodate the diverse information needs of students ranging from basic schools to doctoral studies. However, it is necessary to select a target group in advance, and therefore it is not easy to create a resource for everyone without causing a detriment to quality in the sense of the usability of the given resource for specific educational needs. In order for a resource to meet the fitness for use definition of quality, the intent of its creator is crucial. 

Fitness for use = assessing the quality of given information is individual and always dependent on the context and purpose that the information fulfills. 

An important area in terms of content is the issue of OER content quality. For more on this, see the chapter on quality.

3.1. Target group

Defining a target group is fundamental for creating OER WMS. A target group determines what will be focused on when creating content. If our target group are students, it is advisable to research their requirements for studying. The more exactly the target group is defined, the more focused the resource can be. For example, the success of Czech WikiSkripta is to a certain degree determined by the fact that it has a precisely defined target group that it creates the resource for – students of medical faculties.

 

3.2. Actors

In terms of the creation of a resource, it is necessary to define its actors and the roles which they hold in the project. These can vary from creators and reviewers to project managers or team leaders and organizational employees. OER WMS offer the potential for a number of other actors as compared to classic printed or digital resources (e.g. IT specialist, vandalism controller, etc.). 

 

3.3. Hierarchy

Within OER WMS, a hierarchy of the individual relationships between actors can be created. It should be clear who is responsible for which element of the system’s operation. For these resources, the most well-established is so-called “flat hierarchy” or the “bottom up” approach, i.e. that these resources function best on the internal motivation of their members and not on being centrally directed from the top down. The principle of subsidiarity is supported, which in this case means dealing with problems during their creation on the lowest possible level (typically by a mistake being corrected by the first editor to notice it). Within OER MWS, we can identify several roles within the hierarchy of a resource (See the table below). 

Table. Description of possible OER WMS actors.

Role

Role description

Initiator, project leader

Initiators are key individuals in each OER WMS project. They are the ones who create innovation in the form of creating these resources; their role is to convince others around them of the advantages of these resources and motivate their colleagues to take part in them.

Ambassador

An actor who spreads awareness and motivates potentially interested parties to become involved in the project. These can be people like teachers, students or Wikipedists (within the Wikipedia student education program).

Work supervisor

Assigns and checks content creation (of specific articles). This can be a teacher leading his/her students to create these systems during lessons.

Beginner – student

A newly engaged participant in the project who needs to be given methodological support. The whole community of wiki systems stands on the continual integration of beginners.

Editor

This can be both a student contributing on a voluntary basis or a paid content editor.

Senior editor

Experienced editors become senior editors, who monitor the creation of content and select articles to be created, expanded or updated.

Administrator

The project administrator is responsible for administering the project from a technical standpoint, blocks potential vandals, and gives rights to individual users.

Senior administrator

The senior administrator is in charge of the financial and legal matters associated with completing the project.

Reviewer

An expert who has taken part in reviewing created materials and thus heightens the quality of content.

Technical support

An IT employee supervising continuous updates to MediaWiki software.

General public

Everyone who wants to get involved in content creation or its modification (e.g. only correcting grammar).

Technology (artificial intelligence, bots, etc.)

Software detecting vandalism and unsuitable words.

3.4. Community

A functioning community is crucial for both creation and primarily ensuring the sustainability of individual OER WMS operation. Without a functioning community, these resources stagnate and, in the case of vandalism that is not detected by the members of the community, they can be damaged. 

A functioning community creates a democratic society in which everyone is equal and ensures the collective review of a given resource. Each member of the community can edit anything. The community also creates an editorial team of the given OER WMS. The editorial team can function in a wholly free manner and anonymously, or is can be partially organized and directed. 

An editorial team is the foundation on which OER WMS stands. The members of the editorial staff are responsible for the administration and maintenance of the resource. Their work can be divided into the following points: 

  1. Creation of new articles; 

  2. Inspection of existing articles for vandalism (i.e. entering vulgar or otherwise unsuitable content); 

  3. Updating existing articles by adding new knowledge.

The way the editorial staff works also has a direct influence on the long-term sustainability of the resource. Sustainability is one of the greatest problems in creating OER WMS. At the beginning of the creation of these projects, there may be a certain idealistic enthusiasm, perhaps further supported by targeted financial support provided to create the project. Over the course of time, however, both enthusiasm and finances in many cases begin to dissipate. In a better case scenario, the resource is updated by occasional visitors, and in worse cases it becomes the subject of attacks by vandals, leading to a deterioration in the page’s quality. A good example of ensuring a functional community is by engaging students in the creation of OER WMS. Nonetheless, the best proven method is voluntary cooperation (an example of best practices to ensure the sustainability of a resource is Wikiskripta, which have the largest and best-functioning community of all Czech OER WMS, which functions on a voluntary basis). Communities of resources can be analyzed using the actor analysis method – for an illustration, see the Table below. 

Table. Results of OER WMS actor analysis at Charles University focusing on individual aspects of quality (Petiška et al., 2020).

Resource name

WikiSkripta

Wikipedia education program (Students write Wikipedia)

Wikisofia

Wikiknihovna

Enviwiki

Guarantor of quality

Quality is guaranteed by the community, not an expert – the 2.0 approach functions on a community principle – the principle of demand.

In some cases, there is no quality control at all. In other cases, teacher assessment takes place, or by volunteers from the ranks of Wikipedists. The fact that it is written by people who are professionally focused (i.e. their field of study) should act as a guarantee of quality. 

The editorial board of Wikisofia (due to capacity reasons, however, it cannot check everything).

Original idea for materials to read and checked mutually by students. This did not materialize.

A quality article is one that is reviewed by an expert. However, there is no “capacity” for articles to be truly reviewed.

Perception of quality

This has developed over time. The general negative evaluation is no longer present; there are more specific objections. 

Students attempt to create texts they consider to be of good quality. However, it functions on the basis of internal motivation, not if it is seen as an obligation.

Quality is seen as an attribute of the writer.

Quality is supported by methodology, which writers are to adhere to. 

The final quality is ensured by the coordinator.

The original idea was for students themselves to ensure quality. The project, however, did not manage to build a functional community.

A high-quality text should be approved by an expert in the given field and should be of good quality also in terms of form (formal requirements such as use of references).

Recognition of quality

For example, indirect characteristics: the text is well-organized, readable, and looks good. If the text is not complete, this points to the footprint of a certain author, indicating one should take caution. 

If control mechanisms work well: a) there should be no uncited information on Wikipedia that is not generally known (quality is ensured by sources); b) low-quality articles are deleted.

A number of articles lack citations and review and purely stem from students’ internal study materials. However, there are also articles here that fulfill the basic parameters of more scientific texts. 

Quality was represented by the fact that the source would be used by students and fulfill their needs.

There is a template (which, in several cases, is placed in the article) that declares the article is of good quality (after it is approved of by an expert), but in practice it is not used.

Reliability of information

Ensured by the community: multiple editors ensure content and form (hyperlinks and references     ); multiple people with particular knowledge are able to recognize an error. 

In an ideal case, this is taken care of by Wikipedia’s rules (primarily concerning references) and protected by the community. In practice, however, the community’s supervision “sometimes fails”. 

In some cases (e.g. psychology), it is monitored by an editorial board. However, the editorial board does not have the capacity to check “everything”.

Influenced by the lack of editors.

The administrator sees it as a “relatively reliable source of information”; also thanks to its connection to Wikipedia via links. 



Openness

OER WMS function best on the principle of broad openness. The greater participation they allow for content creation, the better they can make use of the primary commodity of these resources – the possibility of broad cooperation in the collaborative creation of content. However, openness also has its limitations and problems, such as the destruction of content by internet vandals. The general public should therefore be balanced by a properly functioning community, which corrects potential errors and intentional damage to sources. 

Motivation 

An analysis of actors has showed that, in the case of becoming involved in the community and creating OER WMS, internal motivation functions the best, i.e. when participants become involved on a voluntary basis. On the contrary, any form of coercion, for example the creation of these sources as school obligations, proves to be ineffective. In cases where students are required to be involved in the creation of resources and requested content, they do not remain in the project after their obligation has been completed and thus do not become full-fledged members of the community. 

Awards

Internal motivation to create a resource may be supported and awarded by various appraisals that can be given to members of the community within OER WMS. Awards can be given for the result of activity – e.g. evaluating created content in the form of thanks for editing – or by directly thanking a member of the community for his/her long-term activity (various medals and ranks, which they can then place on their user page in the given OER WMS). 

Citing authorship

In the creation of articles, it is suitable to judge whether and in what manner authorship or review by an expert is cited. If an author of an article is listed and the source can be continually edited, a problem arises regarding the fact that the article may be significantly changed by a different editor without the original author’s knowledge. Thus, readers may be confused, thinking that the article they have been reading was written by the cited author. Therefore, it is worth thinking about how to solve this problem. If a resource is broadly open (i.e. possibility for anyone to make anonymous changes), it may be better not to list anything under the author’s article, or to notify the reader of what exactly the author has created (e.g. via a link to a specific version that the author has created, or the last version that he/she has checked). 

Citing a reviewer

Citing a reviewer of content presents a similar case. If the final content of an article is approved of by an expert in a given field, this can significantly improve its quality and, primarily, its usability, as it makes an anonymous resource (similarly to listing an author) into a non-anonymous and reviewed resource. 

Even here, however, care should be taken in making further updates so as not to list the reviewer under a version that he/she has not reviewed. A good example of how to deal with this problem can be green check marks indicating that the given version of an article has been approved of by an expert.

 

3.5. Content organization

Cards and associated pages 

Page tabs are displayed in the upper section of the pages. These tabs allow users to carry out actions or browse other pages that are related to the current one. Available default actions include browsing, editing, and a discussion about the current page. 

Name spaces 

Name spaces are prefixes that come before the name of a page (e.g. “User:” or “Talk:”), which serve to describe the purpose of the page and make it possible for multiple pages with varying functions to exist under the same name. 

Tagging categories 

Users can create new categories and add pages and files to these categories by adding one or more category tags to the content’s text. Adding these tags creates links in the lower section of the page, which lead readers to a list of all pages in this category, making it easier to browse associated articles. 

Groups and access restriction

MediaWiki offers flexibility in the creation and definition of user groups. It is also possible to set up a group of “automatically confirmed users”, of which a person becomes a member after making a certain number of edits and functioning for a certain number of days. 

Installation

Although installing MediaWiki software is relatively simple, it is advisable to use the services of an IT specialist to create it for the required purposes. The installation guide is available here at this address

 

4. Quality

The issue of quality is a problem for both digital and OER resources. In addition, the quality of OER is one of the most frequently discussed issues in the area of these resources and presents an obstacle in using them. Mainly OER functioning on Mediawiki software are a significant challenge in terms of ensuring quality, as they often allow the involvement of unregistered users in the creation of materials and can thus be created by practically anyone. The low level of the quality of some resources influences the attitude towards them as a whole, therefore causing a certain mistrust in using OER for educational purposes. 

This mistrust is often unjustified. The term Open educational resources is used to denote a relatively diverse combination of information/knowledge repositories on the internet. Neither these resource nor their quality can be discussed per se, as they may vary from personal blogs, in which unknown authors present their opinions, to high-quality materials such as reviewed scientific articles. This situation is further complicated by the fact that there are no standards (criteria) of quality that might be generally acknowledged by the creators or users of OER. Therefore, it is always necessary to assess a specific resource (or storage) or, even better, each material contained within it separately – this applies especially to cases in which there is no official guarantor of quality (e.g. a respected author or institution). Issues of OER quality can be broken up into four areas: 

a. The environment of the given system;

b. Content; 

c. Formal and ethical attributes of quality; 

d. User aspects. 

In terms of the environment of the given system, we can define criteria in the sense of how its operation from a technical standpoint is ensured (e.g. how quality is ensured and indicated; who can edit the given resource; how resource sustainability and updating material is ensured; whether a quality guarantor, i.e. an author or reviewer, can be listed – see the table below). It should be noted that a resource may fulfill the criteria for the quality of the technical environment, but it does not necessarily have to contain high-quality texts for study. OER quality primarily means quality of content, but this question presents a significant problem. Answering it in the sense of “is the given resource of good quality?” requires a content analysis, which is demanding both in terms of time and in finding relevant experts who could carry out the analysis in, for example, the form of a peer-review process. 

During such an analysis, various aspects of quality may be considered, such as: 

  • text complexity – in the sense of including various aspects and points of view in order to provide a balanced opinion (i.e. is it sufficiently complex?), 

  • processing (has it been properly processed stylistically, grammatically and visually?), and 

  • credibility (verifiability of information via respected sources). 

Some of these aspects are subjective, not only in the sense of an expert’s subjective opinion, but also in regard to the point of view of a student using the resource (e.g. a student in his/her first year of university bachelor’s study has different complexity requirements than a doctoral student). 

Table. Levels of OER content quality evaluation, which characterizes social and technological aspects (in Clements et al., 2015, see also inserted references). 

Approach to quality evaluation

References

Peer review system/quality evaluation by users, usually using the Likert scale (1-5)

Atenas and Havemann, 2014, Larsen and Vincent-Lancrin, 2005, Schuwer et al., 2010, Windle et al., 2010, Minguillón et al., 2010, Stacey, 2007, Lefoe et al., 2009, Catteau et al., 2008, Li, 2010, Krauss and Ally, 2005, Sanz-Rodriguez et al., 2010, Sampson and Zervas, 2013, Currier et al., 2004

Zervas et al., 2014, Liddy et al., 2002, Waaijers and van der Graaf, 2011, Venturi and Bessis, 2006, Zhang et al., 2004



Tools for quality evaluation by users (e.g. LORI)

Atenas and Havemann, 2014, Clements and Pawlowski, 2012, Downes, 2007, Richter and Ehlers, 2010, Atkins et al., 2007, Sinclair et al., 2013, Vargo et al., 2003, Defude and Farhat, 2005, Kumar et al., 2005, Alharbi et al., 2011

Recommendation tools (best sources)

Manouselis et al., 2013, Atenas and Havemann, 2014, Pegler, 2012, Petrides et al. (2008), Adomavicius and Tuzhilin, June 2005, Duffin and Muramatsu, 2008, Manouselis and Sampson, 2004, Manouselis et al., 2011, Li, 2010, Sanz-Rodriguez et al., 2010, Sabitha et al., 2012, Sampson and Zervas, 2013, Zervas et al., 2014

Commenting

Minguillón et al., 2010, Catteau et al., 2008, Li, 2010, Vargo et al., 2003, Sanz-Rodriguez et al., 2010, Sampson and Zervas, 2013, Waaijers and van der Graaf, 2011

Favorite

Minguillón et al., 2010, Sanz-Rodriguez et al., 2010, Sampson and Zervas, 2013, Zervas et al., 2014

Social tagging

Minguillón et al., 2010, Stacey, 2007, Sampson and Zervas, 2013

Tagging (reporting broken links, unsuitable content, etc.)

Sinclair et al., 2013, Clements and Pawlowski, 2012

However, quality assessment may be made easier by quality criteria that can be used to evaluate the given resources. We will deal with these criteria in the following subchapters. 

 

4.1. Perception of quality

Open educational resources, which function on the basis of MediaWiki software, often face biases in regard to their quality. In this respect, Wikipedia is the most frequently discussed, as it faces significant mistrust in the academic environment. This criticism is often justifiable in light of the fact that OER WMS frequently lack a guarantor for the text as a whole, as is the case with classic textbooks, study materials or scientific publications. On the other hand, it is possible to ensure quality using correctly established processes. The user of these resources should also be sufficiently acquainted with the processes used to ensure the quality of the given resources and the potential risks he/she can encounter. Generally speaking, this type places greater demands on the user in terms of recognizing and evaluating quality. At the same time, the reader becomes the arbiter of this quality and, in the event of insufficiencies, can also edit the resource and thus help increase its quality. 

 

4.2. Guarantor of quality

In the OER WMS environment, quality tends to be guaranteed by a community, not an expert as in the case of classic printed or electronic resources. When materials are created e.g. for university purposes, however, community reviewing can be accompanied by expert assessment. In the form of a simple template, OER WMS make it possible to attribute both an author and reviewer to each piece of content. In these cases, however, it is necessary to consider the fact that a reviewed text can change significantly over the course of time and a reviewer or author can thus be subsequently listed under a distinctly different version than the one he/she approved. This can be dealt with, for example, by listing the version that the expert last assessed and via a notification that the existing version has not yet been checked. 

 

4.3. Recognizing quality

Recognizing the quality of OER MWS content is a complex activity, during which multiple factors should be considered. In addition to traditional aspects such as the authorship of the publisher, it is also possible to assess quality in terms of topicality (via displaying history – see Figure) or structure and readability of individual materials. 

Some creators also use an algorithm in order to evaluate the user-friendliness of the text in the sense of readability (e.g. in the case of WikiSkripta the so-called Mistrík Readability Scale is used). For resources based on MediaWiki software, references and thorough citation of individual passages play a significant role. The emphasis on citing resources should be greater than in the case of classic printed or electronic resources for which a guarantor of the final product’s quality has been determined. In the case of freely-editable tertiary resources, thorough citation of primary and secondary sources helps improve the quality of the resource. Thus, publications which contain this information are also a certain guarantee of quality. 

 

4.4. Reliability of information

The reliability of information is guaranteed by the community of the resource via citing respected sources of information for the given field. The hierarchy of sources differs according to field, but generally in university education there is a visible trend towards the use of scientific publications such as peer-reviewed articles in respected journals. Where it is possible, the use of primary sources heightens the quality of teaching and helps disseminate scientific knowledge from the academic environment among students and thus teaches them the basics of scientific work and methods. However, scientific knowledge is quickly changing, and therefore it is suitable to frequently update resources to include the most recent knowledge. Young professionals from the field can help in this if they engage in the creation of these resources during their studies and continue doing so in their subsequent professions. A good example of this is also WikiSkripta, in which this role is fulfilled by doctors preparing for their medical license examinations who have current knowledge in their fields. 

 

4.5. Topicality of information

Ensuring the topicality of information is the primary commodity that MediaWiki resources provide in comparison to classic publication schemes; they also represent a significant challenge to ensure the sustainability of the resource. If information is not regularly updated, the wiki environment ceases to work as it should and the content of the resource declines. This also applies to URL addresses which the resource cites and which are important for verifying the information. If these addresses are not regularly updated, many links become broken over time. 

 

4.6. Students’ perception of quality

For students, the accessibility of information is becoming the primary criterion for the selection of study materials. Thanks to their structure, OER WMS are often preferred by search engines and appear in the first search results; students thus also prefer them. Their well-arranged structure suits them, as they are accustomed to it primarily from Wikipedia, which they regularly use, similarly to being redirected by hyperlinks to associated topics. However, their perception of quality and other use of these resources is also strongly influenced by those around them, i.e. whether these resources are recommended to them by teachers or older classmates who found them useful in fulfilling their study obligations. 

 

4.7. Processes of ensuring quality

OER WMS offer a wide range of options and processes for ensuring quality. This ranges from partial or complete restrictions to edit individual sections to software to detect unsuitable content such as plagiarism and classic mechanisms to ensure quality such as peer-review processes or citation of authors. In order to ensure the quality of OER WMS, various criteria can be used (see the Table below). 

Table. Possible quality criteria for OER, which function on MediaWiki software for university purposes and their description.

Description of criteria

License tagging

Licensing a resource under an open license is the basic condition for it to be labelled and used as OER. If it is licensed in this manner but not labelled, those who don’t know about the license cannot use it as such. Therefore, tagging the license for all materials that are under an open license (or within the whole resource) is important for dissemination. 

Date of creation

Tagging the date of the resource’s creation is a key sign of quality for its use or its citation in other resources. 

Last update

Just like tagging the date of creation, the tagging of the last update is an important criterion for quality. 

Page version and edit history

If a source is to be cited in a scientific work, it is better to cite the ID version of the page than the time it was accessed. The source may be updated very quickly, especially in the case of current events. Two changes can take place on Wikipedia within one minute. 

Authorship

The author’s reputation is an important criterion of quality both in printed and online resources. Authorship labelling is a factor that can be used to differentiate various OER projects based on MediaWiki software from Wikipedia. 

Review

A peer-review process is one of the primary criteria of quality in the academic environment. Peer-reviewed publications by relevant experts are more respected by those that are not reviewed. Therefore, it is ideal when OER for university purposes allow for a peer-review process and its tagging (e.g. stating the article was reviewed by a relevant expert). Peer-review of OER can be done both by an editorial team in the form of internal community reviewing or by addressing external experts. 

Tagging of quality by creators 

Do creators in some way label the quality of specific materials? E.g. in the form of recommendations by experts in the given field? 

Sustainability

This criterion answers whether the project functions at present or whether it is only online but not being maintained. Updating OER to include the newest information is the primary advantage of these resources over traditional materials. If materials have been made within a project that are no longer being edited or updated with the newest knowledge, it can be assumed that the quality of these materials will decrease. 

Balance

This criterion shows whether the quality of individual sections of texts significantly varies, e.g. whether the resource contains erroneous or outdated sections (but not incomplete ones waiting to be processed) next to properly created ones. 

Quality assessment by users 

This criterion denotes the option for users to assess the quality of materials. It also provides the creators with feedback on whether the resources they are creating are seen by users to be of good quality. This also helps continually analyze the satisfaction of the target group. 

4.8. Risk factors

Many problems arise in regard to ensuring OER WMS quality. The primary problem is ensuring the long-term sustainability of the resource, primarily from a personnel perspective, i.e. via properly functioning communities. To do so, it is necessary to compensate for the departure of some staff members by accepting new ones. Factors that can discourage newly arriving members include any type of closed resource (i.e. no opportunity to contribute to it) and being forced to take part in its creation (e.g. in the form of school responsibilities); it is shown that communities of these resources function better on voluntary cooperation and possibilities of broad, open collaboration so that interested external parties can also become engaged in them.

4.9. Quality criteria for university use of OER

Use of OER can have a negative effect on resulting knowledge among students (if good-quality resources are not recommended to them or they are not led to recognize them). Research carried out in the USA has pointed to the fact that although students function in the digital environment, they have problems recognizing high-quality information. For example, more than 80% of them believe that advertising messages are official news reports. They also have problems with evaluating messages on Twitter and other messages (Wineburg et al., 2016). This can be influenced by the amount of freely available information of varying quality and insufficient competencies in assessing it. Results of research among Czech students of environmental fields of study have shown that a significant number of them is not able to define criteria of quality; those who did list criteria most often gave answers that could be categorized as “Using citations”, “Credibility (verifiability)” or “Well arranged” (see the figure below).  

Figure. Graph depicting individual categories of quality criteria, which students adhered to for judging OER quality. Number of respondents: 233 (Petiška, 2018a). 

Yaari et al. (2011) researched which attributes of quality are relevant for Wikipedia users and found that they often cited things like the amount of information, satisfaction with content and external links. On the contrary, attributes like the number of edits and number of unique editors were contradicting – “both a small number of edits / editors and a large number of edits / editors were listed as attributes of high-quality articles.” Wikipedia (in English) has a sophisticatedly designed system of quality assessment in terms of users, who evaluate individual articles both in terms of various aspects of quality (Wikipedia Contributors, 2019b) and in terms of their fulfillment within the individual thematic categories that the article belongs to. On English-language Wikipedia, articles are then evaluated according to the quality of content as it has been evaluated by users (Figure below). 

Figure. Evaluation of articles on English-language Wikipedia according to quality – articles marked FA (featured articles) are seen as being of the highest quality, whereas “stubs” are of the lowest (Wikipedia Contributors, 2019b)

OER quality is one of the greatest barriers in applying these resources to study curricula. Quality evaluation in the open environment of the internet has its own specific qualities as compared to classic study texts and traditional genres. Dlouhá et al., 2015 suggest creating codes that would help evaluate OER in the form of indicators. Nonetheless, establishing quality criteria for OER is problematic in light of the fact that there is no agreement in scientific literature in quality criteria for online resources or a definition of information quality (Alkhattabi, Neagu & Cullen, 2010). This is due to the fact that the assessment of quality of given information is individual and always dependent on the context and purpose this information fulfills (i.e. fitness for use). Therefore, OER assessment and quality criteria are dependent on the target group and specific purpose (e.g. a specific master’s thesis or term paper) the resource serves. 

Primarily, there is a difference between the quality criteria that can be used for the given OER as an overall system (i.e. technical environment and method of operation) and criteria that evaluate content. A resource (in the sense of storage) that possesses marks of quality in a technical sense can also contain many poor-quality sources of information (e.g. poorly written articles). We will deal with the system environment of these resources and, by using them, provide specific recommendations for their creation and use.

To a certain degree, a significant number of approaches in the OER quality evaluation makes use of the community of their users, which in a certain sense makes them into the co-creators of these resources (Clements et al., 2015). For digital educational resources supported by public budgets, copyright requirements (i.e. placement under an open license) are defined as one of the primary criteria of quality (National Institute for Education, 2016). An important sign of the quality of resources that can be used for university education is the possibility of peer review (i.e. the checking of a given article by an expert) and whether this review is labelled; other important signs are tagging the date of creation and the updating of the resource (Dlouhá et al., 2015). 

The so-called peer-review process can be defined as a basic aspect of content quality evaluation for academic use. This process also creates a basis for assessing the quality of scientific work. It is generally defined as the evaluation of given work by experts in the field. In the environment of scientific work, expertise is quite well-defined, e.g. based on the authorship of scientific publications, in which both the quality of the journals in which these works are published and their perception by the scientific community (e.g. frequency of citation) are assessed. Based upon this, it is fairly easy to recognize expertise in the given field. With OER, the situation is different: contrary to scientific publications, which are highly specialized, these resources can be seen rather as textbooks. They are tertiary resources, and the criterion of quality is thus the way in which secondary sources are used. 

OER often deal with complex and interdisciplinary topics. As their complexity grows, it becomes harder to find an expert that is able to evaluate it as a whole. This applies especially to resources that are intended for the general public (i.e. Wikipedia type); in addition, it is difficult to determine whom and what educational purposes the given resource is intended to serve. The same article that can be sufficient for basic or secondary school students may not be the same for a university student. If OER use a system of quality evaluation, they therefore often rely on users, whom they see to a certain degree as “peers”, i.e. equal to one another (i.e. author and assessor). By doing so, they strengthen the element of their openness. However, users are not experts in the true sense of the word, and therefore this is not a classic peer review process that could be compared to scientific publications. It is rather the evaluation of the majority of users, which does not tell of the quality of the given resource in general, but only on how those who have assessed it perceive the resource’s quality. OER use secondary sources that should guarantee content quality. Thus, this concerns avoiding their use outside the original context or their misinterpretation.

LITERATURE:

 

5. Listing the date and ID of creation and edits

When creating OER MWS, time information in individual parts of the text is crucial. This allows for identification of individual edits and page versions, and also the creators of specific sections of the text or the authors of edits. 

OER WMS allow for time data to be displayed on the creation of the text, its edits and changes in the “View history” tab, which appears at the top right on the page above the text (if not programmed otherwise). Using this tab, it is possible to open individual editing history, compare various versions, or find out who added a specific part of the text and when. However, information on the last edit can often be misleading, as OER MWS do not function like classic text and the date of the most recent edit does not necessarily mean that the text went through its last revision at the same time. The editor, for example, may have only corrected a simply typo. Therefore, it is better to refer parties that are interested in the time information of individual edits to the complete editing history.

When listing time data, OER MWS offer the option of listing a specific version (ID – the identification number of a specific page). For other potential work with a specific text (a page on OER MWS) when citing it, for example, it is suitable to list not only the time information, but also the identification number (ID) of the given page. This is because more than two edits can be made during one minute within OER MWS. The page ID thus serves as a better identifier of a specific version than merely the time of its last edit. A page citation on English-language Wikipedia can thus look like this:

 “Wikipedia Contributors. (2020). Wikipedia. Wikipedia. ID of the cited page version: 934425180. https://en.wikipedia.org/wiki/Wikipedia”.

 

6. Using hypertext

Content that is mediated via OER on the basis of Mediawiki software can sometimes also be understood as a specific genre of text. The resulting text is dynamic and variable; in this context, there is the option of third-dimension (3D) reading, as – aside from offering classic linear text reading – it also allows the user to “click through” or “skip” to associated topics. This is made possible primarily thanks to the hypertext environment, hyperlinks, and redirection to other associated pages. 

Although hypertext technology was created with the development of computers in the second half of the 20th century, the idea of redirecting a text or other relevant parts of it is older. This innovation was created by Ephraim Chambers, who published his two-volume Cyclopædia or “universal dictionary” of arts and sciences in 1728. This Cyclopædia, which contains a wide range of processed topics, was an innovation in the placement of cross-references to other sections of articles. French encyclopedia makers also took inspiration from this. Encyclopedia editor-in-chief Denis Diderot saw the ideal encyclopedia as an index of entries; the aim was not only to describe individual topics, but also the relationship between them (via cross referencing). Encyclopedia authors thus strove for readers to understand contexts, which today is called interdisciplinary or transdisciplinary thinking. Modern technologies like OER MWS have innovated the idea of cross referencing into the form of hypertext. 

The next aspect that is linked to the creation of OER based upon Mediawiki software and makes it to a certain degree a “new genre” is the collaborative process of text creation and the ease of citing statements (via a hyperlink, the cited text can be easily accessed). The collaborative process of creating articles exceeds the classic editing process of texts in its extent and significance. This is linked to the form of articles, which are often not merely brief definitions of the problem as is common with classic encyclopedias. Some are often highly extensive and are more reminiscent in their content and detailed processing of a scientific publication or an introduction in scientific articles rather than an encyclopedia article. The introduction in many entries is often similar to the abstract of a scientific article, which summarizes the whole topic of the article. 

With regard to the fact that the final text does not have a final “guarantor of quality”, an editor or senior editor, it remains to a certain degree in its creation phase. This may often result in a certain unorganized or even chaotic nature of the text, which manifests itself, for example, in the incongruity of individual sentences within the overall context, which makes reading it more difficult. On the other hand, the fact that there is no guarantor of the final text’s quality heightens pressure on the quality of the individual sections. This is primarily evident in Wikipedia, where long disputes are commonly led over things like a single statement or its verifiability by respected sources. There is proof that articles seen by the Wikipedia community as being of the highest quality are also ones that are the best cited (see the table and figure below). 

Table. The median number of edits, editors, words, references and ratio of number of words to number of references to environmental articles on English-language Wikipedia illustrates that the highest-quality articles (according to the Wikipedia community’s evaluation) are significantly different from those of lower quality in all these regards. Number of analyzed articles: 7,048 (Petiška et al., in the process of publication).

Quality category

Number of articles

Number of edits carried out in the article (median)

Number of article editors (median)

Number of words in the article (median)

Number of references listed in the article (median)

Ratio of number of words to references in the article (median)

Category of the highest quality (Best Article Class + Good Article Class

94

513

148

4317

120

33

Category of average quality (B + or C class)

1594

295

123

2124

48

41

Category of lowest quality (Start + Stub Class)



5360

51

26

357

7

47



Figure. The ratio of number of words to number of references in environmental articles on English-language Wikipedia. Blue is used for articles perceived by the Wikipedia community as being of the highest quality; red is for articles perceived as being of average quality; and green is for articles perceived by the community as being of the lowest quality. The vertical axis shows the number of references; the horizontal axis shows the number of words. Number of analyzed articles: 7,048 (Petiška et al., in the process of publication).

 

These articles often do not give the impression of a cohesive text, but rather a list of individual statements. Therefore, we can often understand Wikipedia as a search engine of important knowledge and sources on various topics rather than an encyclopedia with rigidly set rules that guarantee an internally coherent text. 

Hypertext plays a significant role in creating OER MWS. Just like other digital resources, hypertext is one of the primary commodities of OER MWS as compared to classic study resources such as printed study materials. If hypertext is used in the creation of text, it can have a strongly positive effect. We can define three primary advantages that hypertext offers in the creation and use of OER:

  1. Interdisciplinary thinking. With hypertext, it is easy to redirect the reader to associated topics, leading them towards a holistic way of thinking in context. The reader is not studying only the given material, but is presented with the broader context and chains of events. 

  2. Inclusive approach. Hypertext helps to create a certain democratization of study material. It is possible to give more complex texts to students who are less educated in the given issue. If they don’t understand a certain expression, they can click on it and learn about it. On the contrary, readers who are more educated do not have to be slowed down by useless explanations of what they already know. In regard to the fact that today’s students come from varying cultural and social environments, studying from OER WMS, which makes adequate use of hypertext, can help them unify their knowledge to the required level. Hypertext can also serve as a kind of glossary of unknown terms. 

  1. Verifiability of statements. Thanks to hypertext, statements in an article can be easily labelled via the source listed next to it. With one click, anyone can verify whether a statement is truly contained in a given source and potentially read it in the relevant context. 

7. Citations

The OER WMS user environment makes it possible to cite both individual parts of a text and list literature under a created text. In creating OER WMS, it is suitable to consider using thorough citations of individual statements, primarily in cases that involve resources that allow for large-scale anonymous and collaborative editing. 

Even more important than who adds content may be how a portion of the content is cited (and by what quality of the source). The possibility to easily verify a statement in the form of adding a reference is another important benefit of these resources. Thorough citation is called for primarily when university-level materials are involved, as anonymous sources in which uncited information prevails should not be recommended to students. The following arguments for careful content citation when creating resources for university purposes are listed in the table below:

Table. Description of reasons for citing most statements with respected sources in anonymous resources such as Wikipedia (in Petiška, Moldan, 2019).

Reasons

Description of reasons

Problem of verifiability

We do not know the author of a specific statement and therefore we have no reason to believe him/her. If we can verify a statement from a respected source, we can trust it.

Problem of shared knowledge

a) It is impossible to define shared knowledge in sources used by students of various fields and various cultural environments; b) Students should primarily use those sources that have a minimum of shared knowledge and a maximum of new information.

Guidepost and summary of literature

Anonymous sources should be recommended at a university level only as a guidepost to other resources and for summaries of literature.

Context of information

Information without context is less valuable. Students should learn not only to describe reality (the cloudless sky is blue), but also to explain it (the cloudless sky is blue because molecules in the air disperse blue light from the sun more than they do red light). The source can help them understand the context.

Critical thinking

Students should understand that information without a source is less credible than information from a respected source.

The problem of plagiarism

Uncited statements can be acquired from other sources without acknowledgment. This kind of use is unethical and illegal.

The problem of citogenesis 

The problem of circular reporting – an uncited statement in an anonymous source is used in another source and then reused to cite the original statement in an anonymous source.

Bad practices

Students who use uncited information from anonymous sources can learn bad practices in other parts of the learning process, e.g.: a) non-critical reception of information; b) plagiarism (they can learn to use information without references in their university theses). 

8. Media attachment

OER WMS make it possible to embed diverse media attachments into the text, such as images, videos and audio files. When embedding these files, however, one should remember the licenses under which the content is licensed and whether these licenses are compatible with the license of the resource that we are creating (see the chapter on Public licenses).

9. Public licenses

Public licenses (or public copyright licenses) are a type of license that allows for a more open approach to content than is common among traditional licenses. Mainly, they authorize any user to use the work without having to sign an independent license contract with the author. Public licenses are used when the author wants to give others the right to share, use and otherwise utilize the work he/she has created. Conditions of use are specified within the selected license. Although there are several types of standardized public licenses, by far the most popular are Creative Commons licenses, which are also the most widely-used type of license for open educational resources

Table. Summary of the basic types of the most widely used Creative Commons public licenses

A Creative Commons (CC) license is one of several public licenses dealing with copyright, which allows for the free distribution of a work that would otherwise be protected by copyright. The spectrum of various CC types (see the Table below) gives authors various options (e.g. they can decide to allow only non-commercial use of the work) and rids those using or distributing an author’s work of the fear of violating copyrights as long as they adhere to the conditions that are stated in the license under which the author has allowed his/her work to be used. There are several types of Creative Commons licenses. The licenses differ according to various conditions that allow use. The newest is the 4.0 version. All licenses are based on several basic factors (see the table below). 

Table. Description of basic elements of Creative Commons licenses.

Image

Rights

Description

Attribution

Attribution (BY)

Holders of the license can copy, distribute, depict and use a work and create derivative works and remixes based on them only if they cite the author or provider of the license in the way they have specified.

Share-alike

Attribution-ShareAlike (SA)

The holders of the license can distribute derivative works only based on a license that is consistent (“not more restrictive”) with the license under which the original work is licensed. Without this measure, derivatives (i.e. created work based upon the original work) can be sublicensed with compatible but more restrictive license clauses, e.g. CC BY to CC BY-NC, i.e. editing an original work that can also be used for commercial purposes to a work that can be shared but cannot be used for commercial purposes.

Non-commercial

Non-commercial (NC)

Holders of the license can copy, distribute, depict and use the work and create derivative works and remixes based upon it only for non-commercial purposes.

Non-derivative

No derivs (ND)

Holders of the license can copy, distribute, depict and work only with the original copy of the work, not with any derivatives and remixes that have been created according to it.