OntoFox: web-based support for ontology reuse
2010

Zuoshuang Xiang
Mélanie Courtot
Ryan R Brinkman
Alan Ruttenberg
Yongqun He
OntoFox: web-based support for ontology reuse

BMC Research Notes
31752010Open Access TECHNICAL NOTE Technical Note
Background: Ontology development is a rapidly growing area of research, especially in the life sciences domain. To promote collaboration and interoperability between different projects, the OBO Foundry principles require that these ontologies be open and non-redundant, avoiding duplication of terms through the re-use of existing resources. As current options to do so present various difficulties, a new approach, MIREOT, allows specifying import of single terms. Initial implementations allow for controlled import of selected annotations and certain classes of related terms.Findings:OntoFox http://ontofox.hegroup.org/ is a web-based system that allows users to input terms, fetch selected properties, annotations, and certain classes of related terms from the source ontologies and save the results using the RDF/XML serialization of the Web Ontology Language (OWL). Compared to an initial implementation of MIREOT, OntoFox allows additional and more easily configurable options for selecting and rewriting annotation properties, and for inclusion of all or a computed subset of terms between low and top level terms. Additional methods for including related classes include a SPARQL-based ontology term retrieval algorithm that extracts terms related to a given set of signature terms and an option to extract the hierarchy rooted at a specified ontology term. OntoFox's output can be directly imported into a developer's ontology. OntoFox currently supports term retrieval from a selection of 15 ontologies accessible via SPARQL endpoints and allows users to extend this by specifying additional endpoints. An OntoFox application in the development of the Vaccine Ontology (VO) is demonstrated.Conclusions: OntoFox provides a timely publicly available service, providing different options for users to collect terms from external ontologies, making them available for reuse by import into client OWL ontologies.

Background

Biomedical ontologies are sets of terms and relations that represent entities in the scientific world and how they relate to each other. Terms are associated with documentation and definitions, which are, ideally, expressed in formal logic in order to support automated reasoning [1][2][3]. Ontologies have dramatically changed how biomedical research is conducted. For example, since the Gene Ontology (GO) was first published in 2000 [1], it has been used and cited in more than 2000 peer-reviewed journal articles [4]. Ontologies have been used in various applications, such as gene expression data analysis [1], literature mining [5], and as the underpinning of a semantic web [6]. There are currently more than 150 biomedical ontologies and 700,000 entities in the NCBO BioPortal http:// bioportal.bioontology.org/. With new resources continu-ously being developed, maximizing ontology sharing and interoperability has become a growing concern [7,8].

The development of a new biomedical ontology covering a specific domain is often an ambitious, time-consuming project, usually requiring extensive crosscommunity collaboration. The OBO Foundry is an open community that has established a set of principles for ontology development with the goal of creating a suite of interoperable reference ontologies in the biomedical domain [3]. These principles require that member ontologies be open, orthogonal, expressed in a common shared syntax, and designed to possess a common space of identifiers. One way of meeting the goal of interoperability is to reuse existing resources by importing them into the tobe-created ontology. For example, the Vaccine Ontology (VO, http://www.violinet.org/vaccineontology) [9] relies on many terms (e.g., administering substance in vivo) already described by other biomedical ontologies, such as the Ontology for Biomedical Investigations (OBI; http:// purl.obolibrary.org/obo/obi).

OWL currently only provides a mechanism to import ontologies as a whole [10]. This approach is reasonable and recommended for small ontologies that are designed in ways consistent with the importing ontology. However, in many cases this is neither practical nor needed. For example, the source ontology may be too large for editing tools, use different design patterns, or be at an early stage of development. Nevertheless, individual terms in such ontologies may be well-defined and therefore desirable to reuse. As an example, the Chemical Entities of Biological Interest ontology (ChEBI; http://www.ebi.ac.uk/chebi/) currently includes over 455,000 terms. Importing CHEBI as a whole into a target ontology is impractical given current editing (e.g., Protégé ontology editor [11]) and reasoning tools (e.g., Pellet [12] and Fact ++ [13]). Protégé can perhaps handle in the low 10,000s of terms before becoming too slow to use, and with the addition of complex logical restrictions, reasoning performance is noninteractive with the resources used.

As a practical alternative to importing whole ontologies, MIREOT (Minimum Information to Reference an External Ontology Term) was developed in the context of the OBI project [14]. MIREOT proposes selective use of classes from external ontologies that are of direct interest to a target ontology, instead of importing external ontologies as a whole. For example, both the OBI and the VO require the ontology term 'homo sapiens', and have decided to use the NCBI Taxonomy Ontology (NCBI-Taxon) as a common resource for naming taxonomic groups. The corresponding URI for 'homo sapiens' is http://purl.org/obo/owl/NCBITaxon#NCBITaxon_9606. MIREOT specifies that the minimal information needed to specify reuse of this term is (i) this URI, (ii) the URI of the parent term in the importing ontology (http://purl. obolibrary.org/obo/OBI_0100026, organism), and (iii) the ontology IRI of the source ontology. Based on this minimal information, an automated process can be used to retrieve (and periodically refresh) chosen additional information such as the preferred label for the term and elements of the taxonomic hierarchy. MIREOT is being used in a number of ontology projects, for example, OBI, VO, the Influenza Ontology (InfluenzO; http://sourceforge.net/projects/influenzo/), Neural ElectroMagnetic Ontologies (NEMO; http://nemo.nic.uoregon.edu/wiki/ NEMO), ontologies developed in the Neuroscience Information Framework (NIF; https://confluence.crbs.ucsd. edu/display/NIF/), and as part of the eagle-i project https: //www.eagle-i.org/home/.

While editing tools commonly provide means to reference an external term by directly setting its URI, one must also manually enter auxiliary information necessary for practical editing, such as the label and definition, and update such information if the source ontology changes. In addition, it is often desirable to import additional related terms. For example, when the Vaccine Ontology imports a species term, the inclusion of some of its superclasses allows for queries at different taxonomic ranks (e.g., kingdom, phylum, and species). To address these issues, an initial implementation based on MIREOT was created to facilitate managing the tedious aspects of this process automatically http://obi-ontology.org/page/MIR-EOT.

The developers of the MIREOT guideline recognize that such an approach is a balanced compromise. Importing only selected information means that incomplete or incorrect inferences could conceivably be made. Technical approaches such as module extraction [12,[15][16][17] promise to preserve correct inference, under a variety of assumptions, by computationally selecting portions of an ontology. Recent work on modularization casts it as a process that fragments existing ontologies into a set of smaller and possibly interconnected parts or modules [12,[15][16][17][18] that can then be reused as units of ontology [19].

There have been several approaches to computing modules [20]. Structural approaches use the syntax of the axioms of ontologies and mostly only consider the induced is-a hierarchy [17,21]. Logic-based approaches take into account the consequences of ontologies and require that this extracted module captures the meaning of the imported terms used, i.e., includes all axioms relevant to the meaning of these terms. However, Grau et al. [22] proved that it is undecidable, even for description logics simpler than OWL-DL, to determine whether a subset of an ontology is a minimal logic-based module.

These approaches are relatively new, experience using them is limited, and our experience with current Webbased implementations has found them to be unreliable. Moreover the methods do not provide ways to avoid import of certain terms or axioms that might not be considered desirable, or have other issues that prevent their easy use [23]. Nonetheless the syntactic locality approach these methods use is applicable to single-term import and so is compatible with the MIREOT approach.

The OBI project has an implementation of the MIR-EOT mechanism that demonstrates the feasibility of the approach. It is, however, command line-based and requires the specification of terms either by commandline scripts or construction of an ontology document. Specification of which ancillary information should be incorporated is by writing SPARQL queries [24], restricting its adoption by less technically able users. To facilitate application of the MIREOT guideline by the wider ontology community a more user-friendly system facilitating the import and update of external terms into a target ontology is desired. In addition, while MIREOT provides a practical yet simple approach to specifying external ontology terms, the OBI implementation does not pro-vide the ability to consider restrictions on imported terms that a user may desire to import. To preserve the meaning of the imported terms, ontology developers might like to use ontology module extraction, e.g., extraction of the target class and its transitively related (via restriction) closure [17]. Ontology developers may also want the flexibility of including no superclasses, only one direct superclass, all superclasses to the top class, or a subset of all superclasses for a term, in order to provide additional relevant domain terms for their users.

To address these needs for ontology reuse, we have developed OntoFox http://ontofox.hegroup.org/, a webbased application implementing the MIREOT and related ontology term extraction strategies. OntoFox facilitates ontology development by automatically fetching properties, annotations, and related terms from external ontology terms and saving the results as OWL serialized as RDF/XML [25] suitable for use with the OWL import directive. OntoFox provides a web-based package of solutions for ontology developers to extract, for subsequent import, different sets of ontology terms by following and expanding the initial MIREOT implementation and by developing related ontology term extraction methods based on SPARQL [26]. In this manuscript, we will introduce the general OntoFox web system, describe the ways it lets users describe which properties and related terms should be imported, and demonstrate how OntoFox is used in the VO development.

Methods

OntoFox system architecture

OntoFox uses a simple text format and web forms for data input in a user-friendly implementation, and is designed to not require any programming skills. OntoFox is implemented using a three-tier system architecture. At the front-end, data can be submitted using either web forms or by uploading a plain text input file. The input data are then processed using PHP and Java., and SPARQL (middle-tier, application server) queries are then executed against an RDF triple store (back-end, database server), currently the Neurocommons SPARQL endpoint [6]. The web server then processes the result of each SPARQL query sent by the back-end server; as a result an RDF/ XML file is created and offered for download to the user ( Figure 1).

As OntoFox is a web-based system, it is accessible everywhere through the Internet without need for additional software installation. The techniques used in the OntoFox web application were chosen for maximum compatibility by using established W3C standards, specifically, OWL as a web ontology language, RDF/XML as its serialization, and SPARQL for queries.

OntoFox three-tier structure implementation (1) OntoFox web interface

The OntoFox web interface is designed based on iterative testing, thus far informal usability testing and feedback from initial users, following a spiral software development model [27]. It accepts the input from the user, via either web forms or uploading of a local text file, and presents the output data after query processing.

Finding and entering the URIs for desired terms can be tedious. To speed up the term specification process, we implemented an ontology term suggestion feature based on auto-completion of the string of text entered by users after selecting the desired source ontology. The OntoFox server offers a list of potential matches, and upon selection, the associated term ID will show up in an input box next to the label. Additionally, the "Detail" hyperlink next to the term ID provides easy access to an interactive ontology browser allowing visual confirmation of the term definition and its position in the hierarchical ontology tree structure. Lastly, as shown in Figure 2, the user can click "Add" next to "Detail" to insert the full URL of the selected term into the input textbox on the web interface.

(2) Data processing by the web application

The OntoFox application server runs on a Dell Power-Edge 2580 server running the Red Hat Linux operating system (Red Hat Enterprise Linux 5 server). PHP and Java are used as programming languages in the web application server. General web-based programming and query submission are written using PHP. The OWL API http:// owlapi.sourceforge.net/, a Java API for manipulating OWL files, is used in OntoFox to read, process, and rewrite OWL files and save the final results as one OWL file after merging individual query results.

(3) Data storage and access

The OntoFox internal RDF database server runs on a separate Dell PowerEdge 2580 server. The database server is powered by the OpenLink Virtuoso database engine http://www.openlinksw.com/virtuoso/. While VO is loaded within this Virtuoso server, OntoFox also uses RDF data stored in other web accessible servers, for example, the Neurocommons knowledge management platform [6]. Fifteen biomedical ontologies generally used within the OBO community are available for users to select as source ontologies within OntoFox (Table 1). These ontologies, initially chosen to support VO development, were selected based on their specificity, community support, and maturity. They all adhere to a strict deprecation policy, ensuring that the meaning of each term remains stable until the term is deprecated. Though nothing bars serving more resources, these 15 ontologies were all that were required to cover all information needed for import via MIREOT during the VO develop-ment. Users can choose to provide another source ontology URI and corresponding SPARQL endpoint, allowing retrieval of terms outside of the OntoFox source ontology repository resources; however this is done at their own risk as term stability is not guaranteed.

Evaluation of OntoFox SPARQL retrieval of related terms

To compare the performance of the OntoFox SPARQL related term retrieval approach with the OWLAPI modularization, three sets of signature data were used. The first two sets of signature data include either one term (e.g., Figure 1 OntoFox workflow. The input data is parsed internally by the OntoFox web server. SPARQL queries are then constructed and used to query remote RDF triple stores, containing the RDF triples of source ontologies. After successful query execution, an OWL output file is generated and provided to the user for download. the OBI term 'antigen') or a list of OBI terms that were imported to VO. The third set of signature terms for modularization includes all terms in the Neuroscience Information Framework (NIF) Lexicon ontology (nif.owl; http://ontology.neuinfo.org/NIF/nif.owl). The nif.owl file uses approximately 30 external files. The OntoFox method and the OWLAPI modularization method were separately performed and compared. For the OWLAPI modularization, the OWLAPI SyntacticLocalityModule-Extractor with STAR module type was used.

Results

MIREOT implementation

The MIREOT guideline suggests the following minimal set: (1) source term URI, (2) target direct superclass URI, and (3) source ontology URI [14]. These are the first parameters taken as input by OntoFox:

1) Source ontology URI

Box 1 of the OntoFox web input system includes a list of the 15 ontologies a user can select as source ontology (Figure 2 and [Additional file 1). Alternatively, a user can request an unlisted source ontology in Box 2, in which case the URL of a SPARQL endpoint where this new source ontology can be accessed must be provided. For each external ontology term, OntoFox adds an imported-From annotation property http://purl.obolibrary.org/ obo/IAO_0000412, which indicates the URI of the source ontology.

(2) Low level source term URI

This parameter is equivalent to the source term URI in the MIREOT guideline. Box 3 allows users to input one or multiple source term URIs, entering one URI per line. For example: http://purl.org/obo/owl/NCBITaxon#NCBITax on_9606 #Homo sapiens

(3) Target direct superclass URI

This is the URI of the direct superclass of the top-level source term chosen above (i.e., where to position the newly imported term(s) in the target ontology). This parameter is entered alongside the top-level source term URI in Box 4 using the directive "subClassOf" (see more detail below).

These three data items together unambiguously define a single term from the source ontology and where to position (i.e., what class is it a subclass of ) it in the target ontology.

Annotation properties management

OntoFox provides several settings/directives allowing users to select which annotation properties to retrieve, and more importantly, under which format those should be returned.

(1) Source term annotation URIs

By default (i.e., if no annotation URI is specified), Onto-Fox will not fetch any of the annotation properties of the selected term. A user can choose to retrieve specific annotation properties by specifying their URIs, or use the OntoFox command 'includeAllAxioms' to fetch all annotations properties associated with source ontology terms. This parameter is entered in Box 6 in the web input format ( Figure 2).

(2) "copyTo"

This directive is used to map an ontology term annotation to a new annotation property created in the target ontology, resulting in a duplication of the annotation property value in the output file. It is used at the beginning of a line, followed by an annotation URI used in target ontology. For example, the "copyTo" command is used in Figure 3A http://www.w3.org/2000/01/rdf-schema#label copyTo http://purl.obolibrary.org/obo/IAO_0000111# preferred term This duplicates the value of the rdfs:label property into the "preferred_term" annotation (IAO_0000111 from IAO [28]), and both annotations are included in the output file. This directive can be used in the web form (Box 6) or in the OntoFox input text file.

(3) "mapTo"

This directive allows mapping of an ontology term annotation: it will replace an existing annotation property in the target ontology with the value of another annotation property from the source ontology. It is used at the beginning of a line, followed by an annotation URI from the target ontology. For example, Figure 3 (together with [Additional file 2]) contains an example of using the "mapTo" directive:

http://www.geneontology.org/formats/oboInOwl#has-Definition mapTo http://purl.obolibrary.org/obo/IAO_0000115# definition annotation property As ontologies don't always use a common set of annotation properties, this feature provides an easy way to integrate information from a source ontology into a target ontology while retaining a consistent, metadata style. For example, the OBO2OWL script http://www.berkeleybop.org/obo-conv.cgi, used to automatically generate OWL version of OBO ontologies within the OBO Foundry, uses the property "hasDefinition" to relate a term to an instance whose rdfs:label is that term's definition. However VO uses the IAO metadata scheme http:// code.google.com/p/information-artifact-ontology/wiki/ OntologyMetadata, and directly relates the term to its definition via the http://purl.obolibrary.org/obo/IAO _0000115, definition annotation property. The mapTo directive instructs OntoFox to map the definition used in the source to the value of the VO annotation property for definition. This mapping directive is used in Box 6 of the web form input method or in OntoFox input text format.

Managing incorporation of related terms

OntoFox provides a number of mechanisms for selecting related terms for import, all based on structural approaches and that have been used within VO development. Methods are provided for selective retrieval of parent terms, transitive retrieval of restrictions inspired by structural-based modularization techniques, and the extraction of a subtree rooted at a given term. In this section we detail these mechanisms.

The setting "Top level source term URI", is designed to work in conjunction with another term specification when retrieving parent terms between lower and upper level source terms. A typical use is when importing some or all of the superclasses of a species term to allow for queries at different taxonomic ranks (e.g., kingdom, phylum, and species). For example, in between 'homo sapiens' and Eukaryota (our chosen top-level term) in the NCBI Taxonomy, there are 27 intermediate terms (cf Figure 2). It would be very tedious to find, copy and then paste all those 29 terms into the new ontology. By specifying 'homo sapiens' as the low level source term, Eukaryota as the upper level source term, and the setting "includeAllIntermediates", the 27 intermediate terms are automatically retrieved by OntoFox (Figure 2 and [Additional file 1]).

In addition to this retrieval of all parent terms, OntoFox uses an algorithm to compute and retrieve intermediate source terms that are the closest ancestors of more than one low-level source terms, and to remove intermediate terms that have only one parent term and one child term (Figure 4), leaving only terms that present alternatives for query. This setting, "includeComputedIntermediates", provides an option to reduce the number of extracted ontology terms by getting less intermediate ontology terms than that with the setting "includeAllIntermediates" (Figure 2), while still fulfilling many users' requirement. Figure 5 demonstrates the usage of this setting. 11 commonly used animal species are included as the low-level source terms. If we use the setting "includeAllIntermediates", 70 intermediate terms will be included. However, only six intermediate terms are included after the "inclu-deComputedIntermediates" setting is applied ( Figure 5).

Each of these six intermediate terms (e.g., Euarchontoglires) is the immediate parent class for at least two child terms (e.g., Primates and Homo sapiens). Primates and mammals are not leaf nodes in the taxonomy hierarchy when the sole parent term is Homo sapiens. Since we wanted these terms as well in the final result of the Onto-Fox output file, we have intentionally included them as low-level source terms.

A third choice for including selected terms is inspired by structural modularization techniques. Given a set of signature terms, OntoFox retrieves restrictions that are parent classes of a term. This choice is implemented using OntoFox's SPARQL-based related term retrieval algorithm (Figure 6). Where a restriction mentions another class, restrictions on that class are queried, and so on, until a fixed point is reached. The method gives useful results with the ontologies at the typical level of complexity we encounter. It also has the benefit of being straightforwardly implemented in SPARQL. SPARQL is able to easily retrieve the Concise Bounded Description (CBD) of a resource by means of an RDF graph, which is a general and broadly optimal unit of specific knowledge about that resource to be utilized by, and/or interchanged between, semantic web agents [29]. Current modularization algorithms use in-memory representations that require excessive memory for ontologies such as NCBI Taxonomy. In contrast, the SPARQL-based approach is highly scalable. Within the OntoFox user interface, users select this choice by choosing "includeAllAxioms".

To test OntoFox's SPARQL method to retrieve related terms, three sets of signature terms (individual term, small subset of terms, larger ontology file) were given as input to the OntoFox method and OWLAPI modularization method. In all three cases, both methods generated identical results. One comparative test was performed using the set of terms in the Neurodegenerative Disease Phenotype Ontology (NDPO; http://ccdb.ucsd.edu/ NDPO/1.0/NDPO.owl) that imports the Neuroscience Information Framework (NIF) Lexicon ontology (nif.owl; http://ontology.neuinfo.org/NIF/nif.owl). The imports closure of this OWL file contains some 50,000 classes in 87 MB of OWL files. Applying the OWLAPI to the classes and object properties in NDPO.owl yielded a module with 1351 classes and 7 object propertiesroughly 2.5 M OWL file including annotations. The OntoFox generated the same results as measured by the ontology metrics provided by Protégé 4.1. These results support our claim that the OntoFox approach is an effective method for extracting related ontology terms.

Finally OntoFox can extract the whole branch ontology terms below a specific ontology term (Box 5 in Figure 1).

Choices such as which terms in a parent hierarchy should be included are preliminary to module extraction techniques, which take as input a set of terms (signature) that the ontology developer has identified as being of interest. OntoFox supports experimentation by offering more than one choice for making such a term selection.

OntoFox data input and result output

Besides the web form-based data input, data can be uploaded as a text file to the OntoFox web server. This input file contains the same information as the web form input method, but makes it easier to submit batch jobs. The file upload method also makes it possible to keep track of submissions and easily update the input. An OntoFox sample input file (available at http://ontofox.hegroup.org/format.txt) has been developed for users to quickly understand and use the required format. Also, the OntoFox input file can also be automatically gener-

Figure 5

OntoFox demonstration of the includeComputedIntermediates setting. Terms that are common ancestors (e.g., Bovidae) to at least 2 external terms are kept in the resulting hierarchy, in addition to the terms (e.g., Primates) explicitly requested. ated using the button "Generate OntoFox Input File" from data in the web forms ( Figure 2).

Finally, jobs can be programmatically submitted to the OntoFox server via a script at http://ontofox.hegroup .org/service.php. As an example, the following command line can be used to provide an input file (input.txt) and retrieve the corresponding output file (output.owl): "curl -s -F file = @/tmp/input.txt -o/tmp/output.owl http:// ontofox.hegroup.org/service.php.

An OntoFox query can result in either a processing error, in which case an explicit message is provided to the user, or in the production of an OWL file serialized in the RDF/XML format. This OWL file constitutes an ontology on its own and can be visualized using the Protégé ontology editor [11] and directly imported into the target ontology using the OWL import directive.

The OntoFox process can be executed at different times to import updated information of external ontology terms. By keeping and updating the original OntoFox input text file, users can subsequently query the OntoFox server on a regular basis and get up to date information with little effort.

OntoFox application in Vaccine Ontology (VO) development

When using OntoFox to develop VO, we found desirable to apply different settings depending on the source ontology considered, and therefore generated one OWL file to be imported per external resource. Once imported into VO, external terms can be used exactly in the same way as other vaccine-specific VO terms. Different OntoFox settings have been applied for generating these 12 ontology subsets for VO imports (Table 2). In terms of superclass extraction, six were generated with the OntoFox setting "includeNoIntermediates", which is particularly useful when the intermediate superclasses do not generate much more information needed for the target ontology. The setting "includeComputedIntermediates" was used for extracting ontology terms from three external resources, including NCBITaxon, PATO, and PRO [32]. In the case of the NCBI taxonomy it reduces the number of imported classes without losing the information of the most recent ancestor superclasses ( Figure  5). Finally, the setting "includeAllIntermediates" has been used for extraction from OBI, ro_proposed http:// purl.org/obo/owl/ro_proposed, and the Sequence Ontology (SO; http://www.sequenceontology.org/) ( Table 2). These three external ontologies are closely related to VO, and we would like to use all original hierarchies for those terms imported to VO.

Similarly, different annotation property settings have been applied (Table 2). Typically, VO follows the IAO's ontology metadata scheme and uses the properties "rdfs:label" or "iao:definition". To make the annotation styles consistent among all ontology terms in VO, the OntoFox directives "copyTo" and "mapTo" were used ( Table 2).

Discussion

While an implementation of the MIREOT strategy has been performed in the context of OBI, it relies on command line scripts, making it use impractical for the average ontology curator and limiting its adoption by interested users. Comparatively, OntoFox provides a convenient web-based approach to use MIREOT that does not require programmatic skills and allows users to specify their requirements via simple text formats.

In addition, the OntoFox server provides additional options for users to add and rewrite annotations, to include superclasses or subclasses, or select terms via related restrictions (transitively). This last option performs comparable to existing structural modularization methods.

OntoFox uses a RDF triple store and SPARQL for information storage and retrieval, resulting in a system that scales better than in-memory modularization techniques. For the Neurodegenerative Disease Phenotype Ontology, OntoFox extracted the same module that a more sophisticated modularization technique did. While we don't argue that these more sophisticated techniques are not desirable, we do note that the authors acknowledge there are issues with their use. While OntoFox uses simpler methods to retrieve terms and axioms related to MIR-EOT specified terms, it provides a simpler and more Figure 6 OntoFox SPARQL-based algorithm for retrieval of related terms. Its goal is to extract related terms and annotations associated with a set of signature terms (stored in Su) from an external ontology. This method was performed in OntoFox when the setting "includeAllAxioms" is selected.

understandable approach to reuse. This is particularly useful in conjunction with the fact that OntoFox provides an easy approach to incorporate frequent updates from source ontologies that are under active development.

The provision of a simple mechanism for importing selected terms from external ontologies does not shield the user from general issues associated with using external terms. When using terms from other ontologies, care must be taken to avoid a situation in which the meaning of an ontology term in the source ontology is different from the meaning of the term used in the target ontology. To avoid this problem, users are advised to exercise due diligence when selecting terms to import. OntoFox helps prevent this confusion, by first offering a limited set of 15 selected ontologies with good documentation and second by importing annotation properties, providing immediate access to the textual definitions. Where an ontology developer has questions as to the meaning of a term we recommend that they contact the developers of the source ontology and ask for clarification and enhanced documentation. Our 15 initially selected ontologies generally have trackers and mailing lists where questions can be posted.

Another issue is the evolution problem associated with using ontologies that are under active development, as is the case with most current biomedical ontologies. Although at a certain time point a certain term is used in the source and target ontologies equivalently, over time the usage of the term (and the associated classes) in both ontologies may change. It is considered good practice to not use terms from external ontologies in ways not consistent with their definition, and for ontologies to deprecate old and define new terms rather than changing the meaning of terms. While OntoFox provides a way for users to automatically update the annotations of imported terms, it cannot monitor changes in meaning. Therefore it is up to developers to choose ontologies that have practice that will let them monitor for such changes and make adjustments as appropriate. OntoFox's 15 initial ontologies were chosen because they tend to have predictable practices related to ontology evolution.

Conclusions

The web-based OntoFox system supports ontology reuse by following and extending the existing MIREOT principle and developing a new ontology term retrieval method. It dynamically extracts external ontology terms and their chosen annotations from SPARQL endpoint(s), while not requiring any prior knowledge of the SPARQL query language and/or computational programming skills. Finally, OntoFox provides flexible controls over management and customization of the import. Further development is planned. For example, we will include more ontologies in our list of source ontology repositories. These ontologies may come from the OBO foundry or other reliable sources. We are also considering developing an OntoFox plugin for ontology editors (e.g., Protégé). Editors of OBO format ontologies desire a similar facility, and while OntoFox currently supports resources in OWL, we plan to support the OBO format directly by integrating an automatic conversion for OBO files. As usability testing of our web interface has thus far been informal, we will perform more careful usability studies and have designed a survey to solicit feedback from the community.

With more ontologies being developed, OntoFox offers a timely web-based package supporting efficient ontology reuse.

Availability and requirements

The OntoFox web system is freely available at: http:// ontofox.hegroup.org/. Its usage requires Internet access. The source code is also freely available under the Apache License 2.0 [Additional file 3].

List of Abbreviations

Additional material

Figure 3
3OntoFox retrieval of PATO term 'volume' and its annotations. (A) OntoFox input data; (B) Protégé display of OntoFox output data [Additional file 2]. All terms from 'volume' up to 'quality of continuant' in PATO have been imported and positioned under the BFO term Quality. The desired annotation properties (IAO_0000111 'preferred term' and IAO_0000115 'definition') have been specified using OntoFox directives 'copyTo' and 'mapTo'.

Figure 4
4OntoFox algorithm for extracting computed intermediate classes. It removes any intermediate classes that have only one parent class and only child class. Only intermediate terms with at least two child classes are kept.

Figure 2OntoFox retrieval of the term 'homo sapiens' from the NCBI Taxonomy Ontology (NCBITaxon). Input data can be entered via webbased forms (A) or text file upload (B). The output OWL file [Additional file 1] can be visualized using Protégé (C). All terms from 'homo sapiens' up to Eukaryota are retrieved. Synonyms used to annotate each term are also included.Parse
input

Raw OWL
ouput

OntoFox
OWL ouput

Filter
Reformat

RDF stores

SPARQL
queries

OntoFox Web

Database Servers

RDF triple
stores

Web Server

Output file download
OntoFox backend processes

Setting for
intermediate
term mireoting

Top level terms &
target ontology
superclasses

Source
ontology

Annotation
Settings

Low level
source terms

Branch extractions:
source terms &
target ontology
superclasses

A
B

Table 2 :
2OntoFoxed ontologies in VO #Ontology Name # of signature terms # of imported terms Intermediates Annotations1
CARO
2
2
No
rdfs:label copyTo iao:preferredTerm
oboInOwl:hasDefinition copyTo iao:definition
oboInOwl:hasSynonym mapTo iao:alternativeTerm

2
CHEBI
13
13
No

3
DOID
10
57
All

4
FMA
2
2
No

5
GO
2
2
No

6
IDO
1
2
No

7
NCBITaxon
143
198
Computed

8
OBI
41
48
All
rdfs:label
iao:definition

9
PATO
15
17
Computed
rdfs:label copyTo iao:preferredTerm
oboInOwl:hasDefinition copyTo iao:definition
oboInOwl:hasSynonym mapTo
iao:alternativeTerm

10 PRO
2
2
Computed

11 ro_proposed
7
9
All

12 SO
1
1
No

API: Application Programming Interface; BFO: Basic Formal Ontology; CHEBI: Chemical Entities of Biological Interest; GO: Gene Ontology; FMA: Foundational Model of Anatomy; InfluenzO: The Influenza Ontology; PATO: Phenotypic Quality Ontology; MIREOT: Minimum information to reference external ontology terms; MP: Mammalian Phenotype Ontology; NCBI: The National Center for Biotechnology Information; NCBO: The National Center for Biomedical Ontology; NEMO: Neural Electro-Magnetic Ontologies; NIF: The Neuroscience Informatics Framework; OBI: Ontology for Biomedical Investigations; OBO: The Open Biomedical Ontologies; OWL: Web Ontology Language; PATO: Phenotypic Quality Ontology; PHP: Hypertext Preprocessor; PRO: Protein Ontology; RO: OBO Relation Ontology; ro_proposed: RO proposed version; SPARQL: SPARQL Protocol and RDF Query Language; SO: Sequence Ontology; RDF: Resource Description Framework; URI: Uniform Resource Identifier; VO: Vaccine Ontology.
Xiang et al. BMC Research Notes 2010, 3:175 http://www.biomedcentral.com/1756-0500/3/175 Page 10 of 12
This article is available from: http://www.biomedcentral.com/1756-0500/3/175 © 2010 He et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. BMC Research Notes 2010, 3:175
AcknowledgementsCompeting interestsThe authors declare that they have no competing interests.Authors' contributions ZX: Webmaster, software programmer, database administrator, use case testing, and manuscript editing. MC: OntoFox testing, discussion, valuable feedback and suggestions, and manuscript editing. RRB: Discussion, valuable feedback and suggestions, and manuscript editing. AR: OntoFox testing, discussion, valuable feedback and suggestions, and manuscript editing. YH: Project design and management, programming, use case testing, and drafting of manuscript.Additional file 1 Output file forFigure 2. This is the output file in OWL (RDF/XML) format based on the OntoFox input shown inFigure 2B. Additional file 2 Output file forFigure 3. This is the output file used in OWL (RDF/XML) format based on the OntoFox input shown inFigure 3A.
Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. M Ashburner, C A Ball, J A Blake, D Botstein, H Butler, J M Cherry, A P Davis, K Dolinski, S S Dwight, J T Eppig, Nat Genet. 251Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, Davis AP, Dolinski K, Dwight SS, Eppig JT, et al.: Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nat Genet 2000, 25(1):25-29.

B Smith, W Ceusters, B Klagges, J Kohler, A Kumar, J Lomax, C Mungall, F Neuhaus, A L Rector, C Rosse, Relations in biomedical ontologies. Smith B, Ceusters W, Klagges B, Kohler J, Kumar A, Lomax J, Mungall C, Neuhaus F, Rector AL, Rosse C: Relations in biomedical ontologies.

. Genome Biol. 6546Genome Biol 2005, 6(5):R46.

The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. B Smith, M Ashburner, C Rosse, J Bard, W Bug, W Ceusters, L J Goldberg, K Eilbeck, A Ireland, C J Mungall, Nat Biotechnol. 2511Smith B, Ashburner M, Rosse C, Bard J, Bug W, Ceusters W, Goldberg LJ, Eilbeck K, Ireland A, Mungall CJ, et al.: The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol 2007, 25(11):1251-1255.

The PubMed database. The PubMed database [http://www.ncbi.nlm.nih.gov/pubmed/]

Text mining and ontologies in biomedicine: making sense of raw text. I Spasic, S Ananiadou, J Mcnaught, A Kumar, Brief Bioinform. 63Spasic I, Ananiadou S, McNaught J, Kumar A: Text mining and ontologies in biomedicine: making sense of raw text. Brief Bioinform 2005, 6(3):239-251.

Life sciences on the Semantic Web: the Neurocommons and beyond. A Ruttenberg, J A Rees, M Samwald, M S Marshall, Brief Bioinform. 102Ruttenberg A, Rees JA, Samwald M, Marshall MS: Life sciences on the Semantic Web: the Neurocommons and beyond. Brief Bioinform 2009, 10(2):193-204.

Survey of modular ontology techniques and their applications in the biomedical domain. J Pathak, T M Johnson, C G Chute, Integrated Computer-Aided Engineering. 163Pathak J, Johnson TM, Chute CG: Survey of modular ontology techniques and their applications in the biomedical domain. Integrated Computer-Aided Engineering 2009, 16(3):225-242.

Modular Reuse of Ontologies: Theory and Practice. B C Grau, I Horrocks, Y Kazakov, U Sattler, Journal of Artificial Intelligence Research. 31Grau BC, Horrocks I, Kazakov Y, Sattler U: Modular Reuse of Ontologies: Theory and Practice. Journal of Artificial Intelligence Research 2008, 31:273-318.

VO: Vaccine Ontology. Y He, L Cowell, A D Diehl, H L Mobley, B Peters, A Ruttenberg, R H Scheuermann, R R Brinkman, M Courtot, C Mungall, The 1st International Conference on Biomedical Ontology. He Y, Cowell L, Diehl AD, Mobley HL, Peters B, Ruttenberg A, Scheuermann RH, Brinkman RR, Courtot M, Mungall C, et al.: VO: Vaccine Ontology. The 1st International Conference on Biomedical Ontology (ICBO 2009) Nature Precedings 2009 [http://precedings.nature.com/documents/ 3553/version/1].

. N Y Buffalo, Usa, Buffalo, NY, USA July 24-26, 2009 2009

OWL 2 Web Ontology Language document overview. OWL 2 Web Ontology Language document overview [http://www.w3. org/TR/2009/REC-owl2-overview-20091027/]

Pellet: A practical OWL-DL reasoner. E Sirin, B Parsia, B C Grau, A Kalyanpur, Y Katz, Web Semant. 5Sirin E, Parsia B, Grau BC, Kalyanpur A, Katz Y: Pellet: A practical OWL-DL reasoner. Web Semant 2007, 5:51-53.

FaCT++ Description Logic Reasoner: System Description. D Tsarkov, I Horrocks, Proc of the Int Joint Conf on Automated Reasoning (IJCAR 2006). of the Int Joint Conf on Automated Reasoning (IJCAR 2006)4130 of Lecture Notes in Artificial IntelligenceTsarkov D, Horrocks I: FaCT++ Description Logic Reasoner: System Description. Proc of the Int Joint Conf on Automated Reasoning (IJCAR 2006) 2006:292-297. 4130 of Lecture Notes in Artificial Intelligence

MIREOT: the Minimum Information to Reference an External Ontology Term. M Courtot, F Gibson, A Lister, J Malone, D Schober, R Brinkman, A Ruttenberg, The 1st International Conference on Biomedical Ontology. Buffalo, NY, USACourtot M, Gibson F, Lister A, Malone J, Schober D, Brinkman R, Ruttenberg A: MIREOT: the Minimum Information to Reference an External Ontology Term. In The 1st International Conference on Biomedical Ontology (ICBO 2009) Buffalo, NY, USA; 2009.

Just the right amount: extracting modules from ontologies. B C Grau, I Horrocks, Y Kazakov, U Sattler, Proceedings of the 16th international conference on World Wide. the 16th international conference on World WideGrau BC, Horrocks I, Kazakov Y, Sattler U: Just the right amount: extracting modules from ontologies. Proceedings of the 16th international conference on World Wide Web 2007:717-726.

Safe and Economic Re-Use of Ontologies: A Logic-Based Methodology and Tool Support. E Jiménez-Ruiz, B C Grau, U Sattler, T Schneider, R B Llavori, Proceedings of the European Semantic Web Conference. the European Semantic Web Conference5021Jiménez-Ruiz E, Grau BC, Sattler U, Schneider T, Llavori RB: Safe and Economic Re-Use of Ontologies: A Logic-Based Methodology and Tool Support. Proceedings of the European Semantic Web Conference 2008, 5021:185-199.

Web ontology segmentation: analysis, classification and use. J Seidenberg, A Rector, Proceedings of the 15th international conference on World Wide Web. the 15th international conference on World Wide WebSeidenberg J, Rector A: Web ontology segmentation: analysis, classification and use. Proceedings of the 15th international conference on World Wide Web 2006:13-22.

Extracting modules from ontologies: a logic-based approach. B C Crau, I Horrocks, Y Kazakov, U Sattler, Modular Ontologies LNCS 5445 Edited by Stuckenschmidt H. Berlin HeidelbergSpringer-VerlagCrau BC, Horrocks I, Kazakov Y, Sattler U: Extracting modules from ontologies: a logic-based approach. In Modular Ontologies LNCS 5445 Edited by Stuckenschmidt H: Springer-Verlag Berlin Heidelberg; 2009:159-186.

Ontology Module Extraction for Ontology Reuse: An Ontology Engineering Perspective. P Doran, V Tamma, L Iannone, CIKM'07. Lisboa, PortugalDoran P, Tamma V, Iannone L: Ontology Module Extraction for Ontology Reuse: An Ontology Engineering Perspective. In CIKM'07, November 6-8: 2007 Lisboa, Portugal; 2007.

Minimal Module Extraction from DL-Lite Ontologies using QBF Solvers. R Kontchakov, L Pulina, U Sattler, T Schneider, P Selmer, F Wolter, M Zakharyaschev, Proceedings of IJCAI-09. IJCAI-09PasadenaKontchakov R, Pulina L, Sattler U, Schneider T, Selmer P, Wolter F, Zakharyaschev M: Minimal Module Extraction from DL-Lite Ontologies using QBF Solvers. Proceedings of IJCAI-09 2009; Pasadena 2009:836-841.

Specifying ontology views by traversal. N F Noy, M A Musen, Third International Conference on the Semantic Web. 2004Noy NF, Musen MA: Specifying ontology views by traversal. In Third International Conference on the Semantic Web (ISWC-2004): 2004

. Japan Hisroshima, Hisroshima, Japan; 2004.

Extracting modules from ontologies: a logic-based approach. B C Grau, I Horrocks, Y Kazakov, U Sattler, Modular Ontologies LNCS 5445. Grau BC, Horrocks I, Kazakov Y, Sattler U: Extracting modules from ontologies: a logic-based approach. In Modular Ontologies LNCS 5445

. H Editedd By Stuckenschmidt, Springer-VerlagBerlin HeidelbergEditedd by Stuckenschmidt H: Springer-Verlag Berlin Heidelberg; 2009:159-186.

MIREOT: the Minimum Information to Reference an External Ontology Term. M Courtot, F Gibson, A Lister, J Malone, D Schober, R Brinkman, A Ruttenberg, SubmittedCourtot M, Gibson F, Lister A, Malone J, Schober D, Brinkman R, Ruttenberg A: MIREOT: the Minimum Information to Reference an External Ontology Term. Submitted 2010.

SPARQL queries template file. SPARQL queries template file [http://obi.svn.sourceforge.net/svnroot/ obi/trunk/src/tools/build/external-templates.txt].

RDF/XML syntax specification. RDF/XML syntax specification [http://www.w3.org/TR/rdf-syntax-gram mar/]

SPARQL query language for RDF. SPARQL query language for RDF [http://www.w3.org/TR/rdf-sparql-que ry/]

McGraw-Hill Higher Education. R S Pressman, Software Engineering a Practitioner's Approach. Princeton, New Jersey5th editionPressman RS: Software Engineering a Practitioner's Approach. 5th edition. McGraw-Hill Higher Education, Princeton, New Jersey; 2001.

. Information Artifact Ontology. Information Artifact Ontology [http://code.google.com/p/information- artifact-ontology/]

Bounded Description. Cbd -Concise, CBD -Concise Bounded Description [http://www.w3.org/Submission/ CBD/]

. Pato -Phenotypic Quality, Ontology, PATO -Phenotypic Quality Ontology [http://obofoundry.org/wiki/ind ex.php/PATO:Main_Page]

The Mammalian Phenotype Ontology as a tool for annotating analyzing and comparing phenotypic information. C L Smith, C A Goldsmith, J T Eppig, Genome Biol. 617Smith CL, Goldsmith CA, Eppig JT: The Mammalian Phenotype Ontology as a tool for annotating analyzing and comparing phenotypic information. Genome Biol 2005, 6(1):R7.

Framework for a protein ontology. D A Natale, C N Arighi, W C Barker, J Blake, T C Chang, Z Hu, H Liu, B Smith, C H Wu, 10.1186/1756-0500-3-175BMC Bioinformatics. 891SupplNatale DA, Arighi CN, Barker WC, Blake J, Chang TC, Hu Z, Liu H, Smith B, Wu CH: Framework for a protein ontology. BMC Bioinformatics 2007, 8(Suppl 9):S1. doi: 10.1186/1756-0500-3-175

OntoFox: web-based support for ontology reuse BMC Research Notes. Xiang, 3175Cite this article asCite this article as: Xiang et al., OntoFox: web-based support for ontology reuse BMC Research Notes 2010, 3:175


A Study of Barriers and Facilitators of Clinical Practice Guidelines Implementation among Physicians
November-December 2017

Lamya Alnaim firstname.lastname@example.com
Department of Clinical Pharmacy
College of Pharmacy
King Saud University
Riyadh-11451Saudi Arabia

Saja Almaz
Department of Clinical Pharmacy
College of Pharmacy
King Saud University
Riyadh-11451Saudi Arabia

A Study of Barriers and Facilitators of Clinical Practice Guidelines Implementation among Physicians
November-December 2017Accepted 18 September 2017 Revised 20 March 2017 Received 04 October 2016Indian Journal of Pharmaceutical Sciences 923 *Address for correspondence This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical termsClinical practice guidelinesbarriersfacilitatorsphysicians
Alnaim and Almazrou.: Barriers and Facilitators of Clinical Practice GuidelinesThere is a gap in understanding the barriers and facilitators related to guideline implementation in Saudi Arabia. A descriptive cross sectional study was conducted to identify the factors that may act as barriers or facilitators of clinical practice guidelines used by physicians working in Riyadh, Saudi Arabia. A survey was distributed to 215 physicians working at universities and government hospitals in Riyadh city. The survey was pilot-tested on a small group of physicians to improve the clarity and limit response bias. A total of 157 physicians completed the questionnaire. About 88.5% of the physicians indicated that they did implement clinical practice guidelines in their practice while 9.6% declared that they did not implement any guidelines. Clinical practice guideline implementation was not affected by the physician role, years of experience or nationality. There were less female physicians that opted for clinical practice guidelines (P<0.5). Eighty four percent of the physicians chose them because they are evidence-based while only 40.1% incorporated them because it was part of their institutions' regulation. There was a weak positive relation between the total scores of barriers and facilitators assessment tool and the nationality of the physician. The reasons for choosing specific guidelines were not determined by the physician role, years of experience, nationality or gender. Most agreed that clinical practice guidelines served as a good foundation for self-study and still provide space for them to come to their own conclusions, but at the same time, they would like to know more about them before they decide to apply it. The implementation of guidelines was obstructed by three barriers, which were lack of support by the leadership, lack of awareness and lack of knowledge. On the other hand, the ease of access and autonomy has driven the implementation of clinical practice guidelines. Nonetheless, the barriers and facilitators experienced by the physicians may differ depending on their ethnicity and gender.

Research Paper

Clinical practice guidelines (CPGs) are defined by Field and Lohr as "systematically developed statements to assist practitioners and patients in making decisions on an appropriate health care for specific circumstances" [1] . In 2011, the Institute of Medicine (IOM) defined CPGs as "statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options" [2] . The guidelines are designed to support the decision-making processes in patient care. The movement towards evidence-based healthcare has been gaining ground quickly over the past few years, as it is propelled by clinicians, legislators and organizations who are concerned about quality, consistency and costs. CPGs, based on standardized best practice, have been shown to be capable of driving improvements in quality and consistency in healthcare [3] .

The process of collecting, assembling and transferring research evidence into practice in health care settings is challenging, but it is vital to the provision of effective, safe and equitable health care. Hence, the development of CPGs is an important way to facilitate the use of evidence in clinical practice [4] . The principles and methods for developing guidelines have progressed since the 1990s, strongly influenced by the advancement in evidence-based medicine (EBM) [5] . As such, the guidelines are now being developed at international, national and local levels [6] . The Cochrane Collaboration and the National Institute for Health and Care Excellence (NICE) are few examples of organizations that have a long experience in developing guidelines to support decisions regarding safe and effective medical and public health interventions [7] . There are signs of an increased emphasis on better pace and better quality on the development of guidelines in the past decade, which is exemplified by the emergence of guideline clearing houses and many international programs and societies [8] .

The last few years have seen some significant changes in the methods concerning the development of CPGs where several international initiatives have been put forward in order to develop credible guidelines. These undertakings have addressed various problems such as conflicts of interest, balancing the advantages and disadvantages of having input from experts and assuring that the conflicts of interest do not influence recommendations [9] . At the same time, methodologists have charged on to develop frameworks and tools, which will help to facilitate the development and dynamic updating of trustworthy guidelines [10] . The guidelines will be useful in different settings such as to guide the introduction of new procedures or services, improve clinical outcomes, encourage the adoption of cost-effective interventions or to reduce the length of hospitalization [11] .

The literature is growing with studies that explore the barriers to the implementation of CPGs. There are few identified barriers which include, but not limited to, the lack of support by the leadership, lack of financial resources, and behavioral inconsistency. Other barriers might be related to the physician himself such as the lack of agreement and confidence with specific guidelines as well as lack of motivation [12] . Apart from that, the implementation process could also be hindered by system or process barriers ranging from the lack of familiarity and awareness, volume overload, to the time needed to stay informed. Several less frequently identified barriers are related to the perceived quality of the CPGs and evidence, or patient concerns, while specific guideline-related barriers may include the complexity, compatibility with existing values, need for extra resources or acquisition of new knowledge and skills [13,14] .

In Saudi Arabia, data regarding the barriers and facilitators of the implementation of CPGs is scarce, and those available data mainly focuses on the adaption of EBM. Al-Almaie et al. found that the main barriers to the practice of EBM by the physicians appeared to be a lack of knowledge, basic skills facilities, and limited time [15] . In another study, the major perceived barriers in practicing EBM were patient overload and lack of personal time. The respondents thought that the most appropriate way to encourage the EBM was by learning the skills of EBM, followed by using evidencebased guidelines developed by the colleague [16] . These results, although not directly applicable, pave the way for more detailed studies in all aspects of EBM practice and implementation in the Saudi Arabia.

Wahabi et al. [17] investigated the barriers and facilitators on the implementation of a specific CPG, the Pediatrics Asthma Management Protocol (PAMP). They studied physicians' adherence to the PAMP in an emergency department with the objective of examining the compliance of the healthcare providers. They also reviewed the patients' chart to investigate the compliance of the healthcare providers to eight recommendations of the PAMP whereby in the event of non-compliance, a focus group interview was conducted to determine the reasons that led to the noncompliance. They found that organizational barriers and the lack of an implementation strategy for the protocol, in addition to the attitude and beliefs of the healthcare providers, are the main factors behind the non-compliance to the CPG [17] .

These few studies have not looked at the problem from the perspective of physicians in Saudi Arabia. Findings from studies in western or Asian countries cannot be completely extrapolated to local physicians, as there may be unique difference or unidentified factors. Most studies done in Saudi Arabia focused on single specific guidelines. There exist a gap in the understanding of the barriers and facilitators related to guideline implementation. This understanding is critical for development of effective and targeted guideline implementation strategies. In this investigation, we aimed to identify and explore the factors that may have acted as general barriers or facilitators on the implementation of the CPGs by physicians in Saudi Arabia.

MATERIALS AND METHODS

Setting and participants:

This was a descriptive cross-sectional study conducted in several tertiary hospitals in the Riyadh area. The study survey was distributed to available physicians working in the universities and government hospitals in the city of Riyadh. As per the latest census, there are 6715 physicians in Riyadh. The survey was carried out on 215 physicians. The proposal was first submitted to the Institutional Review Board (IRB) of each institution for approval and obtained an expedited approval due to the observational nature of the study. After identifying eligible participants, verbal consent was obtained from each participant where the study aims, objectives, and potential values were explained. All participants were treated with respect from the time they were approached, even if they refused enrolment, as well as throughout their participation. The generated data was stored after each participant was coded appropriately to ensure confidentiality. Hence, none of the research output data was traceable to any participant.

The survey was based on the instrument developed by Peters et al. [18] . The survey design consisted of three main sections. The first section covered the physicians' demographics and relevant characteristics including their nationality, place of work, specialty and current position, in addition to their experience with CPGs. The second section was the barriers and facilitator assessment tool which addressed the physicians' views on the different barriers and facilitators presented to them. The questions required the respondents to indicate how much they agreed or disagreed with each statement. Every question had five answers scored on a scale which is as follows: 1-strongly disagree through to 5-strongly agree. The third component consisted of open ended questions regarding the barriers and facilitators. The survey was pilot-tested on a small group of physicians to improve its clarity and limit response bias.

The survey was delivered manually to an appropriate sample of physicians. The survey was delivered through the secretarial services of the respective departments. The surveys were distributed on different working days of the week to the physicians available on those days. If the response rate was low, the next approach was to deliver the survey by hand directly to the physicians and wait for its completion. The recruited physicians were given the option to refer the survey to a colleague if they believed someone else in their department was better able to answer the questions.

Statistical analysis:

The data was analyzed using SPSS (SPSS, version 21). Descriptive statistics (frequencies and percentages) were used to represent the demographic characteristics of participating physicians. A score was calculated for the barriers and facilitators assessment tool for each participant. The Kruskal-Wallis H test was used to determine the difference in the barriers and facilitators assessment tool score in the different demographics groups. Chi-Square test was used to determine the effect of different demographic variables on the implementation of CPGs in practice. Pearson correlation was used to test the correlation between the barriers and facilitators assessment tool score and the reasons physicians gave to choose particular CPGs. A step-wise multiple linear regression was performed to assess the correlation between demographics and the barriers and facilitators assessment tool score. For all analysis, 2 tailed P<0.05 was considered as statistically significant.

RESULTS AND DISCUSSION

A total of 157 physicians completed the survey. Most of the physicians who responded were residents and registrars at the percentage of 39.4 and 34.8, respectively, which reflected the staffs' demographic. The majority of the respondents were male (63.7%). Only 14.6% of the physicians had over 20 y of medical experience. The majority of the respondents were junior physicians with 0-5 y of experience (38.2%). Saudi nationals comprised 43.3% of the respondents (Table 1).

Regarding the implementation of CPGs, 88.5% of the physicians indicated they did implement CPGs in their practice while 9.6% did not implement any guideline. The implementation of guideline was not affected by the physician's role, years of experience or nationality. It is interesting to note that there was a statistically

TABLE 1: DEMOGRAPHICS OF PARTICIPANTS

The majority of the respondents were junior physicians with 0-5 y of experience (38.2%). Saudi nationals comprised 43.3% of the respondents significant difference in the implementation between males and females (P<0.5) as more females chose not to use guidelines (Table 2). In terms of reasons for choosing a specific guideline, Table 3 shows that most physicians (84.1%) prefer a particular guideline because it was based on evidence, while only 40.1% implemented a guideline as it was required by their institution. Apart from that, the physician's role, years of experience, nationality or gender did not have any effect on the reasons (Table 4). Table 5 outlines the physicians' perceptions of the barriers and facilitators which may affect the implementation of CPGs After performing multiple regression analysis to detect the effects of different demographic characteristics on the total scores of barriers and facilitators assessment tool, we found a weak positive relation between the total scores of barriers and facilitators assessment tool and nationality, since r is 0.297, P-value is 0.003, where r is Pearson correlation coefficient.

In this investigation, we explored factors that may have acted as general barriers or facilitators of CPGs use by physicians by conducting a survey on 157 physicians. The analysis shows that a high percentage of physicians were using CPGs in their practice. Most participants used CPGs because they are evidence-based, while only 40.1% implemented guidelines as per instructed by the institution. This highlights a structural barrier where there is a lack of support by the leadership, hence the physicians make independent choices in deciding    on whether or not to implement guidelines. This barrier has been pointed out in several studies carried out in various settings [9][10][11][12][13][14][15][16][17][18][19][20][21][22] . Moreover, Chapman et al. has observed that if a team practice is tailored according to the team head, the choice of which CPG to adopt might change if a new team head is appointed [19] .

Based on the results obtained, most physicians believed the CPG is a good starting point for their self-study and would like to know more about it before they decide to implement it. This indicates that the physicians would like to participate in the process of choosing which CPG to implement in their practice which is a theme that has been identified in other studies [17] . A lack of participation of the target group of users in the development of the protocol and original guidelines was a significant barrier in the implementation of the pediatric asthma guidelines in the emergency department in Saudi Arabia. It can be argued that the physicians in this study perceived their lack of awareness of the guideline and a lack of knowledge as a barrier to implementation. This has been one of the most reported barriers across studies in this topic [20][21][22]24] . This lack of knowledge could be divided into two types; first, the lack of knowledge about the guideline itself which has been labeled as awareness in some studies [23,25] , and second, lack of knowledge in the usage or application of the guidelines, which has been labeled as skills in other studies [22,24] .

An awareness of a CPG is an essential step to implement it; however, physicians can be aware of the CPG but choose not to implement it. This phenomenon speaks of acceptability, at least from a clinician's perspective, and suggests that the implementation would be greatly aided by efforts to disseminate information about the program more widely. Basic knowledge is essential, but not sufficient, and other modalities of dissemination have to be employed.

There is an evidence of physicians who firmly believed that CPGs are not limiting and leave enough room for their autonomy and the incorporation of patient's wishes. This apparent autonomy seems to be a unique finding in this study. McGuire et al. described an autonomy-supporting leadership where most staffs viewed their leadership as being "supportive", but also emphasized that their leadership delegated decisions and execution to the program-level leaders [26] With regards to the patients' preferences, Lugtenberg et al. found that it was one of the most perceived barriers to adherence, which is contrary to the case in this study [27] .

In addition, most respondents perceived that CPGs are handy to use even when they reported difficulties in changing routines and habits to follow guidelines [27,28] .

The majority of participants agreed that most of the barriers presented to them were not apparent in their practice except two barriers which were a lack of cooperation from other colleagues in implementing the CPGs in addition to those who assumed that the implementation of guideline would require financial compensation. Le et al. saw that in some practices, the practitioners prioritized time and resources on collective implementation activities and organized their everyday practice to support these activities. In another setting, a group of practitioners would discuss the guidelines collectively but the final decision on the implementation will be decided individually [29] . As such, an active implementation support is an important facilitator for the adoption of the guideline [19,20,26] .  In our study, Regression analysis exhibited that the implementation of guideline was not affected by the physician's role, years of experience or nationality. However, there was a statistically significant difference between the male and female practitioners (P<0.5). This phenomenon has been scarcely studied and relatively overlooked. Feldman et al. found that female family physicians are most likely to have positive attitudes toward guidelines [30] . A review by Berger in 2008 reported that the physicians' gender influenced their clinical practice and physicians' practice patterns are influenced by their own demographic characteristics [31] . A previous work has found a more positive attitude towards the implementation of guidelines by younger physicians [32] . In the Tunis study of internal medicine physicians, more recent graduates from the medical school were reported to have more positive attitudes toward guidelines [33] . In this study, the analysis also showed that the physician's ethnicity affected the application of CPGs, albeit weakly. Although this issue was not explored extensively in the literature, a study by Koo et al. found considerable differences in the practitioners' colorectal cancer screening practices in different ethnicities [34][35][36] .

There are some limitations in the study that are worth mentioning. Although our response rate was around 62%, it may nevertheless limit the ability to generalize our findings, as those with a positive attitude toward guidelines may be overrepresented in our sample as well as male physicians. Apart from that, the investigated barriers were not based on a specific guideline, as the study of specific guidelines could have potentially yielded different patterns of barriers. Nonetheless, the general nature of the survey was expected to identify barriers that were representative across all guidelines in practice. Furthermore, any adherence was based on self-report and may represent an overestimate as a result of social desirability bias. However, all participants were assured anonymity and no identifying information was collected which should give the participant the freedom to complete the survey without any external pressure.

Most physicians in Saudi Arabia were receptive of the CPGs and will apply them in their practices if there are more leadership and structural support for the implementations. A lack of cooperation between colleagues is an important issue which needs to be addressed in order to find practical solutions. Moreover, by engaging physicians in the choice of CPGs and providing limited flexibility to accommodate their input and patients' wishes will be a good motivator for them to implement CPGs. To make a CPG more accessible for practitioners, it should be rewrite and summarize in simpler terms. Each team needs to study their own specific barriers and facilitators in order to be able to address the problems specific to their setting and staffs.

TABLE 2 :
2GUIDELINE IMPLEMENTATION AND DEMOGRAPHICSStatistically significant difference in the implementation between males and females (P<0.5) as more females chose not to use guidelines. *Chi-square testReasons for choosing a
specific guideline

Percent of
physicians who
agree

Mean
SD

Evidence based
guideline
88.1
4.59
0.635

Have easy layout and
are user friendly
66.3
4.02
0.935

Recommended by a
colleague
50.3
3.6
1.097

Address most clinical
scenarios seen in my
daily practice

77.8
4.32
0.729

Obligated by institution/
department head
40.1
3.2
1.268

TABLE 3 :
3REASONS PHYSICIANS GIVE FOR CHOOSING A SPECIFIC GUIDELINEMost physicians prefer a particular guideline because it was based
on evidence, while only 40.1% implemented a guideline as it was
required by their institution

N
Mean
SD
P-value
Role in respective department
Consultant
36
19.1
3.9

0.571*

Assistant
consultant/
registrar

51
19.3
4.5

Resident
53
19.2
2.9
Gender
Male
94
19.5
3.8
0.596**
Female
47
18.5
4.8
Years of experience in area (y)
0-5
53
19.5
2.6

0.830*

6-10
23
18.9
3.9
11-15
21
18.9
4.1
16-20
22
18.1
6.5
>20
22
20.2
2.4
Nationality
Saudi
94
19.5
3.4
0.596**
Non-Saudi
47
18.5
4.8

TABLE 4 :
4REASONS PHYSICIANS GIVE FOR CHOOSING A SPECIFIC GUIDELINE IN RELATION TO DEMOGRAPHICS*By Kruskal-Wallis Test, **by Mann-Whitney Test

The lay-out of clinical practice guidelines makes it handy for useSD
Mean Statements
0.94
3.6
Clinical practice guidelines leaves enough room for me to make my own conclusion
1.02
3.3
Clinical practice guidelines leaves enough room to weigh the wishes of the patient
0.82
4
Clinical practice guidelines are a good starting point for my self-study
1.02
2.3
I did not thoroughly read nor remember the clinical practice guidelines
1.02
3.9
I wish to know more about the clinical practice guidelines before I decide to apply it
1.03
2.2
I have problems changing my old routines
0.9
2.4
I think parts of the clinical practice guidelines are incorrect
0.99
2.1
I have a general resistance to working according to protocols
0.98
2.9
Other doctors or assistants do not cooperate in applying clinical practice guidelines
1.03
2.5
Managers/directors do not cooperate in applying clinical practice guidelines
1.02
2.6
Patients do not cooperate in applying clinical practice guidelines
0.96
2.3
Applying the clinical practice guidelines is too time consuming
0.86
2
Clinical practice guidelines does not fit into my ways of working at my practice
1.09
2.9
Working according to clinical practice guidelines requires financial compensation
0.9
3.6

TABLE 5 :
5BARRIERS AND FACILITATOR ASSESSMENT TOOL
Acknowledgements:This research project was supported by a Grant from the Research Center of The Female Scientific and Medical Colleges, Deanship of Scientific Research, King Saud University.Conflicts of interest:There are no conflicts of interest.Financial support and sponsorship:Nil.
Clinical Practice Guidelines: Directions for a New Program, Institute of Medicine. M J Field, K N Lohr, National Academy Press38Washington, DCField MJ, Lohr KN. Clinical Practice Guidelines: Directions for a New Program, Institute of Medicine. Washington, DC: National Academy Press; 1990. p. 38.

Clinical Practice Guidelines We Can Trust. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. R Graham, M Mancher, D M Wolman, S Greenfield, E Steinberg, National Academy PressWashington, DCGraham R, Mancher M, Wolman DM, Greenfield S, Steinberg E. Clinical Practice Guidelines We Can Trust. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Washington, DC: National Academy Press; 2011.

Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. S H Woolf, R Grol, A Hutchinson, M Eccles, J Grimshaw, BMJ. 3187182Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ 1999;318(7182):527-30.

Effectiveness and efficiency of guideline dissemination and implementation. J M Grimshaw, R E Thomas, G Maclennan, C Fraser, C R Ramsay, L Vale, Health Technol Assess. 86Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation. Health Technol Assess 2004;8(6):1-72.

Evidence based medicine: What it is and what it is'nt. D L Sackett, W M Rosenberg, J A Gray, R B Haynes, W S Richardson, BMJ. 312Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: What it is and what it is'nt. BMJ 1996;312:71-2.

Development of evidence-based clinical practice guidelines (CPGs): comparing approaches. T Turner, M Misso, C Harris, S Green, Implement Sci. 345Turner T, Misso M, Harris C, Green S. Development of evidence-based clinical practice guidelines (CPGs): comparing approaches. Implement Sci 2008;3:45.

The appraisal of public health interventions: an overview. A J Fischer, A Threlfall, S Meah, R Cookson, H Rutter, M P Kelly, J Public Health (Oxf). 354Fischer AJ, Threlfall A, Meah S, Cookson R, Rutter H, Kelly MP. The appraisal of public health interventions: an overview. J Public Health (Oxf) 2013;35(4):488-94.

Systematic review of the methodological quality of clinical guideline development for the management of chronic disease in Europe. C Knai, S Brusamento, H Legido-Quigley, V Saliba, D Panteli, E Turk, Health Policy. 1072-3Knai C, Brusamento S, Legido-Quigley H, Saliba V, Panteli D, Turk E, et al. Systematic review of the methodological quality of clinical guideline development for the management of chronic disease in Europe. Health Policy 2012;107(2- 3):157-67.

The vexing problem of guidelines and conflict of interest: a potential solution. G Guyatt, E A Akl, J Hirsh, C Kearon, M Crowther, D Gutterman, Ann Intern Med. 15211Guyatt G, Akl EA, Hirsh J, Kearon C, Crowther M, Gutterman D, et al. The vexing problem of guidelines and conflict of interest: a potential solution. Ann Intern Med 2010;152(11):738-41.

Creating clinical practice guidelines we can trust, use, and share: a new era is imminent. P O Vandvik, L Brandt, P Alonso-Coello, S Treweek, E A Akl, A Kristiansen, Chest. 1442Vandvik PO, Brandt L, Alonso-Coello P, Treweek S, Akl EA, Kristiansen A, et al. Creating clinical practice guidelines we can trust, use, and share: a new era is imminent. Chest 2013;144(2):381-9.

Implementing clinical guidelines in psychiatry: A qualitative study of perceived facilitators and barriers. T Forsner, J Hansson, M Brommels, A A Wistedt, Y Forsell, BMC Psychiatry. 108Forsner T, Hansson J, Brommels M, Wistedt AA, Forsell Y. Implementing clinical guidelines in psychiatry: A qualitative study of perceived facilitators and barriers. BMC Psychiatry 2010;10:8.

Why don't physicians follow clinical practice guidelines? A framework for improvement. M D Cabana, C S Rand, N R Powe, A W Wu, M H Wilson, P A Abboud, JAMA. 282Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999;282:1458-65.

Expected benefits of clinical practice guidelines: Factors affecting their adherence and methods of implementation and dissemination. Al Mazrou, S H , J Health Spec. 1Al Mazrou SH. Expected benefits of clinical practice guidelines: Factors affecting their adherence and methods of implementation and dissemination. J Health Spec 2013;1:141-7.

Barriers facing physicians practicing evidence-based medicine in Saudi Arabia. S M Al-Almaie, Al-Baghli , N , J Contin Educ Health Prof. 24Al-Almaie SM, Al-Baghli N. Barriers facing physicians practicing evidence-based medicine in Saudi Arabia. J Contin Educ Health Prof 2004;24:163-70.

The place of evidence-based medicine among primary health care physicians in Riyadh region, Saudi Arabia. L A Al-Ansary, T A Khoja, Fam Pract. 195Al-Ansary LA, Khoja TA. The place of evidence-based medicine among primary health care physicians in Riyadh region, Saudi Arabia. Fam Pract 2002;19(5):537-42.

Reasons behind non-adherence of healthcare. H A Wahabi, R A Alziedan, BMC Health Serv Res. 12226Wahabi HA, Alziedan RA. Reasons behind non-adherence of healthcare. BMC Health Serv Res 2012;12:226.

Clinicians adopting evidence based guidelines: a case study with thromboprophylaxis. N H Chapman, S P Lazar, M Fry, M N Lassere, B H Chong, BMC Health Serv Res. 11240Chapman NH, Lazar SP, Fry M, Lassere MN, Chong BH. Clinicians adopting evidence based guidelines: a case study with thromboprophylaxis. BMC Health Serv Res 2011;11:240.

Barriers to guideline implementation and educational needs of general practitioners regarding heart failure: a qualitative study. F Peters-Klimm, I Natanzon, T Müller-Tasch, S Ludt, C Nikendei, N Lossnitzer, GMS Z Med Ausbild. 29346Peters-Klimm F, Natanzon I, Müller-Tasch T, Ludt S, Nikendei C, Lossnitzer N, et al. Barriers to guideline implementation and educational needs of general practitioners regarding heart failure: a qualitative study. GMS Z Med Ausbild 2012;29(3):46.

Identifying the challenges and facilitators of implementing a COPD care bundle. L Lennox, S Green, C Howe, H Musgrave, D Bell, S Elkin, BMJ Open Resp Res. 135Lennox L, Green S, Howe C, Musgrave H, Bell D, Elkin S. Identifying the challenges and facilitators of implementing a COPD care bundle. BMJ Open Resp Res 2014;1:e000035.

Barriers to and enablers of evidence-based practice in perinatal care in the SEA-ORCHID project. T Turner, J; Sea-Orchid Study Short, Group, J Eval Clin Pract. 19Turner T, Short J; SEA-ORCHID Study Group. Barriers to and enablers of evidence-based practice in perinatal care in the SEA-ORCHID project. J Eval Clin Pract 2013;19:591-7.

A mixed-methods study of barriers and facilitators to the implementation of postpartum hemorrhage guidelines in Uganda. L Braddick, V Tuckey, Z Abbas, D Lissauer, K Ismail, S Manaseki-Holland, Int J Gynaecol Obstet. 1321Braddick L, Tuckey V, Abbas Z, Lissauer D, Ismail K, Manaseki-Holland S, et al. A mixed-methods study of barriers and facilitators to the implementation of postpartum hemorrhage guidelines in Uganda. Int J Gynaecol Obstet 2016;132(1):89-93.

Barriers and enablers to implementing multiple stroke guideline recommendations: a qualitative study. A Mccluskey, A Vratsistas-Curto, K Schurr, BMC Health Serv Res. 13323McCluskey A, Vratsistas-Curto A, Schurr K. Barriers and enablers to implementing multiple stroke guideline recommendations: a qualitative study. BMC Health Serv Res 2013;13:323.

Which chronic obstructive pulmonary disease care recommendations have low implementation and why? A pilot study. K1 Johnston, K Grimmer-Somers, M Young, R Antic, P Frith, BMC Res Notes. 5652Johnston K1, Grimmer-Somers K, Young M, Antic R, Frith P. Which chronic obstructive pulmonary disease care recommendations have low implementation and why? A pilot study. BMC Res Notes 2012;5:652.

Factors affecting implementation of an evidencebased practice in the Veterans Health Administration: Illness management and recovery. A B Mcguire, M P Salyers, D A White, D J Gilbride, L M White, J Kean, Psychiatr Rehabil J. 384McGuire AB, Salyers MP, White DA, Gilbride DJ, White LM, Kean J, et al. Factors affecting implementation of an evidence- based practice in the Veterans Health Administration: Illness management and recovery. Psychiatr Rehabil J 2015;38(4):300-5.

Perceived barriers to guideline adherence: a survey among general practitioners. M Lugtenberg, J S Burgers, C F Besters, D Han, G P Westert, BMC Fam Pract. 1298Lugtenberg M, Burgers JS, Besters CF, Han D, Westert GP. Perceived barriers to guideline adherence: a survey among general practitioners. BMC Fam Pract 2011;12:98.

Barriers and enablers to implementation of a New Zealand-wide guideline for assessment and management of cardiovascular risk in primary health care: a template analysis. A Mckillop, J Crisp, K Walsh, Worldviews Evid Based Nurs. 93McKillop A, Crisp J, Walsh K. Barriers and enablers to implementation of a New Zealand-wide guideline for assessment and management of cardiovascular risk in primary health care: a template analysis. Worldviews Evid Based Nurs 2012;9(3):159-71.

How GPs implement clinical guidelines in everyday clinical practice-a qualitative interview study. J V Le, H P Hansen, H Riisgaard, J Lykkegaard, J Nexøe, F Bro, Fam Pract. 326Le JV, Hansen HP, Riisgaard H, Lykkegaard J, Nexøe J, Bro F, et al. How GPs implement clinical guidelines in everyday clinical practice-a qualitative interview study. Fam Pract 2015;32 (6):681-5.

Clinical practice guidelines on depression: awareness, attitudes, and content knowledge among family physicians in New York. E L Feldman, A Jaffe, N Galambos, A Robbins, R B Kelly, J Froom, Arch Fam Med. 71Feldman EL, Jaffe A, Galambos N, Robbins A, Kelly RB, Froom J. Clinical practice guidelines on depression: awareness, attitudes, and content knowledge among family physicians in New York. Arch Fam Med 1998;7(1):58-62.

The influence of physicians' demographic characteristics and their patients' demographic characteristics on physician practice: implications for education and research. J T Berger, Acad Med. 831Berger JT. The influence of physicians' demographic characteristics and their patients' demographic characteristics on physician practice: implications for education and research. Acad Med 2008;83(1):100-5.

US family physicians' experiences with practice guidelines. M Wolff, D J Bower, A M Marbella, J E Casanova, Fam Med. 3012Wolff M, Bower DJ, Marbella AM, Casanova JE. US family physicians' experiences with practice guidelines. Fam Med 1998;30(12):117-21.

Internists' attitudes about clinical practice guidelines. S R Tunis, Rsa Hayward, M C Wilson, H R Rubin, E B Bass, M Johnston, Ann Intern Med. 120Tunis SR, Hayward RSA, Wilson MC, Rubin HR, Bass EB, Johnston M. Internists' attitudes about clinical practice guidelines. Ann Intern Med 1994;120:956-63.

Colorectal cancer screening practise is influenced by ethnicity of medical practitioner and patient. J H Koo, M Y You, K Liu, M D Athureliya, C W Tang, D M Redmond, J Gastroenterol Hepatol. 272Koo JH, You MY, Liu K, Athureliya MD, Tang CW, Redmond DM, et al. Colorectal cancer screening practise is influenced by ethnicity of medical practitioner and patient. J Gastroenterol Hepatol 2012;27(2):390-6.

Patterns of 'leakage' in the utilisation of clinical guidelines: a systematic review. S Mickan, A Burls, P Glasziou, Postgrad Med J. 87Mickan S, Burls A, Glasziou P. Patterns of 'leakage' in the utilisation of clinical guidelines: a systematic review. Postgrad Med J 2011;87:670-9.

Do guidelines on first impression make sense? Implementation of a chest pain guideline in primary care: a systematic evaluation of acceptance and feasibility. L Kramer, N Rabanizada, J Haasenritter, S Bosner, E Baum, N Donner-Banzhoff, BMC Fam Pract. 12128Kramer L, Rabanizada N, Haasenritter J, Bosner S, Baum E, Donner-Banzhoff N. Do guidelines on first impression make sense? Implementation of a chest pain guideline in primary care: a systematic evaluation of acceptance and feasibility. BMC Fam Pract 2011;12:128.