in saudi arabia , health care services are provided primarily by the ministry of health through a network of 2037 health care centers and a broad base of 244 general and specialist hospitals.1 other governmental agencies , such as the ministry of defense and aviation , the ministry of the interior , the saudi arabian national guard , and the saudi arabian oil company , finance and deliver primary , secondary , and tertiary care to a defined population , usually employees and their dependants . the saudi government also finances and provides care on a referral basis in specialized tertiary care hospitals , such as king faisal specialist hospital and research center . the private sector also contributes to the delivery of health care services , especially in cities and large towns , with a total of 125 hospitals ( 11,833 beds ) , 2218 dispensaries , and clinics.1 in addition , saudi working for the private sector and expatriates are eligible for a comprehensive package of health insurance benefits . the council of cooperative health insurance , an independent government body , regulates and supervises a health insurance strategy for the saudi health care market.2 the ministry of health is responsible for managing , planning , and formulating health policies , and supervising health programs , as well as monitoring health services in the private sector . the ministry of health was historically the regulatory authority responsible for licensing pharmaceutical products , medical devices , and manufacturing facilities , but the saudi food and drug authority took over this function in july 2009 . in 2012 , the government has allocated sar 86 billion towards health and social affairs , a 26% increase on the previous year s budget.3 despite that , the steady increase in health care costs because of technological advances , the growing number of people with chronic diseases , and high demand resulting from free services , means that saudi decision - makers will struggle to make choices concerning allocation of health care resources . economic evaluation is a technique that has been developed by economists to assist decision - making when choices have to be made between several courses of action . by definition , economic evaluation is a comparative analysis of alternative courses of action in terms of their costs and consequences.4 it addresses the question of whether something is worth doing when compared with other possible uses of the same resources to ensure that efficiency has been attained or approached . inefficiency exists when resources could be reallocated in a way that would increase the health outcomes produced . many countries have started to use economic evidence to support decisions on licensing , pricing , reimbursement , or addition to the formulary.5 in saudi arabia , it is not mandatory to submit evidence of economic evaluation to support licensing decisions ; however , data will be considered if submitted . the new saudi food and drug authority guidelines for pricing of pharmaceuticals indicate that pharmacoeconomic evidence will be utilized to supplement pricing decisions.6 in a survey of 48 saudi pharmacy and therapeutics committee members , two thirds of the respondents stated that they used pharmacoeconomic evaluation in their formulary decision - making processes and 80% of respondents stated that pharmacoeconomics should be used as a decision - making tool , as in the rest of the world.7 if evaluation of economic evidence is to be used increasingly in saudi arabia , a review of the published literature would be useful to inform policy decision - makers on the current state of research and to plan future research agendas . the purpose of this paper is to provide a critical review of the state of health economic evaluation within the saudi context with regard to the number , characteristics , and quality of published articles . a literature search was undertaken on may 8 , 2011 to identify papers on health economics pertaining to saudi arabia in the pubmed , embase , econlit databases , using the following terms alone and in combination :  cost *  ,  economics the databases were searched without language restriction or publication year limits ( ie , from the start of the databases ) . the tables of contents for the saudi pharmaceutical journal and the saudi medical journal from may 2007 to may 2011 were reviewed by the author . the search was restricted to the last 5 years because the task is time - consuming . articles were excluded if there was no statement or word in the title , abstract , or keywords that indicated that an economic ( including cost ) analysis was conducted . articles were also excluded if they were not original economic evaluations ( eg , if the paper was a narrative review on cost - effectiveness ) , not pertaining to saudi arabia , not published in a fully peer - reviewed journal ( eg , conference proceeding abstracts ) , or did not address a health - related topic . titles and abstracts were screened by a group of three pharmd students following a 15-week course of pharmacoeconomics at the college of pharmacy , king saud university . two students independently screened the titles and abstracts of identified citations for potential eligibility using a standardized screening guide . a random selection of title and abstract was independently reviewed by the author to ensure the accuracy of the inclusion and exclusion process . the citations judged potentially eligible by at least one student were retrieved in full text . the author then read the full texts of potential papers to confirm that they satisfied the inclusion criteria . depending on whether both costs and consequences had been considered and whether a comparison with alternative treatment was made , the studies included were classified by the author into two categories , ie , partial economic evaluations ( cost outcome description , cost comparison ) and full economic evaluations ( cost - effective analysis , cost - benefit analysis , cost - utility analysis , cost - minimization analysis ) . data were recorded about the author , year of the study , sample , methods , sample size , study focus , and main findings . a literature search was undertaken on may 8 , 2011 to identify papers on health economics pertaining to saudi arabia in the pubmed , embase , econlit databases , using the following terms alone and in combination :  cost *  ,  economics the databases were searched without language restriction or publication year limits ( ie , from the start of the databases ) . the tables of contents for the saudi pharmaceutical journal and the saudi medical journal from may 2007 to may 2011 were reviewed by the author . the search was restricted to the last 5 years because the task is time - consuming . articles were excluded if there was no statement or word in the title , abstract , or keywords that indicated that an economic ( including cost ) analysis was conducted . articles were also excluded if they were not original economic evaluations ( eg , if the paper was a narrative review on cost - effectiveness ) , not pertaining to saudi arabia , not published in a fully peer - reviewed journal ( eg , conference proceeding abstracts ) , or did not address a health - related topic . titles and abstracts were screened by a group of three pharmd students following a 15-week course of pharmacoeconomics at the college of pharmacy , king saud university . two students independently screened the titles and abstracts of identified citations for potential eligibility using a standardized screening guide . a random selection of title and abstract was independently reviewed by the author to ensure the accuracy of the inclusion and exclusion process . the citations judged potentially eligible by at least one student were retrieved in full text . the author then read the full texts of potential papers to confirm that they satisfied the inclusion criteria . depending on whether both costs and consequences had been considered and whether a comparison with alternative treatment was made , the studies included were classified by the author into two categories , ie , partial economic evaluations ( cost outcome description , cost comparison ) and full economic evaluations ( cost - effective analysis , cost - benefit analysis , cost - utility analysis , cost - minimization analysis ) . data were recorded about the author , year of the study , sample , methods , sample size , study focus , and main findings . in total , the comprehensive search identified 535 citations . based on a reading of abstracts and titles , 477 papers were excluded . the remaining 58 articles were retrieved in full text and reviewed by the author . upon reviewing the 58 articles , searching the tables of contents for the saudi pharmaceutical journal and the saudi medical journal and a google scholar search did not identify any additional citations . ten studies817 described cost and outcomes for two interventions or more and were categorized as full economic evaluations . the earliest study was published in 199722 and the latest was published in 2010.10 four studies were published in saudi journals and 11 were published in non - saudi journals . only one study was published in a specialized economic journal.10 a description of the main characteristics for each of the papers included according to year of publication is provided in table 1 . in two papers,8,15 the aim did not contain any reference to measurement of cost , cost - effectiveness , or cost - benefit . in the remaining papers , the aim was to assess cost - effectiveness ( n = 4),9,13,14,16 cost - benefit ( n = 1),12 or compare costs with outcomes ( n = 3).10,11,17 eight studies compared two alternatives and one compared three alternatives . seven studies were on pharmaceuticals , two were on surgery , one was on a mixture of interventions for intensive care patients , and one was on diagnostic procedures . as table 1 shows , six of the included studies collected effectiveness measures using a prospective design,913,17 three using pre - intervention and post - intervention design,8,15,16 and one using a retrospective design.14 only one study calculated a sample size.12 the study periods were one year or less ( n = 5),8,9,11,12,17 two years ( n = 2),15,16 six years ( n = 1),13 and eight years ( n = 1).14 one study projected outcomes and costs over 40 years using a markov modeling technique.10 the time horizon of the study or the follow - up period was not clear in some studies , but appeared to be duration of hospital stay . the main outcome measures reported were intermediate disease - specific outcomes ( n = 6 ) , survival rate ( n = 3 ) , and quality - adjusted life years ( n = 1 ) . the costs of pharmaceuticals ( n = 5 ) , length of stay ( n = 4 ) , and instruments and consumables ( n = 2 ) , along with personal ( n = 3 ) and productivity costs ( n = 1 ) were considered . the currency used for cost valuation included us dollars ( n = 6 ) 8,9,11,12,16,17 and local currency ( n = 4).10,1315 in seven papers , one intervention was dominant,8,1014,16 ie , cost less and achieved better outcomes , in two studies one intervention was cheaper with equivalent outcomes,15,17 and one study found one intervention to be more expensive but with a better outcome.9 two studies were medication utilization reviews and reported the associated total costs of the medications concerned18,19 ( table 2 ) . one study presented costs and consequences for a surgical intervention.20 one study presented costs savings associated with reuse of a dialyzer.21 another study reported the usefulness of surveys in screening for varicella immunity and presented the cost savings achieved by vaccination of individuals at risk.22 as shown in table 2 , the studies collected costing data using a retrospective study design ( n = 3)1820 or a prospective design ( n = 1),21 and in one study the design was not clear.22 four studies collected data over a period of one year or less,1820,22 and one study collected data for more than one year.20 the costs of medication ( n = 3 ) and instruments and consumables ( n = 2 ) were considered . table 3 shows the extent to which the studies included meet the recommendations for good reporting of economic evaluations . the perspective taken was specified explicitly in only one paper,10 but it appears to have been that of the hospital for most of the included studies . however , there were some studies which also included cost components , like the costs borne by patients themselves11 and loss of productivity costs.12 in all the papers identified , data on effectiveness came from a single clinical study . the sources of unit prices was only reported by three studies , ie , from the literature,8 expert opinion,10 and hospital data.16 four studies reported quantities of resources ( unit costs ) separately from prices . two papers presented cost estimates in their discussion section.14,16 one study stated clearly the base year of the cost data . discounted costs were used in one study by ali et al , who reported costs and outcomes using a 3% discount rate and calculated the incremental cost - effectiveness ratio by calculating the incremental cost per quality - adjusted life years.10 this study used utility scores for quality - adjusted life years from other countries and did not attempt to estimate these in the saudi context . only one study applied sensitivity analyses , specifically one - way sensitivity analysis.10 again , the perspective taken was not explicitly specified in any of the reviewed papers , although it appears to have been that of the hospital . as with full economic evaluation , the quantities of resources used , prices ( unit costs ) , and sources of prices were not reported by most of the studies . the costs of medications ( n = 3 ) , instruments and consumables ( n = 2 ) were considered . a description of the main characteristics for each of the papers included according to year of publication is provided in table 1 . in two papers,8,15 the aim did not contain any reference to measurement of cost , cost - effectiveness , or cost - benefit . in the remaining papers , the aim was to assess cost - effectiveness ( n = 4),9,13,14,16 cost - benefit ( n = 1),12 or compare costs with outcomes ( n = 3).10,11,17 eight studies compared two alternatives and one compared three alternatives . seven studies were on pharmaceuticals , two were on surgery , one was on a mixture of interventions for intensive care patients , and one was on diagnostic procedures . as table 1 shows , six of the included studies collected effectiveness measures using a prospective design,913,17 three using pre - intervention and post - intervention design,8,15,16 and one using a retrospective design.14 only one study calculated a sample size.12 the study periods were one year or less ( n = 5),8,9,11,12,17 two years ( n = 2),15,16 six years ( n = 1),13 and eight years ( n = 1).14 one study projected outcomes and costs over 40 years using a markov modeling technique.10 the time horizon of the study or the follow - up period was not clear in some studies , but appeared to be duration of hospital stay . the main outcome measures reported were intermediate disease - specific outcomes ( n = 6 ) , survival rate ( n = 3 ) , and quality - adjusted life years ( n = 1 ) . the costs of pharmaceuticals ( n = 5 ) , length of stay ( n = 4 ) , and instruments and consumables ( n = 2 ) , along with personal ( n = 3 ) and productivity costs ( n = 1 ) were considered . the currency used for cost valuation included us dollars ( n = 6 ) 8,9,11,12,16,17 and local currency ( n = 4).10,1315 in seven papers , one intervention was dominant,8,1014,16 ie , cost less and achieved better outcomes , in two studies one intervention was cheaper with equivalent outcomes,15,17 and one study found one intervention to be more expensive but with a better outcome.9 two studies were medication utilization reviews and reported the associated total costs of the medications concerned18,19 ( table 2 ) . one study presented costs and consequences for a surgical intervention.20 one study presented costs savings associated with reuse of a dialyzer.21 another study reported the usefulness of surveys in screening for varicella immunity and presented the cost savings achieved by vaccination of individuals at risk.22 as shown in table 2 , the studies collected costing data using a retrospective study design ( n = 3)1820 or a prospective design ( n = 1),21 and in one study the design was not clear.22 four studies collected data over a period of one year or less,1820,22 and one study collected data for more than one year.20 the costs of medication ( n = 3 ) and instruments and consumables ( n = 2 ) were considered . table 3 shows the extent to which the studies included meet the recommendations for good reporting of economic evaluations . the perspective taken was specified explicitly in only one paper,10 but it appears to have been that of the hospital for most of the included studies . however , there were some studies which also included cost components , like the costs borne by patients themselves11 and loss of productivity costs.12 in all the papers identified , data on effectiveness came from a single clinical study . in many papers , the calculation of cost components is not clearly described . only two papers described methods for estimation of resource use . the sources of unit prices was only reported by three studies , ie , from the literature,8 expert opinion,10 and hospital data.16 four studies reported quantities of resources ( unit costs ) separately from prices . two papers presented cost estimates in their discussion section.14,16 one study stated clearly the base year of the cost data . discounted costs were used in one study by ali et al , who reported costs and outcomes using a 3% discount rate and calculated the incremental cost - effectiveness ratio by calculating the incremental cost per quality - adjusted life years.10 this study used utility scores for quality - adjusted life years from other countries and did not attempt to estimate these in the saudi context . again , the perspective taken was not explicitly specified in any of the reviewed papers , although it appears to have been that of the hospital . as with full economic evaluation , the quantities of resources used , prices ( unit costs ) , and sources of prices were not reported by most of the studies . the costs of medications ( n = 3 ) , instruments and consumables ( n = 2 ) were considered . compared with the number of economic evaluations found for other developing countries , such as bangladesh23 ( n = 12 ) , nigeria24 ( n = 44 ) , zimbabwe25 ( n = 26 ) , thailand26 ( n = 41 ) , and korea27 ( n = 45 ) , saudi seems to be lagging behind in conducting health - related economic evaluations . an important observation is that the economic evaluations identified by this research are extremely heterogeneous and assess an intervention after its diffusion into practice . this suggests that health economics assessment is used on an ad hoc basis rather than as a systematic approach to compare alternatives and make decisions that maximize efficiency . there were many flaws in the design , analysis , and reporting of the economic evaluations identified . the perspective of an economic evaluation is an important issue dictating which outcomes and costs should be measured , but the perspective taken was reported by only one study.10 consequently , the author could not assess if all important and relevant outcomes and costs for each alternative were identified . a serious and common methodological pitfall was found , ie , poor quality of effectiveness data . none of the studies based evidence of effectiveness on the  gold standard  of randomized , clinical trials . instead , the effectiveness data came mainly from small , single , retrospective , or non - randomized prospective studies . also , many papers used intermediate disease - specific measures of benefit . intermediate outputs are appropriate only to the extent that a valid link can be established between these and a final health output.5 furthermore , these measures do not allow for meaningful comparison of cost - effectiveness across disease areas . the use of quality - adjusted life years is recommended not only because it enables cross- disease comparisons to be made , but also because it captures the impact of disease on a patient s quality of life . another shortcoming of the existing studies is failure to describe clearly the methods used to collect and evaluate data on use of resources . moreover , details on type and quantities of resources used were not provided , which undermines the external relevance of the results of these evaluations . unit prices were not presented separately from quantities of resources , which limits the possibility of replicating the analysis in other settings . the date to which the prices referred was not reported , and again this limits the reproducibility of the results . another serious methodological pitfall was inadequate handling of uncertainty around benefits and cost estimates by sensitivity analysis . in sensitivity analysis , uncertain key parameters are varied to assess the impact of uncertainty on the robustness of any conclusions . given the poor quality of evidence for estimating clinical effects , this could seriously undermine confidence in the findings of these evaluations and their ability to inform decisions concerning allocation of health care resources . previous studies have reported that developing countries2327 lack sound economic evaluation data . also , reviews of published economic evaluations from europe and the us2832 point to a number of shortcomings in the published literature . formal and informal health economic guidelines have been issued in many countries to standardize and improve the quality of economic evaluation in health care . the findings of this review show that building an economic evaluation research capacity in saudi arabia is warranted . first , there is a need to establish a national agency or research institute to provide the infrastructure required to support and nurture health economic evaluation research . it could , for example , advise a process and criteria for priority setting of future economic evaluations in saudi arabia . it could also commission funds and facilitate procedures for synthesizing and disseminating the results of economic evaluation . this agency needs to build on the organizational and methodological experience of established health technology assessment agencies , such as the australian pharmaceutical benefits advisory committee , the canadian agency for drugs and technologies in health , and the national institute for health and clinical excellence in england and wales . it would also be necessary to educate potential users about methods of economic evaluation , interpretation of economic evidence , and appraisal of cost - effectiveness claims . efforts towards this end include expanding the availability of short - term and long - term courses on economic evaluation for health care professionals and policy - makers . this may mean that unpublished data , such as in government reports , pharmaceutical company reports , and academic theses , were not identified by the literature search . furthermore , the inclusion of only published articles may have introduced publication bias , because studies with positive results are more likely to be published than studies with negative findings . in addition , the methods sections of many studies did not clearly describe what was done , making it difficult to categorize them , and other readers may categorize them differently . compared with the number of economic evaluations found for other developing countries , such as bangladesh23 ( n = 12 ) , nigeria24 ( n = 44 ) , zimbabwe25 ( n = 26 ) , thailand26 ( n = 41 ) , and korea27 ( n = 45 ) , saudi seems to be lagging behind in conducting health - related economic evaluations . an important observation is that the economic evaluations identified by this research are extremely heterogeneous and assess an intervention after its diffusion into practice . this suggests that health economics assessment is used on an ad hoc basis rather than as a systematic approach to compare alternatives and make decisions that maximize efficiency . there were many flaws in the design , analysis , and reporting of the economic evaluations identified . the perspective of an economic evaluation is an important issue dictating which outcomes and costs should be measured , but the perspective taken was reported by only one study.10 consequently , the author could not assess if all important and relevant outcomes and costs for each alternative were identified . a serious and common methodological pitfall was found , ie , poor quality of effectiveness data . none of the studies based evidence of effectiveness on the  gold standard  of randomized , clinical trials . instead , the effectiveness data came mainly from small , single , retrospective , or non - randomized prospective studies . also , many papers used intermediate disease - specific measures of benefit . intermediate outputs are appropriate only to the extent that a valid link can be established between these and a final health output.5 furthermore , these measures do not allow for meaningful comparison of cost - effectiveness across disease areas . the use of quality - adjusted life years is recommended not only because it enables cross- disease comparisons to be made , but also because it captures the impact of disease on a patient s quality of life . another shortcoming of the existing studies is failure to describe clearly the methods used to collect and evaluate data on use of resources . moreover , details on type and quantities of resources used were not provided , which undermines the external relevance of the results of these evaluations . unit prices were not presented separately from quantities of resources , which limits the possibility of replicating the analysis in other settings . the date to which the prices referred was not reported , and again this limits the reproducibility of the results . another serious methodological pitfall was inadequate handling of uncertainty around benefits and cost estimates by sensitivity analysis . in sensitivity analysis , uncertain key parameters are varied to assess the impact of uncertainty on the robustness of any conclusions . given the poor quality of evidence for estimating clinical effects , this could seriously undermine confidence in the findings of these evaluations and their ability to inform decisions concerning allocation of health care resources . previous studies have reported that developing countries2327 lack sound economic evaluation data . also , reviews of published economic evaluations from europe and the us2832 point to a number of shortcomings in the published literature . formal and informal health economic guidelines have been issued in many countries to standardize and improve the quality of economic evaluation in health care . the findings of this review show that building an economic evaluation research capacity in saudi arabia is warranted . first , there is a need to establish a national agency or research institute to provide the infrastructure required to support and nurture health economic evaluation research . it could , for example , advise a process and criteria for priority setting of future economic evaluations in saudi arabia . it could also commission funds and facilitate procedures for synthesizing and disseminating the results of economic evaluation . this agency needs to build on the organizational and methodological experience of established health technology assessment agencies , such as the australian pharmaceutical benefits advisory committee , the canadian agency for drugs and technologies in health , and the national institute for health and clinical excellence in england and wales . it would also be necessary to educate potential users about methods of economic evaluation , interpretation of economic evidence , and appraisal of cost - effectiveness claims . efforts towards this end include expanding the availability of short - term and long - term courses on economic evaluation for health care professionals and policy - makers . this may mean that unpublished data , such as in government reports , pharmaceutical company reports , and academic theses , were not identified by the literature search . furthermore , the inclusion of only published articles may have introduced publication bias , because studies with positive results are more likely to be published than studies with negative findings . in addition , the methods sections of many studies did not clearly describe what was done , making it difficult to categorize them , and other readers may categorize them differently . this review indicates that there are major flaws in the design , analysis , and reporting of economic analyses performed in the saudi health care setting . such deficiencies mean that the findings of this evaluation may not be very useful in informing decisions on health care resource allocation .
backgroundif evaluation of economic evidence is to be used increasingly in saudi arabia , a review of the published literature would be useful to inform policy decision - makers of the current state of research and plan future research agendas . 
 the purpose of this paper is to provide a critical review of the state of health economic evaluation research within the saudi context with regard to the number , characteristics , and quality of published articles.methodsa literature search was conducted on may 8 , 2011 to identify health economic articles pertaining to saudi arabia in the pubmed , embase , and econlit databases , using the following terms alone or in combination :  cost *  ,  economics  ,  health economics 
 ,  cost - effectiveness 
 ,  cost - benefit  ,  cost minimization 
 ,  cost utility analysis  , and 
 saudi  . 
 reference lists of the articles identified were also searched for further articles . 
 the tables of contents of the saudi pharmaceutical journal and the saudi medical journal were reviewed for the previous 5 years.resultsthe search identified 535 citations . based on a reading of abstracts and titles , 
 477 papers were excluded . upon reviewing the full text of the remaining 58 papers , 43 were excluded . 
 fifteen papers were included . 
 ten were categorized as full economic evaluations and five as partial economic evaluations . 
 these articles were published between 1997 and 2010 . 
 the majority of the studies identified did not clearly state the perspective of their evaluation . 
 there are many concerns about the methods used to collect outcome and costs data . 
 only one study used some sort of sensitivity analysis to assess the effects of uncertainty on the robustness of its conclusions.conclusionthis review highlights major flaws in the design , analysis , and reporting of the identified economic analyses . 
 such deficiencies mean that the local economic evidence available to decision - makers is not very useful . 
 thus , building research capability in health economics is warranted .