text:
Yunmeng Mountain
Last updated by chinatravel at 2015/4/24
Yunmeng Mountain Scenic Spot is a high mountain of northern China style, whose main peak is above 1,300 meters high. It is a dividing line between Hebei Province and Shanxi Province. The scenic spot is surrounded by Cuiyang Mountain, Niuqun Mountain, and Jiaoding Mountain. Entering the scenic spot, you will feel as if you are entering a huge kettle with a big stomach mysteriously. You will see steep cliffs all around, barren cliffs with green trees on their tops, and a round sky above. Hence, it is called 'a kettle-shaped fairyland'. It's divided into four floors: the lowest one called 'the bottom of the kettle' with blue streams and quiet valleys, then 'the middle of the kettle' with steep cliffs and splashing waterfalls, 'the upper of the kettle' with fairy Water-Curtain Cave, and 'the world beyond the Heaven' also called the human world paradise.
Yunmeng Mountain has more than 80 scenic spots open to tourists. The scenic spots are closely distributed. The travel schedule is arranged appropriately thrilling but safe. Yunmeng Mountain is well-known for her waterfalls, ponds, and streams. It is a world of waterfalls and a sea of pools, which are formed by the water from the water-Curtain Cave of 'the world beyond the Heaven'. Though the water-Curtain Cave is towering into clouds and heaven, it has abundant water resource and flows endlessly. When it brims with water, the water surges as a dragon rises out of the sea; when it becomes babbling streams, the water ripples like pearls of silver and jade. Converging two springs from the east and the west, the water of the water-Curtain Cave flows from the highest peak of Yunmeng Mountain, forming the Bailong Sream with gleams of light reflecting on waves in the water. Along its way, it roams among the green hillside, falling down the red cliffs, bypassing the pines, meandering the green grass and finally rushes into a cave. It brings Xiying Waterfall, Gunjiao Waterfall, Longlin waterfall, Sailin Pond, Huanyao Pond, Bibo Pond etc., which altogether form nine waterfallls and eighteen ponds. What's more, there is endowed with profound cultural traditions. It's said that a famous Taoist hermit, Gui Guzi, cultivated himself according to the Taoist doctrine and imparted knowledge to his students here in Warring States. Many other earthshaking people had themselves cultivated here, including Sun Bing, Pang Juan, Su Qin, Zhang Yi, Mao Sui. Some relics are still well preserved today, sach as, the road walked by immortals, red millstone, trace left by immortals, and cave carved with Sutra etc..
Solo Adventure Tips:
How to Get There?
You should have a turn at the northern crossing of xingtai from the Beijing-Shenzhen expressway, and then drive through city proper along Yingbing Road to Xinghe Road. Then down at the Bailiangyuan shop, you will find Shibangfang Village. On the other hand, you can take No.3 buses from Xingtai to Yunmeng Mountain.
Ticket Price:
Opening Hours:
More Tips:
Recommended Time for Tourists: half of a day or a day
Nearby Attractions:
Top Things to Do in Xingtai
Kaiyuan Temple, built in the Kaiyuan Period of the Tang Dyna… >> details
Baiyun Cave Scenic Spot of Kongshan Mountain
Baiyun Cave Scenic Spot occupies an advantageous location … >> details
The vast valley in Xingtai, facing Heshun county of Shanxi… >> details
Yunmeng Mountain Scenic Spot is a high mountain of norther… >> details
Qingfeng Building was built in Tang Dynasty and was rebui… >> details
Travel Confidently with Us
10,000 Satisfied Customers
50 Years in China Travel Industry
Quick Response within 24 hours
Secured Online Payment
Group Tours with Solo Adventure
No Hidden Fees and No Traps


text:
Subject: Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv? From: Eugene Loh (Eugene.Loh_at_[hidden]) Date: 2009-09-22 13:32:50
Thanks for responding. I used a linux cluster. I think I would like to create a model that is multithreaded and each thread can make MPI calls. I attached test code as follow. It has two pthreads and there are MPI calls in both of those two threads. In the main function, there are also MPI calls. Should I use a full multithreading?

subset: commoncrawl


text:
Patti Hudspeth
Location: Nottingham NG12 2GW
Concerns I deal with: Usui Reiki, ...more conditions
Age Groups I work with: Children, Teens, Adults, Elders,
I am passionate about Complementary health care; I believe we need an holistic approach to health where allopathic medicine has a place. I believe we can help ourselves to better health given the appropriate help and support, which I aim to do with ... read more about Patti Hudspeth
Beauty Relaxation (Treatments and Training Provider)
Location: Coventry CV3 5AG
Concerns I deal with: Anxiety, Chronic Fatigue Syndrome (CFS) (ME), Depression, Exhaustion, Fatigue, Headaches, Infertility, Pain Management, Pregnancy, Spirituality, ...more conditions
Age Groups I work with: Children, Teens, Adults, Elders,
Beauty Relaxation offers professional Beauty and Complimentary Therapy treatments. The Business prides itself on offering high quality treatments at affordable prices. Flexible opening times with evening and weekend appointments available. Monday - S... read more about Beauty Relaxation (Treatments and Training Provider)
Anne Widdup MAPHP
Location: Northampton NN14 1JD
Concerns I deal with: Usui Reiki, Abuse, Addictions, Adolescence, Aggression, Agoraphobia, Anger Management, Anorexia Nervosa, Anxiety, Assertiveness, ...more conditions
Age Groups I work with: Children, Teens, Adults, Elders,
Clinical Hypnotherapy for Anxiety,Panic Attacks, Stress Related Conditions, Fears & Phobias, Unwanted habit patterns, Bruxism, Nailbiting, Stop Smoking (will-power required), Weight-loss (combined with a calorie-controlled diet), Low Self esteem, ... read more about Anne Widdup MAPHP
Hypnos Healing
Location: Kirkby in Ashfield Ng17 7HY
Concerns I deal with: Angelic Reiki, Addictions, Adolescence, Affairs, Aggression, Agoraphobia, Alcohol Dependence, Anger Management, Anxiety, Aspergers Syndrome, ...more conditions
Age Groups I work with: Teens, Adults, Elders,
Please see www.hypnos-healing.co.uk for details. read more about Hypnos Healing
Sarah Lewin
Location: Northampton NG5 6LE
Concerns I deal with: Anxiety, Dementia, Depression, Fear, Phobias, Stress, Anxiety, Depression, Insomnia, Phobias, ...more conditions
Holistic Therapies are beneficial for a wide range of ailments. Helping to reduce symptoms and often clear up the problem altogether. Stress is the main cause of over 70% of illnesses. Therefore if we treat the underlying Stress, then usually we s... read more about Sarah Lewin
Essentials Therapy
Location: Ripley DE5 3EP
Concerns I deal with: Anxiety, Pregnancy, Rehabilitation, Stress, Anxiety, Stress, ...more conditions
Sports injury's, soft tissue damage, tight muscles, pre/post event massage, shoulder pain, back ache, stress, anxiety, relaxation. Essential Therapies provides Sports and Remedial massage to clients throughout the local area. The company located... read more about Essentials Therapy
sheree@s Reiki & Beauty Oasis
Location: northampton NN3 8QT
Concerns I deal with: Stress, Stress, ...more conditions
Reiki:- is a japanese technique which addresses phsycial, emotional, mental and spiritual imbalances. Swedish massage:- relieves stress,muscular pain and creates a deep sense of relaxation. other beauty treatments offer a chance to relax be pam... read more about sheree@s Reiki & Beauty Oasis
Seeking a Clinic
Swedish Relaxation Massage
- Locations
- Categories
- Acupressure Thurmaston
- Acupuncture Thurmaston
- Allergy Intolerance Therapy Thurmaston
- Aromatherapy Thurmaston
- Art Therapy Thurmaston
- Bowen Therapy Thurmaston
- Chinese Massage Thurmaston
- Chiropractic Thurmaston
- Cognitive Behavioural Therapy (CBT) Thurmaston
- Corporate Massage Thurmaston
- Counselling Thurmaston
- Crystal Therapy Thurmaston
- Deep Tissue Massage Thurmaston
- EFT (MTT) Tapping Thurmaston
- EMDR Thurmaston
- Flower Essences Thurmaston
- Hypnotherapy Thurmaston
- Indian Head Massage Thurmaston
- Kinesiology Thurmaston
- Life Coaching Thurmaston
- Meditation Thurmaston
- Mindfulness Based Cognitive Therapy (MBCT) Thurmaston
- Natural Fertility Therapy Thurmaston
- NLP Thurmaston
- Nutritional Therapy Thurmaston
- Osteopathy Thurmaston
- Psychotherapy Thurmaston
- Reflexology Thurmaston
- Reiki Thurmaston
- Relaxation Therapy Thurmaston
- Sound Therapy Thurmaston
- Sports Massage Thurmaston
- Stone Therapy Massage Thurmaston
- Stress Management Thurmaston
- Swedish Relaxation Massage Thurmaston
- Thai Massage Thurmaston
- Time Line Therapy Thurmaston
- Traditional Chinese Medicine (TCM) Thurmaston
- Weight Loss Thurmaston
- Yoga Thurmaston


text:
Commissioner Corner Message Board
Need replacement for teams- 10 team keeper league 10 keepers
C A.J. Ellis
(LAD - C)No new player Notes
1B Joey Votto
(Cin - 1B)No new player Notes
2B Aaron Hill
(Ari - 2B)Player Note
3B Aramis Ramírez
(Mil - 3B)Player Note
SS Jhonny Peralta
(Det - SS)NANo new player Notes
CI Paul Konerko
(CWS - 1B)Player Note
MI Daniel Murphy
(NYM - 1B,2B)No new player Notes
OF Giancarlo Stanton
(Mia - OF)New Player Note
OF Josh Hamilton
(LAA - OF)New Player Note
OF Jay Bruce
(Cin - OF)No new player Notes
Util Adam LaRoche
(Was - 1B)New Player Note
Util Brandon Belt
(SF - 1B)Player Note
SP Hiroki Kuroda
(NYY - SP)New Player Note
SP --empty--
SP --empty--
SP --empty--
RP Bobby Parnell
(NYM - RP)DLNo new player Notes
RP --empty--
P Greg Holland
(KC - RP)Player Note
P --empty--
P --empty--
BN Omar Infante
(Det - 2B)New Player Note
BN Michael Young
(Phi - 1B,2B,3B)New Player Note
BN Ichiro Suzuki
(NYY - OF)No new player Notes
BN Jonathan Lucroy
(Mil - C)Player Note
DL Willie Bloomquist
(Ari - 2B,3B,SS)DLNo new player Notes
BN Alexi Ogando
(Tex - SP,RP)Player Note
BN David Robertson
(NYY - RP)No new player Notes
BN Koji Uehara
(Bos - RP)Player Note
BN C.J. Wilson
(LAA - SP)Player Note
BN Bartolo Colón
(Oak - SP)DLNew Player Note-
I maxed out under regular id. send invite to tjmagdziak yahoo.
I have questions. Is the league a free league keeper league or a money one? (I only play in money leagues), Is it head to head or roto (I only play in head to head leagues). Is it weekly or daily lineups (I only play in weekly line ups) and where is it being played? I prefer CBS Commish leagues over Yahoo. I've been playing in money keeper leagues for over 10 years now, and won enough money that I've been playing for free every year for almost 5 years.
- 1 Reply to Матт
I'll take either team.
C Brian McCann
(Atl - C)New Player Note
1B Albert Pujols
(LAA - 1B)DLPlayer Note
2B Robinson Canó
(NYY - 2B)Player Note
3B Pedro Alvarez
(Pit - 3B)No new player Notes
SS Zack Cozart
(Cin - SS)No new player Notes
CI Allen Craig
(StL - 1B,OF)No new player Notes
MI Dustin Pedroia
(Bos - 2B)New Player Note
OF Alex Gordon
(KC - OF)Player Note
OF Andre Ethier
(LAD - OF)Player Note
OF Alejandro De Aza
(CWS - OF)Player Note
Util Ryan Howard
(Phi - 1B)DLNo new player Notes
Util Michael Bourn
(Cle - OF)New Player Note
SP R.A. Dickey
(Tor - SP)Player Note
SP Brandon Morrow
(Tor - SP)DLNo new player Notes
SP Wei-Yin Chen
(Bal - SP)Player Note
SP Trevor Cahill
(Ari - SP)New Player Note
RP Rafael Soriano
(Was - RP)Player Note
RP Grant Balfour
(Oak - RP)No new player Notes
P David Price
(TB - SP)Player Note
P Jason Hammel
(Bal - SP)DLNew Player Note
P Zack Greinke
(LAD - SP)Player Note
BN Salvador Pérez
(KC - C)New Player Note
BN Brett Lawrie
(Tor - 2B,3B)No new player Notes
BN Edwin Encarnación
(Tor - 1B,3B)No new player Notes
BN Derek Jeter
(NYY - SS)DLPlayer Note
BN Ryan Zimmerman
(Was - 3B)New Player Note
BN Jarrod Saltalamacchia
(Bos - C)Player Note- 2 Replies to Joe
I'm interested in the second team - mystikalsue yahoo
C Víctor Martínez
(Det - C,1B)No new player Notes
1B Prince Fielder
(Det - 1B)Player Note
2B Martín Prado
(Ari - 2B,3B,SS,OF)Player Note
3B Pablo Sandoval
(SF - 3B)Player Note
SS Asdrubal Cabrera
(Cle - SS)Player Note
CI Adrián González
(LAD - 1B,OF)New Player Note
MI Dan Uggla
(Atl - 2B)DLNew Player Note
OF Matt Holliday
(StL - OF)Player Note
OF Nelson Cruz
(Tex - OF)NAPlayer Note
OF Ángel Pagán
(SF - OF)DLPlayer Note
Util J.J. Hardy
(Bal - SS)New Player Note
Util Adam Dunn
(CWS - 1B,OF)Player Note
SP Justin Verlander
(Det - SP)Player Note
SP Matt Cain
(SF - SP)New Player Note
SP Tim Lincecum
(SF - SP)Player Note
SP CC Sabathia
(NYY - SP)Player Note
RP Joe Nathan
(Tex - RP)No new player Notes
RP Joel Hanrahan
(Bos - RP)DLNo new player Notes
P Yovani Gallardo
(Mil - SP)New Player Note
P Chris Pérez
(Cle - RP)Player Note
P Tim Hudson
(Atl - SP)DLPlayer Note
BN Jason Kubel
(Ari - OF)No new player Notes
BN Danny Espinosa
(Was - 2B,SS)NANo new player Notes
BN Erick Aybar
(LAA - SS)Player Note
BN Howie Kendrick
(LAA - 2B)DLPlayer Note
BN Adam Eaton
(Ari - OF)Player Note
Expert Fantasy Advice
Over/Under: Assessing the ailing Andrew McCutchen
Andrew McCutchen was a consensus top 2 pick in spring drafts, but his stock has fallen drastically since then. … More »
Daily Dime: Starting Gerrit Cole and stacking Yankees
Let's make better daily choices today, folks. … More »
MLB bullpen depth chart: Tracking closers throughout the season
Every major league team's bullpen hierarchy, all in one place. … More »
Closing Time: Waiting game with Stephen Strasburg, Coco Crisp
Two injured players, one coming and one going. Let's go to D.C. and Oakland. … More »
Roto Arcade Fantasy Football Mock 2015, RDs 7-9: Bennett kicks off TE run
As our first mock draft of the season chugs along, the experts fill gaps with Romo, McFadden, Smith Sr. and others. … More »


text:
Questions and Answers
We receive a wide variety of questions about our work. Here are a few of the most frequently asked questions. We hope this provides the information you're looking for! And if not, please feel free to contact us.
What is Pathfinder's mission?
Who is on Pathfinder's Executive Team?
How many people work at Pathfinder?
Who are your Country Representatives and how do I get in touch with them?
Where does Pathfinder work?
How do I partner with Pathfinder?
I'm interested in a job at Pathfinder. Where can I find information about employment opportunities?
What are ways to get involved to make a difference?
Does Pathfinder provide direct services to clients?
What do you mean by technical assistance?
What is Pathfinder's annual budget and funding sources?
What is your position on abortion?
What is the nature of the abortion work you do?
Why give to Pathfinder?
How does Pathfinder compare to other non-profits?
Who founded Pathfinder? And why is it called Pathfinder?
What is Pathfinder's mission?
Pathfinder International's mission is to advance sexual and reproductive health and rights globally by catalyzing change locally.
What we believe: Reproductive health is essential for creating better opportunities throughout life. When people take charge of their life choices such as when and how often to have children, they gain confidence and strength. They can better pursue their education, contribute to the local economy, and engage in their communities.
Pathfinder International places reproductive health services at the center of all that we do—believing that health care is not only a fundamental human right but is critical for expanding opportunities for women, families, communities, and nations, while paving the way for transformations in environmental stewardship, decreases in population pressures, and innovations in poverty reduction.
In more than 20 countries, Pathfinder provides women, men, and adolescents with a range of quality health services—from contraception and maternal care to HIV prevention and AIDS care and treatment. Pathfinder strives to strengthen access to family planning, ensure availability of safe abortion services, advocate for sound reproductive health policies, and, through all of our work, improve the rights and lives of the people we serve.
Who is on Pathfinder's Executive Team?
Pathfinder's Executive Team includes eight members:
- Purnima Mane, President and CEO
- Mike Zeitouny, Chief Financial Officer
- Caroline Crosbie, Senior Vice President
- Scott Schroeder, Vice President of External Relations
- Demet Güral, Vice President of Programs
- Stephen Redding, Vice President of Country and Regional Operations
- Lee Gelb, Director of Human Resources
- Margaret Kemp Carlson, Vice President of Administration
How many people work at Pathfinder?
Pathfinder has a total staff of 1,033 people worldwide. We have 107 staff at our headquarters in Watertown, MA and 18 in our DC office. The remaining staff work in our offices around the world.
Who is the Country Representative in the country where I live? How do I get in touch with him/her?
Information about our Country Representatives can be found here. The best way to contact them is through our Contact Us page.
Where does Pathfinder work?
Pathfinder works in a multitude of countries. Find out more about where we work:
- Latin America: Haiti and Peru
- Africa: Angola, Burkina Faso, Burundi, Democratic Republic of Congo, Ethiopia, Guinea, Kenya, Mozambique, Niger, Nigeria, South Africa, Tanzania, and Uganda
- Middle East: Egypt
- Asia and Pacific: Bangladesh, India, Myanmar, Viet Nam, and Pakistan
How do I partner with Pathfinder?
Pathfinder has valued partnerships with governments, community groups, non-governmental organizations, and professional medical organizations around the world. You can find out more about our partners here. If you're interested in partnering with Pathfinder, please reach out to Alden Nouga at email@example.com.
I'm interested in a job at Pathfinder. Where can I find information about employment opportunities?
We are always looking for great people to join our team! Employment opportunities can be found in our Careers section.
I want to help! What are ways I can get involved to make a difference?
We are always thrilled to have people share our work and help us improve lives around the world. And there are plenty of ways to help! Check out our Get Involved section for ideas on everything from sharing a video to hosting a fundraiser.
Does Pathfinder provide direct services to clients?
No, we do not provide direct services to clients in the sense of building and staffing separate clinics. Rather, our model is focused on providing local partners—government agencies or ministries, local non-governmental organizations, and private sector institutions—with technical assistance and financial support to help them deliver services through their existing health systems and activities. We also place great importance on community mobilization and support to foster and strengthen relationships between health systems and community systems.
What do you mean by technical assistance?
Technical assistance encompasses a wide range of activities that are intended to enhance our partners' expertise, expand their capacity to deliver services, and ensure the sustainability of their vital programs.
Areas of focus include developing and strengthening management skills and systems, building partners' ability to mobilize communities, developing and conducting training programs for clinical and non-clinical health providers, including traditional providers, creating mechanisms for the involvement of local communities in improving health outcomes, developing innovative approaches to improve service delivery," assisting with policy development in critical areas such as reproductive rights, HIV and AIDS, abortion, and sexual and reproductive health services for adolescents.
What is Pathfinder's annual budget and funding sources?
In fiscal year 2014, operating support and revenues totaled $102 million. The breakdown of funding sources is as follows:
- 82% from the US government
- 11% from private donations
- 7% from multilaterals or foreign governments
How are those funds spent?
2014 expenditures were as follows:
- 89% on programs and services
- 10% on management and general administration
- 1% on fundraising
Please see our Annual Reports and Financials page for more information.
What is your position on abortion?
Pathfinder believes unsafe abortion is a critical public health and human rights issue. International statistics show that whether a woman lives or dies when seeking an abortion is largely dependent upon what part of the world she lives in, as well as the legality of the procedure. In the developed world, where nearly all abortions are safe and legal, a woman's risk of dying is minimal. In the developed world where access to modern contraceptives is greater, unintended pregnancies are reduced.
Pathfinder believes in promoting access to quality reproductive health services. We work hard to reduce reliance on abortion by providing women with the means to safely control their fertility through expanded access to contraception. Still, the inability of the majority of women in developing countries to make decisions on issues of sex or reproduction, coupled with poor access to family planning services, means that safe abortion must be available.
Pathfinder believes that all women, no matter where they live, should have access to safe, legal abortion services. We also believe that comprehensive services include proper counseling to avoid future unintended pregnancies and subsequent abortions and to promote adoption of a contraception method. In countries where abortion is highly restricted, Pathfinder is committed to providing comprehensive postabortion care for women who experience complications from unsafe abortion.
What is the nature of the abortion work you do?
Part of Pathfinder's overall mission is to provide safe abortion care and postabortion care in the countries where we work. We are currently implementing abortion care programs in a range of countries, where we work within the legal specifications and local practicalities of each.
Pathfinder trains physicians and midlevel providers in the use of manual vacuum aspiration (MVA) and medication abortion procedures. We also train providers in associated issues such as infection prevention, pain management, use of antibiotics, and the need for privacy, confidentiality, and informed consent.
Why give to Pathfinder?
If you give to Pathfinder, your contribution will be well spent and your support will have a ripple effect. By giving to Pathfinder, you are helping us change attitudes and save lives. Your investment will have a lasting impact. Give to Pathfinder if you believe, as we do, that leadership takes courage, and that all people deserve access to sexual and reproductive health care. Find out more about why you should give to Pathfinder, and what your gift can mean to millions of women around the world.
How does Pathfinder compare to other non-profits?
Pathfinder is deeply committed to remaining accountable and transparent. Our fiscal and programmatic performance has been approved and reviewed by several independent organizations.
Donors are invited to visit these organizations' websites for assistance in evaluating Pathfinder's performance and efficiency. Pathfinder International has been listed and approved by several independent organizations that evaluate charities' fiscal and programmatic performance including the American Institute of Philanthropy, the BBB Wise Giving Alliance, and Charity Navigator.
Who founded Pathfinder? And why is it called Pathfinder?
Pathfinder was founded by Clarence J. Gamble and his wife, Sarah B. Gamble, in 1957. Clarence J. Gamble, a medical doctor, worked to promote access to contraception and was devoted to reproductive health and rights. Sarah Gamble named the organization in honor of a quote by the poet Antonio Machado, "Traveler, there is no path, paths are made by walking." This quote alludes to our groundbreaking work to bring contraceptive information and services to wherever they are needed. See more in the History section.


text: But there is also the matter of perspective. My wife is convinced that I'm hastening my death by being primal. I have to tell her to back off. Conversely, there is no point me telling her about her unhealthy ways because she won't hear it. In the end, one of us will watch the other decline. It's in the contract


text:
Difficult decisions are common for the Food and Drug Administration (FDA). Whether it is allocating scarce resources, deciding how to mitigate newly found risks, or deciding what investments in human capital, facilities, data, or analytic methods would be most useful, decision-makers in the FDA often need to integrate data of varied quality, recognize uncertainties, and make trade-offs to arrive at a decision. Public-health and public-safety concerns must be balanced with the economic realities of budgets and the political constraints of imposing new regulations. How a program is presented by the media and understood by the public is an important determinant of the acceptance and success of the program. Science and public preferences and perceptions must be considered if one is to understand the potential outcomes of different decision options. To inform the decision-making process, data of differing degrees of quality and robustness must be used and sometimes fed through an array of models of varied sophistication. Expert opinion must be used to interpret the relevance of available data and to solve problems on which available data are weak or nonexistent. Immovable deadlines can thwart uncompromising reliance on the most thorough analysis based on detailed quantitative data for a given decision.
To succeed in such an environment, FDA needs a framework within which alternatives can be defined and evaluated systematically. Although it is beyond the scope of the present study to provide a comprehensive decision-making procedure for FDA, the committee proposes a general framework for thinking about and characterizing the human-health dimensions of FDA decisions. Health consequences are only a subset of the large array of factors that must be considered for any given problem. However, they constitute a reasonable place to start the process of developing a decision framework inasmuch as such factors loom large in most FDA decisions and substantial work has already been done on methods for estimating the human-health consequences associated with various risks, hazards, and decisions.
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 17
A Risk-Characterization Framework
Difficult decisions are common for the Food and Drug Administration
(FDA). Whether it is allocating scarce resources, deciding how to mitigate
newly found risks, or deciding what investments in human capital, facilities,
data, or analytic methods would be most useful, decision-makers in the FDA
often need to integrate data of varied quality, recognize uncertainties, and make
trade-offs to arrive at a decision. Public-health and public-safety concerns must
be balanced with the economic realities of budgets and the political constraints
of imposing new regulations. How a program is presented by the media and un-
derstood by the public is an important determinant of the acceptance and success
of the program. Science and public preferences and perceptions must be consid-
ered if one is to understand the potential outcomes of different decision options.
To inform the decision-making process, data of differing degrees of quality and
robustness must be used and sometimes fed through an array of models of varied
sophistication. Expert opinion must be used to interpret the relevance of avail-
able data and to solve problems on which available data are weak or nonexistent.
Immovable deadlines can thwart uncompromising reliance on the most thorough
analysis based on detailed quantitative data for a given decision.
To succeed in such an environment, FDA needs a framework within which
alternatives can be defined and evaluated systematically. Although it is beyond
the scope of the present study to provide a comprehensive decision-making pro-
cedure for FDA, the committee proposes a general framework for thinking about
and characterizing the human-health dimensions of FDA decisions. Health con-
sequences are only a subset of the large array of factors that must be considered
for any given problem. However, they constitute a reasonable place to start the
process of developing a decision framework inasmuch as such factors loom
large in most FDA decisions and substantial work has already been done on
methods for estimating the human-health consequences associated with various
risks, hazards, and decisions.
OCR for page 17
18 A Risk-Characterization Framework for Decision-Making at FDA
The framework offered here provides a common language for describing
potential public-health consequences of decisions, is designed to have wide ap-
plicability among all FDA centers, and draws extensively on the well-vetted risk
literature to define the relevant health dimensions for FDA decision-making.
This chapter first provides a brief description of the proposed framework and the
risk and decision contexts that influenced the committee's approach. Next, the
basis and definition of the risk attributes that characterize the framework are
provided, and then some approaches for estimating the outcomes of decisions
using the risk attributes are described. The chapter concludes with a discussion
of how the output of the framework can be used to support decision-making.
The risk-characterization framework is designed to be as general as possi-
ble while providing consistent risk information in a way that can be used to sup-
port the wide variety of decisions that FDA faces. It is intended to supplement
and augment other risk-based and risk-informed approaches that are in use and
under development by FDA, not to be a replacement or a one-size-fits-all pre-
scription for conducting all risk-informed decision-making. Indeed, the commit-
tee recognizes that the public-health-consequence factors highlighted in this
framework will seldom be the only important considerations in the decision-
making process, but they are almost always some of the key considerations. The
U.S. Nuclear Regulatory Commission has also recognized that risks are not the
only factors that must be taken into account in regulatory decision-making. It
recently embraced the concept of risk-informed decision-making, which it de-
fines as "the use of risk insights, along with other important information, to help
in making decisions" (USNRC 2008, page 1-1). The committee's framework
focuses on risk information but also recognizes that other information will be
relevant for most FDA decisions.
The process is straightforward and involves three steps:
Step 1. Identify and define the decision context: What decision options are
being considered? What are the appropriate end points to evaluate and compare?
Step 2. Estimate or characterize the public-health consequences of each
option by using the risk attributes that are described below. The values of the
risk attributes should be summarized in a table to facilitate comparison of the
Step 3. Use the completed characterization as a way to compare decision
options and to communicate their public-health consequences within the agency,
to decision-makers, and to the public; use the comparison with other decision-
relevant information to make informed decisions.
Although the steps can be easily articulated, they involve thought and ef-
fort to complete. The framework is not a cookbook and will require FDA to ex-
OCR for page 17
A Risk-Characterization Framework 19
ercise judgment in how it is used. As illustrated in the case studies that are pre-
sented in the chapters that follow, successful implementation requires consis-
tency (for example, use of a single set of attributes for every evaluation) and
flexibility (for example, specific causes of death or illness that are considered
important for a decision can be called out separately in the evaluation and sum-
mary). Completing the attribute table (Step 2) may be relatively simple, or it
may require substantial research and modeling or even additional data collection
and analysis. It should be carried out with whatever level of rigor and effort is
necessary and feasible for the decision being considered. Factors to be consid-
ered in deciding how much effort to expend on the evaluation of public-health
consequences include the timeframe in which the decision must be made and the
relative importance of the public-health consequences compared with other key
decision-making factors. If a decision must be made quickly and the public-
health consequences are less important than other factors, it may be reasonable
to complete the attribute table quickly on the basis of available information and
judgment alone. However, if the decision options under consideration could lead
to substantially different public-health consequences or if public-health conse-
quences are highly uncertain or poorly understood and other factors are consid-
ered less important, it may be worth substantial time and effort to develop more
precise estimates for the attribute tables. The decision needs and the available
resources should determine how much time and effort should be put into imple-
menting the risk-characterization framework.
Science and Decisions: Advancing Risk Assessment (NRC 2009) lays out a
"framework for risk-based decision-making" as part of its recommendations for
improving risk assessment at the U.S. Environmental Protection Agency (EPA).
Although that study was focused specifically on formal risk assessment, it sum-
marized its approach into an overall decision framework that closely matches
what the present committee concludes is appropriate for improving risk-based
decision-making at FDA. Figure 2-1 illustrates the Science and Decisions
The three phases shown in Figure 2-1 correspond to the three steps in the
framework listed above, but the emphasis of the present committee is somewhat
different from that of the previous study. Specifically, the bulk of Science and
Decisions focuses on the risk-assessment portion in phase II of the framework,
as is highlighted in the figure. The framework proposed here focuses on the risk-
characterization portion of the decision structure and adopts a more robust view
of the importance of risk characterization in the overall process of making risk-
informed decisions.
Early descriptions of the risk-assessment process identify risk characteri-
zation simply as a process of combining the exposure and dose-response ele-
ments of a risk assessment to summarize and communicate results. In Figure
OCR for page 17
20 A Risk-Characterization Framework for Decision-Making at FDA
2-1, for example, risk characterization is placed in its traditional location as the
last step in a risk assessment. Understanding Risk (NRC 1996, p. 16) laid out a
challenge to that narrow interpretation and greatly expanded the scope and role
that risk characterization should play in the overall process of making risk-
informed decisions:
We have concluded that the view of risk characterization as a summary is
seriously deficient, and we propose a more robust construction. Risk char-
acterization must be seen as an integral part of the entire process of risk
decision making: what is needed for successful characterization of risk
must be considered at the very beginning of the process and must to a
great extent drive risk analysis. If risk characterization is to fulfill its pur-
pose, it must (1) be decision-driven, (2) recognize all significant concerns,
(3) reflect both analysis and deliberation, with appropriate input from the
interested and affected parties, and (4) be appropriate to the decision.
The present committee adopts the broader view of risk characterization and its
relationship to risk-informed decision-making.
Stage 1: Planning
• For the given decision context, what are the attributes of assessments necessary to characterize risks
of existing conditions and the effects on risk of proposed options? What level of uncertainty and
variability analysis is appropriate?
• What are the relative health or
• What problems are associated
environmental benefits of the
with existing environmental
proposed options?
Stage 2: Risk Assessment
• How are other decision-
• If existing conditions appear
making factors (technologies,
to pose a threat to human or • Hazard Identification
costs) affected by the proposed
environmental health, what
What adverse health or environmental effects options?
options exist for altering those
are associated with the agents of concern?
• What is the decision, and its
• Risk Characterization
• Dose-Response Assessment justification, in light of benefits,
• Under the given decision
costs, and uncertainties in each
context, what risk and other What is the nature and
For each determining adverse effect, what is the option?
technical assessments are magnitude of risk associated with
relationship between dose and the probability of the
necessary to evaluate the existing conditions? • How should the decision be
occurrence of the adverse effect in the range of doses
possible risk-management
identified in the exposure assessment?
options? What risk decreases (benefits) are
associated with each of the • Is it necessary to evaluate the
options? effectiveness of the decision?
• Exposure Assessment Are any risks increased? What are • If so, how should this be done?
the significant uncertainties?
What exposures/doses are incurred by each
population of interest under existing conditions?
How does each option affect existing conditions and
resulting exposures/doses?
Stage 3: Confirmation of Utility
• Does the assessment have the attributes called for in planning?
• Does the assessment provide sufficient information to discriminate among risk-management
• Has the assessment been satisfactorily peer reviewed?
• The involvement of decision-makers, technical specialists, and other stakeholders in all phases of the processes leading to decisions should in no way compromise the technical assessment of risk,
which is carried out under its own standards and guidelines.
FIGURE 2-1 A framework for risk-based decision-making that maximizes the utility of
risk assessment. Source: NRC 2009, p. 243.
OCR for page 17
A Risk-Characterization Framework 21
The committee's approach was also influenced by comparative risk analy-
sis (CRA), which was first defined by EPA in Unfinished Business (EPA 1987).
That report embraces the multidimensional nature of risk and ranks 31 identified
environmental threats according to four attributes: cancer risk, noncancer risk,
ecologic risk, and "welfare" effects. In a follow-up report, Reducing Risk (EPA
SAB 1990), EPA's Science Advisory Board endorsed the broad CRA approach
and as a result spawned many applications of CRA at the office, region, state,
and local levels (Minard 1996; Jones 1997). Those early CRA efforts led to
questions about how best to facilitate comparisons and identify useful attributes
for characterizing risks or risk-reduction opportunities.
Progress on CRA method development continues, although its use remains
relatively limited. In February 1994, a workshop organized by Resources for the
Future for the president's Office of Science and Technology Policy brought to-
gether researchers in CRA with the goal of developing a systematic process for
comparing risks among different federal agencies (Davies 1996). As part of that
work, researchers at Carnegie Mellon University developed a framework for
ranking risks that included both quantitative and qualitative measures of relevant
programmatic attributes (Fischhoff 1995; Morgan et al. 1996). They included
health-effect measures (such as morbidity and mortality) and psychometric
measures that research shows play an important role in the evaluation of risks
(such as fairness, scientific understanding, and uncertainty). That work spawned
a series of research projects and papers that refined and applied the framework
(see, for example, Morgan et al. 1999, 2001; Long and Fischhoff 2000; Morgan
et al. 2000; DeKay et al. 2001; Florig et al. 2001; Willis et al. 2004, 2005;
Fischhoff 2006; Gutiérrez et al. 2006; Bronfman et al. 2007, 2008a,b), including
a discussion directly related to food safety (DeKay et al. 2005).
The framework defined here builds on that work while embracing the no-
tion that risk characterization should be decision-focused rather than restricted to
ranking risks. The framework focuses on describing the potential effects of al-
ternative decisions on health rather than on comparing different health and envi-
ronmental hazards. It also focuses on the identification and use of a clear and
consistent set of risk attributes relevant to public health to describe the effects of
alternative decisions.
Although the health outcomes of alternative decisions are not the only fac-
tors influencing regulatory decision-making, such information is often highly
useful in weighing the merits of alternative decisions. For example, in deciding
whether to withdraw approval for a product for which a new adverse health ef-
fect has been identified, knowledge of the extent, likelihood, and severity of the
newly identified adverse health effect is important, but so is knowledge of the
product's benefits and of the effects that are likely to occur if the product is no
longer available. An important question for the present committee was how to
define a set of attributes that would be robust enough such that risk information
relevant to the broad array of decisions that FDA faces could be adequately cap-
tured and used consistently for risk-informed decision-making.
OCR for page 17
22 A Risk-Characterization Framework for Decision-Making at FDA
As noted in Chapter 1, FDA provided the committee with 16 decision sce-
narios (see Appendix C) and used them to characterize the types of decisions
that it faces. FDA asked the committee to consider three types of decisions:
mitigation-selection decisions, targeting decisions, and strategic-investment de-
cisions (Bertoni 2010). For purposes of developing a decision-focused risk-
characterization framework for FDA, the committee adopted FDA's categoriza-
tion of decisions.
Other ways of categorizing decisions are possible, and some decisions that
are within FDA's authority are difficult to fit within the three categories defined
here. For example, decisions about setting or modifying regulatory standards or
establishing certification standards have the potential to affect numerous FDA-
regulated products simultaneously, and the effects of such standards depend not
only on the standards but on the response of the regulated industry to the stan-
dards and on the effectiveness of enforcement actions. The committee has not
explicitly addressed the standard-setting decisions in the present report, although
the concepts presented here could be extended to address such decisions. Others
have addressed the use of a comprehensive risk perspective in setting standards
(Fischhoff 1984), including consideration of when standard-setting is preferable
to case-by-case decision-making.
Mitigation-Selection Decisions
Mitigation-selection decisions require FDA to choose among two or more
options that are available to reduce or mitigate identified risks. The first step in
applying the framework to such decisions is to identify and specify the mitiga-
tion decision and the decision options to be evaluated and compared. For exam-
ple, in the case study described in Chapter 3, the decision context is a hypotheti-
cal situation in which a vaccine side effect is believed to be occurring at a rate
higher than expected. Two decision options are considered—remove or do not
remove the vaccine from the market. In this case study, the committee deter-
mined that it was important to consider the health consequences of both the vac-
cine and the underlying disease that the vaccine is intended to prevent.
The decision scenarios provided to the committee by FDA included sev-
eral additional examples of mitigation-selection decisions, some of which may
require evaluating and comparing a larger number or greater variety of options
beyond simple binary choices of recall or not. For example, for product-recall
decisions, there might be different levels or types of recall that could be exe-
cuted, each of which could lead to different health consequences; there might be
mitigation options that combine recall of unused product with other risk-
reduction options for product already in use. For risk-mitigation decisions with
several options, each of the viable risk-mitigation approaches needs to be speci-
fied, and the consequences of each must be estimated by using the risk attributes
OCR for page 17
A Risk-Characterization Framework 23
defined below. More generally, almost any decision that results in a change in
product availability or the extent of use of a particular product could be charac-
terized as a mitigation-selection decision; the outcomes will differ in the degree
of use of the product, and the resulting health consequences can be characterized
and compared by using the framework proposed here.
Targeting Decisions
Targeting decisions as described by FDA are essentially priority-setting or
resource-allocation decisions. They appear to be made primarily within a pro-
gram and focus on how a particular resource, such as inspection capability,
should be allocated among a broad set of products or product categories. In this
type of decision, the options or alternatives theoretically available to FDA are
vast: virtually any amount of a resource could be allocated to any of a subset of
the identified products or product categories, and the only constraint would be
total resource availability. The large array of options will need to be narrowed
judiciously before the effects of the resulting decisions can be evaluated in de-
For each product, product category, or other item for which resources
might be targeted, FDA will need to identify the resources that it is considering
allocating. In some cases, the decision may be an "all or none" decision, such as
a decision to inspect a facility or not to inspect it. In others, it may be a
particular level of effort and resources, such as a specific rigor of inspection of a
facility. Generally, each potential target of a resource allocation would require
evaluation of at least two decisions: allocating "x" level of resources vs
allocating "y" level of resources. The difference between the values of the risk
attributes of an allocation of "x" vs "y" defines the public-health benefit of
allocation of resources at those levels to each product or product category.
As noted by FDA (Bertoni 2010), targeting decisions can be seen as simi-
lar to the risk-ranking questions that are historically the main focus of CRA
studies and that remain of interest in some FDA centers. For example, the Cen-
ter for Food Safety and Applied Nutrition (CFSAN) has developed a "Fresh
Produce Risk Ranking Tool," which can be used to rank produce-pathogen pairs
in order of risk (FoodRisk.org 2011). Similarly, a detailed risk assessment of
Listeria monocytogenes conducted in 2003 led to a ranking of ready-to-eat foods
by the likelihood and frequency of illnesses (FDA/USDA 2003).
It is unclear, however, how risk ranking can or should be used to make
risk-informed decisions. When different allocations of resources are expected to
be equally effective in reducing the risks associated with the product or product
category in question, a ranking of potential targets according to unreduced risk
may provide sufficient decision support. For example, in making decisions about
what facilities to inspect in a specific year, CFSAN may be able to use a risk
ranking of facilities in their current conditions and operations with the assump-
tion that additional resources will be equally effective in reducing the risks at
OCR for page 17
24 A Risk-Characterization Framework for Decision-Making at FDA
any facility. Given that assumption, resources should be focused on the highest-
risk facilities. To create such a risk ranking, FDA would need to estimate the
public-health consequences of the food products from each facility that is a can-
didate for inspection—or, more generally, for each potential target of the re-
source allocation—on the basis of its best assessment of the current conditions
and operations of the facilities. The resulting ranked list could then be used as
the basis of resource-allocation decisions.
However, the assumption of equal effectiveness is not often appropriate in
real decision problems. If the situation is more complicated—that is, if the risks
associated with some products or product categories being evaluated can be re-
duced much more effectively or at much lower cost than the risks associated
with others—a narrowly focused risk ranking will not be an appropriate tool for
making the decisions. In such situations, FDA should define the decision options
more explicitly, as discussed above. For example, the risk associated with a food
from a particular facility could be determined for two inspection regimes. The
basis of the status quo inspection procedures could be extracted from an existing
risk-ranking exercise. Then FDA would also need to evaluate the public-health
consequences associated with the food from the facility if a decision were made
to allocate a different level of inspection resources to the facility; this evaluation
would include changes in any risks affected by the different inspection. The
same exercise would then need to be performed for any other food-facility com-
bination that is being considered in the resource-allocation decision. That ap-
proach would lead to a "risk-change" ranking rather than a risk ranking and
would enable FDA to take into account the possibility that some inspections are
more effective than others in reducing risks. If resources are to be reallocated
from existing inspection programs to new programs, the increases in risk due to
the reduction in funding in one must be balanced against the decreases in risk
due to increased funding in the other.
In Chapter 4, the committee illustrates the use of the framework to evalu-
ate three food categories for their current health effects. The results of the
evaluation could be used to rank the food categories by risk, and the ranking
could be used, subject to the limitations described above, to support a targeting
decision for a hypothetical example of food-safety inspections. In Chapter 6, the
committee illustrates a targeting decision that involves a more explicit compari-
son of the effects of different allocations of resources.
Strategic-Investment Decisions
Strategic-investment decisions are longer-term internal decisions about
where FDA should invest its resources to enable better risk-informed decision-
making. For example, additional research can be conducted, more data collected
and analyzed, and new tools developed to increase understanding of and reduce
uncertainties about a wide array of potential risks. Investments in such activities
typically enable better evaluations and more informed decisions, so they provide
OCR for page 17
A Risk-Characterization Framework 25
value to FDA and the American public. However, they are tied only indirectly to
the more common metrics of public health, and it can be difficult to demonstrate
their benefits and to decide which strategic investments are most worth while.
The decision-analytic concept of value of information can be used to
evaluate the potential benefits of strategic-investment decisions about data or
information collection. This concept is that new information has decision-
relevant value only if it could lead to actions different from the actions that
would be taken without the information. That is, information and strategic in-
vestments to collect it have value only if they have the potential to change deci-
sions and thus potentially improve outcomes. For example, if a physician orders
a diagnostic test but would recommend the same treatment regardless of the test
results, the information has no decision-relevant value. In contrast, if the treat-
ment recommendation depends on the diagnosis and the test results will be used
to differentiate among diagnoses, the information potentially has decision-
relevant value. The risk-characterization framework can provide some of the key
elements necessary for a formal value-of-information analysis: definition of a
decision context (Step 1) and characterization of decision outcomes (Step 2).
Some extensions to the framework would be necessary to implement a value-of-
information analysis fully, and those are explored in several of the case studies
that follow.
In the case study in Chapter 5, the committee uses its approach to evaluate
the public-health consequences of a strategic-investment decision to enhance
postmarket surveillance of two medical devices. Application of the committee's
proposed approach to this decision category is the most challenging both be-
cause identifying the strategic-investment decision options to be evaluated is
complicated and because evaluating the effects of long-term strategic invest-
ments on public health is difficult. The case study allows some comparisons to
be made but probably represents a narrower scope of strategic-investment op-
tions than what FDA would consider.
In the risk-assessment paradigm, risks have traditionally been character-
ized by a single attribute, such as the number of deaths that could occur as a
result of the hazard being evaluated or the probability that an exposed individual
will experience an identified adverse effect. Several complications arise with
that simple characterization of risks. Multiple outcomes, such as illnesses and
deaths, are often of interest. Uncertainty in the outcomes makes simple charac-
terization and reporting problematic: reporting only expected fatalities will ob-
scure information about the other possible outcomes that are important to deci-
sion-makers and policy-makers. Providing only single values for health
outcomes may convey far greater certainty than is appropriate. Characterizing
the public-health consequences requires recognizing and accommodating the
OCR for page 17
26 A Risk-Characterization Framework for Decision-Making at FDA
multidimensional nature of risk and the uncertainties involved in estimating the
The Many Dimensions of Risk
The comparison of alternative decisions may seem easier when a single
metric is used and when the harm and the benefit are measured with the same
metric. For example, in deciding whether the association of clozapine with the
potential fatal side effect of agranulocytosis is sufficient to warrant keeping
clozapine off the market, the number of deaths caused by clozapine-induced
agranulocytosis can be directly compared with the number of suicides that
clozapine would be expected to prevent. However, those estimates may differ
greatly in their uncertainty, and other benefits of clozapine, such as improved
ability to function, would also have to be considered in such a decision.
When the harm and the benefit of alternative decisions cannot be meas-
ured with the same metric, decision-making is less straightforward, and the im-
portance of informing decision-makers of the various harms and benefits be-
comes more important. For example, infliximab is highly effective in reducing
pain and improving function in people who have rheumatoid arthritis, but its
immunosuppressive properties may permit serious and possibly fatal infections
to emerge. Deciding whether the risk of a serious adverse effect outweighs the
benefits in increased mobility and quality of life is not a technical or scientific
question but rather a question of personal and societal values. An integral part of
the committee's proposed approach is that it characterizes various effects explic-
itly so that decision-makers can make informed decisions that account for them
rather than combining them into a single metric based on implicit weightings
(see Box 2-1).
The risk-assessment paradigm gives rise to one set of attributes for charac-
terizing the health risks associated with FDA-regulated products or FDA deci-
sions: factors that are used to determine the number, type, and rate of occurrence
of adverse health effects (including deaths) that could result from implementa-
tion of a particular decision option. Those attributes are exposed population,
mortality, and morbidity, each described in more detail below.
Studies of risk perception and public attitudes about risks have consis-
tently shown that although the mortality and morbidity components in risk esti-
mation are important, they are not the only things that people care about when
they think about risks and about risk acceptability (Slovic 1992). Numbers of
deaths and illnesses or injuries matter, but so do other factors, such as whether a
risk is voluntary, how much control a person has over risks, and whether the
hazard being considered has the potential to lead to a large number of simulta-
neous deaths. The list of factors hypothesized to be important in understanding
risk is long (see Appendix E). Relatively few risk attributes have been studied in
OCR for page 17
A Risk-Characterization Framework 27
BOX 2-1 Summary Measures vs Detailed Characterization of
Public-Health Consequences
In the public health literature, it has become common to measure and
communicate the burden of disease by using a summary measure that com-
bines mortality and morbidity into a single value. Some measures, such as
disability adjusted life years (DALYs), have been used to evaluate the global
burden of disease (Lopez et al. 2006). Others, such as quality adjusted life
years (QALYs), have been promoted as tools for comparing the cost effec-
tiveness of various medical interventions (Broome 1993) or risk reducing
regulatory strategies (IOM 2006). Each summary measure necessarily em-
beds a set of value judgments about different levels of health impairment,
and each provides a narrow, rather than a robust, characterization of the
risk. The committee finds that a richer characterization of the public-health
consequences of decision options is needed for routine comparison of deci-
sion options. As described in Improving Risk Communication, "reducing
different kinds of hazard to a common metric (such as number of fatalities
per year) and presenting comparisons only on that metric have great poten-
tial to produce misunderstanding and conflict and to engender mistrust of
expertise" (NRC 1989, p. 52).
detail, but the available studies show that the various factors are highly corre-
lated on a relatively small number of dimensions, and it has become common to
refer to these dimensions as reflecting key "factors" that characterize risk per-
ceptions (Slovic 1992; NRC 1996). The first dimension captures the quantitative
aspects, such as the number and type of adverse health outcomes. The second
dimension is roughly characterized by the degree of knowledge about the haz-
ards or risks and how well they are understood. The third dimension is charac-
terized by variables that are less easily summarized by a single category:
whether a risk is voluntary, how much control an exposed person has over it, the
ease with which it can be reduced, and whether it is catastrophic (that is, can
lead to multiple simultaneous deaths or injuries).
Understanding Risk urged that risk characterization explicitly include con-
sideration of those additional factors and described a number of ways in which
they could be taken into account in decision processes (NRC 1996, p. 65). That
approach has also been embraced as a key part of CRA and in the risk-ranking
studies mentioned above. The risk-ranking studies include both quantitative in-
formation about the number of deaths and injuries and more qualitative informa-
tion reflecting some of the variables identified in the risk-perception literature.
Little general guidance, however, has been offered about what risk attributes
might be widely appropriate. Investigators in the studies cited above typically
acknowledge that attribute selection is complicated and then choose attributes
OCR for page 17
A Risk-Characterization Framework 35
tainties to decision-makers and policy-makers without introducing biases, such
as overconfidence and false precision.
Using the Best Available Information to Develop Estimates
The data required to support the evaluation of the public-health conse-
quences of different decisions can vary widely in quality and availability. For
some problems, large volumes of epidemiologic data may be available to de-
scribe the mortality risk of some medical product. For others, there may be an
array of detailed computer models that attempt to characterize the morbidity-
reduction effectiveness of various inspection procedures. For still others, only
scant data or models may be available, and it may be necessary to rely solely on
the judgment of knowledgeable professionals to develop such estimates. Often, a
hybrid approach will be necessary: using experts to identify relevant information
from studies or models and relying on the experts to interpret the data or model
results in the context of the decisions being evaluated. If data are available, their
relevance to the required estimates must be considered carefully. For example,
the available data may indicate deaths that were temporally associated with a
product or device but were not caused by the product or device; judgment will
be required to interpret such data appropriately and to estimate the mortality
associated with the different decision options being evaluated.
Critical decisions often must be made when uncertainty about some rele-
vant factors remains; in those cases, judgment is required. Lack of easily acces-
sible data does not mitigate the need to develop the best possible estimates of
public-health consequences of products being evaluated; in some cases, care-
fully assessed expert judgment may be the only option available for ensuring
that decision-makers have all the relevant information that is available to make
decisions. Delaying a critical decision until additional information can be col-
lected may be an option in some circumstances, but it should not be considered
the "default" decision when uncertainty exists. Instead, all relevant options, in-
cluding the option to delay a decision until later or to allocate scarce resources to
additional data collection or model development, should be evaluated on the
basis of the best available information, including expert judgment, about the
public-health consequences of the decision.
Many risk-assessment and evaluation tools are available and are described
in detail in other studies, including National Research Council and FDA studies
described above. Box 2-3 summarizes current perspectives at EPA about how to
quantify uncertain outcomes, including a hierarchy of approaches and the role of
expert judgment. It is not within the present committee's scope to evaluate,
compare, or recommend specific approaches or models for risk quantification.
The committee simply urges FDA to bring the best available data and expertise
to bear on the evaluation that would be consistent with the importance of the
decision being evaluated and the time and resources available to complete the
OCR for page 17
36 A Risk-Characterization Framework for Decision-Making at FDA
BOX 2-3 Methods for Estimating Uncertain Quantities
A white paper written for EPA outlines a general hierarchy of methods
that can be used to estimate quantities when there is substantial uncertainty
about their "true" values (Frey et al. 2003). Four general categories of
methods are described without any implied preferences or priorities:
Statistical methods based on empirical data, which use classical sta-
tistics to draw inferences from "hard" data alone.
Statistical methods based on judgment, in which expert judgments
and Bayesian approaches to statistical analysis are included, often in com-
bination with "hard" data.
Other quantitative methods that involve approaches not based on
probability theory, such as interval methods, fuzzy methods, and meta-
analytic methods.
Qualitative methods that can be used when key aspects of uncer-
tainty cannot be captured by quantitative methods (for example, uncertainty
caused by problem formulation or the existence of competing models).
Of those approaches, the first is widely used and accepted in risk as-
sessment, the second is widely used outside the risk-assessment commu-
nity and is expanding in use and acceptance in the risk-assessment com-
munity, and the third and fourth approaches are not generally used.
EPA discussed the use of expert elicitation (or expert judgment) and
made the following comment:
Expert elicitation is recognized as a powerful and legitimate quantitative
method for characterization of uncertainty and for providing probabilistic
distributions to fill data gaps where additional research is not feasible.
The academic and research community, as well as numerous review
bodies, have recognized the limitation of empirical data for characteri-
zation of uncertainty and have acknowledged the potential for using
[expert elicitation] for this purpose. In Science and Judgment in Risk
Assessment [NRC 1994] the NAS [National Academy of Sciences] rec-
ognized that for "parameter uncertainty, enough objective probability
data are available in some cases to permit estimation of the probability
distribution. In other cases, subjective probabilities might be needed."
In this "Blue Book" report, the NAS further recognized the "difficulties of
using subjective probabilities in regulation" and identified perceived
bias as one major impediment; but, noted that "in most problems real or
perceived bias pervades EPA's current point-estimate approach." In
addition, the NAS stated that "there can be no rule that objective prob-
ability estimates are always preferred to subjective estimates, or vice
versa" (EPA 2009, p. 5).
Thus, EPA recognized the importance of using statistical methods based on
judgment to derive probability estimates and emphasized that objective
methods are not always preferable to subjective methods.
OCR for page 17
A Risk-Characterization Framework 37
Characterizing Uncertainty
Although substantial effort has been devoted to motivating and guiding the
inclusion of uncertainty in policy analysis throughout the federal government
(see Box 2-4), including it is difficult, and resistance remains high. Individual
researchers and policy analysts are reluctant to make quantitative estimates with-
out what they feel to be "sufficient" data, and they are even more reluctant to
quantify the uncertainties inherent in their domain. It is always possible to col-
lect more data and do more analyses to try to develop "better" estimates, but
there will always be uncertainty, and decisions often must be made on the basis
of existing information. Quantifying what is known and what is not known (the
uncertainty) is an important way to ensure that decisions are as well informed as
The difficulty of developing best estimates and uncertainties when data are
sparse is exacerbated by the norm of using probability distributions for quantifi-
cation. Probability is not a natural way of thinking about uncertainty for every-
one, and often efforts are made to sidestep the problem by using qualitative or
categorical measures of uncertainty, such as "likely," "very unlikely," and "pos-
sible." That approach may make the risk-characterization task more palatable,
but it also makes it considerably less useful. The very ambiguity that provides
comfort makes the task of communicating and comparing uncertainties difficult.
Flexible definitions that vary from domain to domain make cross-cutting analy-
sis impossible. Is the probability value associated with a "high likelihood" of
death equal (either numerically or cognitively) to the probability value associ-
ated with a "high likelihood" of rain? The effort needed to define the uncertainty
categories unambiguously and in sufficient detail would certainly approach if
not exceed the effort needed to follow one of the standardized approaches of
assessing a distribution directly. In addition, for some problem domains, assess-
ing the complete distribution is not necessary; a few key points selected from the
underlying distribution will describe the underlying uncertainty sufficiently to
permit the necessary comparisons and analyses. That approach is demonstrated
in the case studies that follow this chapter.
Options for succinctly describing a probability distribution vary; however,
the shorthand notation should be able to provide an indication of a distribution's
first three moments—central tendency, spread, and skew—and should be easily
assessable from data, model output, or expert judgment. In the case studies that
follow, that task is accomplished by using three values: the distribution's
5th, 50th (median), and 95th percentiles. The 5th percentile is the numerical
value of the quantity being estimated (such as the number of deaths) that
bounds the lower 5% tail of the distribution; for example, the probability that the
actual value will be lower than the 5th percentile is 0.05. The 95th percentile is
the numerical value that will be exceeded with a probability of 0.05; that is, it is
the boundary of the upper 5% tail of the distribution. The median value is a
measure of the distribution's central tendency and generally represents the
OCR for page 17
38 A Risk-Characterization Framework for Decision-Making at FDA
BOX 2-4 Importance of Characterizing Uncertainty
The assessment, quantification, and communication of uncertainty are
essential, as has been recognized in the risk-assessment literature since at
least 1983. Numerous high-level government advisory panels have dis-
cussed the importance of capturing and presenting uncertain values. Risk
Assessment in the Federal Government: Managing the Process (NRC
1983) formalized the integration of uncertainty and risk into policy analysis.
The guidance provided in that early report has been expanded on in a se-
ries of reports (NRC 1993, 1994, 1996) and recently reinforced in Science
and Decisions: Advancing Risk Assessment (NRC 2009, p. 93), which
states at the beginning of a chapter on uncertainty and variability that "char-
acterizing uncertainty and variability is key to the human health risk-
assessment process, which must engage the best available science in the
presence of uncertainties and difficult-to-characterize variability to inform
risk-management decisions." In recent guidelines on probabilistic risk as-
sessment, the U.S. Nuclear Regulatory Commission states that risk-
informed decision-making requires the "appropriate consideration of uncer-
tainty…in the analyses used to support the decision and in the interpretation
of the findings of those analyses" (USNRC 2009, page iii).
Over the last 25 years, EPA has developed a series of guidelines that
detail how uncertainty should be integrated into regulatory policy-making
and evaluation (EPA 1984, 1995, 2000, 2004). Estimating the Public Health
Benefits of Proposed Air Pollution Regulations (NRC 2002) offers advice to
EPA on the importance of characterizing uncertainty even when data are
sparse or lacking:
EPA should move the assessment of uncertainty from its ancillary
analyses into its primary analyses to provide a more realistic depiction
of the overall degree of uncertainty. This shift will entail the develop-
ment of probabilistic, multiple-source uncertainty models based not only
on available data but also on expert judgment … Uncertainty should be
described as completely and as realistically as possible for all regula-
tory options, recognizing that regulatory action might be necessary in
the presence of substantial uncertainty. The regulatory decision proc-
ess will be better informed by a fair assessment of the uncertainty and
a realistic evaluation of the likely reductions in that uncertainty attain-
able through further research (NRC 2002, p. 11).
Explicit evaluation and presentation of uncertainty in risk assessments
reduces the problem of false precision, makes risk characterization more
informative, and increases the credibility of any ensuing risk communication.
"best estimate" of the uncertain quantity. The probability that a value falls below
the median is equal to the probability that it falls above the median. The spread
of the distribution is indicated by the difference between the 5th and 95th per-
OCR for page 17
A Risk-Characterization Framework 39
centiles. The skew of the distribution is indicated by comparing the difference
between the 5th percentile and the median and the difference between the me-
dian and the 95th percentile. For a symmetric distribution, those differences will
be equal. For a negative skew, the first difference will be larger; for a positive
skew, the second difference will be larger.
If the three percentiles are relied on to characterize a distribution, the as-
sessment procedure for experts is relatively simple and well tested. With training
and practice, an analyst can elicit an accurate representation of the three percen-
tiles from experts by using easily described thought experiments, simple tools,
and standard protocols. Numerous references describe a variety of approaches
for assessing quantitative estimates of uncertain quantities, including examples
of how expert assessments have been used to support a wide array of decisions
(Merkhofer 1987; Cooke 1991; Hora 2007; EPA 2009; Jenni and van Luik
2010). It is important to note that a viable approach will require the considera-
tion of the tails of the distribution. Assessed distributions are often too narrow,
but considering rarer and more extreme events leads to assessed distributions
that represent true uncertainties better.
Although the three percentiles are precisely defined, their cognitive inter-
pretation should not be lost. The goal is to have a risk characterization that in-
cludes representation of the uncertainty in the size of the exposed population, in
the number of deaths that may occur, and in the number of injuries or illnesses
that may occur under various decision options. Rather than ask simply for a
range or for low and high estimates of those quantities, the committee suggests
using the concepts associated with the percentiles of a distribution as a way to
ensure consistent assessment and interpretation of the low and high values. The
5th percentile represents a value below which the actual value is not likely to
fall. Similarly, the 95th percentile represents a value above which the actual
value is not likely to fall. The estimates are not upper and lower bounds, but
values that indicate limits beyond which results would be surprising. Fixating on
the precise values for the percentiles could cause unnecessary anxiety and com-
municate false precision where none is intended or needed.
Uncertainty in the health consequences of alternative decisions should be
included in the characterization of risks in this framework. Specifically, a best
estimate, a high estimate, and a low estimate—corresponding to the three per-
centiles described above—should be determined for the size of the exposed
population, mortality, and each of the three morbidity estimates. The other at-
tributes could also be described by using similar distributional measures al-
though the importance of doing so is less. When assessing the probabilities as-
sociated with ability to detect, control, or mitigate, ranges may be appropriate
instead of point values. For example, the probability that an informed institution
will be able to reduce or mitigate adverse health effects associated with a par-
ticular risk could be judged to be 0.80-0.90.
The committee notes that there is likely to be a relationship between the
quality of data available to develop the necessary estimates and the uncertainty
OCR for page 17
40 A Risk-Characterization Framework for Decision-Making at FDA
in the estimates. Specifically, when relevant data are lacking, uncertainty about
each of the outcomes will probably be high (that is, a large spread).
The framework proposed here is similar to the framework for risk-based
decision-making described in Science and Decisions (see Figure 2-1) in its em-
phasis on structuring and conducting risk assessments and risk characterization
in the broader context of supporting agency decisions. Science and Decisions
concludes by making recommendations for improving risk-based decision-
making at EPA; although some of those recommendations may be applicable to
FDA, the present committee was charged not with evaluating the risk-based
decision-making processes at FDA but with developing a robust approach for
characterizing public-health consequences. Those consequences are an impor-
tant consideration in FDA decisions, and the attributes proposed in this frame-
work provide such a characterization. Using this framework throughout the
agency will ultimately lead to robust and consistent characterization of public-
health consequences.
How the information is used will ultimately be determined by FDA. Pub-
lic-health consequences constitute only one of numerous factors—such as eco-
nomic, social, and political factors—that FDA must consider when making deci-
sions, and the committee neither expects nor recommends that the attributes in
this report should form the sole basis of such decisions. However, careful and
consistent evaluation of the public-health consequences of various options is an
essential component of good decision-making, and there are several ways in
which the risk characterizations described here could be used in decision-
One approach was outlined in the committee's letter report, which focused
on developing a risk ranking (see Appendix A). The letter report describes an
approach for involving stakeholders in an exercise to rank risks that used a con-
sistent set of attributes. Specifically, it involved ranking risks on the basis of
judgment and the described characteristics of those risks or ranking risks on the
basis of a mathematical combination of the attribute scores and judgments about
the importance of the different attributes, that is, using the logic of multiattribute
utility theory (Keeney and Raiffa 1976). The approach described in the letter
report is based on the risk-ranking literature described previously and could be
adapted and applied in the decision-focused framework proposed here.
The risk characterization of Step 2 of the present framework is also com-
patible with several other approaches to decision-making—both more quantita-
tive approaches and more inclusive processes. The analytic-deliberative process
described in Understanding Risk (NRC 1996) includes scientists, public offi-
cials, and other interested and affected parties in an iterative process in which all
parties are involved in every step of the decision process. If FDA adopted that
type of approach to risk-based decision-making, the risk attributes defined here
OCR for page 17
A Risk-Characterization Framework 41
could provide a starting point for the risk-characterization aspects of the analy-
sis. Retaining the flexibility to refine the risk characterization throughout the
process, however, is key to the analytic-deliberative process.
At perhaps a different extreme, FDA may determine that for some deci-
sions only a single public-health consequence is relevant. In that case, the other
elements of the risk characterization could be ignored. Alternatively, the quanti-
tative public-health consequences as a group may be considered relevant, and it
may be considered useful to combine the attributes in the risk characterization
into a single metric to compare options. Such integrative measures may be easier
to develop, and especially valuable, when there is a logical set of mathematical
relationships between the various attributes. For example, two rare causes of
morbidity associated with a particular product might reasonably be considered
additive in their effects; other attributes, such as the size of a population affected
and the rate of adverse effects or the number of adverse effects and the likeli-
hood that harm could be mitigated, would combine in a multiplicative fashion.
Proven mathematical approaches for combining attributes into a single integra-
tive measure (Keeney and Raiffa 1976; Fischhoff et al. 1984) could also be ap-
plied here. However, the attributes defined here were chosen to provide a robust
risk characterization rather than an easy mathematical combination, so develop-
ing a single integrative measure would require careful consideration of the rela-
tionship between the various attributes and their relative importance. As dis-
cussed in Box 2-1, using integrative measures alone has several disadvantages,
but relying only on the full list of attributes without integration can make consis-
tency a challenge.
Each case study that follows concludes with a discussion of how the pre-
sent framework could be used to support a specific decision, but they are offered
only for illustration. It remains for FDA to decide how to use the information,
and whether and how to combine it with other decision-relevant factors.
The risk-characterization framework proposed here is a flexible system.
Although it is intended to capture elements of risk that are applicable across the
broad array of risk-related decisions that FDA makes, it is not necessarily a
comprehensive set of public-health-related factors relevant for every such deci-
sion, and it clearly does not aim to include factors that are unrelated to public-
health outcomes. The framework is not proposed as a "one-size-fits-all" method
for all risk-related decisions at FDA for all time. Just as the framework and the
risk attributes were refined through development of the case studies in this re-
port, the committee expects that some elements of the framework will continue
to evolve as FDA gains experience in using it. Additional risk characteristics
may be identified for subsets of decisions and added to the list for those types of
decisions. Furthermore, the existing definitions of attributes may be modified to
OCR for page 17
42 A Risk-Characterization Framework for Decision-Making at FDA
yield more interesting insights, and some attributes may be dropped altogether if
they are not found to be useful.
AHRQ (Agency for Healthcare Research and Quality). 2005. U.S. Valuation of the Eu-
roQol EQ-5D Health States. Agency for Healthcare Research and Quality, Rock-
ville, MD. December 2005 [online]. Available: http://www.ahrq.gov/rice/EQ5D
proj.htm [accessed Oct. 25, 2010].
Bertoni, M.J. 2010. Opening Remarks. Presentation at the 5th Meeting on Ranking FDA
Product Categories Based on Health Consequences, Phase II, February 3, 2010,
Washington, DC.
Bronfman, N.C., L.A. Cifuentes, M.L. DeKay, and H.H. Willis. 2007. Accounting for
variation in the explanatory power of the psychometric paradigm: The effects of
aggregation and focus. J. Risk Res. 10(4):527-554.
Bronfman, N.C., L.A. Cifuentes, and V.V. Gutierrez. 2008a. Participant-focused analysis:
Explanatory power of the classic psychometric paradigm in risk perception. J. Risk
Res. 11(6):735-753.
Bronfman, N.C., E.L. Vazquez, V.V. Gutierrez, and LA. Cifuentes. 2008b. Trust, accep-
tance and knowledge of technological and environmental hazards in Chile. J. Risk
Res. 11(6):755-773.
Broome, J. 1993. QALYs. J. Public Econ. 50(2):149-167.
Cooke, R.M. 1991. Experts in Uncertainty: Opinion and Subjective Probability in Sci-
ence. K. Shrader-Frechette, ed. New York: Oxford University Press.
Davies, J.C. 1996. Comparing Environmental Risks: Tools for Setting Government Pri-
orities. Washington, DC: Resources for the Future.
DeKay, M.L., H.K. Florig, P.S. Fischbeck, M.G. Morgan, K.M. Morgan, B. Fischhoff,
and K.E. Jenni. 2001. The use of public risk ranking in regulatory development.
Pp. 208-230 in Improving Regulation: Cases in Environment, Health, and Safety,
P.S. Fischbeck, and R.S. Farrow, eds. Washington, DC: Resources for the Future.
DeKay, M.L., P.S. Fischbeck, H.K. Florig, M.G. Morgan, K.M. Morgan, B. Fischoff, and
K.E. Jenni. 2005. Judgement-Based Risk Ranking for Food Safety Pp. 198-226 in
Toward Safer Food: Perspectives on Risk and Priority Setting. S.A Hoffman and
M.R. Taylor, eds. Washington, DC: RFF Press.
EPA (U.S. Environmental Protection Agency). 1984. Risk Assessment and Management:
Framework for Decision Making. EPA 600/9-85-002. U.S. Environmental Protec-
tion Agency, Washington, DC. December 1984.
EPA (U.S. Environmental Protection Agency). 1987. Unfinished Business: A Compara-
tive Assessment of Environmental Problems. EPA/230/2-87/025. Office of Policy,
Planning, and Evaluation, U.S. Environmental Protection Agency, Washington,
EPA (U.S. Environmental Protection Agency). 1995. Policy for Risk Characterization at
the U.S. Environmental Protection Agency. U.S. Environmental Protection
Agency, Washington, DC. March 1995 [Appendix A in EPA 2000].
EPA (U.S. Environmental Protection Agency). 2000. Risk Characterization Handbook.
EPA 100-B-00-002. Office of Science Policy, Office of Research and Development,
U.S. Environmental Protection Agency. December 2000 [online]. Available:
http://itp-pfoa.ce.cmu.edu/docs/RChandbk_excerpts_re_exposure.pdf [accessed Oct.
25, 2010].
OCR for page 17
A Risk-Characterization Framework 43
EPA (U.S. Environmental Protection Agency). 2004. An Examination of EPA Risk As-
sessment Principles and Practices, Staff Paper Prepared for the U.S. Environmental
Protection Agency by Members of the Risk Assessment Task Force. EPA/100/B-
04/001. Office of the Science Advisor, U.S. Environmental Protection Agency,
Washington, DC. March 2004 [online]. Available: http://www.epa.gov/osa/pdfs/
ratf-final.pdf [accessed Oct. 25, 2010].
EPA (U.S. Environmental Protection Agency). 2009. Expert Elicitation Task Force White
Paper, External Review Draft. Science Policy Council, U.S. Environmental Protec-
tion Agency. January 6, 2009 [online]. Available: http://www.epa.gov/osa/pdfs/
elicitation/Expert_Elicitation_White_Paper-January_06_2009.pdf [accessed Feb.
23, 2011].
EPA SAB (U.S. Environmental Protection Agency Science Advisory Board). 1990. Re-
ducing Risk: Setting Priorities and Strategies for Environmental Protection. SAB-
EC 90-021. Science Advisory Board, U.S. Environmental Protection Agency
[online]. Available: http://yosemite.epa.gov/sab/sabproduct.nsf/28704D9C420FCB
_5-11-1995_204.pdf [accessed Mar. 17, 2011].
EuroQoL Group. 1990. EuroQol – A new facility for the measurement of health-related
quality of life. Health Policy 16(3):199-208.
FDA/USDA (U.S. Food and Drug Administration and U.S. Department of Agriculture). 2003.
Listeria monocytogenes Risk Assessment: Interpretive Summary. Center for Food and
Applied Nutrition, U.S. Food and Drug Administration, and Food Safety and Inspection
Service, U.S. Department of Agriculture. September 2003 [online]. Available:
ssment/ucm185291.htm [accessed Feb. 23, 2010].
Fischhoff, B. 1984. Setting standards: A systematic approach to managing public health
and safety risks. Manage. Sci. 30(7):823:843.
Fischhoff, B., S.R. Watson, C. Hope. 1984. Defining risk. Policy Sciences. 17(2):123-
Fischhoff, B. 1995. Ranking risks. Risk Health Saf. Environ. 6:189-200.
Fischhoff, B. 2006. Cognitive processes in stated preference methods. Pp. 937-968 in
Handbook of Environmental Economics, Vol. 2. Valuing Environmental Changes,
K.G. Mäler, and J.R. Vincent, eds. Amsterdam: Elsevier.
Florig, H.K., M.G. Morgan, K.M. Morgan, K.E. Jenni, B. Fischoff, P.S. Fischbeck, and
M.L. DeKay. 2001. A deliberative method for ranking risks (I): Overview and test
bed development. Risk Analysis. 21(5):913.
FoodRisk.org. 2011. FDA's Fresh Produce Risk Ranking Tool. Joint Institute for Food
Safety and Applied Nutrition, College Park, MD [online]. Available:
http://www.foodrisk.org/exclusives/RRT/ [accessed Feb.11, 2011].
Frey, C., D. Crawford-Brown, Z. Junyu, and D. Loughlin. 2003. Hierarchy of Methods to
Characterize Uncertainty: State of Science of Methods for Describing and Quanti-
fying Uncertainty. EPA Contract No. 68-D-00-265. Prepared for Office of Air
Quality Planning and Standards, U.S. Environmental Protection Agency, Research
Triangle Park, NC. August 12, 2003 [online]. Available: http://www4.ncsu.edu/
~frey/reports/Frey_CrawfordBrown_Zheng_Loughlin_2003.pdf [accessed Feb. 23,
Gutiérrez, V.V., L.A. Cifuentes, and N.C. Bronfman. 2006. The influence of information
delivery on risk ranking by lay people. J. Risk Res. 9(6):641-655.
OCR for page 17
44 A Risk-Characterization Framework for Decision-Making at FDA
Hora, S.C. 2007. Eliciting probabilities from experts. Pp. 129-153 in Advances in Deci-
sion Analysis: From Foundations to Applications, W. Edwards, R.F. Miles, Jr., and
D. von Winterfeldt, eds. New York: Cambridge University Press.
IOM (Institute of Medicine). 2006. Valuing Health for Regulatory Cost-Effectiveness
Analysis. Washington, DC: The National Academies Press.
IOM (Institute of Medicine). 2009. Initial National Priorities for Comparative Effective-
ness Research. Washington, DC: The National Academies Press.
Jenni, K.E. 1997. Attributes for Risk Evaluation. Ph.D. Dissertation, Carnegie Mellon
University, Pittsburgh, PA.
Jenni, K.E., and A. van Luik. 2010. Assessment of expert judgments for safety analyses
and performance assessment of geologic repository systems. Pp. 580-609 in Geo-
logical Repository Systems for Safe Disposal of Spent Nuclear Fuels and Radioac-
tive Waste, J. Ahn, and M.J. Apted, eds. Cambridge: Woodhead Publishing.
Jones, K. 1997. A Retrospective on Ten Years of Comparative Risk. Prepared for Ameri-
can Industrial Health Council, Washington, DC, by Green Mountain Institute for
Environmental Democracy, Montpelier, VT. January 24, 1997 [online]. Available:
http://www.heartland.org/custom/semod_policybot/pdf/4728.pdf [accessed March
17, 2011].
Keeney, R.L. and H. Raiffa. 1976. Decisions with Multiple Objectives: Preferences and
Trade-Offs. New York: Wiley.
Long, J. and B. Fischoff. 2000. Setting risk priorities: A formal model. Risk Analysis.
Lopez, A.D., C.D. Mathers, M. Ezzati, D.T. Jamison, and C.J.L. Murray, eds. 2006.
Global Burden of Disease and Risk Factors. Washington, DC: World Bank
[online]. Available: http://www.dcp2.org/pubs/GBD [accessed Feb. 14, 2011].
Merkhofer, M.W. 1987. Quantifying judgmental uncertainty: Methodology, experiences,
and insights. IEEE T. Syst. Man Cyb. 17(5):741-752.
Minard Jr., R.A. 1996. Can States Make a Market for Environmental Goals? Pp. 243-280
in Linking Science and Technology to Society's Environmental Goals. National
Research Council. Washington, DC: The National Academies Press.
Morgan, M.G., B. Fischhoff, L. Lave, and P. Fischbeck. 1996. A proposal for ranking
risks within federal agencies. Pp. 111-148 in Comparing Environmental Risks:
Tools for Setting Government Priorities, J.C. Davies, ed. Washington, DC: Re-
sources for the Future.
Morgan, K.M., M.L. DeKay, and P.S. Fischbeck. 1999. A multi-attribute approach to risk
prioritization. Risk Policy Report 6(6):38-40.
Morgan, M.G., H.K. Florig, M.L. DeKay, and P. Fischbeck. 2000. Categorizing risks for
risk ranking. Risk Analysis. 20(1):49-58.
Morgan, K.M., M.L. DeKay, P.S. Fischbeck, M.G. Morgan, B. Fischhoff, and H.K.
Florig. 2001. A deliberative method for ranking risks (II): Evaluation of validity
and agreement among managers. Risk Anal. 21(5):923-937.
NRC (National Research Council). 1983. Risk Assessment in the Federal Government:
Managing the Process. Washington, DC: National Academy Press.
NRC (National Research Council). 1989. Improving Risk Communication. Washington,
DC: National Academy Press.
NRC (National Research Council). 1993. Pesticides in the Diets of Infants and Children.
Washington, DC: National Academy Press.
NRC (National Research Council). 1994. Science and Judgment in Risk Assessment.
Washington, DC: National Academy Press.
OCR for page 17
A Risk-Characterization Framework 45
NRC (National Research Council). 1996. Understanding Risk: Informing Decisions in a
Democratic Society. Washington, DC: National Academy Press.
NRC (National Research Council). 2002. Estimating the Public Health Benefits of Pro-
posed Air Pollution Regulations. Washington, DC: The National Academies Press.
NRC (National Research Council). 2009. Science and Decisions: Advancing Risk As-
sessment. Washington, DC: National Academy Press.
Slovic, P. 1987. Perception of risk. Science 236(4799):280-285.
Slovic, P. 1992. Perception of risk: Reflections on the psychometric paradigm. Pp. 117-
152 in Social Theories of Risk, S. Krimsky, and D. Golding, eds. New York: Prae-
USNRC (U.S. Nuclear Regulatory Commission). 2008. Risk-Informed Decisionmaking
for Nuclear Material and Waste Applications: Revision 1. U.S. Nuclear Regulatory
Commission, Washington, DC [online]. Available: http://www.hss.energy.gov/nu
clearsafety/ns/rawg/docs/RIDM_2008_2.pdf [accessed Apr. 13, 2011].
USNRC (U.S. Nuclear Regulatory Commission). 2009. Guidance on the Treatment of
Uncertainties Associated with PRAs in Risk-Informed Decision Making Main Re-
port, Volume 1. NUREG-1855. U.S. Nuclear Regulatory Commission, Washing-
ton, DC [online]. Available: http://www.nrc.gov/reading-rm/doc-collections/nure
gs/staff/sr1855/v1/sr1855v1.pdf [accessed Apr. 13, 2011].
Willis, H.H., M.L. DeKay, M.G. Morgan, H.K. Florig, and P.S. Fischbeck. 2004. Eco-
logical risk ranking: Development and evaluation of a method for improving pub-
lic participation in environmental decision making. Risk Analysis. 24(2):363-378.
Willis, H.H., M.L. DeKay, B. Fischoff, and M.G. Morgan. 2005. Aggregate, disaggre-
gate, and hybrid analyses of ecological risk perceptions. Risk Analysis. 25(2):405-


text:
Adakole, J.A. and Mbah, C.E. (2005) Seasonal variation of some trace metal concentrations in some fish species on stretch of River Kubanni, Zaria. In: 19th Annual Conference of the Fisheries Society of Nigeria (FISON) , 29 Nov - 03 Dec 2004 ,Ilorin, Nigeria, pp. 774-780.
(The document's language is
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat
Download (450Kb) | Preview |
In a semi-urbanized stretch of river Kubanni, Zaria, seasonal variations of some tree metals (copper, zinc and lead) in fish species (Clarias sp., Tilapia sp and Alestes sp) were investigated over a period of eight months. Water and fish were sampled monthly, pooled separately and seasonal analysis of each trace determined using atomic absorption method. The concentration of each of the elements in water higher in the dry season than during the rainy season. The dry and rainy season concentrations of copper, zinc and lead were 6.85~c10.66 mu gg super(-1)&1.45~c1.10 mu gg super(-1); 2.13~c1.68 mu gg super(-1)&0.1 5~c0.05 mu gg super(-1); and 0.52~c0.50 mu gg super(-1) & 0.31~c0.14 mu gg super(-1) respectively. Similarly, all the three accumulated more zinc and lead in the dry season than during the rainy season. Tilapia species and to some extent, Alestes species being pelagic had a higher concentration of trace metals than Clarias species which is a bottom dwelling fish Tilapia species may therefore be a better indicator species than Clarias species for monitoring trace metals in water bodies. Reasons for the observed variations in the trace metals were discussed
Item Type: | Conference or Workshop Item | ||||||
Title: | Seasonal variation of some trace metal concentrations in some fish species on stretch of River Kubanni, Zaria | ||||||
Personal Creator/Author: |
Page Range: | pp. 774-780 | ||||||
Date: | 2005 | ||||||
Event Title: | 19th Annual Conference of the Fisheries Society of Nigeria (FISON) | ||||||
Event Type: | Conference | ||||||
Event Location: | Ilorin, Nigeria | ||||||
Event Dates: | 29 Nov - 03 Dec 2004 | ||||||
Issuing Agency: | Fisheries Society of Nigeria | ||||||
Uncontrolled Keywords: | Nigeria; fish; seasonal variations; trace metals | ||||||
Subjects: | Fisheries Chemistry |
Item ID: | 4114 | ||||||
Depositing User: | Ms Maria Kalentsits | ||||||
Date Deposited: | 31 Aug 2010 13:11 | ||||||
Last Modified: | 29 Sep 2011 16:32 | ||||||
Related URLs: | |||||||
URI: | http://aquaticcommons.org/id/eprint/4114 |
Actions (login required)
View Item |


text:
Experts Guide
Title | Date Posted | |
Joseph Braun Joseph Braun is a nuclear engineer with over 40 years of experience in the design, operation and safety of U.S. nuclear power plants. He teaches nuclear technology at Argonne training courses conducted for the International Atomic Energy Agency. |
January 30, 2013 | |
David F. Brown David F. Brown has been with Argonne since 1993 and is a principal investigator in projects that include the modeling of chemical and biological agent releases in subways and interior facilities; modeling the release and atmospheric dispersion from chemical accidents; and quantitative risk assessment for hazardous materials transportation. |
February 5, 2013 | |
Daniel Schabacker Daniel Schabacker develops biochips that use proteins and nucleic acids to track bioweapons and diagnose illnesses. |
February 18, 2013 | |
Richard Vilim Richard Vilim is an expert in diagnostics, instrumentation and control, with over 30 years experience developing new applications for the process and power industries. |
February 19, 2013 | |
Vladimir Koritarov Vladimir Koritarov has 30 years of experience in the analysis and modeling of electric and energy systems in domestic and international applications. |
February 19, 2013 | |
Krishna Shenai Krishna Shenai is a Principal Electrical Engineer within the Energy Systems Division at Argonne. |
March 25, 2013 | |
Temitope Taiwo Temitope Taiwo is responsible for the supervision of research groups working on reactor physics analysis, advanced modeling and simulation, fuel cycle and systems dynamics modeling, criticality safety and nuclear data. |
June 28, 2013 | |
Nicola Ferrier Nicola Ferrier's research interests include the use of computer vision (digital images) to control robots, machinery and devices, with applications as diverse as medical systems, manufacturing and projects that facilitate scientific discovery (such as her project using machine vision and robotics for plant phenotype studies). |
June 28, 2013 | |
Richard Brotzman Richard Brotzman is a principal nanotechnology development engineer in Argonne's Energy Systems Division. |
June 28, 2013 | |
Audun Botterud Audun Botterud is an Energy Systems Engineer in the Center for Energy, Environmental and Economic Systems Analysis. |
June 28, 2013 |