text:
Forgot Password? | Not a User? Sign up
Your browser does not support iframes.
Sports Editor
Jefferson Forest boys basketball took on Heritage for the last regular season game on Thursday, Feb. 11. Heritage defeated JF 52-50.
"We played very well, we just didn't make enough plays down the stretch for a win," said Coach Cameron Shepherd.
The Jefferson Forest boys swim team finished first and the girls finished fourth place at the Region 4A swim meet Saturday, Feb. 13. Forest will see 20 swimmers advancing to the State meet.
JF boys finished with a score of 246.5. The second place team, John Champe, scored 200.
The Staunton River girls basketball team defeated district rival, Rockbridge on Tuesday, Feb. 9. After a strong first half the Lady Golden Eagles hung on for a 53-52 win over Rockbridge.
Staunton River started with a commanding 15-8 lead over Rockbridge. In the second period River increased its lead to 31-14 by the half.
Sports Editor
The Staunton River boys basketball team played Lord Botetourt in a make-up game for its last regular season game on Thursday, Feb. 11. River fell 77-70 to LB.
"We played really well. The guys really got after it against a really good team," said Coach Brandon Harris.
By Margaret Pascale
Sports Editor
It was Senior night for the boys and girls basketball teams, as both looked to face off against E.C. Glass for a rematch and the final game of the regular season.
E.C. Glass closed the gap in the third period as the Liberty girls basketball team struggled to get shots to fall. Liberty defeated Glass 45-38 on Friday, Feb. 12.
The Jefferson Forest girls basketball team almost lost their final regular season game in the third period. The Lady Cavaliers rallied in the fourth and secured a 53-44 victory over Heritage last Thursday, Feb. 11.
Tyler Beck of Bedford, left, Peyton McDaniel of Pulaski, Reed Pendleton of Roanoke, Maggie Rooklyn of Clifton Forge, Kaylee Hughes of Martinsville, and Miranda Reynolds, right, of Martinsville placed first at the Elks Lodge Southwest Hoop Shoot on January 30 at Bedford Middle School.
Tyler Beck holding his first place trophy after placing first in the 8 and 9 year old group. Beck will move on to the Virginia State Hoop Shoot held on Sunday, Feb. 14 at St. Anne's Belfield School in Charlottesville.
Three Staunton River girls soccer players signed with their respective college on Wednesday, Feb. 3. Savannah Cowan, left, signed a non-binding contract with Division III Bridgewater College. Emily Hillman, middle, signed with Virginia Commonwealth University. Lizzie Boone, right, signed with Division II King University. This will be Boone's first year playing as a Golden Eagle. Last year Boone was 3A-all state with the Lady Minutemen.
Two Rivers collided on the Golden Eagles courts Friday, Feb. 5 as SR girls basketball hosted James River in a non-district match up.
Staunton River took control of the game and defeated the visiting team 68-32.
The Lady Golden Eagles flexed hard in the first and took a 17-8 lead in the first. Determined to get ahead early SR scored another 17 points in the second and increased its lead to 34-13 by the half.
Copyright © 2016
The Bedford Bulletin is your source for local news, sports, events, and information in Bedford County and Bedford, VA and the surrounding area.


text:
Come with me by GeneRazARTWatch
©2014-2016 GeneRazART
Copyright © by Genesis Raz von Edler/RazielMB. All rights reserved.
My Artworks are registered and protected by Copyrighted.com.
My artwork may not be reproduced, copied, edited, published or uploaded in any way without my written permission. My images do not belong to the public domain.
Copyright number VZA4-CNIZ-OOA1-GGQY
Stock credits:
Dragoroth-stock ~ fav.me/d59arht
Chunga-Stock ~ fav.me/d2o4qzg
kuschelirmel-stock ~ fav.me/d68vah2
SalsolaStock ~ fav.me/d5bbpkv
Everything else is painted and my own resources.
Tools: PS CS6, Knoll Light Factory and Wacom Intous.
Featured here
Well Done, you have Won 2ND Place with your work on G2's Daily Tag Competition:319
Here is your Award: Click Here
Please Visit Us on the G2 Forum: Click Here
Thank You from the G2 Forum, G2 Portal, G2 Networks and the G2 DA Group.
We Appreciate your Contribution as your work has been nominated and entered into our
Daily Tag Competition:319 [24HR Voting] at the G2 Forum DeviantArt Group :
Click Here to Visit the DTC Poll.
Please Visit Us at the G2 Forum:
Click Here to Visit the G2 Site.
Fantastico..me gusta mucho!!
Gracias!! Me alegra mucho que te guste !
Very impressive work and title .....
No fear!! Just love it.
Stunning !
Wow great work.
Love this
Excellent manipulation work
wow..brilliant work!!!!
I sometimes like to imagine that this will be how death is. Nothing scary like the Grim Reaper coming to tear out your soul, just a hand reaching out to. Anyway, excellent work.
This looks incredible Raz!
Awesome work Raz!
everytime my lord of assassins and darkness
Amazing, just plain amazing work my bro!
So beautiful dear!! Love your work well done, my compliments!

subset: commoncrawl


text:
Scarleteen Confidential: Teens and Body Image
[img_assist|nid=7758|title=|desc=|link=url|url=http://www.scarleteen.com/tags/scarleteen_confidential|align=right|width=160|height=160]This is part of our series for parents or guardians. To find out more about the series, click here.
He's so perfect, and I feel too self-concious for intimacy!
I am overweight by about 30lbs, so I do have some extra "baggage". Anyway, most likely, this weight isn't going any where, because I've tried many times, and failed many times. So the problem is, my boyfriend and I have been dating for some time now, and he really is wanting to have sex (and it's not just him, I do too). However, I am too intimidated - he's got like this perfect body - tan, slim and works out a lot, and me, pale in areas, and like I said overweight. If anyone has any advice please tell me how I can overcome my unpleasant feeling of self-consciousness and be able to give into him for once (or I'm gonna be stuck with keeping my shirt on during the whole thing).


text:
Human Resources
The county council is on a journey that will see it transform by April 2016. Find out why and how the transformation will happen and get the latest updates. If you are not already logged in, you will need your pay reference number (make sure you type '10M' before the 7 digit number), NI number and date of birth.
Your Pension is Changing
The Local Government Pension Scheme (LGPS) is changing from 1 April 2014 from a Final Salary scheme to a Career Average Revalued Earnings (CARE) scheme. Changes include more flexibility around when you can leave and take your pension, and a new 50/50 option where you can pay half contributions for half a pension.
For more information, visit www.yourpensionservice.org.uk


text:
Why Work with Older Adults? As a society, we will be facing new challenges and opportunities in the coming years:
What is the Geriatric Leadership Internship (GLI)?
The Geriatric Leadership Internship (GLI) offers a geriatric social work field training experience based on a biopsychosocial model and adult learning theory with a focus on diversity issues and leadership skills. Hence, GSWEC offers a collaborative, student directed, field training experience.
Through an innovative new field training model, the GLI aims to advance preparation of geriatric social workers in order to strengthen the quality of care to a burgeoning older population.
What does the internship offer?
How does the internship work?
What have graduates said about working with older adults?
What types of people are needed?
Return to main Geriatric Social Work Education Consortium (GSWEC) page.


text:
95 dodge ram 2500 is spewing black smoke. I'm only getting about 3 miles to the gallon and it's pretty consistant wihile driving at a steady pace. Fumes smell really bad and I failed emissions testing. Check engin light comes on from time to time
by NewDodgeMan in Grovetown, GA on March 24, 2009


text:
Lucchese Since 1883 N4711-S54 (Women's)
We're sorry — no color of this style is currently available.
Free Shipping
Free Exchanges
No Taxes
100% Price Guarantee
Finish Ordering to Complete Your Exchange Request
Your prepaid return-shipping label will be sent only after you complete your replacement order.
1. Request an Exchange
2. Select a Replacement
3. Review and Finalize
We're sorry — no color of this style is currently available.
Brand: Lucchese Since 1883
Style: N4711-S54
To email a friend about the N4711-S54, click here
To be notified when new Lucchese Since 1883 styles are added, click here


text:
Mobile Site You Are Here > PFR Home > Players > D > Chuck Detwiler > News Archive
News: s-r blog:NFL Combine Results Added to Pro Football Reference
Charles Michael Detwiler
Position: DB
Height: 6-0 Weight: 185 lbs.
Born: March 6, 1947 in Rome, NY
High School: GlendoraCollege: Utah St. (school history)
Weighted Career AV (100-95-...): 11 (6880th overall since 1950)
Automatic Player Linking / Statistical Sharing Tool / Basic Player Linking
A Sports Reference Site:
About PFR |
Found a bug or have a suggestion?
Privacy Statement |
Conditions & Terms of Service |
Use of Data


text:
Do you know who gets the least pay and among the least staff support in the Seattle School District? Maybe you'd guess janitors. Or food-service workers. But it's actually the ones in charge, the members of the School Board, who are supposed to be watchdogging a $560 million-a-year business on no salary and two assistants.
Some readers have written in to say The Seattle Times is hyping the Seattle schools scandal. That the schools chief shouldn't be blamed, let alone fired, for what was low-level cronyism.
"Why is this immediately a story about whether the public servant with arguably the hardest, most thankless job in Seattle, Maria Goodloe-Johnson, should lose her job?" one reader wondered.
It's a fair question. It's made more pressing by the top-line finding of an independent investigator that essentially cleared Goodloe-Johnson of any involvement in, or knowledge of, how a $1 million-a-year minority business program went so wrong.
So why should the School Board show her the door anyway?
- Students seeking sugar daddies for tuition, rent
- Seattle-based seafood company shuts down
- What's the top spelling 'mistake' in Washington state? The answer could make you sick
- UW receiver Isaiah Renfro opens up about depression, announces he's leaving team
- So the NRA sends a questionnaire to a Seattle state senator ...
Most Read Stories
All you really need to know is contained in one e-mail, dated Jan. 15, 2009. Two of Goodloe-Johnson's top managers, Chief Financial Officer Don Kennedy and executive facilities director Fred Stephens, are discussing what to do about a bad report showing this same small-business program was rife with problems. Mind you, this is from two years ago.
"The other consideration is with regards to next week's executive session," Stephens' e-mail reads, referring to a meeting of the School Board. "If I understood you correctly, you and the Supt. didn't want us to hand out the report."
Those italics are mine. Apparently everyone understood everyone correctly. Because that embarrassing report was never handed out.
My sense is the School Board is more upset about this — that the superintendent and her staff may have hidden things from them — than about all the wasted money, lost opportunity and bad publicity put together.
It also suggests something that needs to be fixed about the way the Seattle School District is run.
Do you know who gets the least pay and among the least staff support in the entire operation? Maybe you'd guess janitors. Or food-service workers. But it's actually the ones in charge, who are supposed to be watchdogging a $560 million-a-year business.
School Board members work for per diem only — a max of $4,800 a year. The seven-member board has just two helpers, who mostly do scheduling and office support.
"The way it's set up, the board is almost totally dependent on the superintendent's staff to give them honest information," says Dick Lilly, who was on Seattle's board from 2001 to 2005. "Because of that, what's in that e-mail, alone, is cause for firing."
Since there is no pay, some School Board members keep their day jobs. It's while moonlighting that they try to oversee an enterprise about two-thirds the size of Seattle city government.
Is it any wonder the School Board seems to suffer, at times, from "an abundant lack of curiosity," as I wrote in my last column? They probably are suffering from an abundant lack of sleep.
This is not to feel sorry for them. OK, maybe a little. I'm not saying that if they got six-figure salaries like the City Council then all would be magically right with Seattle schools.
But how about hiring them an analyst or two, so they can have a chance of knowing what they're doing? Or more to the point, what the bureaucracy they're supposed to be watchdogging is doing?
A reader forwarded me a "job announcement" recently that sums it up. The School Board is seeking a professional accountant and auditor, in light of the recent scandal, to help them vet the "internal financial and governance controls of the Seattle Public Schools."
The position is for two years. The pay?
"This is a volunteer position and compensation and benefits will not be provided," the ad says.
So figure this: At the same time the School Board is reduced to begging to fulfill one of its core duties, it has learned that this troubled program in the district not only had some of its activities shielded from the board but was allowed to grow to five full-time, paid employees.
During a hiring freeze.
It's enough to make you want to fire someone.
Danny Westneat's column appears Wednesday and Sunday. Reach him at 206-464-2086 or email@example.com.


text:
* Four-Time All-American *
Career Bests (thru 2015 indoors): • 5,000 Meters: 13:36.91 • 10,000: 29:06.14 • 3,000: 7:50.81i • 1,500: 3:45.04• Mile: 4:02.12i
Record Book:Indoors3,000: 4th5,000: 6thAt Stanford: • Earned his first first-team All-America honor by placing seventh in the 3,000 at the 2014 NCAA indoor championships.
2015 Season:Indoors• Earned second-team All-America honors by placing 11th at the NCAA Championships in the 3,000 (8:03.90).• Placed second in the 3,000 at the MPSF Championships, running 7:51.26.• Ran in five meets: once in the mile, three times in the 3,000, and once in the 5,000.• Ran 4:02.83 in the mile to place fourth at the UW Indoor Preview.• Ran 13:45.25 in the 5,000 to place sixth at the Husky Classic.• The time qualified him for the NCAA Championships, but chose to concentrate on the 3,000.2014 Season: Outdoors • Finished fourth in the Pac-12 in the 10,000, running 29:35.09.• Broke personal record in the 5,000 by six seconds, when he clocked 13:36.91 at the Payton Jordan Invitational, finishing fifth in the second section.• Ran 13:58.62 at the NCAA West Prelims, for the ninth-fastest time of the competition, but did not advance to nationals.• Ran a 4:02 split for Stanford in its runner-up finish in the Penn Relays 4xMile relay.• 11th in the Pac-12 5,000 (14:21.31).• Won the 3,000 in the Big Meet, with teammates Jim and Joe Rosa conceding as a reward for Olson's pacesetting.• Set PR of 3:45.04 at the Stanford Invitational. Indoors • Earned his first first-team All-America honor by placing seventh in the 3,000 at the NCAA championships in Albuquerque, N.M., running 8:14.77.• Ws a second-team All-America after placing 12th a day earlier, in the 5,000 (14:34.40).• Set a personal best in the 3,000 while placing third at the MPSF Championships, in 7:50.81.• Broke personal bests in the 5,000, 3,000, and mile.• His 5,000 best of 13:42.52 came in a fourth-place invite finish at the Husky Classic on Feb. 14.• Ran a 7:57.78 to finish second in the invite 3,000 at Penn State National on Feb. 1.• Shattered his mile best with a 4:02.12 to place third at the UW Indoor Preview on Jan. 18.
2013 Season: Outdoors• Earned All-America honorable mention after placing 19th at the NCAA Championships in the 5,000 (14:25.43).• Placed 11th in the 10,000 at the Pac-12 Championships (30:24.86) and was 17th in the 5,000 (14:37.51).• Earned the final qualifying spot for nationals by running a 13:53.68 at the NCAA West Prelims.• Set a personal best 13:46.12 in his season 5,000 debut, at the Payton Jordan Invitational.• Won the Big Meet in a personal best 3,000 time of 8:04.59.• Set 10,000 PB of 29:06.14 in the fast section of the Stanford Invitational. Indoors• Placed second in the MPSF Championships in the 5,000 in 13:55.66.• Ran personal record 13:54.55 while finishing 18th in the invitational section of the Husky Classic.• Placed sixth in the 3,000 at MPSF's in 8:08.07.• Ran PR of 8:07.03 at the UW Invitational.• Ran 4:10.56 in the mile at the UW Preview.
2012 Season: Outdoors• Placed 10th in the 5,000 at the Pac-12 Championships, running 14:17.20.• Ran 8:07.27 in the 3,000 at the Big Meet.• Member of Stanford's 4x1,500 team that ran 15:20.09 at the Texas Relays. Indoors • Did not compete.
2011 Season: Outdoors • Did not compete. Indoors • Scored for Stanford at the MPSF Championships after running a personal best of 13:59.87 to take seventh in the 5,000.• Ran a personal best of 8:09.93 for 3,000 to place 12th at MPSF's.
Cross Country: • An All-West Region selection in 2012 and 2013.• A four-time scorer in the NCAA Championship meet.• Placed third in the NCAA West Regional.• Ran No. 2 on 2013 NCAA Championship team.
International: • Won the 2011 U.S. Junior cross country title in San Diego with an 8K time 24:28.• Advanced to the 2011 World Junior championships in Punta Umbria, Spain, but did not finish.• Competed for the U.S. team at the 2009 World Youth Track and Field Championships in Bressanone, Italy, placing 10th in the 3,000.• Competed for the NACAC Junior cross country team in Mt. Irvine, Trinidad & Tobago, placing third on the 6K course.
High School: • Captured the 2009 California Division III cross country state championship.• Also won the 2009 North Coast Section Division III title.• Two-time All-State cross country selection (2008 and `09).• Two-time Marin County Athletic League cross country champion.• A four-time All-NorCal and All-Marin County Athletic League selection in cross country and track.• Also captained the track team in 2009 and 2010, winning the MCAL and NCS Redwood Empire team championships.• Triple individual MCAL champion in the 800, 1,600 and 3,200 from 2008-10.• Won 1,600 and 3,200 NCS Redwood Empire titles in 2009 and 2010.• Won the 1,600 and 3,200 at the NCS Meet of Champions in 2009 and 2010.• Holds school records in the 1,600 (4:08.54), 3,000 (8:18), and 3,200 (8:55).• Captained his cross country team for two years.
Personal: • Majoring in human biology.• Son of Tom and Lory Olson.• Has an older sister, Kelsey.


text:
LONG BEACH, Calif. (AP) — Police in Long Beach have arrested two alleged gang members in the 2008 murders of five transients who were gunned down in a homeless encampment near a Southern California freeway.
Police Chief Jim McDonnell said Wednesday that 31-year-old David Ponce and 25-year-old Max Rafael are each charged with five special circumstance murders.
Five people were shot to death in November 2008 in an area of warehouses and apartment buildings where neighbors said people often dropped by to do drugs. Police, at the time, said they received very few phone calls and had no eyewitnesses to the crime.
The announcement comes a day after an Iraq war veteran was charged in nearby Orange County with stabbing to death four homeless men in a weeks-long rampage.


text:
Robin Ince:
Robin Ince: Věda versus zázrak?
Ničí věda kouzlo života? V tomto nevrlém, ale okouzlujícím monologu, zastává Robin Ince roli obhájce vědy. Čím více se toho dozvídáme o neuvěřitelném chování vesmíru, tím více žasneme.
Similar topics
- Comedy
- Entertainment
- Science
- Time
This talk was presented at an official TED conference, and was featured by our editors on the home page.
Related talks
155 comments
Enthusiastically agree? Respectfully beg to differ? Have your say here.
Log in to comment Don't have an account? Sign up now — it's fast and free.
Sort comments by
There are currently no comments for this talk.


text:
Distances given are in a straight line from the centre of Great Cornard but travel routes may be longer.
0.3 Miles
20 Kings Hill, Great Cornard, Sudbury CO10 0EH
Hillside Residential Home provides care for up to 44 residents. The staff ensure that residents can play an active role in the daily activities at the home, and can benefit from a dedicated activities coordinator. Hillside has a close working… more
Group / Provider:
Caring Homes Group
Care provided for Cancer Care
The Briars Residential and Care Ltd
5.8 Miles
23 Hunts Hill, Glemsford, Sudbury CO10 7RP
The Briars is a long established Residential Care Home for the elderly. The home has a unique charm of its own and is privately owned set in the village of Glemsford which has a long and interesting history that dates back to the 15th Century. The… more
1.1 Miles
Walnut Tree Lane, Sudbury CO10 1BD
Catchpole Court Care Home has a proud reputation for the care it provides to older people, either elderly frail or elderly mentally infirm. The Home is divided into three separate units, Lowry, Gainsborough and Constable. Each unit provides specialist… more
Group / Provider:
Four Seasons Health Care
Care provided for Cancer Care
6.4 Miles
off Bear Street, Nayland, Colchester CO6 4LA
Nayland House, in Colchester, specialises in elderly and dementia care for older residents within a homely environment that respects personal freedoms and an active lifestyle. The home is fully registered with the Care Quality Commission, taking the… more
Group / Provider:
Larchwood Care
Care provided for Cancer Care
0.7 Miles
Newton Road, Sudbury CO10 2RN
Chilton Croft Nursing Home, in Sudbury, welcomes a wide range of skilled care packages, with carers trained in modern techniques and with a separate EMI unit enabling them to make the most of their professionalism. An understanding and compassionate… more


text:
The BSE Sensex rose 137 points on Tuesday as lenders recovered from recent steep falls after the RBI's move to purchase bonds eased concerns about the value of the sector's debt holdings.
The markets were also buoyed by a top government official's statement that India will finance its current account deficit fully in the fiscal year ending March without drawing down on its reserves, and will also contain the fiscal deficit at 4.8 per cent of GDP.
Gains also tracked a rise in global shares as investors tended to view the U.S. shutdown as temporary, and also as something that may further delay the Federal Reserve's plans to start closing downs its monetary stimulus.
However, outlook remains cautious for October as traders approach what is expected to be a weak earnings season with Infosys Ltd's results due on October 11, while the RBI is widely seen raising rates one more time in the near term.
"Market may remain sideways for sometime but it will correct further post earnings season as earnings would witness only single-digit growth for the gone by quarter with the second half being particularly bad," said Deven Choksey, managing director, K R Choksey Securities.
The Sensex rose 0.71 per cent, or 137.38 points, to end at 19,517.15.
The broader Nifty advanced 0.78 per cent, or 44.75 points, to end at 5,780.05.
Banking stocks rose after the RBI said it would buy up to 100 billion rupees of bonds via open market operations on October 7, sparking gains in bonds.
Lenders are the biggest holders of government bonds and the Reserve Bank of India's rate hike last month had raised concerns about the holdings.
ICICI Bank and HDFC Bank gained 3.1 per cent each.
Among state-owned banks, State Bank of India rose 1.8 per cent, while Bank of India ended 5.3 percent higher.
Tata Communications surged 7.3 per cent after South Africa's Vodacom Group said on Monday it was in exclusive talks to buy Tata's unlisted local unit, Neotel, in a potential $590 million deal.
Maruti Suzuki gained 1.6 per cent on higher-than-expected sales in September.
TVS Motor Company surged 9 per cent after the company said its sales in September rose 16 per cent to 197,409 units.
Financial Technologies (India) gained 2.3 per cent on short-covering in the futures market ahead of its exclusion from the equity derivatives segment.
However among stocks that fell, Hindustan Unilever fell 1.3 per cent after its parent company Unilever warned that a slowdown in its emerging markets accelerated in the third quarter.
Apollo Tyres ended 1.8 per cent lower after falling as much as 8.9 per cent on worries about possible high debt after Cooper Tire and Rubber Co shareholders approved the U.S. company's $2.5 billion sale to the Indian firm.
(Copyright Thomson Reuters 2013)


text:
home > publishing > alj > 53.3 > full.text > Bridging the gulf: mixed methods and library service evaluation
Bridging the gulf: mixed methods and library service evaluation
Abby Haynes
Manuscript received March 2004
This is a refereed article
What is evaluation?
Evaluation here is concerned with efficacy and the extent to which objectives have been met within the provision of social amenities (Greene, 2000); most often using research methods to question people about a service they have recently experienced (Pawson & Tilley, 1997). Although some consider evaluation to be the third and final step in a linear model: needs assessment > planning > evaluation (Witkin, 1994), others regard it as a part of each step, seeing the development of services as a more cyclical, reflective process in which evaluation is combined with on-going needs assessment and planning.
Hernon distinguishes between the terms 'assessment' and 'evaluation' explaining that assessment is the process of gathering data, while evaluation is the final stage in which the data is interpreted and 'value judgments' are made (2001, pp94-95). This paper will argue that value judgments are made throughout the entire process of needs assessment, planning, service provision, assessment and analysis, and that these judgments strongly affect the nature of the service as well as the means of evaluating it. Therefore the term 'evaluation' will be used broadly with the implication that all aspects are underpinned by a set of assumptions and corresponding values - often at an unconscious level.
Evaluation methodology has attracted considerable attention from a number of disciplines (particularly psychology, social policy, sociology and education) and has been shown to be a highly flexible form of applied research. Depending on the approach used, evaluation can be either narrow or broad in focus, formulaic or dynamic, imposed or participative, purist or eclectic. Although many papers in library and information studies (LIS) recognise these issues and use evaluation methods to address program, service or system provision, there is a lack of discussion about evaluation itself as an information behaviour. Furthermore, the ideas that dominate LIS academic debate and research are not evident in the research that libraries conduct themselves. For the most part it seems that practitioners conduct 'evaluation', often contracting out the whole or parts to market research companies, while academics conduct 'research'. There appears to be gulf between the two practices and the ideas which inform them, despite the huge numbers of evaluations which are continually taking place within the very communities LIS research aims to improve. This is a deeply problematic situation in a discipline which is committed to 'excellence in professional service to our communities' (ALIA, 2002, point 6).
Why evaluate library services?
The principles which inform our discipline require us to engage in on-going service improvement in order to achieve the excellence which ALIA advocates and the 'high standards of provision and delivery of library and information services' outlined by the International Federation of Library Associations & Institutions (IFLA 2003a, section: Aims). Evaluation assists decision-making by providing information about service strengths and weaknesses, indicating where successes can be built on and areas where significant improvements could or should be made, highlighting service gaps and suggesting new directions.
Like all public and private sector services, libraries are fighting for funds and are therefore called upon to justify their expenditure and even their existence. Service efficiency and quality measurements are provided in order to petition for on-going or increased revenues, and in order to engage in dialogue with stakeholders (Sheppard, 2002).
Playing the game
But there is frequently a difference between what we say evaluation is for, and what we really use it for. Evaluation is an inherently political behaviour, often developed in direct response to the socio-economic world in which services operate, and in which the same 'piece' of information may have multiple roles (Greene, 2000). For example, Pelz (1978) regards social research as having three uses: instrumental, conceptual and symbolic. Instrumental uses affect policy in direct ways - guiding a decision or solving a particular problem, while conceptual use enlightens through new understandings. Symbolic (or tactical) uses are political - justifying action already taken or excusing inaction, 'demonstrating' supposed success or failure, or emphasising/masking significant issues. It must be acknowledged that library evaluation also takes place within this political arena and is subject to instrumental, conceptual and symbolic uses.
What are we evaluating?
Are we evaluating the service's efficiency or its quality? Customer opinion or the social value of the library within its community? Most authors distill these concepts into two classifications: customer satisfaction and service quality. Some papers use the terms interchangeably (Pothas, De Wet & De Wet, 2001), while others make a distinction, defining customer satisfaction as a time-limited, subjective reaction to recent or overall service encounters, and service quality as a 'global judgment', an objective response based on the extent to which the service meets a user's expectations (Hernon, 2001, Duffy & Ketchand, 1998). This paper supports Parasuraman's (1994, p112) revised position that customer satisfaction is an aspect of service quality - that as we try to develop better methods for understanding the value of services and ways to improve them, customer satisfaction should be one of the factors we attend to. Customer satisfaction, then, is broadly considered as the extent to which people think and feel positively about a service; but used alone it is not a good indicator for the 'big picture' of service quality (Duffy and Ketchand, 1998).
Current conceptualisations of service quality are dominated by the gap model which surveys and measures the difference between perceptions of a service and other dimensions variously identified as:
The gap model assumes that objective standards for service quality can be revealed by comparing discrepancies between these dimensions and that this, in turn, provides benchmarking measurements that enable libraries to establish priorities, assess their own annual performance and compare it to that of other libraries.
This model tells us more about the service in comparison with what users want and therefore offers greater utility for decision-making than simple satisfaction surveys (Covey, 2002), but it does not provide us with an understanding of the library's social role or the needs of its users. This paper suggests that service quality in libraries relates to a service's value for individuals and communities - the benefits it brings and the ways in which it can enhance people's lives. As Kyrillidou puts it, 'Libraries are social institutions, being part of the social capital available to a community. As such their value needs to be articulated in relation to the value they provide to the user' (2002 p43). Therefore service quality is the extent to which a service meets the information needs of its community: not only in relation to what users say they would like when they tick boxes in a survey, but what the whole potential user community would most benefit from. Quality evaluation, then, also has to consider needs assessment.
Evaluation must engage in dialog with service objectives, but it also needs to influence those objectives. Many libraries are currently evaluating how well they meet aims which were developed by executives years previously. Mission statements and objectives should articulate community needs and evolve in response to changes in those needs. This cannot be an 'objective' process as some authors claim. Concepts such as 'social capital' and 'social inclusion' which underpin many arguments in this paper are acknowledged to be politically loaded. Services are developed within complex cultural contexts and are required to respond to a multitude of social and economic pressures. Evaluation must take account of these contextual factors if it is to provide the deeper understanding necessary for significant improvements in quality; and it must call attention to them if libraries are to maintain their integrity as advocates for 'universal and equitable access to information' (IFLA, 2003a, section: core values, 2)
So, what could we be doing to evaluate our services in relation to individual and community needs more effectively? This paper returns to basics in an effort to construct a framework for thinking about evaluation. It reviews some key philosophical underpinnings in LIS, provides a broad overview of research methods, and looks at current trends in the focus of library evaluation.
Philosophical issues: the 'paradigm shift'
In 1986 Dervin and Nilan wrote a seminal paper that documented a change in the way researchers were viewing people's information needs and behaviours. The paper explained that traditional methods were 'system-centred' because they asked questions that focused on the information service (the term 'service' is used to include human services and electronic systems); and ignored crucial issues regarding people's information needs, uses and abilities. They explained that system-centred research tends to concentrate on what people prefer, and how much or little they use particular resources; and it categorises them in traditional demographic groups such as their socio-economic status. System-centred perspectives see information as an objective 'thing' that is transmitted, and which people process in a rational and consistent manner. These underlying perspectives are generally not articulated in the research however; rather they are assumed as a shared understanding of the way people and information interact, based on the traditions of 'scientific' knowledge, with the core implication 'that by knowing how users have or might use systems, one knows what their needs are or might be' (Dervin & Nilan, 1986a, p10).
Dervin and Nilan argued there was an alternative paradigm - an alternative way of seeing and thinking about things - which offers greater insights into the complex relationships between people and information. This 'user-centred' perspective regards people as complex and sophisticated information seekers and users, actively engaged in a process of making meaning, and varying their behaviours in different contexts. Furthermore, it sees information as fluid, open to multiple interpretations and meanings dependent on when, where and by whom it is being utilised. User-centred research tends to concentrate on the processes people go through in their attempts to access information and make personal sense of it. It looks at what helps and hinders this process, and classifies people by the common information interests or needs that they have in that particular situation. User-centred approaches actively question our understanding of the relationships between people, information and services, and therefore make this inquiry a focal point of the research.
What does this have to do with library evaluation? Because our beliefs and assumptions shape our values and goals, they shape our service objectives, our day-to-day practice, and they shape the methods we use to evaluate them. Service decisions about what we should do and why we are doing it are founded on our core beliefs, our philosophies about the world. As Morris, points out 'assumptions based on the traditional model have dictated the kinds of services we supply and the kinds of libraries we have created' (1994, p21). It would therefore seem reasonable to assess these assumptions if we are to make educated service decisions.
System-centred questions help us to discover the extent to which people use a service, what they know about it, how they rate it, their preferences, and their 'profile'. There will always be a need to ask these questions and reflect on the answers as part of the evaluation process: thus they continue to play an important role in research. But they are not enough. In order to understand the value of a service we need to know its purpose: how people actually use it, what difference it makes in their lives, and why they behave and feel they way they do in their service interactions. We need to ask user-centred questions and select methods which permit users to explore these questions. User-centred philosophy not only guides our questions, it suggests that users have a significant role to play in the development of research methods and the utilisation of findings. For example, Henry (1996) offers suggestions about how communities can participate in the formulation of survey questions, and he demonstrates how user input provides contextual information about outcomes which leads to more accurate and facilitative decision-making about service improvement. Indeed, Marshall (1984) takes this further arguing that 'there is no question that the development of citizen participation in libraries must be encouraged, including the establishment of citizen advisory groups' (p277)
Yet library evaluation remains system-centred. Why? In later sections this paper will explore this question and draw attention to the gulf between academia and the workplace, arguing that user-centred perspectives and research are mainstream within academia but still appear to be peripheral within practice communities and their evaluation strategies. Does this gulf in thinking and application between scholars and practitioners matter? Of course it does. There are valuable lessons to be learnt by each party from the other - ideas that influence the direction of theory and practice, that contribute to the body of learning which promotes better understanding and educates new practitioners, ideas that promote relevant research and creative debate and, ultimately, lead to better information services. It is a two-way street: Durrance finds that '...the integration of research findings into professional practice results in improved outcomes' (2003, p549), while Ingwersen (1995) reminds us that LIS theory has developed out of practice, Biddiscombe points out that practitioners 'play an increasingly important part in the academic process' (2001, p161), and Kuhlthau (1993) explains that her academic work is founded on experience as a practitioner.
Qualitative versus quantitative methods
The primary debate in evaluation has not centred on the philosophical paradigms discussed above, but on the tools used to conduct research: the choice between quantitative and qualitative methods.
Quantitative tools gather numerical and statistical data using experiments, measurements, fixed-response questionnaires, test scoring, et cetera. The approach is underpinned by 'scientific' world views of cause and effect, belief in the objectivity of the researcher and the search for truth. Quantitative methods used in library assessment include web server statistics, electronic counters and surveys. These surveys are usually questionnaire-based and, at their best, are grounded in extensive and on-going piloting which uses qualitative methods such as focus groups and individual interviews to establish that the questions are meaningful to the user community. They often employ empirical testing which targets specific groups and topics while also fulfilling the scientific 'trinity of validity, generalisability, and reliability' (Janesick, 2000, p393).
Qualitative methods gather descriptive information using observation, case studies, reflection-in-action, document analysis, open-response questionnaires, and interviews. The approach is underpinned by social/contextual world views and the belief that interpretation and 'truth' is subjective. Qualitative methods used in library assessment include focus groups, open-response questions (usually as a component of fixed-response surveys), in-depth user interviews and reflection-in-action analysis. These open-response techniques help to explain quantitative data, provide contextual meaning, allow people to raise issues and explore complex situations. They also involve dimensions often excluded from quantitative approaches such as humour, external social factors and, particularly, emotion - a crucial dimension for library users both in terms of their information needs and behaviours (Kuhlthau, 1993) and their perceptions of the institution and the people who work in it (Radford, 2001). Davis & Bernstein (1997) found that careful analysis of a few qualitative comments provided more helpful information with which to improve service quality than all the quantitative questions used; while Lincoln argues that qualitative approaches 'illuminate aspects of libraries, library services, and library users' perspectives in ways we have not had access to in previous research' (2002, in Durrance & Fisher, 2003, p317)
At first glance it may seem that qualitative and quantitative methods fall easily into the user-centred and system-centred paradigms outlined earlier. Yet user-centred research can (and often does) employ quantitative tools as a component of its investigation, while qualitative methods can be system-focused - open-ended questions can ask about user preferences just as well as closed questions can. Nevertheless, conventionally each method is associated not only with asking different questions but with the philosophical leanings of those questions. As Riggen says, 'qualitative and quantitative methods, while not inherent to a paradigm, can act as carriers of paradigm attributes and should be selected for that ability' (in Greene & Caracelli, 1997, p92). Thus quantitative methods are more commonly used in traditional scientific inquiry which is interested in finding facts, patterns and comparative baselines, and therefore more favourable to user-centred inquiry; whilst qualitative methods are more commonly used in social inquiry which is interested in variable, contextualised human experience and therefore more in sympathy with system-centred research.
Both qualitative and quantitative methods have weaknesses as well as strengths. For example, qualitative tools oblige people to fit in with pre-determined statements limiting their answers to a tick against someone else's wording thereby preventing exploratory communication and explanation. This attempt to reduce human complexity to numerical data can result in superficial understanding and simplistic outcomes. However, qualitative research covers smaller samples which, combined with its in-depth, personalised style, means that generalisations are more difficult. It also relies more on (and is arguably more influenced by) the researcher's skills, and does not easily generate statistical data on which to base service decisions.
As Pawson & Tilley note, quantitative tools clearly define the 'what', while qualitative methods provide the 'why' (1997), and Greene & Caracelli add that qualitative approaches provide depth while quantitative approaches offer width (1997). This paper argues that neither makes sense without the other, and strongly advocates the use of creative, mixed method approaches. Witkin (1994) argues that mixed methods are also required for comparability and validity of findings because there is evidence to suggest that 'different methods produce different lists of needs ... [and] yield non-comparable results' (p24). This call for blended methodology applies equally to the paradigm debate above and to the measurement focuses below. Different approaches 'get at' different issues and both are valid when used overtly to achieve clearly defined aims. Library evaluation has employed quantitative methods to establish usage trends and baselines and to gather the demographic data which is vital for public accountability and funding. But it also must use qualitative methods if it is to gain real insights into the experience of service users and the wider community's needs.
Inputs and outputs versus outcomes
The arguments for more user-centred, mixed method evaluation are far from new. Many practice-based researchers have been working towards methods which not only capture data, but also strive to understand the quality of library services and the value they have for users.
Library evaluation has traditionally focused on efficiency calculations: inputs (resources such as funds, equipment, books, staff) and outputs (products or activities like transactions, circulation, and special programs). These measurements provide important 'snapshot' or longitudinal statistics about the daily life of the library which can be used to highlight issues such as comparable funding, staffing levels and resource changes. However, such user-centred, quantitative methodology is limiting. As Durrance puts it 'public librarians, state agencies, and the federal government have come to rely on output measures for public libraries as indicators of public library effectiveness. While the primary values of these measures are as indicators of efficiency and use, they do not reflect value gained by the user' (2003, p546).
More recently there has been a shift away from input/output focus towards outcomes evaluation. Covey explains, 'outcomes are measures of the impact or effect that using library collections and services has on ... users. Good outcome measures are tied to specific library objectives and indicate whether these objectives have been achieved' (2002, p89). Therefore outcomes are not concerned with how much users like the service, but how much it has benefited them in their everyday life or helped them with a specific goal. The approach is still quantifiable and attempts to measure dynamics rather than understand them, but it is interested in changes in a person's skills, knowledge, behaviour, attitude, status or life condition (Motylewski, 2002) such as gaining a job, getting better grades on assignments, improved confidence using software, increased understanding of a language, and can employ both qualitative and quantitative tools in its investigation.
User-centred research is, by definition, concerned with outcomes, but formal outcomes evaluation is not necessary user-centred. Outcomes are the focus of inquiry, and subject to different methods and philosophical approaches. In other words, we can assess outcomes from a system-centred, quantitative approach by calculating test scores of people who participated in a library literacy program; equally, we can take a user-centred, qualitative approach by talking with people about changes in their self-esteem and life experiences since participating in that same literacy program (see Dervin & Clark, 1987, for a practical application of user-centred library outcomes evaluation). Hernon distinguishes between these examples by calling them 'direct outcomes' and 'indirect outcomes' respectively (2002, p61). Nevertheless, overall it can be said that an outcomes focus takes us further in our journey towards user-centred inquiry because it acknowledges the user is actively engaged in a dynamic, contextualised process; not simply consuming a product.
Kuhlthau (1993) believes that librarians tend to approach users with a focus on 'product' (information sources) rather than 'process' (problem-solving). She argues that product-focused mediation assists with access to information sources, but that only process-focused mediation assists with learning from the use of information (p134). It is the focus on product, or sources which is, according to Kuhlthau, one of the chief obstacles to providing better information services. This distinction can also be applied to evaluation: both what we are evaluating and how we are doing it. The marketing emphasis treats libraries as a product, hence we evaluate the relationship between user and product; despite the fact that people actually experience libraries more as a process. Similarly, we use evaluation itself as a product, a tool to produce findings, rather than as an opportunity to engage in a reflective dialogue in which we really learn from our users.
More differences between what we say and what we do
ALIA's library standards publication, a cornerstone of much service planning in Australia, states the main elements of service evaluation are, 'community needs analysis, service objectives, input and output measures, efficiency and ultimate benefits to users' (1990, p88). It argues that objectives and planning should take account of a range of contextual and user needs, that service quality should be assessed in relation to these objectives, and that some qualitative methods should be used. Yet the standards are organised according to inputs and outputs with no further indicators of how needs evaluation might actually be tackled (but it should be recognised that this text is thirteen years old and may reflect attitudes which have moved on significantly since then). Other evaluation manuals describe outcomes evaluation and provide survey templates and analysis guidance (Hernon, 2002) but they remain system-centred with an emphasis on quantitative methods and do not offer any means for user communities or local evaluators to participate in the formulation of evaluation questions. Meanwhile LIS academia is embedded in user-centred research which strongly criticises such approaches.
Morris argues that librarians conduct system-centred evaluation despite the fact that 'we don't talk as if we hold this traditional view of information' (1994, p21). She believes the way librarians think about the relationship between people and information is intuitively more aligned with the alternative paradigm - regarding users as having active, complex information needs and behaviours, and knowing that information is not simplistically transmitted and received but complicated by many socio-cultural, environmental and personal factors. Similarly, Durrance (2003) points out there is a mismatch between library mission statements which tend to be user-centred, concerned with individual or community value-related outcomes ('to satisfy information needs, enhance knowledge, enjoy leisure and stimulate imagination') and the assessment methods they employ which focus on inputs and outputs. It seems there is a gap between the way we conceptualise our services and the methods we use to evaluate them. Thus our reliance on system-centred evaluation, which generates system-centred findings, is out-of-step with service objectives and with our core values and beliefs. This reduces our potential to improve services and enhance the lives of our users, and it also damages us by reducing the librarian's role to that of a shop assistant - a product-orientated customer service operative rather than a creative people-orientated professional.
How do these issues play out in practice? A brief look at some examples of current library evaluation in Australia may provide more context for the argument that evaluation requires a methodology which mixes each of the three areas discussed above: system- and user-centred perspectives, quantitative and qualitative methods, plus input/output and outcomes-focused inquiry. These examples will inevitably draw attention to weaknesses while ignoring strengths. This is regrettable given the enormous amount of time, energy and commitment that staff put into service planning and provision; but this paper will concentrate on areas that could be improved.
Some examples of library evaluation
Academic libraries
In this first example, a university library distributed 'client survey' questionnaires at the library entrance which were also available on their website. The questionnaire, developed by a market research company, asked a total of fifty-four quantitative questions - multiple choice or rated for perceived service importance and performance - followed by two qualitative questions asking for general comments and 'one thing we could improve'. A prize draw incentive was offered.
The questionnaire had a strong system-centred, input/output focus with no identifiable outcomes, and was a good example of Covey's argument that surveys fail to capture an understanding of why things happen because they 'cannot ... establish cause-effect relationships, and the information they gather reveals little if anything about contextual factors affecting the respondents' (2002, p7). The quantitative questions appeared to have insufficient contextual flexibility because of their poor fit - there were no questions about the library's new security measures or the 'Ask Me!' roving assistants pilot program - despite both being significant, high profile, potentially high-impact initiatives. User grounding seemed inadequate because many questions were concerned with issues of which the majority of users may have no experience, or would feel unqualified to answer, such as rapid materials processing, group study facilities, staff initiative, disability access; yet there was no 'n/a' option. Additionally, several questions could be seen as ambiguous (for example, access to electronic databases and journals may depend on where access is attempted - online at a remote location or in the overcrowded and therefore hard-to-access library computer lab). The lack of response options and comment space with which to qualify answers means evaluators have no way of knowing why questions receive particular ratings or why they are left blank (because the respondent does not understand the question, or has no opinion, or doesn't know, or believes 'it depends', or another reason altogether?). The survey's attempt to assess service quality as well as customer satisfaction by employing the gap model still does not help us understand the contextual issues which make up user needs and overall service value - that is, why things are important.
In their evaluation of reference services this library developed a very different, mixed method approach. Staff conducted a comparative survey of other academic libraries and reviewed extensive bodies of literature outlining reference evaluation models. They analysed the criteria and scope of each model before defining their evaluation aims and choosing to blend two focus group approaches which allowed them to incorporate a range of issues already identified as significant by their users such as digital standards. The focus groups were organised by different user communities, thereby recognising their potentially different needs (Whitmire, 2001), and used 'brainstorming' techniques to establish unique agendas for each and so ensure the group's issues were addressed (Widdows, Hensler & Wyncott, 1991, reflects on importance of this flexibility). However these focus groups were limited to the evaluation of reference services alone - only one aspect of the broad library service. Quality reference services are often considered to be the cornerstone of a good library, yet without an analysis of what users feel to be most important there is a danger that this attention is mis-focused. Indeed, formal library services are, themselves, only some of the many uses that people may make of the library. As Kyrillidou (2002) argues, it is likely that many users actually want to use library facilities independently - only asking for assistance if absolutely necessary (p45). We require grounded, mixed-method user needs research at a local level to find out what proportions of people use different aspects of the library, their reasons for doing so, and what would assist them further.
The limitations of pure quantitative research are highlighted by another evaluation example which took place in a different library. Having conducted web server statistical analysis, this library was able to state that their website usage was 'active', but whether this was because users were happily occupied within a well organised site or were desperately searching within poorly designed information structures (or both, or neither) they were unable to say. Qualitative, user-centred evaluation is needed to give meaning to raw data and help us design quality service improvements.
Another university library explained that they recognised the limitations of their input/output evaluation and are considering an outcomes orientated survey package next year. This package is also based on the 'gap theory of service quality' and requires users to rate their perceptions of current library services together with their minimum and desired standards (Cook, Heath & Thompson, 2000). It combines qualitative and quantitative response methods but, like many other outcomes approaches, it has a strong product-orientated market focus and emphasises cross-library comparative analysis. This assumes state or national standards can determine the needs of each user community, as if they were homogeneous across library types and locations. It does not take account of social and institutional factors which may influence services differently from community to community nor, most importantly, can it take account of services which may have been tailored to meet local user needs, possibly on the basis of previous evaluations and user feedback.
This generic instrument is gaining popularity despite the growing emphasis within LIS research on communities of practice which highlight the very different needs that people have in different contexts. Comparative standards and measurements can offer useful baselines for service improvements such as promoting better access for people with disabilities, arguing for increased funding to match that of neighbouring councils or competing institutions, et cetera. but they only go so far. Evaluation must also understand context and the uses each community makes, and wishes to make, of the service. As Hernon says, 'excellence in public libraries is to be defined locally by each library determining its role in the community through an assessment of community needs' (2002, p57). However, it should be noted that outcomes evaluation has been developed by academic and research libraries which, arguably, have a more homogeneous user community and therefore tighter goals than public libraries which, by their very nature, are generalist service providers.
Public libraries
A recent Australian public library survey consisted of nineteen questions offering a range of closed and open responses, many of which required rating on a scale of 'excellent' to 'failing' and included an 'n/a' option. The survey was only supplied in English even though seventy per cent of the local government population speaks languages other than English at home (2001 Census Data). It is not known what proportion of the community would be able to complete a written English language survey, but staff regularly interact with users who struggle with basic verbal English, so the numbers are likely to be significant; particularly as the survey includes examples of jargon ('How do you rate the OPAC?') which might even deter proficient speakers. This survey is a good example of why a system-centred approach is unhelpful. The starting point here is the library rather than the needs and experiences of the people who use it, and this has resulted in an instrument which is unintentionally discriminatory and out-of-step with the library's mission statement 'To provide both adult and children's' material in a variety of languages to meet the needs of [our] diverse ethnic population'.
Library surveys carried out in this locality in 1997 and 1998 indicated that more than two-thirds of users spoke languages other than English at home, but that usage of community language materials and survey feedback about them was low. Employees of the library are aware that the community language books are disorganised and messily shelved, often mixing fiction and non-fiction, adult's and children's books. It seems reasonable to assume that community language users would appreciate the same order as those using English resources, but that lack of complaints by users contributes to the lack of attention by staff. Apart from the obvious barrier of language, and the limitations of the methods used to consult them, there are other contextual factors which might contribute to the silence and could shed more light on their information needs. For example, what proportion are new immigrants? And how many of those are from countries where access to information may be a loaded issue? What are the experiences of the community's Muslim population in the current political climate which might affect their willingness to complain? Witkin notes that 'too many needs assessments are really opinion polls that generate wish lists of solutions to scarcely-articulated problems' (1994, p25). If we really do want to meet the needs of our users we need to find out what their needs are; and this demands a degree of user-centred research to supplement traditional evaluation.
Staff in another public library observe that a significant number of children 'hang out' waiting for their parents to collect them later in the evening, suggesting that the library is used for informal after-school care by some families, a community 'safe place'. This is a fascinating phenomenon, explored in some depth in the United Kingdom (Poustie, 2002); however this library knows very little about it, largely because they have not asked about it. In fact, they have not heard the children's point of view on any matters because surveys are only given to those aged fifteen years and over, despite previous evaluation showing that children and young people make up a higher than usual proportion of this library's users. Focus groups are now being planned for children and young people - a qualitative method which allows reflective discussion and open debate if managed well (Widdows et al, 1991). This takes the library closer to user-centred evaluation and towards addressing its service objectives regarding children and young people.
Another library has a website which users often mention unfavourably to staff when advised of online services. However, this library's evaluation survey does not ask about the website and, therefore, as far as formal evaluation goes, there is no problem with it. The survey was piloted five years ago and so is out-of-date with regard to the 'electronic revolution'. A supplementary website question could be added to the current survey without ruining comparative measurements, but user-centred, qualitative inquiry is needed to understand what users want from the site and how it can help them with their information needs.
One of the contemporary roles Australian public libraries play is that of community information centre (Giese, 2003). Many include this mandate in their mission statements and make some provision for authoritative, up-to-date information on health issues, child care, community language assistance, legal information and other social services; often producing or supplying a free community services directory. Yet seldom is this role evaluated. Is it wanted? Are libraries the right place to host it? How much and in what ways is it used? How does it dovetail with other local services? Where are the gaps? Again, we need to ask these fundamental questions if we are to find meaningful answers that provide quality service direction.
The libraries in all the examples above are staffed with LIS-educated librarians who would have engaged with user-centred research and methodological debates as a component of their training. Many have on-going access to LIS journals and work alongside LIS students who are currently engaged with user-centred studies. Yet the gulf between theory and practice continues. This takes us back to the key question: 'Why?'
The gulf between theory and practice
Some possible explanations for this gulf are considered below.
'There is always a gap between theory and practice'
Many disciplines experience the phenomenon of gaining qualifications by emphasising academic discourse, but subsequently drawing on entirely different resources in 'real life'. However, most user-centred research takes place within 'real' contexts: places such as high schools (Limberg, 1999), school and public libraries (Kuhlthau, 1993; Dervin and Clark, 1987;) and hospitals (Pettigrew, 1999). It explores everyday life situations like reading newspapers and watching television (Savolainen, 1995) and obtaining community information (Pettigrew, Durrance and Unruh, 2002); and examines life crisis situations such as domestic violence (Harris and Dewdney, 1994), health crises (Baker, 1996, in Julien, 1999) and career decision-making (Julien, 1999). These works offer valuable insights into the experiences and needs of people in many aspects of their lives. They identify crucial service issues such as barriers and aids, and the psychological factors which empower or prevent people from getting what they need. This is real life.
'Academic writing is irrelevant to practice'
Academic research is abstract, self-referential and sometimes impenetrable, creating differences of understanding between those who have strong incentives to wade though it, and those who simply want ideas to help them reflect upon and improve services. Fitzgerald (2003) argues that LIS academics are not interested in accessibility or relevance and, in fact, disassociate themselves from practice as much as possible. He concludes that academics are only writing for other academics, paying homage to one another buoyed by 'career concern and ego gratification incentives' (2003, section: relevance to other academics) and 'subsidized vacation opportunities' (Khazanchi, 2001, in Fitzgerald, 2003, section: background: relevance of research to practice). Despite this negative portrait there are notable LIS academics who are committed to applied research and whose work is evident in libraries and information services across the world; see, for example, Kuhlthau (1993), Julien (1999) and Dervin & Dewdney (1986b). A greater commitment to communication styles which allow (and even encourage?) practitioners to engage with the valuable debates currently limited to the academic elite would be very welcome. This does not require 'dumbing down', only more accountability and effort by authors to render their work meaningful.
'Academic research does not place enough emphasis on practical application'
LIS theory 'draws from practice to inform writing but fails to then develop findings of practical value' (Heeks, 2001, in Fitzgerald, 2003, p18). Academics have a duty to address practice - not only to ground a significant proportion of their work in field research, but to give applied examples and practice recommendations in what is, after all, an applied discipline. Clearly not all aspects of LIS lend themselves to service application, but even these might benefit from a 'why does this matter?' analysis: discussion of how further pursuit of the issue may enhance understanding and how this might, in turn, enable better information services. If academics are truly committed to information service quality then surely this should be reflected in the value and usability of the information they are paid to produce.
The arguments made by LIS theorists, using 'persuasive and authoritative rhetoric [which] is hard to discount' (Fitzgerald, 2003, section: relevance to other academics), strongly suggests that they hold the view that their ideas - and their ideas alone - are correct. They effectively promote their theories as truth - even those that embrace paradigms which deny the concept of truth. This may result in part from the 'politics' of academia which make it hard for researchers to collaborate across knowledge domains and to advocate eclecticism, despite the fact that many value collaborative, mixed methods in practice. Dervin (2003) argues that academics suffer from 'extreme information overload' (section: disciplinary tyrannies, para 5) from competing, often contradictory and chaotic research findings. This causes them to become more determined in their disciplinary allegiance and to the blinkered 'tyranny of the quest for certainty' (section: disciplinary tyrannies, para 8). Interestingly, Dervin herself is a good example of this. On the one hand she argues for theory-to-practice relevance and eclecticism noting 'that there are utilities to be derived from both [user- and system-centred] orientations' (Dervin and Nilan, 1986a, p9) and, in her own Sense-Making model, she acknowledges the influence of many theories and the usefulness of both qualitative and quantitative methods (Dervin, 1992). Yet she also criticises 'misuses of Sense-Making' (Dervin, 1999, p729) and implies that it must be employed as an absolute methodology, represented fully using all its dimensions in any research, rather than using the aspects which are most appropriate to each research context.
'LIS journals are biased against practitioners'
The majority of LIS research is published in academic peer-reviewed journals because 'reward structures in academia encourage researchers to publish in scholarly journals which are read by other scholars, [yet] these reward structures breed the type of articles which are eschewed by practitioners' (Durrance, 1991, p282). McNabb argues that journals have the power to determine which ideas are valued, how they must be treated, and who is permitted to talk about them, and 'therefore, can determine whose argument is accepted (scholars, researchers) and whose is considered null and void (practitioners) by perpetuating the theory/practice split' (1999, p27). This position is supported by Fitzgerald (2003) who claims that academic journals are biased against practitioner papers and instead prefer authors who demonstrate sufficient deference to the 'boys club' conventions of academic publishing. The argument that practitioner papers are rejected by scholarly journals due to legitimate quality control is countered by findings which suggest peer-review results in inequalities, biases and an overemphasis on conservative papers (Eisenhart, 2002); also that it fails to provide quality assurance and may, in fact, simply 'pander to egos and give researchers licence to knife each other in the back with impunity' (White, 2003).
'Research and evaluation are completely different'
Some believe that academic research concentrates on the serious understanding of people's information needs and behaviours, while library evaluation simply focuses on measuring service delivery. Further, that since they are engaged in entirely different activities the gulf between them does not matter. However this paper has argued that evaluation is a form of research, the difference being that 'evaluation' describes one focused activity, whereas 'research' is simply the umbrella term. They share a common purpose - to understand phenomena more fully and to use this understanding in some meaningful way.
'Practitioners do not have the time, energy or motivation to attend to on-going research'
Running busy services with scarce resources, funding pressures and increasingly business-orientated directives is a high-pressure activity and not does support reflective practice. Additionally, there is little incentive from many library employers to engage in continued learning or to conduct creative research. These restraints, together with the issues outlined in previous sections, also limit the potential for being published and gaining external recognition. Practitioners will not maintain their links with academic research unless this is encouraged by management structures; and no amount of practitioner enthusiasm for more meaningful evaluation can effect change if higher level managers are unsupportive.
Practice complacency or defensiveness
In 1980, Garrison (in Durrance 1991) argued that both researchers and practitioners were negligent. Researchers do not disseminate their research well, but practitioners chose to ignore it. They use the 'real world' argument to dismiss research and justify an inverse snobbery which protects them from having to deal with challenges they may find too confronting. One senior practitioner, when asked if she had considered using LIS research to inform her evaluation strategy, rolled her eyes and commented she thought after graduating she would never have to read 'that stuff' again. This raises worrying questions about practice culture and also suggests that LIS courses may be failing in their efforts to persuade students of the reciprocal benefits, indeed, interdependency, of the theory > research > practice relationship.
Professional values or lack thereof
Is there is a mismatch between principles claimed by organisational bodies and the values of the average information worker? ALIA asks that its members 'observe the highest standards of service quality' by 'maintaining and enhancing their professional knowledge and expertise [and] encouraging the professional development of their colleagues...' (2000, statement 6). Perhaps information workers do not believe that academic research will enhance their knowledge and expertise and therefore look to other sources. Perhaps aspects of library culture - the hierarchical structure, the retail emphasis, the constant struggle for sufficient resources - discourages a commitment to life-long learning. Or perhaps the stereotype is winning out and librarianship simply attracts people who dislike creativity, challenge and change.
'Libraries are embedded in tradition'
Libraries are not engaged in reflective practice and simply do what they have always done because, tautologically, they have always done it that way: gathering data 'for historical reasons or because they are easy to gather, rather than because they serve useful, articulated purposes' (Covey, 2002, p2). Given the accountability pressures on most libraries, public and private, this seems too simplistic. It is likely that traditional methods are favoured partly because they are familiar, a known quantity, and partly because evaluation in the workplace provokes anxiety: not only because staff may feel judged, but because identifying areas for service improvement may lead to more work in an already pressurised work environment. Quantitative methods which focus on input/outputs and customer satisfaction are 'safer' because they are unlikely to reveal anything surprising.
'Libraries exist in a context which is embedded in tradition'
Primarily though, traditional methods are probably favoured because the world view of library staff and the people to whom they supply their data is traditional, based on the dominant paradigm which tells us that that 'scientific' methods reveal truths, and that results must be measurable and testable if they are to be of any value. Higher level policy makers, budget holders and data analysts do not tend to value research which addresses abstract concepts such as feelings, life enhancement and community value. Even though the way they talk about service quality may sound user-centred and engage with concepts such as social capital, they tend to favour 'hard facts' gathered and presented in a traditional manner with which to justify business plans to their superiors and the public. Thus libraries are encouraged to produce 'symbolic' data which supports their political agenda and which, in turn, is used by higher management to support theirs (but it should be noted that qualitative, user-centred evaluation can be used tactically also; for example, a strong case involving economics, child protection and community safety might be made for better understanding and improving the after-school care role of public libraries already discussed). Against this background we can see that library practitioners are required not only to remain engaged with research developments and to brace themselves for increasingly challenging practice requirements, but are in the unenviable position of having to persuade policy makers and funders of the limitations of system-centred quantitative approaches and the desirability of alternative evaluation methods - at potentially more cost. A big ask.
Evidently none of these possible explanations answers the question by itself. But when viewed together they suggest general areas for further reflection and change: especially in regard to the contributory roles of the major parties within this dynamic - academics, practitioners, and higher level policy/funding bodies. There is a need for more open communication, greater honesty about what we do and why we do it, demonstrated sensitivity to the role limitations of each party, more creative input about how these limitations might realistically be addressed and, finally, a commitment to the needs of the people whom we are all attempting to serve in our work.
So what should we be doing about library evaluation?
We need to understand, acknowledge and harness the benefits of mixed method, eclectic approaches to library evaluation. System-centred quantitative tools are already well in place so the task becomes one of finding meaningful ways to supplement them with user-centred, qualitative approaches which take account of user needs within a social context.
Two examples of mixed method evaluation may offer ideas for practical application.
Pothas et al (2001) developed a qualitative, user-centred gap analysis survey to assess customer satisfaction with their bank. They argue this method allows the issues that really matter to customers to emerge rather than obliging them to answer a set of pre-defined questions judged relevant by the investigator. The approach incorporates 'surfacing issues and monitoring impact from the viewpoint of the customer' (p93) and is based on two simple questions:
The answers to these grounding questions are aggregated to form issue clusters that, in turn, inform the development of further qualitative questions. These questions directly target the specific issues customers raise, and are periodically re-grounded to ensure the evaluation is firmly based in what really matters to service users. This also ensures that evaluation keeps up with environmental changes and service improvements based on previous surveys. Jackson & Trochim (2002) outline methods for analysing and building on the results of such open ended surveys using concept maps - a tool often employed in focus group research.
Norlin's (2000) experience of previous mixed methodology evaluation in an academic library suggested that users had three major needs of the reference staff: 'approachability', 'ability to answer questions correctly', and skills in offering 'ideas on how to get started' (p547). These issues appear to cross paradigms: the first addressing how people feel (a subjective, user-centred perspective), the second their rational information needs (in which information is considered to be an objective product), and the third addressing their context-specific needs as active information seekers engaged in a dynamic process of making meaning. These needs were explored eclectically by survey, focus group and observation methods which resulted in several key changes to the service including roving librarians and better promotion of technology classes. Norlin also found that different student groups had different needs, and that the rating they gave to each librarian (a quantitative measure) related to many subtleties which were only noticeable via observation (one of the qualitative strategies used). Thus an enriched picture was developed which more accurately reflected user's experience of the library and therefore offered better ideas on how its quality might be improved.
The way that we ask questions shapes the that answers we get. All forms of research make assumptions about the world they are investigating, and all have strengths and weaknesses. Mixed method evaluation which combines user-centred with system-centred paradigms and qualitative with quantitative methods offers complementarity. It allows evaluators to design creative interventions which fit with their service aims, and which not only help us to ask the questions we find most pressing but enables respondents to explore their own. A strategy that connects with the needs, experiences and perspectives of service users is one that leads to informed professionals and more effective decision-making about quality improvements. And lastly, as library and information professionals attempting to supplement our evaluations with qualitative, user-centred approaches, surely we cannot afford to neglect the wealth of research produced by our colleagues in library and information studies.
ACT Customer Services and Information (1997), Australian public libraries: 1996-97 selected national indicators [Online]. Available: http://www.info.act.gov.au/functions/librarysurv9697.pdf [Accessed 26 October 2003].
Association of Research Libraries (ARL) (2003), Libqual+™: Charting library service quality [Online]. http://www.libqual.org [Accessed 15 September 2003].
Australian Library and Information Association (1990), Towards a quality service: goals, objectives and standards for public libraries in Australia, Panther, Canberra, ACT.
Australian Library and Information Association (2000), Statement on Professional Conduct [Online]. http://www.arl.org/newsltr/212/libqual.html and http://www.ifla.org/IV/ifla66/papers/028-129e.htm
Covey, DT 2002, Usage and usability assessment: library practices and concerns, Digital Library Foundation, Washington, DC.
Davis, D & A Bernstein (1997), 'From survey to service: Using patron input to improve customer satisfaction', Technical Services Quarterly, vol 14 nº3, pp47-61.
Dervin, B (1989), 'Users as research inventions: how research categories perpetuate inequities', Journal of Communication, vol 39 nº3, pp216-232.
Dervin, B (1992), 'From the mind's eye of the user: the sense-making qualitative-quantitative methodology', in Qualitative Research in Information Management, eds JD Glazier & RR Powell, Libraries Unlimited, Englewood, CO, pp61-84.
Dervin, B (1999), 'On studying information seeking methodologically: the implications of connecting metatheory to Method', Information Processing and Management, vol 35, pp727-750.
Dervin, B 2003, 'Human studies and user studies: a call for methodological inter-disciplinarity', Information Research, vol 9 nº1, http://InformationR.net/ir/9-1/paper166.htm
Dervin, B & K Clark, 1987, Asq: asking significant questions, alternative tools for information needs and accountability assessments by libraries, Peninsula Library System, Belmont, CA.
Dervin, B & P Dewdney, 1986b, 'Neutral questioning: a new approach to the reference interview', RQ, vol 25 nº4, pp506-513.
Dervin, B & M Nilan (1986a), 'Information needs and uses', Annual Review of Information Science and Technology, vol 21.
Duffy, A & A Ketchand (1998), 'Examining the role of service quality in overall service satisfaction', Journal of Managerial Issues, vol 10 nº2, pp240-256.
Durrance, JC (1991), 'Research needs in public librarianship', in Library and Information Science Research, eds C. McClure & P. Hernon, Ablex, Norwood, NJ.
Durrance, JC & KE Fisher, (2002), How libraries and librarians help: context-based, outcomes evaluation toolkit [Online]. http://www.si.umich.edu/libhelp/toolkit/index.htm [Accessed 17 September 2003].
Durrance, JC & KE Fisher (2003), 'Determining how libraries and librarians help', Library Trends, vol 51 nº4, pp541-570.
Eisenhart, M (2002), 'The paradox of peer review: admitting too much or allowing too little?' Research in Science Education, vol 32 nº2, pp241-255.
Fitzgerald, B (2003), 'Informing each other: bridging the gap between researcher and practitioners', Informing Science, vol 6 pp13-19. http://esc01.midphase.com/~inform/Articles/Vol6/v6p013-019.pdf
Greene, J (2000), 'Understanding social programs through evaluation', in Handbook of Qualitative Research, eds NK Denzin & YS Lincoln, Sage Publications, Inc., Thousand Oaks, Ca, pp981-999.
Greene, JC & VJ Caracelli (1997), 'Advances in mixed-method evaluation: the challenges and benefits of integrating diverse paradigms', New Directions for Evaluation, vol 74, Jossey-Bass Publishers, San Francisco.
Harris, RM & P Dewdney (1994), Barriers to information: how formal help systems fail battered women, Greenwood Press, Westport, Connecticut.
Henry, G (1996), 'Does the public have a role in evaluation? Surveys and democratic discourse', in Advances in survey research, eds MT Braverman & JK Slater, Jossey Bass Publishers, San Francisco. pp3-15.
Henry, GT, G Julnes & MM Mark (1998), 'Realist evaluation: an emerging theory in support of practice' New Directions for Evaluation, vol 78, Jossey-Bass Publishers., San Francisco.
Hernon, P & RE Dugan (2002), An action plan for outcomes assessment in your library, American Library Association, Chicago.
Hernon, P & JR Whitman (2001), Delivering satisfaction and service quality: a customer-based approach for libraries, American Library Association, Chicago.
Ingwersen, P (1995), 'Information and information science', in Encyclopedia of Library and Information Science, ed K Allen, vol 56, Dekker, New York, pp137-174.
International Federation of Library Associations and Institutions (IFLA) (2003a), More about IFLA [Online]. http://www.ifla.org/III/intro00.htm [Accessed 24 October 2003].
International Federation of Library Associations and Institutions (IFLA) (2003b), Information for all: the key role of libraries in the information society (Report Prepared for the World Summit on the Information Society, November 2003) [Online]. http://www.unige.ch/biblio/ses/IFLA/rol_lib_030526.pdf [Accessed 20th September 2003].
Jackson, K & W Trochim (2002), 'Concept mapping as an alternative approach for the analysis of open-ended survey responses', Organizational Research Methods, vol 5 nº4, pp307-336.
Janesick, VJ (2000), 'The choreography of qualitative research design', in Handbook of Qualitative Research, eds N. K. Denzin & YS Lincoln, Sage Publications, Inc., Thousand Oaks, Ca, pp379-399.
Julien, H (1999), 'Barriers to adolescents' information seeking for career decision making', Journal of the American Society for Information Science, vol 50 nº1, pp38-48.
Kuhlthau, CC (1993), Seeking meaning: a process approach to library and information services, Ablex Publishing, Westport, Connecticut.
Kyrillidou, M (2002), 'From input and output measures to quality and outcome measures, or, from the user in the life of the library to the library in the life of the user', The Journal of Academic Librarianship, vol 28 nº1, pp42-46.
Latu, T & A Everett (2000), Review of satisfaction research and measurement approaches, Department of Conservation, Wellington, NZ.
Limberg, L (1999), 'Experiencing information seeking and learning: a study of the interaction between the two phenomena', Information Research, vol 5 nº1, http://informationr.net/ir/5-1/paper68.html
Marshall, J (1984), 'Beyond professionalism: the library in the community', in Citizen Participation in library decision-making: The Toronto Experience, ed J Marshall, Dalhousie University/Scarecrow Press, Metuchen, NJ.
McNabb, R (1999), 'Making all the right moves: Foucault, journals and the authorization of discourse', Journal of Scholarly Publishing, vol 31, pp20-41.
Morris, RCT (1994), 'Toward a user-centered information service', Journal of the American Society for Information Science, vol 45 nº1, pp20-30.
Motylewski, K (2002), Outcomes: libraries change lives - oh yeah? Prove it [Online]. http://www.imls.gov/grants/current/PLA-02-2OBE.pps [Accessed 20 September 2003].
Norlin, E (2000), 'Reference evaluation: a three-step approach - surveys, unobtrusive observations, and focus groups', College and Research Libraries 61 [6] pp546-553.
Parasuraman, A, V Zeithaml, & L Berry (1994), 'Reassessment of expectations as a comparative standard in measuring service quality: implications of further research', Journal of Marketing, vol 58 [Jan] pp111-124.
Pawson, R & N Tilley (1997), Realistic evaluation, Sage Publications, London.
Pelz, DC (1978), 'Some expanded perspectives on use of social science in public policy', in Major social issues: a multidisciplinary view, eds JM Yinger & ST Cutler, The Free Press, New York.
Pettigrew, K (1999), 'Waiting for chiropody: contextual results from an ethnographic study of the information behaviour among attendees at community clinics', Information Processing and Management. vol 35 nº6 pp801-817.
Pettigrew, KE, JC Durrance & KT Unruh (2002), 'Facilitating community information seeking using the internet: findings from three public library-community network systems', Journal of the American Society for Information Science and Technology, vol 53 nº11, pp894-903.
Pothas, A, A De Wet & J De Wet (2001), 'Customer satisfaction: keeping tabs on the issues that matter', Total Quality Management, vol 12 nº1, pp83-94.
Poustie, K (2002), Whither Australian public libraries: ALIA Conference 2000 Proceedings [Online]. http://conferences.alia.org.au/alia2000/proceedings/kay.poustie.html [Accessed 26 October 2003].
Radford, GP & ML Radford (2001) 'Libraries, librarians, and the discourse of fear', Library Quarterly, vol 71 nº3, pp299-330.
Rodski Group (date unknown), Two core questions - importance and performance [Online]. http://www.rodski.com.au/research/company/methods.cfm [Accessed 6 November 2003].
Rudd, P (date unknown), Documenting the difference: demonstrating the value of libraries through outcome measurement, [Online]. http://www.imls.gov/pubs/pdf/pubobe.pdf [Accessed 30 September 2003].
Savolainen, R (1995), 'Everyday life information seeking: approaching information seeking in the context of 'Way of Life'', Library and Information Science Research, vol 17 nº3, pp259-294.
Sheppard, B (2002), Showing the difference we make: outcome evaluation in libraries and museums [Online]. http://www.imls.gov/grants/current/crnt_obe.htm [Accessed 19 September 2003].
White, C (2003), 'Little evidence for effectiveness of scientific peer review', British Medical Journal (BMJ), vol 326 (7383), p241.
Whitmire, E (2001), 'The relationship between undergraduates' background characteristics and college experiences and their academic library use', College and Research Libraries, vol 26 nº4, pp233-247.
Widdows, R, TA Hensler & MH Wyncott (1991), 'The focus group interview: a method for assessing users' evaluation of library service' College and Research Libraries, vol 52 nº4, pp352-359.
Witkin, BR (1994), 'Needs Assessment since 1981: the state of the practice', Evaluation Practice, vol 15 nº1, pp17-27.
Biographical information
Abby Haynes is a community development officer with the Commonwealth Government. She recently completed a post-graduate Diploma in Information Management at University of Technology, Sydney, during which time she also worked at a public library. email@example.com
About this article she says: 'You asked how I came to write the article... It was for my Information Management course at University of Technology, Sydney. We had to write a paper as if for a journal. I was doing research on another topic and visiting several academic libraries at the time, and was given customer satisfaction questionnaires to fill in. I found them clumsy and unable to capture the things I wanted to say. I was struck at how little relationship they had to the research methods we were studying on the course. During this period I was also working in a public library in which we gave out English language questionnaires to NESP majority populations. It seemed there was no real attempt to consult with most of our main users groups, yet many decisions about resource allocation were made on the basis of feedback from this survey. I could not understand why library professionals did not question these instruments more and I decided to try and unpick the puzzle. I wrote the article with the broad ALJ readership in mind, hoping to raise these issues with as many Australian practitioners as possible.'


text:
Welcome from SOCAL
This is a discussion on Another Kentucky guy just joined... within the New Members Introduce Yourself forums, part of the DefensiveCarry.com Forum Office category; Welcome from Upstate South Carolina!...
Welcome from Upstate South Carolina!
"Laws that forbid the carrying of arms...disarm only those neither inclined nor determined to commit crimes."
--Thomas Jefferson
Welcome from SOCAL
Associate with men of good quality if you esteem your own reputation; for it is better to be alone than in bad company.- George Washington
Serving Since 1985- "Airborne"
Welcome from West Tennessee!!
,=====o00o _
l_--- \___l---[]lllllll[]
(o)_)-o- (o)_)--o-)_)
welcome from Texas
Certified Glock Armorer
"I got a touch of hangover bureaucrat, don't push me"
--G.W. McClintock
Independence is declared; it must be maintained. Sam Houston-3/2/1836
If loose gun laws are good for criminals why do criminals support gun control?
Welcome from Michigan!
I've got a Charter Pug .44 too, carry in a jacket pocket Fall/Winter. Nice little revolver (usually a BUG to a 5" 1911).
Regards, T Bone.
"Those who would give up essential liberty to purchase a little temporary safety deserve neither liberty nor safety". Benjamin Franklin