text:
Posts filed as 'View'
Oil Can0
Maarten Beijen found this beautiful oil can on a market in Romans sur Isere, France, last Sunday.
Date posted: July 11, 2017No Comments
Paul Gill0
One of the main organisers of the BU History Group is Paul Gill. Paul has been diagnosed with dementia and is part of a campaign to encourage research into the condition. Please watch the video by clicking the link.
Below is a letter published in the Leicester Mercury last month from Dave Tipton in which he pays tribute to Paul. He tells about Paul's crucial role in the seven year struggle, and ultimate victory, by BU pensioners to win back their pensions after the company went into administration in 2000.
Date posted: May 9, 2016No Comments
Leslie Hubble0
I worked in the IVI works as a Tack and Nail maker and also as a utility man. If someone was away l would fill in for them. I found this old forklift truck license about a fortnight ago.
Date posted: April 22, 2016No Comments
Ray Davis0
Ray started with BU in 1945, aged 14; mostly running errands around the factory.
He left to do national service and returned afterwards, to the Tool Room, where this photo was taken in the mid 1950s. Ray's still got all that hair – Just a different colour now!
Date posted: March 21, 2016No Comments
Val Rawlinson0
Val worked in O and M data preparation between 1958-66. These were pre-computer days and the below is one of the wages cards Val would have to punch in data on.
Here is Val in 1960, back row on the left, with the rest of her department. Next but one along is Ann Brearly, one of Val's best friends, and she would love to get back in touch with her. The section boss was Betty Bancroft, and she appears on the front row, far right.
Date posted: March 14, 2016No Comments
Photos from last week's...0
Thank you Mike!
Date posted: March 6, 2016No Comments
For Sale – Cutting Press0
Western Australia again. Donna has written "I have this cutting machine which we were using to press out carpet and vinyl samples. As you can see in the pictures it has one of your identification plates.
I no longer need this machine. Can you please tell me what else it can be used for and how much it is worth as I would like to sell it. Also, would you know of anyone looking for one? I am based in Perth, Western Australia."
0423 623 752
Date posted: February 9, 2016No Comments
1964 BU Training Film (made...0
Date posted: October 18, 2015No Comments
Great Days0
Leslie Hubble has posted this brilliant photo of the Quarter Century Club from the 1960s.
Left to right; Eddie Orme, Glen (Taffy) Evans, Bill Leadbetter, his brother Frank, and Charlie Jerrom.
Date posted: October 11, 2015No Comments


text:
The poor thing was clearly in bad shape and frightened out of his mind, which made staffers at the Austin Animal Center (AAC), think the cat, which they named Klaus was feral by the way he was acting.
Once he was all cleaned up, Klaus who they estimated to be around one-year-old he began to warm up to the people who only wanted to care for him.
Hanna Peterson, senior customer care representative at the AAC said "When he first came in he was hissing and backing away because he was frightened, but now he has warmed up so much, he loves to be pet all over but is still a bit reserved and shy."
When he first arrived he was malnutrition and extremely skinny, and after some tests, it was determined that he has severe anemia. More tests and examinations also revealed something surprising about Klaus - He wasn't a one-year-old... In fact he was nine!
Judging by his spunk and kitten like looks, Klaus must have inherited some really good genetics because he is certainly aging extremely well.
The cat was placed in a foster home, where he can work on gaining some weight and getting strong enough to have his neuter surgery so he'll be all ready for adoption.
Carla Penny, Klaus's foster mom said. ""He is very timid — possibly because he doesn't feel good, but He is eating well and I am hopeful he'll put on some weight and his red blood cells will build back up."
After his traumatic episode and not knowing his history Carla decided to take things slow with Klaus until he puts on a little more weight and becomes more familiar with his surrounds.
Once he feels a little bit more comfortable Clara said she'll try to give him a bath so when he does get adopted and meets his forever home, family for the first time he will be one beautiful cat, as if he isn't already.
If you're interested in adopting Klaus, you can contact the AAC. His animal ID number is A740561.
Realted Story:
Homeless Dog Living Under Highway Is Rescued and Discovers He is Loved
Click On An Image To Open Our Gallery Viewer

subset: commoncrawl


text:
Job Search Engines Worth Checking Out
The hiring procedure is becoming more digital with each passing day, with job postings and public resumes obtainable to anyone seeking to uncover them. The term job search engine may possibly refer to a job board with a search engine style interface, or to a web web site that in fact indexes and searches other net sites. Still, they offer some of the deepest search outcomes in the market, taking their final results from massive websites like Monster and CareerBuilder. When browsing by place, customers can get detailed data about the area, including the average revenue of residents and other job openings in the area.
The site provides posters with an anonymous e mail address, so job seekers will not always know what company has posted listings. It enables potential candidates to network, post resumes and other content material as properly as search for jobs. An advanced search feature delivers the potential to sort results by expertise level, minimum education needed, firm size and revenue. In addition to offering job listings, Glassdoor allows employers to identify job candidates and market place their firms to job seekers. Involves many articles with guidance and information on how to avoid job scams.
Like Indeed, LinkUp is exclusively intended for employers and job seekers to post and search for a single yet another. This is an instance of each metasearch (because these are search engines which search other search engines) and vertical search (given that the searches are limited to a certain subject – job listings). If you're amongst those who aren't, there are job search internet sites just for you, like these.
Eluta also lets you set up an e-mail notification that alerts you as soon as new jobs matching your search are posted. Job search websites for students, like these below, match students with companies that are seeking for the fresh point of view you bring to the table. Glassdoor provides detailed information about each and every organization and job listing on the internet site.
These jobs are worldwide, and the U.S. government isn't shy about listing the actual salary ranges appropriate there in the listing. This tech-only job search site brings networking, job postings and capabilities center – comprehensive with learning opportunities – to these in the tech field hunting to develop their networks and locate new possibilities. In addition to its job search and resume posting tools, Monster also gives profession suggestions , a beta tool for researching companies , and much more. Locating that very first job in a sea of applicants with a lot more sensible encounter can be challenging.


text:
Ulick Varange is the name Francis Parker Yockey used when he was running from the United States of America and its government. It is his alias name. Ulick Varange was an absolutely brilliant attorney from Chicago Illinois. Ulick Varange was somehow called to be an attorney lawyer at the Nuremberg trials in Germany after the second world war. By the way, the Wikipedia article that I just linked to is full of lies and deception. I linked to it for people to reference, but beware of the lies and deception contained within.
Ulick Varange
After Ulick Varange had seen, heard, and witness certain things at these trials, he decided that the Germans were right and the Americans and other allied forces were wrong. He also witnessed the Jewish Zionist control and lies over the whole so called "Holocaust" and then decided he had had enough. His conscience told him that this was not right and he had to do something about it. That was when Ulick Varange decided to leave, and go into Europe and set up what is now called the Fourth Reich.
As you can imagine this put him on the hit list for his home country America. As a result he became a fugitive and worked hard to organize a European movement to save Europe from the total disaster we are seeing right now. Ulick Varange was a visionary and very prophetic in his knowledge of the future. His book Imperium The Philosophy of History and Politics was and is legendary at seeing "the writing on the wall", metaphorically speaking. Eventually he was caught in Oakland California with fake passports and sent to jail in San Francisco where they claim he committed suicide by swallowing a cyanide capsule. Maybe but many believe he was murdered.


text:
I Can't Finish My Food Elementary
Today's lesson is about a picky eater who never finishes their food. Learn some great ways to say that you're full and can't eat any more.
Photo by Megan Lloyde
This is Basic content. Purchase a Subscription in the ChinesePod Store to access.
Failure loading section. Click here to retry.
Comments (12) View last comment
Not sure if your comment is appropriate? Check our Commenting Policy first.


text:
Blood Pressure Monitoring Devices Market Report, published by Allied Market Research, projects that the global market was valued at $1,981 million in 2015, and is estimated to reach $2,926 million by 2022, registering a CAGR of 5.7% from 2016 to 2022. Digital blood pressure monitoring devices segment held around two-fifths share of the total market in 2015.
Early buyers will receive 20% customization on this report! : For Sample PDF – Click Here
Blood pressure is considered as one of the main risk factors for chronic heart disease. Advanced blood pressure monitoring devices are used for regular remote monitoring and transmission of the blood pressure data to the physicians. Factors such as increase in geriatric population, rise in inclination towards home monitoring, and upsurge in incidence rate of chronic diseases, such as cardiovascular disorders and obesity, are expected to drive the market. Moreover, innovations in blood pressure monitoring devices, (such as introduction of digital blood pressure monitors) and rise in awareness among patients regarding advanced devices are expected to fuel the market growth. However, lack of awareness about these devices in low- and middle-income economies hampers the market growth.
Digital blood pressure monitoring devices segment is anticipated to dominate the global blood pressure monitoring devices market owing to the increase in the global incidence of hypertension. However, ambulatory blood pressure monitors segment is projected to be the fastest-growing segment, as a result of increase in the outpatient cases and reimbursement support from the government.
Digital BP monitors at hospitals & clinics segment is expected to dominate the global blood pressure monitoring devices market, as these devices facilitate easy and early detection of patients deteriorating health condition, thereby, enabling speedy intervention and diagnosis of diseases. However, the ambulatory BP monitoring devices at ambulatory surgical centers (ASCs) market is projected to be the fastest-growing segment, owing to increase in the number of outpatient cases..
Do Purchase Inquiry@ https://www.alliedmarketresearch.com/purchase-enquiry/2166
North America was the highest revenue contributor to the global revenue in 2015, accounting for about two-fifths share, and is expected to maintain this trend. This is attributed to the increase in prevalence of hypertension, growth in awareness among patients about cost-effective blood pressure monitoring devices, and availability of advanced healthcare facilities in North American countries. However, Asia-Pacific is expected to grow at the highest rate, owing to its high population base, incomes, and improvement in patient awareness about advanced blood pressure monitoring devices.
Rise in incidence of obesity leads to hypertension, which is the major cause for stroke. Therefore, patients suffering from blood pressure need constant monitoring and this need has fueled the blood monitoring devices market growth. In addition, favorable reimbursements in developed regions and increase in the out of pocket expense in healthcare is expected to lead to growth in the ambulatory blood pressure monitoring devices market.
Key Findings of the Blood Pressure Monitoring Devices Market Study:
- Based on product type, in 2015, digital blood pressure monitoring devices segment accounted for around two-fifths share of the total market.
- Home environment segment is expected to grow at a significant CAGR of 6.5%, as these devices can be easily used by a patient and do not require trained professionals for operating the device.
- LAMEA is projected to be the one of the fastest growing region, owing to the presence of ample growth opportunities in terms of unmet medical needs for the treatment of hypertension, and increased disposable income of the population.
- In 2015, Japan and China jointly accounted for about two-thirds share of the Asia-Pacific blood pressure monitoring devices market.
For more Information – Connect to Analyst@ https://www.alliedmarketresearch.com/connect-to-analyst/2166
Key players in the global blood pressure monitoring devices market include A&D Company, Limited, Contec Medical Systems Co., Ltd., General Electric Company, Halma plc, Hill-Rom Holdings, Inc., Koninklijke Philips N.V., Masimo Corporation, Nihon Kohden Corporation, Omron Corporation, and Smiths Group Plc.
About Us:
Allied Market Research (AMR) is a full-service market research and business-consulting wing of Allied Analytics LLP based in Portland, Oregon. Allied Market Research provides global enterprises as well as medium and small businesses with unmatched quality of "Market Research Reports" and "Business Intelligence Solutions". AMR has a targeted view to provide business insights and consulting to assist its clients to make strategic business decisions and achieve sustainable growth in their respective market domain.
Contact Us:
Rahul Thakur 5933 NE Win Sivers Drive
#205, Portland, OR 97220 United States
Direct: +1-503-894-6022
Toll Free: +1 (800) 792-5285 (U.S. & Canada)
Fax: +1 (855) 550-5975
E-mail: firstname.lastname@example.com


text:
Cooking.com Tip:
Pie dishes aren't just for pies. Using a pie dish is perfect for baking a delicious crisp, cobbler, or in this case, a Blueberry Grunt, along with smaller casseroles. The quality construction of an earthenware pie dish ensures even cooking and browning.
Active Time: 20 Minutes
Total Time: 2 Hours 10 Minutes
Preheat oven to 375 degrees F.
Generously butter 9-inch pie plate. Combine blueberries with sugar, flour and lemon juice in medium bowl. Transfer to prepared pan. Bake until berries bubble thickly in center, about 1 hour. Cool completely.
Combine flour, brown sugar, baking powder and salt in food processor. Pulse to blend. Add butter and hazelnuts and blend until mixture resembles coarse meal. Pulse in cream, mixture will form ball. Pat dough out between two sheets of plastic wrap to 9-inch round. Remove 1 sheet plastic wrap and invert dough evenly onto blueberries in dish. Crimp edges decoratively with fork. Bake until topping is golden brown, about 30 minutes. Cool until warm. Sift powdered sugar over grunt and serve with heavy cream or vanilla ice cream.
View line-by-line Nutrition Insights™: Discover which ingredients contribute the calories/sodium/etc.
|Serving Size: 1 Serving (217g)
|Recipe Makes: 6 Servings
|Calories: 727
|Calories from Fat: 392 (54%)
|Amt Per Serving
|Total Fat 43.6g
|58 %
|Saturated Fat 24.1g
|120 %
|Monounsaturated Fat 14.2g
|Polyunsanturated Fat 2.3g
|Cholesterol 101.8mg
|31 %
|Sodium 301.4mg
|10 %
|Potassium 173.4mg
|5 %
|Total Carbohydrate 85.4g
|25 %
|Dietary Fiber 3.2g
|13 %
|Sugars, other 82.2g
|Protein 4.3g
|6 %
Powered by: USDA Nutrition Database
Disclaimer: Nutrition facts are derived from linked ingredients (shown at left in colored bullets) and may or may not be complete. Always consult a licensed nutritionist or doctor if you have a nutrition-related medical condition.
Calories per serving: 727
Get detailed nutrition information, including item-by-item nutrition insights, so you can see where the calories, carbs, fat, sodium and more come from.


text:
Dw Walsh Architects & Project Managers Limited was set up on Wednesday the 9th of April 2014. Their current address is Merchant House,27/30 Merchants Quay,Dublin 8., and the company status is Normal. The company's current directors Linda Walsh and Donal Walsh have been the director of 1 other Irish company between them; 1 of which is now closed. This Irish company shares its Eircode with at least 46 other companies.
|Company Name:
|Dw Walsh Architects & Project Managers Limited
|Company Status:
|Company Address:
|Merchant House,
27/30 Merchants Quay,
Dublin 8.
|Key Executive:
|Donal Walsh
|Company Size:
|Log-In To View
Donal Walsh is a Company Director of Dw Walsh Architects & Project Managers Limited since 2014 and a listed Director of 2 other companies.
Generate a B2B Marketing List with ease and grow your business. Identify key decision makers and pre-qualified new prospects for your sales and business development teams.
Similar Companies - Create a marketing list of similar companies
The latest documents filed with the Companies Registration Office for Dw Walsh Architects & Project Managers Limited (which can include the Account Details) are listed below.
View CRO company documents and company reports any Irish company or business with ease.
Satisfy due diligence requirments on Dw Walsh Architects & Project Managers Limited in one single 'time-saving' search. Run full background checks for Fitness and Probity on the directors of Dw Walsh Architects & Project Managers Limited and Anti-Money Laundering checks (AML checks) on Dw Walsh Architects & Project Managers Limited
|Donal Walsh
|Linda Walsh
|Company Type:
|LTD - Private Company Limited By Shares
Log-in now to run Due Diligence checks and Compliance checks on Dw Walsh Architects & Project Managers Limited or click Join-up to get started.
Background check Companies, Sole Traders or Individuals and minimise your spend with more efficient Anti-Money Laundering checks and reports.
An overview of the key bad debt judgment awards made against companies and consumers between January and August 2017
A breakdown of the key trends affecting Irish companies between January and June 2017
The new 4th Directive on AML is effective this June 2017. Identify PEPs in your customer base faster and easier with our AML solution.
The Vision-net alerts service is one of the most comprehensive on the market. It takes the cost and time out of monitoring the companies that are important to your business.


text:
Format: Paperback
Language: English
Format: PDF / Kindle / ePub
Size: 7.04 MB
Downloadable formats: PDF
Pages: 367
Publisher: Skyscape (January 27, 2015)
You Are My Only
Addicted to Love
Since You've Been Gone
What Comes After
Sometimes this impact is immediate and direct (injuries and deaths), but, more often, it is long-term, affecting emotional development and overall health download. In this cohort of postmenopausal women, we found a relationship between exposure to abuse and poorer psychological health. At baseline, abused postmenopausal women had lower scores (45.6 vs 53.7) on the RAND 36 MCS scale, greater depressive symptomatology, greater social strain and lower optimism compared with nonabused postmenopausal women epub. At times, they can accomplish tasks far beyond the abilities of their peers. At the next moment, they can be confronted with a task that they cannot accomplish. Many dyslexics call this "walking into black holes." To deal with these kinds of problems, dyslexics need a thorough understanding of their learning disability You Are My Only. At the time you may think that this is some kind of indication of a willingness to work on their past problems, or that somehow it will be different for you. In fact, what they are looking for is absolution in advance for behavior they will later inflict on you. They may even go so far as to say, "I told you this is how I am." Emotional abusers often grow OLD without growing UP Black-Eyed Suzie. Hispanic immigrant parents view these cultural values as important as they strive to teach their children the importance of respecting the elderly and addressing elders with the proper titles Melt (The Rough Romance Trilogy). Radical Feminist Therapy: Working In The Context of Violence. Chapter 8 and 9 are dedicated to the subjects of opposite and same sex partner abuse,both physical and emotional. Also helpful are examples of assessments and treatments from a feminist perspective. Lists of examples of abuse are offered to add to one's insight of the many dimensions of emotional abuse Shift. A Case Not Requiring a CPS Response Within 24 Hours: During the first 3 months of school, the children of a single mother were absent over one-half of the days. When the 7-year-old girl and 10-year-old boy do come to school, they have severe body odor and dirty clothes The Buffalo Tree.
Download Atlanta Burns (Atlanta Burns series) pdf
Everybody Sees the Ants
Promises to Keep (Bluford High Series #19)
The Unspoken
Letting Go of Bobby James: Or How I Found My Self of Steam
Eyes of a Stalker: A Shelby Belgarden Mystery (Shelby Belgarden Mysteries)
The Night Screams
Beaten (Surviving Southside)
A Finders-Keepers Place
Harmony House
Bad Boy
The Journey Back
Finding Katie: The Diary of Anonymous, A Teenager in Foster Care


text:
Here ericdress.com shows customers a fashion collection of current suede wedge shoes.You can find many great items. They all have high quality and reasonable price. You can get big discount sometimes, because we always do promotions. If you need suede wedge shoes,come here to choose. We give you good items and services.
Ericdress Comfortable Wedge Heel Rhinestone Wedding Shoes
Ericdress Suede Point Toe Wedges with Bowtie
Ericdress Floral Print Ankle Atrap Wedges
Pink Sweet Bowknot Peep-toe Lace Flowers Sandals
Sweet Bowknot Suede Stiletto Pumps
Ericdress Suede Metal Ankle Strap Wedges
Ericdress Round Toe Suede Chunky Heel Pumps
Ericdress Suede Platform Metal Chain Tassels Pumps
Ericdress Suede Cross Buckle Peep Toe Platform Pumps
Comfortable White Suede Wedge Heel Shoes
Ericdress Beautiful Satin Bowknot High Heel Wedding Shoes
Ericdress Comfortable High Heel Wedding Shoes
Ericdress Beautiful Peep-toe Pearls High Heel Wedding Shoes
Ericdress Lace-up Peep-toe Wedding Shoes
Classy Appliques Point Toe Ankle Strap Low Heel Wedding Shoes
Ericdress Superb Blue Crystal Handcraft Wedding Shoes
Ericdress Simple Lace High Heel Wedding Shoes
Ericdress Pretty Satin Bowknot High Heel Wedding Shoes
Ericdress Satin Crystal Peep Toe Wedding Shoes
Ericdress Princess Satin Pearl&bowtie Peep Toe Wedding Shoes


text:
Electrical Inspections Contractor IN
Topeka and Lawrence, Kansas
Many components are checked during a Topeka and Lawrence, Kansas area elctrical inspection by Greenwave Electric. The reason behind commercial electrical inspections is to keep a business safe for employees and customers to come in. Every business has an initial inspection done by an official during the building process. To make sure that you will pass this type of inspection, our Topeka and Lawrence, Kansas electrician works to follow all safety and local codes, and can be present with the inspector while the inspection takes place in order to know exactly what needs to be done. Electrical inspections also occur before the sale of a commercial building just to make sure it still remains safe.
The main phase of our Topeka and Lawrence, Kansas electrical inspections is to look for any code violations. Our electricians will work to follow each and every code that is made available to them, but with any jobsite, there might be at least one code violation. After these code violations have been brought to the attention of the builder, the electrican inspections electricians will go in and follow the list made by the inspector to fix everything properly. You can also have your electrician do this same type of inspection on a house you are looking to purchase to make sure everything is in solid working order.
Our Topeka and Lawrence, Kansas Electrical Inspections Contractor Provides:
- Electrician Inspections
- Commercial Electrical Inspections
- Electric Inspectors
- Wiring Inspections
- Electrician Building Inspections
- Commercial Electrician Inspections
- Electrician Inspection Consultations
- Electrical Project Inspections
- Electric Service Inspections
We Provide Electrical Inspections in the Following Areas:
Topeka, KS | Lawrence, KS | Olathe, KS | Silver Lake, KS | Wakarusa, KS | Leavenworth, KS | Atchison, KS | Auburn, KS | Berryton, KS | Tecumseh, KS
Why Topeka and Lawrence, Kansas Businesses Should Hire Us For Their Electrical Inspections
At Greenwave Electric, your experience includes treating you and your business with respect. We enjoy our Electrical Inspections work and it shows! From start to finish, our Topeka and Lawrence, Kansas area Electrical Inspections electrical contractors work with you, the client, every step of the way. Our goal is to ensure that your Electrical Inspections project is completed just the way you expect it. We strive for excellent service, that's on time and at an affordable price.
Our Topeka and Lawrence, Kansas electricians have years of experience in the Electrical Inspections field. This experience allows us to complete our projects on time and with unmatched quality. Call on Greenwave Electric today to find out why countless other Topeka and Lawrence, Kansas Area business have chosen us for their electrical service needs.
We look forward to welcoming you as a customer and being your preferred electrician whenever you have a Commercial Remodels & Additions problem in your Topeka and Lawrence, Kansas business. When you invite our Topeka and Lawrence, Kansas electricians into your business, you will encounter not only expertly trained service, but experts who are trained to deliver to you a tailored and unique service experience. Whether you need electrical repairs, electrical upgrades, or just need a professional electrician, Greenwave Electric has you covered.
What Topeka and Lawrence, Kansas Clients Are Saying About Our Electrical Inspections
"Very pleased with work and I appreciate the ability to do it on Saturday to prevent from downtime in our office. I am also thankful for the additional, unanticipated work they did after realizing cables were put in the wrong location by the phone co."
- Heather | Rated: 3 / 3
Recent Articles About Topeka and Lawrence, Kansas Electrical Inspections
- Why Do Circuit Breakers Trip?
- Reasons for Getting a Lawrence Electrical Safety Inspection
When a circuit breaker trips in your Topeka home, it will cause a power outage, then you'll have to go […]
It's easy to take for granted the luxuries that our modern day electricity in our homes offer but quick to […]
For electrical service in Topeka and Lawrence, Kansas, complete our online request form, or call an experienced electrician today!
(785) 845-2360


text:
Albatros for lunch today. Burrito breath.
September 17, 2012 Leave a comment
Had lunch with some good peeps today. We went for take out Mexican food at Albatros in Lake Forest. I had their Mixed burrito (shredded beef and beans) 4.5 burps.
Check out Nar with his fish taco…that there thing is huuuuuge.


text:
Boole Server is a security platform designed to protect against unauthorized viewing, manipulation or distribution of data. It provides secure storage and access to data along with full auditing capabilities. Boole Server is the result of extensive study in to the vulnerabilities of security systems.
Providing a high level of protection from both opportunistic and determined attempted security breaches. Boole Server enables organizations to securely share and protect all their confidential documents anytime, anywhere through the most advanced on-demand document security solution available.
The first version of Boole Server was released to the market after several years of development and beta testing.
Distributed in India, UK, Italy, France, Switzerland, Luxembourg, Middle East and Asia, Boole Server serves a variety of industries and customers.
Also, Boole Server maintains a number of strategic partnerships with industry leaders such as Accenture.
Boole Server is committed to delivering the highest quality of security standards with its products, as well as exceptional customer service and support.
Read more
Group Websites


text:
Comparison studies between physiological tests are often unsatisfactory for assessing their ability to distinguish between subjects. We recommend a simple but comprehensive protocol, using duplicate testing, that compares tests using1) the discriminant ratio (DR) between the underlying between- and within-subject SDs,2) correlation coefficients adjusted for attenuation due to test imprecision, and3) unbiased estimation of the underlying linear relationship between test results. The following five alternative methods for assessing glucose tolerance were compared: fasting plasma glucose (FPG) as a single sample or as the mean of three 5-min samples (FPG3); the 1- and 2-h glucose during a low-dose intravenous glucose infusion (CIG); and the 2-h plasma glucose from a 75-g oral glucose tolerance test (OGTT). All tests had similar DRs ranging from 2.6 to 4.2. The adjusted correlation between FPG and CIG tests approached unity, and those between OGTT and other tests were ∼0.9, showing that FPG3 provides similar information to the OGTT. FPG concentrations of 6.0 and 7.1 were found equivalent to the 1985 World Health Organization OGTT thresholds for impaired glucose tolerance and diabetes (7.8 and 11.1 mmol/l).
- plasma glucose
- precision
complex physiological responses, such as glucose tolerance, may be assessed by many different methods. For instance, the control of plasma glucose may be measured by the fasting plasma glucose (FPG), by the response to an oral or an intravenous glucose challenge, by the percentage of glycated hemoglobin (HbA1c), or by the plasma concentration of fructosamine. Different clinical tests are used to assess the same physiological variable because of different clinical circumstances, the application of advances in understanding and technique, or as a result of historical circumstances or fashion. The tests may assess differing or equivalent aspects of a physiological characteristic and may express their results in different units of measurement.
In this paper, we recommend a simple but comprehensive methodology for comparing different tests of a continuous physiological variable, which may be applied irrespective of the scales of measurements used. For such comparisons, it is important to include assessment of both the between- and within-subject variation of each test, as this allows comparison of the ability of tests to discriminate between different subjects, determination of the underlying correlation between tests after adjusting for attenuation due to within-subject variation, and unbiased estimation of the underlying relationships between the results of the different tests.
Omission of any of these components in a comparison study will provide inadequate information for choosing a particular test for a particular situation. The imprecision, or the within-subject variation, of a test is of little use by itself and must be considered in relation to the range of the test results. The "coefficient of variation," which relates imprecision to the midpoint of the range, is often inappropriate as it does not take account of the dynamic range of a test (1) and is not always comparable between different scales of measurement. The present paper introduces the concept of "discrimination," i.e., the ability to distinguish between individual subjects within a specified range of interest. This can be expressed as the discriminant ratio (DR), which is defined here as the ratio of the underlying between-subject SD (SDB) to the within-subject SD (SDW). The DR has a defined distribution, and DRs for different tests can be compared statistically.
The correlation between different tests must be included in any comparison. Test imprecision will diminish, or "attenuate," the measured correlation coefficients in a well-described way (11), and it is important to correct for this so as to be able to assess the underlying "true" correlation, as this represents the degree to which the tests are assessing the same physiological trait. This attenuation adjustment can be expressed in terms of the DRs of the respective tests.
Finally, the comparison of tests must, if possible, relate measurements by one test to those by another. Where the relationship is linear, the gradient of the relationship obtained by least squares regression is underestimated ("regression dilution") when both tests are subject to appreciable measurement error. An unbiased technique for deriving the relationship is therefore necessary, and, while many methods are available, a suitable method (27) is recommended here.
The assessment of glucose tolerance is a specific area where several methods are available to an investigator, for instance, by the measurement of steady-state plasma glucose or the response to a glucose challenge, either oral or intravenous. The standard test of glucose tolerance has, until recently, been the oral glucose tolerance test (OGTT; see Ref. 33), but, in practice, this test is not often performed. This is partly because of the inconvenience of a 2-h test and partly because of the marked variability of the OGTT. The poor reproducibility of the test (12, 21, 24, 26), which has a reported coefficient of variation of the order of 15–40%, due in part to the variable rate of gastric emptying, has predictable effects on reclassification of subjects on repeat tests, with a change in status on repeat testing in 30–60% of cases of impaired glucose tolerance (IGT; see Refs. 9, 13, 26, 28, 29) and predictable regression to the mean (26). The simple measurement of FPG has been suggested as a preferable measure (4, 8, 15, 20), and a continuous intravenous infusion of glucose (CIG) can assess glucose tolerance and give simultaneous measures of pancreatic β-cell function and insulin resistance (16). The FPG thresholds for diabetes have been set using outcome data (4). We evaluate and compare the performance of this and other tests throughout the physiological range.
This paper outlines the concepts and components of a physiological comparison study and uses as an example the assessment of glucose tolerance by either 1) the FPG (single sample or mean of 3 consecutive samples),2) the 1- and 2-h responses to a CIG, and 3) the standard 2-h response to the OGTT, in repeated tests in 30 subjects spanning the range of glucose tolerance.
Statistical Methods
This paper considers the following three aspects relating to the assessment and comparison of different tests for measuring an underlying physiological variable such as glucose tolerance:1) the ability of a test to discriminate between different subjects and comparison of discrimination between different tests;2) the correlation between pairs of tests, adjusting for bias due to within-subject variation. Such variation attenuates measured correlation coefficients so that they underestimate the underlying true correlation. This is important in assessing the degree to which different tests purporting to assess the same underlying trait differ with respect to systematic between-subject factors as opposed to random within-subject variation; and3) in cases where the relationship between a pair of tests appears to be a linear, unbiased estimation of the underlying line of equivalence between them.
Each of these aspects is required for a comprehensive comparison study and is based on a combination of well-recognized and novel concepts. Our approach is considered on a conceptual level at this point, with a more rigorous statistical treatment reserved for appendixes i-iii.
Discrimination between subjects. All physiological measurements 1) are subject to imprecision, which may derive from biological, sampling, and analytic sources and 2) relate implicitly to variables taking on values within a particular range of interest. The performance of a particular physiological test will depend on the relationship between both of these characteristics. Absolute measurements of the imprecision of the test are only meaningful in relation to the range of values to which that test will be applied. The smaller the former is in relation to the latter, the greater is the ability of a test to discriminate between individual subjects. In the context of measurements being obtained from a series of individuals representing the physiological spectrum of interest, we propose a novel, simple index of discrimination, the DR, the ratio between the SD of the underlying subject means (SDU), and the SD of repeated measurements on the same subject (SDW).
The discrimination of a test is not a universal property but will relate to the spectrum of values in the population being studied. Hence absolute DR values are not comparable between different populations, but they are essential when comparing the practical application and performance of different tests in the same population.
Underlying SDU.
Because a physiological test may be applied to a variety of possibly nonuniform populations, it is important to assess the test in relation to its expected range of application. Subjects in a comparison study should be chosen to represent and to span the range rather than be randomly selected from particular populations of interest, and this range is characterized statistically by the SDU. The measured SD (SDB) will overestimate the underlying SDU due to the presence of within-subject variation, and it is important to adjust for this, using a standard formula, to yield an unbiased estimate of the SDU.
To relate simply to the between-subject variation, we must be able to assume a common within-subject variation for all of the subjects in the study. This property is called "homoscedasticity" and can be checked by simple plots of the data (see appendix ). Lack of homoscedasticity can often be rectified by an appropriate numerical transformation of the test results.
For homoscedastic data, the common within-subject variance is simply the mean of the individual within-subject variances.
DR. As outlined above, the DR is defined as the ratio SDU/SDW. In a comparison study where kreplicate measurements are performed in each subject, the measured SDB is calculated as the SD of the subject mean values (calculated from thek replicates). The standard mathematical adjustment to yield SDU is so that the DR is calculated simply as This result may also be obtained by an analysis of variance (ANOVA) approach, using a fixed effects model, and this is presented inappendix . The appropriate equation is then where MSB is the between-subject mean square, MSW is the within-subject mean square, and k is the number of replicate tests in each subject, as above.
Assuming that the within-subject variation is normally distributed, we have used an analytical approach to derive confidence limits for estimated DR values and to test for the significance of differences between the DRs of different tests. We present these in appendix , where we also discuss our methodology in relation to alternative measures of test "reliability," in particular the intraclass correlation coefficient (ICC; see Refs. 5, 30).
Correlation between pairs of tests. Two tests designed to assess a complex physiological characteristic, such as degree of glycemic control, may use different methodologies, neither of which may perfectly represent the characteristic in question. The results of the tests may differ in systematic ways, independently from their random within-subject variation. The degree to which the tests measure the same characteristic may be assessed by the correlation between their results in a set of subjects. In the absence of within-subject variation, the degree to which their correlation falls short of unity would represent the extent to which the tests either fail to measure precisely the same characteristic or to measure aspects of the underlying characteristic that are differentially influenced by other factors that vary between subjects. It is this underlying true correlation that we are interested in here.
Test imprecision, however, further attenuates the observed correlation so that imperfect correlation will usually be due to a combination of the systematic between-subject factors discussed above and the presence of within-subject variation. A study comparing tests must distinguish between these two components, and this can be achieved using a standard formula that corrects measured correlations for attenuation (11). Such a correction requires knowledge of both the within- and between-subject variation of the measured values. Because the DR incorporates both of these elements, the correction can be expressed in terms of the DR where (seeappendix ). The corrected correlation coefficient, therefore, represents the degree to which the tests represent the same physiological characteristic, independent of test imprecision.
Unbiased estimation of a linear relationship. Where two tests represent the same characteristic, their results will often be found to be linearly related (although this may require numerical transformation). The determination of this relationship will be important in relating the results of one test to those of the other. Within-subject variability will lead to a noisy relationship, and a statistical approach is necessary to estimate the underlying linear equation. Although least-squares linear regression is often used for this, it is not appropriate here since it assumes that the explanatory variable is free from noise. When this is not the case, linear regression will underestimate the slope of the equation, a well-recognized effect termed regression dilution.
There is no perfect method of estimating the true relationship in these circumstances, but the method chosen here is that of least perpendicular distances corrected for scale differences, which has been shown to perform well in relation to other methods (27). The equations for this are presented in appendix .
Design of comparison studies. There is no single measure that can be used to compare physiological tests. Assessment and comparison of test discrimination and the determination of their underlying correlations and inter-relationships are equally important components. These all require simultaneous consideration of both between-subject and within-subject variation within the physiological range of interest. A comparison study should, ideally, assess both of these factors by using replicate tests in subjects chosen to span that range.
Experimental Protocols
Subjects. Thirty white Caucasian subjects were studied, consisting of 10 normoglycemic subjects, 9 subjects with IGT, and 11 with type II diabetes according to 1985 World Health Organization (WHO) definitions (33). All subjects were on a weight-maintaining diet and had not changed their medication for 4 wk before the tests. Subject characteristics are presented in Table1 by glucose tolerance group.
Protocols. Each subject was studied on four occasions within a 6-wk period. After a 12-h fast, subjects went to the hospital and sat on a bed for the duration of the tests. Tests were each performed on two occasions in the same subject and in random order.
FPG and CIG. Two cannulas were inserted in the same arm. One, for blood sampling, was placed at the wrist or on the dorsum of the hand, which was heated by an electrical blanket to "arterialize" the venous blood. The other cannula, for infusion of glucose, was placed in an antecubital vein. A blood sample was taken at time −10 min, and the plasma glucose concentration at this single time point was termed FPG1. Blood samples were also taken at times −5 and0 min, and the mean of the plasma glucose at the three time points was termed FPG3. At time 0, a continuous 5 mg ⋅ kg ideal body wt−1 ⋅ min−1infusion (22) of 10% glucose was started and continued for 2 h. One-hour CIG and 2-h CIG glucose were defined as the means of the three plasma glucose concentrations in blood samples taken at 50, 55, and 60 min and at 110, 115, 120 min, respectively.
OGTT. A single cannula was placed in an antecubital vein for blood sampling. Fasting blood samples were taken at −10, −5, and 0 min. At 0 min, the subject consumed a 75-g glucose drink, and blood samples were taken at 30, 60, 90, and 120 min.
Biochemical Assays
Plasma glucose was determined by a hexokinase-based method (Boeringer Mannheim UK, Lewes, UK) on a centrifugal COBAS MIRA autoanalyzer (Roche, Welwyn Garden City, UK).
Estimates of Glucose Tolerance
Values for glucose tolerance using FPG1, FPG3, 1-h CIG, 2-h CIG, and 2-h OGTT are presented in Table 2 as median and ranges. Fasting and CIG measures were homoscedastic, and the within- and between-subject variations are illustrated as plots of difference (first test − second test) vs. mean (of the 2 tests) in Fig.1. The 2-h OGTT was found to have within-subject variation increasing with mean values, and this was corrected by log transformation, with the transformed data presented as difference vs. mean plots in Fig. 2 and as medians and ranges for the whole group in Table 2. The underlying SDU and SDW and the DRs of the tests are also presented in Table 2.
DR values for the five measures, together with the one SE range of the estimates, are illustrated in Fig. 3. Although the lowest DR was the FPG1 and the highest the 2-h CIG, there was no significant difference between them on the overall statistical test ( = 6.2,P = 0.19, using Eq.10 in appendix ). Consideration was given to the exclusion of a subject whose inter-test difference in the 2-h CIG was four SDs from the mean of the rest of the group (see Fig. 1). In the absence of an identifiable reason for the large difference between his two test values, this subject was included in the analyses presented here, although, if he were excluded, the DR of the 2-h CIG would rise to 6.1, significantly greater than the DRs of the other tests.
Table 3 shows the Pearson correlations between the tests, both before and after adjustment for attenuation. The correlations were calculated between subject means of duplicate tests, to give the best estimate of the underlying relationships between tests. Adjusted correlations between the fasting and both of the intravenous measures were high, approaching one. Those between the 2-h OGTT results and the other tests were somewhat lower (∼0.9), indicating that there was some biological discordance in their relationship, independent of their within-subject measurement error.
Figure 4 shows the scattergram between the 2-h plasma OGTT (on a logarithmic scale) and the FPG. It also illustrates the line of equivalence, derived as explained above, and the linear regression line of the 2-h OGTT on FPG. The dilution effect of the within-subject variation on the regression line can be seen clearly. Table 4 gives coefficients for the unbiased linear equations relating the test values, and Table5 gives the points on the various scales that are equivalent to the 1985 WHO OGTT thresholds for IGT and diabetes.
SDW of the logarithm of the 2-h OGTT may be interpreted in relation to a standard interval in the range of glucose tolerance, for instance, the interval between the thresholds for IGT (7.8 mmol/l) and for diabetes (11.1 mmol/l). This interval is 0.153 on a logarithmic scale (base 10), whereas the SDW is 0.060. The difference between two individual measurements at either end of this interval would not be significant at the 5% level (P = 0.07), given the imprecision of the 2-h OGTT. In other words, it would not be possible to confidently distinguish between two such individual values. The same applies to all other tests examined, using the unbiased equivalent values to these thresholds given in Table 5, apart from the 2-h CIG plasma glucose, for which the two measurements at opposite ends of the equivalent interval would differ significantly (P = 0.013).
This study has shown that different methods of assessing glucose tolerance were broadly comparable in a range of subjects spanning normal glucose tolerance, IGT, and type II diabetes. This assessment involved the following three separate components:1) comparison of the discrimination of the tests, i.e., their ability to distinguish between different subjects, 2) determination of the degree to which different tests measure the same underlying physiological property, and 3) estimation of the underlying relationship between the test results. The assessment of the within-subject imprecision of each test is a fundamental requirement for this evaluation, so that comparison studies must involve at least duplicate measurements in all subjects in order to determine both the between- and within-subject measurement variation of each test in the same group of individuals. This is best done in subjects who represent a clinically meaningful range of glucose tolerance. To ascribe a single value to within-subject imprecision requires homoscedasticity, and numerical transformation of results may be necessary to achieve this, as illustrated by the 2-h plasma glucose from the OGTT. A measure of imprecision is important for the assessment of changes within subjects, such as over time or after interventions (10).
The determination of imprecision alone is not adequate, however, for the assessment of the practical value of a test. The standard methods of assessing imprecision, including the coefficient of variation, have little meaning on their own, without reference to the range of measurements to which they are being applied. The ability to distinguish between individuals within this range is here termed the discrimination of the test and is assessed using the DR.
The concept of discrimination should be distinguished from the ability of a test to categorize patients by an external gold-standard dichotomy. This is a particular concern in the field of clinical chemistry, for instance, when a biochemical test is being assessed for its ability to detect the presence of a malignancy. Receiver operating characteristics (34) have been used for this purpose. However, they are not suitable for the assessment of test results as a continuous scale of measurement or for the comparison of tests without reference to an external categorization. In the field of glucose tolerance, categories have been defined on the basis of thresholds in a continuous scale of measurement for the OGTT based on external criteria, but this is a notoriously imprecise test (7), and it would not therefore be appropriate to assess possible alternative tests using a categorical approach based on these thresholds. The use of the DR provides a means of comparing how well the subjects studied can be reliably distinguished by different tests, which is an important component of a comprehensive comparison of imprecise tests. For instance, in many research studies using continuous variables, the statistical power to distinguish between groups of subjects or to determine correlations between variables will depend on discrimination.
A very similar concept, referred to as reliability, has been assessed, particularly in the psychological literature, using the ICC. This relates the covariance of replicate test results to the combined between-subject and within-subject variance and is algebraically related to the DR. However, in the context of discriminating between different subjects, it is not as easy to grasp as the DR, which has a direct intuitive relevance when considering a test's application. Furthermore, it is not as easy to derive statistical tests comparing different ICC values. A recently described method for the comparison of two ICCs does not extend to the comparison of more than two tests and was only validated for studies employing repeated tests in 100 subjects or more (2).
The choice of a methodology for comparing different tests is intimately related to its theoretical basis and in particular to the availability of statistical criteria for assessing such a comparison. Much of the theoretical discussion of the ICC and treatments of comparative tests between ICCs have been based on random effects models in which the individuals in which the tests are being performed are assumed to be drawn at random from a known normally distributed population. This presupposes a particular population in which the tests are being applied. Complex populations (for instance those consisting of subgroups) would require appropriately complex statistical models. Unfortunately, even the simplest random effects models are hard to treat analytically, as in the case of the ICC quoted above. Our approach has been to concentrate on evaluating tests across a particular physiological spectrum of interest. In the example presented here, for instance, glucose tolerance represents a physiological and pathological unity irrespective of the its distribution in particular populations. For this purpose, we take the view that it is appropriate to perform a comparison study in subjects selected to span the range of interest, which may be analyzed by a fixed-effects statistical model. Such an analysis is presented here for the DR, allowing the derivation of straightforward expressions for the SD and confidence intervals of the DR and the evaluation of the statistical significance of differences between the DRs of different tests.
Although the comparison of the DRs of different tests in the same study is valid, the DR in itself is not a universal characteristic of a test, as it will depend on the choice of subjects on which the comparison is performed. When the subjects cover a greater range of values, the DR will be larger, and vice versa, and the DR calculated when subjects are selected to span a range of interest will generally be larger than if subjects had been chosen randomly from the same population. However, it is a fundamental property when it comes to a test's practical application, and, unlike imprecision, it can be used as a basis for comparison between tests. The DRs of the five tests examined here were not significantly different, in spite of the increased complexity and expense of the CIG and the OGTT.
Two perfectly precise tests assessing the same physiological variable would be perfectly correlated. Departures from perfect correlation can be due to the following two factors:1) underlying differences not directly related to the variable of interest. These will manifest themselves as systematic differences between subjects; for instance, in the assessment of glycemic control, which is determined by the FPG and OGTT in qualitatively different ways, the underlying correlation may fall short of unity because of the influence of factors such as the effect of gastric emptying and the influence of intestinal incretin effects that differ between the two methodological approaches;2) diminution of the underlying correlation may also arise from within-subject variation, and this is a well-described statistical effect termed "attenuation," which may be adjusted for by standard techniques to enable the estimation of the underlying correlation (11). This adjustment depends on both test imprecision and the degree of variation between subjects and so can be expressed in terms of the test DRs.
Adjustment for attenuation will establish the degree to which the underlying correlation differs from unity due to the factors detailed in factor 1 above.
In this paper, the adjusted correlation coefficients between the fasting glucose and the CIG approached unity and were only slightly lower between these tests and the 2-h OGTT. Although additional factors unrelated to the homeostatic control of plasma glucose, such as variable gastric emptying, would contribute to this, the relatively high overall intercorrelations and the simplicity and cheapness of the FPG would recommend this as the measure of choice for the assessment of glucose tolerance.
When two tests have an underlying structural relationship between their measurements (or transformations of these) that is linear, it can be instructive to determine the equation of the "line of equivalence." Linear regression, although often used, is unsatisfactory since it assumes perfect precision in the independent variable and is subject to regression dilution. There have been many approaches to deriving an unbiased estimation, as comprehensively reviewed by Riggs et al. (27), and the "weighted least squared perpendicular distance" approach (Riggs' "PW" method) has been used in this paper. In the present study, the assumption of linearity, within the limits set by the imprecision of the tests, was supported by visual inspection of plots of the relationships (data not shown). The FPG threshold concentrations recommended by the American Diabetes Association (4), based on studies of the prevalence of retinopathy in three distinct populations, were confirmed as equivalent to the established OGTT thresholds for IGT and diabetes.
We also calculated the SD of loge(DR) estimates for different numbers of replicate tests, using the Taylor series expansion, subject to a constraint on the total number of tests performed. For a study comparing two methods using a total of 60 tests, the power to detect a difference between DRs of 2.5 and 4.0 is 58% using 30 subjects and two replicate tests, rising to 72% with 20 subjects and three tests, an increase in power of 24%. Further increasing the number of replicates gives smaller increases in power, e.g., for four tests in 15 subjects the power is 78%, with even smaller gains for more than four replicates. There would appear to be some advantage in using three, rather than two, replicate tests in each subject, but there is little advantage in increasing the number beyond three given the need to obtain sufficient subjects to adequately cover the range of interest.
This study showed that, with OGTT done under carefully controlled conditions, with a reproducibility that is somewhat better than reported elsewhere, it was not possible to distinguish between the WHO thresholds for IGT and diabetes at a 5% significance level, and this was also the case for FPG, even when the mean of three samples at 5-min intervals was assessed (the between-sample variation being relatively small in relation to the between-day variation). It is therefore not surprising, with two thresholds close together, that repeat measurements often give change of status. Improved classification could be achieved by taking the mean of determinations on more than one day. However, although these classifications may be useful for epidemiological purposes, for practical purposes the actual OGTT or FPG value is more informative (31, 32).
In summary, we have outlined a comprehensive but simple methodology for the comparison of imprecise tests, encouraging1) comparison of test discrimination, expressed as the DR,2) the evaluation of the degree of agreement between tests based on correlation coefficients adjusted for attenuation, and 3) in the case of a linear relationship between test results (or their mathematical transformations), the use of an unbiased method for estimating the underlying equation. For such a comparison study, it is important to determine the within-subject variation of each test as well as the variation between subjects. Application of these methods to various tests of glucose tolerance demonstrated similar discrimination, acceptable agreement, and an unbiased estimation of the FPG values equivalent to those of the 2-h OGTT. The latter agree closely with the outcome-derived thresholds currently being recommended by the American Diabetes Association. However, because the thresholds for IGT and diabetes are within measurement error and cannot be reliably distinguished, the absolute 2-h OGTT or FPG is more informative than the categorization.
We are grateful for the assistance from Dr. Sue Manley and Nuala Walravens.
This section presents a more detailed mathematical treatment of the concepts outlined in methods.
Discrimination Between Subjects
Statistical model. We consider the comparison of different tests, each measuring the same physiological variable. Each test is performed ktimes on each of n subjects, with the order of the tests being randomized for each subject.
Considering first a single test in isolation, an appropriate model is Equation 1whereXij is the result of the test performed for thej'th time on thei'th subject, μ is the overall mean value of the variable in question on the scale of the current test, and αi is the true value of thei'th subject, measured as a deviation from the mean (thus Σi =1, nαi = 0); εij represents day-to-day variation, which includes both biological and assay variation; the εij are assumed to be independent, normally distributed random variables with mean zero and variance ς2.Equation 1 is a standard one-way ANOVA.
The assumption of constant variance (or homoscedasticity) of the error term, ς2, can be checked graphically. If k ≥ 5, we can calculate the quartiles of the kreplicate test results for each subject and plot log(interquartile range) against log(median) (14). If 2 <k < 5, plot the SD of thek replicates against the mean for each subject (23), and if k = 2 plot the differences (1st − 2nd replicate) between the pairs of tests against the subject means (6). If the assumption of homoscedasticity holds, the plotted measure of variation [log(interquartile range), SD, or difference] should be approximately constant across the range of subjects. If there appears to be a systematic relationship between the measure of variation and subject medians or means, this can often be removed by mathematically transforming the results ofXij . A common case in physiological measurements is where the SD increases in direct proportion to the mean, when a log transformation of theXij stabilizes the variance and log(Xij ) can then be used in place ofXij in Eq. 1 . Other transformations can be considered for different relationships between the subject SDs and means (14, 23).
It is also possible to check the assumption that the εij have a normal distribution by plotting the ordered residuals from fitting Eq.1 against standard normal deviates in a "normal probability plot" (3). However, the ANOVA procedures used here are fairly robust to moderate departures from normal distribution and can be used without such sophisticated checking, provided homoscedasticity of variance holds and the data do not exhibit marked skewness.
In these experiments, subjects are selected to span a range of glucose tolerance and are not chosen randomly from a prespecified population. The subject effects αi are therefore considered as "fixed" rather than "random" effects.
DR. As a measure of discrimination between subjects, we define the true DR, Δ, as the ratio of the underlying SDB to the SDW Equation 2Unbiased estimates of the between- and within-subject variances are given by (MSB − MSW)/kand MSW, respectively, where MSB and MSW are the between- and within-subject mean squares from a standard one-way ANOVA, i.e. Equation 3 Equation 4and Mi = Σj =1, k Xij /kand M = Σi =1, nMi/n, the subject and overall means.
We then estimate Δ empirically as the ratio of the between- to within-subject standard deviations Equation 5The DR is algebraically related to the ICC, which is commonly used as a measure of the reliability of tests (5, 30) Equation 6
However, the methodology developed for ICCs is in the context of a random effects model, rather than the fixed effects model used here, so published results for SDs and confidence intervals cannot be used. The DR gives a measure that is intuitively closer to the idea of discrimination between subjects, whereas the ICC is a measure of correlation. In addition, for tests with good discrimination, ICC values tend to cluster unhelpfully close to their upper limit of one. Furthermore, there is no simple practicable test available for the comparison of ICCs from different tests in a random effects model. A recently described method for the comparison of two ICCs does not extend to the comparison of more than two tests and was only validated for studies employing repeated tests in 100 subjects or more (2). We have derived straightforward expressions for the SD and confidence intervals of the DR in a fixed effects model and a test for the equivalence of DRs in a comparison study.
Confidence limits for DR. Confidence limits for the DR can be found by noting that Equation 7whereF 0 = MSB/MSWis the standard F statistic from the one-way ANOVA. F 0has a noncentral F distribution with degrees of freedom ν1 =n − 1 and ν2 =n × (k − 1) and noncentrality parameter λ can be estimated by (n − 1) × k × DR2, and a 95% confidence interval for Δ is Equation 8whereF L andF U are the lower and upper 2.5% of the noncentral F(17).
Noncentral F tables are not widely available (18), and a reliable approximation toF L andF U can be made using a central F distribution (25) where and are the lower and upper 2.5% of a centralFν,ν2distribution and Comparison of DRs. We derived a test for the equivalence of several DRs by assuming the following model Equation 9 Xijh is now the result of the h'th test performed for the j'th time on thei'th subject, μh is the mean value of the variable in question on the scale of testh, and αih is the true value of thei'th subject measured using testh(Σi =1, nαih = 0 for each testh); εijh are again assumed to be independent, normally distributed random variables with zero mean and variance .
Using this model, the DRs for each test are statistically independent. We used simulations (see appendix ) to show that the distribution of loge(DR) is approximately normal if the model assumptions hold. We then used Cochran's theorem (19) to show that the statisticQ has a χ2 distribution withr − 1 degrees of freedom, where Equation 10
The DRs are unequal at a significance level of 0.05 ifQ exceeds 95% of the distribution.
We derived an expression for sh, the estimated SD of Lh, from the mean and variance of the noncentral Fdistribution using a Taylor series expansion; details are given inappendix .
Alternative models. The models we have used, given by Eqs. 1 and 9 , for observations from the comparison study have been deliberately chosen for their relative simplicity. Although some of the algebra is intricate, all of the calculations we have presented can be easily implemented using spreadsheet software and do not require the use of specialized statistical packages. However, some of our model assumptions do warrant further discussion.
First, the choice of a fixed rather than a random effects model is unusual in this kind of context. However, subject selection in our study is clearly nonrandom in that we have deliberately chosen roughly equal numbers of normal glucose tolerance, IGT, and diabetic subjects. Even within each of these subpopulations, sampling is unlikely to be random as subjects are sought to span the range of interest as evenly as possibly, which is likely to result in oversampling from the extremes of the distribution. Such a sampling scheme is likely to produce a DR that is higher than that which would be obtained from a random sample from the population, and its use is restricted to comparison with other tests in the same study. It is not appropriate to formally compare test DRs that have been derived from different populations.
A population consisting of clearly defined subpopulations might be best treated using a mixed model, or even structural equation modelling. This would require more sophisticated analytical techniques and a larger scale of comparison study than we have presented in this paper. In the particular example presented here, however, the "subgroups" are not clearly separable but are arbitrarily defined by thresholds in a continuous spectrum. In this situation, which is relatively common in physiology, the approach taken here would be adequate, relatively simple, and practical. A formal comparison of the use of more complex models and the simple approach made here is beyond the scope of this paper.
The assumption of independence of the error terms εijh is unlikely to be completely true since most biological measurements exhibit some degree of "autocorrelation," i.e., correlation between successive measurements made on the same subject. In the context of these studies, where repeat measurements are almost always made on different days and often several days or even weeks apart, the magnitude of such autocorrelation is likely to be small compared with the total within-subject variation in which we are interested. Furthermore, accurately estimating autocorrelation coefficients would be difficult in relatively small studies, and the degree of mathematical complexity would increase such that specialized statistical methodology and software would be needed, rendering the procedures inaccessible to many researchers. However, our methodology might not be appropriate where repeat measurements were made within the same day or where other biological reasons existed for suspecting nonnegligible autocorrelation.
Correlation Between Pairs of Tests
The nature of the relationship between a pair of tests can be examined graphically by plotting the subject means for the first test against those for the second. In many cases, particularly after transformations to ensure homoscedasticity, the relationship will be approximately linear, and the degree of correlation can be assessed using the Pearson product-moment correlation coefficient,r (3).
In the model of Eq. 9 forr = two tests, we are interested in the correlation between the underlying subject means αi 1and αi 2. However, in the presence of within-subject variation, the sample correlation coefficient, i.e., the correlation between the two sets of observed subject means, underestimates the true correlation between the tests; this effect is known as attenuation and means that, even if the true subject means αi 1and αi 2were perfectly correlated, the correlation between the observed subject means would be less than unity because of the random fluctuations due to within-subject measurement variation.
Standard results from measurement error theory (11) show that the correlation between two measurements, both of which are subject to error, is attenuated by the factor where κ1 and κ2 are the reliability coefficients of the two tests. From Eq.6 where DRM is the DR of the means Mi,i = 1,...,n, of thek replicate measurements on each subject, rather than of the individual measurements themselves.
Taking the mean of Eq. 1 over thek replicates yields where ξi is normally distributed with mean of zero and variance ς2/k. Thus, in Eq. 2 , ς must be replaced by ς/ , which we estimate by , and fromEq. 5 Equation 11Thus The Pearson correlation coefficient rcan be adjusted for attenuation by dividing it by η, i.e. wherer adj is the adjusted r.
In cases where the relationship between the tests is clearly nonlinear, the Spearman rank correlation coefficientr s should be used in place of r to assess the comparability of the tests. However, there is no universal formula for the attenuation ofr s in the presence of measurement error.
Unbiased Estimation of Linear Relationship
In the case where the relationship between a pair of tests is linear, it may be useful to obtain unbiased estimates of the gradient and intercept of the line. Linear regression gives biased estimates because it only considers errors in the dependent variable, and clearly both tests here are subject to error; the gradient is always underestimated, and regression of subject means from test 1 on those from test 2clearly gives a different relationship to that of test 2 on test 1.
The method that we have chosen to estimate the linear relationship between the subject mean measurements from test 1 and those from test 2 is that of "perpendicular least squares, properly weighted." This essentially minimizes the sum of the squared perpendicular distances between the observed data and the fitted line, but with an adjustment that makes the method invariant to linear transformations of the measurement scales. If Mi 1and Mi 2,i = 1,...,n, are the subject means (over the k replicate tests), then the estimated gradient is where S11 = Σi =1, n(Mi 1− M1)2; S22 = Σi =1, n(Mi 2− M2)2; and S12 = Σi =1, n(Mi 1− M1) × (Mi 2− M2). M1 and M2 are the overall means for each test, i.e., Mh = Σi =1, nMih/n,h = 1, 2, and θ = / .
The and are estimated from their respective MSW, so that we estimate θ by The intercept is then estimated as This method is described and contrasted with other methods by Riggs et al. (27), where it is shown to perform well under a range of values of θ when the correlation between the Mi 1and Mi 2is fairly high (above ∼0.5) and θ is estimated fairly precisely. Such conditions are likely to apply in these experiments: the Mi 1and Mi 2are measuring the same underlying physiological variable so the correlation will be high, and ς1and ς2, and hence θ, are directly estimated from the repeat measurements using each test.
We used simulations to examine the distribution of the DR and loge(DR) and to check the accuracy of the Taylor series formula for the SD of loge(DR), given the form of model described by Eq. 1 . These were performed for all combinations of the following values ofn (number of subjects),k (number of replicate tests), and Δ (the true DR) For each of these combinations of n,k, and Δ, the following procedure was performed.
1) An arbitrary subject mean μ was chosen, along with a set of n equally spaced subject effects αichosen symmetrically around zero so that Σi =1, nαi = 0.
2) The ς2, the within-subject variance, was calculated as 3) For eachi = 1,...,n andj = 1,...,k, a random observation εij was generated from a normal distribution with mean zero and variance ς2.Xij were then generated from Eq. 1 .
4) The DR and hence loge(DR) were calculated from theXij using Eqs. 3-5 .
5) The SD of loge(DR) was calculated from the Taylor series approximation (Eq. 17 ofappendix ), using the noncentrality parameter λ evaluated from the DR estimate at the current step of the simulation.
6) Steps 3–5 were repeated 500 times, yielding a distribution of 500 values for each of DR, loge(DR), and SD of loge(DR).
7) The distributions of DR and loge(DR) were checked for normality using the Shapiro-Wilk test and were plotted as histograms.
8) The true SD of loge(DR) was estimated from the simulated distribution of loge(DR).
9) The distribution of Taylor series estimates of the SD of loge(DR) was compared with the true SD by plotting the median, upper and lower quartiles, and 5 and 95% against nfor different values of k and Δ.
Examination of P values from the Shapiro-Wilk test showed some evidence that loge(DR) was not quite normally distributed (slightly >10% of the Pvalues examined were <0.05, but there was no apparent relationship between low P values andn, k, or Δ). However, this was a marked improvement over the DR itself, for which >50% of the P values were <0.05. Histograms also showed the distribution of loge(DR) to be symmetric, whereas that of DR was markedly positively skewed (data not shown). Loge(DR) was deemed to be sufficiently close to normal for use in the χ2 test for equality of DRs. Plots showed that the median Taylor series estimate for the SD of loge(DR) was generally within ±10% of the true value for n ≥ 10. However, for k = 2, the SD could be overestimated by as much as 20% for 10 ≤n ≤ 20. The distribution of SDs is positively skewed and, for n ≥ 10, the 5% SD was at most 10% below the true value. Overestimation could be more marked, but even the upper quartile SDs were within +25% of the true value (data not shown). Because the χ2 test statistic is a function of the reciprocal of the SD, the test is conservative with respect to overestimates of the SD, i.e., one is unlikely to wrongly reject the null hypothesis (no difference between the DRs), but the test may not be particularly sensitive to genuine differences for small values ofn, especially ifk = 2.
Estimates of the SD become very inaccurate forn < 10, and the approximation should not be used in this range. However, we would not recommend performing an evaluation study of this kind on such a small number of subjects, since the objective is to characterize the performance of the tests over a reasonable range of the variable of interest.
We derived an estimate of the variance (and hence the SD) of loge(DR) using a first-order Taylor series expansion. From Eq. 7 whereF 0 = MSB/MSWhas a noncentral F distribution with degrees of freedom ν1 =n − 1 and ν2 =n × (k − 1) and noncentrality parameter Let LDR = loge(DR)
Expanding LDR as a Taylor series inF 0 about its mean gives, to first order Equation 12where LDR′ is LDR evaluated at and d(LDR)/d is also evaluated at . Hence Equation 13where var indicates variance. Now Equation 14where DR′ is DR evaluated at . From general properties of the noncentral Fdistribution (17) Equation 15 Equation 16
Substituting for DR′ from Eq. 15 into Eq. 14 and for d(LDR)/d fromEq. 14 and var(F 0) fromEq. 16 into Eq.13 gives Equation 17which is evaluated by noting that and replacing Δ2 by DR2, i.e.
Address for reprint requests: J. C. Levy, Diabetes Research Laboratories, Radcliffe Infirmary, Woodstock Rd., Oxford OX2 6HE, UK.
This study was done with aid of grants from Servier and the Alan & Babette Sainsbury Trust.
- Copyright © 1999 the American Physiological Society


text:
Alec Jack is a 48 year old beef & dairy farmer from Pakaraka. He and his wife Kelly have 4 children aged between 8 and 13 years old.
As a Massey University graduate, a Ballance Farm Environment Award winner, and a Nuffield Scholar, Alec has been involved in local farming leadership through:
· PPCS North Island Suppliers' Council (Northland Rep)
· Meat & Wool NZ Monitor farm program (Monitor Farmer)
· Beef & Lamb NZ Northern Farmer Council (Deputy Chair)
· Waitangi Catchment Collaborative Stakeholders Group (Public Interest Rep)
She is the CEO of Manaaki Solutions Limited and COO for Te Putea Whakatupu Trust. She serves as a Director of Northland Inc, NorthTec as a Council Member and humbled to be a trustee for Kaikohekohe Education Trust.
She is especially proud to call Tai Tokerau (Northland) home. Her whakapapa is to Te iwi Morehu, Te Aupouri, Ngapuhi-Nui-Tonu.
She has previously held other governance roles in Tai Tokerau in the Iwi commercial sector of fisheries, asset management and her marae. She acknowledges having been blessed to have grown up with amazing strong wahine around her and they are forever in her heart and mind when she makes decisions.
Karleen is very proud to own her own business in Tai Tokerau and the mahi she does for her whanau, haahi (church) and iwi. She is honoured to be able to contribute to the Kaikohekohe Education Trust, a game changer in education today for our future.
Sign in|Recent Site Activity|Report Abuse|Print Page|Powered By Google Sites