text:
Kahlon presents ministers with Arrangements Bill
Netanyahu: Justice will be dealt to Jerusalem gay pride parade attacker
Knesset closes summer session with antic-filled all-nighter
Livni: Justice Minister Shaked playing into BDS hands, will bring Israel to ICC
Fashion: From Barcelona to Bnei Brak
Food: The continuing saga
Recipes: Time for lime
July 31: Top 10 things to do
Late goal boosts Kiryat Shmona's hopes
Hasarma to make Arab history on sidelines
Kroyter jumps back into picture as high-flying Israeli phenom
Blue-and-white moves up gear in EuroBasket preparations
Journalists to appeal to High Court against IBA shutdown
Hotovely: Netanyahu government will back more settlement building
It takes a train to cry
Palestinian baby killed in suspected West Bank 'Price Tag' attack
Force-feeding law is 'license to kill Palestinian prisoners,' PA says
US, EU: Israel's authorization of 300 new settler homes harmful to peace
Seven Arabs arrested for attacking Jews on Jerusalem promenade
Why is Muslim harassment of Jews on the Temple Mount increasing?
Ex-hasid who committed suicide had mental issues, says family
Survivors appeal to Pope to intercede with Germany over Holocaust reparations
Jewish egg donations are hot commodity, but are they 'kosher'?
Increase in UK anti-Semitism this year 'due to better reporting'
World Jewry remembers Samuel Pisar, 'moral voice of a generation'
Netanyahu: The more one looks at the Iran accord, the worse it looks
Foreign Ministry calls Sunni Arab states 'Israel's allies'
Top US general distances himself from choice of Iran deal or war
Strategy emerges to reimpose US nuclear sanctions on Iran under a different name
Nusra Front abducts US-backed rebel leader, blunts efforts against ISIS
Iraqi militia leader says US not serious about fighting Islamic State
Canadian-Israeli anti-ISIS fighter: If we don't fight them there, they'll come here
My friends went on vacation and all I got was this stupid ISIS flag
Report: Top French official says if Congress rejects Iran deal, a better deal could be achieved
ISIS conned by Chechen women in internet recruitment scam
Arab MKs claim brucellosis 'epidemic' and demand budgets
Quality of Health Ministry's staff better than its image, says D-G
Knesset drops requirement for doctor's authorization for adults who want to work out in gyms
Israeli researchers: Type II diabetics should never forgo breakfast
Israel's largest solar field begins flowing to the national grid
Hydrocarbons found in second wells at Golan Heights oil project
Establish commission to examine water flow to Jordan, MK says
Steinitz, Netanyahu postpone vote on gas compromise outline
Mark Zuckerberg: Virtual reality is Facebook's next frontier
Budget deal to give NIS 18,000 'piggy bank' to 18-year-olds
Tech talk: Cybereason takes on Operation Kofer
Israeli safety app Musketeer saves lives
RX for readers: Spoiled by a sunburn
Jerusalem Post editorial: Don't parole Pollard
Jerusalem Post Letters to the Editor: US Jews and Iran
The fight for Israel
The lesson of Jonathan Pollard
Column One: Obama strikes again
Take two tablets!~
Sin Offering
Lincoln and the Jews
Portrait of an assassin's wife
The light makes his life
An American-Jewish idiom
The story of the Sabena pilot hijacked en route to Israel
'We were the only ones left alive'
The Tower of David– Full speed ahead
This Week in Jerusalem
Get on the shabus
Remembering three lone soldiers
No worst, there is none
Walking the line:the Animix festival
Wandering through fascinating Florentin
Who Cares? Israel does
מִשִירֵי רָחֵל: אֶל אַרְצִי
מִתְנַדְבִים מִיִשְׂרָאֵל
שִירַת רַבִּים 3
'נְתִיבֵי יִשְׂרָאֵל'
The path to aliya- Indian style
The journey of Ukraine's embattled Jews to Israel
A historic win, but no slam dunk
Art is for everyone, everywhere
Susiya: Injustice On Display
Trump change the race
The new KKK is radical Islam
Politically Koret: "We're on the Table"
Immoral World Cares More about a Lion than Jewish Victims of Arab Terror
Israel's Northern stars
Israeli Church leaders blame anti-Christian attacks on government inaction
Poll: Pope Francis' approval rating dips in US ahead of September visit
Oren: Jesus today would be a settler
Israeli Christian group challenges BDS with new guide
New Campsite for Youth Dedicated in Maginim Forest
New Hadid Single Dedicated in Ben Shemen Forest
Eucalyptus Park in Beit Shean Inaugurated
Tranquil Weekend in Nes Harim for Gaza Border Families
Italy's National Day at EXPO 2015
The Ambassadors' Club of Israel Tours the Galilee with KKL-JNF
The International Solidarity Mission Says Goodbye in Jerusalem
Chinese Ambassador Visits the Israel-China Friendship Fores
KKL-JNF Global Landscapes Seminar at Expo 2015
New Bio-filters Purify Surface Runoff Water for Reuse
Future Tour Guides Discover the Negev with KKL-JNF
The Shamir Drilling Project
$100 Million JNF Boruchin Israel Education Advocacy Center Opens
JNF Canada Pays Tribute to the Honourable John Baird
From Sunny Israel to Cold Canada
Walking for Water with KKL France in the Mountains of Eilat
Renewable Energies for the Future in the Arava
LDA Conference: Science and Technology for Forests
The Negev Conference on Agricultural Research and Development
Arava Open Day: Food Security for Israel and the Entire World
An Olah Reports From Nepal
How One Oleh is Helping to Transform the Way People Drink Water
From Think Tank to Environmental Consultant
Award Winning Designer Launches Career as Wedding Dressmaker
How Natalie Edwards Built a Successful Career in Israel's Startup Industry
5 Tips to Land a Job in Israel
Starting Your Own Business in Israel: Real Life Examples
5 Strategies for Finding Your Professional Place in Israel
36 Hours in Southern Israel
Food Shopping In Israel
How Women Can Have it All in Israel
The Eisenberg's Long Journey Home
Why Retirees are Thriving in Israel
Thanks to Aliyah video
6 Frequently Asked Questions about Israeli Healthcare
Starting Over in Israel
Sam Eisenberg: From Nativ to Aliyah
WATCH: Why Olim #LiveLoveIsrael
5 Ways that Olim Kick Back
VIDEO: Are you ready for the South?
Make yourself at home… in Israel
NBN Announces Bonei Zion Prize Recipients for 2015
NefeshB'Nefesh Provides Services to Bnot Sherut
How Doctors Are Impacting Israel's South
07.31.2015 | 15 Av, 5775
Residents of Jisr az-Zarqa beckon Israel Trail hikers to enjoy their town
Pope receives Madonna lily seeds from BGU scientists to bloom for spring Annunciation
Architect, Israel Prize laureate Prof. Avraham Yaski dies at age 87
Harvard law students get a legal tour of Israel
MK Regev forms new Knesset panel headed by Henin to address Haifa Bay area pollution
Peres: Purim is a great time to dress up, but not the only time
Tel Avivians run for cover as wild weather hits the country
Deputy finance minister backs Cinema City's Shabbat closure reversal petition
STEM education, another successful export from Israel
Piron: Even if a teacher is wrong, no room for dismissal over politics
Jpost.com is looking for a Jewish, Israeli name for the storm
Security forces on high alert as Israelis mark Yom Kippur
04/08/2010 02:28
(photo credit:Ariel Jerozolimski)
While the big names behind Jerusalem's luxurious Holyland housing project awaited trial in what is being labeled "one of the worst corruption cases ever," the complex itself – by far one of the capital's most lavish and expensive – sat quietly in the moonlight above Malha Mall and Teddy Stadium on Wednesday night, as a handful of residents trickled in and out.Home to some of Jerusalem's top businessmen and lawyers, the sprawling private neighborhood, which is made up of a series of multi-story buildings holding 340 housing units along with the "Holyland Tower" – a more than 30-story, 140-unit high-rise that is awaiting completion – the entire "Holyland Park" is a symbol of affluence, complete with an underground parking garage stocked with shiny luxury cars.Described by developers as "the peak of Jerusalem's luxury life," a sign at the entrance to the complex reads, "Holyland Park – a private neighborhood for people like you."Equipped with automatic glass doors and keypad entry systems, the lobbies of the Holyland buildings also include marble floors, flowing fountains of water and assorted replicas of fine artwork on the walls.And although the corruption case against the Holy Land's proprietors and their associates led the nightly news on Wednesday, residents of the opulent complex were mostly mum when it came to the details."I heard it on the news tonight, but we obviously knew nothing about these allegations," a middle-aged resident who was walking outside one of the buildings told The Jerusalem Post on Wednesday."I moved here with my wife about a year ago, and it's been lovely," he added. "It's a quiet area, the view is spectacular, and we always wanted to live in a place like this."While the man, who declined to give his name, said that he didn't expect any fallout from the legal proceedings to affect daily life at the complex, he said the allegations surrounding the Holyland's owners were "shameful, if they are true."Other residents told the Post that many of the buildings' residents were French and American olim and that despite a recent upswing in sales, many of the apartments were still empty.In February, reports surfaced that Holyland Park Ltd., the development company behind the residential project, had not met its sales target for the housing complex, which is set to include some 1000 units when completed.The reports were based on filings by Kardan Israel Ltd., which owns 30 percent of the development through its subsidiary Kardan Real Estate Initiative and Development Ltd.In a park across the street from the complex on Wednesday night, next to the Ramat Sharet neighborhood, two teenage girls, Inbal and Noa, said that residents in the neighborhoods surrounding the Holyland development resented the complex as it had "destroyed the view."
Your comment must be approved by a moderator before being published on JPost.com.
Disqus users can post comments automatically.
Comments must adhere to our Talkback policy. If you believe that a comment has breached the Talkback policy, please press the flag icon to bring it to the attention of our moderation team.
Sign up for our newsletter
Please insert a valid email address
By signing up to The Jpost mailing list, I hereby


text:
- 176Promote
Tote Bag
Catby Volkan Dalyan
Our quality crafted Tote Bags are hand sewn in America using durable, yet lightweight, poly poplin fabric. All seams and stress points are double stitched for durability. Available in 13" x 13", 16" x 16" and 18" x 18" variations, the tote bags are washable, feature original artwork on both sides and a sturdy 1" wide cotton webbing strap for comfortably carrying over your shoulder.
lovely design :)
Thanks Grace
Gorgeous ! : ))
Thanks Keren
love it! :)
Thanks Dorc
Thank Megstuff
Thanks Carol
super cute!
Thanks istillshootfilm
this is super, super cute!
Thanks According to Panda
Extraordinary =^..^=
Thanks Paolo
Yes. His Feline Majesty. Buddha's got nothing on a happy cat basking in the sun, or curled up purring in your couch.
Thanks John
what a proud looking cat
Thanks Alice
Thanks JCMaziu
Great design!
Thanks Angelo
Thank you Fan art
Please like and follow me too! I really like your art style!
Hey I Really like your stuff. Have you tried making them into leggings? Like these: http://society6.com/product/pretty-pi...
ngs#56=417 I think it would look good. Anyway, I liked and followed you!
See More Comments

subset: commoncrawl


text:
Visit our other Wisconsin Historical Society websites!
Refine or narrow your results by clicking any term in the gray boxes below.
Expand your results by removing any breadcrumb in a gray outlined box above. Click on the blue "X" to remove the term.
Narrow your results by entering a keyword into the search box above. Then click the "Search Within" checkbox, then click on the orange "Search" button.
If you did not find what you are looking for or you did not get any results, try changing your search term(s). See our Search Help page for more tips.
Date: 1887
Description: View up Langdon Street of the new Science Hall on the University of Wisconsin-Madison campus.
Date: 1870 ca.
Description: Two men are in the foreground on a shoreline, one standing in a boat. In the background on the other shoreline are adults and children sitting in the gras...
Date: 1892-07-23
Description: Frank Lenz entering Minneapolis escorted by members of the St. Paul Cycle Club, the Minneapolis Business Men's Bicycle Association, and the Minnesota Divis...
Date: 1894-05-08
Description: A man transports nitroglycerin in a boat which has a red danger flag at the bow (unseen against the dark background).
Date: 1869 ca.
Description: Stereograph of the Axtell House building, with a storefront next to it advertising "Clothing Hats Caps Sailors Outfits". A bridge is on the left and a man...
Date: 1886 ca.
Description: Stereograph of a group of men "witching up."
Date: 1875 ca.
Description: Exterior view of the Giles house, residence of Hiram H. Giles, state legislator and social welfare authority.
Date: 1871-07-01
Description: Railroad surveying party in camp somewhere north of Green Bay. The men were surveyors for the Chicago & Northwestern railroad extension.
Date: 1874 after
Description: Stereograph of elevated view lookinf northwest from Exposition Building. There is a church on a street corner on the left, and the Best Brewing Company in...
Date: 1896 ca.
Description: View of the Old Water Works on East Washington Avenue, as seen from Capitol Square.
Date: 1886
Description: Stereograph of Ashley Bennett ("Cookie") who volunteered his services during the float, and the cook Mike Lane, on the Arpin fleet of lumber rafts.
Date: 1885 ca.
Description: Winter scene with a European American man with a beard and moustache sitting in a sleigh pulled by a single horse on snow-covered ground in front of the C....
Date: 1895 ca.
Description: Dr. Edward A. Bass pushing his son Cary in a baby carriage along a wooden sidewalk on Nebraska Street (now West Montello Street).
Date: 1890 ca.
Description: The "J.C. Ames," a Lake Michigan car ferry, probably in the South Chicago harbor. There is a railroad bridge visible in the background.
Description: Stereograph; view of the earlier City Hall building, on Market Square at the corner of E. Water Street and Oneida Street. Several signs are in the image, i...
Description: A large, white brick building is in the middle center, surrounded by trees. A smaller white building is in the right background. In front of the building...
Date: 1892-06 before
Description: Frank G. Lenz posed with his bicycle between railroad tracks in the northeastern United States.
Date: 1871 ca.
Description: Andrew Dahl's wagon is on the side of a two-story log house in Blue Mounds. On one side of the wagon cover is "A.L. Dahl Landscape Photographer." On the ba...
Date: 1895
Description: Exterior view of the round, large frame barn of Chas. Tisch. A man in a carriage and a team of horses wearing fly-nets poses in front. On the right a man a...
Date: 1878 ca.
Description: The Reverend Peter Andreas Rasmussen (1829-1898) is shaking the hand of a woman, possibly his wife Ragnhild Holland Rasmussen. Someone has apparently just...
If you didn't find the material you searched for, our Library Reference Staff can help.
Call our reference desk at 608-264-6535 or email us at:
© 1996-2015 Wisconsin Historical Society, 816 State Street, Madison, WI 53706


text:
Play Video
some Guys burn their hair with hairspray
Posted by Chironex on Jun. 09, 2007
Become a Breaker! ?? http-//bit.ly/BreakSubscribe Kevin's Shia Lebeouf Moti...
Good morning Mr. Phelps. Relive the first four movies in the mega-action fr...
Blondes suck at math, pugs rock at climbing, and sharks rock at EVERYTHING!
Posted by Chironex on Jun. 09, 2007


text:
News Headlines
- Fri, Jul 31
- Lara Croft and God of War headline August's free PlayStation Plus games
- Vote to Play allows PlayStation Plus subscribers to help choose their next free game
- Worms crawls onto Xbox One and PC with Worms WMD, and iOS devices with Worms 4
- Nintendo confirms Splatoon's latest Splatfest for NA: Hot Dogs vs. Marshmallows
- Rumor: Nintendo to start offering N-Boxes to compete with Loot Crate
New Articles
Related Articles
QNAP TS-109 Turbo Station NAS Review - PAGE 1William Henning - Wednesday, January 9th, 2008 Like Share
next: What's In the Box? »
No comments posted yet. Why not be the first to have your say?


text:
— Militarythe Accuracy International L115A3 sniper rifle was the weapon used in the most prodigious feat of marksmanship in military history – three consecutive strikes from 2.47 km in combat. Now a British sniper in Afghanistan has reportedly killed six insurgents with one bullet using the L115A3. Read More
Accuracy International's L115A3 sniper rifle does it again – six kills from one bullet
By Mike Hanlon - April 1, 2014 7 Pictures


text:
Got a minute?
Got a minute?
The Research & Development Consultant will be expected to:. Possess expert knowledge in government funding mechanisms used to support research and development.... - Indeed
Thanks! We'll send you a welcome newsletter as soon as we can.
In the meantime, check out our research center.
Looks like your email already exists in our database.
Please log in here.
You are already logged in.


text:
Cordova Jobs
Looking for jobs in Cordova? Snagajob has great opportunities with the area's leading companies. And applying for Cordova jobs has never been easier. Look at the job listings below, click on a job title that interests you and hit the "apply now" button. Part-time jobs, teen jobs, student jobs and summer jobs are now just a click away. Start your job search now.
Cordova, TN Job Openings
Red Robin Gourmet BurgersCordova, TN
Cordova, TN
CVS Caremark
Cordova, TN
O'CharleysCordova, TN
Cordova, TN
Brookdale Senior LivingCordova, TN
Cordova, TN
Red Robin Gourmet BurgersCordova, TN
Cordova, TN
The Cleaning Authority235 Germantown Bend Cove
Cordova, TN
Cities within 20 miles of Cordova, TN
- Memphis, TN (1095 jobs)
- Southaven, MS (125 jobs)
- Germantown, TN (124 jobs)
- Arlington, TN (102 jobs)
- Olive Branch, MS (98 jobs)
- Collierville, TN (76 jobs)
- Millington, TN (30 jobs)
- Horn Lake, MS (20 jobs)
- Bartlett, TN (18 jobs)
- Atoka, TN (10 jobs)


text:
How to Cope with Other People's Bad Decisions
Hopeful Assurance Ruchti continues: When we stand in a muddle of misery someone else created for us, too weary to be creative, too worn down to follow a seven-step plan to a new, improved life, we need an arm around our shoulder assuring us God hears, God understands and is not stingy with hope. How ragged is the hope you're clutching? It's no less valuable or essential than it was when it was new. Is it hard for you to admit you're struggling with the aftermath of other people's choices? Did you think confessing how awful it is would make it worse? Or make your pain seem pathetic, or cheapen the tenacity you're working so hard to maintain? You'll find hope that – even when it's tattered – glows in the dark. When you walk through the fire, you won't be scorched and flame won't burn you.
10 Prayers For HealingJesus wants you to know that He has both the capacity and the desire to heal...
Is Social Media Sabotaging Real CommunicationSocial media tools like Twitter, Instagram, and Snapchat are becoming the new...
Are My Insecurities Ruining Relationships?Are promising relationships slipping through the cracks because of insecurity...
Success Sundays: Sir Richard Branson, Founders of...Sir Richard Branson, the creator of the Virgin brand - Virgin Records, Virgin...
More from Beliefnet


text:
Wednesday, 29 July 2015
Last updated 12 hours ago
Jun 14 2010 | 11:46am ET
A good year for Och-Ziff Capital Management's hedge fund meant a great year for the firm's London-based employees.
The 58 people working at Och-Ziff Management Europe saw their average pay almost double last year, according to regulatory filings. On average, Och-Ziff paid the Londoners £830,000 apiece, with an addition £90,000 spent per head on pension costs and other benefits. All told, the firm paid out £53.5 million in total compensation in 2009.
In 2008, Och-Ziff Europe's 64 employees took home a total of £34 million in compensation and benefits.
Och-Ziff's hedge fund enjoyed double-digit returns in 2009.
Jul 7 2015 | 4:03pm ET
In addition to easing capital formation for small companies, Regulation A+ has enormous...
Jul 23 2015 | 5:05pm ET
Italian sportscar maker Ferrari has taken a step closer to a stock market listing...
Jul 28 2015 | 3:01pm ET
The lifting of restrictions on foreign ownership limits in Vietnam later this year...
May 27 2015 | 2:15pm ET
Support Hedge Funds Care, also known as Help For Children (HFC), by participating in this year's raffle. All proceeds go to support HFC's mission of preventing and treating child abuse. Read more…


text:
Sunday, 7 August 2011
Paws for thought
'And they're off!' the race commentator announces, as eight assorted terriers tear up the field after a stuffed thing on a piece of string.
Junior wins by default, as Mullet, Holly, Rascal and Rivet get two feet past the starting line and decide to have a fight.
We're at the terrier races, an event that's been going for 37 years and just up from the road where I was born. Now I've completed my OU studies, I have time on my hands so I suggest to Mr Grigg we head for the hills to find out what it's all about. Amazingly, this is the one event not to be featured in any Hugh Fearnley-Whittingstall programme.
And in amongst the plate smashing, the hoopla and tombola, and a commentator who is wearing a red jacket, top hat and a new pair of teeth for the occasion, I see glimpses of my past. People who were at school with me stroll around with cobweb tattoos on their elbows, baseball caps at jaunty angles and gobble up fat hot dogs washed down with a nice pint of Otter.
And then I see a thin old man with a stick, shuffling along in slippers. The walk is hard for him, painful even, but he is smiling. It is my old art teacher, the teacher who inspired me the most and to whom I wrote a letter a few years ago just to tell him that.
He is delighted to see me and delighted with my result.
I then bump into one of my favourite nephews, the one who does intricate tattoos on himself but all upside down.
'I come here every year,' he says, having been born two miles up the road. 'It's local, innit?'
And to complete a hat trick of familiar faces, the one-legged Aga man saunters by and Mr Grigg goes over and gives him a friendly jab in the beer belly. He is already up in the winnings stakes, having raked in 30p on the last race.
As the sky turns black and the rain comes down, we head for a pint in the village pub run by the community for the community.
That's about it.
Love Maddie x
Posted by Maddie Grigg at 11:46 a.m.
1 comment:
Subscribe to: Post Comments (Atom)
Popular Posts
I turn my back for five minutes and find my husband in bed with another female. Arty has climbed up onto the mattress and is having a cud...
As the future of Greece hangs on a souvlaki stick, I wonder how things will pan out. So much has been said about this crisis, nothin...
'I've got a plan,' Mr Grigg says, when he gets back from walking Arty around the block while I work on my laptop to the beat of ...
Some 330 years ago, ancestors of mine were on a battlefield in Somerset, engaged in a hopeless fight. It became known as The Monmouth Rebe...
We've just picked up a vehicle for my big brother from Kostas and Antonis at the appropriately-named Sunrise Car Hire. They'r...


text:
Russian Crisis: The main players: Alexander Rutskoi
Tuesday 05 October 1993
It was in fact only after being appointed Vice-President of Russia that Alexander Rutskoi, a pilot in the Afghan war, was promoted to the rank of general. Russians contrast him with Alexander Lebed, who, unlike Mr Rutskoi, was a general already in Afghanistan. General Lebed is now the controversial commander of the Russian forces in Moldova, and has recently taken to expressing the low morale of the army by warning that it is on the brink of catastrophe. It is he, they say, who is the war hero to be reckoned with - but as yet it is unclear where his loyalties lie.
Last year, Mr Rutskoi said in an interview he was in no hurry to replace Mr Yeltsin. 'If I serve two terms with President Yeltsin, I will be 53 years old. What's that for a politician?' Fourteen months later, he had parliament declare him rival president. After spells as a Communist, reformed Communist and democrat, he began opposing Mr Yeltsin and claimed to represent the so-called 'centre' of Russian politics. When Mr Yeltsin announced plans for direct rule in the spring, Mr Rutskoi defected to the enemy camp, declaring: 'This order will lead to a split in state and society.'
Polling evidence has suggested Mr Rutskoi was at the zenith of his popularity when criticising Mr Yeltsin's programme, but fared badly when taking the initiative. He rapidly lost what credibility he had over his amateurish attempt to run a parallel defence ministry on the 13th floor of the White House, just below the windows that spewed smoke after the building was stormed yesterday.
(Photograph omitted)
- 1 National Orgasm Day: Six reasons (plus one bogus one) why they're good for us
- 2 The 'world's most beautiful vagina' has been debunked by science
- 3 Whoopi Goldberg tells Cara Delevingne to suck it up: 'She's not famous. I'M famous'
- 4 John Green schools morning show hosts after awkward interview with Cara Delevingne
- 5 Doctor Who: Christopher Eccleston says why he left the BBC series after just one series
What turns someone into a conspiracy theorist? Study to look at why some are more 'receptive' to such theories
Stuart Baggs dies: Apprentice star 'The Brand' found dead aged 27
Whoopi Goldberg tells Cara Delevingne to suck it up: 'She's not famous. I'M famous'
John Green schools morning show hosts after awkward interview with Cara Delevingne
Supermodel Gisele Bundchen mocked for wearing a burka to avoid being seen visiting plastic surgeon in Paris
Yvette Cooper: Our choice is years of Tory rule under Jeremy Corbyn – or a return to a Labour government
Labour leadership contender Jeremy Corbyn says 'we can learn a great deal from Karl Marx'
Is Britain really full up? Are migrants taking our jobs? Leading academic answers the most common anti-immigration claims
Public anger after French sunbather beaten up by gang for wearing a bikini in Reims park
Labour leadership: New poll shows party is now even 'less electable' than under Ed Miliband
While we fixate on Calais, the Home Office is quietly deporting dozens of migrants on 'ghost flights'
Independent Dating
iJobs General
Recruitment Genius: Travel Customer Service and Experience Manager
£14000 - £17000 per annum: Recruitment Genius: The fastest growing travel comp...
Recruitment Genius: Cleaner / Caretaker / Storeman
£15500 - £17680 per annum: Recruitment Genius: A position has become available...
Recruitment Genius: Head of Sales - SaaS B2B
£60000 - £120000 per annum: Recruitment Genius: This conference call startup i...
Recruitment Genius: Web Developer
£25000 - £30000 per annum: Recruitment Genius: This digital and print design a...


text:
More Topics
Creator's Hub
Collect Leads
Get Started
Tips & Tricks
For Business
DB2 System Performance, the Basics...
Upcoming SlideShare
Loading in...
Like this document? Why not share!
DB2 Performance Primer
by Tess98
664 views
DB2 System Performance Introduction
by Tess98
554 views
Db2 10 memory management uk db2 u...
by Laura Hood
134 views
DB2DART - DB2Night Show October 2011
by Laura Hood
2935 views
DB2 V 10 HADR Multiple Standby
by Dale McInnis
1623 views
Monitoring and Reporting
by Tess98
886 views
Share SlideShare
Email sent successfully!
Size (px)
Start on
Show related SlideShares at end
WordPress Shortcode
DB2 System Performance, the Basics...
Published on
Jul 19, 2010
0 Comments
Full Name
Comment goes here.
12 hours ago
Are you sure you want to
Your message goes here
Be the first to comment
Be the first to like this
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
No embeds
No notes for slide
Transcript of "DB2 System Performance, the Basics..."
1. Session: A03 DB2 System Performance, the Basics... …and a Bit, to a Lot More Joel Goldstein Responsive Systems May 19, 2008 1:30-2:30PM Platform: DB2 for z/OS Abstract This presentation addresses the approaches, values, thresholds, triggers, and relationships of performance data analysis. Performance monitoring, analysis, and tuning should be a pro-active and continuous process; however, this often gets neglected because people are too busy with other work, or they are not sure what data is really important and which data quickly shows that there are problems - when nobody is complaining. Some basic thresholds and data provide quick insights about problems both at the application level, and the system level. Charts and graphs illustrate the surprising CPU and dollar costs of sequential prefetch and IOs. This presentation does not address SQL coding or application design considerations. 1
Major Bullet Points • Sources of data • Key performance thresholds and indicators • Do you have problems? • Rules of Thumb • Good numbers, bad numbers, or ??? • Can't dot every i or cross every t in this presentation…. • Using and applying data for analysis • Performance examples from the real world 2 ©Responsive Systems 2008 Outline: 1. DB2 Data Sources a. Application Accounting Data b. System Statistics Data c. DB2 performance trace data 2. The important performance variables and indicators: a. Variables and relationships b. Inter-relationships between Statistics and Application data c. How do you know when you have a problem, and just how bad is it? 3. Top down tuning approach a. Tuning from available DB2 data sources b. How much can you tune - the payback potential c. How long will it take? 5. Bottom up approach a. Data interpretation b. Determining where the benefits are… 6. Summary, Guidelines and recommendations 2
We care about performance because.. • Competitiveness in the marketplace • Providing service to your customers • Both internal & external • Reducing costs/avoiding costs • Tune, or buy hardware • Tuning is a free payback for the money already invested • Time is money • The system is slow today... I can fix that… Or maybe you're just a computer geek like me.. 3 ©Responsive Systems 2008 In today's competitive economic environment we can't be complacent about performance. Information is critical to the success of your company, and the ability to both retain existing customers and get new ones, may determine long term growth. The enterprises that can deliver information quickly will grow and thrive - those that cannot, may either cease to exist or will be swallowed by other more aggressive corporations. Ever increasing processor capacities are often driven more by application in- efficiencies and other performance related problems than by true volume demands. Throwing multi-million dollar hardware solutions at performance problems has become the norm. If corporate stockholders became aware of the magnitude of corporate waste and in- efficiencies, many corporate officers might find themselves on the receiving end of legal actions - and many boards of directors would be replaced. Some reasonable amounts of system and application tuning can provide dramatic paybacks at many companies. The multi-million dollar tuning success stories exist - and should receive more press than the large system failures. If management could only become truly aware of the opportunities, and reward staff for improving performance - they would be amazed at the results and long term savings. 3
So – performance is all about numbers.. Raw data, aggregated data, statistics 8426 99 367 73.02 913 4,576 25 8 9,653 881 501 642 4 ©Responsive Systems 2008 Numbers, numbers everywhere. What do they all mean? What is important? 4
Mean and Median • How many of you remember the difference? Mean is an average, and averages are easy – right?… Mean is an average, and averages are easy – misleading... 4, 8, 12, 2, 10, 8, 104, 4, 6, 10 = 168/10 = 16.8 4, 8, 12, 2, 10, 8, 104, 4, 6, 10 = 62/8 = 7.75 5 ©Responsive Systems 2008 5
Mean and Median Median is a mid-point 2, 27, 11, 19, 33, 22, 4, 2, 19, 24, 22, 40, 38 2, 2, 4, 11, 19, 19, 22, 22, 24, 27, 33, 38, 40 odd # values 2, 2, 4, 11, 19, 19, 22, 24, 27, 33, 38, 40 even # values 23 22+24=46/2 6 ©Responsive Systems 2008 6
What does this mean to us? • We need multiple data or focus points • Drill down to lower levels of detail • Exception reporting should be pushed from detail upwards…. • Long averages across periods mask/hide problems • 10, 10, 10, 10, 10, 100, 10, 10, 10, 10, 10,10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10 • Average is 12.4 but it's 24% higher than the median, or the mode 7 ©Responsive Systems 2008 7
DB2 Performance is relative to work • Work performed by the application • Getpages • Fetches • I/Os • Class 1 elapsed time • Ratio - compared to class 2 elapsed time • Class 2 elapsed time (online transactions) • .050 seconds • .100 seconds • .500 seconds 8 ©Responsive Systems 2008 Performance is relative. What are your expectations? Are they reasonable? What constitutes good performance? What is a reasonable amount of work for an online transaction? How do we characterize this? Many well designed online transaction applications have average elapsed times of less than .020 seconds, and provide true sub-second response time to the end user. However, applications like this are becoming exceedingly rare in today's environment of bloated application code, poorly designed databases, and poorly coded SQL. We will try to address what is good, what is reasonable, and basic metrics that indicate when performance is poor or marginal. 8
Keys to better performance • Thought • Data relationships • Inquisitiveness • Knowing what is good, or bad • Reasonable • And why • Expectations • Caring about it • Monitoring • Not waiting for complaints • Test/Dev - Early Warning • Performance History • It can Always be Better • Yesterday? • Last week? 9 ©Responsive Systems 2008 Performance is everyone's job, and responsibility! Simply caring about performance issues, and looking at data will provide opportunities to make things better. Those who wait for complaints, or wait for someone else to take action are not serving their company well - they are having a negative impact on the profitability of the entire corporation. When you find something, and fix it - tell the world!! Tell them how much you saved them, and how many more opportunities exist in your systems and applications. Delaying or avoiding a processor upgrade is worth millions!! 9
Top down tuning • Important/Meaningful periods • Summary data – don't get buried in detail • What looks big, or strange, or not like most other things • Don't assume others are doing their job properly • Use multiple tools to look at performance 10 ©Responsive Systems 2008 It is not usually helpful to look at reports summarized over an entire days, or several days. You should be primarily concerned with the peak workload periods - if you can process well there, the rest of the time will run just fine…. Don't assume that others are doing their jobs, or even that they always know what they are doing - unless you are confident of a persons ability. Even then, they may be overloaded and not have the time do everything you think they are doing. Priorities of work assigned to others changes at the discretion of their management, so unless you work for the same person you can't always be sure what the priorities are. Different tools often use a slightly different perspective to look at the same set of data - so seek input from all tools available to you. 10
Top down tuning • Know what values are good, what is OK, what is bad • Know what is normal for your system • Exception reporting • Don't get too bogged down in detail… • But - be able to drill down to find the lower level data 11 ©Responsive Systems 2008 It is really important to understand what performance numbers are good, and then what is normal for your system and applications. Tracking performance over time is a great way to find problems before they become critical. 11
Performance is like a traffic circle • Application problems may lead to the DB2 system • DB2 system problems often lead to application design and SQL coding • Application and system problems may lead to DASD, z/OS, CICS, IMS • z/OS performance problems may lead to applications or DB2 memory consumption Memory and paging V8 and V9.. 12 ©Responsive Systems 2008 As you find a problem, as correct it, other performance problems will surface that may not have been obvious before. Monitoring and tuning is a constant process, not a one time exercise. The only exception to this is if/when your environment is static - no application changes, no workload or data growth, no migration to newer releases or maintenance levels…. 12
System Overhead • (C2 Elapsed - C3 Wait) / C2 CPU • So what's a reasonable ratio? • Online IMS/CICS transactions < 1.5 • Batch 10.0 + or - • These are affected by: • Processor busy rate, possible system paging • Address space priorities, WLM setup 13 ©Responsive Systems 2008 An area that is often overlooked is the overall system overhead. If the priorities for all the address spaces do not have the correct relationship, varying amounts of overhead will slow your work. Is your processor more than 95% busy on a regular basis? If so, your performance is hurt by the machine busy rate. Running at 100% is not a case of using the machine to its fullest capacity - it's a case of hurting all performance because the processor is overloaded! 13
Important application performance data • Workload • Buffer pool performance • I/O per second (important) • Class 1 - 2 relationship • Type of I/Os • Application hit % (useless) • Class 3 Wait time • System hit % (historical • Components (GP-PagesRd)/GP & not useful) • Avg I/O times • Total % of Class 2 • Ridpool failures • How many • Deadlocks • Why • Timeouts • Online transaction - Seq. scan 14 ©Responsive Systems 2008 You must always consider the amount of work your application is performing - look at the number of getpages, fetches, I/Os, locks, etc, etc. Look for the problem areas. What can be improved? Calculate the System BP IO Rates, calculate the avg. I/O wait time by dividing the total synch wait time by the number of I/Os. Look at any failures of resources, and look for things that don't seem normal to you. Much of this can be programmed into your online monitor exception reports, or other reporting facilities. You really want a reasonable level of exception reporting so you don't have to look at tons of detail reports. 14
How much do you think Getpages cost? MIPS for Sequential Scans 40 35 30 25 20 15 MIPS 10 5 0 5M 10M 20M 40M 80M 150M Sequential GP/Hr Many large systems today are > ¼ Billion GP/Hr 15 ©Responsive Systems 2008 Finding one object dominating your system with sequential I/O? One company fixed a problem like this years ago, and got back 18% of the machine and cancelled their upgrade ! Think you don't have sequentially accessed data in your system? Don't have some indexes with sequential scan? Think again…. Most installations do – and nobody is complaining about performance. But management is complaining about CPU consumption and having to upgrade….. But they won't spend a nickel to tune!!! 15
Cost of Sequential Getpage Activity Dollars per Year $1,200,000 $1,000,000 $800,000 $25 CPU Min $600,000 $35 CPU Min $400,000 $45 CPU Min $200,000 $0 25 50 100 250 500 1,000 GP, Millions per Day 16 ©Responsive Systems 2008 This is based on a 2064 processor with 210 MIP engine speeds. 16
Application Delay issues LOCATION: CASISDBDG OMEGAMON XE FOR DB2 PERFORMANCE EXPERT (V3) PAGE: 1-4 GROUP: DBDG ACCOUNTING REPORT - LONG REQUESTED FROM: 06/14/07 20:05:00.00 MEMBER: DDG2 TO: 06/14/07 20:09:00.00 SUBSYSTEM: DDG2 ORDER: PLANNAME-REQLOC INTERVAL FROM: 06/14/07 20:05:25.65 DB2 VERSION: V8 SCOPE: MEMBER TO: 06/14/07 20:08:59.11 PLANNAME: DISTSERV REQLOC: 10.2.20.23 ELAPSED TIME DISTRIBUTION CLASS 2 TIME DISTRIBUTION ---------------------------------------------------------------- ---------------------------------------------------------- APPL |===========================================> 87% CPU |==========> 21% DB2 |==> 4% NOTACC |======> 12% SUSP AVERAGE |====> 9% APPL(CL.1) DB2 (CL.2) IFI (CL.5) SUSP CLASS 3 SUSPENSIONS |=================================> 67% AVERAGE TIME AV.EVENT HIGHLIGHTS *** ------------ ---------- ---------- ---------- -------------------- ------------ -------- -------------------------- ELAPSED TIME 0.139919 0.018450 N/P LOCK/LATCH(DB2+IRLM) 0.003620 1.28 #OCCURRENCES : 33700 NONNESTED 0.139919 0.018450 N/A SYNCHRON. I/O 0.005955 2.12 #ALLIEDS : 0 STORED PROC 0.000000 0.000000 N/A DATABASE I/O 0.000017 0.02 #ALLIEDS DISTRIB: 0 UDF 0.000000 0.000000 N/A LOG WRITE I/O 0.005938 2.11 #DBATS : 3370 TRIGGER 0.000000 0.000000 N/A OTHER READ I/O 0.000006 0.00 #DBATS DISTRIB. : 0 OTHER WRTE I/O 0.000321 0.18 #NO PROGRAM DATA: 0 CP CPU TIME 0.004498 0.003869 N/P SER.TASK SWTCH 0.000016 0.00 #NORMAL TERMINAT: 3370 AGENT 0.004498 0.003869 N/A UPDATE COMMIT 0.000002 0.00 #ABNORMAL TERMIN: 0 NONNESTED 0.004498 0.003869 N/P OPEN/CLOSE 0.000000 0.00 #CP/X PARALLEL. : 3370 STORED PRC 0.000000 0.000000 N/A SYSLGRNG REC 0.000000 0.00 #IO PARALLELISM : 0 UDF 0.000000 0.000000 N/A EXT/DEL/DEF 0.000014 0.00 #INCREMENT. BIND: 0 TRIGGER 0.000000 0.000000 N/A OTHER SERVICE 0.000000 0.00 #COMMITS : 33639 PAR.TASKS 0.000000 0.000000 N/A ARC.LOG(QUIES) 0.000000 0.00 #ROLLBACKS : 64 ARC.LOG READ 0.000000 0.00 #SVPT REQUESTS : 0 IIPCP CPU 0.000000 N/A N/A DRAIN LOCK 0.000000 0.00 #SVPT RELEASE : 0 CLAIM RELEASE 0.000000 0.00 #SVPT ROLLBACK : 0 IIP CPU TIME 0.000000 0.000000 N/A PAGE LATCH 0.000151 0.06 MAX SQL CASC LVL: 0 NOTIFY MSGS 0.000000 0.00 UPDATE/COMMIT : 1.92 SUSPEND TIME 0.000000 0.012340 N/A GLOBAL CONTENTION 0.000185 0.22 AGENT N/A 0.012340 N/A COMMIT PH1 WRITE I/O 0.000000 0.00 PAR.TASKS N/A 0.000000 N/A ASYNCH CF REQUESTS 0.002085 2.94 STORED PROC 0.000000 N/A N/A TOTAL CLASS 3 0.012340 6.80 UDF 0.000000 N/A N/A 17 ©Responsive Systems 2008 Within this five minute interval, there were 33,639 commits, and 64 ROLLBACKS. Note that 67% of the application elapsed time is WAIT. Log Write IO is 48% of the Class 3 Wait Time ! 17
Application Delay issues ELAPSED TIME DISTRIBUTION CLASS 2 TIME DISTRIBUTION ---------------------------------------------------------------- ---------------------------------------------------------- ------ APPL |=====================================> 75% CPU |=====> 11% DB2 |==> 5% NOTACC |====> 9% SUSP |==========> 20% SUSP |========================================> 80% AVERAGE APPL(CL.1) DB2 (CL.2) IFI (CL.5) CLASS 3 SUSPENSIONS AVERAGE TIME AV.EVENT HIGHLIGHTS ------------ ---------- ---------- ---------- -------------------- ------------ -------- -------------------------- ELAPSED TIME 0.181775 0.045574 N/P LOCK/LATCH(DB2+IRLM) 0.024806 2.51 #OCCURRENCES : 20890 NONNESTED 0.181775 0.045574 N/A SYNCHRON. I/O 0.004653 2.21 #ALLIEDS : 0 STORED PROC 0.000000 0.000000 N/A DATABASE I/O 0.000098 0.04 #ALLIEDS DISTRIB: 0 UDF 0.000000 0.000000 N/A LOG WRITE I/O 0.004555 2.17 #DBATS : 2089 TRIGGER 0.000000 0.000000 N/A OTHER READ I/O 0.000028 0.01 #DBATS DISTRIB. : 0 OTHER WRTE I/O 0.000259 0.09 #NO PROGRAM DATA: 0 CP CPU TIME 0.005798 0.005033 N/P SER.TASK SWTCH 0.000018 0.00 #NORMAL TERMINAT: 2089 AGENT 0.005798 0.005033 N/A UPDATE COMMIT 0.000007 0.00 #ABNORMAL TERMIN: 0 NONNESTED 0.005798 0.005033 N/P OPEN/CLOSE 0.000000 0.00 #CP/X PARALLEL. : 2089 STORED PRC 0.000000 0.000000 N/A SYSLGRNG REC 0.000000 0.00 #IO PARALLELISM : 0 UDF 0.000000 0.000000 N/A EXT/DEL/DEF 0.000011 0.00 #INCREMENT. BIND: 0 TRIGGER 0.000000 0.000000 N/A OTHER SERVICE 0.000000 0.00 #COMMITS : 20839 PAR.TASKS 0.000000 0.000000 N/A ARC.LOG(QUIES) 0.000000 0.00 #ROLLBACKS : 64 ARC.LOG READ 0.000000 0.00 #SVPT REQUESTS : 0 IIPCP CPU 0.000000 N/A N/A DRAIN LOCK 0.000000 0.00 #SVPT RELEASE : 0 CLAIM RELEASE 0.000000 0.00 #SVPT ROLLBACK : 0 IIP CPU TIME 0.000000 0.000000 N/A PAGE LATCH 0.000985 0.17 MAX SQL CASC LVL: 0 NOTIFY MSGS 0.000000 0.00 UPDATE/COMMIT : 2.04 SUSPEND TIME 0.000000 0.036660 N/A GLOBAL CONTENTION 0.001066 0.60 SYNCH I/O AVG. : 0.002103 AGENT N/A 0.036660 N/A COMMIT PH1 WRITE I/O 0.000000 0.00 PAR.TASKS N/A 0.000000 N/A ASYNCH CF REQUESTS 0.004844 4.32 STORED PROC 0.000000 N/A N/A TOTAL CLASS 3 0.036660 9.91 18 ©Responsive Systems 2008 Hopefully obvious the big problem is locking…. 18
Little Red Flags… SUBSYSTEM SERVICE COMPONENT ---------------------------- IDENTIFY 31529 CREATE THREAD 757527 SIGNON 28235 TERMINATE 789499 ABORT 180102 COMMIT PHASE 1 3546312 COMMIT PHASE 2 2910873 READ ONLY COMMITS 643648 UNITS OF REC GONE INDOUBT 0 UNITS OF REC INDOUBT RESOLVED 0 SYNCHS (SINGLE PHASE COMMIT) 1857513 19 ©Responsive Systems 2008 Hopefully, it should be obvious that large numbers of aborts can be a problem. 19
Little Red Flags… From 03/28/08 09:00:00 To 03/28/08 09:30:00 EXCEPTIONS Crit Warn Info Subsystem 11 2 16 EDM POOL BUFFERS Free Pg 20687 % Total 41.4 ADDRESS SPACE CPU Warnings 2440 DBD Lds 0 % Rqsts 0.0 DBAS 10:24.95 Act Pools 0 CT Lds 0 % Rqsts 0.0 SSAS 55.92 %NStl Pgs 7.8 PT Lds 26 % Rqsts 0.0 IRLM 2.70 Getpages 135233941 Dyn Ins 0 % Rqsts 0.0 DIST 55:00.61 Sync Rds 5296912 Read Eff 25.5 LOCKING LOGGING POOL FAILURES Buf Updts 8210578 Suspend 2629 Dlyd Wrts 0 RID 0 Pg Writes 937334 Escalate 0 Arch Read 0 EDM 0 Write I/O 76331 Timeout 0 Min/Ckpt ***** Deadlock 0 Warnings 0 THREADS SQL DATA SHARING STORED PROCS Created 44345 Dynamic 25710 Group DSNDBXX CALLs 352 Terminated 44653 In+Up+Dl 2171956 Member DB66 Fails 0 Aborts 5986 Open+Sel 1153851 Commits 84603 20 ©Responsive Systems 2008 Hopefully, it should be obvious that large numbers of aborts can be a problem. At this time, there's a lot of free space in the EDM pool. This should be tracked across other time frames to see if memory is wasted. 20
DB2 Read I/Os Two basic types of object access and resulting I/Os • Random or Synchronous I/O • Prefetch or Asynchronous I/O • Sequential • Dynamic • List Prefetch 21 ©Responsive Systems 2008 There are many thresholds and exceptions that should be monitored and tracked on a regular basis, and some are more critical than others. SPTH, DMTH, WITH, and EDMPool failures are critical and require immediate action - IWTH is not a problem unless you have hit SPTH and DMTH first. Log Buffer unavailable and Unavailable Read/Write engines often points to poor dasd subsystem performance, and may have to be addressed from both sides to eliminate the problem. 21
Buffer Pool – Tuning Methodology • Mostly Random – RAMOS • Small/Medium Working sets WkSet is subjective – every • Large Working sets system and application is • Very Large & Very Random different • Never have a good Hit % Change the size of a pool, and • May have high I/O rate every object wkset changes almost…. • Indexes Move an object to a different pool, and the WkSet size changes • Mostly Sequential – SAMOS • Small/Medium Working sets You must be able to predict the • May be able to get a decent Hit % effect of changes, or you are Large Working sets guessing with your performance • Never get a decent Hit % life…. • Don't need a large pool 22 ©Responsive Systems 2008 A proven pool tuning methodology is the proper grouping of objects based on access type and working set size (number of resident pages in the pool). Object catalog statistics indicating the number of physical pages are not useful for this approach. An object may have a million physical pages, but the important thing is how many you re-reference within a few minutes – and the impact this object reference pattern has on other objects in the pool and vice versa. 22
Buffer Pool – Tuning Methodology • What is a transient object? • Large wkset size • Wkset grows proportionally to pool size increases • Pages not frequently re-referenced • When do we care? • When it impacts other objects in the pool • If it has a low GP and IO rate, we don't care • High, we care a lot - and immediately.. • Typically very large, and random or sequential • Random may have low or high Dynamic Prefetch… 23 ©Responsive Systems 2008 We don't have any control over dynamic prefetch. This is determined by the buffer manager, at the application cursor level. It may help performance of the object in use, and it may also hurt other objects in the pool. 23
Pool Performance • Is rarely linear • More memory does not always improve performance • Small pool increases often don't show any improvement • Sometimes additional "smallish" increases can provide substantial improvements • Many times large increases do not help • Sometimes they do…. • It may depend on what "large" means to you… • A number, percentage, or it depends…? DB2 Performance – it depends… 24 ©Responsive Systems 2008 We will see from data later in this presentation that pool performance is not linear. Doubling the pool size does not double the hit ratio, or cut the I/O rate in half. We will always reach a point of diminishing returns, when adding buffers to a pool. Now, I realize that some of the above items seem contradictory, and will explain them in more detail during the presentation. The data shown in future slides will also illustrate all the above points. 24
Performance tuning complaint… "I modified a bufferpool to set the DWQT from the default 50% value to DWQT=4 and VDWQT = 0 - No other change - After the change, a program doing the inserts (in ascending key), and delete on it, took a lot of time, 40min instead of 17min (it is a temporary table), STROBE shows that 90% of the time was on the Insert, and 80% of the Wait was on "OTHER WRITE". I proved to them that the INDEX and TS has never been organized , and an increase in volume can magnify the problem. (Statistics showed 70,000 inserts instead of 10,000 during this day) That's 7 times the workload, but only 2.3 times the elapsed They told me that the problem comes from the change, because even if it was disorganized , other executions were good… " Three blind mice…. History data, history data… 25 ©Responsive Systems 2008 Wow, do you think that increasing the workload by a factor of 7 has any impact? Actually, performance was much better, not worse, and not longer. The elapsed time was only 2.3 times the original job elapsed times. 25
Eliminating I/O Saves Money !! Cost Savings per Year I/O based on DB2 I/O Rate reduction Rate/Sec 100 $2,000,000 200 $1,500,000 500 Dollars 750 $1,000,000 1000 $500,000 1500 2000 $0 2500 $25 $30 $35 3000 $ per CPU Minute 26 ©Responsive Systems 2008 These I/O rate per sec savings, up through 2,500 per second, have been achieved by clients 26
Buffer Pool Performance Data - Red Flags BP5 GENERAL QUANTITY /SECOND /THREAD /COMMIT --------------------------- -------- ------- ------- ------- CURRENT ACTIVE BUFFERS 410.10 N/A N/A N/A UNAVAIL.BUFFER-VPOOL FULL 0.00 0.00 0.00 0.00 NUMBER OF DATASET OPENS 8446.00 0.10 0.01 0.00 BUFFERS ALLOCATED - VPOOL 7507.00 N/A N/A N/A DFHSM MIGRATED DATASET 0.00 0.00 0.00 0.00 DFHSM RECALL TIMEOUTS 0.00 0.00 0.00 0.00 VPOOL EXPANS. OR CONTRACT. 0.00 0.00 0.00 0.00 VPOOL OR HPOOL EXP.FAILURE 0.00 0.00 0.00 0.00 CONCUR.PREF.I/O STREAMS-HWM 300.00 N/A N/A N/A PREF.I/O STREAMS REDUCTION 3382.00 0.04 0.00 0.00 PARALLEL QUERY REQUESTS 88003.00 1.02 0.12 0.03 PARALL.QUERY REQ.REDUCTION 215.00 0.00 0.00 0.00 PREF.QUANT.REDUCED TO 1/2 866.2K 10.06 1.21 0.34 PREF.QUANT.REDUCED TO 1/4 73224.00 0.85 0.10 0.03 What critical data items are missing from these sets of data? 27 ©Responsive Systems 2008 There are a lot of red flags in this data report, all related to a lack of buffers available for prefetch. 27
Buffer Pool Performance Data - Red Flags BP5 READ OPERATIONS QUANTITY /SECOND /THREAD /COMMIT --------------------------- -------- ------- ------- ------- BPOOL HIT RATIO (%) 31.59 GETPAGE REQUEST 179.4M 2083.30 250.37 70.40 GETPAGE REQUEST-SEQUENTIAL 130.1M 1510.58 181.54 51.05 GETPAGE REQUEST-RANDOM 49309.8K 572.71 68.83 19.35 SYNCHRONOUS READS 5285.0K 61.38 7.38 2.07 SYNCHRON. READS-SEQUENTIAL 1797.4K 20.88 2.51 0.71 SYNCHRON. READS-RANDOM 3487.7K 40.51 4.87 1.37 GETPAGE PER SYN.READ-RANDOM 14.14 SEQUENTIAL PREFETCH REQUEST 3856.3K 44.79 5.38 1.51 SEQUENTIAL PREFETCH READS 3684.0K 42.79 5.14 1.45 PAGES READ VIA SEQ.PREFETCH 109.6M 1273.45 153.05 43.04 S.PRF.PAGES READ/S.PRF.READ 29.76 LIST PREFETCH REQUESTS 1328.8K 15.43 1.85 0.52 LIST PREFETCH READS 392.2K 4.56 0.55 0.15 PAGES READ VIA LIST PREFTCH 2049.5K 23.80 2.86 0.80 L.PRF.PAGES READ/L.PRF.READ 5.23 DYNAMIC PREFETCH REQUESTED 197.4K 2.29 0.28 0.08 DYNAMIC PREFETCH READS 184.4K 2.14 0.26 0.07 PAGES READ VIA DYN.PREFETCH 5726.7K 66.51 7.99 2.25 D.PRF.PAGES READ/D.PRF.READ 31.06 PREF.DISABLED-NO BUFFER 0.00 0.00 0.00 0.00 PREF.DISABLED-NO READ ENG 74.00 0.00 0.00 0.00 PAGE-INS REQUIRED FOR READ 0.00 0.00 0.00 0.00 28 ©Responsive Systems 2008 Running out of Read Engines is often a sign of poor dasd performance. 28
Buffer Pool Performance Data - Red Flags BP5 WRITE OPERATIONS QUANTITY /SECOND /THREAD /COMMIT --------------------------- -------- ------- ------- ------- BUFFER UPDATES 18460.7K 214.41 25.77 9.39 PAGES WRITTEN 2220.6K 25.79 3.10 1.13 BUFF.UPDATES/PAGES WRITTEN 8.31 SYNCHRONOUS WRITES 25517.00 0.30 0.04 0.01 ASYNCHRONOUS WRITES 1063.6K 12.35 1.48 0.54 PAGES WRITTEN PER WRITE I/O 2.04 HORIZ.DEF.WRITE THRESHOLD 1.00 0.00 0.00 0.00 VERTI.DEF.WRITE THRESHOLD 451.00 0.01 0.00 0.00 DM THRESHOLD 0.00 0.00 0.00 0.00 WRITE ENGINE NOT AVAILABLE 85.00 0.00 0.00 0.00 PAGE-INS REQUIRED FOR WRITE 0.00 0.00 0.00 0.00 Running out of read engines & write engines is often a sign of DASD performance problems. 600 read engines, 300 write engines (apar). What critical data items are missing from these sets of data? 29 ©Responsive Systems 2008 The number of buffers in the pool, and pool thresholds.. And the ELAPSED TIME for the data !! V8 has 600 read and 300 write engines; however, this number is for the entire DB2 system, not one pool. 29
What does a Hit Ratio really tell you? 7% The first 50% does not get much payback 30 ©Responsive Systems 2008 Ok, it shows you that performance is better. But how much better is it? How much CPU and elapsed times have been saved from I/O avoidance? Increasing the pool by 50% does not give much payback, the next 50,000 shows a large improvement, and then the improvement curve flattens. Again, it looks nice, but you can't take any of the numbers to the bank. 30
The I/O rate is a measurable Metric Why does the next 50% help so much? A critical WKSET was reached 31 ©Responsive Systems 2008 The I/O rate is convertible into CPU costs, and elapsed time savings. This is not just a suggestion to make the pool larger, it shows you the real benefit, and where to stop. It shows you that the first 50,000 additional buffers don't provide much payback, but the next 50,000 give a huge payback. The large payback from the second increment of 50,000 buffers is because we passed a critical working set threshold for a heavily accessed object. As stated earlier, the wkset size of an object has nothing to do with the number of pages shown in the catalog. It is the number of pages in the pool at a specific point in time. 31
The I/O rate is a meaningful Metric INTV INTV GET SYNC SEQ LIST DYN IORATE HIT DATE TIME PAGE IO PREFETCH PREFETCH PREFETCH /SEC RATIO ---------- -------- ---------- ---------- ------------ ---------- ------------ ---------- -------- 2007-02-21 00.00.00 1,187,068 43,331 625,280 156,611 202 459 30.47% ++++ 2007-02-21 00.30.00 10,913,350 342,348 3,226,519 289,002 1,069,556 2,737 54.85% 2007-02-21 01.00.00 671,955 51,854 373,032 49,730 893 264 29.24% 2007-02-21 01.30.00 743,700 67,202 435,841 2,948 95 281 31.95% 2007-02-21 09.30.00 4,105,758 124,423 2,961,328 12,364 69,313 1,760 22.85% 2007-02-21 10.00.00 4,232,240 100,715 3,088,771 9,277 64,952 1,813 22.88% 2007-02-21 10.30.00 3,127,959 70,646 2,452,744 18,310 58,303 1,444 16.88% ++++ 2007-02-21 11.00.00 3,890,741 112,328 2,520,929 9,911 51,934 1,497 30.73% 2007-02-21 11.30.00 4,011,848 117,173 2,394,283 11,760 74,255 1,443 35.26% 2007-02-21 12.00.00 4,580,930 109,663 2,339,369 639,288 68,607 1,754 31.09% 2007-02-21 12.30.00 6,150,020 188,803 3,202,333 11,852 64,002 1,926 43.63% If the hit ratio was meaningful, it would not show a big increase when there is a large increase to the IO rate % GP increase vs. IO yes we did find a lot more of the pages in the pool…. 32 ©Responsive Systems 2008 The IO rate can increase, and the hit ratio can increase. The IO rate can decrease, and the hit ratio can decrease. This is the opposite the "expectation" Changes in the workload, the type of accesses taking place, and the objects in use, cause the counter-intuitive swings of the hit ratio. But it's counter- intuitive only if you are looking uniquely at getpages and IOs. 32
Bigger is not always better - 1 33 ©Responsive Systems 2008 The increase for the buffer pool hit ratio flattens, and drops to .4% for every 10,000 buffers, 40 meg of memory. As stated many times, these gains cannot be equated to elapsed times or CPU reductions. 33
Bigger is not always better - 2 34 ©Responsive Systems 2008 The IO reduction/gain from increasing the pool size flattens, and eventually drops to only 1 IO/Sec per 10,000 buffers. This is not considered a useful payback for 40 Meg of memory. 18,000 to 36,000 saves about 15 IO/Sec., and we can see how the rate of saving drops off to almost nothing as we continue to add more memory. Overall, the 36-38,000 buffer range is the best range for performance/memory trade-off. 34
The previous slide with graph showed.. • When a pool is much too small, more memory will provide substantial improvements • There is a point of limited, and possibly no return • Further increases provide very little gain, not enough to justify the added memory • The first tripling of size cut the IO rate 50%, 70 IOs • Then a doubling cut the IO rate by 15 IOs • The third doubling cut the IO rate by 5 IOs 35 ©Responsive Systems 2008 Roughly, increase 6,000 to 18,000 has a large payback – 70 IO/Sec, and is a 54% saving of the IO rate/sec. 35
Look at Hit Ratio again – Looks good… Misleading – implies very good performance 36 ©Responsive Systems 2008 If we show the overall System Hit Ratio, it looks and sounds great. But the IO rate is quite high…. The is a lot of room for tuning to reduce the IO rate, save CPU, reduce transaction elapsed times, and improve productivity. The pools with the highest IO rates are quite a bit lower than that overall System Hit Ratio. 36
Sort Pool Performance LOCATION: DB2PROD GROUP : DSNGDSN MAINVIEW FOR DB2 INTERVAL START: 2020-01-01-00.00.31.0000 SSID : DSN MEMBER: DSN1 PERFORMANCE REPORTER INTERVAL END : 1991-01-01-00.00.01.0000 VERSION : 81 SCOPE : MEMBER STATISTICS LONG REPORT INTERVAL : 00001 PAGE 0006 ----BP7 GENERAL------------ --------QTY ----BP7 READ OPERATIONS---- --------QTY ----BP7 GLOBAL BP---------- --------QTY CURRENT ACTIVE BUFFERS 21775 GETPAGES 17054.28K SYNC RD INV BUFF -WITH DATA 0 VP BUFFER POOL FULL 0 GETPAGE SEQ REQ 11306.15K - NO DATA 0 SUCCESSFUL OPEN 0 GETPAGE RANDOM REQ 5748.13K SYNC RD NOT FOUND-WITH DATA 0 VP BUFFERS ALLOCATED 25000 SYNC READ I/O 2255 - NO DATA 0 HP BUFFERS ALLOCATED 0 SYNC READ IO SEQ REQ 2154 ASYNC READ - DATA RETURNED 0 EXPANDED STORAGE HP BUFFERS 0 SYNC READ IO RANDOM REQ 101 - NO DATA 0 MIGRATED DS ENCOUNTERED 0 GETPAGE/SYNC READ RANDOM 56912 SYNC PAGES WRITTEN -CHANGED 0 RECALL TIMEOUTS 0 SEQ PREFETCH REQ 77737 -CLEAN 0 HP EXP/CONTRACTION 0 SEQ PREFETCH READ IO 57118 ASYNC PAGES WRITTEN-CHANGED 0 VP EXP/CONTRACTION 0 SEQ PREFETCH PAGES READ 449.87K -CLEAN 0 EXPAND SOS FAIL 0 SEQ PREFETCH PAGES/READ 7 CASTOUT -PAGES WRITTEN 0 HWM PREFETCH IO STREAMS 0 LIST PREFETCH REQUESTS 0 -NO ENGINE 0 PRFETCH IO STREAMS REDUCED 0 LIST PREFETCH READ IO 0 -CLASS THRESHOLD 0 REQUESTS FOR PARALLELISM 0 LIST PREFETCH PAGES READ 0 -GBP THRESHOLD 0 PARALLEL REDUCTION-NO BUFF 0 LIST PREFETCH PAGES/READ N/C NO WRITE ENGINE 0 PREFETCH QTY CUT TO 1/2 0 DYNAMIC PREFETCH REQUESTS 0 READ FAIL -STORAGE 0 PREFETCH QTY CUT TO 1/4 0 DYNAMIC PREFETCH READ IO 0 WRITE FAIL -STORAGE 0 DYNAMIC PREFETCH PAGES READ 0 RD STG STATS(5.1)/OTHER(4.1) 0 ----BP7 WRITE OPERATIONS--- --------QTY DYNAMIC PREFETCH PAGES/READ N/C GBP CHECKPOINT 0 PAGE UPDATES 13349.07K PF DISABLED - NO BUFFER ++ 171 GBP REBUILD 0 PAGES WRITTEN 626.00K PREFETCH DISABLED-NO ENGINE 0 UNLOCK CASTOUT 0 BUFF UPDATES/PAGES WRITTEN 21 MVPG PAGES SYNC HP->VP 0 READ CASTOUT CLASS 0 SYNC WRITES 0 MVPG PAGES ASYNC HP->VP 0 READ CASTOUT STATISTICS 0 ASYNC WRITE IO 21696 HP->VP MVPG FAIL 0 DELETE DIR/DATA ENTRIES 0 ASYNC WRITES + SYNC WRITES 21696 DATA MOVER ASYNC HP->VP 0 READ DIRECTORY INFORMATION 0 HORIZONTAL DEF. WR REACHED VERTICAL DEFER WR REACHED ++ 1478 0 DATA MOV ASYNC FAIL HP->VP PAGEINS FOR READ IO 0 0 REGISTER PAGE UNREGISTER PAGE 0 0 DM CRITICAL REACHED 0 BPOOL HIT RATIO-ALL(%) 97 REGISTER PAGE LIST 0 NO WRITE ENGINE 0 BPOOL HIT RATIO-RANDOM(%) 99 REGISTER PAGE LIST-RD CHNGE 0 PAGES SYNC VP->HP 0 REGISTER PAGE LIST-RD CLEAN 0 PAGES ASYNC VP->HP 0 ----BP7 SORT/MERGE--------- --------QTY EXPLICIT CROSS INVALIDATES 0 PAGES WRITE FAIL VP->HP 0 MAX WORKFILE IN MERGE 0 DUPLEX-WRITE REQUEST 0 DATA MOVER ASYNC VP->HP 0 NUMBER MERGE PASSES 31803 -WRITE FAIL 0 DATA MV ASYN FAIL VP->HP 0 MERGE PASSES/INSUFF BUFFER 0 -DELETE NAME LIST 0 PAGEINS FOR WRITE IO 0 WORKFILES REJECTED LOW BUFF 0 -DELETE NAME 0 TOTAL WORKFILES IN MERGE 64253 -READ CASTOUT STATS 0 WKFILES NOT CREATED-NO BUF 0 PREFETCH DISABLED-WK FILES 0 PAGES FOR DESTRUCTIVE READ 4148.62K 37 DEQUE FROM VDWQ DISTRUCT RD 3707.49K ©Responsive Systems 2008 Effect of setting the sort pool thresholds vdwqt and dwqt too high. The pool hit spth, and prefetch was disabled. Current Active Buffers – is a snapshot at the moment the Statistics record is produced, not a high water mark, and it's at 87% of the total pool buffers. While performance looks good based on the Hit Ratios, the pool is having performance problems highlighted in Red. Other important metrics are highlighted in Blue. 37
Sort Pool – BP1 The sort pool was increased from 51,200 buffers to 307,200 buffers Bigger is NOT always better… saved .4 IO/sec - insignificant Wasting a Gigabyte of memory….. vdwqt=90, dwqt=90 pool hitting spth and turning off prefetch 38 ©Responsive Systems 2008 Since the vdqwt and dwqt thresholds were incorrectly set at 90%, the pool was hitting spth, and turning off prefetch. An exception reporting system flagged the threshold problem, and suggested making the pool larger. It continued to hit spth for large sorts. The number of "buffers unavailable, or in-use" was generally 80% of the buffers, no matter how large the pool was made. 38
Sort Pool – Bigger is not always better # Buffers Read IO/Sec Write IO/Sec 51,200 1.37 .18 256,000 1.25 .10 307,200 0.93 .06 The sort pool was oversized from the beginning, and memory would have been used more effectively on other pools. The reason for the expansive growth was…. Looking at one performance variable or exception may miss the real problem, and bad recommendations from an exception reporting product You need to understand performance, and evaluate recommendations… 39 ©Responsive Systems 2008 An analysis tool that only looked at the exception hitting spth, and merely recommended making the pool larger. No consideration was given to the vdwqt and dwqt settings, or the information that the pool consistently showed active (unavailable) pages at 80% of the pool. 39
Sort pool facts • New pages are created in the pool, they are not read into the pool first – no initial read IO delay • Pages that are written out, are not always read back • Pools often have a high % of random getpages • The prefetch read qty is 8 pages This is not an unusual access & usage illustration 40 ©Responsive Systems 2008 There is a lack of published information regarding sort functions and processing methods. 40
Looking at interesting data 41 ©Responsive Systems 2008 BP8 has the killer IO rate. It has one object, a TS BP9 has two objects, both indexes on the TS in BP8. This is a heavy batch system, even though it is mid-morning. 41
Looking at interesting data 42 ©Responsive Systems 2008 BP8 has the killer IO rate. It has one object, a TS BP9 has two objects, both indexes on the TS in BP8. 42
Email to client after looking at data You have a classic pool thrashing scenario, complicated by an application design/access issue. Are you running multiple concurrent batch jobs against this, attempting to get better throughput by hitting different partitions? BP8 As you said, one object, 10 partitions. The access to this object is 100% sequential. The high Synch IO tells me that the prefetched pages are thrown out before a low priority batch job can be dispatched to read them, and they are re-read using synch IO. A major performance, and cpu/cost killer. ** you need to have the batch job(s) run at a higher system priority ** if you are running several, against different partitions, try running a few less concurrent jobs BP9 Two indexes on the TS in BP8 PDPAYMENT is 100% SP access - for an index??? This is a design or sql coding problem, or lack of full stats for the index and TS.. It appears that there were at least 9,351 complete scans of this index. The other index has almost no usage, and all random. 43 ©Responsive Systems 2008 43
Response from the client Joel: Dang you are good!! We had 3 jobs running going against 3 different partitions on the same table. I can't remember what SRVCLASS they were running in. Everybody has the ability to change SRVCLASS, which stinks, but that is my problem. You say "9,351 complete scans of this index". I assume you are getting that number from this screen. How do you interpret this to be "complete scans" and not "pages read" since SYNC I/O's are one page at time? Let's go back two slides and look at the data This isn't genius, just experience. 44 ©Responsive Systems 2008 All the getpages issued were sequential. So what causes a synch IO? When a prefetch stream starts, it issues 1 synch IO for the first page, a SP for pages 2-32, and a SP for pages 33-64. When you hit page 32, it issues a SP for page 65-96, etc, etc I am making an assumption here, since I am not looking at the actual trace data from the collection file, but since this is how sequential prefetch is initiated.... I'm assuming that each synch IO indicates the start of a scan... 44
Back to the client All the getpages issued were sequential So - what causes a synch IO? When a prefetch stream starts, it issues 1 synch IO for the first page, a SP for pages 2-32, and a SP for pages 33-64. One synch IO, and two concurrent prefetch IOs. When you hit page 32, it issues a SP for page 65-96, etc, etc I am making an assumption here, since I am not looking at the actual trace data from the collection file, but since this is how sequential prefetch is initiated.... I'm assuming that each synch IO indicates the start of a scan... I already covered this in the discussion of 3 slides back 45 ©Responsive Systems 2008 All the getpages issued were sequential. So what causes a synch IO? When a prefetch stream starts, it issues 1 synch IO for the first page, a SP for pages 2-32, and a SP for pages 33-64. When you hit page 32, it issues a SP for page 65-96, etc, etc I am making an assumption here, since I am not looking at the actual trace data from the collection file, but since this is how sequential prefetch is initiated.... I'm assuming that each synch IO indicates the start of a scan... 45
What's happening in BP0? Why does it have twice as many Getpages as any other pool? 46 ©Responsive Systems 2008 Very high number of Getpages, almost twice any other pool in the system. Application objects in the pool? 46
50% Sequential Prefetch 47 ©Responsive Systems 2008 This is very unusual access for BP0. 47
What's this? - using Sysdummy 48 ©Responsive Systems 2008 Heavy application usage of SYSDUMMY …. 48
It's Pool Resident…. 49 ©Responsive Systems 2008 This object is "supposedly" one page in size. Updating catalog statistics for it did not get rid of the sequential activity. 49
Pagefixing Buffer Pools • Will save 8% of the IO CPU cost • About a 15-20% reduction of DBM1 CPU cost • Application savings are harder to measure, but not impossible • You need REAL Memory availability before fixing memory. If the system starts to page, you die… 50 ©Responsive Systems 2008 Application saving would have to be measured from the 101 application accounting records, and accumulated over a period. Looking at a few records would not show anything. 50
Which Buffer Pools would you fix? 10 Mins High IO Rate/Second? IO Intensity? Pages r/w / # Buffers BP1= 6.4 BP2=15.9 From the performance & capacity planning side, Intensity has to factor in time… and this is missing. Memory, memory, memory…. 51 ©Responsive Systems 2008 If you have the memory… the HIGH IO pool gives you the greatest saving, and CPU reduction. 51
Which Buffer Pools would you fix? 30 Mins High IO Rate/Second? IO Intensity? Pages r/w / # Buffers BP2= 14.2 BP12=30.8 not the best indicator If you have Memory, memory, memory…. 52 ©Responsive Systems 2008 If you have the memory… the HIGH IO pool gives you the greatest saving, and CPU reduction. If you have the memory, BP2 will give you much better payback than BP12. So, IO Intensity is NOT the best indicator. 52
Memory usage – different system 54M DB2 is not the only user of memory Sometimes we care about the Minimum, A frame of memory is 4096 bytes sometimes the Maximum 53 ©Responsive Systems 2008 If you have the memory… the HIGH IO pool gives you the greatest saving, and CPU reduction. But you don't dare pagefix anything when available memory is this low, and yu already see a count for system paging. 53
Memory usage – pictures make it obvious Be careful… 54 ©Responsive Systems 2008 If you have the memory… the HIGH IO pool gives you the greatest saving, and CPU reduction. But you don't dare pagefix anything when available memory is this low. 54
Tracking & comparing performance…. • One of the largest issues we face, is trying to determine if performance and workloads are similar or We know there has been a change in performance, so what changed? • What and where is the difference? • Application workload? • System changes? • Both? 55 ©Responsive Systems 2008 Some performance differences are obvious, but the reasons may be difficult to find. In most cases, we are looking at performance degradations, and need to find out why the users are complaining about poor response times, or batch jobs are running too long. An automated way of looking at two sets of data, and comparing performance, and highlighting the differences – but at the system level, and at the workload/object usage level would make life much easier. 55
Performance is better – what changed? 56 ©Responsive Systems 2008 We have both configuration and workload changes. Objects not accessed in the second run, a pool not utilized, pool thresholds have been changed. 56
Performance is better – what changed? 57 ©Responsive Systems 2008 This highlights the system configuration changes from the earlier run. 57
DB2 V8 and V9 • Exploitation of 64bit memory, and moving more control blocks out of the DBM1 address space • Access to many gigabytes of memory for pools • Does not change basic pool tuning methodologies • Grouping by Random and Sequential, and then by working set size is the industry proven technique (RAMOS, SAMOS) • Working set size has no relationship to number of pages for an object (catalog statistics) • An object has 1,000,000 pages, but the maximum number in the pool during a specific time period is 2,967 • The wkset is 2,967 • This may change, as the pool size is increased or decreased 58 ©Responsive Systems 2008 The working set size of an object is the number of pages in the pool at a given point in time. There is no relationship between the working set size, and the information you will find in the DB2 catalog. 58
DB2 Version 9 • More areas of memory moved above the 2Gig Bar • Large parts of the EDM Pool • SKCT and SKPT • DBDs • Parts of CT and PT • Parts of Dynamic SQL You need REAL memory behind everything… 59 ©Responsive Systems 2008 A very large memory relief for big systems. Keep in mind that you still need the REAL memory available on the machine!! 59
DB2 Version 9 • Increasing prefetch quantities and using larger page sizes to reduce IO (vpseqt * #Buffers) > 40,000 • Double the prefetch and deferred write number of buffers • IO remains a huge throughput concern for system scalability • Supports buffer pool sizes > 5 Gigabytes • Up to 1 Terabyte of memory for DB2 • Increased usage of 32K sort file to reduce IO • Larger and multiple 32K sort objects • Improved sort logic – row < 101 bytes uses 4K longer uses 32K • It's possible to limit the amount of sort space for a thread (zparm) 60 ©Responsive Systems 2008 IO remains a primary concern for system scalability, and these concerns are being addressed in every new version of DB2. 60
DB2 Version 9 • Automatic Pool Size Management – option autosize=yes • Function integrated with WLM • Can increase/decrease pool size by 25% of initial size • Increments? Not much real information available yet • Based on long term trends • This is not defined anyplace • Long term performance data is a major problem… • Tries to take the memory from other low activity pools first • Based on a random hit ratio – hit ratios are not valid as a performance metric • Seeming fallacy of this approach – lack of IO prediction capability • We already proved bigger is not always better • Will WLM reduce it if the increase does not improve performance? 61 ©Responsive Systems 2008 There are good long term possibilities for this type of approach, and it's obvious that this is just a first cut implementation. It remains to be seen if this provides any real benefit. I suspect it may for small to medium systems at installations where there isn't a lot of DB2 performance knowledge. Large and high performance systems still need real tuning expertise. Also, remember that the effective way to get good performance is through the proper grouping of objects (Ramos/Samos), and not by throwing memory at few large pools. 61
DB2 Version 9 • What changes in regard to pool tuning? • Absolutely nothing • Methodologies remain the same - Ramos/Samos & Wkset sizes • There are opportunities for ever larger pools, if you have the memory…. and are sure you get a real benefit • Remember, paging is to DASD, not expanded memory, so that's 1,000 times slower • Just as previous environments, if you start to page, your pool performance may look better statistically, but the users performance is worse… be really sure memory is available…. • WLM is supposed to manage memory better, and it can, if it's set up properly 62 ©Responsive Systems 2008 The basics of performance tuning have not changed over the last four decades, and certainly won't change over the next decade. CPU, Memory, and IO are the important tuning metrics. 62
DB2 Version 9 – some Important Zparms • SJMXPOOL - Memory pool for Starjoins, must be enabled by the STARJOIN zparm. Up to 1 Gig of memory, above the 2 Gig bar • MXTDCACH – in memory data caching, other than Starjoin, allocated per thread • DSVCI – allows DB2 to create datasets with a CI size that matches the page size, such as 8K, 16K, 32K • MAXTEMPS – the max workfile storage an agent can use 63 ©Responsive Systems 2008 While there are a great many zparm changes in V9, and they are ALL important to your system, these are a few you may want to start with. 63
DB2 Version 9 – overhead comparison Trace filtering capability provides an overhead reduction From DB2 9 for z/OS Performance Topics Redbook 64 ©Responsive Systems 2008 So – Red is bad, Green is good. The cost of the base C1, C2, C3 accounting traces has gone up. The cost of adding package level information has decreased a lot. Classes 7, 8, 10 are the high overhead classes, just as V8, but substantially reduced. The traces can have Include and Exclude lists by many criteria, such as USERID, WRKSTN, APPNAME, PKGLOC, PKGCOL, PKGPROG, CONNID, CORRID, and ROLE. 64
The basics of performance... Have not changed within the last four decades !!! There are no Magic solutions…. solutions… There are no Silver Bullets…. Bullets… There are no 'self tuning' systems…. yet tuning' systems… 65 ©Responsive Systems 2008 Systems and tools are becoming better all the time. While there are some attempts at self tuning systems, or some parts that have a potential for self tuning, we're a long way from realizing these goals. The largest obstacle is the overhead cost of effective approaches. Doing it properly is a lot of work, and very expensive from a CPU perspective. So – it still requires some work to achieve better performance, and the rewards of lower operational costs that better performance provides. 65
Session #### Session Title Session A03 DB2 System Performance, the Basics... Speaker Name and a Bit, to a Lot More Speaker Company Speaker e-mail address Joel Goldstein firstname.lastname@example.com 66 ©Responsive Systems 2008 66


text:
Pawan Kumar Bansal draws Railway reforms blue-print for meeting with PM
New Delhi: Railway Minister Pawan Kumar Bansal has prepared a blueprint of his action plan for rejuvenating the country`s largest public transporter to be presented at Thursday`s meeting of the Union Council of Ministers.
The blueprint consists of some big ticket rail projects and reforms to be taken up in the coming days to boost the finances of the Railways.
The Prime Minister has called a crucial meeting of the entire Council of Ministers tomorrow to set the agenda for his new team for the less than two-year period that remains before the next general elections slated for mid-2014.
Cash-strapped Railways is passing through a critical phase with many of its ongoing projects facing fund curtailment.
Bansal was briefed about the status of the proposed projects like locomotive factories at Madhepura and Marhora in Bihar and other important rail projects, Railway sources said.
There are many challenges ahead given the Railways` financial condition. Bansal has to concretise the action plan for the coming years.
With the change of guard in Rail Bhawan, it is expected that Railways will pursue reforms with great zeal. Tomorrow`s meeting is crucial for Railways as the Prime Minister would set the agenda to sustain the reforms in the national transporter.
Railways is also keen to push high speed rail projects in public-private partnership (PPP) model. There are seven corridors being identified across the country for undertaking a feasibility study.
Besides there are several ongoing rail expansion projects, steps to meet freight target, preventing accidents, improving passenger amenities among others are also being shortlisted in the action plan.
Besides the Prime Minister, Finance Minister P Chidambaram will address the meeting.
More from India
More from World
More from Sports
More from Entertaiment
- 6 leaders were in contact with Dawood Ibrahim till 26/11: Report
- Rameswaram waits to give final salute to Dr Abdul Kalam
- DNA analysis of Yakub Memon's final moments in Nagpur Jail
- SC order on Yakub Memon huge disappointment, setback: Owaisi
- Last rites of Gurdaspur attack martyr Baljit Singh in Kapurthala
- APJ Abdul Kalam's funeral to be held in Rameswaram; PM Modi to attend
- Yakub Memon hanged in Nagpur jail: As it happened
- Has President Pranab Mukherjee rejected Yakub Memon's mercy plea?
- SC refuses stay on Yakub Memon's hanging: As it happened
- 2 NDA, 4 UPA leaders were in contact with Dawood Ibrahim before 26/11: Report


text:
Product Description
Additional Information
Additional Information
|Material Blend
|100% Polyester
|Fabric Weight
|Visit our Tablecloth Sizing page for more information
|Care Instructions
|Visit our Care Instructions page for more information.
- Vibrant ColorReview by BARBARA
I bought these for the buffet table, cake table and DJ table for my daughters wedding in a few months. The red will add a vibrant pop of color to the room!(Posted on 8/12/13)
- AwesomeReview by Catherine
I ordered 2 of these for the head table at my wedding reception and they are great.(Posted on 5/16/12)
- AwesomeReview by Catherine
I ordered 2 of these for the head table at my wedding reception and they are great.(Posted on 5/16/12)
- Finding Tablecloth for Custom-Made TableReview by Judith
I was wondering how to find a tablecloth for our custom-made 4' x 10' dining room table. Cutting off and hemming one side of this tablecloth made for a perfect solution. Much cheaper than trying to find something at a fabric store. PLUS you have the option of pre-made matching napkins here, also!(Posted on 4/20/11)
- Well pkgd, well priced, well made!Review by Roberta
I ordered 3 of these tablecloths, for the headtable and 1 as a spare.(Posted on 12/11/10)
The fabric quality is quite nice, and while the edges are serged and not hemmed, the serging is quite tight and small - about halfway between a regular serged edge, and a rolled hem.
The fabric has a good weight, definitely isn't see-through, and drapes quite nicely. The serging is also the same color as the tablecloth, so it blends in completely.
If you are wondering about what 'shade' of red the tablecloths are, it is more of a slight 'orange-red' than a 'pink-red'. My fiance preferred this as he loathes pink! I also layered black lace over it, which I found also gave it more of a pink cast.
In any case, I'm quite happy with my purchase!
We have many ways to help make dream designs come to fruition.
Whether you want to see different ways of tying a sash for your chair or learn how to find the correct tablecloth size,
we have made many different videos and tools to share our expertise with you.
Tablecloth Sizing Widget
Available exclusively at LinenTablecloth.com, this easy-to-use tool allows you to view all of our different tablecloth sizes
on the most widely used table sizes in the industry.
We also have sizing charts available if you are not using a flash enabled device.
Click here to view our tablecloth sizing widget & charts
Event Layout Planning Tool
Our newest addition! The event layout planning tool allows you to recreate a digital space of your event venue so that
you may choose where you would like all of your banquet tables to be placed - and even label them so you know who will sit where!
To make it even better, you can print out your event layout to bring with you when you start setting up!
Click here to start creating your event's layout now!
Instructional Videos
We have many tutorial videos ranging from how to make elegant napkin folds to how to choose the correct chair cover.
These videos can be found on our site, and on our YouTube channel.
Click here to view our on-site instructional videos
Click here to view our YouTube channel for even more helpful tutorials
Customer Service
As always, if you need any extra help when choosing your table linens, you can contact our knowledgeable customer service representatives.
Customer Service is available Monday through Friday from 8am - 5pm PST (11a - 8pm EST)
Phone: 877-835-5617
Fax: 503-345-9604
Live Chat
Email: email@example.com
You may also be interested in the following product(s)
70 in. Round Polyester Tablecloth Pink
Regular Price: $9.79
Special Price: $6.36
70 in. Square Polyester Tablecloth Pink
Regular Price: $9.89
Special Price: $6.43
70 in. Round Satin Tablecloth Pink
Regular Price: $8.69
Special Price: $5.65
90 in. Round Satin Tablecloth Pink
Regular Price: $11.39
Special Price: $7.40
20 in. Polyester Napkins Pink (1 Dozen)
Regular Price: $8.89
Special Price: $4.45
Silk Rose Petals Pink
Regular Price: $0.99
Special Price: $0.64
118 in. Round Satin Tablecloth Pink
Regular Price: $15.99
Special Price: $8.00
10 in. Dark Pink Hydrangea Silk Plant
Regular Price: $1.99
Special Price: $1.39
10 in. Light Pink Hydrangea Silk Plant
Regular Price: $1.99
Special Price: $1.39
90 x 156 in. Rectangular Polyester Tablecloth Pink
Regular Price: $16.59
Special Price: $10.78
90 x 132 in. Rectangular Polyester Tablecloth Pink
Regular Price: $14.19
Special Price: $9.22
88 x 130 in. Rectangular Rosette Satin Tablecloth Pink
Regular Price: $39.99
Special Price: $25.99
88 x 154 in. Rectangular Rosette Satin Tablecloth Pink
Regular Price: $49.99
Special Price: $32.49
3.5 ft. Pink Fairy Lights
Regular Price: $6.99
Special Price: $4.54
13 in. Blush Pink Charger Plates 4/pack
Regular Price: $10.99
Special Price: $5.49
Pink Heart Tealight Candles 10/pack
Regular Price: $9.99
Special Price: $6.99
Pink Favor Bag with Flower / Ribbon 10/pack
Regular Price: $7.99
Special Price: $5.19
2.7 in. Pink Heart Mint Tin 5/pack
Regular Price: $4.99
Special Price: $2.50
27 in. Pink Silk Carnations
Regular Price: $2.99
Special Price: $1.94
27 in. Light Pink Silk Carnations
Regular Price: $2.99
Special Price: $1.94
36 in. Pink Silk Hydrangea
Regular Price: $9.99
Special Price: $6.49
Pink Silk Rose Garland
Regular Price: $8.99
Special Price: $5.84


text:
If you are gonna talk the talk, then walk the walk
Isaiah 30:21 Whether you turn to the right or to the left, your ears will hear a voice behind you, saying, "This is the way; walk in it."
Friday, July 31, 2015
Just a little reminder....
Of what Jesus did just for you:
Isaiah 53:3-12
3 He is despised and rejected by men,
A Man of sorrows and acquainted with grief.
And we hid, as it were, our faces from Him;
He was despised, and we did not esteem Him.
4 Surely He has borne our griefs
And carried our sorrows;
Yet we esteemed Him stricken,
Smitten by God, and afflicted.
5 But He was wounded for our transgressions,
He was bruised for our iniquities;
The chastisement for our peace was upon Him,
And by His stripes we are healed.
6 All we like sheep have gone astray;
We have turned, every one, to his own way;
And the Lord has laid on Him the iniquity of us all.
7 He was oppressed and He was afflicted,
Yet He opened not His mouth;
He was led as a lamb to the slaughter,
And as a sheep before its shearers is silent,
So He opened not His mouth.
8 He was taken from prison and from judgment,
And who will declare His generation?
For He was cut off from the land of the living;
For the transgressions of My people He was stricken.
9 And they made His grave with the wicked—
But with the rich at His death,
Because He had done no violence,
Nor was any deceit in His mouth.
10 Yet it pleased the Lord to bruise Him;
He has put Him to grief.
When You make His soul an offering for sin,
He shall see His seed, He shall prolong His days,
And the pleasure of the Lord shall prosper in His hand.
11 He shall see the labor of His soul, and be satisfied.
By His knowledge My righteous Servant shall justify many,
For He shall bear their iniquities.
12 Therefore I will divide Him a portion with the great,
And He shall divide the spoil with the strong,
Because He poured out His soul unto death,
And He was numbered with the transgressors,
And He bore the sin of many,
And made intercession for the transgressors.
Oh what a Savior we have in Jesus!!!
Posted by Tom Jehnzen at 7:39 AM No comments:
Thursday, July 30, 2015
Your faith tested. 7-30-15
Posted by Tom Jehnzen at 5:50 AM No comments:
Tell others of our Great God!
May we all take these words to heart and give glory where it is due...to God!
Psalm 96:3-13
3 Declare his glory among the nations,
his marvelous works among all the peoples!
4 For great is the Lord, and greatly to be praised;
he is to be feared above all gods.
5 For all the gods of the peoples are worthless idols,
but the Lord made the heavens.
6 Splendor and majesty are before him;
strength and beauty are in his sanctuary.
7 Ascribe to the Lord, O families of the peoples,
ascribe to the Lord glory and strength!
8 Ascribe to the Lord the glory due his name;
bring an offering, and come into his courts!
9 Worship the Lord in the splendor of holiness;
tremble before him, all the earth!
10 Say among the nations, "The Lord reigns!
Yes, the world is established; it shall never be moved;
he will judge the peoples with equity."
11 Let the heavens be glad, and let the earth rejoice;
let the sea roar, and all that fills it;
12 let the field exult, and everything in it!
Then shall all the trees of the forest sing for joy
13 before the Lord, for he comes,
for he comes to judge the earth.
He will judge the world in righteousness,
and the peoples in his faithfulness.
Keys point to grasp in this Psalm:
1. With your voice brag upon God and tell what He has done for you to others! Don't keep it to self!
2. Fear God and praise Him!
3. Idols are worthless! Worship God who made all including heaven.
4. Strength, beauty, splendor, and majesty all describe God.
5. Give to God glory and give offerings to God.
6. Worship and tremble before God after all He is God!
7. Let all know God reigns and He will judge all one day.
8. Creation worships God.
9. All will be judged righteously be cause God is faithful!
Oh what an awesome God we have...praise Him!!!!!
Posted by Tom Jehnzen at 5:49 AM No comments:
Wednesday, July 29, 2015
What are you feet looking like????
Was reminded of this verse last night:
Romans 10:15
And how are they to preach unless they are sent? As it is written, "How beautiful are the feet of those who preach the good news!"
Short and sweet today...what are you feet looking like? Beautiful because you are sharing the good news of Jesus Christ or covered in dead skin and ugly because you are keeping the news to yourself???
Pray, evaluate, and GO!
God keeps His promises! 7-29-15
Posted by Tom Jehnzen at 5:26 AM No comments:
Tuesday, July 28, 2015
Do good for God's glory not your own!
The ole devil likes to see us brag about doing things for others for pride has then entered us. Be very careful about this because when we brag about doing something for others, we then are patted on the back and that is our reward! On the other hand if we do good for someone and keep it to ourselves, God sees it and will give us rewards in heaven! Look at these verses:
Matthew 6:1-4
6 "Beware of practicing your righteousness before other people in order to be seen by them, for then you will have no reward from your Father who is in heaven.
2 "Thus, when you give to the needy, sound no trumpet before you, as the hypocrites do in the synagogues and in the streets, that they may be praised by others. Truly, I say to you, they have received their reward. 3 But when you give to the needy, do not let your left hand know what your right hand is doing, 4 so that your giving may be in secret. And your Father who sees in secret will reward you.
Reread that slowly and listen to what God is telling us! Here are a few other verses to think upon also:
Colossians 3:23-24
23 Whatever you do, work heartily, as for the Lord and not for men, 24 knowing that from the Lord you will receive the inheritance as your reward. You are serving the Lord Christ.
Luke 6:35
35 But love your enemies, and do good, and lend, expecting nothing in return, and your reward will be great, and you will be sons of the Most High, for he is kind to the ungrateful and the evil.
I wonder how many rewards I have lost in heaven because I just had to tell someone what I did for somebody else!!! Why is it so hard to keep it to ourselves? Pride! Don't let the devils tool steal your rewards!
You are serving Jesus and He knows all you are doing and that is all we need to know!
Posted by Tom Jehnzen at 6:10 AM No comments:
Don't get ahead of God! 7-28-15
Subscribe to: Posts (Atom)


text:
Nick Tepesch scratched from Wednesday start due to blister
May 21, 2013, 6:25 PM EDT
MLB.com's T.R. Sullivan reports that the Rangers have scratched Nick Tepesch from his scheduled Wednesday start against the A's due to a blister on his right middle finger. Ross Wolf will be called up from Triple-A Round Rock to pitch in his place.
Tepesch developed the blister Friday evening on the only slider he threw during his solid five-inning outing against the Tigers. He is hoping that it will be fully healed by his next turn in the Rangers' starting rotation — Sunday, May 26 against the Mariners.
Tepesch, 24, has posted a decent 3.98 ERA, 1.33 WHIP and 31/13 K/BB ratio over 43 innings this season for Texas. Wolf, 30, has a 2.02 ERA, 1.09 WHIP and 26/9 K/BB ratio in 35 2/3 innings this year on the farm.
- beefytrout - May 21, 2013 at 7:46 PM
the Rangers have had SP issues that rival any team out there.
- reeggss - May 21, 2013 at 8:10 PM
Their success is amazing considering the injuries. Well done by RW for keeping thus team competitive and management for keeping the system stocked with talent.
- danagidner - May 22, 2013 at 9:35 AM
Reblogged this on Dana's Quiet Place and commented:
LETS go RANGERS, don't be swept by the A's
Top Posts
- Title or no title, Dave Dombrowski's tenure in Detroit was a success 0
- Dave Dombrowski out as Tigers General Manager, team president; Al Avila takes over 40
- Blue Jays' Aaron Sanchez, John Gibbons disciplined in the wake of Sunday's plunkings 49
- Believe the hype: Carlos Correa is already a superstar 29
- Gregg Zaun to Yordano Ventura: "stop writing checks with your mouth that your skinny ass can't cash" 91
- And That Happened: Monday's scores and highlights 55
- Adrian Beltre needs just five innings for the third cycle of his career 16
- Mets blow out Marlins, move past Nationals for first place in NL East 26
Most Commented
- The benches cleared in Friday's Giants-Rangers game (208)
- Blue Jays acquire David Price from the Tigers (113)
- Rangers land ace left-hander Cole Hamels from Phillies (106)
- Royals make another big move, get Ben Zobrist from A's (95)
- Gregg Zaun to Yordano Ventura: "stop writing checks with your mouth that your skinny ass can't cash" (93)
- C. Gomez (5626)
- Y. Cespedes (4907)
- C. Hamels (4343)
- D. Price (3890)
- B. Revere (3565)
- G. Parra (3452)
- H. Olivera (3369)
- T. Tulowitzki (3295)
- M. Latos (3233)
- J. Soria (3224)


text:
He who puts his lips to it would wish them never to be removed
From "Travels" by Ibn Jubayr
Ibn Jubayr 10 p.m., July 29
Posted October 18, 2011
Singer-songwriter Kevin Martin, frontman for Get Back Loretta, is best known for his Beatle-esque melodies. Indeed, the Vista resident's new solo album is titled Throwback Pop. However, the Occupy San Diego movement has inspired Martin ...


text:
1912 $10 Canada's First Gold Coin "bank Of Canada Hoard" Hand-selected, Rare For Sale
This"REAL TRUE sale" for:
1912 CANADIAN $10Goldcoin
The Canadian $10 Goldcoin had a very brief life span of 3 mintage years which included: 1912 -74,759 coins, 1913-149,232 and 1914- 140,068
Eachpreceeding year being rarer than the next!
We ship immediately! All itemsshownare as issued from the Royal Canadian Mint.
Common Characteristics:
Weight:16.7185 grams
Alloy: 22 karat
Fineness: .900 gold
Actual Gold Weight(AGW): 0.4838 troy ounces
26.92 millimetres diameter
TOTAL WORLDWIDE MINTAGE: 74,759(lowest of allearly $10Canadian gold coins!)
Items shown in picturesarethe actual items for sale. This sale is for(1) $10 gold coinonly.
No taxes are ever charged on any of our salesor on our shipping.
Check outour other items! Be sure to addus to your favorites list!
Please do not hesitate to ask questions about thisor any of our other sales!
We answer all inquiries. Need more scans or pictures, just ask.
Buyer warrents that they have read, understand and agree to every word printed in this listing.
This item has been shown times.
1912 $10 Canada's First Gold Coin "bank Of Canada Hoard" Hand-selected, Rare:
2014 Canada Superman - All 2014 Superman Coins Set - Comic Book Covers -in Stock
1981 Canadian Silver Dollar Bu In Original Govt Package--50% Silver
Canada 1967 Unc Set 100 Yrs Birth Canada All Silver Coins Except Penny And 5c
2012 .9999 1 Oz Fine Silver Robert Bateman Moose Canada Proof Coin
2006 1/2 Oz 9999 Fine Silver Canada Wolf Wildlife Series Rcm Sealed
2012 $20 The Queen's Diamond Jubilee Silver Coin
Canada 2015 $20 For $20 - Bugs Bunny Looney Tunes- 0.9999 Pure Fine Silver Coin
2013 Canada - The Bald Eagle - Original Government Packaging - No Coin
Canada - 2014 'royal Winnipeg Ballet' Colorized $20 Silver Coin 1oz In Music Box


text:
Engaging Infographics
Visually provides on-demand access to the world's top designers and copywriters.Learn More About Visually Infographics
How To Stay In Touch On The Cheap When You Travel
45 views
0 faves
0 comments
Ever wondered how you can save money and still keep in touch with your family and friends when you travel? Well wonder no more.
Source: http://cruis...ou-travel/
Rank: 2159 of 3799 in Travel
Did you work on this visual? Claim credit!
Published by
Designed ByUnknown Did you work on this visual? Claim credit!
Embed Code
Switch to Wordpress Code