Andy Geggan headed the League One side in front when he met Declan McDaid's corner kick shortly before half-time.
Robbie Crawford's dazzling solo run almost brought a stunning second, only for his his shot to be saved.
Premiership Kilmarnock saw plenty of the ball but seldom troubled Ayr goalkeeper Jordan Hart.
Lee McCulloch's side have not tasted victory at Somerset Park since January 1993, a run that now stretches to five games.
The hosts, backed in big numbers in a crowd of 6,417, looked the sharper of the two sides in the first half.
Ayr captain Ross Docherty's deflected shot asked an early question of goalkeeper Jamie MacDonald.
Then MacDonald had to save again from Jamie Adams' point-blank header before Craig Moore smacked the rebound into the side netting.
Kilmarnock had a strong appeal for a penalty kick when Kirk Broadfoot appeared to be pulled down in the box by Adams, but referee John Beaton instead awarded the home side the free-kick.
It may have been the first competitive game of the season but the pace was relentless.
Kilmarnock's Alan Power raced in at the back post to meet a corner kick, but blasted over.
At the other end Adams squeezed a shot between defenders, but MacDonald got down low to palm away.
A goal was coming and after McDaid's free-kick was pushed wide, Geggan raced from deep to head in the resultant corner.
The game became messy in the second half as the tempo dropped, though Paddy Boyle almost sneaked a shot inside the near post from distance.
Ayr were being pushed further into their own half, and Jordan Jones had a close-range shot blocked, but it was the League One side who continued to create the better chances.
Moore got his head to a free-kick but could only direct it straight at MacDonald.
Then a brilliant slaloming charge from Crawford could have settled the match when he skipped past four players, but saw his shot blocked by the legs of the Killie goalkeeper.
Despite those near misses, Ian McCall's men comfortably saw the game out with Kilmarnock struggling to find the energy or accuracy to snatch a late equaliser.
Ayr United manager Ian McCall: "I thought all the best chances in the game fell to us. They performed really well, worked their socks off, and it's one for the fans and directors.
"Michael Rose will probably be out for four weeks now, he took a hell of a clatter on his ankle. Arguably our two most talented players didn't play - Alan Forrest and Craig McGuffie.
"The players worked their socks off and in derbies you have to.
"I know Ross Docherty got man of the match but Jamie Adams was just a colossus. He's probably trained three times and played half a game in between working on his farm."
Kilmarnock manager Lee McCulloch: "We're bitterly disappointed to lose against our biggest rivals. We won't be taking the result lightly that's for sure.
"What cost us the game was our first-half display and switching off at corners. That's not good enough.
"There was too much lumping the ball instead of getting the ball down and playing in little pockets, which we've been encouraging the players to do.
"We didn't deserve to win that game. Maybe a point would have been fair but our final ball was lacking."
Match ends, Ayr United 1, Kilmarnock 0.
Second Half ends, Ayr United 1, Kilmarnock 0.
Attempt saved. Declan McDaid (Ayr United) right footed shot from outside the box is saved in the top left corner.
Foul by Calum Waters (Kilmarnock).
Craig Moore (Ayr United) wins a free kick on the right wing.
Kris Boyd (Kilmarnock) wins a free kick in the attacking half.
Foul by Chris Higgins (Ayr United).
Corner,  Ayr United. Conceded by Stephen O'Donnell.
Substitution, Kilmarnock. William Graham replaces Rory McKenzie.
Rory McKenzie (Kilmarnock) wins a free kick in the attacking half.
Foul by Michael Moffat (Ayr United).
Stephen O'Donnell (Kilmarnock) wins a free kick in the attacking half.
Foul by Craig Moore (Ayr United).
Corner,  Kilmarnock. Conceded by Jordan Hart.
Attempt saved. Jordan Jones (Kilmarnock) right footed shot from a difficult angle on the left is saved in the centre of the goal.
Attempt missed. Dom Thomas (Kilmarnock) right footed shot from outside the box is high and wide to the left.
Attempt saved. Chris Higgins (Ayr United) header from the centre of the box is saved in the centre of the goal.
Corner,  Ayr United. Conceded by Jamie MacDonald.
Attempt saved. Robbie Crawford (Ayr United) right footed shot from the right side of the six yard box is saved in the bottom right corner.
Corner,  Kilmarnock. Conceded by Jamie Adams.
Attempt saved. Jordan Jones (Kilmarnock) right footed shot from a difficult angle on the left is saved in the top centre of the goal.
Corner,  Kilmarnock. Conceded by Jamie Adams.
Attempt blocked. Dom Thomas (Kilmarnock) right footed shot from the centre of the box is blocked.
Attempt blocked. Rory McKenzie (Kilmarnock) right footed shot from outside the box is blocked.
Attempt saved. Craig Moore (Ayr United) header from the centre of the box is saved in the top centre of the goal.
Foul by Rory McKenzie (Kilmarnock).
Declan McDaid (Ayr United) wins a free kick in the attacking half.
Substitution, Kilmarnock. Alex Samizadeh replaces Alan Power.
Corner,  Kilmarnock. Conceded by Andrew Geggan.
Foul by Rory McKenzie (Kilmarnock).
Ross Docherty (Ayr United) wins a free kick in the attacking half.
Adam Frizzell (Kilmarnock) wins a free kick in the defensive half.
Foul by Robbie Crawford (Ayr United).
Foul by Dom Thomas (Kilmarnock).
Brian Gilmour (Ayr United) wins a free kick on the right wing.
Attempt missed. Kirk Broadfoot (Kilmarnock) header from the centre of the box misses to the right.
Dom Thomas (Kilmarnock) wins a free kick in the defensive half.
Foul by Patrick Boyle (Ayr United).
Attempt saved. Patrick Boyle (Ayr United) left footed shot from outside the box is saved in the bottom left corner.
Substitution, Ayr United. Brian Gilmour replaces Michael Rose because of an injury.
Ayr United continued their 24-year unbeaten home record against rivals Kilmarnock with a 1-0 win in their League Cup Group E opener.