The sign outside Chimes cafe in Machynlleth, Powys, says: "The more you weigh, the harder you are to kidnap! Stay safe. Eat cake!!"
April, five, went missing on 1 October 2012 near her home, sparking the biggest search in UK history.
The cafe's owner said the sign was not meant to be distasteful or offensive.
She added the sign had been on "display for many weeks" and she had "only received one comment".
"April Jones's family have been in here recently, when the sign was up, and have not said anything," she said.
However, she admitted she could "appreciate how others would interpret the message", but believes those who have complained "have taken it the wrong way".
Machynlleth resident, Angharad Penrhyn Jones, took to Twitter to say: "A sign outside a cafe in #Machynlleth: "The more you weigh, the harder it is to get kidnapped. Eat cake!" In view of #AprilJones, shocking.
"I went to talk to cafe owner and apparently the sign is 'humorous' and the kidnapping happened 'years ago', and I am being 'hypersensitive.'
Author and former parliamentary candidate, Mike Parker, said: "I think the sign is totally inappropriate and my first thought was how distasteful it was considering what had happened previously in the town".
The cafe is situated behind the landmark clock tower in Machynlleth, which played a prominent visual role during the search for April when a large pink ribbon was placed on the railings outside.
Mark Bridger, 47, of Ceinws, Powys, was found guilty of abducting and murdering April in a sexually motivated attack and must spend the rest of his life behind bars.
Her remains have never been found.
A cafe owner has defended a sign urging people to eat cake to avoid being kidnapped in the town where April Jones was abducted and murdered in 2012.