Using a dimmer on a light plugged in to a receptacle
No, you can't do that. It sounds great until the first time you are short a receptacle and plug in something else - perhaps a vacuum cleaner. And have big problems. Or to give a really bad scenario - your regular receptacle circuits in the kitchen are out but the lighting circuit is working so you move your refrigerator to the "oops, still dimmed" ceiling receptacle on an extension cord until the electrician can get to you...by which time your food has spoiled and your refrigerator ruined.

According to this reference there is a way to do it. But I don't recommend it.

It is usually not that hard to rewire the switch so that it controls a direct (hardwired) lighting fixture instead of a receptacle and change the receptacle to be always on. That is the first step for many people when they buy a house and find that all the light switches control receptacles instead of ceiling fixtures.

It is also very easy (no new wiring involved) to convert a ceiling receptacle to a hardwired light fixture, which would work well for your situation.

But the bigger question is: Why are you switching from fluorescent to incandescent?

You should be switching to LEDs. A LED bulb roughly equivalent to a 40W incandescent (and much brighter than a 25W incandescent) will use around 6W. Plus you'll be replacing incandescent bulbs on a regular basis while the LEDs will last for years. Just make sure you get dimmable LEDs - not all of them are dimmable.

You should also make sure you get enough light. Your 2 48" fluorescent bulbs (if you actually have 2 bulbs in each fixture then the numbers here all get doubled, but I'd actually agree that's more light than you need for a typical kitchen, so I'll stick with the 2 48" bulb scenario) are producing about 2400 lumens each for a total of 4800 lumens. Each 40W incandescent or 6W LED will produce about 450 lumens for a total of 2700 lumens - a little more than half the light. While you may think the current light is too bright, a lot of light is a good thing in a kitchen and the dimmer can give you less light but it can't (obviously) give you more. If you're stuck on screw-in bulbs in a light bar, then something like 12.5W, 75W equivalent, 1100 lumens will do quite nicely. 6 x 1100 = 6600 (more than you have now), total power at full brightness = 6 x 12.5 = 75W (that's less than 2 of your incandescent bulbs) and dimmable so you can lower the light when you want to.

Based on comments, the issue here is more than just making the lights dimmable. There are issues of light quality. Traditional fluorescent "work lights" are relatively poor quality and bother many people due to:


flickering
brightness
color temperature


The flickering can be solved very easily with most other lighting technologies, including incandescent, halogen, LED and even more modern fluorescent fixtures.

The brightness problem can be solved, obviously, with a dimmer. I do recommend a dimmer for this situation as it allows for dim light for those who need it and bright light for more typical users. Working in a relatively dark kitchen is fine when you're pouring a cup of coffee. But if you're working on a Thanksgiving feast with chopping &amp; mixing &amp; cooking &amp; baking &amp; 3 people running around, a bit of extra light is very helpful.

Color temperature, CRI and other qualitative factors of a lighting technology have also been dramatically improved with the latest fluorescent and LED bulbs.

Recommendation:


Convert the plug-in fixture to a hardwired multi-bulb (Edison base) fixture


For the average user, I would pick a reasonable-looking LED fixture (i.e., not screw-in bulbs). In your situation since you have serious concerns about light quality, screw-in bulbs provide total flexibility.


Install a LED-compatible dimmer switch. Even if you don't end up using LED bulbs now (but please try - some of them are really good!), that will allow flexibility for any future user. Check the specs carefully - most LED-compatible dimmers are also compatible with a reasonable load of incandescent lighting as well.
Go to a lighting store or a big box store with a decent lighting department and compare light color/quality of the available options.