Why do we need air-filled tires?
Is this a conspiracy of bike and wheel manufacturers, to maintain a steady demand for tubes and tires?

No, because if it worked as well as you suspect, someone would have made it happen and taken all the money tube and tire companies are getting.
The reason pneumatic tires continue to be used is because air itself is a great spring.  Additionally, being a fluid, it has the benefit that all of the air in a tire can work as a spring all the time. The rubber in the tire casing also works great as a shock absorber, and just like the air, the entire tire flexes at once, not just the contact patch, no matter which way the tire is oriented. These two effects mutualistically complement each other, and make the tire what it is.
If you had a solid strip of rubber, only the rubber in the immediate area can do the job, so you'll either have a harsher ride, or a heavier tire. The same problem exists with other air-free alternatives as well.
Now sure, you could make a full-suspension bike with a suspension tuned to make a smooth ride despite solid rubber wheels. But you'll quickly find out that such a suspension, both the changes to the frame, and the components, will be considerably heavier than a regular tire. And bikes are an application that is highly weight sensitive.
Other applications, such as Tanks, have a different set of constraints and trade-offs, and so there tracks are preferred, and they have some pretty sophisticated suspension systems allowing them to drive over bumps and stuff at high speeds, because when hauling around 50 tons of armor, another extra ton for a good suspension is worth it.
Sometimes, things stay the same way forever and ever because they just work.