Can I plug in too many LEDs (i.e. 26) on Arduino UNO?
You are mistaken, the pins do not have a rated maximum of 40mA. That is the rating of permanent damage happening, and very much outside of normal operating conditions.
The rating for normal use conditions for a single pin is 20mA.
Also do note that the pin voltage is not any more 0V or 5V  when 20mA flows, it is guaranteed to be max 0.8V sinking 20mA and min 4.1V sourcing 20mA.
But there are also other limits.
There are three pin groups which each are limited to sink 100mA, and there are two pin groups which each are limited to source 150mA.
The MCU total limit is listed in the absolute maximum ratings, again which does not mean normal usage, is 200mA.
So this basically means, no, you cannot plug a 20mA load on each GPIO pin and expect it to work.
Due to the limits, that's less than 10 loads of 20mA for the whole MCU, but due to the group limits, it must be below 11mA per pin.
What you need is the MCU to control some sort of LED driver that can handle the load.