What actually determines whether I see H+ or 4G in the top panel? So, I read the answers to What are the different signal indicators avalable on the notification bar? and as one answer says:


  
  3G: UMTS (384 kbit/s to 7.2 Mbit/s)
  H: HSPA (enhanced 3G, sometimes shown as 3.5G; up to 21 Mbit/s)
  H+: HSPA+ (even more enhanced/faster HSPA; up to 42.2 Mbit/s)
  4G: LTE (thanks to eldarerathis for confirmation); up to 300 Mbit/s
  


I'm guessing that these speed definitions are determined by some international convention or something like that.

I have different SIM cards from different carriers.

When I use Carrier A I see H+, H or 3G in the panel. I get speeds ranging from 4 to 12 megabits per second.

When i use Carrier B I always see 4G in the panel. However, I get speeds ranging from 1 to 2 megabits per second.

By the way, these are general trends for these carriers in various areas, not isolated instances of speed.

So, it seems that what symbol I see in the panel needs not matter so much for the actual speed? (Except for E, which is always terrible regardless.) 

If not speed, what is the real reason for what symbol I see? Is it arbitrarily set by the carrier? Can they simply choose to call their network whatever they like? Or does it have to do with what sort of technology they have in their tower? Maybe Carrier B actually does have 4G towers, only their networks are overloaded and therefore cannot supply proper speeds to all the users?

I am in India where 4G is still new. AFAIK, Carrier B might have more traffic than their network can handle.
It is the speed that determines what is shown in the indicator
For the conventions regarding the symbols and speed , see this - which are in line with what is mentioned in your question
Advertised throughput speeds are not the actual, real world throughput speeds experienced by the end user. See this for factors affecting the speed


Coming to your main concern regarding different speeds and variance with symbols displayed:

Situation described is India specific and regulated by Telecom Regulatory Authority of India. AFAIK, they are aware of operators delivering lesser speed than advertised and are in the process of streamlining it. Details are being skipped here being broad / opinionated


Operators are not at liberty to falsify speeds - some typical practices have been covered in comments
It is best to reach out to the carriers and or TRAI to get to the operator version of it