How can I tell if my new AC is overloading my home electric service?
WHY WE CAN'T ANSWER THIS QUESTION
Welcome to stackexchange JDS. The problem everyone is having with your question is that you have given an excellent physical description of your problem, but almost no electrical information. To truly answer:

How can I tell if my new AC is overloading my home electric service?

We need to get the operating amperage of your AC. Then for us to try and assist in the rest of your dialog, we need to get information using and Ammeter and probably a Multimeter that gives us voltage in 1/1000 increments. Otherwise the best we can do is try to figure out if it is a utility brownout or your panel brownout.

I have attached a Panel Report and Checklist. Look at it and try fill out as much as you can so we can see what you are looking at electrical wise. Remember all of this will do no good if we cannot get at least the circuit number, what it serves, breaker size and amperes which are the first four columns from outside to inside. Also look down at the Buss Inspection area and check off what you can, and add anything that looks funny in comments. 



I know it looks like a lot but fill out what you can and we will see what we can do.

Stay safe you will be working around live equipment.