voltage stabilizing Transformer 120 to 24 volt.

I doubt it is the transformer, more likely the 120v input is sagging and the 24v output is just following along. Monitor the 120v from an outlet and see if it sags. Do the lights dim?

If your HVAC causes that big of a sag then it seems to me you have bigger problems. I am not an electrician, but if there is a problem with the service to your home, the line into your home is inadequate, the HVAC is defective, or you have some poor connections somewhere in the line it could cause this problem.

The majority of the problem is that when there is high demand, the power company reduces power to the area to 105-110v, so that the sag is starting from a low voltage to begin with.

It seems to me that I should be able to convert 100v to 24v without difficulty. Isn't there some sort of capacitor I can put in line to prevent the momentary sag?

Humphry you can use a special transformer called a SOLA voltage regulating transformer. Internal buck/boost winding regulates the output voltage to <1% variation over a large input voltage range. Available from a local electrical supply house or industrial electronics vendors such as Allied Electronics, Newark, etc.
Size the unit according to the current of your load; I suspect the smalllest model will work. Order the model for 120Vac in to 24Vac out of course.