It could be a simple relay that is making a bad connection. When you cycle it off, it will clean itself and make a new connection, at turn on.
Another possibility is, somewhere in the circuit, there can be a bad connection. The inrush current (at turn on) can cause a little arc, and temporarily weld/solder the contact back together. This also can happen inside a semiconductor itself.
Another possibility is, somewhere in the circuit, there can be a bad connection. The inrush current (at turn on) can cause a little arc, and temporarily weld/solder the contact back together. This also can happen inside a semiconductor itself.