Turning the batteries upside down is a mistake. You are reversing the polarity. If the circuit is made you could be burning out a diode. Which is one way. To do what you want to do put something over the battery ends to isolate from the contacts. Something thin like a guitar pick.