The AAA batteries can be a pain sometimes. It might just be that one's popping out slightly each time you move the saber. The boot-up sound each time makes me think so, at least.
Try some electrical tape wrapped tightly around the entire 4 batteries to keep them snug. That might just be it.
If not, I'd certainly follow Therion's advice.