Given that the Speed Force is often referred to as a sentient thing and how often Barry screws things up via time travel, you'd think by now it would just take away his speed when he attempts time travel of more than a day...
I imagine that by doing all of that traveling he's causing huge ripples. Regardless of Henry's speech about time being fractured every time he travels, Barry has messed with he timeline so much I'm waiting for the real backlash; I mean, even the show dedicated to time travel, Legend's of Tomorrow, doesn't impact the timeline's nearly as much as Barry does.
So I agree, the Speed Force is due to punish Barry for his recklessness.