I think you answered your own question - the use of your second hand. I've noticed the same thing plenty of times. I think it just acts as a sort of shock absorber so the board doesn't sense the impact.
Another thing is that sometimes if I hit one area of the blade, it triggers it, while other areas don't. It's just one of the quirks of the board.
I guess I might have answered my own question, yeah. Haha.
I just wanted to see what other people have experienced.
If it's one of the quirks of the design then I suppose it can't be helped.
Still, it would be nice if it would trigger the clash every time I made solid contact.