Yes, false accusations are indeed a bad thing. But the point is that people choosing to no longer associate with you is very different to people forcing you to live in a cage under the threat of violence.

Both are unpleasant. One is worse. Thus the burden of proof is different.