You have to be so careful in the public eye, look at history, with The Greek, Howard Cosell, Al Campanis, all were never the same publicly after what they thought were just 'oh by the way' statements...
We all know Joe Paterno didn't mean anything by it, some group will call him on it, he should have known better though, agreed.