Free speech is critical because anything short of that means that there is someone who has the power to restrict the speech of others, and whomever that 'someone' or committee is will ultimately bully people they don't agree with. Free speech isn't to protect what most of us agree with, it's a check on power so that anyone can speak up, even if a large majority happen to disagree or find the statement distasteful.