Criminal violence. Your free speech rights exist vis-a-vis the government. The First Amendment to the U.S. Constitution reads "Congress shall make no law. . . abridging the freedom of speech."
Congress, which the courts have interpreted to mean the whole government. Radical Muslims are not the government in France.
There are many ways to limit your freedom of speech, some legitimate, some not. If you are an employee, your employer can tell you what you may or may not say, and fire you if you disobey. Perfectly legal and legitimate. You have no free speech rights vis-a-vis your employer. If you own a shop, your customers may limit your freedom of speech by declining to do business with you if you say certain things, such as "Hitler was right." Again, perfectly legal and legitimate. If you go to a party at someone's house and espouse racism, the owner of the house can limit your freedom of speech by telling you that if you continue, you'll have to leave. Perfectly legal and legitimate.
The key here is, again, that your "free speech" right exists vis-a-vis the government, not private parties.
Then there are illegitimate ways to limit your freedom of speech. One of these is private individuals threatening or committing violence against you. But if they do that and are caught, they will not be punished for limiting your freedom of speech. They will be punished for the crime they committed. If you beat the heck out of somebody because of what he said, the law will punish you just as it would if you beat the heck out of him for looking at your girlfriend, or supporting the wrong sports team, or because you want his wallet. That demonstrates that the Paris thing was not about freedom of speech, it was about mass murder.