But the UK government faces tricky decisions when any of these men try to return to the UK.
How will those who went to Syria to help the humanitarian effort be separated from those who are simply thugs?
What about the threat from within? The callous, brutal murder of Lee Rigby was a lesson to us all.
But one thing is certain - dismissing all members of the Islamic faith as would be terrorists is foolish. We face other threats which could be missed if we follow that line.
And in the end all such hate will do is breed more hate and violence.
Blinded imam Hull, police launch fresh appeal
Tell MAMA Reporting 2013/14: Anti-Muslim Overview, Analysis and ‘Cumulative Extremism’