Because you're angry about not winning (or, in our case, losing) and thus perceive everything that happens thought that prism and as either contributory to or symptomatic of that fact.
Football crowds - in case you hadn't noticed - are not wise, rational, considered or dispassionate things.