Remember that Juande Ramos went to Spurs from Sevilla, ditching Monchi and his long nurtured youth project on the way, so he probably has a wry view on both Ramos and Spurs from that experience.
Context, old boy, is everything. Sometimes you have to look quite hard to find the specific reason people hate Spurs.