Hang on, don't move the goalposts. We are talking about London. There were black immigrants in London in the 18th century. The 19th and early 20th century saw large scale immigration from Ireland and European Jewry, many of whom didn't speak English as a first language and were certainly not treated as homogenous arrivals.
If we are going to restrict it to skin colour then it is starting to feel a little 'race-based'