One of the points argued is that these statues were not put up just after the civil war but either in the 20s or 60s - the latter in opposition to the civil rights movement. Also, that no-one complains when statues of Lenin and co are removed in the former USSR.
I'm not really on any side here, btw. Though by a coincidence I got a book on London statues and monuments for my birthday this week. I might go and pull a few down if I disagreed with something they did.
Oh, and I also read some remarkably racist quotes from Abraham Lincoln which somewhat deflates the virtue balloon of the Unionist side.