This happened to my mate though he sort of did it in the wrong order,
He went to a wedding and a lady he used to know many years ago was there saying she always liked him and you should have been with me. They didn't do anything but he went all mental next day dumped his missus strangely picked me up in his car all confused and just drove around for hours.
Then he went to contact the girl he met at the wedding and she wasn't having none of it
You don't find her attractive, you find her annoying, she is new so you are not her friend and your boss has noticed.
You can let it play out but you may not be in control or, you could record your interactions and keep them for the inevitable meeting with your HR department.
Of course, you could just be honest with her..... in a nice way of course. I suspect if you both go for a coffee, she might lighten up
"Scoring a goal is better than sex" - Whoever said that was sticking it to the wrong woman