On the chimney is not so bad they will not want to live on the outside....Inside the chimney maybe See what they are up to tomorrow. They would normally find a place to 'roost' for want of a better term, the queen stays with the main swarm whilst others look for a new home, when found the whole swarm will move off! It could be that they will be gone by morning (if night is cool) or maybe in a couple of days if nights are hot!
If you are worried Piskie I have the number of the Witney swarm controller he will be able to give much better advice.