Would this offend the Pagan community if I add these groups in my novel?

A story is a story. But some might still get irritated - much the same way some church groups were irritated by the Harry Potter novels (when they first came out).