I am not sure that the level of chlorine in tap water is in anyway harmful but I have noticed that, along with many dogs I know, that the hens prefer to drink any old natural rainwater they may find in puddles etc in the garden, the more rank looking the better!
I occasionally give mine water collected in a water butt but some people advise against this but Ihaven't seen any ill effects so far.