How horrible for you, Mrs B. Hope you're feelng a little better now.
Before you do anything else, I would write down everything you can remember and while it is fresh in your mind. It will help you process it before you contact anyone else.
Yes, as others said, you should contact the police given that he's said he has damage to his car. If there's an accident on a road like yours, the drivers are supposed to swap names, addresses, and vehicle registration details. If this doesn't happen, you have to contact the police as soon as possible and in any event within 24 hours. Give them a ring on 101 as soon as you can.
Don't go round to the pub to find out his details - just let the police do that. The landlord shouldn't give out any details of people renting rooms, and as the driver has already shown himself to be aggressive, don't put yourself back into what could be a volatile position.
As others said, you should also contact your insurance company as there's a possible incident. Unfortunately it may well increase your premiums but you have no way of knowing whether he will simply go to his insurers with your details - and it could be worse if they found out from him rather than you.
Try not to worry about it too much. If you don't think you hit him, then don't admit that you "must have done". If the damage to his vehicle is allegedly huge, and you didn't feel any bump, then say so. If you were unaware of any accident at the scene then you shouldn't get into trouble for not stopping at the scene. You aren't in a position to say whether his vehicle was in fact damaged beforehand.