I have houses much closer than that on 2 sides of my small plot and muntjac are quite happy to wander across it to attack crops.
The whole site is surrounded by houses and is not large. We have a very old cemetery nearby that is about the same size and I think that is where they live. People in the neighbouring houses often say they see them in gardens. I occasionally see them in daylight on the allotment site, but they generally charge off the moment they hear the gate being unlocked.