I've got a lot of sympathy for the tree lopper. It might be classic and beautiful, but from my interpretation of the photo it's significantly taller than the house, and it's closer to the house than it is tall. It will be sucking moisture from the garden and it will have been blocking the light. Also it's not just what's happening now, but what wil happen in 10 , 20 years time. Personally I'd not have bought the house, which is not a particularly helpful comment to make, and had the tree been on my land (assuming that my assessment of the photo is correct) then it would have been felled and grubbed out.
Assuming that there was no TPO, and that I'd discussed the situation with my neighbour, and that my intended actions were lawful, I would also have considered the option of cutting it back to the boundary of my property. It will grow and in a few years it, and although it might look a bit quirky it won't stand out too much. I'd also have considered root pruning it along the boundary line too.
Mature trees are one of the best assets that an established garden can have, but a lot of thought needs to go into where they are sited, and the species selected. 6 years ago I removed 27 laurel trees and two mature hornbeams from my small garden (approx 300sqm); and with permission removed another hornbeam from the boundary and pollarded a sycamore (which I'd have felled given the chance/authority). The two birches remain, but were pollarded and now look very well proportioned. It was a real shame to see the hornbeams go but the benefits far outweigh the 'vandalism' and there are no worries about tree roots damaging property foundations.
SS
Edited to remove numerous typo's!