Byrony - English Mandrake - sounds very unpleasant but it's common over most of England, moreso to the southern areas. Don't get the juice and such on your skin as it's an irritant, the berries are poisonous.
Not sure how you'd rid yourself of this plant, but I'd try lopping off all the top growth and directly pouring on (very carefully!) some straight Roundup onto the stumps. Be very careful not to let it drip off and touch your shrubs.