the signed agreement between 2 plot holders is probably worthless.
1 each plot is rented to individuals or man and wife co owners
2 any agreement between 2 plot holders must be ratified by the allotment commitee.
in this case it would seem the committee knew nothing about it.
i would think that the chap packing in his plot has had the fruits of your labour free of charge for the past 1.5 yrs and i think there is a way out of this for you.keeping both plots until your plants etc have been harvested. you should go to the committee and speak to them stating what you have said happened and ask for the decision to pass on the plot be delayed this year giving you time to get your harvest in .. it would bode well if you payed the rent up to next years rent due date. then say you will surrender the plot for the next person.
can i also point out to all folks who want to rent allotments ,,,,,, read all the small print at least 3 times and ask questions re your plot before you do anything on it and i mean about anything you may want to do or build or even plant in some cases.
allotment commitees are only as powerful as the site owners let them be ... some commitees have allowed things then owners( councils) have come along and charged plot holders hundreds of pounds to put right things that they do not allow... even if they didnt tell you in the rental agreement