When we had 'The Patch', I found that the bigger the shed there, the more stuff I kept down there instead of at home, like pots, spare tools, yards of wire, plastic bags, acres of netting and cloches etc! Now we only garden at home, all that has either been diminished to necessities only or given away, and seems to work OK.
I'd have thought a 6 x 4 was perfectly OK for an allotment, but of course, if there's no space at home, you may like the idea of extra storage space!
(I knew one lovely old boy down at the plots, who just kept his hand tools under a plastic sheet on the ground all the year round, and was perfectly happy)!