when to harvest aubergines

What a relief!  I grew 4 aubergine plants from seed.  But...2 doing nothing and two have one fruit each on, one of those being of the 'udder' lookalike, just like strangerachaels.  Flowers just drop off.  Thought I was useless but it appears I am not too bad.  Am pondering whether to take the 'udder' off to eat and hope it grows another one as a new lovely purple flower on it at the moment.  I don't have a greenhouse so they have been outside for weeks.  Could this be why the flowers go brown and drop off beofre polination?