Peppers/Chillis. When to deflower?

I think your peppers are quite advanced if I remember rightly. I would leave them be and take a decision about it if the peppers set. You really only need to remove the fruit if the plant can't handle producing fruit and growing.  If they're in the raised beds in your PT Id leave them be.

It is quite common for peppers and chillies to produce flowers when very small.  I've seen them on baby veg plants at work when they are really pretty tiny.

I do quite a bit of pinching out when my plants are small, to make them compact and more heavily branched, so tend to nip out the early flowers that way.  It works for me and the plants are much more productive for the space they then take up 

As I don't leave early flowers on the plants, I can't say whether it does harm or no harm, but plants do tend to have growing phases, followed by flowering and fruiting phases.  This is a pure guess, but if they start setting fruit when still fairly small, there might be a danger they think 'seeds set - job done' then and stop growing 

Well that makes my pinching out look positively timid 

It might be a bit drastic as we have a relatively short growing season for tender plants like peppers.  If you have a heated greenhouse or somewhere else warm and light you can move them to in the autumn, you could experiment