Patio Nectarine

I was given a patio nectarine last year.  Kept it in the (unheated) greenhouse to protect from peach leaf curl and frost/wind damage, but left the door open in the day to allow for pollination.  Also tried pollinating using a paintbrush.  The flowers just withered, died and no fruit grew.  The plant itself is really healthy and is once again covered with buds that are just about to open and I would really like some nectarines this year!  Any ideas? 

Can't remember the variety off-hand, but it was self-fertile - supposed to be great for tiny gardens or balconies with only space for one. 

I have a patio peach (which is also self fertile) on a balcony in London, so pollination is a real problem. I have found the most reliable was to get a decent yield of fruit is to pollinate with a paintbrush every other day while it's blossoming.