Did you cross-pollinate that flower? My Physical Graffiti flower pollinated with its own pollen (by hand) is quite a bit smaller than the two other fruits on the same plant which I cross-pollinated. I don't know if that's because it's on a thinner/weaker branch or because of metaxenia as discussed in
http://aob.oxfordjournals.org/content/93/4/469.full.pdf. Others have claimed cross-pollination results in larger fruit so I tend to think that's probably shown in my PG fruits as well.