Just my observation ...Many yards in Florida have privacy fences in wide yet short backyards because of the setback of the home.
I have seen a few banana mats planted too close to the fence and some about 12-15 feet away.
If I had the extra room I would opt to plant farther from the fence
When a plant is heavy with fruit it will also lean. Which way? Over the fence will give your neighbor kids a banana piņata .
Some mature plants have a large leaf footprint with leaves 12+ feet long....these will overhang your back fence for sure ,will your neighbors mind?
I have seen where a neighbor had taken his weed trimmer and zapped a few leaves that were encroaching over his fence.
JP is not too far off on his idea plant shorter ones closer to the fence and taller ones farther away where they will not become an encroachment issue, and if you stagger them they would make a nice block.
A clean slate is nice to have....let er rip.