Not sure if it would drive me nuts but thats only because I have never run a center mounted helm boat. One thing worth noting if considering a center mounted downeast or other POWER boat is that the highest portion of the boat that you need so see over is in the center unless you are only jogging along. Standing to port or starboard helps with visibility quite a bit. Sailboats run flat so center doesn't influence them much.
I personally think that on a boat of that size (is that a 28 BHM?) it is an inefficient layout of the house to have the wheel in the center. That aside, I would have no issues driving from the cnter of the boat. Beam isn't wide enough to cause any visibility issues docking
I enjoy sitting while driving. I'm not as young as you two.
I've pulled my 10 traps off a swim platform requiring a "crew" who were real good at tangling lines in the props and then after the addition of the ghetto davit having to "quickly move" from an elevated helm around a seat grabbing the hook pole while watching the buoy sail by.
This boat I bought is not big. The wheel, controls and hauler will ALL be within an easy reach.
And most importantly, if I find it doesn't work out, I'll move it around next winter!