With cutting end of Forstner bit facing out? Ouch.
Understand short cylindrical section not suited to the other way. How about somewhat deeper counter bore and the backside short enough the chucking end sticks out? You'd eject chosen bit with a finger tip.
But I like idea of a shadow box/seiton index near area of use covering fractional bits. I store tap drill sizes with the corresponding cutter.

Haven't solved counterbore pilots or back spot-facers yet.