I was thinking your charts looked difficult, then I worked out that your starting point is imperial. Having grown up here in a metric Australia I start with metric sizes and "fit" the imperial sizes between. I have several sets of drills in different measuring systems and material, metric 0.5 rises and 0.1mm rises from 1-10mm, imp fractional and number and letter but I always think of the sizes in metric terms as that is my native language for want of a better term.
I am just trying to come up with a better storage method for my Morse Taper drills above 13mm and 1/2" and am considering large boards and using a scale making larger sizes further from the start of the board and in different rows. Metric full mm first row then metric 0.5mm rises second row then fractional rises then anything else in another row. I would label all the positions and that might also act as a rough size chart for general drilling for tapping or reaming. Anyone have other ideas?