Nice work Steve.
I too have very basic bench drill (RECORD) which I inherited, I have made a few mods for milling non ferrous metals with an old XY table. The main limitations were the Jacobs taper chuck mounting and the flimsy tubular column. I replaced the column with a solid one and after removing the chuck drilled and reamed the end to take ¼ end mills. The next problem was being able to lower the head down to the table for rigidity, easily fixed by cutting away part of the belt guard and fitting a stop collar to the column. Then a fine feed was made for the quill together with a quill lock.
I mainly use it for coordinate drilling but a decent quill as your design would improve matters and thanks for the drawings.
Replacing the column is definitely a future plan for me as well, your solution for lowering the head is a good one!