This was my milling machine for a while before I bought an actual milling machine. While it worked, it suffered from the same problem that all other drill press conversions suffer from - they're too light and consequently rattle and vibrate too much to do accurate work. I did use it some on aluminum with acceptable results but these days it's relegated to drilling.
I started with a Wun Hung Lo drill press, shortened the spindle shaft and then grafted it onto an ER-20 straight shank collet chuck. I replaced the spindle bearings, installed a fine downfeed made from a garage door opener kit, converted a scissors jack into a table support to stop table deflection, added a cheap XY table and started making chips. These days it's been retrofitted with an XY drill press vise and it works well as a precision drilling station.
It was a fun project at the very least and it's actually still useful, though not much to look at.