Carbide, while near diamond in hardness, a very resistant dense body, means the cutting edge is brittle. That can be alleviated but not eliminated in setup. It hinges on whether they're ground positive, negative or neutral, in respect to it running 'on center' like we'd describe a lathe. Carbide is intended to push material vice cutting it as HSS. Good thing about import bars usually don't have a clamping flat...instead of placing on plane to center [let's call that 3 o'clock] rotate it back to 3'15'' or so, figuratively speaking. That puts more material at underside of cutting edge.
I don't fault you, the mill, spindle, or .020 per side, but carbide needs to work not scrape. Chip ejection carries away heat, so a consistent feed-rate is important.
Regard this as an open letter to all readers. We really don't 'know' each other personally, or can watch goings on, everything depends on symptoms reported.
There isn't any suggestion here I haven't seen occur one time or another.
If a HSS drill bit scratches the surface, it will certainly bore with HSS. I looked to see what pattern a Clausing 8520 is, they are substantial.
Rigidity isn't spindle alone. Are the column clamps snug? Is the ram retracted or extended, snug either way? Part clamped to table? Is the hold down bolt properly placed at nose of clamp? Wish I had a dime for every hold-down I've seen closer to the step blocks. Wrong! In a milling vise, or drilling vise? 'Drill vises' have near zero lateral strength and practically no hold-down ability. Bad vises aren't even good C-clamps.
As far as the Morse Taper not releasing, that's all guess work.
I don't recall an OK condition Morse taper giving me any trouble, let alone brass punch and engineers hammer.
If the spindle has a slot for drift wedge, it's probably slotted for the Morse tang.
If they are retained by a drawbolt very little torque is needed to retain the taper. Morse tapers are self holding. One should be able to sock in a clean #2, 3/4" bit and drill into solid material, and remove after with a drift and simple but solid tap.. Is taper clean AND bore dry? NO oil other that from handling..
My Bridgeport Cherry rig is MT for endmills, it doesn't drill at all, but never spun an endmill or had drawbar much more than fingertight with a 3/4" boxend wrench.
One more, on drawbolt; clean the threads, there should be a thick washer on the spindle under head of drawbolt with a thin film of oil. Thickness of washer unknown...but could be established by assembling outside the spindle.
Look for interference at end of threads to collet, protrusion into cavity for tool, or short thread engagement.
There are more but these are a good start.