It's a bit hard to see what you are about here, but one thing that jumps out is whatever your tool rest, it needs to be even or a little higher than your mandrel center. The way you have it now, whatever you try and cut is going to immediately jam between the blade and the rest. That's why bench grinder rests are close to center. You might consider mounting the drill with the handle going down because you could then have it below the table and give yourself more access to the blade. I hope you are only going to be cutting small stuff as it's tough to mount a drill like that steady with the plastic case as they were meant to be hand held. About the time you get it really well mounted it will crack the case. The first clamp needs to be right behind the chuck.
Thin aluminum is not very strong and when you start talking the kind of stress if something jams up, especially with those kinds of cutoff blades, you are into dangerous territory. We used to call those suicide blades because they are hard to control especially on a drill and they want to twist. Don't trust them after you've got them bound up. You can lose digits and eyes in a heartbeat with those kinds of blades.
Don't get in a hurry, make it 5x's stronger than you think you need, and safety first because eye patches and lost fingers are NOT sexy.