Fly press upgrades - video

[Toolmaker51]

Almost like this quick change die holder; but that type of retention is greatly improved by side opposite register (square edge) tapered somewhat, 2 or 3 degrees. This narrows front edge. That multiples grip with a temporary interference, and insures it is set back uniformly against stop button. Combined with the cam having a spring to push or pull it into contact, the die cannot move or be seated other than one position.
Another detail, countersunk screws aren't dowels or repeatable locators. Swapping them around or any deviation in torque is almost certain to move an assembly from original. The angled face is never concentric with screw body, and without great care a countersunk hole not concentric with clearance hole in clamp plate.