Spot punch marks for drilling should always be light, when confirmed to be spot on then use a larger punch to set the mark to get the hole where you intend to. If a prick punch is off just lean the prick punch and tap sideways until you are over you spot and set the correct position. Drilling off is just from being in a hurry and rushing. If it is worth doing do it right.
Happy boring
Nelson