I keep my magnet in a plastic bag. Take the bag off, sweep the area, and and all the swarf falls into the bin.
Mike