SPIDERS: overview of the X-ray galaxy cluster follow-up and the final spectroscopic data release