What should I do with my private key?

Your private key should always remain private. The only person that should see your private key is your hosting company, if they ask for it. However, do not delete your private key, as it is required for your certificate to work.