The reason for deleting install folder is that it contains the PHP code that configured your store. So it might be very easy for anybody who knows it is there to rerun the install process with erroneous information and mess up your store.
Steps to remove Install Folder:
Steps to remove Install Folder: