In this article we go over the steps needed to create the robots.txt file for your website. This file is used to give instructions about your site to web bots. This gives you the ability to allow or disallow certain aspects of your website to web bots.
Let's get started.
- First login to your Website Portal.
- Navigate to Site > Hosting > Robots File from the side menu.
- Enter the lines of text you require in your robots file. Follow the guidelines of robotstxt.org.
- Click "Save" or "Save and Close" if you are done testing your Robots file. Click grey "Cancel" button if you don't want to change your robots.txt file.
