How to Create a Robots.txt File

gplea1958
44 Posts
gplea1958 posted this 23 February 2022
Ask a Question

OK. I know how to create the actual file, but I don't understand WHAT I'm creating. I don't understand the real consequences of the commands. Can you PLEASE explain very simply what rules I should apply and why? Next, I'm assuming since GSC advised that I could simply insert the file in the Meta Tags section that I can put it there. Correct? If not, how do I insert it in it's proper location?

OK. I know how to create the actual file, but I don't understand WHAT I'm creating. I don't understand the real consequences of the commands. Can you PLEASE explain very simply what rules I should apply and why? Next, I'm assuming since GSC advised that I could simply insert the file in the Meta Tags section that I can put it there. Correct? If not, how do I insert it in it's proper location?
Vote to pay developers attention to this features or issue.
3 Replies
Order By: Standard | Newest
gplea1958
44 Posts
gplea1958 posted this 23 February 2022

So, here's the contents of my robot.txt file: (If you don't mind, could you quickly peruse my site and advise where I MIGHT need a disallow stmt?

User-agent: *

allow: /

I've just learned that I also need to insert the URL with the link to my sitemap. So, would that look like this?

User-agent: *

allow: /

https://myproaudiogear.com/sitemap.txt

Do I have to use an XML ext? If so, how do I do that within the txt file?
You guys REALLY REALLY should have SOME kind of tutorials on this stuff!! Honestly, HTH do you expect total noobies to just pull this stuff out of thin air?

So, here's the contents of my robot.txt file: (If you don't mind, could you quickly peruse my site and advise where I MIGHT need a disallow stmt? User-agent: * allow: / I've just learned that I also need to insert the URL with the link to my sitemap. So, would that look like this? User-agent: * allow: / https://myproaudiogear.com/sitemap.txt Do I have to use an XML ext? If so, how do I do that within the txt file? You guys REALLY REALLY should have SOME kind of tutorials on this stuff!! Honestly, HTH do you expect total noobies to just pull this stuff out of thin air?
Support Team
Support Team posted this 24 February 2022

Hi George,

We do not create robots.txt file and do not provide any instructions. You can find some tutorials over the internet like this one:
https://neilpatel.com/blog/robots-txt/

...................................................
Sincerely,
Hella
Nicepage Support Team

Please subscribe to our YouTube channel: http://youtube.com/nicepage?sub_confirmation=1
Follow us on Facebook: http://facebook.com/nicepageapp

Hi George, We do not create robots.txt file and do not provide any instructions. You can find some tutorials over the internet like this one: https://neilpatel.com/blog/robots-txt/ ................................................... Sincerely, Hella Nicepage Support Team Please subscribe to our YouTube channel: http://youtube.com/nicepage?sub_confirmation=1 Follow us on Facebook: http://facebook.com/nicepageapp
JRC
53 Posts
JRC posted this 22 February 2024

Nicepage support team.

I think you might be missing the point on this one.
IF someone is using your App (like I am, and the person above), to generate the entire site in HTML....
No one wants to Manually re-create the robots.txt file each time they do an export.
Its too easy to skip and miss the step.

So, what we're looking for, is the ability to add one or more "include files" in the HTML export.
And robots.txt would be a text file that we define what its contents are...
And when we export the site to a folder, the robots.txt file we defined, would be exported there.

Fairly simple request.
And it'd save your users a ton of headaches.

Nicepage support team. I think you might be missing the point on this one. IF someone is using your App (like I am, and the person above), to generate the entire site in HTML.... No one wants to Manually re-create the robots.txt file each time they do an export. Its too easy to skip and miss the step. So, what we're looking for, is the ability to add one or more "include files" in the HTML export. And robots.txt would be a text file that we define what its contents are... And when we export the site to a folder, the robots.txt file we defined, would be exported there. Fairly simple request. And it'd save your users a ton of headaches.
You must log in or register to leave comments