I like to have clean logs, at the moment there are entries caused by bots searching for robots.txt. I'd like to put a robots.txt file so ...
I have a general error in my log saying I'm missing robots.txt. I just read a 5 year old thread here that phpbb doesn't use robots.txt???
Old Hard to Find TV Series on DVD
Hi guys. I created a robots.txt to stop robots getting to some information such as member profiles... Does this look right to you, ...
I am wondering how to correctly do a few things however. 1) Would this correctly block search engines from accessing a forum area? Disallow: / ...
I currently have an install of phpbb 3.0.5 set up on a subdomain (example.domain.com). Everything is going fine except for the fact that ...
Hello all, I recently upgraded to phpbb3.2 and decided to check the google web tools site, too my suprise when I clicked on the robots.txt checker it showed ...
Robots.txt is often used to tell robots where to look (and more often where not to look) for content. Expanding on what ameeck said, you wont ...
I'd suggest Googling, you will find plenty of information. But basically, it's a file search indexers can download and see what parts of the website they can ...
The newest version of phpBB doesn't contain a robots.txt? How important is this for SEO? Should I create one? How do I create one?
Hello Is there an easy way to prevent search engine indexing? I am aware of the noindex tag, but I have no idea how to implement it into ...