Can you give me an idea of what this means. It is what I found in my webmaster tools robots.txt file for my domain on hubpages.
If the file is supposed to tell the search engines not to crawl certain pages, wouldn't the information in this file mean that the search engines should not be crawling any of the below given hub sections?
Sitemap: http://roohikhan.hubpages.com/sitemaps/ … han.xml.gz
# Google was crawling our custom analytics URLs, so tell them not to
I don't think so. I wouldn't worry until I find a line that says "Disallow: /hub/*" Why do you ask?
by Gail Meyers3 years ago
This was my site map uploaded into my Webmaster Tools. At roughly the same time I noticed NONE of my hubs were any longer indexed. The site map is generated by HubPages and was found previously found...
by Kristen Howe17 months ago
Yesterday I updated it by adding a new poll. It has two ad capsules and nothing spammy about it. What gives?
by Wendy Iturrizaga2 years ago
Hello everyone, I am a bit early this week with the new HubMob topic, I hope no one minds. First of all I want to thank all the HubMobsters who contributed to the Thanksgiving HubMob. So far we have 20 great...
by Simone Haruko Smith4 years ago
A couple weeks back, a friend of mine got a $500 ticket for a driving violation I had never even heard of. Granted, I’m not a very frequent driver, but I doubt that all of us know every driving rule out there. Are...
by Howard S.4 years ago
Robot.txt disallows the following for every topic:Disallow: /forum/topic/autos/I don't believe that actually stops anything. I believe what is really intended is:Disallow: /topic/autos/We disallow /search/. That's good....
by Wendy Iturrizaga6 years ago
Happy New Year!!!!This is this first Monday of the year, the best time to start new projects and work on New Year's resolutions to try to improve our quality of life. And this month, HubPages is helping Hubbers to...
Copyright © 2017 HubPages Inc. and respective owners.
Other product and company names shown may be trademarks of their respective owners.
HubPages® is a registered Service Mark of HubPages, Inc.
HubPages and Hubbers (authors) may earn revenue on this page based on affiliate relationships and advertisements with partners including Amazon, Google, and others.