Does HubPages have a robot.txt file limiting crawling of our hubs?
I just started using Google webmaster tools to look at my hubs. I have 26 errors regarding robot.txt. I checked here and it said don't worry about robot.txt. But how can robot.txt not be an issue if Google can't crawl my hubs? I'm sooo confused! Thanks in advance for any light you can shed on this.
If you click on the link "restricted by robots.txt" in google webmaster tools you will see a list of specific URLs that are not being crawled. Restrictions for crawling are for links like getting a widget, asking a question, or for rss feeds. Crawling to your hubs is NOT restricted. Robots.txt can be set up for any website to tell a search crawler not to crawl certain pages.
Thank you, ktrapp! I saw all of those errors on my first use of Google webmaster and tried to find the answer on my own. I found more confusion.
Oh, we newbies! We probably provide the best (and worst) entertainment!
by David Hunt 2 months ago
Received the following from Google Adsense (identifying data redacted). Anyone in Hubpages staff know what it means?:Urgent: Potential revenue decrease if no action taken.Dear Publisher,We’ve noticed that the ads.txt file on one or more of your sites that you monetize through this AdSense account...
by Quilligrapher 8 years ago
Google AdSense Site Diagnostics reports that a Robots.txt file is blocking their crawler from accessing one of my hubs. Can anyone suggest how I can resolve this issue?
by Ben Guinter 6 years ago
I'm curious to hear if there are any hubbers, who saw a big drop in traffic after the most recent Google Panda and Penguin updates, have fixed their issues and are getting good traffic again yet?I think it would be good for us all to know what changes brought about your renewal of traffic, since...
by Peg Cole 5 years ago
Is there a way to get rid of unwanted traffic sites that attach a link to our hubs without our knowledge? I try to individually monitor my hubs for suspicious traffic and have written two letters to the offender who has sent me 8 visits on one hub this week. Now HubPages has marked my hub with a...
by Nathan Bernardo 2 months ago
Adsense sent me a message that says that my publisher ID is not in my Hubpages and HowTheyPlay ads.text files. My earnings are fine generally, not too concerned, just wanted to bring the issue to your attention.
by cashmere 8 years ago
In my site diagnostics in Google adsense these hubs of mine are being shown as blocked. Reason blocked is Robots.txt File.Can you tell me how I can unblock my following hubs?My favourite Tarot layouts,How to teach English and Multilevel marketing - the good the bad and ugly.This has happened since...
Copyright © 2018 HubPages Inc. and respective owners. Other product and company names shown may be trademarks of their respective owners. HubPages® is a registered Service Mark of HubPages, Inc. HubPages and Hubbers (authors) may earn revenue on this page based on affiliate relationships and advertisements with partners including Amazon, Google, and others.
|HubPages Device ID||This is used to identify particular browsers or devices when the access the service, and is used for security reasons.|
|Login||This is necessary to sign in to the HubPages Service.|
|HubPages Traffic Pixel||This is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.|
|Remarketing Pixels||We may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.|
|Conversion Tracking Pixels||We may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.|