hi Just a quick question, just checking my Adsense, as you do, and see that on three of my hubs the google crawler is blocked because of robot.txt. What the hell is that and how do I fix it?
I don't actually know but I would speculate that google thinks your hubs have been copy/pasted from somewhere else or that you didn't write them by hand...you could try editing them and re-typing maybe? Sorry I'm just guessing here...
I just googled it and this page might help;
to fix this, go to the control panel(cpanel or whatever your hosting site gave you) of your hosted blog, and browse to /public_html/robots.txt using file manager.
Open the robots.txt and then enter these lines here in quote or copy it from the google adsense error page that you got :
Once you done adding these you can save the robots.txt and you'll find adsense bot crawling without any problem.
the robot.txt was added on your side. google does not have the ability to add this to your hubs.
Here's my response to the same question a few days ago...http://hubpages.com/forum/topic/33803#post750921
check the settings for the affected hubs. make sure you have the box checked for the question about " content meets the conditions of the google adsense policy"
not necessarily...google indexes hubs on their owns servers for retrieval and if somebody hit translate then a new"translated" copy with the .txt string is added so that every time a translate request is received, it will always translate from the original and not in series from translation 2 translation so that the meaning is not obscured over time.
robots.txt is used that way to keep the "translated" .doc from being indexed and other requests from grabbing it and so uses the original indexed one every time. otherwise it would look like gleep , glop and ascii code over time probably and look like wingdings hieroglyphic font eventually LOL !
the next time someone orders up your hub from googles search engine in original english, than the message will be gone.Your original copy is still in tact and indexed at googles server waiting for another query request.
in other words. it got translated. and will set itself straight next time someone clicks her on the serps. Google has a translate button on their home search page for ppl in other countries to read in their own language.
I know what the translate feature is. It has absolutely nothing to do with a robots.txt file.
Here, read this from Google
http://www.google.com/support/webmaster … wer=156449
There was a thread about this a few months back Cindy - it may provide you with a solution, quite a few hubbers added to it with what it means, how to work with it etc
by cashmere7 years ago
In my site diagnostics in Google adsense these hubs of mine are being shown as blocked. Reason blocked is Robots.txt File.Can you tell me how I can unblock my following hubs?My favourite Tarot layouts,How to teach...
by Quilligrapher7 years ago
Google AdSense Site Diagnostics reports that a Robots.txt file is blocking their crawler from accessing one of my hubs. Can anyone suggest how I can resolve this issue?
by Mike Bouska7 years ago
When I logged into Adsense today, it gave me an Analytics diagnostic warning for hubpages, saying that they could not crawl because of a Robots.txt error.Anyone else getting this?
by Tammy6 years ago
I tried to asked this question before, but it didn't show up. I logged into Google Webmaster Tools and it said restricted by robots.txt. What is that and how do I fix it? It looks something like this:Show URLs:...
by Carolyn Blacknall8 years ago
I was looking at Google analytics and it said that one of my hubs (Disney-Snow-White-Birthday-Party-Ideas)is blocked from Google crawler because of a Robots.txt file.Their help file says:How can I make sure my...
by pinkytoky6 years ago
When I do URL submissions, sometimes I get a message saying my robots.txt is blocking the robot crawlers. Also, it says my robots.txt file does not comply with the current “Robots Exclusion Standard” and this...
Copyright © 2017 HubPages Inc. and respective owners.
Other product and company names shown may be trademarks of their respective owners.
HubPages® is a registered Service Mark of HubPages, Inc.
HubPages and Hubbers (authors) may earn revenue on this page based on affiliate relationships and advertisements with partners including Amazon, Google, and others.