hi Just a quick question, just checking my Adsense, as you do, and see that on three of my hubs the google crawler is blocked because of robot.txt. What the hell is that and how do I fix it?
I don't actually know but I would speculate that google thinks your hubs have been copy/pasted from somewhere else or that you didn't write them by hand...you could try editing them and re-typing maybe? Sorry I'm just guessing here...
I just googled it and this page might help;
to fix this, go to the control panel(cpanel or whatever your hosting site gave you) of your hosted blog, and browse to /public_html/robots.txt using file manager.
Open the robots.txt and then enter these lines here in quote or copy it from the google adsense error page that you got :
Once you done adding these you can save the robots.txt and you'll find adsense bot crawling without any problem.
the robot.txt was added on your side. google does not have the ability to add this to your hubs.
Here's my response to the same question a few days ago...http://hubpages.com/forum/topic/33803#post750921
check the settings for the affected hubs. make sure you have the box checked for the question about " content meets the conditions of the google adsense policy"
not necessarily...google indexes hubs on their owns servers for retrieval and if somebody hit translate then a new"translated" copy with the .txt string is added so that every time a translate request is received, it will always translate from the original and not in series from translation 2 translation so that the meaning is not obscured over time.
robots.txt is used that way to keep the "translated" .doc from being indexed and other requests from grabbing it and so uses the original indexed one every time. otherwise it would look like gleep , glop and ascii code over time probably and look like wingdings hieroglyphic font eventually LOL !
the next time someone orders up your hub from googles search engine in original english, than the message will be gone.Your original copy is still in tact and indexed at googles server waiting for another query request.
in other words. it got translated. and will set itself straight next time someone clicks her on the serps. Google has a translate button on their home search page for ppl in other countries to read in their own language.
I know what the translate feature is. It has absolutely nothing to do with a robots.txt file.
Here, read this from Google
http://www.google.com/support/webmaster … wer=156449
There was a thread about this a few months back Cindy - it may provide you with a solution, quite a few hubbers added to it with what it means, how to work with it etc
by Quilligrapher6 years ago
Google AdSense Site Diagnostics reports that a Robots.txt file is blocking their crawler from accessing one of my hubs. Can anyone suggest how I can resolve this issue?
by cashmere6 years ago
In my site diagnostics in Google adsense these hubs of mine are being shown as blocked. Reason blocked is Robots.txt File.Can you tell me how I can unblock my following hubs?My favourite Tarot layouts,How to teach...
by puter_dr6 years ago
When I logged into Adsense today, it gave me an Analytics diagnostic warning for hubpages, saying that they could not crawl because of a Robots.txt error.Anyone else getting this?
by Tammy5 years ago
I tried to asked this question before, but it didn't show up. I logged into Google Webmaster Tools and it said restricted by robots.txt. What is that and how do I fix it? It looks something like this:Show URLs:...
by Ray Choiniere aka Cagsil2 years ago
Okay, I know I'm not the sharpest tool in the shed, but I went to Google Adsense day before yesterday and I inserted my new hub into the URL Channels.When I inserted it, I also decided to check my URL channels against...
by Eileen Hughes6 years ago
I posted this in a follow up in Maddies article on this. But didnt know if anyone would go there.I went into Diagnostic for first time and some of mine are blocked by the robot.txt fileGooggle adsense ?? says...
Copyright © 2016 HubPages Inc. and respective owners.
Other product and company names shown may be trademarks of their respective owners.
HubPages® is a registered Service Mark of HubPages, Inc.
HubPages and Hubbers (authors) may earn revenue on this page based on affiliate relationships and advertisements with partners including Amazon, Google, and others.