HubPages Blocked And Not Crawled By Adsense Crawlers?

1. 62

   SunSevenposted 17 years ago

   Hi Paul,
   
   When I checked my adsense account today I found HubPages articles and profile page in listed in the site diagnostics area. Is anyone else having this issue? How do I go about solving this problem?
   
   http://hubpages.com/hub/ says multiple errors as the reason blocked.
   
   http://hubpages.com/profile/SunSeven/map  says Robots.txt File as the reason blocked.

   replyreport
   1. 70

      jstankeviczposted 17 years agoin reply to this

      Yep. I strongly encourage you all to:
      
      1. Take a deep breath; this is a common thing amongst successful online writers, and is easy to deal with
      2. File a DMCA complaint (you can use the template we provide in our Learning Center or the template Melbel shared above, which is the same)
      
      I'm sorry this happened! Just keep in mind, this is a somewhat normal thing. No cause for alarm. Yes, it's frustrating, and it's a hassle to send over the email, but it's no cause for serious concern.

      replyreport
2. 60

   paddy_avfcposted 17 years ago

   I have just checked my account and the same message is appearing in my site diagnostics area.

   replyreport
3. 70

   waynetposted 17 years ago

   Yep the same for me, any help hub mods!!!??

   replyreport
4. 70

   jstankeviczposted 17 years ago

   All gone, none blocked now. Yet another aspect of this online experience that I don't fully understand!

   replyreport
5. STAFF

   pauldeedsposted 17 years ago

   We block certain urls from search engine web crawlers to prevent the same page from getting into the search index under two different urls (which can lower ranking in search results).
   
   However, there is no reason to block the adsense bot from any pages, since it is just crawling the page to determine what ads to run on it.  I've just made a change to allow that bot on any page.  It should have only been affecting pageviews that had query string parameters in the url (which is normally a very small minority of all pageviews), for instance something like this: "?utm_source=rss".
   
   Thanks for bringing that to our attention, SunSeven.

   replyreport
6. 62

   SunSevenposted 17 years ago

   Thank you for the reply Paul. But the issue still persists.

   replyreport
7. STAFF

   pauldeedsposted 17 years ago

   Google caches the robots.txt file for a period, so it may take a day or two for it to clear up.

   replyreport
8. 62

   SunSevenposted 17 years ago

   Thanks a lot Paul. It is cleared now. Sometimes how Google works is beyond me. I have one doubt, though. The adsense ads seem to be away from the context. And CTR has gone down a bit. Perhaps I am seeing different ads. Can you have a look please?

   replyreport
9. 60

   Prince Maakposted 17 years ago

   No, Everything is OK, I didn`t found any blocked page.

   replyreport