HubPages Blocked And Not Crawled By Adsense Crawlers?

Jump to Last Post 1-9 of 9 discussions (10 posts)
  1. SunSeven profile image67
    SunSevenposted 11 years ago

    Hi Paul,

    When I checked my adsense account today I found HubPages articles and profile page in listed in the site diagnostics area. Is anyone else having this issue? How do I go about solving this problem? says multiple errors as the reason blocked.  says Robots.txt File as the reason blocked.

    1. jstankevicz profile image85
      jstankeviczposted 11 years agoin reply to this

      Yep. I strongly encourage you all to:

      1. Take a deep breath; this is a common thing amongst successful online writers, and is easy to deal with
      2. File a DMCA complaint (you can use the template we provide in our Learning Center or the template Melbel shared above, which is the same)

      I'm sorry this happened! Just keep in mind, this is a somewhat normal thing. No cause for alarm. Yes, it's frustrating, and it's a hassle to send over the email, but it's no cause for serious concern. smile

  2. paddy_avfc profile image56
    paddy_avfcposted 11 years ago

    I have just checked my account and the same message is appearing in my site diagnostics area.

  3. waynet profile image73
    waynetposted 11 years ago

    Yep the same for me, any help hub mods!!!??

  4. jstankevicz profile image85
    jstankeviczposted 11 years ago

    All gone, none blocked now. Yet another aspect of this online experience that I don't fully understand!

  5. pauldeeds profile imageSTAFF
    pauldeedsposted 11 years ago

    We block certain urls from search engine web crawlers to prevent the same page from getting into the search index under two different urls (which can lower ranking in search results).

    However, there is no reason to block the adsense bot from any pages, since it is just crawling the page to determine what ads to run on it.  I've just made a change to allow that bot on any page.  It should have only been affecting pageviews that had query string parameters in the url (which is normally a very small minority of all pageviews), for instance something like this: "?utm_source=rss".

    Thanks for bringing that to our attention, SunSeven.

  6. SunSeven profile image67
    SunSevenposted 11 years ago

    Thank you for the reply Paul. But the issue still persists.

  7. pauldeeds profile imageSTAFF
    pauldeedsposted 11 years ago

    Google caches the robots.txt file for a period, so it may take a day or two for it to clear up.

  8. SunSeven profile image67
    SunSevenposted 11 years ago

    Thanks a lot Paul. It is cleared now. Sometimes how Google works is beyond me. I have one doubt, though. The adsense ads seem to be away from the context. And CTR has gone down a bit. Perhaps I am seeing different ads. Can you have a look please?

  9. Prince Maak profile image65
    Prince Maakposted 11 years ago

    No, Everything is OK, I didn`t found any blocked page.


This website uses cookies

As a user in the EEA, your approval is needed on a few things. To provide a better website experience, uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at:

Show Details
HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
LoginThis is necessary to sign in to the HubPages Service.
Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
AkismetThis is used to detect comment spam. (Privacy Policy)
HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the or domains, for performance and efficiency reasons. (Privacy Policy)
Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
MavenThis supports the Maven widget and search functionality. (Privacy Policy)
Google AdSenseThis is an ad network. (Privacy Policy)
Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
Index ExchangeThis is an ad network. (Privacy Policy)
SovrnThis is an ad network. (Privacy Policy)
Facebook AdsThis is an ad network. (Privacy Policy)
Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
AppNexusThis is an ad network. (Privacy Policy)
OpenxThis is an ad network. (Privacy Policy)
Rubicon ProjectThis is an ad network. (Privacy Policy)
TripleLiftThis is an ad network. (Privacy Policy)
Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)