How to Increase Crawl Rates on Your Subdomain

Jump to Last Post 1-12 of 12 discussions (28 posts)
  1. Simone Smith profile image95
    Simone Smithposted 5 years ago

    Hey y'all!

    From time to time we hear concerns about newly-Featured Hubs not getting crawled for aaaaages, so we tried out a couple of different solutions (er- rather Paul Edmondson did; I can't take the credit) and finally found something that helps: increasing your subdomain's crawl rate through Google Webmaster Tools.

    I've added tips on doing this to our Learning Center guides to Google Webmaster Tools and Featured Hubs, but the how-to is all in this FAQ entry: http://hubpages.com/faq/#increase-crawl-rate

    Should you be concerned about the rate at which your Hubs are Featured, give this a go! smile

    1. vocalcoach profile image96
      vocalcoachposted 5 years agoin reply to this

      Thanks Simone.  This will be a big help.  Gonna go study your tips as well as the FAQ entry!

      1. Simone Smith profile image95
        Simone Smithposted 5 years agoin reply to this

        Sweet! This one is pretty straightforward. The only catch is that the increased crawl rate has to be re-set every 90 days!

        1. AMFredenburg profile image77
          AMFredenburgposted 5 years agoin reply to this

          A valuable nugget of information; thanks!

        2. tillsontitan profile image84
          tillsontitanposted 5 years agoin reply to this

          Increased crawl rate has to be re-set every 90 days??? Can you explain that one Simone?

    2. shampa sadhya profile image74
      shampa sadhyaposted 5 years agoin reply to this

      Hello Simone Smith,

      Thanks for this info but while I was fixing the crawl rate I noticed that 16 hubs of mine have URL errors. It suggests: 404 Page Not Found. I tried to fix 3 or 4 of them but not confident that I did it correctly. I am not sound in these type of technical matters. I simply need your help.

      May be this is one reason why my traffic has gone down.

      Even I have noticed that in 1 or 2 hub the URL which is mentioned is never used by me. Fact is, I have changed the title of many hubs but did not knew about this kind of error.

      I am feeling helpless so kindly help me to fix the error in a step by step method.   

      Thanks in advance!

      1. Simone Smith profile image95
        Simone Smithposted 5 years agoin reply to this

        Hey shampa sadhya- are you talking about error messages in Webmaster Tools, or on HubPages?

        Rest assured that when you change a title of a Hub, its URL does NOT change. Those cannot be edited once created.

    3. sholland10 profile image92
      sholland10posted 5 years agoin reply to this

      Thanks, Simone!  I had some adjustments to make.  So glad I found your post! :-)

  2. Ethan Green profile image88
    Ethan Greenposted 5 years ago

    That's interesting - I didn't know that was possible. Great advice for my own websites as well.
    Thank you Simone!
    Ethan

    1. Simone Smith profile image95
      Simone Smithposted 5 years agoin reply to this

      Oh, totally! Hope it helps!

  3. EricDockett profile image98
    EricDockettposted 5 years ago

    If we increase our crawl rate, wouldn't that also mean that we'd have a smaller window to edit our Hubs when they fall into idle status before Googlebot comes back and de-indexes them?

    1. Simone Smith profile image95
      Simone Smithposted 5 years agoin reply to this

      Yep! So it's all up to you. It definitely goes both ways.

    2. shampa sadhya profile image74
      shampa sadhyaposted 5 years agoin reply to this

      What does smaller window mean?

      1. EricDockett profile image98
        EricDockettposted 5 years agoin reply to this

        It means, if you increase your crawl rate, you'll have less time to edit and fix up your Hub when it falls into non-featured status before Google re-crawls your subdomain and de-indexes your Hub.

        1. Marisa Wright profile image95
          Marisa Wrightposted 5 years agoin reply to this

          Good point.

  4. LCDWriter profile image94
    LCDWriterposted 5 years ago

    Shampa, were those pages unfeatured?  (no H beside them).  That might be the reason they have an error.

    1. shampa sadhya profile image74
      shampa sadhyaposted 5 years agoin reply to this

      No, they are all featured hubs. That's why I am very much anxious.

      1. LCDWriter profile image94
        LCDWriterposted 5 years agoin reply to this

        Very strange!

      2. wilderness profile image97
        wildernessposted 5 years agoin reply to this

        I wouldn't worry too much - I think that's not too unusual.  For whatever reason google couldn't follow a link from somewhere, so the page couldn't be found.  Maybe HP had a 2 second down time, maybe someone else did.  If often see a few of those, gone a few days later.

        If it continues, though, it could indicate something actually wrong.

        1. Simone Smith profile image95
          Simone Smithposted 5 years agoin reply to this

          That's what I'm thinking. I haven't really seen this happen much though (not that I spend much of any time looking at my Hubs through Webmaser Tools....)

  5. AMFredenburg profile image77
    AMFredenburgposted 5 years ago

    Well, I've done it; we'll see if there's a change. I've only got 4 Hubs so far, so perhaps it won't show all that much.

  6. Rosie2010 profile image73
    Rosie2010posted 5 years ago

    Great useful tip!  Thanks, Simone. smile

  7. paradigmsearch profile image87
    paradigmsearchposted 5 years ago

    I checked some hubs at random. Both ones Google loves and ones Google hates. The absolute oldest crawl date I could find was 3 days. I've decided that since it ain't broke, I'm not going to fix it. What with the oldest crawl date being only 3 days, this kinda indicates to me that Google actually likes my subdomain. Or am I living in fantasy land here?   big_smile

  8. LuisEGonzalez profile image83
    LuisEGonzalezposted 5 years ago

    I did as suggested by Simone and some of my hubs have started showing red arrows. Let's hope this was a good move..................smile

  9. Marcy Goodfleisch profile image94
    Marcy Goodfleischposted 5 years ago

    Thanks, Simone - I will go read and learn how to do this!

  10. Li Galo profile image69
    Li Galoposted 5 years ago

    Thanks for the helpful info.

  11. profile image0
    Beth37posted 5 years ago

    How do you do this?:

    Navigate to your HubPages subdomain on Google Webmaster Tools

  12. profile image0
    Confucius Vijayposted 5 years ago

    I think, to say to google to crawl the page, we should submit the RSS feed to google. as after feeding the rss google crawlr will be able to crawl your page fast.

 
working

This website uses cookies

As a user in the EEA, your approval is needed on a few things. To provide a better website experience, hubpages.com uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at: https://hubpages.com/privacy-policy#gdpr

Show Details
Necessary
HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
LoginThis is necessary to sign in to the HubPages Service.
Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
AkismetThis is used to detect comment spam. (Privacy Policy)
HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the googleapis.com or gstatic.com domains, for performance and efficiency reasons. (Privacy Policy)
Features
Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
MavenThis supports the Maven widget and search functionality. (Privacy Policy)
Marketing
Google AdSenseThis is an ad network. (Privacy Policy)
Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
Index ExchangeThis is an ad network. (Privacy Policy)
SovrnThis is an ad network. (Privacy Policy)
Facebook AdsThis is an ad network. (Privacy Policy)
Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
AppNexusThis is an ad network. (Privacy Policy)
OpenxThis is an ad network. (Privacy Policy)
Rubicon ProjectThis is an ad network. (Privacy Policy)
TripleLiftThis is an ad network. (Privacy Policy)
Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
Statistics
Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)