jump to last post 1-12 of 12 discussions (28 posts)

How to Increase Crawl Rates on Your Subdomain

  1. Simone Smith profile image92
    Simone Smithposted 3 years ago

    Hey y'all!

    From time to time we hear concerns about newly-Featured Hubs not getting crawled for aaaaages, so we tried out a couple of different solutions (er- rather Paul Edmondson did; I can't take the credit) and finally found something that helps: increasing your subdomain's crawl rate through Google Webmaster Tools.

    I've added tips on doing this to our Learning Center guides to Google Webmaster Tools and Featured Hubs, but the how-to is all in this FAQ entry: http://hubpages.com/faq/#increase-crawl-rate

    Should you be concerned about the rate at which your Hubs are Featured, give this a go! smile

    1. vocalcoach profile image93
      vocalcoachposted 3 years ago in reply to this

      Thanks Simone.  This will be a big help.  Gonna go study your tips as well as the FAQ entry!

      1. Simone Smith profile image92
        Simone Smithposted 3 years ago in reply to this

        Sweet! This one is pretty straightforward. The only catch is that the increased crawl rate has to be re-set every 90 days!

        1. AMFredenburg profile image79
          AMFredenburgposted 3 years ago in reply to this

          A valuable nugget of information; thanks!

        2. tillsontitan profile image91
          tillsontitanposted 3 years ago in reply to this

          Increased crawl rate has to be re-set every 90 days??? Can you explain that one Simone?

    2. shampa sadhya profile image86
      shampa sadhyaposted 3 years ago in reply to this

      Hello Simone Smith,

      Thanks for this info but while I was fixing the crawl rate I noticed that 16 hubs of mine have URL errors. It suggests: 404 Page Not Found. I tried to fix 3 or 4 of them but not confident that I did it correctly. I am not sound in these type of technical matters. I simply need your help.

      May be this is one reason why my traffic has gone down.

      Even I have noticed that in 1 or 2 hub the URL which is mentioned is never used by me. Fact is, I have changed the title of many hubs but did not knew about this kind of error.

      I am feeling helpless so kindly help me to fix the error in a step by step method.   

      Thanks in advance!

      1. Simone Smith profile image92
        Simone Smithposted 3 years ago in reply to this

        Hey shampa sadhya- are you talking about error messages in Webmaster Tools, or on HubPages?

        Rest assured that when you change a title of a Hub, its URL does NOT change. Those cannot be edited once created.

    3. sholland10 profile image92
      sholland10posted 3 years ago in reply to this

      Thanks, Simone!  I had some adjustments to make.  So glad I found your post! :-)

  2. Ethan Green profile image91
    Ethan Greenposted 3 years ago

    That's interesting - I didn't know that was possible. Great advice for my own websites as well.
    Thank you Simone!
    Ethan

    1. Simone Smith profile image92
      Simone Smithposted 3 years ago in reply to this

      Oh, totally! Hope it helps!

  3. EricDockett profile image94
    EricDockettposted 3 years ago

    If we increase our crawl rate, wouldn't that also mean that we'd have a smaller window to edit our Hubs when they fall into idle status before Googlebot comes back and de-indexes them?

    1. Simone Smith profile image92
      Simone Smithposted 3 years ago in reply to this

      Yep! So it's all up to you. It definitely goes both ways.

    2. shampa sadhya profile image86
      shampa sadhyaposted 3 years ago in reply to this

      What does smaller window mean?

      1. EricDockett profile image94
        EricDockettposted 3 years ago in reply to this

        It means, if you increase your crawl rate, you'll have less time to edit and fix up your Hub when it falls into non-featured status before Google re-crawls your subdomain and de-indexes your Hub.

        1. Marisa Wright profile image92
          Marisa Wrightposted 3 years ago in reply to this

          Good point.

  4. LCDWriter profile image94
    LCDWriterposted 3 years ago

    Shampa, were those pages unfeatured?  (no H beside them).  That might be the reason they have an error.

    1. shampa sadhya profile image86
      shampa sadhyaposted 3 years ago in reply to this

      No, they are all featured hubs. That's why I am very much anxious.

      1. LCDWriter profile image94
        LCDWriterposted 3 years ago in reply to this

        Very strange!

      2. wilderness profile image97
        wildernessposted 3 years ago in reply to this

        I wouldn't worry too much - I think that's not too unusual.  For whatever reason google couldn't follow a link from somewhere, so the page couldn't be found.  Maybe HP had a 2 second down time, maybe someone else did.  If often see a few of those, gone a few days later.

        If it continues, though, it could indicate something actually wrong.

        1. Simone Smith profile image92
          Simone Smithposted 3 years ago in reply to this

          That's what I'm thinking. I haven't really seen this happen much though (not that I spend much of any time looking at my Hubs through Webmaser Tools....)

  5. AMFredenburg profile image79
    AMFredenburgposted 3 years ago

    Well, I've done it; we'll see if there's a change. I've only got 4 Hubs so far, so perhaps it won't show all that much.

  6. Rosie2010 profile image82
    Rosie2010posted 3 years ago

    Great useful tip!  Thanks, Simone. smile

  7. paradigmsearch profile image89
    paradigmsearchposted 3 years ago

    I checked some hubs at random. Both ones Google loves and ones Google hates. The absolute oldest crawl date I could find was 3 days. I've decided that since it ain't broke, I'm not going to fix it. What with the oldest crawl date being only 3 days, this kinda indicates to me that Google actually likes my subdomain. Or am I living in fantasy land here?   big_smile

  8. LuisEGonzalez profile image80
    LuisEGonzalezposted 3 years ago

    I did as suggested by Simone and some of my hubs have started showing red arrows. Let's hope this was a good move..................smile

  9. Marcy Goodfleisch profile image93
    Marcy Goodfleischposted 3 years ago

    Thanks, Simone - I will go read and learn how to do this!

  10. Li Galo profile image87
    Li Galoposted 3 years ago

    Thanks for the helpful info.

  11. 0
    Beth37posted 3 years ago

    How do you do this?:

    Navigate to your HubPages subdomain on Google Webmaster Tools

  12. 0
    Confucius Vijayposted 3 years ago

    I think, to say to google to crawl the page, we should submit the RSS feed to google. as after feeding the rss google crawlr will be able to crawl your page fast.

 
working