We have added a feature to the Search tab under the stats box on each Hub that enables you to see when that Hub that has last been crawled by search engines.
When an Idle Hub is crawled, search engines will see that it has a noindex tag, which indicates to search engines that the page should not appear in search results. When a Hub falls into Idle status, it may take some time before the Hub is crawled, so it may remain in search engine results for some time. If you update an Idle Hub quickly, right when you see it is no longer Featured, you may bring it out of Idle status before it is crawled, meaning you can avoid having that Hub removed from search engines' indexes entirely.
Do keep in mind, though, that even after a Hub has been crawled, it might take a search engine some time to take it out of the index or add it back in. Essentially, once a Hub has been crawled and a search engine knows what we say it should do with a Hub, the ball is in the search engine's court.
On the flip side, it may take a while between the point in time at which an Idle Hub becomes Featured again and the point at which it is added back to search engine indexes, as the Hub must be crawled first.
We added this feature to make it easier to see why your Hub is being indexed (or not indexed). We hope you find it useful!
For more information on Featured, Pending, and Idle status, stop by our official Learning Center entry.
On more than one level the "why" is huge--thanks!
What a brilliant idea! This is great Simone. Thanks for letting us know
What a fantastic feature! Thank you Simone and the HubPages Team!
Not only a great idea, but a great help. This is a benefit not only for idle hubs but hubs we are trying to promote! Knowing when (and if) Google or Bing have hits on our hubs is just another great weapon in our arsenal of hub promotions. Thank you, thank you!!!
Hey everyone! I am so pleased that you find this feature to be helpful.
Many thanks to Paul Deeds and Derek Gulbranson for getting this in order and making sure these things are as clear as possible!
This feature has allowed me to see that my latest hub - http://izzym.hubpages.com/hub/People-Wh … re-Pathway - which covers a very important and potentially life-saving topic, was crawled by the googlebot 8 hours ago, when the noindex tag was still on it.
Now it could be weeks before it comes around again, during which time this hub will not be read except by hubbers and social traffic.
Thanks a lot, Hubpages!
It may actually take Google some time to remove your Hub from the index, so you might still be getting search traffic for quite some time.
You misunderstood me Simone.
This is a new hub that was put into pending mode.
It was visited by the googlebot during this period, presumably because other hubbers linked it to Facebook and Twitter.
Google saw the noindex tag. It'll likely be weeks before it gets indexed now.
If you are going to hide newly published hubs, then take away all linking from it to social networks to stop the Googlebot from seeing it.
The last two hubs I published were visited by G bot within 30 minutes, only to be slapped and told to go away! Hiding doesn't work when the hubs are sent to followers, etc. I used to love the way new hubs were indexed so quickly, now this good feature works against us and imposes an indexing delay and lost income. Why is this beneficial to established and trusted writers? I have never had a pending hub fail to get the Prince's Kiss and awaken. This is just a pain.
This "exclusive" page
http://janderson99.hubpages.com/hub/Slo … oes-Recipe
Is listed as being crawled 7 days ago, with an update to say 22 hours ago.
Yet it does not appear in the SERPS using the search term " Slow Roasted Dried Tomatoes Recipe hubpages" and cache:http://janderson99.hubpages.com/hub/Slow-Roasted-Dried-Tomatoes-Recipe produces a 404 error
This test works
so what is going on with the tomatoes one!!!
All we can do is make sure that Hubs are properly tagged and let you know when they are crawled- once they are, it is up to a particular search engine to make a move, I'm afraid!
I looked at one of my hubs still in the pending period
The tool says it was crawled 14 hours ago - with the NOINDEX tag in place.
This means that the bot won't be back for days or weeks
This is bad for authors and bad for HP
The only way I can get pages indexed is to submit them to webmastercentral
Surely there must be a better way. This is extremely annoying!
Sending pages to the sin bin tells Google not to index them
Does HP have any stats on the proportion of pages that are crawled during the pending period and don't get indexed for days or weeks!
I suspect that EVERY new page gets crawled while pending - the bot sees the NOINDEX tag and goes away - not to return for days or weeks - surely this is bad and should be addressed. The hubber and HP loses income, which can never be regained!!!
To be fair, most Hubs do not start making significant income until they're quite mature (e.g. have been around for months- if not longer).
In my experience that notion is out of date.
Why doesn't HP introduce 'Trusted Authors' who wouldn't have all their new hubs sent to the 'sin bin'. Many other sites do it. It would free-up scarce resources. A Win Win!
In theory that's a good idea but I can see problems - it would lay HP open to accusations of favouritism.
Simone....I'm still trying to discover why my traffic hasn't picked back up on my hub (a DIY article) that was, prior to late September, garnering half of my total traffic. I've lost more than half of all hub traffic and half my earnings this month, even after tweaking my "best" hub. It still shows the same Google ranking, so I'm bewildered that its traffic is so decreased. It doesn't help that I'm "technically challenged." Do you have any explanation about the fate of this hub? Thanks in advance for any help you can give me. JAYE
And just how are they supposed to 'mature' if they get hidden in pending status from the get-go? Saying 'it's up to the search engine to re-crawl,' is a cop-out, because it was HP who installed the 'no index' PENALTY!! (For that is what it truly is!) If it's not indexed, the crawler may not come back for weeks, months or at all...lost income ALL AROUND. This was NOT a good move on HP's part!
Being able to "see" when it was last crawled does nothing to get rid of a 'no-index' tag. We need information on HOW to do THAT!
Re-write the article? Add/change tags or keywords? Or, once it's de-indexed, that's that--too bad, so sad?
"To be fair, most Hubs do not start making significant income until they're quite mature (e.g. have been around for months- if not longer)."
Its still lost income - If you impose delays so that an income of $100 that would have occurred in 4 weeks is delayed a week, after 5 weeks your net income is $100 not $200!
The tool should test whether the page is ACTUALLY indexed - that is what is important!!!!
Not to a Smoke and Mirrors Salesperson!
Maybe the tool can also now upgrade all that missing information on the measuring of hub Viewing Durations? It seems that half of mine, even though some of the hubs are over two years old haven't yet been measured yet for some rather obscure reason?
Thanks for the info. Great feature that likely as not I wouldn't have found without seeing this.
Awesome added feature! Thanks Hubpages! I never learned how to find out when the bots come a-crawlin' my hubs, so this added feature is very helpful to me. Cheers!
This is one of the best features yet. Thank you so much for giving us insight into it as well.
Great job HP! Thanks for adding this feature. I can use all the data I can get.
This is an excellent feature! I have been away for several months, and coming back, I see many useful tools that the HP team has implemented. It's a great, and wonderful surprise, to come back to a site that has so many useful tools for the writer.
Thank you and kudos to the HP team!
HubPages suddenly experiences a significant surge in traffic as all hubbers go through their accounts checking when their hubs were last crawled
The points DzyMzLzzy and janderson99 made in posts (above) are correct--seeing when hubs are crawled is not helpful if hubbers cannot do anything about indexing. Those are the "tools" HP needs to provide--so we can see if our hubs are indexed and, if not, show us how to ensure they are indexed by Google and other search engines.
Its a wonderful feature Simone and HP team. i like that i can know when my site has been crawled.
It will be nice to know when the bots are about to come crawling too
this is such a very good feature. now i can manage my hubs even better. Thanks for this one!
Excellent idea for hub management. We can never have enough analytical ideas.
Many thanks for this feature, HP! I was pleasantly surprised when I first saw the message: "congratulations , etc.etc."
Thanks for sharing this info. I will definitely watch carefully for my hubs to come out of idle status.
This is a really cool feature. It definitely takes the guess work out of the equation for the authors. Which gives us more time to research and write hubs!
By the way, Simone, I used the new feature to see when my "best" hub was last crawled by search engines. Not good news. Even though my title uses words from engine searches, Google last crawled the article THREE DAYS AGO, and Bing, two days ago!
Great idea! Thanks you Simone and the rest of the HP team. Thank you very much for your efforts and keeping us informed.
Thanks a lot for Simone it's really helpful for us to be able to see this...appreciate your effort as it gives a broader context to our understanding of our idle hubs...cheers
This is a really nice feature. As soon as I read this, saw it in slow hub, I used it and made some modifications to existing.
Thanks for the info Simone.
Copyright © 2018 HubPages Inc. and respective owners. Other product and company names shown may be trademarks of their respective owners. HubPages® is a registered Service Mark of HubPages, Inc. HubPages and Hubbers (authors) may earn revenue on this page based on affiliate relationships and advertisements with partners including Amazon, Google, and others.
|HubPages Device ID||This is used to identify particular browsers or devices when the access the service, and is used for security reasons.|
|Login||This is necessary to sign in to the HubPages Service.|
|HubPages Traffic Pixel||This is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.|
|Remarketing Pixels||We may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.|
|Conversion Tracking Pixels||We may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.|