Google Caffeine - seo implications of Google's Next Generation search engine

Google is testing a new version of their search engine - nicknamed Google Caffeine.

On their webmaster central blog, they say that

The new infrastructure sits "under the hood" of Google's search engine, which means that most users won't notice a difference in search results. But web developers and power searchers might notice a few differences, so we're opening up a web developer preview to collect feedback.

and they are inviting webmasters to offer feedback.

Where to find the test search engine

To test out the new search engine go to Google Caffeine [edit: Caffeine has now gone live, so the test version has disappeared], and type in queries as normal.

To spot the differences it's helpful to open a second tab with the existing search engine and perform the same queries.

The differences in the results mean that webmasters will need to optimize for Google Caffeine.

What are the differences between Google Caffeine and Google?

[Note: - some of this has changed - I have kept the original findings and put an update below, so it's obvious just what has changed in the last month]

1. Google Caffeine is way faster to load.

2. The new index is much bigger (to check this, type in your query and see the number returned on the top right of the screen and compare to the old search engine).

However, on certain subjects, the new engine has a smaller index. eg at the time I tested, "make money online" has 98.7 million results in the new engine but 169 million results in the old engine. "Weight loss" has 68.4 milion results in the new engine but 101 million results in the old engine. "credit cards" has 86.7 million results in the new engine but 131 million results in the old engine. This tells me that the new engine has deindexed a lot of the sites that scammers have put up - the fake copy/paste jobbies designed to lure some desperate unsuspecting visitor to parting with money.

What about certain domains?

Well Hubpages comes out better - putting in the operator site:hubpages.com into the new engine returns 1,830,000 results, compared to 1,800,000 results in the old engine. Squidoo also gets more pages indexed; 2,200,000 results compared to 2,180,000 in the old engine, though notably they didn't gain as many pages as Hubpages. Ezine articles does significantly better - 4,110,000 in the new engine compared to 4,000,000 in the old one - a 2.75% improvement. Infobarrel, the new article site that everyone is getting excited about loses pages: 12,000 in the new engine compared to 12,400 in the old. EHow also loses: 4,290,000 in the new engine compared to 4,330,000 in the old engine.

3. The algorithm is slightly different and thus the order of the results is different. From my initial tests, they seem to give more weight to having the entire keyword string in the URL. In my test, I saw pages in the old engine where the URL had numbers after the domain name disappear in the new engine to be replaced with pages that had the keyword string or at least part of it in the URL.

4. pages with the keyword in the title, in the snippet of text and in the URL seem to do best.

5. twitter pages seem to be showing up higher in the new engine as are Facebook pages.

6. They seem to be focusing on real time, so that pages are being popped into the results even before they've been fully indexed for breaking news subjects (i.e. you will see them there without a cache indicating that it's the first time the bot found them).

7. More weight seems to be given to on-page SEO. For instance I've spotted pages with keywords bolded on the page in the new engine which weren't anywhere to be found in the first three pages of the old engine.

I will be testing a ton more over the next few days to find out the search engine optimization implications of Google Caffeine, and will update this page. If people have any observations, please leave your thoughts in the comments.

Update 12th Sept 2009 & 12 Oct 2009

It's been interesting to revisit point number 2 above.

There has been much churning of pages in both the old and the new Caffeine index. Google seems to be making it's pagerank changes in the existing index first, and then amending the new Caffeine index. And the way they seem to make the changes is to remove pages from the index, and then gradually add pages back that fit their new ranking criteria.

Rather than subject you to a lot of text to read about which changes have occured, I've summarized them in the tables below. It provides a fascinating snapshot of how Google changes the composition of it's index on an almost continual basis.

keyword: Make money online

date 
old (existing index 
New Caffeine index 
12 Aug 2009 
169,000,000 
98,700,000 
 
 
 
12 Oct 2009 
174,000,000 
146,000,000 

Keyword: Weight loss

date 
old (existing) index 
New Caffeine Index 
12 Aug 2009 
101,000,000 
68,400,000 
12 Sept 2009 
101,000,000 
66,400,000 
12 Oct 2009 
101,000,000 
89,000,000 

Keyword: Credit Cards

date 
Old (existing) index 
New Caffeine index 
12 Aug 2009 
131,000,000 
86,700,000 
12 Sept 2009 
129,000,000 
80,100,000 
12 Oct 2009 
42,400,000 
107,000,000 

Site: Hubpages.com

date 
old (existing) index 
new caffeine index 
12 Aug 2009 
1,800,000 
1,830,000 
12 Sept 2009 
1,100,000
1,140,000 
12 Oct 2009 
1,080,000 
1,110,000 

Site: Squidoo.com

date 
old (existing) index 
new Caffeine index 
12 Aug 2009 
2,180,000 
2,200,000 
12 Sept 2009 
2,200,000 
2,220,000 
12 Oct 2009 
4,780,000 
2,320,000

Site: Ezine articles

date 
old (existing) index 
new Caffeine index 
12 Aug 2009 
4,000,000 
4,110,000 
12 Sept 2009 
3,750,000 
3,740,000 
12 Oct 2009 
3,690,000 
3,640,000 

Site: EHow.com

date  
old (existing) index 
new caffeine index 
12 Aug 2009 
4,330,000 
4,290,000 
12 Sept 2009 
4,170,000 
4,170,000 
12 Oct 2009 
4,310,000 
4,300,000 

Why does the composition of the index matter so much?

It's clear from everything Google has said (and the results above) that this update to their search engine is more about the composition of the index than about algorithm changes (which appear to be slight).

However, changes in the composition of the index can have a bigger impact than algorithm changes. For instance if they suddenly find 200 pages that have links to you, you will rise in the rankings. If they suddenly deindex 200 pages linking to your site, you lose the value of any links to your site. This is particularly important if you are using Ezine articles to get backlinks.

I'm surprised that there is not more discussion about the impact of Caffeine to be honest. Apart from the initial flurry, when everyone talked about how much new social media was included, everyone has gone quiet. In particular people seem to have missed the massive deindexing that is taking place in some topics.

Update Feb 2010: it appears Google Caffeine has started to be rolled out in most data centres

Update 2012 It seems that the Caffeine infrastructure has helped Google roll out the Panda and penguin updates as it has provided them with the computing power needed for their new machine learning capability

Other pages you may be interested in

More by this Author


Comments 28 comments

expectus profile image

expectus 7 years ago from Land Downunder

nice hub, it seems good and it definitely felt much speedier :)


sandra rinck 7 years ago

Cool, I am gonna check it out.


Insurance Guy profile image

Insurance Guy 7 years ago

Nice hub, and yeah it's much faster, thnx for the info.


kephrira profile image

kephrira 7 years ago from Birmingham

thanks for the info, its very interesting. Seems like an improvement on old google to me, I just hope the changes put my sites up and not down!


rb11 profile image

rb11 7 years ago from Las Vegas

This is the second time I've heard of this, caffeine. For us who have an interest in SEO your testing will be of great help, Thanks..

Regards


kiwi91 profile image

kiwi91 7 years ago from USA

This is interesting. Hopefully all of our SEO efforts will help us with Google Caffeine if it's giving bold tags more of a push. I'll have to try it out and see how it works for me. Thanks!


jill of alltrades profile image

jill of alltrades 7 years ago from Philippines

Cool! I will have to check this out.

Thank you for sharing!


prasetio30 profile image

prasetio30 7 years ago from malang-indonesia

it looks like interesting. I just know about google caffeine form this hub. I think its good for SEO. I'll find it directly. And get the advantages.


Eric Graudins profile image

Eric Graudins 7 years ago from Australia

Thanks Silver rose, but

It's not available at the moment so I can't check it out.


Jane@CM profile image

Jane@CM 7 years ago

Silver Rose, very interesting article, I had not heard of this before. Going to look into it.


Silver Rose profile image

Silver Rose 7 years ago from UK Author

Eric - yes, they've put a message up saying they are upgrading their data centre. Should be available again in a few hours.


RockyMtnHi profile image

RockyMtnHi 7 years ago from Denver, Colorado

Another excellent scoop.


nahid5692001 profile image

nahid5692001 7 years ago

Great article. I tried the new engine, and the first thing that caught my eye was the extremely fast speed. It was unbelievable hopefully it stays this way. Great article!


MistHaven profile image

MistHaven 7 years ago from New Jersey

Cool, any word when Google Caffeine will become official?


MagicStarER profile image

MagicStarER 7 years ago from Western Kentucky

Keywords in the url, the title, and bold print... hmmm...very useful information. Thanks a lot for sharing!


Research Analyst profile image

Research Analyst 7 years ago

I think it has to do with natural organic search, Google wants to make sure that when a person searches for something they will get relevant pages demographically.

So if I type in "foot fungus home remedy" I will get only the sites that actually cover this whole topic, because I searched for a specific niche.

So your right the sites with that exact phrase in the title and url will rank high on the first page and more than likely I will click on the site for more information.


Mark Stroud 7 years ago

Great read - any thanks and what a relief seem like it favoring proper white hat seo so if you been following google guidelines looks like we should be ok


Direct Response profile image

Direct Response 7 years ago from Melbourne, Australia

To add to this information - There was an interview where someone (from Google) stated that Google Caffeine loves the layout of Word Press sites.

This along with other comments I have heard tells me that Google is Customer focused. What I am saying is that they have always wanted to give the best results and experience to people searching for information.

Over the last few months I have noticed that it is very simple to beat long established websites with lots of backlinks if you have your keyword in your Domain name, Title, Description, Keywords, H1, h2 and text (below 6% within text).

These results show me that Google is moving towards relevancy having more importance within websites. This is a great step forward for us marketers that stay within the guide lines as it will reduce the Spamer Sites.


waynet profile image

waynet 7 years ago from Hull City United Kingdom

I've noticed the de-indexing of some of my websites and it is frustrating when this happens but then there is the ability to learn to adapt to these things if we plan on making online our home for business.


scheng1 6 years ago

Someone did observe that Google Caffeine seems to emphasize onpage seo more than offpage seo. The twitter or whatever real page search is useless, hope they can create a separate one for twitter user.


Toronto12 profile image

Toronto12 6 years ago from Toronto

I found my sites to jump a couple spots in the SERPs after caffeine. I'm feeling that they're giving more weight to relevant links.


Csjun89 profile image

Csjun89 6 years ago

This is some pretty interesting stuff, have you thought about giving it a more recent update?


Mandar Thosar 6 years ago

I have read just now that launch of Google Caffeine has not impacted the search listing in big way. May be Google has introduced Caffeine slowly and now when the process is 100% complete published the news.

I have not checked the results myself yet.


vocalcoach profile image

vocalcoach 6 years ago from Nashville Tn.

This is good news. I will definetely check this out. Also, I will nake sure the keyword is in my link, title and as you suggested in bold. Thanks for the great info. Thumbs up!


Entourage_007 profile image

Entourage_007 6 years ago from Santa Barbara, CA

I've been hearing a lot about Google Caffeine, great article.


Louis 6 years ago

Great article. I've been wondering how exactly Google's new algorithm has been effecting SEO. Thanks for the info.


FindYourSearch profile image

FindYourSearch 6 years ago from Las Vegas, NV

Great analysis. I'm definitely liking the increased speed of indexing since the update.


fdoleac profile image

fdoleac 6 years ago from Hollis, New Hampshire

Thanks for the informative article. We are attempting to adapt.

    Sign in or sign up and post using a HubPages Network account.

    0 of 8192 characters used
    Post Comment

    No HTML is allowed in comments, but URLs will be hyperlinked. Comments are not for promoting your articles or other sites.


    Click to Rate This Article
    working