Making Sense of Scientific Literature

Many companies and products make promises that sound too good to be true. They often claim that they have science backing their product and that they can heal whatever ails you or fix whatever is wrong. So how are consumers supposed to discern which products really work and which products are less than what they claim to be? Check the science.

When a scientist invents a new product, process or technology that is scientifically sound they have an ethical duty to publish a scientific article about it so other scientists are aware of the progress in the field. This means that consumers can ask for the scientific literature behind a new invention or product and can check the science for themselves!

If there isn’t literature, be wary. Ask if literature is forthcoming. If not, proceed very cautiously. The product is probably not going to live up to all of its claims. If the literature is forthcoming, and it’s realistic, ask for a publication date and reconsider the product when the literature is available.

Be very careful to simply accept testimonials they give you, especially if this is an MLM company. Of course there are lots of people who want you believe it works! Are there people who aren’t getting paid by the company who say it works and want you to believe it works? If not, you can probably chalk it all up as a fraud.

The company should be able to provide you with a copy of any articles published about their product. If they can’t, ask for a citation. Even with just the citation you can learn a lot.

One of the most important things you will learn from the citation is the name of the journal in which the article was published. This is important for two reasons. 1) So you can find the article. 2) So you can find the impact score of the journal, which will tell you how prestigious the journal is.

Many public and university libraries have access to journals. Consider calling the nearest libraries and checking to see if they have a license to the journal. If so, you can review the article there for free.

A journal’s impact score is the measure of its prestige and is an estimation of its impact on the scientific community. Journals with impact scores in the double digits have very high impact scores and are considered to be reliable. Unfortunately, there’s more than one reason that a journal may have a single digit impact score. Narrower journals are going to have lower impact scores as impact scores are directly related to the number of citations their articles receive the year after they were published and narrower journals will have a smaller audience. Narrow journals will typically have impact factors between 0.5 and 4. If the impact factor of the article is less than 2, there’s a red flag. It’s possible that the science is well done, and a publication journal was simply poorly selected. However, it’s also possible that the science is questionable or not as novel as the company would like customers to believe.

If you’re unable to see the full article, a simple google scholar search of the citation of the article should at least allow you to see the front page or abstract of the article. From this document you should be able to gather the companies or universities that supported the research, any government funding and the names of the authors. In this case, governmental funding is usually good. It means that they submitted a plan of their proposed work, and scientists that work for the government reviewed it and chose them over many other proposals for the funding. Furthermore, if you recognize the university’s name for more than just being your local junior college, that’s also a plus for the article. And, you can do a google scholar search on the names of the authors. Authors who have published consistently for many years are more easily to trust than people who have only published one article. At least one author on the article should have written several papers on products, processes, or topics other than the current product in question.

If you have hit red flags and don’t have further access to the article, you can probably pass the product off as a fraud and move on.

If you do have access to the article, there are a few more things you can check:

Turn to the back of the article. Does it cite a good number of other papers? New science isn’t developed in a vacuum. Real science is built on the shoulders of previous science. This means, that if the article is worth its weight in salt, it’ll site a good number of articles that helped the authors arrive at the conclusions they claim in their article.

Next, turn to the experimental portion of the article. Read through what they did. Try to digest as much as you can, and don’t worry about what you don’t understand. Check for sample sizes. Small sample sizes mean that their results could be nothing more than flukes. Look for whether or not they repeated their experiments several times. Brownie points should be given if the experiment was repeated by separate labs.

After the experimental portion of the article, read through the results portion. Are all of their results reported with a margin of error? Does their margin of error make sense? Or is it so large that the data doesn’t mean anything? (Remember, with larger data sets will come larger margins of error.)

Once you have read the experimental and results sections, read the article from introduction to conclusion. Does the science seem logical to you? Can you come up with other things that might explain their results that weren’t addressed in the article? If you can, ask the company about those possibilities. If they can’t address your question, you’re probably right that they jumped to conclusions and should look elsewhere.

If everything checks out, you’re probably safe to proceed in your purchase or investment. If there were several red flags, consider doing more research, giving the company more time to address the concerns, or find a new investment.

If you’re still unsure about whether or not you should get involved with the company, find an expert in their field at your local university and ask if he or she would be willing to review the article with you. Send it to them a week or two in advance so they can look over it, and bring a hard copy with you to review with them. Try not to take more than 10-15 minutes of their time.

Reading scientific articles can be hard, especially when you feel like they’re talking over your head. But, they can contain a wealth of information and, with a little practice, you too can learn to decipher good science from not so good science! This basic understanding of how to use scientific literature can save you from many poor investments and make you a much wiser consumer!