Peer Reviews and Academic Publishing

Science intersects with mainstream society when controversial topics are debated. Well-meaning non-scientists seek to bolster their positions by referencing "peer reviewed" studies and "published research". Typically, both sides of any remotely scientific debate can point to documentation supporting their point of view. Television news channels introduce scientific experts with publishing credits and letters behind their names to participate in mini-debates sandwiched between Michael Jackson updates. I have some experience with peer reviews and academic publishing; it's an interesting world that might surprise most people.

Scientists, engineers, and researchers don't write papers simply for fun. They may enjoy hunkering over keyboards while the philosophers and art historians are outside tossing Frisbees, but their madness does have a method. In general, a scientist or researcher in academia needs to publish.

A career in teaching at the college level is characterized by the quest for tenure. Earning tenure doesn't ensure a job for life, but it's the closest thing to a guaranteed no-cut contract outside of serving in Congress until the age of 90. Tenured professors can still be fired for extreme moral turpitude or blatant fraud. Publishing papers represents a hugely significant step in the process of earning tenure. "Publish or perish" summarizes how entry-level professors view their early careers in academia.

Given that academic publishing is a necessity, numerous mechanisms have sprung up to meet the demand. Hundreds of conferences take place every year. Each conference targets a specific aspect of science, engineering, or technology. Each conference publishes a compendium of papers from experts in the field. The relationship between conference and writer typifies the dictionary definition of symbiosis. Conferences need authors and authors need outlets for their work.

Consider the general process for publishing a conference paper:

• The conference publishes a "Call for papers" in relevant trade magazines and web sites.
• Authors submit abstracts to the conference organizers.
• The conference organizers consider the abstracts or solicit association members to do so.
• Authors are informed of acceptance or rejectance ;)
  
• Authors write the actual paper and submit to the conference.
  
• Peer review takes place against the complete paper.
  
• Authors are informed of acceptance, rejectance, or suggestions for improvement.
• The paper is added to the conference schedule and becomes part of the published record of the event.

Peer Reviews take place at least twice; upon submission of the abstract and submission of the complete paper. A peer is a specially selected human being meeting some qualifications to be recognized as an expert in the field (without question every conference identifies experts differently.) These folks are responsible for evaluating the content and veracity of the papers they are assigned to review.

Peer reviewers generally work independently. Their methodology is their responsibility.

I have served as a peer reviewer for several conferences. Logistically, the process has been greatly simplified with the advent of the Internet. Experts from all over the world can serve as peer reviewers for conferences they would otherwise have no contact with. A physicist in Albania can evaluate papers submitted by physicists in Alabama to a conference in Algeria. This greatly increases the pool of potential peer reviewers.

I visited the conference web sites and signed up. I provided a brief overview of my qualifications, including my degrees, graduation dates, publications, and academic affiliations. Shortly thereafter I received log-in information that allowed me to download several submissions from potential authors. Keep in mind that the conference needs me and I need them; they require peer reviewers while I need service credits to pump up my curriculum vitae (academic-speak for resume). Should I decide to go for tenure, my application will be more attractive if I've served on peer review committees for conferences.

I received no compensation to serve as a peer reviewer. To my knowledge, this is predominantly a volunteer position.

After being selected as a peer reviewer, I received three 'blind' papers from unidentified authors. Except when I didn't. Theoretically, any identifying information should have been redacted. No names or university affiliations should have appeared in the papers. This requirement prevents reviewers from favoring some authors or punishing others. Believe it or not, science is sometimes beset with political baggage.

In actuality, several of the papers included full contact information for all the authors. As I read the papers I was fully aware of the university affiliations and academic credentials of the work I was reading. I tried to ignore that information. Subconsciously, might reviewers be influenced to favor papers submitted from MIT over papers submitted from PIT (Poughkeepsie Institute of Technology)?

I read the papers and made fun of them.

Not really. I did read the papers. I considered the structure, content, veracity, and relevancy. I tried to understand the equations. I studied the charts and graphs. If a paper focused on wombat habitats in Australia but the event was "The 100th Conference on Technical Computing in Community Colleges", then I suggested that it be rejected. I may have spent 5 minutes on each paper. I may have spent 5 hours. No one knows.

I submitted my conclusions via a form provided on the conference web site. Each conference has a different format for accepting peer reviews. Some conferences simply ask for an overall score. Some conferences allow for extensive comments to be entered. No conference with which I've been associated actually allows a single peer reviewer to accept or reject a specific paper; each submission is subjected to review by several evaluators who are not in contact with each other.

Peer Reviews provide some level of validation for scientific papers that would be published in academic journals. The peer review process varies somewhat between publishers and conferences.

Every conference is different. I haven't been to every conference, but I have served as a peer reviewer and I have published and presented papers.