You can use just about any comic image within the realms of copyright "fair use". Here is a link to the Fair Use provision specifically.
The key part being...
"Section 107 contains a list of the various purposes for which the reproduction of a particular work may be considered fair, such as criticism, comment, news reporting, teaching, scholarship, and research."
It is always good practice to research your artwork and properly credit it. And it's greatly appreciated by the creators if you properly credit them as well. For instance, a normal credit would say something like:
Silver Surfer #1 Copyright 1968 Marvel Comics
But do yourself a favor and take the extra step to properly credit the creators and artists of the artwork. So add:
Created by Jack Kirby,
Cover artwork by John Buscema and Joe Sinnot,
Written by Stan Lee
More than not, I tend to see only the publisher being credited when using artwork. And in reality the publication would be nothing without it's creators. So please do your due diligence and go that extra step. I realize this is not always possible, but I'd venture to say that 90% of all comics can have identifiable creators and artists.
Many publishers did not credit creators until the 60's or 70's. And some like Gold Key and Dell rarely if ever credited creators. However there is a great resource in the Grand Comics Database where thousands of comic experts have taken the time to research and find those missing creators and list them properly in their database. Check it out, it may also be a great resource for your artwork research as well. In which case I would highly recommend crediting the GCD for providing the artwork.
Good luck with your writing! The world needs more comic writers writing about comics! Thanks for your passion in comics!
Your Friend and Fan,
CE Publishing Group