Can you really prevent duplicate content from a sort function with a robots.txt, or is that...
better addressed through paramaters?
It depends, with robots.txt you can disalow search engine bots to look into some directory, this means this directory will never be indexed. cool thing if you have a great amount of subscriptors, this is a way of doing that directory for subscriptors only and not for search engine spiders.
Hope that helps.
Hi dusy7969, and all the rest of the people
If you are talking about cPatchs...etc be aware that this is often only for Google Crawler info
And is true that it known to cause duplicates.
The Robots.txt is only a file and has absolut no power over the Spiders or Web Crawlers, they only respect what you have in the "text document" if they want, and since they belong to diferent companies and organizations they woul not acept the others rules.
I do not know where you host your pages and if you can edit and use any html code inside your web pages, but one thing that all or almost all Web Crawlers "obey" is to the Metatags that you insert in the Html Head's document.
If you know how to use it then i will show you where you can generate them easy
If you do not know, you can always use the same link from SINALSOFT where you can generate your MetaTags in less than 1 minute (freeware - FreeOnlineTools)
here: ( http://www.sinalsoft.com/tools/?lg=en&a … &pgs=1 )
You can also read my 3 Meta Tags Articles Here: ( http://hubpages.com/hub/programming-a-b … ags-robots )
I hope this helps
by Brett A Rogers 5 years ago
Not sure if this will work, but if it can, then it would be god to do so.As you probably know duplicate content is not allowed, which makes sense from a search engine point of view, but not from the perspective of someone surfing the directories of Hubpages and/or other such websites.If hubs can be...
by Ceres Schwarz 6 years ago
I have a hub that I have yet to publish. I've been working on it and then this warning suddenly shows up "WARNING [?]NOTE: A substantial portion of the content on this hub appears on other pages". How do I get rid of this warning? And how can I find exactly which part of this hub has the...
by Mark Knowles 11 years ago
I am moving this question over here to ask for help from the hubpages team.2 of my hubs and some of hovalis' hubs seem to have had a penalty applied by google and have disappeared from the search rankings down to the last page. The conclusion we have come to is that they have been flagged for...
by Peeples 5 years ago
I am doing a 20 "such and such" quotes about "such and such" hub. I even went back in after they marked it duplicate and made sure 400 of the words are my words, even though I know people googling it aren't going to want to read my words and are instead just going to be coming...
by Raye 10 years ago
I noticed my Hub on Angelina Jolie's tattoos was being flagged as a duplicate, and it turns out that the content has been stolen/copied and now appears on no less than 71 blogs or other pages.How the heck is a person supposed to keep up with this?I could take a flamethrower to the lot of them....
by lifeboost 7 years ago
Hi there One of my hubs has been unpublished, and it says it's because there's duplicate text, but I don't know which part is duplicate? I've written it all myself, and I've just done a "plagiarism check" to be sure that someone else hasn't copied and pasted my writing elsewhere, but it's...
Copyright © 2019 HubPages Inc. and respective owners. Other product and company names shown may be trademarks of their respective owners. HubPages® is a registered Service Mark of HubPages, Inc. HubPages and Hubbers (authors) may earn revenue on this page based on affiliate relationships and advertisements with partners including Amazon, Google, and others.
HubPages Inc, a part of Maven Inc.
|HubPages Device ID||This is used to identify particular browsers or devices when the access the service, and is used for security reasons.|
|Login||This is necessary to sign in to the HubPages Service.|
|HubPages Traffic Pixel||This is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.|
|Remarketing Pixels||We may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.|
|Conversion Tracking Pixels||We may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.|