Can you really prevent duplicate content from a sort function with a robots.txt, or is that...
better addressed through paramaters?
It depends, with robots.txt you can disalow search engine bots to look into some directory, this means this directory will never be indexed. cool thing if you have a great amount of subscriptors, this is a way of doing that directory for subscriptors only and not for search engine spiders.
Hope that helps.
Hi dusy7969, and all the rest of the people
If you are talking about cPatchs...etc be aware that this is often only for Google Crawler info
And is true that it known to cause duplicates.
The Robots.txt is only a file and has absolut no power over the Spiders or Web Crawlers, they only respect what you have in the "text document" if they want, and since they belong to diferent companies and organizations they woul not acept the others rules.
I do not know where you host your pages and if you can edit and use any html code inside your web pages, but one thing that all or almost all Web Crawlers "obey" is to the Metatags that you insert in the Html Head's document.
If you know how to use it then i will show you where you can generate them easy
If you do not know, you can always use the same link from SINALSOFT where you can generate your MetaTags in less than 1 minute (freeware - FreeOnlineTools)
here: ( http://www.sinalsoft.com/tools/?lg=en&a … &pgs=1 )
You can also read my 3 Meta Tags Articles Here: ( http://hubpages.com/hub/programming-a-b … ags-robots )
I hope this helps
by Brett A Rogers 4 years ago
Not sure if this will work, but if it can, then it would be god to do so.As you probably know duplicate content is not allowed, which makes sense from a search engine point of view, but not from the perspective of someone surfing the directories of Hubpages and/or other such websites.If hubs can be...
by Mark Knowles 10 years ago
I am moving this question over here to ask for help from the hubpages team.2 of my hubs and some of hovalis' hubs seem to have had a penalty applied by google and have disappeared from the search rankings down to the last page. The conclusion we have come to is that they have been flagged for...
by Ferhan Johar 4 years ago
Submitting article to ezine articles,from article that we already posted in our blog,can we do this?Let say i have a blog, and i create unique content, each day, then i want to create backlinks to my blog, i do this by submitting my article that i have already posted on my blog to any of the...
by Ceres Schwarz 5 years ago
I have a hub that I have yet to publish. I've been working on it and then this warning suddenly shows up "WARNING [?]NOTE: A substantial portion of the content on this hub appears on other pages". How do I get rid of this warning? And how can I find exactly which part of this hub has the...
by Jessica 5 years ago
I found out this morning that this site had stolen at least 2 of my hubs. The more I looked, the more I noticed that it appears every single post on the site is a stolen hub, completely copied in most cases (including the authors' names and copyright notes, ironically). I see lots of names of...
by Anjali 7 months ago
I just published my content which was written by me (I did not copy from anywhere, Only did the content based on our sales data), but I got email saying it is Duplicate.How will HubPages determine it as Duplicate Content?
Copyright © 2018 HubPages Inc. and respective owners. Other product and company names shown may be trademarks of their respective owners. HubPages® is a registered Service Mark of HubPages, Inc. HubPages and Hubbers (authors) may earn revenue on this page based on affiliate relationships and advertisements with partners including Amazon, Google, and others.
|HubPages Device ID||This is used to identify particular browsers or devices when the access the service, and is used for security reasons.|
|Login||This is necessary to sign in to the HubPages Service.|
|HubPages Traffic Pixel||This is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.|
|Remarketing Pixels||We may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.|
|Conversion Tracking Pixels||We may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.|