Some serious flaws with Quality Assessment Process

Jump to Last Post 1-12 of 12 discussions (38 posts)
  1. profile image0
    Giselle Maineposted 11 years ago

    Yesterday I went into edit mode on one of my non-featured (idled) hubs but DID NOT make a single change.  (The reason I went in there was because I just wanted to look at my summary, which I can't do without going into edit mode).  I did not do any typing whatsoever.  To my surprise, my hub became featured again.  I think that this sort of experience illustrates that there are some serious problems with the QAP, since NOTHING about the content of the hub was changed.

    I've also had problems where I've gone into one of my featured hubs to update a broken link, nothing else.  The hub then became non-featured. 

    These sorts of experiences, I believe, show that the QAP is non-robust.  A more robust method should not so easily give variable results.  I know of one other hubber that has had something similar happen (a response to one of my previous forum posts).  Has anyone else had this sort of thing happen to them? And are you worried about the QAP too, or not?

    1. MelissaBarrett profile image59
      MelissaBarrettposted 11 years agoin reply to this

      My personal guess would be that the hub was borderline. 

      There's a human element involved in all ratings.  There is no way to remove it.  If a hub is borderline the human aspect comes into play.


      If you had any suggestions about how to change QAP to make it better now would be the time to make them.

      1. profile image0
        Giselle Maineposted 11 years agoin reply to this

        Yes thanks, that explains some of it.  I think it would help the QAP if the following changes were made:
        1.  If a hub passes the QAP, that it is "guaranteed" not to become idled again unless an edit triggers a new QAP. 
        2.  I don't know if this is possible, but if during an edit no changes were made to the body of the text, that this edit does NOT trigger a fresh QAP. 

        Those would be my suggestions, for what it's worth.  I believe that would not only keep things straightforward for hubbers, but it would also mean less HP resources go into repeatedly doing a QAP when nothing has changed about the hub.

        1. MelissaBarrett profile image59
          MelissaBarrettposted 11 years agoin reply to this

          Number two sounds great.

          Number one kinda defeats the purpose of the "engagement" portion of the idling.  I believe that one of the purposes of editing an idled hub is to 1. add more engaging material and 2. add freshness-which google seems to like-as much as it can like anything.

          Basically to get the hub more traffic... which if it's been idled for engagement purposes it obviously needs.

        2. wilderness profile image95
          wildernessposted 11 years agoin reply to this

          I wonder what would have happened had you opened the hub, edited it (with no changes) and closed the window.  Never touched the "publish" button.

  2. DrMark1961 profile image96
    DrMark1961posted 11 years ago

    Marisa Wright commented that she made some minor, "cosmetic" changes to some of ther idle hubs and a short time later they became idle again. I wonder if the same thing is going to happen to this hub that you did not change.
    Like Melissa commented, it may be a borderline hub and may be okay now, and idled again the next time it is looked at.
    I dont think your second suggestion would work. Anytime a hub is "edited", even if you just open the edit window and click on the key to indicate you are finished editing, it will have to be reviewed again. (QAP). Interesting proposal though.

  3. LindaSmith1 profile image61
    LindaSmith1posted 11 years ago

    I have done same thing. Click edit, do nothing, hit done editing, and it is back on board again.

  4. Simone Smith profile image85
    Simone Smithposted 11 years ago

    Thanks for sharing your insights, Giselle Maine (and everyone).

    The QAP is very much a work in progress still- we're still refining it!

    We definitely want to factor in engagement with deciding which Hubs to Feature over time, as that seems to be something that can impact the overall success of the community.

    It is hard for us to set a minimum threshold for change (instead of just using an edit action) to re-ignite the Quality Assessment Process because of instances in which mostly undetectable changes (e.g. the replacement of one horribly pixelated image with a good one) can still make the difference between a borderline Hub being Featured or let alone.

    That said, we're definitely trying to make both quality and engagement aspects of the QAP more transparent and easy to understad. It just takes a while to get everything right! big_smile

    1. Millionaire Tips profile image91
      Millionaire Tipsposted 11 years agoin reply to this

      One thing that might help is a "cancel" button.  Right now the only way to get out of the edit mode is to click "done editing" which implies that you changed something.  A cancel button would not save changes, and the hub wouldn't have to go through the (expensive) Mturk process again.

      1. wilderness profile image95
        wildernessposted 11 years agoin reply to this

        What happens if you simply close the browser window?

    2. Marcy Goodfleisch profile image83
      Marcy Goodfleischposted 11 years agoin reply to this

      Simone - it would really (really) be helpful for us to know the level of quality the filtering system looks for  in a hub.  What are the elements that will let a hub be featured, and what will hold it back? 

      Without having these guidelines publicly known, it is like trying to pass a college course without knowing the grade criteria, or having an evaluation at work without having goals and objectives on which you are rated.

      Certainly, you would not want writers to 'game the system' by publishing junk that happens to have all the criteria for being featured, but is poorly written.  That's subjective.  The other factors, though, are objective.  We can produce work that meets those goals the first time around if we know what they are.

      So far, I haven't had a hub idled longer than the initial publication person (unless it was an old hub, that had everything done right but got idled for reasons I will never know).  But it is very frustrating for those who do get gigged and could have prevented it.

      It's also frustrating for those of us who have to tweak and re-tweak older hubs that are solid, but get idled.  We need to know the factors that cause hubs to be idled.

      1. Marisa Wright profile image86
        Marisa Wrightposted 11 years agoin reply to this

        Marcy, not sure if you missed Derek's post.  He stated very clearly that if an existing Hub gets idled, then you can be confident you know why - because the only reason an old, un-edited Hub can be idled is lack of traffic.

        Hubs only go through the QAP if they're new, or if they're edited. … ost2315660

        1. Marcy Goodfleisch profile image83
          Marcy Goodfleischposted 11 years agoin reply to this

          Marisa - you're right, I did miss that post. I had looked for it, but didn't find it (I could see people referring to it). Thanks for guiding me to it. One thing that might help is to know the level of traffic that kicks a hub into idle status, because writers could then spot hubs that are at risk and do some promoting or editing.

    3. profile image0
      Giselle Maineposted 11 years agoin reply to this

      Simone, thanks so much for the explanation.  I see what you mean about the difficulties of setting a minimum threshold for change.  I especially appreciate you shedding some light on the matter in general in this forum.  I think others in a similar situation to myself will also find your post extremely helpful.

      I also liked what Marcy had to say about making the QAP a little more transparent (something that Simone had also touched on).  It would certainly be a lot more encouraging for hubbers to have a bit better idea of what they are aiming for when they are publishing a new hub or editing an old one. 

      As an example of where it can get confusing, I had a hub get idled even though it ranked on the first page of Google results for a 3-word search term (a logical-sounding term, not just 3 unrelated words).  I concluded that the idling in this particular instance must have been due to something about other aspects of the writing than the SEO-friendliness of it, so it did give me a clue at what to look.  Yet for those of us such as myself with more than just a few hubs, I don't have time to go through that processs and make all these sorts of detective-work guesses for ALL my idled hubs - it would help to have more specific criteria at which to aim at the beginning.  Anyway, I just wanted to explain the thought processes of what happens when a hub gets idled.

      I do want to re-iterate that I'm grateful to Simone for helping shed some light on the matter in general, and I agree with Marcy that some specific criteria at which to aim would be of help to everyone. It sounds from what Simone said that HP also is planning for the QAP to become a bit clearer to hubbers so hopefully this is something that we can expect to hear more about in the future.

      1. Marisa Wright profile image86
        Marisa Wrightposted 11 years agoin reply to this

        As I just pointed out to Marcy - there is only one reason why an existing, un-edited Hub can be idled.  It's lack of traffic.    Only new and edited Hubs are sent through the QAP.  … ost2315660

  5. vinayak1000 profile image67
    vinayak1000posted 11 years ago

    Interesting. Thanks for sharing!

  6. janderson99 profile image52
    janderson99posted 11 years ago

    See these suggestions on another thread

    1. profile image0
      Giselle Maineposted 11 years agoin reply to this

      Hi Marisa and Janderson99, I checked out the links you each provided and am familiarizing myself with these other discussions on the topic. Janderson, I liked the ideas that you and others put forward on the other thread.  Marisa, thanks for the helpful update on the fact that low traffic is the reason why existing non-edited hubs can be idled, and for making the distinction that they haven't been through the QAP (something I wasn't previously aware of - very helpful thanks).  However, I'm curious as to why low traffic need be the criteria for idling an existing hub? As Janderson (I think) mentioned in the other thread, why not have Hubscore of the particular hub be the criteria instead? Otherwise, as has already been mentioned in the other thread, we have a Hubscore for each hub - which isn't even being used in determining whether the hub gets idled.  Seems a bit pointless to have the Hubscores then if they do not factor into whether or not a hub gets idled.

      I hope that in what I have mentioned that I have understood what is being said, but please do not hesitate to correct me if I have gotten confused somewhere.  I am trying to come to grips with understanding this.

      1. janderson99 profile image52
        janderson99posted 11 years agoin reply to this

        HP won't idle a hub that's generating traffic - that's shooting itself in the foot.
        HP is not churning through OLD hub and giving them QAP scores - that's too expensive. So the tactic is to idle all hubs that get low traffic. If they are edited they get put through a QAP process, but its a lot less stringent than for new hubs.
        What this means is that HP is not improving the overall quality of hubs on the site - but is only filtering out new hubs with low quality, and hubs that get no traffic and are not edited (it assumes that hubs that get no traffic must be bad or have some fault because Google gives them no engagement). Lots of low quality hubs (low QAP scores) remain on the site because they get traffic. Lots of fabulous high quality hubs are being de-indexed or being removed to other sites, because they get low traffic. Will G downgrade the ranking of HP because of this? That is the big unknown. There has been little sign of improvement in overall traffic for HP in the 5 months since hubs began to be idled (sept 2012).

      2. Marisa Wright profile image86
        Marisa Wrightposted 11 years agoin reply to this

        If you read the post I linked to, I asked Derek exactly the same question - because it made no sense to me. 

        Read Derek's reply.   Basically, he's saying that it would cost too much time and money to assess every single Hub on the site, so they've taken the view that idling Hubs that have low traffic is likely to remove a lot of low quality stuff that Google doesn't like. 

        If you read this forum: … ost2317470

        you'll see that HubPages knows that HubScore is meaningless as a measure, so they wouldn't want to use it to make the decision.  The only reason they haven't replaced it with something else, is that they have other priorities right now.

        1. janderson99 profile image52
          janderson99posted 11 years agoin reply to this

          The point I am making is that HP's aim to lift the the overall quality of articles on HP to boost Google ranking and avoid a Panda penalty is hampered by its inability to eliminate old poor quality pages from the site, because it will only find those that have low traffic.
          Meanwhile heaps of fabulous hubs that get little traffic are being idled and moved to other sites. Morale is being destroyed by the idling of quality hubs (AP, HOTD, Stellar, Exclusives etc). No one knows what to fix, why the hub was idled.
          I think a change of approach is required to lift the quality of ALL hubs on HP, lift author morale, for the benefit of ALL the authors, with an increase in traffic.

          Perhaps its time to abandon human ratings and shift to an algorithm approach with a spell checker, grammar checker, word counts, photo counts, capsule diversity, etc. HP already has something like this
          It may be rough but it could be run for ALL the hubs targeting the low quality stuff - whether hubs are good, very good or excellent does not matter as much - as HP already applies the test "IF Google gives it Traffic it must be OK." High QAP does not correlate with high traffic
          The idling criteria would then be:
          => LOW quality - show what needs to be fixed via revived hubscore. For low quality hubs that are getting traffic, don't idle them but send an email requesting improvement so they will get more traffic and have better quality. For marginal ones, loss of a few hits for a 'bad' hub may be worth it
          => LOW traffic, Quality OK - Author advised to change the title, get more links etc. to get more traffic.
          => Low Quality HIGH Quality hubs - Allow more High Quality hubs to be retained and featured on the site, despite low traffic. Lower the standards, so more are retained, and advise authors to work harder to get traffic to these hubs. Don't assume that all hubs that get low traffic have something wrong with them because Google sends them no traffic - it is probably more to do with a non-competitive title, links, Pinterest images etc.

          If the overall quality of hubs on the entire site could be improved, I think traffic would improve.

          1. meloncauli profile image90
            meloncauliposted 11 years agoin reply to this

            I absolutely agree with you!

            "Perhaps its time to abandon human ratings and shift to an algorithm approach with a spell checker, grammar checker, word counts, photo counts, capsule diversity, etc. HP already has something like this"

            If this is the case, how come so many newly registered hubbers can post a first hub that is of extremely poor quality? I regularly see new hubbers writing a single paragraph, often with spelling mistakes etc. They are also often devoid of any other capsules and photos. I see titles without capitalization too. I saw one article recently that was about 800 words long and it didn't contain one single paragraph!

            This is what I are they able to get those on to HP in the first place? How and why has the pending process enabled them to be published? This is where new and much stricter criteria is needed.

            1. Marcy Goodfleisch profile image83
              Marcy Goodfleischposted 11 years agoin reply to this

              I see a problem with using only an algorithm - I think they don't always find the bad writing, or the problem content.  But as staff has said here, the cost of human filtering would be huge.

              Many people have volunteered to help do a clean-up. Maybe that is such a huge task that it's feasible, but if there would be a way to divide the work, so many of us truly care and see this as a community, there may be some willing participants. Many of us have been editors in our careers.

        2. profile image0
          Giselle Maineposted 11 years agoin reply to this

          Marisa, thanks - yes I knew from the link you provided previously on this page that old hubs were being idled based on traffic for reasons of economy, as opposed to passing them through QAP.  The Hubscore info you put was extremely helpful and this was news to me (i.e. the reason why they can't just use Hubscore as a measure of quality) - thanks.  I still find it odd that they can't make or change Hubscore into a useful measure of quality, instead of relying only on traffic for idling the older non-edited hubs. 

          I personally still don't understand why we need a QAP at all. Why can't we just go with what Janderson suggested and have an automated spell checker and grammar checker, and word counts etc - surely that would manage to eliminate low quality hubs very quickly and with much less resources than the QAP requires.

          I agree with what Janderson said that the QAP is taking morale down.  By which I am not saying that my writing is perfect and that I'm miserable at being told it isn't.  Instead when a hub of mine is idled I understand that there's something wrong (probably traffic-related since I haven't published any new since the QAP arrived), but I find that I have to 'guess' what to change on the idled hub, and when I do, there is very little incentive to put it on Hubpages again.  This is because I feel like it is uncertain whether or not it will pass the QAP, so it's more tempting to want to put it elsewhere on the web.

  7. ajwrites57 profile image83
    ajwrites57posted 11 years ago

    Do the Panda and/or Penguin figure into any of these changes?

  8. profile image87
    Robbie C Wilsonposted 11 years ago

    I agree that the system of idling hubs is flawed and needs revision.

    I think that hubs should be given more than one month to generate traffic and that low traffic should not automatically cause hubs to be idled.

    One of my favourite hubs complete with excellent photos and content got idled because of traffic (it is rather niche). What do I do now to get that hub back on Google for a decent period of time?

    1. Marisa Wright profile image86
      Marisa Wrightposted 11 years agoin reply to this

      Robbie, if it's one of those topics that gets very low search volumes, then the only option is to move it elsewhere - because the only long-term solution is to get traffic, and that's not easy on a topic which doesn't get searched much.   

      You can get it back to Featured by tweaking it, but that only works if you can drive some traffic to it while it's in the Featured state - otherwise it will just idle again, and you'll have to keep an eagle eye on it.  I don't think that's worth it for that level of earnings potential.

      As Bac2basics says, it's still worth doing something with it because a low, steady trickle of income is still worth having.   You could try moving it to Wizzley, ThisIsFreelance, PubWages or Xobba, none of which have any kind of idling process.

  9. bac2basics profile image82
    bac2basicsposted 11 years ago

    Hi Robbie. I agree with you. Even if hubs are Niche, someone is going to read them at some stage and in my view even 10 hits a year is better than nothing. I have spent a day this weekend looking at idled hubs and making changes, it doesn´t take much to get them un idled, but as you say may be just for a short time. Many comment´s have told me to unpublish these hubs and place them on other sites.

  10. Sue Adams profile image91
    Sue Adamsposted 11 years ago

    I have a few featured hubs that I would like to tweak but I dare not push the "publish" button because if I do so, they will become idle.  Many of us haven't published a new hub since September when QAP was introduced. Now how does that help anything!?

    There are now 4 forum threads (maybe more?) all on the same time consuming issues dealing with Some serious flaws with Quality Assessment Process.

    posted 5 weeks ago by Marisa Retire Hubber Score (and Hub Scores?)

    posted 10 days ago by janderson99 Suggestions for Changes to Hubscore and Hubberscore

    posted 4 days ago by Melovy Idling and pending hubs are killing traffic

    posted yesterday 27 Jan by Giselle Main Some serious flaws with Quality Assessment Process

    To Summarize
    The majority of authors on all 4 forum threads ask for...
    A re-working of the criteria for scoring to show more Transparency and Correlation between *QAP Scores and Hub Scores.

    I believe that Janderson is leading in the right direction with some positive suggestions.

    BTW, this thread should be moved to "Technical Problems on HubPages".

    1. Marcy Goodfleisch profile image83
      Marcy Goodfleischposted 11 years agoin reply to this

      Sue - lately I have noticed that when I tweak a hub that's a,ready featured, it doesn't start the arrows spinning again. I don't know if that's a fluke, or a new way they've adjusted it. As with you, I have dreaded editing featured hubs because of the subsequent idle time.

      1. SimeyC profile image86
        SimeyCposted 11 years agoin reply to this

        Once a hub is featured you can edit without changing the's been like that for a while now!

    2. ajwrites57 profile image83
      ajwrites57posted 11 years agoin reply to this

      How very discouraging--I've been writing on Hubpages for a month now and have 15 Hubs. You're saying it might not be worth it now? What a disappointment..................

      1. profile image87
        Robbie C Wilsonposted 11 years agoin reply to this

        It is worth it don't worry. Your hubs have to get next to no traffic to be idled. Some of my hubs got idled, but I keep going and believe me it is worth it. My traffic has gone up by 30 times from my first month to now. Just keep writing articles and the traffic will come and then the money will come too.

        At some point I will go back and "rescue" my idled hubs, but at the moment I am too busy writing new ones.

        As many people have already said, the Hub Pages community is wonderful and this is a great site.

  11. LindaSmith1 profile image61
    LindaSmith1posted 11 years ago

    We can flag hubs, but what is the point? Flag them, yet they remain active.

    Expense?  Well here is another way for HP to look at it.  A lot of good writers have stopped publishing here altogether.  Others are packing up their hubs and moving them elsewhere.

    How expensive will it be, when nothing but junk is left on HP?

  12. Sue Adams profile image91
    Sue Adamsposted 11 years ago

    No AJ, we don't want you to go. HubPages is the best, easiest and friendliest place to publish for money. It'll take you a while anyway to fully come to grips with all the wonderful features on HP. Just keep writing while this QAP business is resolved. New Hubbers are less affected than old ones when HP decides to change its system. And , in the long run, changes usually prove to be for the best.  smile

    1. ajwrites57 profile image83
      ajwrites57posted 11 years agoin reply to this

      Lol--Thanks so much--just was reading your Hub about the 10 Skills for handsome men. I won't go, not with great Hubbers like you to encourage me.  Seriously, though, I hope these problems can be solved. Thanks to you and all the others for the work and suggestions to improve this site.Thanks again!

    2. Marcy Goodfleisch profile image83
      Marcy Goodfleischposted 11 years agoin reply to this

      +1 on Sue's  request for you to stay!  Yes, we are all submitting constructive comments here, but we are all still here!  You won't find a better staff or community anywhere else on the Net.

      1. cascoly profile image61
        cascolyposted 11 years agoin reply to this

        what's needed is a different approach -- eliminate hubscores, and stop the idling process

        instead there needs to be a 2 step process

        no new hub should be published unless it can meet minimal grammatical, etc technical standards [exeptions for literary postings?]  treat it just like current reqs for minimal length, overly promotiomal, etc

        second, tougher is to address older hubs - gradually work thru the dormant hubs - sorted by last visited or last hubscore - alert those hubbers their hub is about to be retired


This website uses cookies

As a user in the EEA, your approval is needed on a few things. To provide a better website experience, uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at:

Show Details
HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
LoginThis is necessary to sign in to the HubPages Service.
Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
AkismetThis is used to detect comment spam. (Privacy Policy)
HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the or domains, for performance and efficiency reasons. (Privacy Policy)
Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
MavenThis supports the Maven widget and search functionality. (Privacy Policy)
Google AdSenseThis is an ad network. (Privacy Policy)
Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
Index ExchangeThis is an ad network. (Privacy Policy)
SovrnThis is an ad network. (Privacy Policy)
Facebook AdsThis is an ad network. (Privacy Policy)
Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
AppNexusThis is an ad network. (Privacy Policy)
OpenxThis is an ad network. (Privacy Policy)
Rubicon ProjectThis is an ad network. (Privacy Policy)
TripleLiftThis is an ad network. (Privacy Policy)
Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)
ClickscoThis is a data management platform studying reader behavior (Privacy Policy)