Some serious flaws with Quality Assessment Process

1. 0

   Giselle Maineposted 11 years ago

   Yesterday I went into edit mode on one of my non-featured (idled) hubs but DID NOT make a single change.  (The reason I went in there was because I just wanted to look at my summary, which I can't do without going into edit mode).  I did not do any typing whatsoever.  To my surprise, my hub became featured again.  I think that this sort of experience illustrates that there are some serious problems with the QAP, since NOTHING about the content of the hub was changed.
   
   I've also had problems where I've gone into one of my featured hubs to update a broken link, nothing else.  The hub then became non-featured. 
   
   These sorts of experiences, I believe, show that the QAP is non-robust.  A more robust method should not so easily give variable results.  I know of one other hubber that has had something similar happen (a response to one of my previous forum posts).  Has anyone else had this sort of thing happen to them? And are you worried about the QAP too, or not?

   replyreport
   1. 58

      MelissaBarrettposted 11 years agoin reply to this

      My personal guess would be that the hub was borderline. 
      
      There's a human element involved in all ratings.  There is no way to remove it.  If a hub is borderline the human aspect comes into play.
      
      *Shrugs*
      
      If you had any suggestions about how to change QAP to make it better now would be the time to make them.

      replyreport
      1. 0

         Giselle Maineposted 11 years agoin reply to this

         Yes thanks, that explains some of it.  I think it would help the QAP if the following changes were made:
         1.  If a hub passes the QAP, that it is "guaranteed" not to become idled again unless an edit triggers a new QAP. 
         2.  I don't know if this is possible, but if during an edit no changes were made to the body of the text, that this edit does NOT trigger a fresh QAP. 
         
         Those would be my suggestions, for what it's worth.  I believe that would not only keep things straightforward for hubbers, but it would also mean less HP resources go into repeatedly doing a QAP when nothing has changed about the hub.

         replyreport
         1. 58

            MelissaBarrettposted 11 years agoin reply to this

            Number two sounds great.
            
            Number one kinda defeats the purpose of the "engagement" portion of the idling.  I believe that one of the purposes of editing an idled hub is to 1. add more engaging material and 2. add freshness-which google seems to like-as much as it can like anything.
            
            Basically to get the hub more traffic... which if it's been idled for engagement purposes it obviously needs.

            replyreport
         2. 88

            wildernessposted 11 years agoin reply to this

            I wonder what would have happened had you opened the hub, edited it (with no changes) and closed the window.  Never touched the "publish" button.

            replyreport
2. 100

   DrMark1961posted 11 years ago

   Marisa Wright commented that she made some minor, "cosmetic" changes to some of ther idle hubs and a short time later they became idle again. I wonder if the same thing is going to happen to this hub that you did not change.
   Like Melissa commented, it may be a borderline hub and may be okay now, and idled again the next time it is looked at.
   I dont think your second suggestion would work. Anytime a hub is "edited", even if you just open the edit window and click on the key to indicate you are finished editing, it will have to be reviewed again. (QAP). Interesting proposal though.

   replyreport
3. 60

   LindaSmith1posted 11 years ago

   I have done same thing. Click edit, do nothing, hit done editing, and it is back on board again.

   replyreport
4. 82

   Simone Smithposted 11 years ago

   Thanks for sharing your insights, Giselle Maine (and everyone).
   
   The QAP is very much a work in progress still- we're still refining it!
   
   We definitely want to factor in engagement with deciding which Hubs to Feature over time, as that seems to be something that can impact the overall success of the community.
   
   It is hard for us to set a minimum threshold for change (instead of just using an edit action) to re-ignite the Quality Assessment Process because of instances in which mostly undetectable changes (e.g. the replacement of one horribly pixelated image with a good one) can still make the difference between a borderline Hub being Featured or let alone.
   
   That said, we're definitely trying to make both quality and engagement aspects of the QAP more transparent and easy to understad. It just takes a while to get everything right!

   replyreport
   1. 86

      Millionaire Tipsposted 11 years agoin reply to this

      One thing that might help is a "cancel" button.  Right now the only way to get out of the edit mode is to click "done editing" which implies that you changed something.  A cancel button would not save changes, and the hub wouldn't have to go through the (expensive) Mturk process again.

      replyreport
      1. 88

         wildernessposted 11 years agoin reply to this

         What happens if you simply close the browser window?

         replyreport
   2. 80

      Marcy Goodfleischposted 11 years agoin reply to this

      Simone - it would really (really) be helpful for us to know the level of quality the filtering system looks for  in a hub.  What are the elements that will let a hub be featured, and what will hold it back? 
      
      Without having these guidelines publicly known, it is like trying to pass a college course without knowing the grade criteria, or having an evaluation at work without having goals and objectives on which you are rated.
      
      Certainly, you would not want writers to 'game the system' by publishing junk that happens to have all the criteria for being featured, but is poorly written.  That's subjective.  The other factors, though, are objective.  We can produce work that meets those goals the first time around if we know what they are.
      
      So far, I haven't had a hub idled longer than the initial publication person (unless it was an old hub, that had everything done right but got idled for reasons I will never know).  But it is very frustrating for those who do get gigged and could have prevented it.
      
      It's also frustrating for those of us who have to tweak and re-tweak older hubs that are solid, but get idled.  We need to know the factors that cause hubs to be idled.

      replyreport
      1. 86

         Marisa Wrightposted 11 years agoin reply to this

         Marcy, not sure if you missed Derek's post.  He stated very clearly that if an existing Hub gets idled, then you can be confident you know why - because the only reason an old, un-edited Hub can be idled is lack of traffic.
         
         Hubs only go through the QAP if they're new, or if they're edited.
         
         http://hubpages.com/forum/topic/108768? … ost2315660

         replyreport
         1. 80

            Marcy Goodfleischposted 11 years agoin reply to this

            Marisa - you're right, I did miss that post. I had looked for it, but didn't find it (I could see people referring to it). Thanks for guiding me to it. One thing that might help is to know the level of traffic that kicks a hub into idle status, because writers could then spot hubs that are at risk and do some promoting or editing.

            replyreport
   3. 0

      Giselle Maineposted 11 years agoin reply to this

      Simone, thanks so much for the explanation.  I see what you mean about the difficulties of setting a minimum threshold for change.  I especially appreciate you shedding some light on the matter in general in this forum.  I think others in a similar situation to myself will also find your post extremely helpful.
      
      I also liked what Marcy had to say about making the QAP a little more transparent (something that Simone had also touched on).  It would certainly be a lot more encouraging for hubbers to have a bit better idea of what they are aiming for when they are publishing a new hub or editing an old one. 
      
      As an example of where it can get confusing, I had a hub get idled even though it ranked on the first page of Google results for a 3-word search term (a logical-sounding term, not just 3 unrelated words).  I concluded that the idling in this particular instance must have been due to something about other aspects of the writing than the SEO-friendliness of it, so it did give me a clue at what to look.  Yet for those of us such as myself with more than just a few hubs, I don't have time to go through that processs and make all these sorts of detective-work guesses for ALL my idled hubs - it would help to have more specific criteria at which to aim at the beginning.  Anyway, I just wanted to explain the thought processes of what happens when a hub gets idled.
      
      I do want to re-iterate that I'm grateful to Simone for helping shed some light on the matter in general, and I agree with Marcy that some specific criteria at which to aim would be of help to everyone. It sounds from what Simone said that HP also is planning for the QAP to become a bit clearer to hubbers so hopefully this is something that we can expect to hear more about in the future.

      replyreport
      1. 86

         Marisa Wrightposted 11 years agoin reply to this

         As I just pointed out to Marcy - there is only one reason why an existing, un-edited Hub can be idled.  It's lack of traffic.    Only new and edited Hubs are sent through the QAP. 
         
         http://hubpages.com/forum/topic/108768? … ost2315660

         replyreport
5. 58

   vinayak1000posted 11 years ago

   Interesting. Thanks for sharing!

   replyreport
6. 52

   janderson99posted 11 years ago

   See these suggestions on another thread
   
   http://hubpages.com/forum/post/2317081

   replyreport
   1. 0

      Giselle Maineposted 11 years agoin reply to this

      Hi Marisa and Janderson99, I checked out the links you each provided and am familiarizing myself with these other discussions on the topic. Janderson, I liked the ideas that you and others put forward on the other thread.  Marisa, thanks for the helpful update on the fact that low traffic is the reason why existing non-edited hubs can be idled, and for making the distinction that they haven't been through the QAP (something I wasn't previously aware of - very helpful thanks).  However, I'm curious as to why low traffic need be the criteria for idling an existing hub? As Janderson (I think) mentioned in the other thread, why not have Hubscore of the particular hub be the criteria instead? Otherwise, as has already been mentioned in the other thread, we have a Hubscore for each hub - which isn't even being used in determining whether the hub gets idled.  Seems a bit pointless to have the Hubscores then if they do not factor into whether or not a hub gets idled.
      
      I hope that in what I have mentioned that I have understood what is being said, but please do not hesitate to correct me if I have gotten confused somewhere.  I am trying to come to grips with understanding this.

      replyreport
      1. 52

         janderson99posted 11 years agoin reply to this

         HP won't idle a hub that's generating traffic - that's shooting itself in the foot.
         HP is not churning through OLD hub and giving them QAP scores - that's too expensive. So the tactic is to idle all hubs that get low traffic. If they are edited they get put through a QAP process, but its a lot less stringent than for new hubs.
         What this means is that HP is not improving the overall quality of hubs on the site - but is only filtering out new hubs with low quality, and hubs that get no traffic and are not edited (it assumes that hubs that get no traffic must be bad or have some fault because Google gives them no engagement). Lots of low quality hubs (low QAP scores) remain on the site because they get traffic. Lots of fabulous high quality hubs are being de-indexed or being removed to other sites, because they get low traffic. Will G downgrade the ranking of HP because of this? That is the big unknown. There has been little sign of improvement in overall traffic for HP in the 5 months since hubs began to be idled (sept 2012).
         
         
         
         
         
         

         replyreport
      2. 86

         Marisa Wrightposted 11 years agoin reply to this

         If you read the post I linked to, I asked Derek exactly the same question - because it made no sense to me. 
         
         Read Derek's reply.   Basically, he's saying that it would cost too much time and money to assess every single Hub on the site, so they've taken the view that idling Hubs that have low traffic is likely to remove a lot of low quality stuff that Google doesn't like. 
         
         If you read this forum:
         http://hubpages.com/forum/topic/107466? … ost2317470
         
         you'll see that HubPages knows that HubScore is meaningless as a measure, so they wouldn't want to use it to make the decision.  The only reason they haven't replaced it with something else, is that they have other priorities right now.

         replyreport
         1. 52

            janderson99posted 11 years agoin reply to this

            The point I am making is that HP's aim to lift the the overall quality of articles on HP to boost Google ranking and avoid a Panda penalty is hampered by its inability to eliminate old poor quality pages from the site, because it will only find those that have low traffic.
            Meanwhile heaps of fabulous hubs that get little traffic are being idled and moved to other sites. Morale is being destroyed by the idling of quality hubs (AP, HOTD, Stellar, Exclusives etc). No one knows what to fix, why the hub was idled.
            I think a change of approach is required to lift the quality of ALL hubs on HP, lift author morale, for the benefit of ALL the authors, with an increase in traffic.
            
            Perhaps its time to abandon human ratings and shift to an algorithm approach with a spell checker, grammar checker, word counts, photo counts, capsule diversity, etc. HP already has something like this
            It may be rough but it could be run for ALL the hubs targeting the low quality stuff - whether hubs are good, very good or excellent does not matter as much - as HP already applies the test "IF Google gives it Traffic it must be OK." High QAP does not correlate with high traffic
            The idling criteria would then be:
            => LOW quality - show what needs to be fixed via revived hubscore. For low quality hubs that are getting traffic, don't idle them but send an email requesting improvement so they will get more traffic and have better quality. For marginal ones, loss of a few hits for a 'bad' hub may be worth it
            => LOW traffic, Quality OK - Author advised to change the title, get more links etc. to get more traffic.
            => Low Quality HIGH Quality hubs - Allow more High Quality hubs to be retained and featured on the site, despite low traffic. Lower the standards, so more are retained, and advise authors to work harder to get traffic to these hubs. Don't assume that all hubs that get low traffic have something wrong with them because Google sends them no traffic - it is probably more to do with a non-competitive title, links, Pinterest images etc.
            
            If the overall quality of hubs on the entire site could be improved, I think traffic would improve.

            replyreport
            1. 93

               meloncauliposted 11 years agoin reply to this

               I absolutely agree with you!
               
               "Perhaps its time to abandon human ratings and shift to an algorithm approach with a spell checker, grammar checker, word counts, photo counts, capsule diversity, etc. HP already has something like this"
               
               If this is the case, how come so many newly registered hubbers can post a first hub that is of extremely poor quality? I regularly see new hubbers writing a single paragraph, often with spelling mistakes etc. They are also often devoid of any other capsules and photos. I see titles without capitalization too. I saw one article recently that was about 800 words long and it didn't contain one single paragraph!
               
               This is what I mean...how are they able to get those on to HP in the first place? How and why has the pending process enabled them to be published? This is where new and much stricter criteria is needed.

               replyreport
               1. 80

                  Marcy Goodfleischposted 11 years agoin reply to this

                  I see a problem with using only an algorithm - I think they don't always find the bad writing, or the problem content.  But as staff has said here, the cost of human filtering would be huge.
                  
                  Many people have volunteered to help do a clean-up. Maybe that is such a huge task that it's feasible, but if there would be a way to divide the work, so many of us truly care and see this as a community, there may be some willing participants. Many of us have been editors in our careers.

                  replyreport
         2. 0

            Giselle Maineposted 11 years agoin reply to this

            Marisa, thanks - yes I knew from the link you provided previously on this page that old hubs were being idled based on traffic for reasons of economy, as opposed to passing them through QAP.  The Hubscore info you put was extremely helpful and this was news to me (i.e. the reason why they can't just use Hubscore as a measure of quality) - thanks.  I still find it odd that they can't make or change Hubscore into a useful measure of quality, instead of relying only on traffic for idling the older non-edited hubs. 
            
            I personally still don't understand why we need a QAP at all. Why can't we just go with what Janderson suggested and have an automated spell checker and grammar checker, and word counts etc - surely that would manage to eliminate low quality hubs very quickly and with much less resources than the QAP requires.
            
            I agree with what Janderson said that the QAP is taking morale down.  By which I am not saying that my writing is perfect and that I'm miserable at being told it isn't.  Instead when a hub of mine is idled I understand that there's something wrong (probably traffic-related since I haven't published any new since the QAP arrived), but I find that I have to 'guess' what to change on the idled hub, and when I do, there is very little incentive to put it on Hubpages again.  This is because I feel like it is uncertain whether or not it will pass the QAP, so it's more tempting to want to put it elsewhere on the web.

            replyreport
7. 86

   ajwrites57posted 11 years ago

   Do the Panda and/or Penguin figure into any of these changes?

   replyreport
8. 76

   Robbie C Wilsonposted 11 years ago

   I agree that the system of idling hubs is flawed and needs revision.
   
   I think that hubs should be given more than one month to generate traffic and that low traffic should not automatically cause hubs to be idled.
   
   One of my favourite hubs complete with excellent photos and content got idled because of traffic (it is rather niche). What do I do now to get that hub back on Google for a decent period of time?

   replyreport
   1. 86

      Marisa Wrightposted 11 years agoin reply to this

      Robbie, if it's one of those topics that gets very low search volumes, then the only option is to move it elsewhere - because the only long-term solution is to get traffic, and that's not easy on a topic which doesn't get searched much.   
      
      You can get it back to Featured by tweaking it, but that only works if you can drive some traffic to it while it's in the Featured state - otherwise it will just idle again, and you'll have to keep an eagle eye on it.  I don't think that's worth it for that level of earnings potential.
      
      As Bac2basics says, it's still worth doing something with it because a low, steady trickle of income is still worth having.   You could try moving it to Wizzley, ThisIsFreelance, PubWages or Xobba, none of which have any kind of idling process.

      replyreport
9. 85

   bac2basicsposted 11 years ago

   Hi Robbie. I agree with you. Even if hubs are Niche, someone is going to read them at some stage and in my view even 10 hits a year is better than nothing. I have spent a day this weekend looking at idled hubs and making changes, it doesn´t take much to get them un idled, but as you say may be just for a short time. Many comment´s have told me to unpublish these hubs and place them on other sites.

   replyreport
10. 91

    Sue Adamsposted 11 years ago

    I have a few featured hubs that I would like to tweak but I dare not push the "publish" button because if I do so, they will become idle.  Many of us haven't published a new hub since September when QAP was introduced. Now how does that help anything!?
    
    There are now 4 forum threads (maybe more?) all on the same time consuming issues dealing with Some serious flaws with Quality Assessment Process.
    
    posted 5 weeks ago by Marisa Retire Hubber Score (and Hub Scores?)
    
    posted 10 days ago by janderson99 Suggestions for Changes to Hubscore and Hubberscore
    
    posted 4 days ago by Melovy Idling and pending hubs are killing traffic
    
    posted yesterday 27 Jan by Giselle Main Some serious flaws with Quality Assessment Process
    
    To Summarize
    The majority of authors on all 4 forum threads ask for...
    A re-working of the criteria for scoring to show more Transparency and Correlation between *QAP Scores and Hub Scores.
    
    I believe that Janderson is leading in the right direction with some positive suggestions.
    
    BTW, this thread should be moved to "Technical Problems on HubPages".

    replyreport
    1. 80

       Marcy Goodfleischposted 11 years agoin reply to this

       Sue - lately I have noticed that when I tweak a hub that's a,ready featured, it doesn't start the arrows spinning again. I don't know if that's a fluke, or a new way they've adjusted it. As with you, I have dreaded editing featured hubs because of the subsequent idle time.

       replyreport
       1. 81

          SimeyCposted 11 years agoin reply to this

          Once a hub is featured you can edit without changing the status...it's been like that for a while now!

          replyreport
    2. 86

       ajwrites57posted 11 years agoin reply to this

       How very discouraging--I've been writing on Hubpages for a month now and have 15 Hubs. You're saying it might not be worth it now? What a disappointment..................

       replyreport
       1. 76

          Robbie C Wilsonposted 11 years agoin reply to this

          It is worth it don't worry. Your hubs have to get next to no traffic to be idled. Some of my hubs got idled, but I keep going and believe me it is worth it. My traffic has gone up by 30 times from my first month to now. Just keep writing articles and the traffic will come and then the money will come too.
          
          At some point I will go back and "rescue" my idled hubs, but at the moment I am too busy writing new ones.
          
          As many people have already said, the Hub Pages community is wonderful and this is a great site.

          replyreport
11. 60

    LindaSmith1posted 11 years ago

    We can flag hubs, but what is the point? Flag them, yet they remain active.
    
    Expense?  Well here is another way for HP to look at it.  A lot of good writers have stopped publishing here altogether.  Others are packing up their hubs and moving them elsewhere.
    
    How expensive will it be, when nothing but junk is left on HP?

    replyreport
12. 91

    Sue Adamsposted 11 years ago

    No AJ, we don't want you to go. HubPages is the best, easiest and friendliest place to publish for money. It'll take you a while anyway to fully come to grips with all the wonderful features on HP. Just keep writing while this QAP business is resolved. New Hubbers are less affected than old ones when HP decides to change its system. And , in the long run, changes usually prove to be for the best. 

    replyreport
    1. 86

       ajwrites57posted 11 years agoin reply to this

       Lol--Thanks so much--just was reading your Hub about the 10 Skills for handsome men. I won't go, not with great Hubbers like you to encourage me.  Seriously, though, I hope these problems can be solved. Thanks to you and all the others for the work and suggestions to improve this site.Thanks again!

       replyreport
    2. 80

       Marcy Goodfleischposted 11 years agoin reply to this

       +1 on Sue's  request for you to stay!  Yes, we are all submitting constructive comments here, but we are all still here!  You won't find a better staff or community anywhere else on the Net.

       replyreport
       1. 61

          cascolyposted 11 years agoin reply to this

          what's needed is a different approach -- eliminate hubscores, and stop the idling process
          
          instead there needs to be a 2 step process
          
          no new hub should be published unless it can meet minimal grammatical, etc technical standards [exeptions for literary postings?]  treat it just like current reqs for minimal length, overly promotiomal, etc
          
          second, tougher is to address older hubs - gradually work thru the dormant hubs - sorted by last visited or last hubscore - alert those hubbers their hub is about to be retired

          replyreport