Getting More Hubs Hopped

1. STAFF

   Paul Edmondsonposted 11 years ago

   We first launched the new Hub Hopper in early August http://blog.hubpages.com/2012/08/hub-ho … o-feature/
   
   We are looking to increase the amount of Hub Hopping and last night deployed an application to Mechanical Turk to find more people to Hop Hubs.  For those that are good at it, there is an opportunity to be compensated for Hopping.  For folks that are interested in trying it out, go to Mturk.com (you'll need an account) and search the HITS for HubPages.  You'll find an almost identical experience to Hub Hopping, but with a few more hoops.  Our goal is to find more people that can follow the scale and provide accurate ratings. 
   
   We use the Hub Hopping data as one data point in assessing quality on HubPages.  Our goal is to collect enough Hub Hopping data that it helps identify high quality Hubs. 
   
   Please let us know if you have any questions.

   report
   1. 75

      jacharlessposted 11 years agoin reply to this

      Hey Paul,
      I read through this thread and had a Turk stopover.
      Noticed you mentioned you have a team of people, on site, who hop. Obviously the goal is to reduce clutter of low quality, while sustaining//increasing user interaction, which, in my opinion, is brilliant. Writers screening articles. However, PPC for Hub Hopping makes me a tad leery...
      
      What if...
      HP selected//approached a group of Hubbers, like a Welcome Wagon or Nuggets group type of scenario, and asked them to hop say 50 Hubs in a week. If a pool of 130-1300 (1-10% of total Hubbers) did this, an average of 6,500 to 65,000 Hubs could be screened by members, weekly. The pool could be rotated or static.
      
      This encourages Hubber interaction, adds incentive, gives writers the opt-in to screen articles -whereby increasing their knowledge base for their own article structure- and lightening the load on the back end. A nice touch would be a "frog" icon on their profile or another accolade.
      
      I for one would participate in such an program, as am sure many others would. Critique by ones peers is essential and the best form of building camaraderie and honing skills.
      
      James.
      
      Hopper Accolade:
      This Hubber has hopped 10, 50, 100, 500, 1000, 10,000 hubs.
      
      Hub Hopper Icon
      

      report
      1. 68

         Rosie2010posted 11 years agoin reply to this

         +1 I support this idea.  I'm in!!!

         report
         1. 75

            jacharlessposted 11 years agoin reply to this

            Woo-hoo! That's two. Anyone, anyone...

            report
            1. 52

               janderson99posted 11 years agoin reply to this

               Afraid of burnout - De-qualification
               
               

               report
               1. 75

                  jacharlessposted 11 years agoin reply to this

                  poor fellow looks positively frightened.

                  report
                  1. 52

                     janderson99posted 11 years agoin reply to this

                     A few more suggestions
                     

                     report
            2. 86

               WannaB Writerposted 11 years agoin reply to this

               I'm in, too.

               report
      2. STAFF

         Paul Edmondsonposted 11 years agoin reply to this

         Their is a group of Hubbers that hop regularly and their data is evaluated the same.  It flows into the quality system weighted for accuracy.
         
         We discussed an accolade in the past, and I'll put this in our tracking system. 
         
         Thanks for all the feedback.

         report
      3. 95

         Daughter Of Maatposted 11 years agoin reply to this

         I suggested they add this to the curriculum for apprentices, but apparently that got ignored.
         
         I still say it would make more sense for the apprentices to participate in this type of program (as well as other hubbers of course) because they are currently learning about HP's standards and writing online. As an apprentice myself, I think adding this type of editorial task to the curriculum would be highly beneficial.
         
         But I guess no one else does.

         report
         1. 75

            jacharlessposted 11 years agoin reply to this

            Actually, yes, adding such a course requirement would certainly be beneficial to Apprentices, in many ways. The only downside//drawback would be full community interaction with this, as well. Example is someone like myself, who is ineligible for the Apprentice Program, due to a lack of Ad Sense, ergo a lack of Yield Build [Hub Premium], which is a requirement for application. On the upside, rotating community members to handle the task would really bridge the divide and increase their own editorial skills, resulting [hypothetically] in more quality content. I wonder why it was ignored. It is a very solid idea. Perhaps Apprentices have a lot on their plate [?].
            James.

            report
            1. 52

               janderson99posted 11 years agoin reply to this

               Apprentices are supposed to be learning how to do it - so how can they be judges of quality as well. I know some of the 'a's are very experienced. This is just stating the obvious. I guess there's the green 'a's. Maybe HP is tracking to become a 'AO' site!

               report
            2. 95

               Daughter Of Maatposted 11 years agoin reply to this

               Apprentices do have quite a few hubs to write as a requirement for the month, but the curriculum isn't so harsh that it wouldn't allow for us to do this.
               
               As for other hubbers, I agree. My suggestion was supposed to be in addition to other hubbers hopping. They need all the hoppers they can get from what I understand.
               
               @janderson99 While we are just "learning," we do have more of an insight to what HP wants since our lessons come straight from HP. We are taught exactly what HP wants when it comes to substance, organization and grammar and we get feedback on the hubs we write. I think we're much more qualified to hub hop than people on MTurk.
               
               And the test to qualify for hopping is not all that difficult.
               
               I don't mean to be negative Nancy, but I just think there are better options and that it should be an internal thing, not something that any joe shmoe can do.

               report
            3. 85

               DzyMsLizzyposted 11 years agoin reply to this

               I believe I probably would qualify for the apprentice program..I just cannot take it on right now.  It's too much of a time commitment with all my husband's health issues on my plate; that's in addition to normal running the household and tending to my (very small) ceramics business, right now.  I sometimes have to drop everything on a "right this minute" basis.
               So, I don't spend a lot of time hub-hopping anymore.  I used to do it more often before the new style hopper went in.  But I still agree with everyone who thinks outsourcing this task is a horrible idea.

               report
2. 77

   MyWebsposted 11 years ago

   I think your going to get a lot of users from Amazon's Mechanical Turk for whom English is a second language. So now they will be accessing the quality of our hubs? SMH
   
   If you was willing to pay people to hop why didn't you just recruit from within the community instead? I do understand that MTurk has some automation advantages and all. But it's not like you don't have programmers who could have created something from scratch on HP. Just one more in a lengthening line of bad choices in my opinion.

   report
3. STAFF

   Paul Edmondsonposted 11 years ago

   For the folks coming through Mturk, there are assessments and they continually get reassessed to see if their rating profile changes.  Access to the application is limited to folks in the US right now.

   report
   1. 77

      MyWebsposted 11 years agoin reply to this

      You did make a good choice by only allowing those from the US initially to insure they understand English well. But there is still a trade off to be had here. I don't know exactly what your paying Turkers to hub hop, but I'm sure it's not much. Many Americans won't wish to hop for a penny a hub or whatever it pays. Too time consuming for a small amount of money. Those who would be more willing would be those from other countries used to working for small amounts of money. Among many of these foreigners English isn't their primary language.
      
      While I'm sure you will get an increase in hopping your really stuck between a rock and a hard place here without an English assessment in place to first test foreigners before allowing them to take on your HITS.

      report
4. 74

   carol7777posted 11 years ago

   Could you explain hub hopping and how we find hubs to hop.  I know this may seem dumb but I really don't understand this. Sounds like something I might like to do.  Thanks.

   report
   1. 77

      MyWebsposted 11 years agoin reply to this

      Go to your Hubs Statistics page and click up top where it says Help us out, Hop some hubs.
      
      Hub hopping is something HP uses to try and judge how good, or bad, a hub is via a rating system built into the hub hopping feature.

      report
5. STAFF

   Paul Edmondsonposted 11 years ago

   Here is a link to the new Hub Hopper to try it if you haven't. A link to it can always be found in the sidebar on http://hubpages.com/feed
   
   http://hubpages.com/hop/

   report
6. 83

   psycheskinnerposted 11 years ago

   The funny thing is, if some people are paid to do it (albeit three and a half sense per thousand hubs, or whatever) I start wondering why I should do it for free.

   report
   1. 87

      Simone Smithposted 11 years agoin reply to this

      Because it can be a cool way to discover new Hubs and remind yourself of the own standards to which you might aspire to adhere (this is why I do it, at least).
      It reminds me of what I should be going for, and I think my Hubs are better as a result.

      report
      1. 52

         janderson99posted 11 years agoin reply to this

         Define 'quality' for me please. [ Not that gal Stella ]
         
         Some of my best quality hubs get "idled".
         It depends on what you are trying to achieve.
         Google already assesses quality, in many ways which governs SERP position and ranking.
         Users assess quality by clicking on the SERP links, staying longer, saying 'WOW', linking to it, and telling their friends about it.
         => The ultimate measure of quality is sustained TRAFFIC within the niche.
         So how is Mturk going to help? Beauties who yawn?

         report
         1. 87

            Simone Smithposted 11 years agoin reply to this

            What I was referring to was using the Hopper as a means of looking at general, human-centric quality standards and considering how to exceed them in one's own work.
            
            Clearly the Hopper isn't the best place to learn about what search engines determine to be great.

            report
7. STAFF

   Paul Edmondsonposted 11 years ago

   I took a screenshot of the listing in Mturk for folks that are curious what it looks like. 
   
   
   update: The image is hard to read, so here is the text from it.
   
   Requester:       HubPages
      
   HIT Expiration Date:       Oct 26, 2012  (1 day 5 hours)
   Time Allotted:       40 minutes
      
   Reward:       $0.05
   HITs Available:       115
   
   Description:     We award bonuses of up to $1.75 for every 10 HITs you complete, depending on your accuracy. You must complete 10 paid practice HITs before you will be eligible for this type of HIT. Search for "HubPages practice" to find our practice HITs. Our HITs involve reading an online magazine article and rating it on three scales: substance, organization, and grammar. It typically takes 1 to 2 minutes to rate an article. Although we do our best to remove inappropriate content from our user-generated articles, you might rarely encounter adult content.
   
   Keywords:     reading,  judging,  rating,  article,  evaluating,  writing,  grading,  HubPages
   Qualifications Required:     Your Value     
   Online article rating accuracy is greater than 172     188     You meet this qualification requirement
   Adult Content Qualification is 1     1     You meet this qualification requirement

   report
   1. 78

      Sally's Troveposted 11 years agoin reply to this

      There's no way to see this screen shot, not even with a 300% increase in size through my browser.

      report
   2. 85

      DzyMsLizzyposted 11 years agoin reply to this

      "Although we do our best to remove inappropriate content from our user-generated articles, you might rarely encounter adult content." 
      
      And your point?  We are adults, here.......
      I really hate censorship....
      I may not go out of my way to find so-called "adult content," but if I do, oh, well..big deal, so what?

      report
      1. STAFF

         Paul Edmondsonposted 11 years agoin reply to this

         That's a disclosure for MTurk workers.  Before you can work on HubPages HITS you need to certify that you are over 18 and may be exposed to adult content.

         report
8. 60

   Hollie Thomasposted 11 years ago

   I find it somewhat disconcerting that HP is willing to pay none HP writers to hub hop. Because firstly, people who work for so little money such as those who work for Mechanical Turk, are hardly concerned with quality, just the quantity of hubs which they can hop for money. Secondly, they may be US citizens but that does not necessarily mean that English is their first language, nor does it mean that they have an adequate grasp of the English language, even if English IS their first language. If they are working for Mechanical Turk, they are highly unlikely to be editors either, which is basically the task you are asking them to undertake. Thirdly, it's quite insulting that those who hub hop regularly have not been offered some form of payment for hub hopping, when it appears that HP is more than prepared to offer money to those outside of the HP community, for the same task.

   report
   1. 78

      Sally's Troveposted 11 years agoin reply to this

      +++++

      report
   2. 87

      Simone Smithposted 11 years agoin reply to this

      1, The app has means of assessing the quality and accuracy of ratings.
      2. See above answer.
      3. You're totally welcome to rate through that app! As I understand it, we found that it would be much harder for us to develop a proprietary means of paying people for hopping Hubs- we're just trying to be practical and efficient here, but would really appreciate ratings from real Hubbers.

      report
   3. 85

      DzyMsLizzyposted 11 years agoin reply to this

      Agreed 100%

      report
   4. 77

      MyWebsposted 11 years agoin reply to this

      "Because firstly, people who work for so little money such as those who work for Mechanical Turk, are hardly concerned with quality, just the quantity of hubs which they can hop for money."
      
      I have took on HITS in the past from Amazon's Mechanical Turk system so I speak from experience here. Almost all HITS pay so little that your trying to rush through the HITS as fast as possible to try and earn a bit of money without it taking forever while also doing a good job, or for many users the barely acceptable standard. I'm quite sure many users would be more concerned with the quantity of completed HITS than the quality of their ratings. Doing a good job means you make less money! I quite doing it because well it took way  too long, if your concerned about quality like I was, for what it paid 99% of the time.
      
      About the same time you posted this I was writing something similar above. I totally agree with you.

      report
      1. 95

         Daughter Of Maatposted 11 years agoin reply to this

         Wow, ya this is a really bad decision on HP's part. It would have made more sense to just put a team together and pay them for hopping so many hubs per day, or week, or whatever. MTurk is like Microjobs, you don't care about the quality of the work, you just want to get it done to get paid. I worked for microjobs and I have an MTurk account  and have never used for just that reason.
         
         Why don't you just add hopping as a task for the apprenticeship and increase the bonus for doing so many hops per day (or week or whatever)? It would make more sense since the apprentices are learning about writing online and what HP expects. They would be the best people for the job.

         report
         1. 85

            DzyMsLizzyposted 11 years agoin reply to this

            Agreed, DOM, I think it's a terrible idea, and I like your suggestions. 
            
            However, I hold out little hope...we've seen plenty of evidence recently that HP no longer cares about its writers, and makes changes willy-nilly, regardless of harm done to our income or traffic, only in the interest of its own bottom line.  In other words, losing sight of their initial objectives and roots,  just as we have seen with E-Bay, Etsy, Netflix, and countless other smaller start-ups that began with great ideas.  They sold out and are following the poor example of much larger corporations....

            report
9. 74

   carol7777posted 11 years ago

   How do you find hubpage hubs on mturk? How much does it pay?

   report
   1. STAFF

      Paul Edmondsonposted 11 years agoin reply to this

      Search under HITS for HubPages.  You'll need to take an assessment (it's paid).  The rate is 5 cents a Hub and bonuses for accuracy and volume are paid.
      
      We have a number of ways of going through content today from automatic filters, to moderators, to the HubPages community.  This is just another source where we can dial it up or down depending on the data we need.

      report
      1. 91

         Brainy Bunnyposted 11 years agoin reply to this

         I use Mturk from time to time, so I checked it out. I only got through 4 practice hubs before they were all used up, so I couldn't get the required 10 to start in for real. Can you release more practice ones?

         report
      2. 71

         galleryofgraceposted 11 years agoin reply to this

         Ok , so I should stop hopping here and go to mturk and get paid 5 cents. Ok thats better than nothing.

         report
10. 0

    DigbyAdamsposted 11 years ago

    Wow, I had never heard of this place and just went to visit. I can't imagine the situation of people who would be willing to work for so little money. If they are there because they need money, they will have to work very quickly to make any supper money at all. Can't believe that those results will be valid in any way. Just people pretending to care to make a few pennies.

    report
11. 52

    janderson99posted 11 years ago

    Can I suggest that the hopper information is a bit out of date:
    to help us quickly identify Hubs to feature across HubPages Topic Pages, Related Hubs and the Best, Hot and Latest Hub feeds.  This will create a better browsing experience for readers and offer more rewards to Hubbers that create great Hubs.
    The Best, Hot and Latest Hub feeds are not accessible any more. The new topic layout means that users effectively will only look at the top 10 or 20 hubs displayed (images will count most). Popularity - likes, hub score, traffic, comments etc. would appear to be a simple criterion with a revolving set to give more people a go. So why all this stuff?
    Or is the aim to vet new hubs and reject ones that don't pass the quality test!

    report
12. 74

    carol7777posted 11 years ago

    Each hub should take at least five minutes to make an accurate judgment or opinion. It depends what you are looking for-style, topics, grammar and spelling errors. I am just curious how this is supposed to work effectively.

    report
    1. STAFF

       Paul Edmondsonposted 11 years agoin reply to this

       @Carol7777 - That's a really good question. 
       
       The way it works is we have a team at HubPages that Hops Hubs.  We just can't hop enough of them.  So, we look at how other people hop Hubs and calculate the value of their inputs.  Then we compare their adjusted inputs to that of the internal HubPages team. 
       
       Internally, we don't always agree, so we use a triple standard.  Multiple internal Hoppers have to agree for a rating to be gold and used in the assessments.
       
       From there, we continually reassess the accuracy of the ratings and we calibrate internally as well.  When we put this on HubPages we learned all sorts of interesting things.  It turns out there are people that can rate highly accurately and quickly, we just don't have enough of them. 
       
       For us to get higher degrees of confidence that a Hub is accurately rated it helps to have more ratings from highly accurate raters.  The less accurate and more variance the more ratings are required. 
       
       So, our goal is to get a team of highly accurate and consistent raters to help keep HubPages clean.  If someone does a poor job, they won't be allowed to work on HubPages HITS and their ratings are worth little.  If you're really good at it, I think it's possible that it's a rewarding job and the ratings are very helpful.

       report
13. 71

    galleryofgraceposted 11 years ago

    On second thought this could be a very bad thing. I am a member of mturk and I used it before I found HP to make  a few cents .The responders on mturk will be ones who could not qualify to write on respectable content sites and they may also be ones who hold grudges and will destroy articles.

    report
14. 52

    janderson99posted 11 years ago

    Why are some people being allowed to by-pass "Pending'' - the smallest wait period I have seen is 3 minutes - Are their hubs being hopped?
    
    If the major reason for hub hop checking is to block poor quality hubs, then why not introduce a 'Trusted Author' scheme which applies when 10 hubs have been checking and found to be OK. After this hubs would be published without going through 'Pending'. You could do random checks and re-impose the 'Pending' for defaults. This would free-up resources and established authors would be happier because of the delays in indexing caused by the bot seeing the NOINDEX tag while hubs are pending (bot visits within 30-60minutes after hub is published). Perhaps 'Trusted Authors' could nominate their better quality ones for consideration as 'featured hubs'. This would free up scarce resources, improve moral, get hubs indexed fast - A Win Win Win

    report
    1. 86

       IzzyMposted 11 years agoin reply to this

       That's what I want to know too.

       report
    2. 96

       Peggy Wposted 11 years agoin reply to this

       A trusted author system makes a certain amount of sense with perhaps only random checks to make sure that they are still writing the same quality articles.  If they go off the "deep end" their articles can still be flagged by others.  That would free up some time for HP.  Makes sense to me!

       report
15. 85

    Azure11posted 11 years ago

    I actually thought this was a good idea. Why not use an existing system to pay people. I have an account with murk myself although I haven't used it much recently. I am a little but insulted about the generalisation of people who use mturk. There are plenty of Hits requiring intelligence and top level English. So I thought I might give it a go. But I live in the UK so I am not eligible. Can it not be extended to Brits who obviously also have a pretty good grasp of English?

    report
16. 82

    CMHypnoposted 11 years ago

    As I am British, I m concerned that the only people who are being paid to evaluate hubs are American.  There are hubbers from many other countries where English is the first language, so are we going to be penalised because reviewers don't realise that we sometimes use different spellings for words or use language slightly differently?

    report
17. 0

    DigbyAdamsposted 11 years ago

    There are many people living in America, who can barely speak English. I think we should all be concerned about this new strategy.

    report
    1. 74

       WriteAngledposted 11 years agoin reply to this

       Agreed!
       
       I am particularly concerned that exclusively US residents are being allowed to determine the fate of hubs written by English speakers in other parts of the world.
       
       I am even more concerned that people who are willing to take on a job like this for a pathetic 5 cents a piece will understandably put speed over quality and care.
       
       To value the quality evaluation of a hub at such a ludicrous pittance is demeaning both to hub authors and to the poor individuals forced to scrabble for such pitiful earnings.

       report
18. 0

    EmpressFelicityposted 11 years ago

    Why not post it as a job on Elance and O-Desk and specify that you want native English speakers only (be they American, Canadian, British, Australian or whatever)?

    report
19. 95

    cryptidposted 11 years ago

    I'm not a complainer around here, but this seems like it's going in a really weird direction.  I think when determining whether this outsourcing program makes sense there is only one question to ask:
    
    Is the average person at this Mturk place an equal to the good writers here at HubPages? 
    
    If the answer is "no", then IMO there is no way these folks are qualified to review our work.  I never heard of Mturk until now so maybe they are.  But I also so know that I spent a fair amount of time in some of the darker money-making places of the internet before I came to HubPages, and the people who gravitate to sites where they can make pennies at a time aren't usually the most quality-conscious. 
    
    But all of this is almost beside the point.  HubPages seems very concerned with serving up high-quality content lately, yet clings to the idea that anyone and everyone should be able to publish on this platform.  It seems like the days of having it both ways are gone.
    
    People in the forums have pleaded for a stronger automated system for weeding out garbage.  I don't think everything has to be a "stellar" Hub, but if a Hub is too short, badly formatted and contains numerous spelling and grammatical errors it ought to be pretty easy to catch using the computer.  Outsourcing services should not be required.  Yet in the HubHopper, more often than not, you stumble over 10 messy Hubs for every one good one. 
    
    Also, you'd probably get more people hopping Hubs here if the new hopper wasn't so clunky.  I always use the old one, simply because it lets me breeze past Hubs on topics I don't like.  Maybe it would help if we could hop based on topic.  I hope the Mturk people can do this, because I can't think of anything worse than someone who barely cares about what they are doing rating a Hub on a topic they neither know nor care about.  Yikes!
    
    Anyway, I still love you, HubPages, but I just don't quite get this move.  Hope it works out.  I'm tempted to go and sign up for this thing just to check it out.

    report
20. STAFF

    Paul Edmondsonposted 11 years ago

    It sounds like the main concern is an uneducated person hopping a Hub.  This is the garbage in, garbage out problem.  We care a lot about getting high quality data and have built a system that's pretty robust at figuring the accuracy of the raters.  Poor raters, aren't allowed to access the HITS.  Folks should give it a try.  It's not easy to work for HubPages on Mechanical Turk. 
    
    The good news is there are a few people that appear to be very good at it.  I'm hoping we can tune it and find more.
    
    There was a bug that with the assessment where we weren't putting in enough HITS to complete it.  That's getting worked on today.
    
    There were a few suggestions on why not hire folks internally, or go to another place. In other words, why MTurk.  We looked at several solutions.  Even interviewed folks from an outsourcing service.  The challenge is we need a large number of ratings, people turn over, and finding people that are good at it is challenging.  So, there needs to be a large pool of people to pull from.  Most of the services that offer moderation and rating services use Mturk.  They charge a premium for doing it, so we decided to try our own app first.  We may work with a service like this in the future if we need more volume.
    
    So, we decided to develop a pool of people that rate accurately to incorporate with our other data sources.  Since it's very early days, we still need to figure out how to significantly increase the number of ratings.

    report
    1. 74

       carol7777posted 11 years agoin reply to this

       I think many of us would really enjoy this process of hubbing and judging. I would look at it as a great learning experience. Viewing other people's hubs has greatly improved my writing.  I look forward to see how this unravels.

       report
       1. 86

          IzzyMposted 11 years agoin reply to this

          You are the first person I have seen welcoming the new hub hopper and all the work it entails, or did you mean signing up with Mturk and becoming a wage slave?
          
          This whole thing could be avoided simply by downgrading the site from an open publishing platform, to a site where you have to prove your writing ability first, as you have to do on every other writing site or content farm.
          
          I am also with the other British writers on this (or they could be Canadian or Australian). Why should the grading of hubs be left to Americans only?
          
          I have seen some pretty terrible writing from those who claim to hail from the USA.
          
          Still, no-one has answered why it is that some people are considered to be good writers here and allowed instant featuring of hubs without going through all this palaver, and the rest of us aren't.

          report
21. 74

    carol7777posted 11 years ago

    If this process was generated from within..I used to hub hop  everyday with new hubs..not grade them...just comment.  so I am not really sure where this is going.

    report
22. 76

    jenniferrpoveyposted 11 years ago

    I really don't like this.
    
    I think you would do better to pay an incentive through the system itself. The reason is this:
    
    If an Mturk user gets blocked by two clients within six months, they are GONE. There are very few safeguards to protect users from malicious clients - I got maliciously blocked myself when I used the service, and also blocked by another client by accident. Literally by accident.
    
    You could build up a great corps of raters only to find some of your best people get banned. It's beyond your control. Outsourcing places things *outside the control of hubpages*.
    
    And I don't particularly want to have to go through Mturk to hop hubs...which I haven't been doing because I need to do paid work. Sure, now I can get paid...if I jump through several extra hubs.
    
    Ultimately, though, I don't entirely trust Mturk's system and the way it works. It's too open to certain kinds of abuse. I would never use it to hire people through because I would hate to find a good person, have them be banned (potentially by accident) and not be allowed, per the TOS, to find them again by another means.

    report
23. 52

    janderson99posted 11 years ago

    MECHANICAL HUB HOPPER
    
    Just for fun I ran the MTurk example hubs through Grammarly (general).
    I counted the number of words in the article
    I counted the number of images
    I then played with various formulae in Excel (see the image)
    The results are shown below
    The outcome is pretty good!!!! - 5 minutes work!!!
    The main exception was the last one Rated 9 which has cleaner image layout.
    It is interesting that Grammarly gave it a poorer score than the one rated 7, perhaps the number or words could be given more weight!
    
    Surely Grammarly which includes a spelling and plagiarizing test+ lots more AND number of words AND number of images and their layout/text, would be far simpler than human hub Hopping! (sooo subjective)
    
    

    report
    1. STAFF

       Paul Edmondsonposted 11 years agoin reply to this

       Human ratings are one input.  There is a fairly robust automated system as well. We use people to validate and train the algorithmic quality rater as well.  Grammarly is pretty cool and a good tool:)

       report