{"id":729,"date":"2020-03-04T14:19:35","date_gmt":"2020-03-04T14:19:35","guid":{"rendered":"http:\/\/wordpress.cs.vt.edu\/cs6724s20\/?p=729"},"modified":"2020-03-04T14:19:35","modified_gmt":"2020-03-04T14:19:35","slug":"03-04-20-lee-lisle-combining-crowdsourcing-and-google-street-view-to-identify-street-level-accessibility-problems","status":"publish","type":"post","link":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/2020\/03\/04\/03-04-20-lee-lisle-combining-crowdsourcing-and-google-street-view-to-identify-street-level-accessibility-problems\/","title":{"rendered":"03\/04\/20 &#8211; Lee Lisle &#8211; Combining Crowdsourcing and Google Street View to Identify Street-Level Accessibility Problems"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><strong>Summary<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Hara, Le, and Froehlich\ndeveloped an interface that uses Google Street View to identify accessibility issues\nin city sidewalks. They then perform a study using three researchers and 3\naccessibility experts (wheelchair users) to evaluate their interface. This\nsevered as both a way to assess usability issues with their interface as well\nas a ground truth to verify the results of their second study. That study\ninvolved launching crowdworking tasks to identify accessibility problems as\nwell as categorizing what type each problem is. Over 7,517 Mechanical Turk HITs\nthey found that crowdworkers could identify accessibility problems 80.6% of the\ntime and could correctly classify the problem type 78.3% of the time. Combining\ntheir approach with a majority voting scheme, they raised these values to 86.9%\nand 83.8%.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Personal Reflection<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Their first step to\nsee if their solution was even feasible seemed like an odd study. Their users\nwere research members and experts, both of which are theoretically more driven than\na typical crowdworker. Furthermore, I felt like internal testing and piloting would\nbe more appropriate than a soft-launch like this. While they do bring up that\nthey needed a ground truth to contextualize their second study, I initially felt\nthat this should then be performed by only experts and not as a complete\npreliminary study. However, as I read more of the paper, I felt that the comparison\nbetween the groups (experts vs. researchers) was relevant as it highlighted how\nwheelchair bound people and able-bodied people can see situations differently. They\ncould not have collected this data on Mechanical Turk alone as they couldn\u2019t\nguarantee that they were recruiting wheelchair bound participants otherwise.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It was also good to\nsee the human-AI collaboration highlighted in this study. That they\u2019re using\nthe selection (and subsequent images generated by those selections) as training\ndata for a machine learning algorithm, it should lessen the need for future\nwork.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Their pay level also\nseemed very low at 1-5 cents per image. Even assuming a selection and\nclassification takes only 10 seconds, their total page loading only takes 5\nseconds, and they always get 5 cents per image, that\u2019s $12 an hour for ideal\ncircumstances.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The good part of\nthis research is that it cheaply identifies problems quickly. This can be used\nto identify a large amount of issues and save time in deploying people to fix issues\nthat are co-located in the same area rather than deploying people to find\nissues and then solve them with lesser potential coverage. It also solves a public\nneed for a highly vulnerable population which makes their solution\u2019s impact even\nbetter.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Lastly, it was good\nto see how the various levels of redundancy impacted their results. The falloff\nfrom increasing past 5 workers was harsher than I expected, and the increase in\nidentification is likely not worth doubling the cost of these tasks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Questions<\/strong><\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>What other public needs could a Google Street\nView\/crowdsourcing hybrid solve?<\/li><li>What are the tradeoffs for the various\nstakeholders involved in solutions like this? (The people who need the fixes,\nthe workers who typically had to identify these problems, the workers who are\ndeployed to maintain the identified areas, and any others)<\/li><li>Should every study measure the impact of\nredundancy? How might redundant workers affect your own projects?<\/li><\/ol>\n","protected":false},"excerpt":{"rendered":"<p>Summary Hara, Le, and Froehlich developed an interface that uses Google Street View to identify accessibility issues in city sidewalks. They then perform a study using three researchers and 3 accessibility experts (wheelchair users) to evaluate their interface. This severed as both a way to assess usability issues with their interface as well as a [&hellip;]<\/p>\n","protected":false},"author":105,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[77,74],"class_list":["post-729","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-class7","tag-crowdstreetview"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/729","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/users\/105"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/comments?post=729"}],"version-history":[{"count":1,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/729\/revisions"}],"predecessor-version":[{"id":739,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/729\/revisions\/739"}],"wp:attachment":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/media?parent=729"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/categories?post=729"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/tags?post=729"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}