Summary
Hara et al.’s paper “Combining Crowdsourcing and Google Street View to Identify Street-level Accessibility Problems” explores the crowdsourcing approach to locate and assess sidewalk accessibility issues by labeling Google Street View (GSV) imagery. Traditional approaches for sidewalk assessment relies on street audits which are very labor intensive and expensive or by reporting calls from citizens. The researchers propose using their designed interactive user interface as an alternative to proactively deal with this issue. Specifically, they investigates the viability of the labeling sidewalk issues amongst two groups of diligent and motivated labelers (Study 1) and then explores the potential of relying on crowd workers to perform this labeling task and evaluate performance at different levels of labeling accuracy (Study 2). By investigating the viability of labeling across two groups (three members of the research team and three wheelchair users), the results of study 1 is used to provide ground truth labels to evaluate crowd workers performance and to get a baseline understanding of what labeling this dataset looks like. Study 2 explores the potential of using crowd workers to perform the labeling task. Their performance is evaluated on both image and pixel levels of labeling accuracy. The findings suggest that it is feasible to use crowdsourcing for the labeling and verification tasks, which leads to the final result of better quality.
Reflection
Overall, this paper proposes an interesting approach for sidewalk assessment. What I think most is how feasible we can use that to deal with real-world issues. In the scenario studied by the researchers, the sidewalk under poor condition has severe problems and relates to a larger accessibility issue of urban space. The proposed crowdsourcing approach is novel. However, if we take a close look at the data source, we may question to what extent it can facilitate the assessment in real-time. It seems impossible to update the Google Street View (GSV) imagery on a daily basis. The image sources are historical instead the ones that can reflect the current conditions of the road sidewalks.
I think the image quality may be another big problem in this approach. Firstly, the resolution of the GSV imagery is comparatively low and sometimes under poor light conditions, which is challenging to let the crowd workers make the correct judgement. There is possibility to use some existing machine learning models to enhance the image quality via increasing its resolution or adjusting the brightness. That could be a potential place to introduce the assistance of machine learning algorithms to achieve better results in the task.
In addition, the focal point of the camera was another issue which may reduce the scalability of the project. The CSV imagery is not collected merely for the sidewalk accessibility assessment, which would usually contain a lot of noises (e.g. block objects). It would be interesting to conduct a study about how much percent of the GSV imagery is of good quality in regards to the sidewalk assessment task.
Discussion
I think the following questions are worthy of further discussion.
- Are there any other important accessible issues existing but not considered in the study?
- What are improvements you can think about the authors could improve their analysis?
- What other potential human performance tasks can be explored by incorporating street view images?
- How effective do you think this approach can deal with the urgent real-world problems?
I agree with your question on other accessibility issues that have not been considered in the study. Indeed, people with particular accessibility issues won’t be able to label the ground truth as effectively if they have reduced vision, etc. Nonetheless, these are accessibility problems that need to be addressed. We can use volunteers who work with such people to generate the ground truth. Additionally, I think you are correct regarding the comment on improving the image using machine learning or other computer vision techniques to improve the quality of images.
I agree with you that the image quality and age problem is urgent to be addressed in this system. For the low quality images, it is hard for the crowd workers to find and label the accessibility problems. Even though the workers labeled some of the problems, some of the images are captured from years ago, and there will be new issues or the labeled problems may have been fixed.
I think it would be good to have a function for the users to submit their own captured photos. As a result, these users can do the first time labeling and we can have images from different points of view. However, this is a great work and may consume lots of time and money. Also, it is hard to have an amount of users willing to share photos and do the labeling job. So, I am not sure whether it is worth to build such a system.
I think a version of this is already being used to monitor real-world problems. For example, satellite imagery is used to monitor forest cover, ice cover, etc. This helps us in responding to them and also control human actions accordingly. For example, ozone layer was a major issue for quite some time, but monitoring and controlling emissions has now brought it back to safe levels.