{"id":3048,"date":"2023-09-13T10:10:00","date_gmt":"2023-09-13T15:10:00","guid":{"rendered":"https:\/\/wordpress.cs.vt.edu\/3digroup\/?p=3048"},"modified":"2025-01-29T10:06:35","modified_gmt":"2025-01-29T15:06:35","slug":"eyest-predicting-information-relevance-from-eye-gaze-data-in-immersive-space-to-think","status":"publish","type":"post","link":"https:\/\/wordpress.cs.vt.edu\/3digroup\/2023\/09\/13\/eyest-predicting-information-relevance-from-eye-gaze-data-in-immersive-space-to-think\/","title":{"rendered":"EyeST: Predicting Information Relevance from Eye Gaze Data in Immersive Space to Think"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large is-resized is-style-default\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/EyeST_teaser-1024x576.jpg\" alt=\"\" class=\"wp-image-3049\" style=\"width:848px;height:477px\" srcset=\"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/EyeST_teaser-1024x576.jpg 1024w, https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/EyeST_teaser-300x169.jpg 300w, https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/EyeST_teaser-768x432.jpg 768w, https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/EyeST_teaser.jpg 1504w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Eye gaze patterns vary based on reading purpose and complexity, and can provide insights into a reader&#8217;s perception of the content. We hypothesize that during a complex sensemaking task with many text-based documents, we will be able to use eye-tracking data to predict the importance of documents and words, which could be the basis for intelligent suggestions made by the system to an analyst. We introduce a novel eye-gaze metric called `GazeScore&#8217; that predicts an analyst&#8217;s perception of the relevance of each document and word when they perform a sensemaking task. We conducted a user study to assess the effectiveness of this metric. We explore potential real-time applications of this metric to facilitate immersive sensemaking tasks by offering relevant suggestions. <\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"alignleft size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"981\" src=\"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/rate_word-1024x981.jpg\" alt=\"\" class=\"wp-image-3050\" style=\"width:211px;height:202px\" srcset=\"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/rate_word-1024x981.jpg 1024w, https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/rate_word-300x287.jpg 300w, https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/rate_word-768x736.jpg 768w, https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/rate_word.jpg 1076w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">The system retrieves a set of words and documents with different levels of predicted relevance determined by GazeScore. The users go through each of them and rate them based on their perceived relevance. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We found strong evidence that documents and words with high GazeScores are perceived as more relevant, while those with low GazeScores were considered less relevant. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"373\" src=\"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/combined_relevance_gazeRank_significance-1024x373.png\" alt=\"\" class=\"wp-image-3051\" srcset=\"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/combined_relevance_gazeRank_significance-1024x373.png 1024w, https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/combined_relevance_gazeRank_significance-300x109.png 300w, https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/combined_relevance_gazeRank_significance-768x280.png 768w, https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/combined_relevance_gazeRank_significance.png 1440w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Here&#8217;s an overview of our system and the flow of the user study.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Predicting Information Relevance Based On Eye Gaze Data in Immersive Analytics\" width=\"1778\" height=\"1000\" src=\"https:\/\/www.youtube.com\/embed\/6gAcD0t01y0?modestbranding=1\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen><\/iframe>\n<\/div><figcaption class=\"wp-element-caption\">Overview of the EyeST system with details on how the user study was conducted.<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Our findings from this study are scheduled to be presented at the 22nd IEEE International Symposium on Mixed and Augmented Reality (ISMAR).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><div class=\"teachpress_pub_list\"><form name=\"tppublistform\" method=\"get\"><a name=\"tppubs\" id=\"tppubs\"><\/a><\/form><div class=\"teachpress_publication_list\"><h3 class=\"tp_h3\" id=\"tp_h3_inproceedings\">Proceedings Articles<\/h3><div class=\"tp_publication tp_publication_inproceedings\"><div class=\"tp_pub_info\"><p class=\"tp_pub_author\">Ibrahim A Tahmid; Lee Lisle; Kylie Davidson; Kirsten Whitley; Chris North; Doug A Bowman<\/p><p class=\"tp_pub_title\"><a class=\"tp_title_link\" onclick=\"teachpress_pub_showhide('209','tp_links')\" style=\"cursor:pointer;\">Evaluating the Feasibility of Predicting Information Relevance During Sensemaking with Eye Gaze Data<\/a> <span class=\"tp_pub_type tp_  inproceedings\">Proceedings Article<\/span> <\/p><p class=\"tp_pub_additional\"><span class=\"tp_pub_additional_in\">In: <\/span><span class=\"tp_pub_additional_booktitle\">2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), <\/span><span class=\"tp_pub_additional_pages\">pp. 713\u2013722, <\/span><span class=\"tp_pub_additional_organization\">IEEE <\/span><span class=\"tp_pub_additional_year\">2023<\/span>.<\/p><p class=\"tp_pub_menu\"><span class=\"tp_resource_link\"><a id=\"tp_links_sh_209\" class=\"tp_show\" onclick=\"teachpress_pub_showhide('209','tp_links')\" title=\"Show links and resources\" style=\"cursor:pointer;\">Links<\/a><\/span> | <span class=\"tp_bibtex_link\"><a id=\"tp_bibtex_sh_209\" class=\"tp_show\" onclick=\"teachpress_pub_showhide('209','tp_bibtex')\" title=\"Show BibTeX entry\" style=\"cursor:pointer;\">BibTeX<\/a><\/span><\/p><div class=\"tp_bibtex\" id=\"tp_bibtex_209\" style=\"display:none;\"><div class=\"tp_bibtex_entry\"><pre>@inproceedings{tahmid2023evaluating,<br \/>\r\ntitle = {Evaluating the Feasibility of Predicting Information Relevance During Sensemaking with Eye Gaze Data},<br \/>\r\nauthor = {Ibrahim A Tahmid and Lee Lisle and Kylie Davidson and Kirsten Whitley and Chris North and Doug A Bowman},<br \/>\r\ndoi = {10.1109\/ISMAR59233.2023.00086},<br \/>\r\nyear  = {2023},<br \/>\r\ndate = {2023-01-01},<br \/>\r\nurldate = {2023-01-01},<br \/>\r\nbooktitle = {2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},<br \/>\r\npages = {713\u2013722},<br \/>\r\norganization = {IEEE},<br \/>\r\nkeywords = {},<br \/>\r\npubstate = {published},<br \/>\r\ntppubtype = {inproceedings}<br \/>\r\n}<br \/>\r\n<\/pre><\/div><p class=\"tp_close_menu\"><a class=\"tp_close\" onclick=\"teachpress_pub_showhide('209','tp_bibtex')\">Close<\/a><\/p><\/div><div class=\"tp_links\" id=\"tp_links_209\" style=\"display:none;\"><div class=\"tp_links_entry\"><ul class=\"tp_pub_list\"><li><i class=\"ai ai-doi\"><\/i><a class=\"tp_pub_list\" href=\"https:\/\/dx.doi.org\/10.1109\/ISMAR59233.2023.00086\" title=\"Follow DOI:10.1109\/ISMAR59233.2023.00086\" target=\"_blank\">doi:10.1109\/ISMAR59233.2023.00086<\/a><\/li><\/ul><\/div><p class=\"tp_close_menu\"><a class=\"tp_close\" onclick=\"teachpress_pub_showhide('209','tp_links')\">Close<\/a><\/p><\/div><\/div><\/div><\/div><\/div><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Eye gaze patterns vary based on reading purpose and complexity, and can provide insights into a reader&#8217;s perception of the content. We hypothesize that during a complex sensemaking task with many text-based documents, we will be able to use eye-tracking data to predict the importance of documents and words, which could be the basis for <a href=\"https:\/\/wordpress.cs.vt.edu\/3digroup\/2023\/09\/13\/eyest-predicting-information-relevance-from-eye-gaze-data-in-immersive-space-to-think\/\" class=\"more-link\">&#8230;<\/a><\/p>\n","protected":false},"author":330,"featured_media":3054,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_ppma_block_editor_authors":""},"categories":[11],"tags":[14,24,27,509,494,510],"ppma_author":[395,398,396,402,391],"class_list":["post-3048","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-projects","tag-3d-interaction-techniques","tag-ar","tag-augmented-reality","tag-eye-tracking","tag-immersive-space-to-think","tag-sensemaking"],"authors":[{"term_id":395,"user_id":330,"is_guest":0,"slug":"iatahmid","display_name":"Ibrahim Tahmid","avatar_url":{"url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2020\/12\/ibrahim.jpg","url2x":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2020\/12\/ibrahim.jpg"},"author_category":"","user_url":"https:\/\/www.iatahmid.com\/","last_name":"Tahmid","first_name":"Ibrahim","job_title":"","description":"Ibrahim is a fifth year Ph.D. Student advised by Dr. Doug Bowman. His research interest focuses on assisted sensemaking in virtual reality. More specifically, he works on how eye gaze can be used to infer the user-perceived relevance of information during sensemaking and how to leverage that inference to build intelligent recommender models for the immersive space to think."},{"term_id":398,"user_id":105,"is_guest":0,"slug":"llisle","display_name":"Lee Lisle","avatar_url":{"url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2020\/12\/leeLisle.jpg","url2x":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2020\/12\/leeLisle.jpg"},"author_category":"","user_url":"","last_name":"Lisle","first_name":"Lee","job_title":"","description":"Lee is a 5th-year Ph.D. candidate. His Ph.D. work centers around human-computer interaction and immersive analytics with multimedia document sets."},{"term_id":396,"user_id":302,"is_guest":0,"slug":"kyliedavidson","display_name":"Kylie Davidson","avatar_url":{"url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2020\/10\/20181228_113322-1-copy-scaled-1.jpeg","url2x":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2020\/10\/20181228_113322-1-copy-scaled-1.jpeg"},"author_category":"","user_url":"","last_name":"","first_name":"","job_title":"","description":""},{"term_id":402,"user_id":0,"is_guest":1,"slug":"chris-north","display_name":"Chris North","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","author_category":"","user_url":"","last_name":"","first_name":"","job_title":"","description":""},{"term_id":391,"user_id":331,"is_guest":0,"slug":"dbowman","display_name":"Doug Bowman","avatar_url":{"url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2021\/01\/professional_photo2_2019-cropped-square-smaller-scaled.jpg","url2x":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2021\/01\/professional_photo2_2019-cropped-square-smaller-scaled.jpg"},"author_category":"1","user_url":"","last_name":"Bowman","first_name":"Doug","job_title":"","description":"Doug A. Bowman is the Frank J. Maher Professor of <a href=\"http:\/\/www.cs.vt.edu\">Computer Science<\/a> at <a href=\"http:\/\/www.vt.edu\">Virginia Tech<\/a>. He is the principal investigator of the <a href=\"http:\/\/wordpress.cs.vt.edu\/3digroup\/\">3D Interaction Group<\/a>, focusing on the topics of three-dimensional user interfaces, VR\/AR user experience, and the benefits of immersion in virtual environments.\r\n\r\nDr. Bowman is one of the co-authors of <a href=\"https:\/\/www.pearson.com\/us\/higher-education\/program\/La-Viola-3-D-User-Interfaces-Theory-and-Practice-2nd-Edition\/PGM101825.html\">3D User Interfaces: Theory and Practice<\/a>. He has served in many roles for the <a href=\"http:\/\/ieeevr.org\">IEEE Virtual Reality Conference<\/a>, including program chair, general chair, and steering committee chair. He also co-founded the IEEE Symposium on 3D User Interfaces (now part of IEEE VR). He received a CAREER award from the National Science Foundation for his work on 3D Interaction, and has been named an ACM Distinguished Scientist. He received the Technical Achievement award from the IEEE Visualization and Graphics Technical Committee in 2014 and the Career Impact Award from IEEE ISMAR in 2021.\r\n\r\nHis undergraduate degree in mathematics and computer science is from Emory University, and he received his M.S. and Ph.D. in computer science from the Georgia Institute of Technology.\r\n\r\n<a href=\"http:\/\/people.cs.vt.edu\/~bowman\/cv.pdf\">Curriculum vitae (PDF)<\/a>"}],"jetpack_featured_media_url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2023\/08\/EyeST_teaser-1.jpg","_links":{"self":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts\/3048","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/users\/330"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/comments?post=3048"}],"version-history":[{"count":6,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts\/3048\/revisions"}],"predecessor-version":[{"id":3262,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts\/3048\/revisions\/3262"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/media\/3054"}],"wp:attachment":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/media?parent=3048"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/categories?post=3048"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/tags?post=3048"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/ppma_author?post=3048"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}