Month: February 2024

Starting with GTA work again this week office hours were busy for both courses. For CS5590 the Cross Site Scripting Attack Assignment is due this week so students were frequently asking questions. For CS1054 the students had their Project 1 due which I have to grade before the Spring break and a Quiz today which I had to proctor too. This meant office hours last week were pretty occupied for this course too.

For coursework, Machine Learning lectures are being posted asynchronously as recordings which are going in the back log for me. I have to catch up on the lectures since Home Work 2 is out and we there’s a mid for this course after Spring Break. For Advanced Topics in HCI, I had three readings due this week related to different side channel attacks. So coursework wise this week was a little easier.

For research, I started off the week with the goal to port the Surf Share project to Magic Leap 2 device due to hardware availability. Since the project is in Unity 2019 version as the first step I had to upgrade the project to 2022 version for the porting to be made possible. This required some work with the existing packages. The next thing was to change the project settings to make the project compatible with Magic Leap 2 this require hand tracking enablement and reconfiguration of MRTK profile to Magic Leap default with some other tweaks. After configuring the project, the project could still not be ported since the OpenCV package used in the original package had been deprecated and is no longer supported by the 2022 version. Having reached this point I switched back to HoloLens 2, searched for the availability of a second device in library but their device is occupied. While I still look at other avenues of getting the second device for the project, I started implementing a MVP for the occlusion attack on the single Surf Share user. For now I have implemented a basic occlusion attack, I will probably work on making that more sophisticated and Denial of Service Attack for the next week.

For GTA work of CS5590 I completed XSS Seed lab myself. I noticed students have already started coming into my office hours right after the assignment was released maybe because they struggled with the last one. Whatever, the cause it is good for them to start early and good for me since I hope that will lead to lesser amount of emails a day before submission. Also graded more paper summaries for this class. For Intro to Java, students got their project assignment so that is mainly what has been occupying my office hours.

For courses, Machine Learning classes are async for a while so have to play catchup with that. I haven’t started on writing Friday Seminar reflections yet so that is also something on my list. for Advanced Topics in HCI, got the project proposal out of the way, and been reading some papers about differential privacy. The formal proofs in those papers is not something I have a great grip over but the discussant presentations really helped understand the paper those papers better.

For research, there were 3 different tasks I was focusing on. Starting with Surf Share, I did get access to one HoloLens 2 which is a start, spent one whole day debugging the code playing with project settings and unity versions to finally get the code running on the one device. The surf share however requires another client to connect to go beyond its first interface of drawing the portal. So then I shifted to understanding the networking mechanism of this code and unity in general. After one research on Network Manager and WebRTC I would like to think I understand how the networking is happening but still need another HoloLens2 device to connect to verify it and basically start looking into security threats. So that is the progress on technical end.

The other important progress for this week was the meeting with the medical advisor on the ICAT project Dr. Andy Muelenaer. Hi insights really help understand the motivation of this project, the current status quo in the medical field n telehealth and the true value we can provide with this study. Brendan suggested I should pen these down in the paper format to ease paper writing down the road. I couldn’t exactly find the time to write the paragraph itself but I did create a structure for it so I remember what to write when I get to it. I made sure to read through the motivation sections of the last year’s publications of the targeted venue and cover the major points according to them.

Lastly, continuing my literature review of the project I read the following paper:

That Doesn’t Go There: Attacks on Shared State in Multi-User Augmented Reality Applications – Slocum et al

This paper talks about how AR devices use camera feed input as the key for authentication in multiuser environment. And how a picture of the physical environment can be used to generate the key and create Read and Write attacks in Augmented Reality

This week GTA work was less teaching more grading. Graded Assignment 1 on Buffer Overflow Attack and Paper Summaries for CS5590 and also the first Quiz for CS1054. Also helped with the preparation of Assignment 3 on Ransomware for CS5590. Now I have to attempt and understand Assignment 2 on XSS attacks for the next week before it is released for the students.

For coursework, I submitted the Machine Learning Homework it was mainly theory based but the questions were very well designed with tricks that made you think harder. For Advanced Topics in HCI read a few papers and submitted their paper summaries, some were related to my own research that talked about perceptual manipulation in MR devices. I am currently working on the project proposal for this class with my roommate to be presented tomorrow.

My research work is kind of overlapping with the Advanced Topics class, so while I was explaining the project idea on security threats in collaborative mixed reality systems to my project partner, I got a chance to zoom out from my work and evaluate it at the higher level and plan out its contributions. This led to a fruitful discussion with Brendan on our aim for the study and somewhat formulation of a research question with a user study that seems technically feasible and now has defined experimental conditions and evaluation metrics. All of this although assumes the correct working of SurfShare. Unfortunately, the current unavailability of Microsoft HoloLens 2 has become a blocker in testing this out. The user study design however, led to me brainstorming on possible experimental tasks that are not directly medical in nature but measure important metrics for carrying out successful medical tasks. These metrics include precision, timeliness and accurate understanding of expert instructions.

This week was yet again swarmed with GTA work. Students have their first programming assignment for CS5590 this Thursday which means my office hours and inbox have been super crowded with queries ranging from the installation, setup and the actual concept and implementation of the assignment. I myself, ran into issues while doing this assignment so it was kind of expected. Also, started grading the paper summaries for students for each class. Speaking of paper summaries had to do loads of them for my own courses as well. For CS1054 students had their first quiz which once again called for a lot of pre quiz questions in and out of office hours. But I do enjoy being a part of someone’s first interaction with computer programming and them being fascinated by it.

For my own courses, another person reached out for Advanced Topics in HCI project, they have their own set of unique skills and I can think of ways they can be beneficial too but at the same time justifying more than two students on my project might be hard. I have however, left the ball in their court to decide if they want to extend this project in there area of expertise. Apart from that the Machine Learning Homework is due this week but the syllabus was not covered till yesterday’s class so I still have questions left to attempt before submission.

For research, Brendan had reached out to the authors of SurfShare to get access to their open-source repository. Which they did share with us, so that is good news. We now have a potential alternative to ML Remote Assist that has an AR device on both ends. It also has some other cool features such as duplicating object from the real world and manipulating the virtual replicas that can be of great assistance to our user study. However, I need two HoloLens 2 to run that code. The HoloLens 2 in our lab has been occupied for this week so that has been a blocker in actually testing the surfshare out. Furthermore, in the search of potential security threats while using VR devices in the medical context or otherwise, I read the following paper:

OVRSEEN: Auditing Network Traffic and Privacy Policies in Oculus VR

This paper evaluates the network traffic of applications from the oculus store and analyze the kinds of data collected, the purpose of it and finally how they disclose this in their privacy policies. The way they collect this network data and decrypt it specifically for oculus can also be of benefit to our study. I have the git repository for their code but again the headsets in the lab have been occupied so will hopefully be able to test this out as soon as there is an oculus available for use. The SurfShare will still be higher on priority list though.