SUMMARY

In this paper, the authors propose a flexible workflow to enable the creation of visual blends. The authors specifically focus on the ‘Hybrid’ visual metaphor wherein objects are ‘fused together’. The authors propose a blending design pattern ‘Single Shape Mapping’ which can be used to blend two objects with similar shapes and blends all of one object into a part of the other one. The workflow consists of different microtasks including brainstorming, annotation, and evaluation. The entire workflow consists of 6 steps – brainstorming, finding images, annotating images, detecting images that blend together (by the system), automatic synthesizing of the blends, evaluating the blends (by the user) – and the users were made to watch a 15-minute training session before they started with the task. The authors evaluated the task decomposition with three studies namely decentralized collaboration,  group collaboration on blends for messages, and novice users with and without VisiBlends. The authors found that VisiBlends was helpful as it enabled people to meet all of the constraints associated with visual blending.

REFLECTION

I enjoyed reading this paper and found the motivation for this study to be very nice. It was interesting to see how creativity, which is predominantly a human affordance, was being presented in a mixed-initiative setting.  I liked the examples that were chosen throughout the paper and found that they helped me to better understand the challenges associated with blending images (orange + healthy, with apple as the health symbol) as well as to appreciate the images that were generated well (orange + healthy, with a health symbol that was not food).

The study on ‘decentralized collaboration’ reminded me of the paper on ‘The Knowledge Accelerator: Big Picture Thinking in Small Pieces’ which was discussed last week. I liked the study on the ability of novice users to create these visual blends with and without VisiBlends. I also agree that having a flexible iterative workflow similar to the one used in the paper is very useful as it aids the users to identify issues with the original results and then improve upon the same.

I liked that the authors discuss how creative design problems also have patterns in the ‘Discussion’ section. I think it would be very interesting to see a similar study be conducted in the domain of story writing. Text data poses a multitude of challenges and having a decomposed workflow such as this proposed for writing would be very interesting – especially given that authors may have very varied writing styles.

(Interestingly, “Wash your hands. It’s the smart move” was one of the messages used as part of this study.)

QUESTIONS

1. What modifications can be made to the interface design of the VisiBlends website to better aid users in creating these visual blends? What are the main drawbacks of the existing interface?
2. The authors propose the decomposed workflow for ‘Hybrid’ visual metaphors. Is it possible to create such a workflow for the ‘Simile’ or ‘Contextual Metaphor’ visual blends? What kind of changes would be required to be made to enable this?
3. The authors conduct a study to evaluate the usage of this system by novices. What results would have been yielded if the users of the system were people with a background In marketing, but who were not graphic designers? Would their approach to the problem have been different?

Read More

Summary

The authors define a method to include graphics to draw attention to messages, specifically the blending of two images from two different concepts. They started the paper with the beautiful example of Starbucks and summer, and how the two ideas generated the resultant image. The paper mentions that the process they used was the hybrid blend of two images into one object, which represents both the concepts effectively. The process of visual blending is defined here to be two concepts resulting in two objects and then the integration of both the objects into one, yet, making sure that both the original objects are visible. The VisiBlend system involves various steps in the entire workflow. Firstly, the users need to brainstorm associated concepts, then find images from both the concepts, followed by annotating them correctly. Finally, the algorithm chooses the images to blend and generates final images for the mix, which is evaluated by the user. It was found that the tool helped in the improvement of the resultant image. 

Reflection

This is an interesting study that can be used in the field of advertisement and marketing, among others. The main attraction of the paper is that the system helps people who are novices in graphic designing. The steps are straightforward to follow, and the results showed that using the VisiBlend system proved to be more helpful. This paper essentially tries to find the similarity between images as opposed to texts. 

One of the drawbacks that the users pointed out was the time taken to find relevant images to a concept. I think this problem could be mitigated if we were to even consider scenes as opposed to a single image. Various computer vision algorithms provide tools to get bounding boxes and improve image quality. This would help users save time and get components from larger images too.

I also kept on thinking about how we can extend this idea to other possible areas. One thing that comes to my mind is if we could combine images and text to find analogies between them. By combining them, I do not mean just providing a keyword and getting an image. That is what Google or any search algorithm does. However, I mean, if a user searches for a word, then gets an image they like, could we link articles to that? We could also try and find text related to the final blended image. I am not sure about the feasibility; however, this would be an interesting experiment. 

Additionally, I also was intrigued to know if participants having good technical (CS background) or artistic background (Graphics) are faster in finding images. I believe that the more “internet searching” talent we have, the lesser the time. 

Questions

1. What other fields can this idea be extended to? Could we possibly find texts related to the blended image? For example, for New-York and night, we could find articles that talk about both the concepts. This may result in finding more images too. 
2. Would using complex images like scenes as opposed to only singular images help? Computer Vision algorithms can be used to get bounds boxes out of such images to get individual objects. 
3. Do people with better technical or graphical skills perform better?

Read More

Summary

Chilton et al.’s paper describes a flexible workflow for creating visual blends that the authors dub “Visiblends’ (do you see what they did there?). Lack of imagination in naming notwithstanding, the workflow involves an input of two concepts, brainstorming, image classification and blending, and evaluation of the automatic blending. They performed three studies where their workflow was tested, showing that decentralized groups of people could appropriately brainstorm and generate blends in microtasks. The first phase involves associating different words with the input themes to create a broad base of kinds of images. The second phase involved the searching for related imagery. The third phase asked crowdworkers to annotate the found images for basic shapes and coverages of those shapes. The fourth stage is performed by an AI and involves shape matching between images to combine the two themes, while the final stage (also by AI) blends the images based on the image matching. Their studies confirm that decentralized groups, collaborative groups, and novices can all use this workflow to create visual blends.

Personal Reflection

               I liked this work, overall, as a way for people to get interesting designs out of a few keywords of whatever they’re working on. I was somewhat surprised that the second step (“Finding Images”) was not an automatic process. I had figured when I read the introduction that this step was automated by image recognition software, since these are not complex images but images of single objects. However, when it was explained in the Workflow section, it makes it clear that these images are essentially another phase of the brainstorming process. However, I was concerned that it was perhaps a complex microtask since it asked for an implementation of several somewhat complex filters as well as ten images from those filters.

               I thought the images in Figure 7 were somewhat deceptive, however. They stated in the caption for that image that there was “aesthetic editing by an artist,” which implies they had a visual designer already employed. If that was the case, why is the expert not performing the expert task? I would have liked to see the actual resultant images as they show (some of) in the later studies.

               The refinement process they introduced in the first study was also interesting in that the refinement was more than just asking for more results – the user actually iterated on the design process to find similar shaped items between the two categories. This shows an aspect of human intelligence to solve a problem that the AI had difficulty solving – realizing why the AI was having trouble was a key part of the process.

               Lastly, I would have liked to see what would have happened if all three groups (decentralized, group collaboration, and novices) were given the same concepts to generate ideas from. Which group might have performed the best? Also, since this is quite decentralized, I would have liked to see an mTurk deployment to see how the crowd could perform this task as well.

Questions

1. As discussed above, an interesting part of this paper was how human intelligence was employed to refine the AI’s process, thereby giving it better inputs. Are there other ways that using human insight into why an AI is having issues is a good way to solve the problem?
2. When a workflow that creates microtasks like this, is it more helpful to test it with participants that come into a lab or through a crowdworking site like Mechanical Turk? Should they both be performed? Why or why not?
3. Would you use a service like this for a creative image in your work? Why or why not?

Read More

Authors: Lydia B. Chilton, Savvas Petridis, and Maneesh Agrawala.

Summary

This paper discusses the concept of visual blends, which are an advanced design technique to draw attention to a message. This paper presents VisiBlends, a workflow for creating visual blends, by decomposing the process into computational techniques and human micro-tasks. Users can collaboratively create visual blends with steps involving brainstorming, synthesis and iteration. An evaluation of the workflow showed that it improved novices’ ability to create visual blends, and it works well for both decentralized and co-located groups.

Reflection

Stemming from a poor design sense personally, I appreciated how this paper read and what it has to contribute a lot. Creativity is a complex topic, and designing computational tools to support it can get really tricky. This paper was easy to read, and very well supported by figures for helping readers understand not only the concept of Visual Blends, but also the findings. VisiBlends opens up new possibilities of how tools can extend support to other design applications such as web design. (As per my knowledge, there are AI engines for generating web design templates, color schemes, etc. but I am not aware of user studies for these AI solutions)

This paper echoes something that we have read in a lot of papers — decomposing big tasks into smaller, meaningful chunks. However, decomposing creative tasks can become tricky. The steps were simple enough for onboarding novice users and the algorithm was intuitive. This human-AI collaboration seemed a bit unique to me particularly because the success of the whole endeavor also depended on how well the user understood how the algorithm works. This is a stark contrast to the black-box vision of algorithms. Will it become difficult as the algorithm gets more complex to support more complex blends?

All of the findings supported the usefulness of VisiBlend approach. However, I wonder if there’s a possibility of the task or the concept of visual blends (ticking off the checklist of requirements) being complex enough to understand in the first attempt. I am sure the training process, which they stress to be important, was thorough and comprehensive. But, at the end of the day, it boils down to learning from experience. I feel one would understand the requirements of visual blends better through the tool and may face difficulty in the control condition.

I really liked how the participants iterated in different ways to improve upon the initial visual blend. This is a great demonstration of human-machine collaboration, where people use machine suggestions to refine the original parameters and improve the whole process. I am also glad they addressed the issue of gender stereotypes for the “Women in CS” design task as that was something on my mind as well.

Questions

1. What do you feel about decomposing creative tasks? What are the challenges? Do you think it’s always possible?
2. Do you think users should always have a good sense of how the algorithm works? When do you think it’s necessary?
3. What changes are necessary for this to scale up to more complex designs? How would a complex algorithm affect the whole process?

Read More

Visual blending is the idea of taking two objects or concepts that you want to represent and combining them in a way that both concepts are identifiable, while also implying that one concept is applicable to the other. The creation of visual blends is a very creative process. But with Visiblend, the authors have created a system that has greatly streamlined the process of creating visual blends, and even split the process up as microtasks. The process of creating visual blends is split up into tasks of ideation of related concepts, searching of singular representative images with simple iconic shapes for the ideated concepts, and annotate the shapes on the images. The system then takes this information to create different combinations (blends) of the images and returns them to the user for evaluation and further iteration. They conduct three case studies looking at how Visiblend can be used under different situations. They also make note of the limitations that Visiblend can only deal with simple iconic shapes and cannot do more complex stuff like animations.

The visual blended images are some of the most powerful imagery I’ve ever come across. They convey ideas so well. I think that this is a really good project that is streamlining the process of creating these powerful images. I am actually shocked at how simple the steps are (granted, it takes a lot more work to actually make it look good). But still, very surprising. Initially, I felt that the system was very limited because all it was doing was cropping and overlaying one picture on top of the other. How could that be of possible use? But then I realized that the real value was coming from the fact that it is able to perform so many blends automatically with no human effort and demo them all. It’s utility comes from the speed and iteration of the visual blends that we can do through it. It’s also really interesting how the tool allows to visualize really unintuitive combinations (like the McDonald’s + energy example). Where a human doing it would be really limited by their preconceived notions of both those concepts, a machine doesn’t have those blocks and can therefore present any combination of zany ideas that a human can look at go “Oh! That does work!”. So it serves as the perfect tool to come up with ideas because it does not have any inhib
itions.

1. What kind of workflow would be necessary to do something like this, but for animation and gifs instead of static images?
2. Do you think this streamlined workflow would impede some creative ideas from being conceptualized because people’s thought processes are trained to think this way?
3. In your opinion, does Visiblend function better as a centralized collaborative tool (where everyone is in the same room) or as a distributed collaborative tools (i.e. using crowd workers on crowd work platforms)?

Read More

Summary:

This article is devoted to researching a flexible system to complete visual blends. Visual blends is an advanced graphic design technology designed to attract people’s attention. It combines two objects together, which has applications in many fields, because it can express information well. However, visual blends do not simply combine or paste the two pictures together. It need to combine the characteristics of the two images, and it also need to be able to separate the two images. The previous working methods have some drawbacks, such as assigning tasks to a work fixedly person. Therefore, the author of this paper proposes a new workflow that can achieve flexible visual blends. The workflow of the system decomposes the flow of integrated computing technology and human microtasks, so that users can collectively generate visual fusion through brainstorming, synthesis and iterative steps. The workflow includes: brainstorming, Finding Images, Annotate Images for Shape and Coverage, Matching Algorithm, Automatic Blend Synthesis and Evaluation.

Reflection:

Before reading each article, I will anticipate the method of the article from the title of the article. Also before reading this article, I am thinking about how to implement a flexible workflow for visual blends. I think that when combining two images to form a new image and complete a new idea, increasing the flexibility of the workflow requires a wider selection of pictures and the order of work that can be changed, for example, to determine the images then to achieve the purpose, or determine a purpose, and then find two images. But I have never thought about starting from the worker’s perspective, through the participation of more workers, to build a flexible workflow.

And, I think the more important benefit of introducing brainstorming is inspiring workers. Compared to the method I mentioned before, and combined with real life, when completing a creative task, inspiration is the most difficult to find. So I think this is indeed a very good way. The benefits of setting up the first step of the system as brainstorming are:

1. Help workers better understand the purpose of the task, which is not limited to the purpose of the current task, and at the same time, the understanding of visual blends can also be improved.
2. Inspire inspiration. In the multi-person work style, people can choose the tasks they want to do, and can look at the results of others, and even collaborate with each other, which is very helpful for the construction of ideas.

So I think that this system has improved the flexibility of inspiring ideas and work. At the same time, because of the introduction of annotations, merge two images by computers and the evaluation system, I think the quality can be guaranteed in all aspects. But I think there are some limitations, for example, the generation of the machine may not control some details, but I think that a good design must pay great attention to details.

Question:

1. Do you think this system can realize a flexible workflow, or it is more inclined to a heuristic system.
2. How the system should control the details of the images automatically synthesized by the machine
3. What are the limitations of this system?

Read More

This work presents VisiBlends, a system for creating visual blends by providing a flexible workflow that follows an iterative design process. The paper explores the feasibility of decomposing the creative task of designing a visual blend into microtasks that can be completed collaboratively. The paper also aims to explore if this system is capable of enabling novices to create visual blends easily. The workflow proposed decomposes the task process of creating a visual blend into computation techniques and human tasks. The main tasks involved in the workflow are brainstorming, finding images for each concept, annotating images for shape and coverage, synthesis, evaluation, and iteration. Three studies were conducted as part of this paper. The first was to test the feasibility of decentralized collaboration and involved 7 individuals taking part in only one of the above-mentioned tasks. The results demonstrated that although they were working only on parts of the workflow, together they were able to successfully create visual blends. The second study was to test group collaborations and involved groups of 2-3 people working together to create visual blends. The third study involved testing the ability of VisiBlends in helping novice users create visual blends. Overall, the results showed that due to the flexible, iterative nature of the system, the complex task of creating visual blends was successfully decomposed into microtasks.

I feel that human cognition and AI capabilities have been leveraged innovatively to decompose a complex, creative problem into distributable microtasks. VisiBlends breaks down the task into microtasks that involve both human cognition as well as artificial intelligence. This paper addresses the fundamental elements involved in the creative task of visual blend designing and introduces the concept of the Single Shape Mapping design pattern. 

I feel that VisiBlends is a great tool that allows novice users to create exciting visual content to promote their idea/product. It provides a platform that abstracts the complexities of creating a visual blend away. The user is tasked with finding the relevant images and annotating them for shapes and coverage. The rest of the magic happens on its own wherein VisiBlends runs a matching algorithm and synthesizes various potential blends from the user’s images. The iterative design flow allows users to modify the suggestions as required. I really liked the resue functionality of VisiBlends as well. This would definitely help with the brainstorming phase and would save a lot of time and effort.

• It was interesting to note that each participant was trained on the entire workflow although they were made to contribute only to one step. This helped them understand the big picture and they were able to function more efficiently. Can this finding be applied to other systems to improve their performance?
• Can similar techniques of decomposition be applied to other creative domains?
• One challenge mentioned in the paper was regarding the difficulty faced during the search for relevant images where users were not able to find appropriate images. What are some techniques that could be employed to overcome this?

Read More

The authors designed the VisiBlends system, which is used to do visual blends. This is a quite challenging task before because the generated object should contain the two input objects while these objects should be still distinguishable. However, the system leveraged both human strength and AI strength to achieve this goal. Humans need to find relative images, annotate images for shape and coverage, and evaluate the outputs generated by Automatic Blend Synthesis. Machines will apply matching algorithm according to human’s classification and blend the objects. Though the evaluation section mentioned that the workflow often fails, the system is flexible, and users can iterate and adapt easily. With the system, even novices can collaboratively complete the difficult visual blending tasks easily.

Reflections:

The system is interesting and useful since a user who knows nothing about visual blending can obtain a good blended design from a group of people who knows little about design, program, or visual blending. This can be a good example of human-AI interaction. Neither the inexperienced workers nor the AI itself cannot design such a kind of challenging task. However, with the system, they can collaborate to complete the tasks.

I like the experiment in which workers are divided into two groups. One group is exposed to a set of previous blended objects while the other one does not. I was surprised about the result that the group of workers who have experience with what the previous outputs look like performs worse than the other group. I think this is because their minds are limited by the successful outputs so they cannot have novel ideas of blending after brainstorming. The system works like it uses an exhaustive mechanism to get human ideas. If the human’s ideas are limited by some previous success, the result will lose some blending with novel ideas. If I am a worker who has some experience with image blending, I would consider whether I can blend the images or not instead of just thinking about whether the output would be interesting or not. However, the blending tasks are carried out by automatic mechanisms, so those guys who list all kinds of possibilities would probably obtain better ideas.

I think the system can be better used by experts instead of workers. If the workers have no experience with this system, it is hard for them to find proper images that can be used in the blending task later. They may spend a huge amount of time uploading images having no objects with proper shapes. Instead, experts can have an image of how the output would be looked at and provide only the proper images. Or a database of images should be provided to the users so that they can only make selections from the images which are good for blending.

Questions:

How can the system’s success rate be improved?

Does it worth to hire some professionals to involve in some of the stages of the system?

Can we design a system in which the user can upload pictures, annotate the images on himself, and the system generates the output according to his annotations? What is the advantage of hiring workers to do these tasks?

Read More

Summary

The author in this paper introduced an advanced graphic design technique which combines two objects or concept in a novel and meaningful way in conveying a message symbolically. To achieve this, the author presents a tool, VisiBlends, a flexible hybrid system that facilitates the generating of visual blends based on an iterative design process. The author first introduces and defines the problem of vidual blends and then decomposes the process of creating visual blends into sub-task. The baseline of this iterative design process is that let users brainstorm first regarding the concept and then find certain relevant types of images. Then the user annotates images for the convenience of the system automatically detects which images to blend. Finally, users evaluate each blend and decide whether or not iterate the process. To find out whether the system could support decentralized collaboration and co-located teams generate the visual blend, and whether this system would help novices create blends efficiently, the author conducted three user studies. The study results indicate that both decentralized groups and co-located groups can generate visual blends to express their messages efficiently. Further, the system VisiBlends indeed helps novices generate visual blends.

Reflection

I really like this paper, and I even want to try the system. Create something novel out of thin air is always hard. Therefore, people are continually looking for tools that can stimulate creativity and brainstorming, hoping that these tools can inspire us. The paper we discussed last week, which presents a tool to help find analogy from papers also trying to do the same thing. This really shows the essence of creative inspiration.

It really enjoins to see the study process, especially study 2, group collaboration on blends for messages. They discovered many constraints, but they also solved these constraints cleverly, such as focusing on the images of the other concept to increase the chance of finding a blend when the image of another concept is limited. I particularly like the example of women + CS. Workers were trying to avoid gender stereotypes, even though it’s tough to think about the creative way. Thus, the author concludes that it’s hard to meet all the constraints, and we have to decide where to compromise.

The human visual system inspires me of a way to create a database of visual patterns. Since human tends to recognize an object based on its 3D shape, silhouette, depth, color, and details, we could let a group of people identify a blurred shape, which contains certain features but does not clear enough to recognize the actual object. Then we can base on participants’ visual perspectives to perceive the user perspective of the metaphor of this shape.

For the third study, there is an interesting phenomenon pointed out by the author. Participants who saw VisiBlends first then removed VisiBlends have much worse performance than a participant who did not see VisiBlends at all. This reminds me of the participant in one of the previous papers said they are afraid to be spoiled by the automatic system leads to no active thinking. So I think this might be one of the cases.

Questions

1. Do you think this system would help you generate a visual blend? Will you use the system to help with your design?
2. It is mentioned in the paper that sometimes we need to compromise to achieve our goals, what do you think about this perspective? Can you think about the examples of this situation?
3. What is the essential tool for you when you are trying to do the brainstorm?
4. For the iterative design process described in the paper, which part is the most significant for you? Which part you think could be replaced by an automatic machine.

Read More

Summary

Advanced graphics designs such as visual blending are a very difficult art to master. Let alone for a singular person to understand and able to create them. Utilizing different designs through various companies to ensure the widest market share and the greatest public opinion requires time, effort, and typically a team of experts to ensure quality. Aiming to ensure a lower barrier of entry for younger or smaller companies, this team created VisiBlends, a crowd sourced framework aiming to ease the burden on teams. This framework breaks down the task of visual blends into 6 different stages to ensure there is validation throughout the whole framework. Throughout the various studies the team had created to measure the results, there was a majority success stapling this usage as a potential tool to generate visual blend assistance through the whole process. 

Reflection

I think this problem domain is similar to the research problem domain in various aspects. Researchers look at the bleeding edge of technology and aim to solve new or old problems using new techniques. These techniques are usually accumulated through various studies, various other methods, or even a new method that uses existing technology to extend previous work. It is interesting to see how the process for creating a new research idea can also be applied throughout the 6 different steps within this teams framework, however with the lab mates, advisor, and committee filling in for the users being crowd sourced within the original framework. 

I do appreciate the machine learning algorithm learning to match the two different pictures (or ideas). This provides several starting places for uses while continuously learning the better matching locations. I believe this could be improved by allowing a human in the loop factor or an override however. Within this kind of work the art is continuously being improved upon in a real time scale. To keep up design artists may have a better idea of new ideas to be used and could be used as an early wave, better feeding the algorithm with new results. 

Questions

• Visual blend problems are potentially a factor when marketing for a new item or product. Blending two (and potentially very different) objects helps to grab the attention of passersby who only give it a split second while either scrolling past the story or going past the ad. Have you had any experience in creating a visual blend for one of your products and did you use this technique to reach a specific demographic? 
• One of the perceived problems throughout Visual Blends is the specific pairing of elements that can be related to each other. Notably within this study this was overcome by obtaining the consensus (throughout the Mechanical Turk Workers) the initial idea that came to their head with the idea. Do you think this could be alleviated by using another Machine Learning algorithm pairing with the stream of information from a social media website (accounting for the social consensus of a group)?  
• This technique could potentially remove part of the need for specific marketing teams who specialize in visual blending. Do you think this program could become widespread or integrated with another bigger system already in use by another company? 

Read More