What does the Pictorial Research Lab do?
Researching and designing pictorial information.
Pictures are everything, from the oldest cave paintings that often appear surprisingly fresh to the contemporary eye, until the latest synthetic images created with AI. Indeed, the pictorial recipes and media have changed but one thing maintained rather unchanged: the human eye. The passion for pictures as a nexus where everything comes together is shared by many different scientific disciplines, and the Pictorial Research Lab aims to embody this multidisciplinary nature. The founders come from vision science (Maarten Wijntjes) and design (Catelijne van Middelkoop), and we collaborate with a great variety of colleagues, companies and institutes.
Yes, when we claim to study and design a wide variety of pictorial subjects, we are quite serious about *wide*. One of our favourite subjects is the study of image collections. We curate our own collections for teaching about visual literacy (image on the left) and also study heritage collections and explore designerly ways to access museum collections. At the other end of the spectrum, here represented by the Lululemon legging on the right, we investigate differences between pictorial expectations (such as product photos) and reality. Here we combine our creative talents with robust quantitative methods to infer about the effectiveness of visual communication. To create some order on this eclectic chaos, here are our focus points:
- Pictures as data
- Pictures and reality
- Pictures in context
- Pictures in eduction
- Pictures as data: Image collections containing unique sociocultural information and also serve as one of the two (the other is language) most important diets of AI systems. There are various interesting challenges concerned with ‘pictures as data’. First, the curation of a picture dataset has both practical (where to scrape images, how to annotate metadata) and ethical (how to cope with bias) challenges. Secondly, while ‘distant viewing’ analysis is becoming the standard approach to analyze picture datasets, it is still an open question what combination of computational and behavioral (crowdsourcing) facilitates this analysis. Thirdly, there is a conceptual challenge of how to use pictures as material to design with.
- Pictures and reality: Pictures offer promises that are sometimes abstract or metaphorical but can also be concrete such as in online retail. When the order arrives, the expectation is compared to reality and ideally the consumer is not confronted with unexpected surprises. This challenge can be met by using various visual techniques (photos, movies, animations, renderings) in combination with rigorous user studies that aim, for example, to quantify perceived material properties. Also, pictures have been used to document reality for example in press photography, or forensic contexts. However, the trustworthiness is threatened by contemporary image synthesis which demands novel approaches to verify visual data.
- Pictures in context: Depending on the surrounding, a picture may take on different meaning. One of the most well known contexts is the computer or smartphone screen and many pictures are specifically made for this (e.g. social media pictures). When it comes to artworks, NFTs or digital art match well with the digital context, but analogue art such as paintings and sculpture match better to a physical, museum context. Many contextual crossovers exist (e.g. NFTs in galleries and digital reproductions of paintings) which offer an interesting area of both research and design. More in general, this theme focuses on the picture and its situatedness in both space and time.
- Pictures in eduction: We have recently developed the Vision and Depiction framework. We apply this in both teaching and research, we even organised an awesome conference about it: https://visionanddepiction.github.io
