Multi-Sensory Design Towards Inclusion and Access

Many multi-sensory exhibitions aim to create immersive experiences, yet remain inaccessible for a variety of visitors. This session will examine multi-sensory exhibition design and how engaging various senses can provide not only a rich and immersive experience, but can facilitate greater accessibility, widening audiences, increasing demographics, and facilitate inclusion. Examples from different museums will be examined for not only what they achieved, but the potential they hold (or held) for increasing inclusion and access.


Unknown Speaker 00:00
Alright, let's get started. How's everybody doing? Good. Not bad hanging in there. So, today we're going to be talking about multi sensory design. My name is Serena Brahm, this is Corey Timpson to my left your right. And a lot of our work concentrates at the intersection of experiential design inclusive design, multisensory on both sides of the house things that our visitors are facing, and that people interact with on a daily basis, but also in terms of operational aspects and efficiencies they're in.

Unknown Speaker 00:38
Yeah, so yes, pretty quick presentation. So we're, it's going to be a fast fly through just I mean, some premise setting, I guess, before we get into some of the examples we want to talk about is this idea of the ecosystem. So whenever we're thinking about experience design, or you know, inclusive design and accessibility, we're always thinking about the ecosystem that we're operating in. So these many different aspects that operate with one another, within the environment that people find themselves in. So we know that our visitors, our patrons, our audience, our users exist without within both physical and digital spaces like this is just the reality of today. This is the ecosystem, we're thinking about many variables combined to define the user experience within these spaces. So we want to think about what all of these variables are when we're thinking about inclusive design and accessibility, in particular, and consideration of either of these spaces cannot be done in isolation, we have to think about it all. When visitors are in the gallery, they have their phones with them, they're doing a lot. When our staff and our people are operating museums, this is a very rich, blended ecosystem as well. The ethos that we operate from, is that rather than designing develop something, and then figure out how to make it accessible, we designed with an inclusive design methodology in place where we're considering all vectors of human difference at the outset, you've probably heard of accessibility and universal design, what are we thinking the distinction is when we think about inclusive design. And it's really about all of those vectors beyond the technical vectors of human difference, like physical and cognitive ability, disability, or really thinking about everything. Every vector of difference could be ethnic, cultural, geographic, etc, etc, etc. The premise we operate from is that inclusion exclusion rather, is the result of mixed mixed. I think I'm trying to speak too quickly, I'm trying to keep up with you doesn't work belfry, you'll see what I mean when he starts talking is the mismatch between design intent and user intent. So our design intentions are responding to strategic objectives, learning objectives, etc. The design is trying to realize those, but users are coming in with their own intentions of things they're trying to do. And when these intentions don't line up, that's when we get exclusion. So resolving these points of exclusion yields not only inclusion, but immersion and innovation, and then things like interoperability, scalability, and, you know, rich and meaningful experiences. The facets of our approach is we look for a mixed interaction design, so passive, we're all familiar with that at museums, read, watch, listen, read, watch, listen, active, get the users to do something, get the visitor to, you know, participate, to collaborate, and interactive. The difference there is not you know, push a button to read, watch and listen, which are all passive activities. It's really dialogic, it's back and forth. It's like a conversation, you don't know what you're gonna say, at least I hope you don't know what you're gonna say until I'm done speaking. And I don't know what I'm going to say until you're done speaking, that kind of dynamic back and forth is what we think of when we think of interactive. So the facet of mixed interaction design is really important. It doesn't mean they're all equal, doesn't mean we need an equal amount of passive activity as interactive activity. But it means we do want some kind of variety, analog and digital blends mixed in transmedia storytelling is a facet that we really focus on stylistic variety, as an example, documentary and photographic versus a lot of illustrative and animation. So having some kind of variety, providing different entry points to Content and Experience for different people who have their own individual preferences, learning styles, etc. Immersion how to immerse someone in something can be done through lighting, it can be done through scenographic, it can be done through anything. But this idea of immersion is something we pay attention to. And then multi sensory design. And that's really what the focus of this talk is about in terms of inclusion and accessibility is this last facet.

Unknown Speaker 04:43
So in terms of the senses, I don't want to spend too much time on this, but it's important to think about things just beyond the audio and the visual stuff, right? So we have vision, we have audio that I think most of us are familiar with in terms of vision and hearing. There's things like somatosensory, right so so things like cutaneous responses that Feeling of things on your skin. This can be broken down into things like haptics, vibrant tactile, feedback, texture, right? Different kinds of textures. And these all have, by the way, different effects for different audiences. Some people might be sensitive to particular textures, but it's also a rich way of conveying information. Right? You also have senses such as, for example, proprioception, right? That sense of your body independent of whether or not you can see it, where is your pinky right now on your left hand, right? You know where that is, without looking at it, right? And so this is the sense of proprioception. And you can play with that as one of these dimensions. When designing multisensory exhibits. Other things like balance, of course, come into come into play there. Lastly, you know, you have things like Osmel reception, right? So this idea of thirst or hunger, those things that kind of border on maybe like a little bit of a pain threshold, but there are different feelings that we have different, different urges that we have, that we could we could perhaps play with just something to be thinking about. gustatory, right, like, how do things taste? Right? And how does that intermix with other things, for example, like olfactory, right, and what are what are things smell like? And what kinds of things can that trigger as part of elements in terms of rich storytelling,

Unknown Speaker 06:15
and maybe just to like, highlight that. Certainly, the first three senses that Siena spoke about are the ones that we probably experienced the most in terms of experience design, particularly in exhibitions, and the cutaneous, certainly not as much like experience around temperature. Although isn't that interesting how you can convey meaning through temperature, just like you can through texture.

Unknown Speaker 06:39
And it's also a great analog because somebody who might not be able to see the differences in color, because it's a continuous thing from light to dark could appreciate the difference between of sunlight shining on your face when you're under a tree and half of your faces in the shadow. So it's darker or colder, and half of your face is lit by sunlight, so it's warmer or brighter.

Unknown Speaker 06:57
So we're going to talk about some examples to sort of try and concretize these concepts and like show where the points of consideration are. And we don't have all of the answers for these projects. These are all projects we've been involved with in one way or another. But you know, this is where we want to lead to the discussion, so tactile, and audio navigated photos. This is a shot from an exhibition that we produced, called sight unseen, where the photography was all produced by blind photographers. When we presented this exhibition at the Canadian Museum for Human Rights, we translated the photography into tactile versions, you can see there's a little silver.on the side of this man's face. And when you hit that with your finger, you get an audio description of what it is you're feeling. So in this example, we took a purely visual exhibition, a photography exhibition, and we made it multisensory, we made it tactile, and we made it audio as well. This was a very rich, multi sensory experience for all of our visitors. This was also an extremely accessible experience for our blind and low vision visitors. We didn't just do this once, we've now done this four times. This is another exhibition called points of view. This was a also at the Canadian Museum for Human Rights. And what it was, was a crowd sourced exhibition was a national call for people to contribute their photos based on certain themes. This was the award category award winning photo from the inclusion and diversity image. So what was really amazing about this was, clearly the artists intent was to highlight the woman's face and hijab in her hands, she's dressed all in black, and she's on a black background. Translating that you can see that tactile audio described version on the right into a tactile relief, presented me with a bit of a problem, I wasn't sure how to do that properly, because I didn't want to break the magic of the photo and that artists intent, but we had to sort of outline her body in order to have a navigation system for navigating and feeling the photo with your fingers. So first thing we did was contact the subject of the photo and say, like, are you okay with the fact that when to translate this, people are going to effectively be putting their hands on your likeness, she was cool with it. The artist was the photographer was cool with it. Also, what ended up happening is, she's almost like at a one to one scale. So you end up holding hands with her when you're navigating the photo. And it's an, you know, it's a very intimate gesture to hold hands with someone. And this is in the inclusion and diversity category. And we didn't plan on that happening. But that happened because we were trying to figure out how to make this experience more accessible and inclusive for everyone. So like happy accidents or innovation does happen when we think about these things. So we've done this now four times the Tet Offensive exhibition at the museum, and the Rohingya exhibition back at the Canadian Museum for Human Rights. And the big consideration around doing this that we find is not just an easy question to answer or a solution to find is respecting artistic intent while translating the art for In one medium to another in order to increase inclusion and increase multisensory design. The next example is the multi sensory installation designed from the senses, designed Beyond Sight exhibition at Cooper Hewitt Smithsonian Design Museum. So this is one of the installations, this was about tactile feedback, where, you know, visitors pass through a space and they encounter these fuzzy balls, which, you know, provoke different effects. And the idea is like, you're actually navigating through this and causing responsive environment through tactility. This is another part of another installation where on the right hand side of the photo, it's all based on navigating content through smell. And on the left, it's through texture. This is a close up of the smell. To give you a better example, these are coffee mugs made out of coffee grounds. So you were actually accessing the content through the scent of coffee. This is wallpaper that was a tactile wallpaper. So the image and relief was actually what was conveying the design of the wallpaper. This was an installation where it was audio visual and vibrant, tactile, you sat on these chairs, he listened to the headphones, and you saw the visual projections. But what if you are coming into the space using your own chair? How do you get the vibe or tactile experience? Well, they created a pillow that did the same thing. So if you're in a wheelchair, you can hold the pillow. But the thing here is that this installation was visual was auditory, olfactory is vibrant, tactile haptic. But in some of those installations, I put the fuzzy balls for people to sort of catch them with their head properly, if you're in a wheelchair, you missed out on that experience. So the sort of consideration here is how to really create multi sensory installations that just don't exclude one group in service of another. And that was something that, you know, is a really, really good sort of problem to try and consider with that exhibition.

Unknown Speaker 11:58
So the next one is on tactile reproductions. So this is out of the Andy Warhol Museum in Pittsburgh. And these are tactile reproductions of a Warhols works, the soup can is one of the images that we one of the things we see in the image on screen, as well as the Brillo box. Now the Brillo box is interesting, because it's not just a flat work, it's actually five sides of the Brillo box all individually lasered into a material called asset Hall and stitched together. So it's on a pivot, it's like on a lazy Susan, you can spin it around and really explore this tactically, right. These are all mounted. They're wheelchair accessible. They have Braille labels, they have guided tactile descriptions. So they only have visual descriptions, which is an explanation of what's there for maybe somebody who can't access that information visually. But they have guide a technical description so that as you are touching the thing, you are being told what those textures are mapped to that thing that feels like Polish concrete means a darker color, right? Or certain things like something that is in the candles, for example, the raised lettering implies a brighter color. And so this mapping that was decided on was a very, it was a very tight mapping to the visual information. And the point of consideration here was tactile clarity, versus aesthetic accuracy. So tactile clarity, for example, would mean well, you would space things out a little bit more, you would make things a little bit more pronounced so that it's easier to touch because vision is a much, much richer fidelity sense than touch is. But they didn't choose to go with that on purpose on this project, because the aesthetic accuracy was important, but then how did that can mitigate it that got mitigated by providing these guided tactile descriptions that help explain what you are touching and then provide that sense of exploration. Right. Now historic house museum, so on the historic house museum, just out of a Joplin, Missouri is Joplin, historic neighborhoods. This is architecture from about 120 to 135 years ago, so turn of the two centuries ago. And the idea here is that there is a lot of rich detail right from just the the the architectural aspects, the color aspects, the historical context. And a lot of times we concentrate on the visual sense of all of this, not thinking that there's a staircase with a lot of intricate detail, but at the same time, that staircase, well, it's a wonderful opportunity in terms of tactility and different aspects such as such as texture, like we've talked about, it's also a barrier to somebody who example is a wheelchair user, right? And so how do you then come you know, how do you juxtapose these two things? So for example, one of the things that's being done here is a an elevator is being installed. But it's being installed in such a way that is not just, Oh, we're doing this for accessibility. It's being installed in a way that's part of the narrative that's actually going to be conducive to onboarding that actually has a story behind it in terms of specifically where it is being installed in the house. And so we're able to then take this, this distinction and use it as a benefit instead of something that's, that's, you know, necessarily an obstacle. So this is one way that we're distinguishing between historical accuracy and providing this kind of inclusive

Unknown Speaker 15:20
experience. And maybe just to say that, you know, the architecture and construction teams on that project are, you know, really sort of in it to win it as well in terms of inclusion. So they're, they're not like dropping the elevator in the middle of like, first floor level floorplate. Like they're actually integrating it in a way that doesn't take away from the historical accuracy. But the way to contextualize the value spend the return on investment of creating that putting an elevator into this, you know, super old house is that like to Siena's point, it's going to be used as a multi for multi modal modal functions. Like it's not just a people mover, it's actually a programmatic space that will be used to satisfy certain experiential objectives as well.

Unknown Speaker 16:03
And it's important, like when we were there visiting, you know, they're showing us like tiles from the roof. And the one of the things that happened is one of the main persons on the project ran over because they were about to throw away all of these tiles they had collected from the roof. And he's like, No, these are amazing. Why would you did these tiles are, first of all 1000s of dollars. And second of all, are amazing aspects that can be used in components of exhibit design later, to offer ways of not only providing insight into the process, but also that richer, multimodal experience, because this is something that you can't go up on the roof and touch but because we have the tiles from the roof reconstruction, you can bring that available to make that available to everybody. In terms of some final thoughts, because we do want to leave some time for for questions here is using multisensory design is really a benefit. It's it's not a burden when we when we have proper intentional decision making around it. And we're doing it in service of inclusive of inclusion, right. And this is not just a cliche, this is not just something you know, that feels good to say this is this is real, we encounter this on a daily basis. In our work, happy accidents do happen. Cory highlighted that, that thing with Holding, holding hands, we just came across that while writing that guided tactile descriptions for that piece. And so these are the kinds of emergent effects that come out of doing the work of inclusion. Lastly, this is not a zero sum game, we're not taking away from some other experience or disadvantage in some other population in service of another. These are this is something we feel really passionately about that it's that whole rising tide raises all ships concept is completely true in terms of inclusive design. In terms of other things to think about. multisensory design does not guarantee inclusion, right? That's not the same thing. multi sensory is great, multimodal is wonderful, but it does not guarantee inclusion, multi sensory design does not it haptics does don't always, also, like provide all of the accessibility goals that you're you're you're trying to achieve. But if you do that, and if you use multi sensory design, in service of inclusive of inclusion, through immersion, through motifs, through multimodal exhibits, etc, then you're able to arrive at a place through deliberate decision making that does achieve inclusion that does achieve accessibility that does raise the equity for all visitors. And that's really, really critical.

Unknown Speaker 18:39
I just think that final point is that we have a pretty hard stance on inclusive design and accessibility to begin with, and there's like no good reason to not do it. We you know, whether we're talking about the experience for our visitors, whether we're talking about the operational sustainability of the underlying infrastructure, protocols and process that we're developing in order to be able to satisfy that experience and manage our operations ongoing, whether we're talking to, you know, the sort of sea level of the museum and demonstrating the return on investment that happens in terms of strategic performance indicators, like audience, reach audience, just numbers, audience demographic differences, you know, visitation and repeat that, like all of the earned revenue. You know, one in five people has a disability, one in two people over the age of 35 will have a disability in their life. You know, I know the number in Canada, I don't remember in the US, but it's like $40 billion discretionary discretionary spending like this is a historically marginalized community. I'm thinking just about accessibility at this point, like that is you know, that it has been historically marginalized, that can really contribute. So all of the there's no good reason to not do it. But if you're doing something multisensory, like there's even less of a good reason not to do it,

Unknown Speaker 19:59
and it's implied Given that a lot of it comes from deliberate decision making, not necessarily additional resource spent, if you're already doing something multisensory, you can raise the the equity and raise the inclusive aspects of that by a great deal and include way more people by just making a few decisions a different way, six months before open. And that's really at the heart of what we, you know, what pains us when we see those decision points not being considered upfront, it's oftentimes not a matter of spending more money. And so that's, that's really important in terms of deliberate decision making. I think we have some time for we have six minutes for q&a. Yes. Oh, sorry. Thank you so much for that wonderful presentation.

Unknown Speaker 20:44
I saw some accessory artworks. And we also saw some artworks that were sort of translated into other like modalities, decryption and touch objects. And I'm curious if you have any stories that you've kind of talked about engaging one of the subjects of a photograph and that process, and maybe a little bit about the artist, but I'm curious if you have any other stories about sort of including artists in that process.

Unknown Speaker 21:09
So I'm one that I'm reminded of is on a Kiara tilava, Telly. Previously, the MCA Chicago, led a series of artists led tours, artists LED touch tours. And what was interesting about this is it's contemporary art. So the artists are still alive, right? And they kind of come into this like, Okay, I'm a visual artist, I'm sure I'll lead some blind kids around, you know, like, there's sort of, like this nebulous Ness around what's, what's the intent here and such, what ends up happening is after that process, was, first of all, we had more than one artist essentially break down into tears, because it was such a moving experience for them. Because they had people who they thought were absolutely not anyone who could appreciate their art, appreciating their art in different contexts that they could have never imagined before. And sometimes this is accidental, like, particularly thick brush strokes, or different aspects of sculpture or what have you. And sometimes it's through intentional modifications or intentional enhancements on behalf of the artist or, or the museum. And so what ended up occurring from that is a couple of takeaways. One was the artists started incorporating this these aspects of, oh, there's all these different facets I haven't been thinking about into their practice. That's number one. Number two, is it offered a connection point between the artist and this community. And that I think was just really meaningful for everybody involved in a deeply emotional way that frankly, we wanted, but really didn't predict. Does that answer your question?

Unknown Speaker 22:47
Yes. green jacket.

Unknown Speaker 22:50
Questions about? So related to what she was talking about? Photography? Seems like that would be a good idea, the material, sort of the review type of installation like you had, so maybe you could do it a little more about the cost? Or does it also sort of create this whole sort of foggy area where people like to have sort of say you did it for multiple photographs? How does that work and salary really like to add to that first?

Unknown Speaker 23:25
Sure, um, we've done a few different sort of creations in that process. We're working with different vendors. I mean, in terms of the clogging, like sort of, like visitor flow and stuff that's never actually been presented as an issue. I think the densest show that we did like that was the Tet exhibition at the museum, because they also had an event for the National Federation of the Blind. There was like 5000 people at the event, and that exhibition was like the highlight, I'd say, and still not really an issue, but um, sound when you're doing the audio overlays, and how you manage sound is something that needs to be carefully done. With the TED exhibition, one of the really interesting aspects of the evolution of that, of that technology was there was oral histories from the Marines overlaid. And that prints like I'm getting goosebumps right now thinking about it, because it wasn't just describing the image. It was actually, there were different audio choices. And you could hear the Marines telling their story as you're exploring, you know, the tactile image. So that was extremely phenomenal. experientially, I'm not sure if it creates

Unknown Speaker 24:44
it creates a shared sense of Wow, so one of the one of the metrics that we use in our work is do different audiences with different abilities go Wow. At the same time, do they experience delight at the same time, not for the same exact reasons, but for equitable, same reasons until like the one year We're talking about, I could imagine a very poignant scene in a video that would have provided that sense for you, Cory, because you can see the video, but not for example for me, right, but by touching it and something that you happen to be able to do as well, it's multisensory, but it's also that shared sense of WoW amongst two different populations. or longer. It's almost always necessary for for tactile because tactile is rarely independently clear on its own. And it's a possibly an offline discussion, I'm happy to get into, but almost always, when you have something tactile audio can only serve to enhance that.

Unknown Speaker 25:37
And we won't really get into it in this talk. But, you know, actually, Cena and myself and Bruce published a paper at MWC, 20, I guess, about like the sort of work breakdown structure and project management around like this type of development. And just like we're talking about working with artists, in the creation of the installation, there's also like working with prototyping and testing groups, and what is the user and what is an expert and like how we work with those groups as well. So in finding the right relief, and how we're going to provide like tactile navigation, it's not necessarily it's not just that we created a standard, the real art is actually in the application of that standard, because it's not always going to be consistently applied in the same way in order to achieve the same outcome. Orange, orange, you're good? Yep. Yes.

Unknown Speaker 26:26
I was wondering if you might be able to talk at all about the visitor engagement, multisensory, that you did, in their relationships, or museums or habits change? That,

Unknown Speaker 26:41
yeah, Should I go first? Or did you want to go? So I would say that part of this is about to quote, I think one of the most brilliant statements on the subject, Susan Chun has the statement that, you know, by by doing this work, the work of inclusion, we are acknowledging this population, whether it's people with different abilities, or what have you, as creative beings and welcoming them into a global discourse. And that's really important to me. And so I think that when you talk about the the visitor engagement aspect, part of it is about building allies, part of it is about welcoming this community and, and recognizing that yes, there actually is something there for you, or for your loved one who may not be able to use the stairs or see the paintings or hear the music, to be able to enjoy both together and by themselves. So that's, that's one aspect of visitor engagement. The second is that it helps build an a consistent sense of expectation. So if you are making a commitment towards doing the work of inclusion, one of the things is that visitors over time, are then expecting that to be in the work to be inclusive, right. And I think that's really important. Because otherwise, what you end up with is one offs, you end up with some work that happens once you get a little bit of a blurb or a press sort of boost from it, and then it goes away. And that's a flash in the pan, it's not actually doing the work of inclusion. So sometimes it's slow and steady, as opposed to something very flashy. But by building those relationships, and welcoming people in Cory mentioned, the difference between testers, and experts, and advisory panels, and so on, and so forth. All of this is actually a multifaceted way of making sure this is sustainable with respect to visitor engagement, as opposed to a one off with respect to visitor engagement. I don't know if that exactly answers your question.

Unknown Speaker 28:30
Also, I think the summative evaluation on the actually, it's a project that we didn't show here was a artifact based project where we added tactile recreations to every artifact in the show. And will the dwell time was, you know, way higher. And we just wanted to confirm that that was because it was an enjoyable experience, not because it was frustrating. And, and, and that actually built a huge amount of loyalty. And then, you know, we had like, we have like quotes, like, you know, there was no reason for me to go to a museum before and you know, now there is, you know, like these, like, really amazing quotes. And then, you know, for the sight unseen exhibition, there was like no promotion budget for that exhibition, but we earned $8 million worth of media through that exhibition. So like that's because there was and then more people came and then that revisit, revisit repeat visit and then like, our membership went up, you know, like it was substantial. Three minutes over, but you know, no one's kicking us out. So if there's any more questions alright, thanks a lot. Thank you. That's

Unknown Speaker 29:44