Introducing Cooper Hewitt’s Interaction Lab

With the introduction of the Pen in 2015, Cooper Hewitt established itself as a pioneer in digitally-integrated visitor experience. Four years on, we’re creating what’s next. Rather than working behind closed doors, we’re opening up our design process and living our mission – “to educate, inspire and empower people through design.” The recently-launched Cooper Hewitt Interaction Lab will introduce a user-centered process to the museum’s visitor experience design process. We’ve designed the Lab’s activities to bring our strategic plan to life - emphasizing inclusive, digitally-integrated design. As a space for research, experimentation and assessment, the Lab will keep with the pace of new technological development, grounded in visitor insight. In addition to an already collaborative design process, we will also launch a slate of public programs to involve an even wider community. This year’s MCN theme aligns beautifully with our aims for the Interaction Lab, which itself will be a collaborative, transparent interface between the museum, art/technology/design communities and the audiences we’re designing with and for. This session will provide an overview of the Lab, with emphasis on “Lab as interface” and the methods and approaches we’ll use to bring this to life."


Unknown Speaker 00:00
Hi, everybody, thanks for not being by the pool, we really appreciate that. I did have to drag a couple of people in, I'm not going to mention their names Adam and Guna. But we're so pleased to have you here this morning. And we are going to talk a little bit about a new initiative that we have just launched at the Cooper Hewitt in New York. My name is Carolyn Royston. I'm the chief experience officer at the Cooper Hewitt. And this is my colleague, Rachel Ginsburg, who is the director of the Interaction Lab. So I'm going to start off by just talking a little bit for a few minutes about the lab, sort of how it came about. And Rachel is going to take over and talk about some of the initiatives that we've been working on over the last few months. So I've been at the Cooper Hewitt for almost two years. And in when I joined the museum, there have been huge digits digital initiatives that had been launched when the museum was reopened after being shut for two years, in 20, December 2014. And

Unknown Speaker 01:27
as technology and the world moves on, technology is starting to come to the end of its natural life. But also the museum is at an interesting place, and that it's coming up to five years since since reopening, and makes a great inflection point to think about, you know, what, of whatever we wherever we come from, what have we learned from that? And where do we want to go next? And so that was really the the sort of question that I was presented with and thought about. And so you'll know about, I'm sure some of you about the Cooper Hewitt pen and the digital tables and the great success of that project for the museum and putting it on the map in terms of really being at the forefront, I guess, with museum technology. And the museum has learned a lot from the pen over the last four years. But as I said, it's sort of coming to the end of its technological life. And we're at this moment where we want to think about what next. So I really spent a lot of last year thinking about how to do that. I built a new team at the museum. And we started to put together a roadmap for all of our digital experiences and initiatives, and to think about how that roadmap fits together, both in terms of our online experience, in terms of the spaces in the galleries, but most importantly, I think about a holistic visitor experience. So how do we think about the visitor journey from before they come to visit the Cooper Hewitt and their encounter with our website to when they come through the door to what happens in terms of further engagement. And so our digital roadmap is really placed within that context of a total visitor experience. And, and that enables us to think about, you know, the human side of that, the physical side of that, and the building and the importance of the building and, and its place in that visitor journey, but also about the digital experience. So this was really an opportunity for us to look at that all together and to think about where do we want to go next? And also an opportunity to think about, you know, how do we want to work and evolve our practice as an organization. So it really has been a significant inflection point for us. And out of this, really came this idea of creating an r&d space. That was a space that really enabled us to think about very strategically about what we want of his new visitor experience to be at Cooper Hewitt, how do we want to evolve, where we've come from, and where we want to go. And to think about that, you know, as a as a place of exploration, as it were a design museum, that gives us a perfect opportunity to think about the process, to think about how to actually design a visitor experience. And to think about how to do that in a way that is embedded across the organization becomes part of our practice, and really places less focus on what's the one thing that we're going to do next to really thinking about a whole range of experiences. And so, we have a set of objectives that we have come up with. One was to focus on innovation, but not just for its own sake, which is really why we chose the word interaction. lab over an innovation lab, we really wanted this to be really focused on on the on interactions, and not to be driven by technology, we want to put the visitor at the center of our design activity. We want to identify and create opportunities for new interactions, and to think about how those interactions work across that whole visitor experience. We really want to embed a prototyping culture into the institution. And this is a really, really important point because as I said, we're a design museum. And it's I think, should be at the heart and the ethos of what we're doing. And be able to talk about prototyping as a process. And as part of what we're doing with less emphasis on perhaps the output. And we want you to think that, as I said, holistically across all platforms and environments, and to live our mission as a design museum. So we had a very purposeful set of objectives around what we wanted to do with this lab and,

Unknown Speaker 06:04
and sort of came up with this sort of desk definition, I suppose for it, the Interaction Lab is reimagining the museum experience for the 21st century, as a space of inquiry and experimentation, the lab keeps pace with emergent technology, while remaining grounded in data led audience insight. We're bringing interactive design methodology to the very heart of our visitor experience across digital, physical and human interactions. And I should say as well that were a lab without walls, it is not a physical space at the museum, it really is a space where we are encouraging collaborations, and Rachel will talk more about this, but collaborations in the broadest sense about who we're working with, from from tech firms, to individual designers, and technologists. And we are very excited about about the lab and about the possibilities. We are really seeing this as an emergent space. We have we have a plan. But that plan has evolved and changed, even in the last few months, you know, considerably and our thinking has changed considerably. From you know, where Rachel and I sort of started this this project. So I'm going to hand over to Rachel now he's going to talk a little bit more about the work that we've been doing.

Unknown Speaker 07:20
So hi. So yeah, we have some some fairly lofty goals. And we have some fairly big sort of shoes to fill and, and obviously, a lot of work to do. And so I've been at the museum since March of this year. And we just announced the lab recently. But we've been sort of quietly working in the background for a while just kind of getting our feet under us and figuring out really what the opportunities were kind of consolidating knowledge and information about what had been happening so that we could sort of chart a path forward that actually brings as many stakeholders as possible with us. And so the way that I know to do that, I mean, when I come from a practice that's quite emergent, more sort of storytelling driven, I'm new to the museum world, which I am those of you who I've met over the past couple of days, that's typically the first thing out of my mouth, mostly just because I'm excited to be able to be around people who who aren't new to the museum world. So I can learn from all of you. And so, brainstorming is really like the first like for me, if I don't know what to do, I get a bunch of people together and talk about it. And so that's that's basically what we started with what we started with was a couple of brainstorms. And we basically just reached out to all of these designers that we know and you know, for some very generous folks in the museum's sector, and then also brought in a whole bunch of Cooper Hewitt staff, and and just kind of convened ourselves around. One will sort of two questions. One, what is your ideal museum experience, like look and feel and smell? And, you know, how does how does it make you feel? What does it look like? What happens? And so that's kind of one question. And then the second question was, now what is your ideal do design museum experience look like but not just look sort of all of the various sensory engagements that one might have with a museum experience. And we did that a couple times. And we, we sort of did some conversation in a room as you see. And then and then sent people out into the galleries to continue the conversation in situ and then brought everybody back and did some some discussion. And so when we pulled together the themes that emerged from those brainstorms, I mean, I don't imagine that that really this will surprise anyone in this room, but the idea of multi sensory multimodal interactive design so design that engages your senses that you can engage with in multiple modes, and also this interactive, a desire to amplify exploration and discovery. So to really give people a space to own the things that they learn in a museum space and designing experiences that help them discover layered accessible storytelling so like excessive ability is really critical, not just based on, like physical ability or mental ability, but also based on preference to, um, that's the position we're taking on accessibility, Cooper Hewitt. And the fact that it's layered. And actually, the layer language also came out of curatorial too wasn't it wasn't something that was just driven by a bunch of outside designers. But in multiple conversations we've had with curators that desire to create a sort of layered approach to storytelling, understanding that some people want to go deeper, and some people less and there's many different spaces to do that. Multiple Guided Pathways arose out of a conversation about Choose Your Own Adventure, which is the language that came up a lot, but people but the truth about museums is they are the most Choose Your Own Adventure possible, because you literally just walk in and you can do whatever you want. But when people talked about Choose Your Own Adventure, what they were talking about was choosing from adventures that had been designed that they could step into that they didn't actually have to construct themselves. And so we transform that language into guided pathways, and then just ongoing engagement, like how is the visit extending past the point of being in the physical museum? So then, we had this internal kickoff, sorry, I'm just keeping time, we had this internal kickoff, where we basically had a many sessions with different folks about how things were like, what what is the shared reality that maybe isn't so shared, because we haven't actually all sat in a room and really talked about it. So there was some of that there was also exploring constraints and affordances of the space, we were talking about visitor experience, and how to collectively and collaboratively define visitor experience. So that we're all on the same page when we're talking about these things. And so this was really like, the time in the space where, and we worked with the incredibly talented folks at Franklin, green and Webb to do this, they facilitated, designed and facilitated for us. And the group that you see on the bottom right are some advisors that we pulled together to support that conversation, um, that they acted kind of as listeners and provocateurs in the experience, which was incredibly helpful. So this sort of gave us that kind of internal understanding to continue pushing it forward.

Unknown Speaker 12:09
Then lots of workshopping, so I'm gonna go through this kind of quick. So then this is sort of the space where we recognize that we have this pen, we have this sort of legacy of collecting and creating that the pen really did shift the paradigm in the museum in terms of centering interactivity in a way that's actually quite profound. And now we are posed that paradigm shift. So like what a futures for collecting as a primary interaction look like. And when I say collecting, I'm not talking about art collecting, I'm talking about the interaction that drives much of the pen experience, which is collecting objects, and then the value of collecting objects being that visitors are walking around telling us what they like, and we give them the agency to collect that, which is fundamentally what the pen did. But what does it mean to take that as an interaction into the future? So exploring how might collecting be always at the core at the as a baseline? But we introduced some other kinds of design questions in this workshop. So for example, how might collecting be the passive result of another kind of interaction, that collecting isn't the end. But collecting is a service that we provide when someone indicates interest in an object, for example? And so using that question as a jumping off point, or how might having collected transform a subsequent experience in the future? Because that's data that we now have about our visitors? So how can we use that data to continue advancing the work that we're doing and engaging them? And then, last workshop, we did a workshop with a group from Facebook called the AI and new experiences team. And it was interesting, because I mean, in addition to sort of having to think about like, Okay, this is Facebook, okay, this is AI, obviously, we have to sort of have a conversation about ethics. And we have and that team, I have to, like, give tremendous kudos to that team, because they are incredibly talented, incredibly conscientious people, which was really gratifying to experience obviously, given all of the politics, etc, the issues with Facebook around privacy and exploitation. But what was profoundly interesting about this workshop was that the ways that the folks from Facebook, were talking about AI was pretty fundamentally different from other kinds of creative AI conversations I've had. And for those of you that don't know, weren't in the session yesterday, I'm an artist and have done work with AI and I'm in quite a lot of conversations about creative applications. And Facebook came at it from a really interesting angle around Korea, you know, creating social connections and and sort of deepening people's relationships with themselves and other people visa vie AI, which is actually quite closely connected to the artistic work also that I'm developing. So it's really interesting to explore that together. Just Even to expand our thinking. And I would say the interesting example that this workshop provides was really engaging a partner that is expert and thinking about things that we are not necessarily expert at at the museum and having a very focused conversation around applications of that thing.

Unknown Speaker 15:16
So, this has been the process, right, we had this brainstorm the triangle, sort of like this is the widest and we're narrowing down. So we started from the top, the widest approach, this is a brainstorm, we need to talk about what ideal museum experiences are like, then we moved into an internal kickoff, which is really about establishing shared understanding, exploring constraints, and affordances. And sort of honestly having a little bit of group therapy. And then these workshops, these specific workshops are much more granular and much more focused on a certain topic. And surprise, we don't get to a solution. After all this, we just get to prototypes, because this is not, as Carolyn was talking about this, we are in an emergent process that we are, I mean, thankfully, I have to say this in front of all of you, I have a boss who is senior enough at the museum to be able to shepherd that process through. And I know that that's not the case for everybody. So, I mean, obviously, there are other kinds of conversations to be had. But the purpose of this process is to land us in a place where we have some stuff to test, where we have some some ideas to explore. But we are not at a place yet where we have solved this or where we have an answer. So really, the sort of first seven months have been about framing, they've been about surfacing these various things. So we went from themes to design provocations into narrative frameworks. And I'm going to present those in just a second. And a really important thing to note is that we got to a place where we have some ideas about prototypes. And what we will be doing with those prototypes is intensive user testing and evaluation and co creative development as we go through. And I'll talk more about that in a second. So in thinking about design provocations, so how might we so one of the things that the Interaction Lab is also really trying to do is to inject design practice into the design museum, and though there is some of that it's definitely not diffused throughout the whole museum. And so we really try to frame things in the language of design. So So framing design questions as design questions, and the how might we structure is incredibly useful for that? Because it just means a different thing than how can we? Or how should we? Or how will we, because this is still a speculative space? So how might we extend our visitor experience beyond the walls of the museum? How might we use storytelling to make meaning with audiences? How might we be transparent about the design of the museum experience and use it to engage visitors accordingly? How might we use immersive approaches to expand access to the collection, and then how might we build on the legacy of our current interactive approaches, because we do have this strong legacy and, and we've, we have the privilege of being able to take some space from it and really look at it critically, and take away the things that have worked very well. And then make sure that we don't repeat the things that haven't worked so well. And so this is where I'm going to talk a little bit about narrative frameworks. And I'm just before I do that, I'm just going to kind of introduce what that actually means. So narrative frameworks are really just a story that allows you to think about a specific kind of problem or situation. It's like a very fancy language for just like, This is a story that I'm using to contextualize my thinking around a specific thing.

Unknown Speaker 18:29
And really, I mean, provocations, narrative framers is all kind of the same thing. So the first thing that we're really thinking about a lot is the idea of the museum as a platform. And we are, if you were in the previous session, and heard about strategic plans, we are lucky enough to have a strategic plan. And though there are things about it that I think remain at times a little confusing, one of the things that's really inspiring is this goal, one of our strategic plan, which is to be a platform for design. And so the Interaction Lab is actually looking literally, literally at the museum as a platform for design. Because right now we have a very well defined approach. At exhibition design, we have a very well defined approach at designing the specific pieces. And we have this pen that links it all together. But when we're thinking about what's next, and we're what we're really talking about is designing connective tissue for those various exhibitions, and a Cooper Hewitt, our we have a constantly rotating exhibition calendar, we have a fairly small physical footprint, and we don't have things that are permanently on display, except for the building itself. So how can the building be a platform? What does it even mean to be a platform because platforms do all kinds of things, platforms, elevate platforms, launch platforms support, so also really exploring all the various ways that the museum is a platform for education for exhibitions for the work of emerging designers, all of that stuff? Next, the next really cool museum. Museum is interfaith So like the museum is a place where, where visitors interface with our collections where visitors interface with the building where visitors interface with each other, we have many interfaces inside of the museum that support that. But if we think broadly about the idea of what it means to design an interface, what does it mean to design a connection between two things and make sense of that. And that's really something that museums are uniquely qualified to do, because we are such centers of knowledge and authority about that knowledge. Museum is prototype, which is Carolyn, this is like one of Carolyn's favorites. So how can we look at the museum as a living prototypical space? How can we let go a little bit of like the work in progress aspect, the sort of the fear and the stress about exposing work in progress, that everything has to be finished and perfectly polished in order to put it out there. And it doesn't mean that we can launch things at the scale of the entire museum in an alpha or beta version at all. But what it means is, we have this amazing r&d space that is the the whole museum, and we have people in it all the time, who can contribute to our knowledge building about what actually that looks like. And then lastly, museum as knowledge system. And this is something that I'm particularly like nerdy about, and if you get me on this topic, good luck with getting away. Just the idea that what museums fundamentally are, if we step back from the practical and look at them, theoretically, is that they are systems of knowledge that have specific kinds of inputs, they have specific kinds of outputs, there are ways that those knowledge systems are designed to feed back the pen is a really interesting one, because it delivers knowledge in a certain kind of way. One of the things that we're thinking about a lot at the museum is how we can make those knowledge systems generative. So so how can we respect and pay homage to the kind of curatorial knowledge that we're building around the museums all the time and the research base and expert base knowledge but that there are other kinds of knowledge that we want to be building at the museum. And not only do we want to be building them, but we want to be designing systems to capture them and reintegrate them into the system. So building intelligence, building generative systems to support that process, that also expands recognition that curatorial knowledge isn't the only kind of valuable knowledge about our collection, and our exhibitions and our building.

Unknown Speaker 22:29
So we are working on prototyping a co creative audio platform called Story base. And I just want to be really clear, this is a very, very, very early stage thing. And we could decide that it you know, we hate it, and we think it's garbage. So you know, don't get too excited yet, because we don't know yet if it's really going to work. But the idea of story base is is sort of reframing the museum itself as kind of a database and think and though it is actually database, right? We have the collection TMS. But if we sort of expand that metaphor to include the physical space of the museum, and consider each object is kind of unknown in the database, and what is the metadata around each object? So like, obviously, there's labels, there's tombstone information, there's the catalog entry, as Rich or not rich as that is, but how can we expand the stories that are attached to these various objects? How can we consider an expanded definition of metadata as not just small individual labels, or factual pieces of information, but emotional reactions, stories of context, things that they remind you of, and if you heard Adams presentation the other day, and we'll talk about this in a few minutes, we're looking at building a systems architecture to support to elevating the importance of stories. And so the idea of story base is that if you imagine that a visitor would sort of walk into the museum, and every exhibition would launch with a with some audio content that the museum created, and maybe one of those contributions, you know, sets of contributions would be from a curator, and maybe a conservator, and maybe another kind of expert that, you know, the exhibition folks identified as being appropriate for the particular exhibition. And so that would be a kind of audio tour, that you'd be able either to access on an object by object basis, or that you could follow through the museum. But rather than being a very formal, heavily scripted audio tour, the purpose of story base is a more kind of ephemeral like interaction with I'm standing in front of this object. And this is what I'm thinking about not I've have written a script, and I'm going to sit in a studio and record this. And so what do curators who are also human beings, but have this amazing knowledge base about this particular topic, actually think about when they're standing in front of an object, then we really want to create space to share that. And then to open that space up to users and visitors to actually share those same kinds of reflections and capture them and put them into a database and treat them as precious. And obviously, there's lots of challenges and I'm sure that every single one of you You could imagine all of the many ways that that could go wrong. And I thank you. And please tell me about it, we've probably already thought of it. But I'm always open to hear more. But at the part of the prototyping process for us, we're fairly certain that visitors will be excited about hearing audio content generated by the museum because we have a fairly long history of visitors being excited about that. The co creative pieces, so what we're not sure about and how that will look and how that will be framed. And like I said, maybe story based will actually decide halfway through that we think that it's crap, and we'll throw it out and do something else. But at least getting started on a prototyping process that pushes us in the museum kind of a little bit out of our comfort zone, and forces us to think differently about how we we approach it at all. Yep. Oh, yeah. Well, cool. So I'm gonna go really fast. This is we're exploring how we might use AI to to do a little more storytelling about the Hewitt sisters, who were the originators of the collection that is now the Cooper Hewitt. And we have all these blog entries and all of this really rich information, but it's not sort of organized in a way that we can actually build story around it right now. So we're looking at using AI to do that until it makes some semantic connections. We are also looking to build an accessibility chat bot, and we're talking to a couple of folks about collaborating on that project with us. I mean, should I name those people? No for talking about it? Well, you'll hear more about it. So the idea here is that it would, that people would be able to text with a chat bot to get access information. And that hopefully, that would be the intention is to integrate it into the CRM system so that we actually have the capacity to provide access services for those people in the future. And we're actually building intelligence about what their needs are.

Unknown Speaker 26:48
Also, we're really excited about exploring some mixed reality storytelling. And one of the areas that we're looking to do this is some onboarding around what it means to look at design objects, because design stories are mostly invisible when you look at a design object unless you happen to be a historian or you understand process in a way that most people don't. And we feel like Mr is a really amazing way to onboard people in a specific location with a specific set of goals, and then send them out into the museum with these new eyes that you know, mixing virtual reality is capable of giving. And this is a reprise of the linked architecture that Adam was talking about the other day. But the idea that in the future, we imagined some scenario where there is a collection database, there is a story database, that those come together through the API into a linked set and then are are able to be served out as a unified as a unified set of data in the various channels that were serving digital content. We're also looking, we're also building a commissioning program. And we'll be commissioning around these these various areas, pathways through the museum, activating user generated collections, content development, and then also new experiences for existing technologies. We have these beautiful tables, we have this beaut these beautiful interfaces, we have the immersion room, they're amazing. We love them, we think they work really well how could we consider expanding what they do? We've already just about to kick off, come in commission for a navigation system. So this is our second floor. This is the exhibition plan for the nature by design the set of nature by design exhibitions. The reason this is up here is because you can kind of see like we have all of these small rooms. And so being able to give people a story rich experience involves some kind of order. And so coming up with concepts that will help us direct people around the museum, to be able to follow this order is something that we're really excited about and want to figure out how we can do it. We're also working on this public program series, which we launched on September 17. With a program that we did in collaboration with the museums and AI network called curator, computer creator, we have another one next week on interpretation of storytelling, we're working with a bunch of three, threes a bunch, right, different designers from different disciplines to talk about their process and how their process might sort of layer into a museum approach to storytelling, and how that then informs interpretation. We're looking at access, we're looking at data and information design, we want to talk about augmented audio. We want to talk about embedding intelligence into the built environment and then also designing interfaces for shared futures. And I'm excited to explore more what that means I we were talking about our public program, and I was like, let's talk about interfaces. So that's what we're going to do. We're only two minutes late. Thank you so much. We just said a lot of things so we don't really have time for questions. Maybe just one and then we can take it outside if anybody wants to talk to us more. No, are you all just like blown back by all the words We just said really fast mented audio. So augmented audio is like, what's augmented reality using audio. So sort of considering audio as an augmentation to reality, so like the Bose AR glasses that are that use, it's actually not literally bone conduction, but they use bone conduction to give you audio. So the idea that putting something in someone's ears so like an audio tour is actually augmented reality. We don't necessarily think about it that way because the definition of augmented reality right now involves like holograms and two and a half d and visual stuff. But that like actually if we think about augmented reality is a thought framework like labels are also augmented reality so that's what that is. Any other questions? Yeah. Master Control System. Like Adam, do you want to answer that question? I so can back i right there. He's the person I wish that I were the person I'm not the person they are there is a lot that is essential now. Okay, any other questions? Yeah, sorry. We just don't want to we don't want to eat this. This lovely gentleman's time. Thank you guys so much. Thank you all so much.