Transcript
Unknown Speaker 00:00
Hello, everyone. I'm going to do this in song. My name is Nora Rodriguez, I produce interpretive content at the Jewish Museum, I'm so happy to be here. We produced a not kids audio guide about three months ago. So I'm going to be talking a little bit about that process. And since it's still relatively new, I think really what we learned from the process rather than the massive analytics that we have three months in, it's 1107. I'm going to stop talking in 11. Seven teen can snatch this out of my hand at that point. Right? can't quite understand that there's multiple microphones somehow. Okay, so before Oh, and there's this great. So I thought I would sort of like set the table a little bit and give you a tiny bit of context for this kids tour that we made. It was a kids who are for a permanent collection. And we had just reinstalled the permanent collection. So formerly, we had, you know, a very linear overview at the Jewish Museum of Jewish art and Jewish culture. In 2017. We really shook that up, we have a pairing of contemporary objects with Judaica, material culture, ritual objects, fine art, they're all in the same space. And you have these kind of really unusual, exciting juxtapositions. And, you know, I think that this is obviously really part of a pattern of a lot of institutions thinking about how they show their permanent collection, what is the big story they're telling. And oh, my gosh, I don't know why I can't get used to this clicker. But I do have to do that, right. This will maybe give you a little video, at the same time, we had designed a new mobile platform. So this was, you know, a basically a vehicle to access all of our audio tours. It was not an application, we'll be talking actually at a different session a little bit about this journey. But it was basically a single page web application. So the point of this is basically to say that we were not only thinking differently about the content itself, but how people were accessing the content, you know, what were what was the user interface? What was the experience of getting this content? So these kind of two things were happening, kind of, as we were, you know, re entering the conversation about a kid's tour slash a family tour. So Oh, perfect. Yeah, I truly can't do two things at the same time. So oh, maybe we have to, though. Okay, great. So why did we make this kid's tour, I guess, what I'll, I'm actually going to stand up that feels more natural to me for a second.
Unknown Speaker 02:47
You really are gonna do it. I'm just gonna put this down. So the first thing I'll say, as an educator is that I think all interpretive content is, for me at least less about what we are looking at, but how we're looking at those things. So less about, you know, the, the artist or why this object is significant, and more about how do we look in a museum, what is the experience of being in a museum, and I'll say, I don't know if this is controversial. But I always, I am actually not all that invested in having young people leave knowing what impressionism is, I am very invested in young people leaving a museum thinking museums are a place where we get to be really weird. Museums are a place where we get to laugh, museums are a place where I get to hang out with my family, museums are a place where I feel safe, I feel very invested in that. I feel less invested in Impressionism. But I think we should be invested in that also. So that's to say that as we as we enter this audio guide, you know, here's sort of the key things that we wanted to communicate. The first the Jewish Museum is an art museum that's really less about like, our institutional identity or ego differentiating ourselves, as from a history museum, it's really more about saying, you know, maybe there are some ways that we look at art, that are sort of are freeing or different, or weird or exciting. And, and, and we want you to experience that type of framing in this place. There are really cool stories that you're going to find here. And the way that we get to investigate those stories, primarily, is through looking really, really closely. And actually, I mean, I could talk for hours about, you know, trying to shift to other, you know, other ways of discovery, beyond looking but but for the most part, it was through close observation. And then I guess the last two points are really about you know, there isn't a right way to understand the things you're finding so people can have different interpretations. And we also really, really wanted to model the fact that you can have, you can ask questions, you can have conversations, you can make jokes. Making jokes is a really great way to understand art and objects. And so that was one of our goals is to sort of model that way of looking. So approach, how do we make this audio tour, we were actually really inspired by some of the audio tours at the Whitney in New York, because they were using kids voices on the audio tour, we thought that was really interesting. We really loved some of the rich sound of design that we saw at the MoMA, saw on the MoMA, heard in the MoMA audio guide. So we decided to bring students in and record their conversations in our audio guide, we brought in fifth and sixth graders, and we had them come to the museum for four sessions that were each about an hour. And we really let it like a school tour. So we had an educator come who's a very experienced warm educator, and lead tours with them, which we recorded. And then we did one final session at the end, where we sort of did like clean up narration so little pieces of narration that we wanted to that we wish we had been able to include, but didn't the but didn't catch during the recording sessions. So with that, that's sort of like the, you know, aerial view of our process, I'm gonna play a little bit of audio. So this is for a piece called Oyo by Deborah Cass.
Unknown Speaker 06:20
sculpture. The word boy comes from the Yiddish term believing we use it to mean No, or but what if you walk around to the other side?
Unknown Speaker 06:36
Oh, yo, yo, yo, yo, yo, yo, yo.
Unknown Speaker 06:42
We're looking at the sculpture boy yo, by the artist Debra Katz. And I'm curious what questions you have about it?
Unknown Speaker 06:49
Why did Devere cast choose the color yellow? And why did he choose it listener service.
Unknown Speaker 06:56
This piece was inspired in part by a painting by another artist named Edward Shea. And that painting was a big block letters, but it was just the word. Oof. Oh, f. Do you think this little building got in this bottle?
Unknown Speaker 07:14
I'd like to think that the person who created this advanced science 100 years and greater shrink ray. So they shrunk it and put it in a bottle.
Unknown Speaker 07:23
I think they cut off the bottom of the bottle and then created the tassel out of clay or whatever materials, get ready
Unknown Speaker 07:33
to have your mind blown. The bottle was unharmed in the making of this thing.
Unknown Speaker 07:41
How is this possible? Does anyone think
Unknown Speaker 07:43
that maybe this could have been possible with very long? Tweezers?
Unknown Speaker 07:53
No, because the tassels too big to fit through the bottle.
Unknown Speaker 08:00
This is an example.
Unknown Speaker 08:03
Yeah, they're free. Cute. So, you know, I think it yielded some really fun audio, we had a lot of fun doing it. I think that they had fun. I hope I hope it's fun to listen to. I'll really breeze through some of the things we learned. Because I think that there was a lot that we learned. And I don't I wish I could share a lot of analytics. But since this is so new, I don't have those. But I'll talk a little bit about the process. The first was, you know, we partnered with a school that's across the street from the Jewish Museum, the nightingale Bamford Academy, we put asks out to several schools that we partner with for various school programs, public and private, this was the only one that was able to make it work, you might be able to guess by the name of that school, it is a you know, it's a private school on the Upper East Side, that means those students, it's an all girls school also. So those schools, those students are bringing with them a certain set of experiences, and, you know, you know, their own identities. And I wish that we had had been able to bring in a more diverse set of students. And, you know, I think the real learning there is that, you know, every single thing, this is how I feel at least every single thing you're doing in a museum, it's about relationships, and you need to put in the time to really develop those relationships to, to, you know, for those to like pay dividends later on. So what I would say is, I think we actually should have probably spent like a third of our time just building relationships with students, or even more, I mean, I think in an ideal world, this could be like a long term strategy, thinking about bringing young people into the interpretive content. But I would say that, like, in an ideal world, we would have much more time for that relationship building to then produce this content. The other thing is, I think that the most magical moments of this tour are those kind of spontaneous moments of real exchange between mark our educator and the students, those moments where he's like, you know, just being himself being goofy and I would say that building More of those in and also thinking about ways that you could sort of facilitate some of that goofiness. So for example, with audio, we did sort of an activity where we said everyone has to go around and say, Boy in a different way. And so you got sort of that like funny, silly goofiness, but it also yielded cool audio. So the My question would be, you know, were there more of those sort of games we could play or things where we could think about it, like actually making it a fun experience for them. But then also, how could that create content? You know, another big learning was, the editing was a lot of editing, it was a lot of editing. And John was such a champion, and we read a lot of transcripts. But you know, this is probably obvious. But this, I think, is like a heavier lift in terms of how much, you know, refining of content, and you just have a lot more content, because you have all these sessions. And the last thing I'll say, very quickly, two minutes over is, you know, one thing that I think about, Okay, John, one thing I think about is, you know, there were a couple moments where we used interviews with artists in this tour, because we had interviewed them for other audio tours. And what I would really like to see in the future is thinking about how we could have leveraged those interviews with artists for some of the kids content. So for example, you know, when we do an interview with an artist asking, how would you want to talk to a kid about this work? Or if you were to explain this work to a child, what would you want them to know? And not only audiences that are kids, but thinking about just other audiences? In general, you know, if you were to talk to someone who's blind or has low vision, what would you want them to know about this work? So I think that's something I'm always thinking about is, you know, how is the content we're creating in other areas going to be, like useful across our various audiences. So I will end it there, and hand it over to Sarah.
Unknown Speaker 11:55
Great. Hi, everyone. Hi, everyone. I'm Sarah Durkee, from the National Gallery of Art, where I serve as the head of interpretive resources. And earlier this year, we had an opportunity to provide audio interpretation for a temporary exhibition called The Life of animals in Japanese art. This was a massive show featuring animals over 17 centuries. And I think, because animals, everybody thought that a family audio tour might be a great project for us to work on. And we agreed, we thought this would be a great match for families. But we also sort of had the opportunity and the challenge to think about balance because we only had the resources to create one audio to our for this exhibition. And the National Gallery of Art has a a very traditional tone and approach in our audio interpretation for exhibition. So we knew that we would have a lot of visitors coming to this exhibition expecting a certain a certain output in our audio interpretation. So we met together as a team to sort of think about if we were to create a family audio tour, what does that mean? And how can we make that as inclusive as possible, sort of going back to what John introduced at the beginning, we wanted to think about something that was intellectually and developmentally accessible for a wide range of visitors to the show. So we thought we would take on that challenge, we brought together a group of staff in interpretive resources in our exhibition department who are sort of serving as the onsite curators. So we were thinking about this together, as well as staff from our family programs department. So we kind of brought a cross disciplinary group together to think through defining family audio for this particular project. And we sort of landed around the idea that this would be for kids, eight to 12, and their adult companions. So that was sort of our very narrow scope. But in order to hit these other accessibility points, we thought about learning outcomes that were knowledge based, but then also enjoyment and social based as well. So we thought about, what would it look like to define comment, content that inspired conversations that made for joyful moments of discovery together? That was this is the only Chatham House part of this presentation, fun, which is not a word we'd like to use at the gallery, but we wanted those moments. So so the outcome sort of drove that for us. And we also decided to hit some of those, we would need a storytelling approach. Stories are universal and something that kids and adults could enjoy together like and then we started digging into the details. So we knew from a lot of our visitors, both family and adult Audio interpretation can feel like a drag, it can lock you in, it feels like it takes too much time. So we wanted when designing for a family audience to create something that felt flexible and choice based, where families could listen to as few or as many stops as they wanted. And we wanted people to feel that way. That worked out very well, because as we really dug into the content, sort of two content tracks emerged, one that really focused on the stories and the folklore and a lot of the objects, and one that sort of spoke to the symbolism of the different animals. So we ended up creating two tours, two short tours, each one one had 12 stops, one had 13. They were associated with those two icons. So instead of seeing the little headphones in the gallery, you would either follow the crane path or the monkey path. Turns out, Nobody followed either one. But that's, that's for later.
Unknown Speaker 15:58
It was a great idea. And then, so we thought about the content and the experience for our families. And then we also thought a lot about invitation and messaging. So we have this family audio tour that we think is for everyone. How do we tell people that so one of the things we did was we created an activity guide for families that had a scavenger hunt, and it had a art activity where you could choose an animal that best represented your family and draw it together. So that was a very clear invitation for our family audiences. But we also thought about signage, and we thought about how we script the volunteers who sit at our, at our front desk in terms of how they can talk to people about what this tour is the first, the first sign said family audio tours for visitors of all ages. That stayed up for about a week. And then we switch that out to family audio tours for children eight to 12. And up and we kind of played with the language a little bit, which I can talk about a little bit more. The stops for these tours are extremely varied. We do have one clip from Black Bear plays with a camera, which was one photograph in a series of four photographs in the show. So I want to give you a little bit of a taste of of one of our stops. Can you play I don't know if I can play that from here. It starts very quiet.
Unknown Speaker 17:32
We are out in the forest at night. Try not to make a sound. The photographer Miyazaki Manabu. has set his camera up outdoors. When an animal comes past, he will automatically trigger the camera. His photographs, let us see what animals do when people are not around to watch them. Look at the black. He looks like he wants to be the cameraman himself. I wonder where the zoom button? How do you get it to take a selfie?
Unknown Speaker 18:11
I should have mentioned one of the other things we did was hire a Japanese American award winning storyteller Matobo Dorkin, whose voice you heard here. And we hired her both to narrate. But also do script review with us to sort of develop the story in her own voice as well. So she took all of our families through their journeys. So what did what happened with this tour in the galleries? Well, just under 7000 People listen to it. On average, about 60% of our visitors listen to 10 stops or more. And I do want to point out we have these analytics because we worked with a company guide ID to distribute this audio tour and we were able to sort of see what was happening in the galleries because of that. Given that each tour was 12 stops. This felt like people were really listening to the tours. But we were also curious about who they were. And so we worked with our evaluation department who did post visit surveys with a sample size. They interviewed about 52 visitors in July and August. As you can see here, this was kind of striking to us about three quarters of those who listened to the tour were either sometimes or rarely engaging with audio. So that was of note to us. And then I think even more important you'll see here of those sample size. Are there any children in your group today? Not a lot of children coming with the people who listen so our family audio tour had had an adult audience for three quarters of the time. So that was interesting for us to see as well. But here's what they said about it. It showed history humor and it talked about details. It made it relatable. It feels more personal like a tour guide. It has emotions built into it, I was thinking that it would make it more accessible for my grandkids when they visit. So all of these comments were sort of hitting at some of these outcomes that we were looking at for the beginning. We wanted multigenerational experience, you wanted conversations, we wanted you to feel immersed in it. So we were feeling very encouraged by that feedback. And then we also got ones like seemed more geared to kids and was immature in its approach. I think it's better when the kids and the adult guides are separated, the pace is too slow. And the information is our favorite dumbed down. We weren't surprised to get some of these comments. And I have to say there weren't that many. And I think I know we're sort of running out of time. I think what, as we began to dig into the data, one of the big learnings for us on that was the management of expectations. So we actually went back in and for the people who bought at the content on the tour, we asked them, Did anybody tell you that this was an immersive fun tour designed for kids and adults? And 89% of those people said no, no one had managed that expectation. So we think that with more careful messaging, more consistent messaging, we might be able to, to plan that because at least anecdotally, when I was sitting at the desk, and you sort of lead with this is not your average scholarly audio tour, this is designed to be fun and immersive. People said, Well, that's what I want anyway, and they happily took it and they came back and they were really happy. So when we think about that kind of family into adult and multigenerational content, that seems to be a really important piece.
Unknown Speaker 21:40
So I'm gonna end there with just one more comment. And this is sort of forward looking and kind of an invitation to everybody in this room, we've been working with our evaluation team to use the overall experience rating, which some of you may be familiar with in terms of sort of counteracting, or counterbalancing courtesy bias in museums. So a five point rating of either poor, fair, good, excellent, are superior. And really anything that's good or below is not that good. So you're really sort of looking at benchmarks and the excellent and the superior. And you can see in that sort of pixelated, not very clear slide. We're we have a lot in the excellent and the superior, but we don't quite hit the benchmarks. And so we're beginning to see if we can glean a little bit more information about what is causing people to just rate it as good as opposed to excellent or superior. We did do some preliminary looks, and it doesn't look like technology was really one of the key points, which was interesting. It did seem like expectation management was important for some people, but not everyone. So we're going to kind of dig into that a little bit more. And it was very clear that the more stops people listened to the higher they rated it, which is obviously a virtual circle anyway, people who are liking it are gonna rate it high and listen to more stops. So we're kind of curious to see where this cut of information might take us in the next year. And if any of you are interested to please reach out, because I think there's a lot that we can do on this front as well, in terms of making sense of this kind of content. Did I make it? Okay.
Unknown Speaker 23:24
I don't think there's anybody right after us in this room. So if there's some questions, I think, is there anybody right after this? Oh, good. There's no Go ahead. Turned out,
Unknown Speaker 23:49
we didn't do a lot of research into the length of stops, other than the fact that we have analytics from all of our previous tours about how long people listen, what I will say is that, you know, most of those analytics are for adults. And so this is another thing that we talked about is that I that we do wish we had done more evaluation, you know, in or before we initiated the process and during the process in terms of how long kids are listening, because my instinct is that they're listening that they need shorter content.
Unknown Speaker 24:17
Yep, we are researched both on site and just from the field was sort of a 92nd average audio tour stop. We went longer on some of those and we saw people click out. They were they were too long. We felt for the story we had to but we were going with a 92nd as our as our benchmark.
Unknown Speaker 24:39
Yeah. Just from our experience at acoustic guy like we always that's our like general rule is 90 seconds. I will say that one of the things that's been great working with the Jewish Museum on their projects is it's been a kind of a iterative process where we're working through things and figuring them out as we go. And so that I think that we like this We'll continue with them. That's just the way we work.
Unknown Speaker 25:05
More important, to some extent, you had challenges with some of the schools schools getting a large population of participants, and that you felt that cultivating those relationships, was one of the key challenges wasn't getting participation in turn. They didn't know you what they're trying to do.
Unknown Speaker 25:24
Yeah, that's a great question. And I should first say that I, you know, our school programs manager was really facilitating that piece of the process. But what I know from her is that, you know, we were looking to engage students after school. So, and it's a significant time commitment, you know, they have to come, it's at least an hour of recording. It's like, a lot of these kids have homework, they're hungry, it's you need to provide snacks, you need a parent to come pick them up. So I think that, you know, the proximity was one reason why the school really worked there, literally across the street, so students could just walk across the street. And, you know, we had a relationship with them, although I will say that every school we reached out to we had some type of relationship with. So I think that sort of the logistics of how easy it was for them to come on site is really the reason, although I think we could have made it work. But you know, you're right, that there's so many things like you need to you need someone to come transport them to the site you need. Like, obviously, you need immediate releases, you need a parent population that's, like pretty engaged and excited to be a part of it. So yeah, I mean, I think and again, I think that with planning and with, if we had really set that as an intention, and had a little bit more time, I think we could have done better in that area. So
Unknown Speaker 26:53
got in the back. And then and then then you go ahead, me Yes, yeah.
Unknown Speaker 26:59
Are there any resources that you use to determine what would be helpful for that intergenerational learning? Or was it just sort of like instinctive that like, kids like things that are fun and cool and interesting, adults will also like those things, because that's something that work that we're thinking about how do we get that intergenerational engagement and learning?
Unknown Speaker 27:22
So yeah, we look to a lot of the research that drives our family programming. So having them in in the conversation was a big part of that. And developing conversations around art is one of their core tenants. Looking closely is one of their core tenants. So we made sure that prompt questions as well as looking prompts that bring out interesting details were part of sort of our approach in the audio. So that's how we did it for this sort of building up from our existing research and programmatic tenants.
Unknown Speaker 28:05
Yeah, I would just echo that and say that the resource we relied on most heavily was our internal Education Department, like our educators, both, you know, staff and part time educators who are working with students and asking them like, you know, what are you seeing what are the questions students are asking? But again, I would say that like that could have been an area for more research at the outset.
Unknown Speaker 28:27
And I'll just add that like having done a number of these, and really large versions of these types of tours were literally like, hours of content for one site. I tried to Deek add some guidance into that in terms of like, just and again, it's a lot of his anecdotal, but it's a lot of just sort of experience in doing this for multiple sites and seeing how different things whether it's a museum or historical site or something in terms of that and trying to try to engage audiences, different audiences, but that in that way.
Unknown Speaker 29:02
So this is sort of a community reading question. But I'm wondering if you guys use any tools in the sort of editing process if the script is
Unknown Speaker 29:12
out there right now. Just when you have so many nicknames and transcripts, that you're trying to piece together,
Unknown Speaker 29:18
anything, that would be my computer screen, just reading it, we saw an acoustic guide, we have an in house studio. So we have like, what happens is, we go through the transcript, and we have the raw audio and I'll listen and they'll listen and we'll pick up the pieces we think and then we'll try and usually we'll work on scripting and putting the script together in a flow and like a narrative flow because the way that this was just like the start this up to here, that's not the way that like the we did like that's not the way they were recorded. And then luckily, I have we have great engineer that occurs to guide who will then do rough cuts and they'll get to hear, you know, hear it and then we'll we'll clean it all up and produce it at the end. decision making
Unknown Speaker 30:02
process from designing your family for and how you get it realized we probably have so many factors. How you make the decision?
Unknown Speaker 30:15
Yeah, that's a great question. And we had done family programs and activity guides for shows before. But this was our first time actually doing family audio. And I think literally, it was driven because the subject matter of the show was animals. I hate to say it, but people felt like of all of the things that the National Gallery of Art would do this is this is family friendly. And I also think it came down to budget as well. So in education, we had the money to produce this. So those two factors came together, and we developed a family audio tour. I think that because of this experience, we're already thinking about what we where we could apply this next because it did feel different than the audio that we have, and really impactful for our audiences. So now we're taking a look at parts of the permanent collection where we may be able to do similar sort of treatments.
Unknown Speaker 31:16
Yeah, I would echo that I feel like resources really drive a lot of this decision making. Unfortunately, we tend to not do kids audio ads for special exhibitions, we did a kid's audio ed for permanent collection, because it is not going anywhere for the most part. And so we felt like it would have the most longevity. We have talked about potentially doing very, very short kids audio guides, you know, like five stops just choosing five objects or something. And, you know, another thing we talked about is like, capacity building in terms of, you know, if we have a group of kids coming in, could we record them and and just make something very, very short and do it as sort of a pilot project. So we've definitely talked about that as a way of like maximizing resources. But currently, yeah, it was really about like, what if we're doing a kid's guide, what is going to stick around the longest? Anybody else? Thank you. Thank you.