Learning from an Open Approach: Activating Smithsonian Open Access as a Commissioning Case Study

Unknown Speaker 02:14
everyone, thanks for joining us. We're gonna get started in just a sec. Also just to note, we were having a little bit of technical difficulty with one of the panelists getting into the Zoom Room so I may be momentarily distracted, checking Slack on my phone, looking for him but we will be we'll make it happen. So thank you so much for joining 45 minutes is not a long time, so to jump in and get started, if folks could mute that would be great. Hi, I'm Rachel. I'm a 40 year old non binary woman with short light hair that I add my hairdresser describe as ash blonde but everybody else thinks is brown. I'm wearing gold aviator style glasses and have two piercings on my left nostril. I'm the founding director of the Interaction Lab at Cooper Hewitt Smithsonian Design Museum. Today I'm joined by Jono Brandel new media artists and technologist, Jackie Lee who will be joining us momentarily, CEO and co founder of science VR, Lisa Walters executive producer at experiencial design studio red paper heart. Ryan King. Open Access Program Manager from the newly formed Office of Digital transformation at the Smithsonian and Kate Haley Goldman founder and principal of evaluation firm HG and CO. We are all here today to talk about activating Smithsonian open access, or for short, which is a creative commissioning program launched in spring of 2021 by the Interaction Lab that funded seven creative technology teams through an open call for digital interactions with the Smithsonian's open access data set, the slide you see on screen right now, It's just a title slide with all of our names, and the title of the panel. So next, in case you're not familiar, the Interaction Lab is a visitor experience focused r&d program at Cooper Hewitt Smithsonian Design Museum in New York. the slide on screen right now says Cooper Hewitt interaction live in the upper left hand corner, and then the image in the background is a whole lot of colorful, and less colorful sketches and a photo of my hand holding a very small mock up of a remote control that we developed in a workshop we did a couple years ago. The Interaction Lab was created to lead the development of a reimagined visitor experience at Cooper Hewitt, while also convening thinkers and doers in the design world and museum sector to explore futures for museum experience since its fall 2019 launch the lab has injected new ideas into the museum's work through internal workshopping and strategy, a highly participatory Public Program Series merging interactive design and museum practice and a commissioning program that engages the design community as creative collaborators, and it's that last piece that we're here to talk about today, I'm going to start by just running through some info about a SOA, and then we're going to hear from each of our panelists and then hopefully we will have time for questions. We're going to try to push it with this 45 minutes.

Unknown Speaker 05:20
So, this slide on screen right now says seeking interactions with open access collections beyond passive quote looking, and so supported by Verizon 5g Labs, this first iteration of creative commissioning was intended to demonstrate the potential of this open innovative approach by engaging a community of creators beyond those who typically work on museum projects, Rather than focusing on recreating the kinds of passive looking experiences already commented physical museums, the open call encouraged creators to propose interactions that take advantage of the incredible range of possibilities, arising from digital collections with unrestricted creative use. And one of the most exciting things about esoa is the diversity of thought that it represents and it was able to support. So from web based extended reality to multiplayer social learning time and writing based collections exploration tools, and even simulated echolocation. The so prototype speak to a wide range of audiences enter interaction styles, very much living up to our ambitions for the program. The slide on screen right now shows an image from each of the seven projects which, as you can see are quite diverse, I'm going to describe them there including top clockwise from the top left, art clocks digital interface on a tablet sitting on a wooden shelf, butterflies, augmented reality depiction of butterflies flying around the phone screen with a real life garden background, a still image of the period room from doorways into open access which depict beautifully rendered period Decor from LaBella puck in Paris in the early 19 hundred's a moonscape from science yard treasure hunts multiplayer game, the writing with open access interface depicting keyword analyzed writing on the left, alongside a gallery style display of related open access images. To the left of that we have a 3d rendering of a bronze king's head from the kingdom of Jeanine, that is available to download for free thanks to casting memories, and a 3d woolly mammoth skeleton, that's part of the multimodal exhibition art echo. You can imagine that getting all of these images on one slide, and into one collage was a little bit of a trial because they are very different from each other and there's really a lot going on. Unfortunately we don't have enough time today, for me to dive into each prototype in depth, but I urge you to play with them and share your feedback with us. They're all accessible from the SOA homepage which we'll put into the chat in just a few minutes, and they will remain available at least through the end of January through the homepage, and likely far beyond that from the team's individual web pages. So, all of that said, let's talk about the potential for creative commissioning. So really, the idea here was that we have the Interaction Lab see a really tremendous opportunity for creative commissioning to do all kinds of things for the museum. The first and most importantly, we think, is to show the range of possible interactions, missing with with museum assets. In this case, open Access assets but in general museum collections and research that by engaging a community around Cooper Hewitt design challenges or sub your museum name there. It helps to create a closer connection between the museum and the design world so really thinking about like museums as these massive asset store houses that can just be activated by engaging with folks from creative communities in the design world, expanding the range of creators who are building tools and experiences for the museum sector. So thinking about the way that commissioning is typically used in museums which is to commission like art pieces to be displayed as part of an exhibition for example as opposed to what the Interaction Lab is doing, which is actually thinking about creative commissioning as a pathway to new interpretive technologies or experiences or other kinds of analog interactions we have lots of ideas about how we can use this. And then lastly, and this one is a big one, I think for any museum anywhere, is to put a wide variety of design practices to work without having to build internal competences. So all of those prototypes I just described very rapidly. Imagine if we had had to actually hire or in some way recruit and bring in house, all of that capability, it would have been incredibly expensive and I mean to the point of being totally prohibitive, but we had the opportunity to engage all of these really talented creators to make a ton of really cool stuff without actually having to hire them, which sorry guys, I wish I could have hired you, but the fact that we can't and still got to work together, it's pretty great. Um, and so what's on the screen right now is a diagram about what a pathway to scale looks like for creative commissioning, which is not dissimilar to what it would look like for any kind of startup so starting with an open call, then a small piece of funding through permission Award, a prototyping and evaluation process. And then hopefully, significant funding to then scale the experience, assuming that the evaluation is successful.

Unknown Speaker 10:27
So this next slide has a whole lot of text on it, that I will not go through in depth right now, but I will share the deck afterwards so you can go through it in depth, but basically it breaks down the process into five stages and the timeframe attached to each was the actual timeframe that it took more for the so a program to do this. So there's pre planning, planning, awarding builds and post launch and so pre planning is really about, like, actually getting the relationship together with a partner, they're really like, Oh, somebody's dog is barking not unlike my dog might be barking but not right now. So really, like, getting things together with a partner, which in this case was Verizon 5g labs, and then moving through this process of MANY concurrent steps, eventually leading into a post launch process where we're evaluating the prototypes and determining what we can do next with them. And then lastly, what you're seeing on screen right now is a long list of names there are 62 names on this slide and they're not even everyone. So, as you might imagine, it took a tremendous amount of labor and collaboration for MANY people to make a so happen, we have the SOS staff so our executive producer Jade wells, a program assistant Catherine Miller, who is typically working in visitor experience to actually dove into supporting the lab during the museum's closure, we had fundraising partners from Cooper Hewitt and from Central Smithsonian, we had the lawyers we had her judges and mentors and advisors, plus the 40 people who did user testing whose names are not on this list, collaborators in the Smithsonian Office of the Chief Information Officer, of course, our partners at Verizon 5g labs, and I don't draw attention to this massive list of people to dissuade you from doing creative commissioning on your own. Actually, the opportunity here is that these 100 Plus in reality 100 plus people who we interacted with over the course of the program actually become a significant and invested community in the success of these prototypes. So, the idea that we were able to commission seven different prototypes, run a program where they were made at the same time, and then engage over 100 people in the process of their existence and ultimately hopefully their success was really meaningful. So that was a very rapid fire overview of a SOA, and I absolutely could talk about it WAY longer and I'm happy to answer any questions you may have about it and which I'll do in the chat because we're kind of short on time, but the panel that we assembled today. The idea was really for everyone in attendance to get an understanding of the impact of this program from different people's experiences. And so I'm going to direct a question to each person on the panel, they're going to speak for a few minutes, and then hopefully as I said we will have time for questions at the end. So, John Oh, um, what about the so a challenge appealed to you as an independent new media artist.

Unknown Speaker 13:46
Yes. Hi, my name is Jono Brandel, I'm a new media artists, and technologists, I'm the project team lead for writing with open access for this prototyping phase of. So activating Smithsonian open access, and I am a male dark complexion with white glasses and a top notch hairdo with a beard and a black shirt. Um, and to go to Rachel's question. So it to me was, these three things. It was an action oriented grant, A safe creative space and the accountability that myself and the two other team members that comprise our team. So three in total needed to actually, you know, realize, kind of our project concepts. So, next slide please, before I actually go into depth of those three kind of questions though, I just want to give a quick overview of what the prototype we made was, and is available to us today. So, this is our project writing with open access, it's an animated GIF. On the left side, there's a text being written in a grey box that says, can we use the Smithsonian's collection to transform the writing experience, you're writing into the introductory statement of a museum exhibition. And then on the right side, images kind of pop in and the main image that we can see on the right side is actually an illustration of the Smithsonian Museum, one of the older building illustrations, and so this is actually the experience of the prototype, you can go to the website which I'll put in the chat after my, my answers are given, and you can right in this left hand area, as you would, and the text is analyzed and keywords are chosen, which are the highlighted words and those keywords are used to query the Smithsonian database and bring and connect those keywords to the to the images that the Smithsonian Open Access API results and so it kind of bridges images and words together. So that's our prototype. Next slide please. So what made a SOA action oriented. For us, you know, in the original proposal this grant and prototyping period was about making something tangible, and from an artistic perspective working working in the arts as an emerging artists, a lot of grants and opportunities. Explore kind of a research phase and don't have a tangible output, and to have a 10 week program that had, you know, a prototype that you're working towards was something that we're really interested in. And so this image is actually our project schedule that we proposed. Each week we kind of had milestones that we wanted to reach. Next slide please. Um, it was also a safe and creative space for us to fail. And so that was kind of the big takeaway for me it was during this 10 week period, you know, our team always felt like we belonged, and we were connected to, you know, people directly within the Smithsonian, with Cooper Hewitt and people outside, you know, Rachel mentioned that Verizon 5g Labs was a supporting member, and they helped with, you know, their network, as well as an inclusive design consultancy that we had. So we had mentors kind of from lots of different areas that we had access to. And this image is actually a Slack screenshot from our Open Access class, Slack channel, and I had issues using the API, and it made such a big difference to be able to directly ask those questions to the people who are building, building the API, so I won't go into detail about what the question was, but suffice to say like, it was amazing to just have that direct access from, from an artist and not just work with a documentation page and, you know, send an email into the ether. Next slide please.

Unknown Speaker 18:01
The last element that I felt like was unique and important for this grant, compared to other ones that I've participated in, and particularly, particularly that I'm proud of about making this project is that, you know, they provided the accountability that we were looking for, you know, in art school and in the art domain critique is a very important aspect of developing your practice, but when you're not in school, you know, it's sometimes hard to find that that kind of rhythm and discourse with other people. And so this image is actually a legal document that I had to sign that's publicly available that the Smithsonian has about privacy and security, but it kind of symbolizes the, the support that we had where, you know, we were meeting with people every week, and kind of showing our progress, talking about our vision about wanting to connect people's writing to the Smithsonian Open Access collection. And that really kind of created, not just a fulfilling project, but also made us, you know, hit the milestones that we wanted. We were very ambitious with what we wanted, but because of the 10 week period. The proposals in all of the projects were really specifically scoped. And I'm really proud of the project that we made and also to be part of this cohort. Next slide please.

Unknown Speaker 19:34
So, to wrap up, those are kind of my answers to Rachel's question about what makes us so unique. And I just want to finish off by saying that this project actually started earlier. I first got to explore what the museum industry is kind of like pulse was at MC and last year when I was part of the new museums, cultural futures track, and during that time I met some of you in the audience and a lot of other people to learn about you know what the challenges that museums had and the amazing opportunities that museums, provide, you know, not just for the general public but for, you know, organizations and, and artists alike. So I also just want to say like it's a amazing opportunity to come back to MC N and share this, this project and it comes from a lot of questions and discussions that happen through the NCI network as well. So, if you are interested in learning more about this project I will share a link. Rachel just published a design retrospective for the writing with open access project, so it goes in detail about how we kind of came to the decisions that we made, has some other kind of video material to explain the project, and I'll pass it back to you, Rachel. Thank you.

Unknown Speaker 20:57
Thanks so much Jana. Um, so, Jackie did make it into the room but Jackie is on the phone and cannot see what's happening on Zoom. So Jackie I am just going to direct the question to you, and then, um, and then if you, I'll just kind of like we have some slides but Jackie you let me know what your what your preferences in terms of how to handle that. And thanks again for being such a good sport about the technical challenges. Um, so, Jackie, are you there, do I have you.

Unknown Speaker 21:32
Let's see. You said you can hear me, are you 617 Probably. Alright, Jackie messaged me on Slack, we'll jump to Lisa for the time being. So I'm Oh Here you are. Hi.

Unknown Speaker 21:48
Hello. Thanks for

Unknown Speaker 21:51
joining. Sorry about the drill.

Unknown Speaker 21:55
That's fine, it's fine, it's great. Jackie from science, we are we looking at a slide called Treasure Hunt face mission we are now. Yep. Oh, great. Thanks. Great, so it's really been exciting, inspiring journey to where we Rachel and her team and also to learn from, from the Smithsonian Learning Lab, and those are colori from the Verizon 5g lab. Um, you know, I love museums and love visiting museums. I study computer human interaction, and have been building 3d and interactive media for all my career. I am a creative technologist. So, I believe, all the great ideas can have the visual forms in virtual space for people to interact with. So this is where I'm coming from. And, and this unique opportunity to collaborate with Smithsonian museums and open assets. Allow me to really think about what digital museum looks like. Well, we all have different ideas about digital museums, right. And, and to narrow it down. The we focus on classrooms and families with middle school students. And that's actually my son's age, and he also helped us due to the advertise Edison over there. So, we started from building a multiplayer treasure hunt game, we choose, Apollo 11 space mission as our main topic, and it turns out to be very popular. We build a museum actually calling from the museum and the museum is populated from with the CC zero digital assets from the Smithsonian open assets. And if we can go to an expert. Next slide please. So, my son, he wants to play games with his peers. He does coders. So I tell him, Well, there are other avatars collecting artifacts with you, and they can be your friends, And we're all here to play and this actually resembles a typical museum setting, and people explore and learn as a informal social group. So, so it seems to me, our job is to get them to have more conversations with other bands around that. And good in this game. My son and his friends can also learn about artifacts and get to see the ego more lenders landing on the moon, virtually, and to get the scan more interesting and also harder with that different treasure hunt objectives. And so, players, they have to authenticate artifacts, by typing in the observations, before they can collect it. And this is our discover authenticate and collect process. And one of my sons, tells me, and he come into it. And this is a game that I have to think, and it's quite funny. So now we have more, something more to chat about artifacts right. Can we go to the next slide. So, more conversations, also mean that we are sharing our perspectives, and they have been great efforts from the Smithsonian Learning Lab, where they apply project zeros, the Think, Wonder educational guidelines to encourage prospective sharing. When learning about artifacts. And so we asked ourselves, can we do some of the perspective sharings a scam MCN mechanics. So this is what we call the social learning game mechanics. So for example, as you see in the, in the slides. When the player authenticates the other set his or her notes will be broadcasted to all players. So in this case, there were learn from each other as they are making progress. So that is one example. And okay. So, to sum up, learning in this virtual space and building social learning game mechanics is such an interesting area for me. And we will keep exploring with Rachel and her team. I'm also hosting a tour in the museum, and there should be a URL, and I'll be in in the museum. After the panel, and I hope you can check it out, and happy to answer any questions. And you, you're able to use your browser to join, and we'll, we'll meet in the moment with you. Thank you so much Rachel.

Unknown Speaker 26:36
Thanks, Jackie, and to everybody, I put just put a link to science VR to the New Museum in the chat so if you'd like to join after the panel, you can do so. And now I'm going to stop screen sharing for a moment and ask Lisa Walters a question. Um, so, from your perspective as an executive producer, how is the seller process different from other kinds of commissioning, or requests for proposals that you typically get at Red paper heart.

Unknown Speaker 27:10
So, Hi, I'm Lisa. I'm an elder millennial with orange hair, and I think the exact same gold aviator glasses as Rachel. So I guess it's, it behooves us to talk about kind of typical RFP process first. Our red paper heart. We're part of the rarefied world that receives RFPs from museums and other institutions, both for experienced design, as well as custom art installations. A typical RFP goes out to three to five companies and they'll come up with some version of the same idea. The winner of the company is generally the one that can throw the most money at renders, and then crafted emotional narrative around what can be a fairly basic concept. I've seen companies that will allocate well over $100,000 Just to a pitch deck. I've seen asked of companies to shoulder over a million dollars in production costs in the RFP. So with that being the norm, the barrier to entry is actually quite astronomical, those types of barriers make the typical resource pool very tiny, and then you have to ask the question, Am I getting the best ideas from this RFP. And I mean, probably not. I think that the best ideas involve some form of risk. And when companies are spending that much to craft a pitch deck or carrying that type of debt. There can't be a lot of risk involved. And on the other side if you don't spend that much on your pitch deck, it doesn't look as good. So a truly great idea can be lost to some Glossy renders. So, in contrast to what I've traditionally seen with RFPs. The SOA RFP was fairly short, and asking for the straightforward deliverable of how can we explore the massive online catalog of the Smithsonian. To further this ask the outcome was expected to be a prototype. There were no parameters on how a team interpreted the task, which allowed for a wide variety of disciplines. There was a strict budget. There was a strict and short timeline for both proposals, as well as prototypes. There were page limits on the actual deck page counts to address each section from BIOS to ideas to work history, forcing brevity, which probably also helped in the auditing process as well. By the associate you doing this type of thinking up front, the limits ensured a very level playing field where a team could submit a proposal and the power was completely within their team and the idea. Also, when you level this playing field, upfront, it allows for a lot more diversity and inclusion, rather than the traditional route which limits ideas to what has been done before. And edition the RFP was almost answering, it's quite the problem itself because it drove a ton of people to the catalog to explore and look for inspiration. But at the end of the process, I was at a table filled with people that I'd never heard of or seen before. And to me that was really exciting. Another difference that I noticed was the support and access to information in order to achieve the ideas, the support that was given to the awarded teams was fairly incredible. The Interaction Lab had assembled a group of mentors creators of the database and API, stakeholders and assembled them all in the same Slack channel. So not only were you given this kind of unprinted precedented access to all people involved, but also the subject matters for content and accessibility were there. This wasn't really traditional as far as US versus DAM roles, you got to overcome that traditional vendor relationship, you were able to see challenges that other teams were encountering and adjust accordingly or chime in if you had already overcome that same challenge. It very much felt more of a partnership. The outcome of this RFP process are some truly new ideas. I guess that have legs that can be pushed further ideas that address shortcomings in accessibility in all forums, and I think that it was really important to see an institution like the Cooper Hewitt, the attraction lab, and the Smithsonian Museum ating this type of work.

Unknown Speaker 32:05
Lisa, thank you. You're welcome. Oh much. I just I have to say this and it's gonna embarrass you, Lisa and I apologize for that but when we were talking when we were rehearsing and Lisa was running through some of these ideas, I was so excited because this is such an important perspective, to share especially with this group because there's so MANY perspective commissioners in this room right now I'm sure folks who work at museum so really like hearing your point of view as an executive producer is awesome and so appreciate it so thank you. Thanks for having me. Yeah, of course, thank you for joining us, Ryan King. Hello. Thank you for joining. Yeah. Um, so, my question to you, um, what was the impact that so had on your thinking about possibilities with open access and has this process changed anything about future approaches that you might take.

Unknown Speaker 33:01
Yeah, good question. Hello and happy MC and everyone amazing stuff so far right and I have to say it has been incredible working with these teams and seeing development from ideation to polish prototype. And over the course of just a few weeks, honestly. We've been so lucky to work with these teams and a big thanks to all the thought and work that was put in behind the scenes so Rachel, I'll be getting to your question but think it'd be helpful to start with a little background info first. But even before that I think all of you so much for joining us today, and a big thank you to the NCM team for organizing yet another fantastic conference, and to my fellow panelists for the support of the SOA project. My name is Ryan King, I'm the program manager for the Open Access Initiative, and I'm most commonly use he, him, pronouns, I'm a 40 year old male with mousy brown hair. I'm wearing a navy shirt and blue rimmed glasses and sitting in front of a bookcase topped with plants and artwork on the wall at home in Washington, DC. So to get it racialist question a bit of context, the Smithsonian. Open Access initiative launched in February of 2020, at that point released 2.8 million 2d and 3d assets into the public domain SEC zero. It was an amazing one Smithsonian effort. And we continue to extend our gratitude to the over 100 staff members who contributed to the success of that launch, but it was also a very timely release right before, before a global pandemic that seemingly overnight made digital access a necessity. Um, so really kind of the why of the alignment with open access, and this creative commissioning process and creative teams really I mean, while we're quite proud of releasing millions of objects to the public. Objects in an image of themselves aren't necessarily, you know, have impact or fully engage our audiences, it's the context that we can weave and the expertise that we potentially could bring to the table and creative storytelling that connects those objects, both to each other and to the public in meaningful ways. And this solar project really, really nails that through the Interaction Lab, Rachel and team put innovation front and center, both in digital products delivered, and also very importantly in experimenting with new ways the Smithsonian can identify and work with potential partners, which I think is really kind of the meat of what Rachel's asking so the creative commissioning process allowed us to build new relationships with creative individuals and teams that we normally just would not, most likely have had the opportunity to collaborate with through normal contracting processes. So these groups brought fresh perspectives and connections to new audiences and applied their own lenses to the Open Access Objects and data sets, and then also encouraged us to think anew in different ways about potential product development. So while we're rapidly prototyping a variety of different ideas along the lines of impact learning outcomes and connections to our audiences were able to elevate the top vetted products for further development into, like for example items in our gift shop or exhibitions in our galleries, and as referenced by everyone like a large component of the process involved mentoring, where the Interaction Lab partnered si staff and external advisors with the creative teams to share ideas and practices. And one example is the emphasis on accessibility that was baked in to the mentoring sessions, those discussions. Establish both the baseline and common vocabulary around digital accessibility and helped us envision how we can internally infuse accessibility practices more holistically. And I just want to pause for a huge shout out to Sina Bahram and Tori, Corey Timpson of Prime Access consulting for guiding our teams along the way. We found this investment not only resulted in stronger prototypes. But as we build these ongoing relationships with new creative teams we know they have the baseline understandings of our expectations and standards of excellence for future collaborations. I encourage all of you of course to review all the prototypes in the link, Rachel posted, but we'd like to spotlight, how a few of directly tied back into the work underway at this at the Smithsonian. So our echo is pushing the envelope and how we build augmented audio experiences for in gallery and virtual

Unknown Speaker 37:38
experiences of our objects, our own 3d digitization team is currently exploring this area, and the cross pollination of their findings has helped inform and better both goals science VR brings virtual access into classroom settings a key audience group for the Smithsonian, our clock is infinitely extensible an application and its crowdsource model allows for greater access to not just discovery but contribution and writing about open access applied accessibility practices to not only their interface and screen reading functionality but also thinking broadly about access, built in multiple languages from the beginning. So these advances give access to the Universal. Universal Design primacy and are crucial in the Smithsonian strategic goal of reaching 1 billion people a year through digital strategy. The ASL project also allowed us to focus on a group feedback loop on our Open Access API or application programming interface, the API was rolled out as part of our launch efforts and allows for anyone to tap into our assets and data sets programmatically to integrate into digital projects, or run through machine learning for research. The feedback has allowed us to better understand how enter users interface with our API and helped us refine our own documentation and functionality. And to that, we are establishing in API advisory group, As one example of continuing these relationships and bringing some of the participants into that group. And finally we are excited to announce the newly creative, Office of Digital transformation at the Smithsonian, this department has been established to integrate harness and prioritize ongoing digital initiatives as well as create a pan institutional digital strategy. This, the silver project as a whole, it's a fantastic example of the type of collaborative and innovative process, we hope to promote with an audience focus mindset, across the institution. And to that point, we really wanted evaluation to be a key component of the project, to measure and assess the creative commissioning process and identify areas where we can leverage such models of innovation in the future. So Kate, Haley Goldman and her team HGM co lead a series of focused interviews and deployed assessment tools throughout the process, and she'll be sharing on that next. And just personally wanted to say, again, it's been an amazing team to work with a lot of fun to see the various ways the creative teams made use of open access and connect with audiences, and I'm so thankful to be involved in various aspects of the project from planning. The selection process, and now hoping to extend the prototypes further, and these types of initiatives really help us get a step closer to realizing Smithsonian Secretary bunches vision of being in every American home and classroom. I'm going to share a few links in the chat if you'd like to learn more about open access, but I really look forward to hearing your questions and creative ideas and with that I'll turn it over to Kate Haley Goldman, principal of Hg and CO to highlight evaluation component of the project. Thanks.

Unknown Speaker 40:41
Thanks so much, Ryan, I just want to note that I know we're running a little short on time so Kay is going to try to squeeze it in. And if we, if you don't mind sticking with us for another minute or two, that would be amazing and then Barry Joseph just has a little announcement to make at the end. So Kate, take it away. Thank you all so much for your continued attention,

Unknown Speaker 41:01
and Kate Haley Goldman principle of he and Co. whites this woman, I'm coming to you from Miami today we're we're working at that Perez. Art Museum of Miami and I'm going to share rapidly, some slides I think. So our role was really about supporting the design teams about conducting that user testing with my colleagues Leslie at menlyn did interviews with both users and then later on with with the all of the different stakeholders to look at the impact of creative commissioning and of these projects in general. So, from a design team perspective, there was quite a bit of different structures in place, right, that supported the things that you've just heard of, and then from the Smithsonian perspective, there was a variety of different elements that that made up creative commissioning is as a very different way of doing things and there were issues that Lisa so well covered about how an RFP process locks us into certain set of ideas or certain set of individuals who can propose those ideas, and in order to break out of this process, the individuals at Cooper Hewitt and elsewhere, creative this creative commissioning model since this was a first round go there's sort of a couple of different elements to the evaluation that and one is to see what was the impact of this process on the individuals who participated in it, both internally and externally. And the other part of that was to see if there are pieces of this process that we've had these elements from the external point of view these elements from the internal point of view, to change if we were to do this again, what would those elements be now I have very little, just a minute to cover that. But I want to say that there were significantly different stakeholders, and each of those stakeholders in our end interviews felt like their goals had been achieved. When one common unifying heroes the list of the stakeholders including Cooper Hewitt, the design teams the open access team, Ryan and others the funder varizen And then the public and I've grayed out some of these because we spent less time on the public given that this was a prototyping element. It did achieve some of those things that JOHNNo and Lisa and Rachel and others are talking about space in structure to develop these ideas, having different voices, fresh approaches equalizing that playing field and balancing out that risk and viability, there were some really important elements for the design teams in terms of getting user testing was one of the, the high points, but even more so that inclusivity components, and that structured feedback back and forth and advising and inclusivity and how to deal with accessibility issues. Overall I would say for the design teams, and for the Smithsonian folks, we were talking about relationships so on each of these slides that I'm now going to stop so that we can have questions, relationships and building those deeper conversations, was one of the key outcomes, which isn't it really interesting model, as we move forward. I wish I could talk about more they're such rich ideas about how do we change contracting to build a more equitable and a more diverse field with risk based ideas and viable ideas and that we'll have to wait as we continue this conversation. Thank you.

Unknown Speaker 44:53
Thanks everyone so much and Barry I know you have a quick, a quick announcement before we close, and also just FYI, my email Rachel is in the chat so if you have questions also Ryan King put his if you have questions please feel free to hit us up and I think, based on what Barry saying I will be available this afternoon also to ask questions okay now I'm very sorry.

Unknown Speaker 45:14
Thank you, Rachel. Hi everybody I'm Barry Joseph, I'm one of the three chairs of the nCn experience design and immersive technology sessions that we're all experiencing today, after the next session, the one following this one all six sessions will come together to review highlights from today's theme and hold the discussion across all attendees, like yourself, this will be a 315, Eastern Time. Cade if you want to share more during that session, please just let us know. And in addition to all. Please keep in mind this will be strictly BYOD J, bring your own bad joke. Well, hope you all come and join the conversation, Ben.

Unknown Speaker 45:50
Thanks everyone.