Expanding and Enriching Metadata through Engagement with Communities

Unknown Speaker 13:14
Hello everybody. Wait a second, so that the rooms is complete. Okay. Should I start then. Are you ready. Okay, so now we are having a panel where about with the title expanding and reaching Metadata through engagement with communities. The panelists are Isabel Brador of the ACLU of wall Sonian Jessica BrodeFrank, of the other planetarium, Bri Watson, of the University of British Columbia is call me average. She is currently a digital curator at the British Library and the panel will be will discuss how cultural institutions are engaging various communities to co create academic research and object Metadata in order to increase their presentation, access to collections highlighting how this is done in different ways to engage specific audience says, and goals. So to graduate students assesses assistantship Museum, interactive experiences crowdsourcing and professional action groups. So with that in mind. You can also ask for questions, comfortable for you to cover about ethics design, interaction, crowdsourcing projects action groups, and student based assess assistantships, so I'm very happy to welcome you to lead you to the room.

Unknown Speaker 15:28
Thanks everyone for joining us, Isabel, are you able to share our slides,

Unknown Speaker 15:39
just while that's happening, so we each speak for about seven minutes and hopefully have time for discussion afterwards. And please do share questions and comments in the chat as well. We'll keep an eye on that. Thank you Sam for that introduction. I'm Mia Ridge, I work for the British Library. I'm a middle aged white woman working from home in London, and today I'm thinking about how we can engage glands themselves are galleries libraries, museums and archives, with the process of designing projects to enrich Metadata. Next slide please. So one of the first things I have to get out of the way, is the idea that asking the public to collect and enrich records is new. The image shows James Murray one of the early editors of the English Oxford dictionary in the 19th century behind him, are slips of paper that were sheets of paper sent in by the public, to help them define and provide early examples of words for the dictionary. And then we've moved through the sort of folksonomy and tagging prey craze around web 2.0 In the early 2000s And an example from the Steve dot museum project thinking about workflow for tags for artworks. Finally, thinking about how machine learning and image classification computer vision methods can be used to classify and work together with the public, to provide terms that are appropriate to describe works of art in this example from art UK. Next slide please. So crowdsourcing enables galleries, libraries and museums and archives to co create Metadata by contributing tags or stories. I'm looking at some British examples that went beyond that to really thinking about opening up collections for reinterpretation and richer records then just tags or transcriptions. In 2006, revisiting museum collections was a project and a process that supported museums and archives in opening up collections query interpretation and capturing knowledge from community groups and external experts. The idea was to help build and share a new understanding of the multi layered meanings and significance of objects, and the records about them. They aim to Capture and permanently associate this newly acquired knowledge within the documentation of the object to give it a new voice, and a new perspective. So they were aiming to build in a process of community interpretation, into the very fabric of the systems that recorded information about their objects. And the Museum of London's reassessing what we collect project was, which is shown as a screenshot. Aim to engage proactively with London's diverse communities and to develop the museum's collections, so that they reflected and recorded that heritage, with a very specific aim of redressing under representation, thinking about diverse groups along lines of gender diversity minority minority ethnic groups, faith groups, groups representing different sexual orientations and gender identities, and in disability as well. So that was a real moment in the mid 2000s Next slide please. So we know that crowdsourcing is hugely successful that could be, whether you're defining it as a number of people reached the number of tasks completed or the opportunities for learning that it provides. So the screen shows some screenshots of history projects on Zooniverse representing a range of different kinds of projects often transcription or classification projects working with historical records from different kinds of organizations. The number of pages transcribed on the from the page platforms, which is focused around the transcription of handwritten text particularly using kind of collaborative transcription processes where people successively improve each other's transcription of text so they've worked on over 1.3 million pages at this point. And finally, the screenshot shows conversations on History Hub where volunteers, share questions and share things that they've noticed, and hopefully learn through some of the conversations they're having around the process and the historical records that they're working on. So we know that Kress I think is really successful, but. Next slide please. Where does the where does the data that results in is actually live.

Unknown Speaker 20:27
The results aren't always as integrated into internal processes, as you might like them to be, and you've probably guessed that this image, the farmhouse and the silo is the result of an image search for silos inflicted comments which in itself is an example of an early crowdsource project that collected tags and stories about images and aimed to bring those back into the records of the institutions, of course, that process of reviewing the tags and reviewing the records and reviewing the stories, took a lot of work and MANY institutions weren't actually able to find ways to take that data back into their collections. The results of the crowdsourcing projects might also be lost when social media platforms close as we've seen with people using Facebook groups to collect community history, their tags of the transcriptions might be lost when systems migrate or when the person who knows where they are moves the institution. So collections management systems still don't really easily allow knowledge from external communities external experts, people with lived experiences of the objects or cultural knowledge about the objects to be recorded, alongside the Metadata created by catalogers and alongside research from internal people. The next slide please. So having seen both the opportunities and the challenges of integrating crowdsource data with our collections management systems and uniting views of objects with internal and external views. I think the challenge that we face is that the tasks that volunteers find most enjoyable, don't always align with the kinds of Metadata cataloguing standards that we might have, where you might be applying standardized vocabularies, for example, rather than free texting. So it's a kind of classic square peg in a round hole issue, but I think more than ever before we need to ensure that our collections have information that represents diverse views about them they don't only have an institutional viewpoint. So my challenge is that I want to think about how we can really finally address the structural issues that create those information silos, how do we apply the code design principles that we apply so well when we're thinking about community engagement and crowdsourcing, how do we use those principles with our colleagues, so that we can think more creatively about the kinds of tasks that both engage people but also enhance and expand Metadata. So that's the challenge that I want to leave you with today. Thank you.

Unknown Speaker 23:10
Hello, I'm Jessica Bertie Frank, I'm the Digital Collections Access Manager at the Adler Planetarium and today I'm just going to talk a little bit about how the Adler has been inviting the public in, and engaging in some crowdsourcing projects both online and on site, inner interactive, I can get the next slide please. So just to start, I'd like to kind of set the scene by providing some context for why this work is taking place at the Adler and via a particular platform at the other we have been using this universe.org platform, which is a platform for people powered research. It's a way to connect some curious volunteers with researchers who are looking for volunteers who are interested and able to take part in public research projects. It actually began as a single astronomy project in 2007, and has since then, supported hundreds of research projects and collaborative teams all over the world, amassing a community right now of 2.3 million registered volunteers. And if all those volunteers aren't enough the Adler's universe partnership has also come to be because half of the Zooniverse team is actually embedded at the Adler Planetarium, as part of our division of science engagement and visualization in 2016 the Adler received a National Leadership Grant from the IMLS, which included some support for an internal project aimed at increasing access to the collections held in the Adler's collections department where I set the larger grant funded project was called Transforming libraries and archives, through crowdsourcing and it aim to expand the tools and resources available on his universe platform to support galleries, libraries, archives and museums, as well as other project teams who wanted to build crowdsourcing projects to support research in the humanities Zooniverse was really good at science based projects but had not expanded very far into the humanities. So this IMLS funded effort led to the first Adler's universe collections collaboration which we called mapping historic skies and you can see it on the right here in the interactive exhibition space at the Adler, the project ran for 15 months and three of those months, it was on site in our exhibition space but the last 12 months it had to be moved completely online on Zooniverse org, due to the COVID closure of the Adler, the project asked volunteers to identify individual constellations within a collection of about 4000 constellation depictions within the Adler's collection. While attempting to expose volunteers to the way that constellations have changed and been represented across different cultures and times, while also simultaneously building a database of these individual constellations, and due to the success of this project, both from an engagement perspective with over 7000 volunteers participating in that time, and a Metadata enhancement perspective, as, as we saw an accuracy rate of identification of constellations of about 98% the Adler greenlight a secondary Zooniverse project that began this year can get the next slide. So our second universe host your project is called tag along with Adler, and it looks to bring volunteers into the world of cataloging it launched this March, and is a Metadata tagging project that looks to bring volunteer generated Metadata tags and terms into the Adler's cataloging system to improve our search ability and find the ability of our collections online, but while also providing an engaging experience in which to explore with guests and volunteers, the way that language underpin search functionality in their everyday life online, in part due to the success of the constellation images we use for mapping historic skies we again use the Adler's visual arts collection which includes Rare Books archival photographs as well as MANY works on paper, but unlike the mapping historic skies project the tagalong project was not designed around a single collection or a single output goal of building a database, instead it's around changing the way that collections are cataloged at the Add Layer, and also really exploring this need for transparency in our decisions that affect search and discovery for our users. The tag along with Adler features two different workflows, and one of those focused on verifying tags that were created by two different AI models, and other the other focused on just adding volunteer generated tags. The project text that we created, how to very specifically mentioned that there was no wrong answers in this project because this diverges, quite a bit for my typical Zooniverse project that often operates under a need for consensus or a single right answer. It also clearly states that this project unlike our previous projects is a part of an active research into the use of crowdsourcing as a tool for actually engaging our audiences in a process, in this case a process to tackle a semantic disconnect between how museum staff and museum users describe objects as well as between algorithms like aI tiny models, and the language used by the general public.

Unknown Speaker 27:58
Next slide please.

Unknown Speaker 28:01
So as of last week the tag line project is about 80% complete and we are processing data sets as they are completed in chunks of 100 images. So so far we've done early evaluations on approximately 600 of the 1000 images in the project. We've had over 1000 3000 registered volunteers take part so far, and they've created over 200,000 tags for these images, and we're already demonstrating this in the anti gap this project look to address a full survey of our other cataloging data was conducted as part of the project development to create a baseline understanding of what kind of language the other actually used, and was available on our online search portals, and you can see that here in the red. Our most frequent terms really were, years of be of creation. Item Type significant maker the locations of where objects were made. And in comparison the terms that our users have been adding which you can see in purple on the right have really lacked that form of specificity, but have really been geared towards describing what is visually represented in the actual images. So a further importance of this project was actually the primacy of the participation on the AI centered workflow of the two we've consistently throughout the project now seeing that the AI workflow has seen an engagement two to three times that of the generalized tagging model, and MANY of our top board conversations have centered around questions of why AI models aren't more accurate why they're not better why some of these tags don't really make sense and you can see some of the tags here in blue in the center, and we've had opportunities to discuss the true promise and possibilities of AI and machine learning while also being able to transparently show why human centered participation is so very much key to overcome some of those faults and Lakin's. We're actually currently brainstorming how to develop this project into another interactive to be placed on site when we reopen in March. But we have to rethink that approach just based on the limited capability of this universe museum mode tool that we had previously used to make the mapping historic skies interactive, so we're also considering designing a gamified version of this tagging project to better test, engaging Mac metrics as well as kind of enticement of participation. And can I get the next slide please as well. So this project also included an optional survey and the responses shown here, only account for about 5% of our project participants, but they do help to support some additionally identified needs for the crowdsourcing projects within museums, outside of just outsourcing that cataloging labor, approximately 22 to 23% of our survey respondents have told us that they don't feel like their stories are represented in our collections or in museum exhibitions and 13% of the respondents actually said they didn't feel like they saw anybody in science today who remind them of themselves. When we combine that with highly extremely high markers that participants felt that museums are essential to communities and that communities are essential to museums, there's a marked opportunity for museums to leverage their place within the community to begin participatory experiences that would bring the public into the process of description, helping to increase that representation that's obviously notably lacking. There's also an opportunity for museums as a place with Mark standing in the community to help foster discussions on searchability and discoverability on the internet, our survey showed, roughly 30% of our participants did not trust what they found online. But conversely, about 75% of them believed they could find things easily. And we just believe that the level of transparency in this study and in crowdsourcing projects in general, it can help a Ford have an effect on the trust of online search, and really place museums in a place to have that discussion with our guests and audiences. And so with that I'm going to pass it over to Greg.

Unknown Speaker 31:57
Hello. Thank you everybody for joining today. Next time, before the beginning, I just like to say them speaking here Carrasquillo Saber Tooth and Squamish, have a traditional ancestral unceded territory taken, often violently of a so called settler nations of Canada and the United States on Turtle Island. I am a descendant of Scots Irish and English settlers, and as such I am a beneficiary of colonialism and genocide, Canada and just continue this genocide today through pollution, criminal neglect cultural extermination and murder. I aim to learn to do when I am able to do on the land and the people I live and work with, and I hope you do as well and take some time to reflect on that. You're not aware of the nation do you live upon native nationally on the.ca is a great resource for this. Next slide please. Main Screen Watson, Bri, hi cope I'm Ryan and Watson may still be publishing with that as well, a PhD student at the University of British Columbia's high school, I focus on Ecuador cataloguing and archives galleries libraries, museums Special Collections GLMs. I have a master's in library science, Indiana University and I have an earlier history degree from during university. I am queer non binary to Staples and white settler so in what follows I'm not presuming to speak for trans or non binary folks of color indigenous sexualities and so on. But I am really excited to talk and see and make some new connections, and learn from all of you, because as someone who's trying to work in the museum world unfold the archives and the special collection roles. A lot of these conversations are going on and they're happening in very similar ways and happening at the same time and I think there could be a lot more to gather by bringing everybody to the same cable. So in that spirit, and next slide, I have a question. Oh no, I think we skipped a slide. While the previous one is the name, there we go, thank you. I guess my slides have not loaded all the way in there, that's fine. My question was what is the name, and across the, and I wanted to say across the GLAMs world there's a knowledge that we need to revise revisit and redescribed both our collections and the vocabulary we used to label things. But I think that we're not taking full advantage of the vocabularies available to us within roles and the gallery roles and so on and so forth. But in doing that, we should privilege the language of the communities whose stuff we have. And to be clear stolen stuff should be returned. But if that's not or cannot be possible, it's not possible or it cannot return. We should reproach the communities that we're talking to that we're describing, and we should try to think about if these labels are the right labels for those communities and that stuff. And because my slides aren't reflective the slides up here. The next slide really was just me saying, I'm going to talk to you about two projects today. The first is the homesource which you have a little preview of here. And the second, and that's like more actual usable project that can hopefully take back to you, tuition. And the second is a project in development, with the transmitted data collective, the homeless service, which is the timeline you have up here was originally created in 19 in the 90s by a Dutch lesbian Library and Archive in a Dutch lesbian, a gate library. It was used as a standalone to stores to describe their collection, but it's kind of flat, it wasn't very well used, it's just meant to describe within their own particular collection, and over time they started adding terms for bisexuality intersex turns and so on but not in Metadata mythological way 2013 Jack Bender well with the help of Ellen Greenblatt transformed the original version into a link data vocabulary, which was discovered by KJ Ross and in 2015 when he was looking for a subject vocabulary for the homeless source for the digital Transgender Archive, which is a wonderful contract if you're not familiar with that, um, this vocab really only existed on a few websites, lived on a laptop for a while circulating among queer conferences, but when digital Transgender Archive adopted it, it became a link data vocabulary, it became, It was placed on line, and we found an editorial board this editorial board has been responsible for the second revision which happened 2019 Two years ago now, and that version really focused on changing it to a clear specific vocabulary that was meant to be used in association with another subject vocabulary library Congress subject headings recently and 2021, I think just about just about three weeks ago we introduced the version three, which doesn't really have any major changes but does have new stable your eyes so that way if language or hierarchies change to your eyes will still point to the exact same website.

Unknown Speaker 36:57
But next time, this is just say that we have been sorry, go back. Yep, this is this is just to say that we have been successful in our craft. I think more successful than we hope to originally at home a service is used in minimum of cataloguing of 7.7 million items and 33, petitions across 11 countries, in some institutions even replaced of CFH of the dominant cataloging classification language, but from a source alone cannot fix all the problems associated with these policies and procedures. Next time, and that is the job of the transmitted data collective, which is a group of about 100 cataloger librarians activists scholars, information and these DAM professionals. The work is still ongoing and open if you are interested in involved in this community and you're on to become more involved, feel free to reach out and get in touch. I don't know if the next slide will be reflective, so can we go to that one. We'll see if it's actually why it is great. Um, this is the origins and the history of the Medicaid collective drives this work from the archives for black lives in Philadelphia, which is a, is a Resource Description Framework for how you describe black lives and people of color in cultural heritage institutions. That really helps to move away from this neutral voice or traditional description, it helps encourage us folks to use, very active voice when describing oppressive relationships. Next slide. Great. And then this is just the kind of final slide and before I move on, but I just wanted to say these are the working groups that we have broke out and broken out into. Currently our plan, the descriptive profit classification name authorities and access and subject headings and 13th and most you have to work conditions on work group. These all these working groups are working together create several documents, which will all be combined right about September January, summer of this year January of next year, Hoping for circulation to the larger lambs professional in the first quarter of next year. So I hope you like to look out for that, and I hope to have competition for this, and I will hand it over to next vendor.

Unknown Speaker 39:13
Thank you, Bree Mia and Jessica, as mentioned before I'm Isabel Brador and the digital asset and collection data manager at the wolfsonian F IU and I will close out our panel by talking about the Metadata squad, a research incident engagement initiative, hosted by the Wilsonian F IU. So before we get into the nitty gritty, I'll give a little bit of context as to why the initiative was started so the wolfsonian F IU grew out of a private collection, we were gifted to the state of Florida by Mitchell Wilson Jr. In the 1990s, and at the time that we were gifted over, we were actually the largest donation in this history of the state of Florida, Just for context. And so it's a large collection with varying levels of Metadata in various states, some of it more complete than others. Lots of blank fields, and now into the 2020s. There are records that really haven't been examined or revisited, since their initial introduction into our database. So one of the challenges that we were having is, how do we verify and put our best foot forward on our digital catalog and to the public for 200,000 objects. The other issue or challenge that we were having really is, we are part of Florida International University and while that is an amazing and wonderful opportunity. We are located very far from campus, so we are actually on Miami Beach, where as campus is about 20 miles away, so student engagement was something that we were really passionate about, especially when looking closely at the demographic of Florida International University or FSU. So, f Ru is designated as an HSI which is Hispanic Serving Institution, 65% of the student population is Latin x, with 50% being Pell Grant recipients and 25% identify as first generation college students. So all of these demographics and numbers are really representative of some of the, of the peoples and individuals that have been kept out traditionally out of museum professions, and are inherently, you know discriminated against and excluded by these systems. So, that is something you're really passionate about, but the distance between campus in the museum was another issue. We wanted to provide engagement opportunities that were equitable and accessible and didn't pretty students that perhaps had more resources or more access to more time, so we wanted something that was a paid opportunity to help us solve our Metadata issue. And then in their comes B Metadata squad. So, the solution that we were able to, to work out with the help of the university is three graduate level history students they're usually in the MA track in the history department particularly looking at public history, we get three every year, and they do reap assistantship benefits so what that means for all of us in the room who might not speak, academic, is they get a tuition waiver for a full course load, they get a stipend that is paid out bi weekly like a salary, and importantly they do get the opportunity to enroll in the university health care plan. In exchange, and to also help us with our Metadata.

Unknown Speaker 42:26
They digitize our object files which are hardcopy and in a record room so very difficult to access. And they also research our object collection and we'll look a little closer at that process in just a minute. And then most importantly, and perhaps most excitingly, they participate in professional development opportunities. So what that looks like is, one is a cover letter and resume writing workshops specifically geared towards the glam sector, they do one on one interviews and to get an intensive review of their cover letters and resumes. We also help apply for specific jobs, they participate in career readiness sessions, which include things such as like how to network, we did recently do a workshop on how to address microaggressions in the workplace. And then we also have department talks, so students come away with a holistic understanding of what different museum departments do and how they advance a Museum's mission. So what does this work look like, right now you're looking at a very large CSV file. This is the safe way for our students to interact with our database without worrying about accidentally deleting any information or changing information, unintentionally, and you'll look at the first seven or eight fields that are in blue. Those are the fields that they're really working with so they're really looking at who is the artist maker, what are the dates of that artist the nationality, the country that this object was made in the dates of that object when it was made and a description, and they're really looking for answers so you'll notice that some of these fields are empty. So those usually take priority but then they go back and they also look at all the fields that are filled and start verifying, so what does that look like. Oh, and before I move on, you'll notice that here on the right hand side there are some fields that aren't highlighted, those are more for just the completeness of the record. Students have access to that as well. So, this is to mimic the chaos that is usually on a student's screen, there are all sorts of sources and screenshots. So what they do is here in the top left, you'll look you'll see a very old looking document it's been scanned quite a few times. It's a condition report that's part of the original object file from 1989, and they'll have access to that they scan that so that other staff also have access, and then they start doing online research, so that takes the shape of a few forms so one is they consult sources that we all know like Library of Congress Google Books, the Getty union List of artists names. They also reach out to different professionals at various museums around the country and across the globe. So they do get that networking experience, they get the fear of cold emailing kind of out of their system as well which is something I think we all could could benefit from any practice with. And what's really interesting is they save all of these screenshots, which is how I had access to them, and they're put together in a digital object file that is then kept up in perpetuity for other researchers to then build off of their work, and they get credit for all of their work to they have a research log in a text document. that's also accessed in an evergreen format. So a quick example, are these three figurines that were in our collection. They were researched by a student in our collection we had them as being created by an artist named address Eckhart, so through research, we were able to actually unearth a really interesting story. Turns out the address at CART was originally Edith at CART she was part of the WPA program in Ohio, so the federal Arts Project in Ohio, specifically. And what was really interesting is when Edith now address started her career, it was really interesting because she consistently kept losing in art competitions, and she was losing two male colleagues who really didn't have the same technical expertise or the same artistic skill. So she decided she was going to change her name on the applications, and she went in as address, which she deemed as being more of a gender neutral name, and she immediately started winning, which was fascinating,

Unknown Speaker 46:35
because these were judged just based on applications so it's a fascinating little story of how this work does make a difference so all of the work that's done is immediately sent to our curatorial department for verification for just review uploaded to our database and then pushed out to our digital catalog. So very quickly, just to close out before we get to the q&a. Some of our outcomes and pardon the different colors in my bullet points, they're all supposed to be hot pink. This is the sixth year of the program. We have digitized and researched about 6500 objects. And that's over the course of 18 different graduate assistantships post program there's 100% employment rate, and that breaks down to about 80% in the glam sector, and we have 100% graduation rate from respective programs. So I'm really excited. We're going to be expanding this program. So we're hoping to move beyond just the history department as a unit, and engaging others on campus and maybe even beyond the humanities, and I'll now turn it back to Sam and to the q&a Thank you

Unknown Speaker 47:43
so much. Thank you so much.

Unknown Speaker 47:48
I've seen a lot of discussion on on the chat I was contributing to whether I didn't finish it. So there are questions there, that could we could start from, like it, and other issues. If someone has direct questions we could start with the Diag boys question. Otherwise, I have some questions to put. So, is there anyone who wants to put question, or don't see rising hands. Start. This topic is permitted professionally is usually valuable and really interesting. Also because it is now, it's more than in the past has to do with our knowledge and our history. In, one point is the fact that artificial intelligence nowadays, USC you have not mentioned it with machine learning, and Jessica has made also very nice representation of it in the slide. What is your, your perspective what your point of vision, about the fact that there are human biases when imitating, so it's one side is when you have the persons that are mutating that our general public. So there are biases that we have, because we have prejudices in general cultural ethnicity, religion and whatever. And on the other side, artificial intelligence is biased itself, because it's developed by humans. So it is an hidden bias inside of artificial intelligence, why, on the third position the first and the very traditional scientific annotations, even though there is more hidden. But even there, is there some kind of biases. So how do you face this topic with your your activities.

Unknown Speaker 49:41
I think just to just to get started, um, anything that we say is the content is a product of our times, so there's no, we can't escape bias, we can just acknowledge it, and we can step away from claims to objectivity. And I think people are doing work now to acknowledge subjectivity of the terms that they're getting. I think things like computer vision methods based on machine learning or AI, at least have the advantage of applying classifications that you can see so you can review them. But I think people fell into a trap of imagining they didn't have to review them that the computer was going to get it right and that's just so not the case. But that, at the same time means that you can't just apply a machine learning tool and get computer tags for all your images and your job is done. It actually means it's almost as much work to use those computer tools on machine learning tools, at least for the setting like levels of confidence with some of the tags that they might apply, and having to like actively maintain things like stock word lists, so there's just some words that can never ever apply to images, but it means that it's, yeah, it takes away that that sense of like just press a button, then it's all done, but I'm sure Jessica's got a lot more to say in that,

Unknown Speaker 51:13
but also before Jessica, and also take into account that the point would be also. This does this mean that you have continued to renew the Metadata annotations because as soon as we find that you, as it appears, you, you have to refresh it with the our age, so you said as Mia said, there's our age that influences what we, we, right. So, all these aspects, might be, and it is not taken because we don't refresh, so much, often the Metadata imitations of our collections. So just

Unknown Speaker 51:49
know that actually ties nicely, and I'm kind of following what he has that I think that's the important piece to this first and foremost notate we've moved a decent amount in the field to recognizing like museums are not neutral, and it is not no matter what we're doing in the museum world whether we're cataloging or we're using an AI to catalog, it's not neutral, no matter what we say. So I think that's first and foremost important, and that's actually part of what we're working at at the Adler to push some of these projects into an engagement space and into having some learning objectives so our tagalong project specifically features two different AI models we did not build, because we were very transparent in our project to say we don't have enough of a collection to even build a model that would operate. Functionally, and be able to recognize our collection is very small, we only have about 3000 objects and we can't train a model to really notice anything so we've used the model. One of them is from The Metropolitan Museum of Art, and obviously the Atlas collection is very different. So when we ran our images through there we're going to get some wonky classifications because it's not ours, but at least shows some of what a museum built, and the other one we used was the Google vision. Cloud Vision and that partially underpins a lot of the searches that our users would be doing throughout, you know their daily life on Google and so that gives a little bit more but again there's already some great ways to engage with our volunteers in that discussion because they're starting to see some of the bias that these models were trained with where we'll have two pictures of scientists in our archives, wearing white lab coats because it was the 1960s and that's how we made them look professional at the Adler, and the ones of men are tagged with Doctor scientist and the ones for women or tag nurses, and they did the same job. And so it's, to have that discussion of who trained these models and what did, what were the images available to them and what was, what were they using to make these kind of classifications. And to make that clear to me as point there's a great promise to AI and machine learning in certain things, it's really good at, but other things, it has some struggles with so when you don't have a collection that is you know, none of us have a collection that represents everything in the world so our models are never going to be perfect and I think that's the piece. We really see great engagement with and especially with our AI, the public is just as interested in AI as we are and they hear a lot of really kind of fascinating promises of what's coming up and I think it gets to be a nice discussion point, and then that also plays into our continuation of running this project is making it clear with the public that terms that were used to catalog 50 years ago are not terms we want to use today. And we've had to have that discussion of why some of our cataloging records include terms like we are not going to describe people as negros oriental anymore but that is in the old records, so it needs to be discussed and they need to be enriched and, you know, really fixed up for modern language and we can acknowledge that in 50 years, it's probably still going to need to be so creating it into an engaging experience that kind of lets our guests on site as well as online users just kind of continually add to the narratives we have has seemed to kind of be our. We're hoping our response to those kind of questions.

Unknown Speaker 55:19
Let's try one question so for Bri and then Isabel is your. Sorry, can your experience and your, what you have explored the use as Monitoring Instrument or some aspect that you can use inside this experience so that it detects the gender issues that has to do with Metadata annotations and data flow.

Unknown Speaker 55:52
Question I think it reflects, maybe to some of the differences between the fields, maybe perhaps link data could be used in a manner to the tech, but as like most of the vocabulary are open link data, they purposely don't have any like sort of tracking or like tracking stuff built into that. And I think just building off like one of the questions here in the chat too in I can't speak comfortably to New Museum context but one of the things, there's a lot of institutions are finding that these problems are way too big for them to tackle themselves, which they are their societal problems there in fact we all bear responsibility for them so we all responsible for fixing them. And some of these terms like are they, like, if something is Metadata is from that has been used in court battles to decide whether or not an indigenous nation gets land back from the Canadian or the American government, and certain terminology for queer folks especially has, there's a direct line between the harm caused to a queer person by the description term library or archive or museum for a long time in the asexual community. The only term and lcfc or library Congress subject headings was asexual plant reproduction. So that's led to a very long running bitter joke and that community about being a houseplant because of that, so you can see these two really direct lines between between like what we think is like neutral terminology and like the harm it has in people's lives.

Unknown Speaker 57:22
Yeah, that's me. Thank you, and Isabel. What about your next steps. From the Actions, I know that we have short time probably but. So what, what is your point, that.

Unknown Speaker 57:39
Thank you Sam. So one of the things that we're doing is we're currently looking for grant funding to branch out to other units so particularly we're interested in partnering with on the stem side the GIS so that your graphical information sciences team, but also other humanities based units, so we're particularly interested in working with the Gender and Women's Studies department here on campus, as well as the African African Diaspora Studies, the Latin American and Caribbean center that's here as well, just to make sure that we're getting a variety of perspectives on our collection. And I think this kind of feeds into one of the questions in the in the chat about curatorial authority, and how that works in a museum context and one of the ways that we really started kind of chipping away at that that hierarchical nature that happens in museums, is really inviting curators to work alongside with us we have a curatorial liaison that meets with our students, once a week, and then we really start establishing the students as authorities on research so whenever a student, or rather a curator has a question about research or research assignment we chat with our students and kind of have like a rolling pool of like who would like to take this, and by the end of the semester by the end of the year really traders do come to see or beginning to see our students as experts in their own right and you know colleagues that they can reach out to so I hope that answers to questions at once.

Unknown Speaker 59:04
This Perfect, thank you so much. Other questions. Others want to enter in the discussion. Okay, that's, that's great. I really appreciated your presentation really really interesting and really up to date, it is the topic that is very hot. And so, good job, and really enjoyed. So if there is nothing else I think that we have finished. Okay, yeah, I'll leave you that that link, That is, I think it's really, really, on the topic that you're working on. So, goodbye.