Collection Explorer: A generous interface for the Williams College Museum of Art

Unknown Speaker 00:00
Hello, everyone. Welcome. Thank you so much for coming. My name is Chad Weiner. I'm the Mellon manager of Digital Initiatives at the Williams College Museum of Art.

Unknown Speaker 00:13
And we're going to talk about a project called collection explorer. At generous interface for the Williams College Museum of Art, one of the things I've been in museum technology for for a little while, and I've just recently dipped my toes into the digital humanities community. One of the things I love about a digital humanities conference is that at the top of the slide deck, they'll put a credit slide often. I love that idea. It's something that I'm going to do from here on out. And it makes the point that none of these projects are done by one person. And certainly, unfortunately, my team can't be here today. This project was certainly not done by me alone. There's a whole host of people and I wanted to call that out right away. In particular, my with my digital team, Rachel disown our associate registrar, Jim Allison, our collections developer, Beth Fisher, digital humanities, postdoc, and I'm fortunate enough to lead that team, it's a Mellon funded project, three year project, and we're in year three, and our work is still not done. We're we're excited to to move on. Along those lines, what I'm going to show you is work in progress, you're gonna get a little bit of a behind the scenes look at our thought processes, some of the systems and ideas and open source code that we looked at, to get where we are. And I'm going to try to give you a sense of where we're gonna go as well. In between those things, we're going to do some live demo in the browser. So see the pants stuff. 233 sections of this talk that important to note, one I'm going to talk about the Williams College Museum of Art, this project is pretty distinctive and fits the needs that we've seen at the Williams College Museum of Art and fits the way Wickman works in some interesting way. So we'll we'll start out with that. Then I'm going to show you some examples of some prototypes and some of our thinking that really started off the project. And then we'll do that browser tour. And we'll see how that goes. So yeah, let's start off. This is the Williams College Museum of Art. Here you see it in the snow, which it often is. It's in the beautiful Berkshires in Western Massachusetts, not far from New York State not far from Vermont, nestled up in that north west corner. The Art Museum is a teaching museum and has about 15,000 objects. It's about a century old, and a relatively small staff. We've got about 2000 students at Williams College, which is a small liberal arts college. And one thing I really want you to know about Williams College Museum of artists that of all the institutions I've worked with, it's one of the best that I've seen at using its collection and empowering others to use the collection. And I'd love to show you some some of the some examples of that. So this is a gallery shot of a gallery called Object lab. This is a space that we encourage faculty members from across campus to use to teach. We encourage them to teach. Of course, within their own discipline, we're not expecting them to teach art history. We want them to teach chemistry through works of art, or neuroscience through works of art. This happens to be a geosciences professor. What you can't see on the other side of those walls is that we've made it a working space, there's a place for a projector on the left, that he's pointing to. And on the right, there's a little shelf to keep his laptop. And so it's an active teaching space. Here's a better view of the whole gallery you can see that faculty members have been encouraged to select four to six works of art from the collection to teach their respective classes. You can see there's a there's a cluster on the right bright colors, hard edges, that is a group of works of art that were selected by the neuroscience professor, very interested in in how the eye works with the brain. There's a single painting straight ahead that was used by a math professor and geosciences is in there as well. So again, very keen To empower

Unknown Speaker 05:03
faculty from a range of disciplines to use the collection to do their own work. And I say that and I emphasize it because it's it's certainly it's it's a particular strength of Williams College Museum of Art. But it's also an opportunity for other museums to think through how how best to, to make their collections relevant and usable to their audiences. Our key audiences, of course, are our faculty members and our students and our community more broadly. This is a nice close up from a biology class of works of art that were chosen. For this class, you see a range of works of art, a range of media, as part of my project, and we're not going to talk about this too much today. But we're really interesting in gathering the use data of our collection. So the idea that these works of art were used, carefully chosen, and brought to bear on a biology class. That's never, that's not something we ever would have come up with. As a as an art museum. That's, that's focused on largely art historical context. So we're eager to gather the context of what happens in that biology class, and bring it back into our collection data and, and then expose it in really interesting ways. And this is maybe the third example. And I love showing these of the collection being used in really interesting ways. This is an example of our walls program. So there's a subset of the collection that we allow students to take, essentially on loan back to their dorm rooms. And so a student can have a work of art, from the Art Museum on view in their room for the semester. Yes, it's a fantastic program, we actually have students, it runs through the through the length of the semester, we have students that camp out, in order to get the work of art that they really want. I went to Duke University as an undergrad, we used to camp out for basketball games, at Williams College, they camp out for works of art. Those, this is also kind of gives you a sense of what Women's College Museum of Art is about. Each of those works of art in that walls program comes with it. A little journal that travels with that work of art. And each student that that borrows that work of art is encouraged to write a little something journal entry, a personal reflection, about their time, their semester without work of art. And that reflection is passed on then to the next borrower. And on and on as the chain continues. From the digital side, we have a really compelling and amazing in many ways dataset made up of these journal entries, which had been digitized transcribed. They're set aside, this is a particularly special dataset, not for the public, some of these reflections are personal and touchingly, so but they give us a real sense of how these works of art are used and loved and a little view into the students worlds as well. Another way that that as part of our Mellon funded project, we've really struggled to, to open up the collection and allow it to be used in new ways is to make our data available. So we have our collection data available as a CSV file, which we've gotten the most use from computer science class, a stats class, student projects, all of these different areas of study are eager for datasets. And it's particularly compelling for them to have one that's close to home. And that relates to something that that they have a personal and emotional attachment to. And so our dataset has been used across campus in, in classes and in projects. Along with that,

Unknown Speaker 09:31
we have an open image set. And so not very many museums that I know of have one of these, these are essentially a zip file with all of our collection thumbnails available as a download. Because we found that the students were we're getting interested in visualization software particularly. And some of that software is easy enough to where you can point it to a directory and say go and it'll it'll get you started. At least, and so in this drive to, to jumpstart interaction, and drive use of the collection in interesting ways. We we decided to package up our images, just not thumbnail size, but perfectly adequate for these visualization tools and open them up for faculty, students and for the public. So that gives you a sense of how the collection is used. For almost all of those uses, you can imagine there's a step in there, where the student or faculty member is, is trying to make selections, they're going through the entire collection, trying to decide what of these 15,000 things might work best for my math class? That's a tough question. And we actually have, we're fortunate to have an academic curator, whose primary job is to connect faculty members and students to the collection. So it's almost a concierge service. Quick Find, find five works of art that might have to do with calculus go. And she's amazing. And she does that one of the tools she uses is our online collection. And it's it's nothing special. It's It's looks a lot like most online collections, we've found that it's not especially useful for our community. Because we're we're so interested in broadening the the use of the collection, our art history, faculty members and students that certainly know how to put a search term in that top box, you can see the the keyword searches is front and center. And there's even an advanced search and they they can get what they want from that no problem. art faculty, probably in that camp as well. At 10, tenured faculty member in physics, maybe not so much. They might not know exactly what terms to put in that search box to get the sorts of works of art that they might be interested in. Often they come to us with with more of a visual idea. Actually, I remember in our interactions with the neuroscience professor, for instance, he was most interested in contrast, bright colors, this color next to that color. And that wasn't something we were prepared to search for, especially through an interface like, like this one. So one of the things that that we explored to address this problem was a generous interface. And I put an asterisk here, because there have been some great papers here at this conference, and other conferences about generous interfaces, I'm not going to dive into it too much. I just put an asterix there, just so that you know, it's not something that I came up with. There has been some some wonderful rich work done in this area. And we have been fortunate to explore it for our own needs. And to utilize some of the some of the tools and projects that are out there. For us, though, if we can tie it to arts more broadly, and art history in particular, we're doing really well. And so there is a precedent for this idea. And I should backtrack a little bit. The generous interface is one in which you're not confined to that keyword search box. You're not expected to type something in to get something back. You're given the whole collection, you are given

Unknown Speaker 13:59
a view into the images that that you can then browse and look through this idea, at least from an art context. You can trace back decades, probably centuries. One of my favorites. exemplars though is this Andre Malraux, the Museo imaginaire, this is in 1947. He was imagining, bringing together images from across collections, across classifications, across timelines and geography, finding similarities amongst them and and trying to imagine themes and linkages between them. And this is this is more than the idea that that we were really interested in one of our first forays into thinking about this and this is within the first few months of our project was taking Those images from that image set and working out some prototypes, what does it look like we didn't know what the collection looked like, all together, all at once. So this is this is made from an open source bit of software called Image pot. And it's one of those where you can point it to a directory full of images and say, do your magic and get some semblance of an image like this, which is the entire collection kind of sorted north and south by hue. And Eastern west by brightness gives us some sort of look into what we have, and to what of what is ours. And this was this was an image that inspired and, and started conversations. And I've got to say it's one of the only projects that I've had that ended up as a product in the shop. So this is a postcard made from one of our very first visualizations. And that led to further exploration. So this is a we went from the static images to exploring some of these ideas through Tableau, which is a free, there's a free version online Tableau Public, a way to interactively explore these ideas. So this is a this is a view of the entire collection. This is 15,000 objects, organized in blocks by classification. So one of the classifications is painting. One is drawing prints. The different colored dots are the maker field, and our collection data. So you can get a sense of all the different makers in each of the classifications. Again, conversation starters for getting people interested, curious, wondering about the sorts of tools that we can come up with, this is a better view of of some of the data that we're really curious about, again, this was made in Tableau, you can see the mouse up in the corner is hovering over a square, which is a kind of a light blue, I think. And the popup tells you that this is a Picasso dance of Sallie Mae. And it has been exhibited seven times. So the darker blue squares, the works of art, those have been exhibited the most in the collection. So it gives you a sense of almost a heat map of what has been used in the collection. This was something that that we were able to go to the academic curator and discuss, you know, what parts of the collection had been used the most. And then she took it as a as a challenge taken on herself, to go to those places where there's nothing where there's no semblance of use, and pull those out and see what they look like and get faculty members interested in them. We also explored 3d spaces. There are some wonderful open source projects. The LDH lab has one called pics pot, which we modified and played with with some of our partners. And imagine the almost a 3d virtual space full of our collections. It was immersive and and amazing all within the browser. In the end, it was it was too much. So we turned up the volume to the max to 11. And so very valuable experience but but found that our audiences weren't as comfortable navigating through that 3d space. Much more comfortable with the Andre Malraux. Just lay it out on the table, the flat space. And this is another project

Unknown Speaker 18:54
along the lines of this sort of prototyping idea, and what would it be like idea? And I should say, too, this is an exhibition that we just launched this, this this fall. And it's part of the reason why on that second slide, I said, this is a work in progress. We put some of the development work on collection Explorer on hold to be able to do this. The opportunity came for us to do as a digital group to be involved with the exhibition program. And so we jumped on it. This was an immersive installation done with a design group in New York City called studio the green aisle. It's called all at once it takes one of our brand new galleries in a renovated section of the museum fills it with our collection. So that in a public space you can see the entire collection laid out before you you can get kind of a sense from this from this image. The collection itself is actually organized by visual similarity. So you can see a little bit and I'm dying to get up and be art historian here. But there's, there's an island up on the right of of lightly colored works of art, those are those are drawings, you can see just below that and to Agnes is right there, there's some bright colors that kind of jumped out that's hard edge abstraction that are all clustered together right there. On the bottom, you can see almost a shelf of objects, where the algorithm was seeing gray backgrounds and tall things, and essentially clustered those together. This, this allowed us to talk about computer vision, and visual similarity in a way that really connected with in a, in a physical way with our audiences, and allowed us to explore these ideas of seeing the collection in new ways and making connections visually, that we hadn't before. She's you can see she's holding up an iPad, there was a, there is an augmented reality layer to this as well. So that as you hold up the iPad, and find landmark works of art, you're given a little bit of information about those works of art. And you can see how they're connected across the visual field to other similar works of art. Some of these clusters have to do with basic metadata, like the same artist, or the same year. Others have to do with our specific metadata. So that neuroscience cluster that I pointed out before, that's actually cluster over here as well. And you can get a sense of that within the augmented reality experience. So all this out there, you there, you get a sense of what the screen looks like as you, as you pointed at the at the wall. One other component of this that we were fortunate enough to work on with the studio was essentially making this immersive experience. So visual similarity. compressed into two dimensions for the walls. Here, it's compressed into one dimension, one dimension is time. And so we're exploring visual similarity through through video, as well. This is only a snippet. If you see the whole collection, it'll last about eight minutes. But this is a this is a clever one, where the and maybe we'll find this in a minute to the Andy Warhol is merged into American painting in a really interesting way. All right, make sure you some stuff. So all that was meant to give you an idea of who we are, what the needs are, what the opportunities are, and how we've already begun to explore some of these things. So right now, I'm going to X out of this and dip into the browser. And we're gonna hope it all works. And we're gonna give you the latest on some of our prototypes for how we're thinking about addressing some of those needs we talked about. So again, a generous interface. So this is, how are we doing on time to 20 Oh, boy, a generous interface. This is the whole collection, it's helpful for us to see the whole collection laid out in front of us, just like that, that Andre Malraux.

Unknown Speaker 23:42
This gives you a sense of scale, certainly, but also allows you to do interesting things. Let's focus on the different areas of the collection. So you'll see up at the top, we have some of our metadata fields, kind of laid out before us. I'm gonna press on some of these, it'll highlight areas of the collection. So I'm I'm hovering over photograph now. And you're seeing which of these in the collection are photographs a fair number, I'm going to click on photograph. And those are all going to come to the top there. I'm going to click down, you get a sense of all the photographs we have here. We're going to zoom all the way in to see individual works of art here as well. And we went right into a deep view with high resolution. And just a little bit of metadata here as well. Again, unfinished, we plan on linking this to the online collection itself. One of the things we'll talk about that this is this is just another view of the collection. This isn't meant to be the penultimate view of the collection. In fact, one of the reasons we really start data and images are so students and faculty members can make views into the question we'll get to that. We can zoom all the way out. That gives you a sense of how things can be filtered. And framed, there's also there is a keyword search as well, if you want to go in that direction. And you'll have a similar interaction. So not as many Picasso's in the collection, but you get a sense, I will show you.

Unknown Speaker 25:40
This is a timeline view of the collection. So this is organized by accession date. So this is when works of art came into the collection, I would love to show you recreate one of the conversations I had with our curators who was interested in photography. looked at this, and we started exploring this together, zoomed in, just out of curiosity, just to see which was the earliest photograph taken into the collection. And unbeknownst to him, or me is this was a revelation to both of us was this picture. Beautiful African American lady, we ended up doing research on this, and this, this totally changed and is reflected in our reimagining of the permanent collection. That's that has just opened as well. We did some research on the on the artist, how the work of art came into the collection, we're still not exactly sure. But just an example of how looking at this, even internally, even with curators has has created a new conversations, new ways of looking at the collection.

Unknown Speaker 26:50
zoom all the way out.

Unknown Speaker 26:53
And then I would love to show you.

Unknown Speaker 26:56
Let's see. Let me go back to this one. Okay, so I shouldn't have done it that way. But these are all the Picasso's.

Unknown Speaker 27:12
This is a third view that I would love to show you. So this again, we talked about visual similarity, and something more that we were really interested in because that physics professor would much rather go in here and browse, then think of something to put in that search term. I'm going to show you that this area would have been really interesting for the neuroscience Professor these colors would be something that that he would he would be able to go in and and really dive into and find those that that that fit his uses. Those Andy Warhols are down here. Where are they? Oh, right here. So you can see these in a 2d way. Those Warhol portraits are jammed right up next to some strange kind of surrealist works. And then modern portraits. Not something that one of our curators would have put together well, one of them might have done this. But, but something that that this new view, this new access, and this new visual way to look at the collection allows us to see and again

Unknown Speaker 28:36
allows you to get kind of filtered views of the collection as well. And so we are going over time, but I'm going to close this out. And I'll have these slides available. This is a little bit of a of a look under the hood of what we're using here. This is not a framework that we developed ourselves that is key. This is based on the Vicus viewer by UC lab, the urban complexity lab in Potsdam, they have made this open source and we have been able to build upon it. It's built on d3, JS and Pixi. JS for animation, TensorFlow, and TC for the visual similarity. Goodness, what I love about it, it's essentially a bunch of files. All the pre processing is done by node. And you can drag it up onto a web server and the magic is done in your browser. These are some of the big concept ideas that that we were getting at with this and we're still working on. We're still trying to find ways to do these things. The idea of bringing the distance and the close together Not making them separate. Digital Humanities is sometimes chided for concentrating on that distant view. Art History is sometimes treated as as being too focused on individual works of art. We're curious about that, that zoom in between. We wanted it to be have a measure of delight, delight fulness and to be approachable for those faculty members, especially, the metaphors have been important for us. So a curators table is one that that we use the Andre Malraux putting all the images out on the floor. It has been helpful for us as well. And again, to stress we're interested in we see the final version of this as just one of many views into the collection. And we'll be encouraging students and faculty in classes to create those views with us.