Transcript
Unknown Speaker 00:00
Good afternoon, everyone. So my name is Johanna, I'm the Digital preservation manager at LACMA. And I'm joined by Morgan Kessler, media collections manager at LACMA. And our session today is called upward mobility challenges and preserving app based artworks. And I know you're all super excited to be here today, because today, as you know, is World digital preservation day. So this is just in the spirit of fun. We're continuing that right now. And so I should also mention that this has been research and experimentation that has been taking place over the last year and a half along with our colleague at MoMA, Flaminia Fortunato who wasn't able to come today. But maybe Amy can answer any questions about that, if it comes up. I'll give a little kind of digest of what they've been working on at MoMA. But for the majority of the presentation day, we'll be talking about the case study at LACMA. And I should also give a disclaimer that, you know, we've given this talk and a few formats, but it's pretty much exclusively been for audiences of media art conservators. But we thought this would be a really great opportunity to bring this talk to MCN given, you know, the people that are working in app development, and that is certainly a new area of inquiry for us. And so thinking about other strategies that might be employed there, though, you know, the case study is on app based artworks. And so we're building off of a history of software based preservation, and applications for artists based materials, which arguably are a lot more restrictive in terms of what kinds of acceptable loss can be experienced across time, we have very specific parameters that we've agreed upon with artists around what can and cannot change across time. But perhaps based on some of the things we'll cover, can open up discussions around what are the broader issues of sustainability that are affecting our field at large, particularly, as we increasingly face challenges with Apple and whether or not their partners in these issues? And so if nothing else, I think one question that could be explored today is what is going to happen with all of this stuff. And that's to say, you know, over the years, especially recently, Apple has released more and more peripherals and services and tools that are entirely proprietary, so they only work with designated devices. The newest Mac Pro's are becoming increasingly hard to work with in terms of accessing the internal hard drives. I'm still not really convinced that iCloud does what it purports to do. You can find me on that. But I mean, it's all to say that there might come a time when the field at large needs to find a way to hold Apple accountable because they're certainly not doing it of their own free will. Or are we just going to get further into bed with them until one day they decide to smother us with their proprietary pillow. So we'll see what happens. But I think we are finding more and more that if we're going to work with them, there's going to be a moment where we have to start making demands. So we're drawing from two case studies, one from LACMA composition for marimba by Mungo Thompson as well as w YDRN by Martine Sims, or you could read it as what you're doing right now. And again, we'll mostly be going into the LACMA case study. But we'll touch a little bit on momos as they have similar challenges, but also delve into some different territory and bring up some new issues in ongoing conservation. Just a explanation of who we are at LACMA. So, as you probably know, we are an encyclopedic art institution in Los Angeles. So collect everything from antiquities to contemporary art. So media art is just one narrow slice of the overall collection though we have around 500 media artworks in some form. And we do have a time based immediate committee that's comprised of curators registrar's, people in collections management and media installation, though the majority of the ongoing preservation work is sort of split between Morgan myself and Abby. And so as the digital preservation manager, I am there to envision what is the pathways for the future for all of LACMA is digital content that is considered to be retained for the long term. So archives are collections as well as the various kinds of
Unknown Speaker 04:48
interpretive materials marketing, trailers, things of that nature. So more just kind of architecting what is our larger strategy around that? More again, is specifically the Keep her and hold her for the media art collections. So the incoming acquisitions, ensuring everything is in a good format, the ongoing refreshing and maintenance of media. And then Abby is our dedicated well not dedicated, but she's an object conservator with specialization in electrical, electrical components, light bulbs, batteries, things of that nature. So, you know, physical components also come up in this these works a lot too. So, through our powers combined, we feel we have a good kind of angle on this stuff. So in looking at these case studies, we asked ourselves three main questions when we started out. So first of all, what does it mean to collect these kinds of works? Are we thinking about these in terms of their specific placement in a history of usage? Or are we thinking of them purely as display devices? Just what are some of the issues that come up when you're thinking about apps, and whether or not they're downloaded by users? If they're more part of sculptural works, it brings up a lot of questions. Also, what tools and methods are we going to borrow from our understanding of software based Art Preservation, though, as I'm I hope that we can kind of draw out in this presentation. You know, there's, we maybe assume a little bit of understanding this audience of what are some of these tactics, but disk imaging, emulation, virtualization, things like that are kind of challenged within this environments versus specifically with Apple based technologies that are kind of resistant to these forms of portability? And the last question is, how might this inform our understanding of who our partners are in working together on this, particularly within the tech sector in the commercial market. So these are our main kind of concerns, as we embarked on this exploration. Study, in case you're not familiar, just a brief overview of apps. So they can be broken down into three different categories, native apps, which are built for specific operating systems and remain faithful to those systems. So they won't be platform portable, necessarily. mobile web apps are delivered or rendered through web browsers. And so to that degree, they are a lot more interoperable. And then hybrid apps kind of make use of both to some degree. And in the case of the two case studies were going over. They are hybrid apps, though, I would say they kind of fall more into the for blackmans, at least the native apps as it is performed on a very specific device. And in terms of app development, within the iOS environment, in pretty much all cases, it will be performed through Xcode. And so that is a software compiler and package installer that's used for a variety of purposes, but is largely there to assist with the architecting of the behaviors and the events and the interactive components of an app. Most of the time, people are working with Swift though there is the underlying programming language, though Objective C is another option. But even, you know, not considering the choices you make within X code, around the software, there still has to be a destination device. So different versions of an iPhone will have different haptic functions, different screen dimensions. And all of these depending on what the what the work is doing will be highly significant as you make a choice in the destination device. And so as Apple is known for they are very slick, they're very about easy design, and things that are very palatable to the eye, which I understand is very important for a lot of reasons. But it does also mean that there is a certain reliance on how their kind of interfaces are pulling things together. And I would say kind of obscures the actual programming aspect of it or the way that things are encoded. There's very specific areas where you configure files within a directory and where certain behaviors are placed. So that is kind of another risk associated with working in this environment is in terms of defining what is the source code, it does have further reliance on these kind of user interface systems.
Unknown Speaker 09:21
And another kind of consideration is the way that it's distributed. So ad hoc or sideload, is when an app is loaded directly onto a phone. And so through X code, you would choose the device it's going to and then you can simulate it, but then you can also sign it to the device. And then there it resides. But then most commonly, you know, the App Store is where people place their their apps to be downloaded ad hoc by users. But of course in considering that routes, you do give a certain authorship over to Apple to make that app continue. As the available to decide when to take it down. In most cases, if an app does not work with a given operating system, I don't believe that they would do anything to make sure that it is migrated forward, it would then all likelihood just be taken down. But it's not to say that one is more sustainable and the other, even with ad hoc or sideloaded apps, there's still a requirement to sign up through a developer account. And so that will rely on database calls that authenticate that developer account on a yearly basis. So in the case where you would put an app on a phone, even if you left it untouched, and didn't do anything with it a year later, if you went to turn it on, and it hadn't been re authenticated, the app effectively is dead. So it's not That's to say neither one is necessarily the better routes. So now, Morgan will give you an overview of the case study.
Unknown Speaker 10:55
Yes. And I'm gonna be talking about this artwork, it's called composition from a rumba by Mungo Thompson. And as you can see, it's the iPhone six plus mounted on a tripod, a music stand. And the tones are linked to the playing cards, of course, let's see how slow the internet goes. It's a little hard to hear, but you can see it's very beautiful sound. So, I'll be going through over here, the artists deliverables the acquisition at LACMA condition assessment in our treatment. So this work was actually a gift from artist John Baldessari, and this was in 2016. Usually gifts don't come with artist interaction. But we were lucky to have Mungo Thompson and his studio on hand to actually deliver us the phone. He also included an Xcode project with all the needed files. So that was great. But the artist was very careful to mention that we should never update the operating system of the phone, because this might interfere with the playability of the work. And this actually went right on exhibition as soon as it came in. So they put it on airplane mode, and let it play. And we all thought, okay, great, we've got the phone, we've got the source code, we've got the X code, this acquisition is complete. And so this was only a couple years ago, back in 2016. So in summer 2017, we finally had the resources to devote to the preservation of this work. And we wanted to document the operating system, all the basic parts of the phone. And as we plugged it in and turned it on, we found out Oh, the app launched and crashed immediately. So we checked the software update feature, and that had not been turned off. So we figured, oh, what probably happened is we turned it on and automatically updated the operating system. And now it's not working. But our next step was to see if we can find the log of operating system updates. And of course, Apple makes it extremely difficult to figure out that kind of information. iPhones are especially under lock and key when it comes to content and system logs. And those are usually contained in the P list files. And you can get some data. But this is really only a partial view of the phone's activity, as many of the files are hidden. And even if you get them open, they're often encrypted. So without the aid of expensive forensic tools and the ability to translate this encrypted language, we were not able to figure out the operating system updates. But we did have a source code and Xcode project. So we looked to see if we could get the app working using that. And that was when we learned that we needed to save much, much more than what we had have been given. So we did have the source code, we had the X code, we also needed the precise version of Xcode in order to sign the code to the phone. Now, we didn't have a document of what that version was. And nor was there a version of that software saved along with the code. So we were effectively unlocking a chest that only contain more mysteries and error messages. So we started our experiments on a computer that had Xcode 9.4 installed on it. And we knew that this was going to be a problem because it didn't have the capability to go from Swift two to Swift three. And it didn't work. So then we installed on the same computer version 8.2 point one. And it also did not work because it was using all the dependencies installed with 9.4 point one. And simply by chance on my computer, we had only the version 8.2 point one installed. And this was a we were so happy worked. But this was a big red flag for us that we need to be saving much, much more we need the Xcode swift Mac operating systems and checking to see if there's any libraries that we need to save and affect the whole developer environment needs to be encapsulated with the work given all these tightly bound dependencies. So once we had the working Xcode project, we decided to update it To the current OS of the phone that was 11.4. And that was successful, we were able to get it working. And Xcode also has a simulator app, which can be very, very useful, especially if you don't have a phone on hand.
Unknown Speaker 15:16
But the drawback is that the simulator isn't necessarily accurate. In the middle, this is the artist documentation. And this is the simulator and this is slow down to 30%. So you can see how it kind of drifts in and out. So if you're going to be using this for documentation, it's really not the best idea. So we have the app working on our backup phone and the Xcode project. And now we really wanted to understand the work by digging into the source code. So we had enough information that we were ready for an interview with the artist to answer our questions. And it's really important to wait until you have a thorough understanding of how the application is working. Because that way you can act ask very specific questions that will help you in your preservation. So things we want to know. We're choices and timing and responsiveness. The cards shuffled randomly, but we wanted to know what the nature of random meant. Does that mean if it goes through one iteration, it can never do that one again? Or is it so random that yes, they can be duplicated. We wanted to know if the cards and the tones were mapped. And they were. And there was another slider that allowed someone to adjust the speed, and the delay. And so we were curious, who is expected to use that turns up curatorial was led to make that decision. And we also want to know, because iPhone six plus its lifespan is not very long. So what is it going to look like for the hardware? Is it okay, if we go to another device once these are no longer working. And so this artist interview was really, really, really interesting. And if you're interested, we have a bitly MC N, LACMA, and that's our transcribed interview with Mongo. And we were just so shocked that in two years, we had to do so much work to rebuild the app, and all of the programs and software that we needed. Were all already outdated. So we found the four things, this is really going to change our acquisition practices, because now we know that we need a budget for duplicate equipment and specialized software, you obviously can't do your preservation work on the original, because you might just ruin it. And also, you need to be rebuilding the work to understand if you have all the assets that you need. Do you have all the pictures? Do you have all the tones? Do you have the source code, and this is something that really needs to happen kind of regularly. So we're hoping that every year or two years, we're going to get it up and running, and then also save any kind of software that we needed in order to accomplish that. And the interview with the artist is also crucial. And ask your artists, the preference for tech support. And I know we have a lot of content creators. So it's good to think about. Do you leave your judgment up to the institution for preservation? Do you want tight control? Would you like to be the only one working on iterations of your artwork? Does the artwork have an expiration date, and you can put these in your purchase agreement as well. And so we have a suggested content package here, the source code Xcode project, along with all the assets, any code libraries referenced, capture the developer environment, that's the host computer OS X code, Swift, possibly Diskimage. video documentation of the app and action, consider a long term support plan for the institution acquiring the work and convey the important aspects of the work that will go beyond technological obsolescence. So these are really starting off points. They're very particular to your artwork to your content. But this is a good place to start thinking about that. And so we had originally thought that the updated operating system was a reason why it was not working anymore. But it turns out, you need an Apple developer license in order to sign the app to a phone. This is only good for one year. So after one year, every single edition of this artwork crashed at the same time. So now we needed to get our own Apple Developer License. But it's really important that you don't tie it to a personal to an employee's email. Because if they leave now, you're going to have to re tie it all. And again, so if you can have a facility email address that link these two, that's crucial as well. And we talked a lot about software, but the hardware is just as important. Is the work reliant on a particular brand and version of a phone or computer? Can this be upgraded 50 years from now? Does the institution have the ability to stockpile this equipment? Can the facade of the original hardware be used with new technology and internal batteries need to be addressed as well as a maintenance plan for checking this equipment? And our objects conservator is currently looking into strategies for stabilizing iPhone batteries. And we're also looking into capacitors. So I'll hand it back to Julie.
Unknown Speaker 19:53
And I think in the interest of time I'm I'm not going to spend too much time on the moment case study but the work by Martine Sims What you doing right now is unique from LACMA in that it makes use of an augmented reality component. And it was meant to accompany an exhibit of prints and video. And for 12 of the Select works, it would invoke what were described as events. So different images and GIFs, that would pop up on the screen relative to the work that you're standing in front of. And so some of the additional issues that they've been working through at at MoMA is related to the AR kit, which is in Wicca tude. And then noticing some of the platform discrepancies given that it's an app that users install on their own phone. And so noticing some of the differences between certain platforms and versions of iPhones even that it won't always invoke all of the events, it will do so in different speeds. And so that's largely why Flamini has been doing a lot more research around the video documentation piece of it, in order to tie those behaviors to specific environments. But then also, yeah, to note the difference between like an iPhone seven and iPhone 10, for example. But then also to use things like PhoneGap as a means of exploring, you know, what isn't is not portable, two successive versions of iPhones. So, you know, if you're curious about those aspects of preservation, I would suggest reaching out to Flaminia Tasker more about that.
Unknown Speaker 21:28
So a lot of this rehabilitation work that we did took place a year ago. And so then since then, we kind of wanted to crack open the vaults even just a year later, and see how much of it had held true. And it turns out a lot more issues have come up. And so this has brought up a questions of just like, what kind of collaboration is possible with Apple? Or how much are they thinking about these issues. And I think it will come as no surprise that either they're not telling us that they are or they're very aware that it's part of their business plan, or they just straight up are not considering it. And despite my best efforts through our business account at LACMA, and the Genius Bar, and no one seems aware of an institutional archives who might be working on this even just archiving their own history, it's not really clear if that's really a thing there. And what they've usually told me to do is oh, well, you know, do a mobile sync backup, and you can backup from, you can restore from that if you're having issues with your phone. But then I found that with recent versions of the iPhone, they've discontinued the ability to restore from backup, you can in effect restore from it, but it forces you to upgrade to the current iOS. So you know, that will obviously be a big problem if the current iOS is not compatible with the app that you're working with. And so that led me to look further into this thing called jailbreaking, which some of you might be familiar with. And I should mention, you know, jailbreaking is technically not illegal, but depending on its application, and its viability in court for Apple to be successful, it could be litigated against you, and particularly in the case of high valued artworks, there certainly is no capacity for them to seek some recompense in court if they wanted to. And so we're still just kind of investigating this. But it has been found that you know, since this is open source software that kind of the pirate web is putting out there. It's not always stable. iOS and OS 11, was reported to have a lot of issues with bugs. And so unfortunately, the version that of IOs that we currently have the work on in 11, does not have a very stable jailbreak. Nor would we want to do that on the original phone. Until we've done some analysis, we were able to do it on iPhone, the iPhone 12 and over iOS 12. And that jailbreak seemed to work really well. And what that's supposed to allow you to do is sideload apps onto an iPhone without the developer account. But the issue then came, you know, following downloading different software that changes or alters the file system and then puts the actual jailbreak tool on there that were we then had a lot of issues in migrating the work forward. So while it was on Swift tune, we brought up to Swift three with basically no observable differences or losses in behavior between swift three and swift four, which is required for the current iOS version, they've discontinued swift three. There are now a lot of issues with the compatibility of the original source code with the current version of Swift and they've even used the word obsoleted to describe some of the behaviors that don't port over. So, you know, now we're faced with a few options. None of them are easy and particularly given the fact that, you know, this is one work, but we may be acquiring more and more iPhone based works. And we don't really have control over what's acquired this, at this point, this is just a collaboration with Christian Marclay. It's not discussed for acquisition, but you never know what happens.
Unknown Speaker 25:17
But we've had, we've come to a few conclusions. And you know, as we have learned from software based preservation, you do often have to pursue simultaneous activities at once. And so some of the choices were presented with now. So you know, freezing the original Avaya environment. As we said, with the original dev environment, the original Xcode was swift dependencies, and all of the documented dependencies, all encapsulated into one thing would hopefully allow us to be able to sign that app to a device within the year that it's slated for exhibition. So it's really just going to be alive during that period, we could have the option of jailbreaking the device, or hopefully, if we're able to with the iOS 11, we could sign the app to the device and then put it in storage and ideally years down the line when we want to exhibit it, we can pull it out and it will function without the need for the developer account. We could continue to migrate the work forward into oblivion. But we obviously don't want to do that. There have been some new options that have come up like, we've definitely considered the thought of rebuilding the work with an understanding of how the code actually functions. And knowing that all that is necessary is the dimensions of the screen. And to some extent, something that looks like a phone, it doesn't conceptually need to be running as an app on an iPhone. So we could port it over. But as conservators we want to do, we want to take the course that is the least invasive, that's possible. However, with developments, like just this afternoon, or late morning, I heard a presentation that was talking about progressive web applications as a move towards sustainability in this kind of work. And I could certainly see that being one avenue for exploring with this work. But we would need to be in close collaboration with the artist and guiding that migration to that kind of platform as that's not how it was originally designed. So anyways, it's to say we're, we're still very much exploring, and we're curious to hear other thoughts and ideas and how this could work. But hopefully, we've imparted some of the issues and risks and maybe a call to arms to either discuss this with content creators or any ideas of how to call apple to the carpet to start thinking about this, though, of course, in their business model, they're benefiting from this, so they probably don't see an issue there. But at any rate, the research is ongoing as it probably always will be. But we want to acknowledge our partners at MoMA who assisted in this research and then as well as at LACMA. So thank you very much.