Draft, 11 October 2016
This page should be in a useful state, but still needs work before it's finished.
I’m sure you’ll be excited to know that I’ve been continuing to work on my ASIO redactions. ;-) In fact I’ve now created a site where you can explore the contents of ASIO files through their redactions. I like to think it’s sort of turned things upside down – instead of hiding things, the redactions now provide a point of discovery. Anyway you can play with it here (still a work-in-progress).
I also conducted an experiment to see what happens when I put all 230,000 redactions on top of each other in a single image. The answer is you create a portal to another universe…
This week we’re exploring multiple realities – augmented reality, virtual reality, and even the amazing world of 3D! But I have to admit I’m a bit uncertain. With the other tools and technologies we’ve examined, I feel like I’ve had a pretty good understanding of their value and use. But while I think there are some exciting possibilities ahead for 3D imaging, I’m not sure about AR and VR in the cultural heritage setting. Perhaps it’s because I feel like we’ve been there before. I remember playing around with VRML in the ’90s; when Second Life was going to be the next big thing for museums; and when everybody was going round sticking QR codes on things. Are these technologies different? Will they change the way we experience cultural heritage?
I’d like to know what you think. Instead of trying to tie these things up in a neat bundle for you today, I thought I’d just offer examples and raise questions. As you explore each of the examples I’d like you to think about what they might contribute to the way we experience and understand cultural heritage. Are they gimmicks or groundbreaking? The future or mere fad?
As I said, I think 3D imaging is becoming an established part of the digitisation toolkit. Basic photogrammetry – extracting spatial data from photographs – has moved from research projects to consumer products. Indeed, if you have a smartphone you can try it for yourself. Download 123Catch and have a play. Unfortunately the documentation of the website seems to refer to an older version of the software, but it’s very easy to use.
Create a free account on the 123D site. This will enable you to save and share your models.
Set up the object you want to image in an evenly lit space that you can easily move around (you need to be able to photograph it from all angles).
Open the app and log in to your account.
Click on the ‘+’ to start a new project.
Move around the object taking photos from different angles.
Also take some from above the object.
Once you have about 20 to 30 photos you’re ready for the next step. Click on the blue tick.
You can then review your photos, deleting any that didn’t work.
Once you’re happy click on submit.
Your photos are then uploaded to the 123d server where they’re magically stitched together. It can take quite a while, so be patient!
Once your model is complete you can view it under your account on the 123D site.
The result will probably be a bit messy, with various bits of the background included. If you want to clean it up you can use tools like MeshMixer (available for free from the 123D site) to select and delete the bits you don’t want.
Of course there are much sophisticated and accurate tools available for 3D imaging. Using technologies such as LIDAR it’s possible to create 3D models of complete heritage sites.
Cultural institutions are starting to make selected collection items available as 3D models:
Not only can you view the models online, you can download them and feed them into a 3D printer to make your own physical copies. This seems to me to have important educational uses, particularly in remote areas – what do you think? Does the ability to handle an item (even a 3D printed plastic version) add to our understanding of it?
This is not really 3D and not augmented reality, but I thought it was worth mentioning here. As you may know, Google’s StreetView technology has moved inside some institutions, allowing people to take virtual tours. Here’s the National Museum of Australia.
The NMA also provides laser-guided virtual tours courtesy of two mobile telepresence robits.
I suppose one theme across all of the technologies we’re looking at today is the attempt to create new understandings or experiences of space that blend the physical and the digital. But how new is this? How are technologies such as augmented reality different to old-fashined audio guides? Is access a key point – the fact that these experiences can be delivered remotely?
As I noted above, 3D scanning technologies can be used to help build up models of complete sites. These can then be populated with reconstructed buildings, and even animated people.
The Visualising Angkor project is a particularly impressive example, where researchers have not only tried to reconstruct a site, they’ve tried to represent to rhythms of everyday life – the sun rises and sets, people and animals move about their lives. All of this is based on detailed archaelogical evidence. Unfortunately the computing power required to run the simulation means it can’t be accessed online, but you can find some videos on the Google Cultural Institute page.
How about a tour of ancient Rome? The Rome Reborn project has created a detailed 3D model of Rome in 320 CE.
Distance in time is not the only reason to build 3D worlds. Sites can be inaccessible for a variety of reasons, and 3D models can provide a way of documenting and experiencing locations that might be at risk.
Researchers in Western Australia have created a 3D model of the wreck of the HMS Sydney. As you can see from this ABC report it offers a powerful emotional experience.
You’ve probably also heard of attempts to use 3D modelling to document and possibly even reconstruct parts of Palmyra destroyed by ISIS. Recently a replica of the Palmyra Arch of Triumph, created using 3D modelling techniques, was unveiled in London.
It’s also possible to create models of places that never actually existed. Researchers on the Digital Panopticon project have created a 3D model of Jeremy Bentham’s ideal prison, designed in the 18th century, but never built. This them to think about how the space might have worked, and what the experiences of prisoners might have been within such a space.
But not everyone thinks these types of projects are a good idea. Critics of the reconstructed Palmyra arch, for example, have pointed to the danger of creating a Disneyfied version of the past. Might such reconstructions obliterate memory rather than preserve it? How do we experience and document the processes of destruction and decay alongside such freshly-minted versions of the past? Are these reconstructions all a bit too clean?
Nowadays if you want to explain augmented reality to someone it’s easy – you just say it’s like Pokemon Go. AR is a blend of physical and digital, with new experiences, information and activities layered on top of our physical environment. There’s some definitions and a good dash of added hype in this article about ‘Augmented Museums’. Clearly some people can see the business opportunities.
Location-enabled applications that present historical photographs based on your location have been around for a number of years. A couple of years ago the Museum of London created an iPhone app called StreetMuseum that blended past and present.
Earlier this year the Art Gallery of NSW presented an immersive 3D experience that allowed visitors to explore intricate cave murals from the UNESCO World Heritage listed Mogao Grottoes in China. This work has been presented in a variety of forms around the world. It’s not just a 3D reconstruction, because various elements of the paintings come to life. Here’s a video showing it on display in HongKong.
Augmented reality apps can be used within a museum context to provide additional layers of information and interactity, as in this example.
If you view this video on YouTube you’ll see that the caption proclaims:
Visiting a museum is no longer a one-sided, passive experience, as we’ve developed an interactive exhibition tablet app using the newest innovative mobile technologies…
Does giving a visitor an iPad app make the experience less ‘passive’? Isn’t their experience still carefully curated? Clearly there’s value in giving visitors access to more information, but is this a good way to do it?
Virtual reality, on the other hand, immerses you in a totally manufactured digital environment. And you have to wear funny-looking goggles. All the big players are working on VR technologies – most notably FaceBook and Microsoft. (Let’s not mention Google Glasses).
Microsoft is building up to the release of its HoloLens and there’s much hype about the ‘Minority Report’ style of interactivity it will enable. It’s interesting, however, that Microsoft has used the possibilities for cultural heritage as part of its pitch to potential developers:
One of the problems with VR is of ourse that you need special hardware for it to work. Videos don’t capture the experience. If you’d like to explore some of the possibilities and you’ve got a smartphone handy you can order Google Cardboard. It’s a simple VR viewer that uses your existing phone – for $30 or so you can start experimenting.
The Google Cardboard app provides some fun examples, including museum exhibits. You can also search the App Store or Google Play for ‘Google Cardboard’ to find other VR apps. Beware – the experience can be a bit unsettling!
You might we use this sort of technology? One of the nice features of SketchFab is that its 3D viewer is compatible with VR devices, including Google Cardboard. If you created a model with 123D Catch, you can easily upload to SketchFab and view using Google CardBoard. (Yes you’ll need to create a free SketchFab account.)
Log in to the 123D site.
Hover over ‘ME’ in the top menu and select ‘Models’.
You should see the model you created with 123D catch. Click on it.
Click on the Edit/Download button and select ‘Download 3D models’.
In the list of downloads select the ‘Mesh package’ – you can uncheck the other options.
Once the file has downloaded, login to your SketchFab account.
Click on the Upload button and then select the file you just downloaded.
Click Continue and wait while the file is processed.
Once it’s finished, click Continue again and wait some more.
Eventually your model will open in the 3D viewer. Click on Edit 3D settings to play around with how it looks.
To view in VR, just hover over the image and then click on the Google Cardboard icon. It will give you a link to open on your phone.
Note that the free SketchFab acount will only let you upload models that are less than 50mb.
If you didn’t create a model with 123D catch you can still play around in SketchFab. Try downloading a 3D model from some of the collections mentioned above. This jug from the Powerhouse and this trilobite from the Museum of Victoria are both under SketchFab’s free 50mb limit. Once you’ve downloaded the files, just upload them to SketchFab as described above.
For further experimentation download Google’s Street View app. You can use it to create 360 degree ‘globe’ images of any location that you can then explore in VR using Google Cardboard.
This is all a lot of fun. But how might you use this technology in a cultural heritage setting? Will we really want to spend our time immersed in a virtual museum? What sort of experiences do we want to create?
Perhaps the really exciting thing is not what cultural institutions will do with this technology, but what ordinary people will do with cultural collections using these sorts of tools.
There is a big overlap between VR/AR and the world of gaming. Indeed, the game creation platform Unity is often used as the platform for building and exploring 3D worlds.
If you’re interested in the connection between cultural heritage and gaming, keep an eye on the Play the Past blog. Their most recent post is about creating interactive histories using Twine (which I’ve mentioned previously).
One final video describes a project undertaken by the Auckland War Memorial Museum. Working with a local school they ‘recreated’ the Gallipoli campaign using Minecraft.
Would you be interested in such a project?