The Importance of 3D Standards Development for the Museum Sector
Looking towards the future of 3D for heritage with Scott Geffert, General Manager Imaging at The Metropolitan Museum of Art
I am Scott Geffert, the General Manager of Imaging at The Metropolitan Museum of Art. In my role, I oversee a department dedicated to photographing the museum's extensive collection since 1906. In recent years, the museum has expanded its efforts to include 3D imaging, a journey we embarked on around 2016. A good portion of our work in 3D supports conservation and technical documentation and is not yet public-facing, our team has digitzed hundreds of objects from scarabs and cylinder seals, to massive Bijs Poles and more recently the entire Temple of Dendur.
Our team is actively involved in developing imaging standards for both 2D and 3D formats and works closely with standards and user communities to bridge the tools and techniques used in 2D and 3D imaging.
You can learn more about our work via recent technical papers and exhibition content:
3D Imaging
Taking a Technical Leap: 3D Imaging of New Guinea Bisj Poles
From the Inside Out: Practical Application of 3D Imaging Techniques in Art Conservation
2D Imaging
At The Met Museum's Imaging department, our goal is to elevate 3D images to the same high standards of color accuracy, fidelity, and lighting quality that our dedicated team achieves through our traditional 2D photography. We are leveraging our extensive experience in developing 2D imaging standards, such as ISO 19264, to help shape emerging standards in the 3D imaging field, including initiatives like OpenUSD. While some may view OpenUSD merely as a file format, it is actually an industry-wide movement aimed at addressing the challenges of interoperability within 3D imaging.
OpenUSD, formerly known as Universal Scene Description (USD), is a 3D file format and open-source framework that allows users to create, simulate, and collaborate on 3D projects. This video provides an overview: What is USD? The basics of Pixar’s 3D file format in 15 minutes (Version 1)
For 3D imaging to be effective in the cultural heritage sector, it is crucial to address related core technical challenges that extend beyond file formats such as color accuracy (through technologies like OpenColorIO and NanoColor) and materials representation (using standards like MaterialX and OpenPBR). In many ways, these technical quality hurdles coupled with incompatible proprietary delivery mechanisms are the primary obstacles for open interchange of 3D content. The most interesting aspect of OpenUSD is that it was created out of necessity and over years of refinement to solve the problems of interchange and collaboration for content creators. When it was released as open source it gained wide adoption and is on a fast track towards formal standardisation. The cultural heritage community is well-positioned to play a neutral, pivotal role in developing these industry standards because of our expertise in establishing “ground truth” data sets. At the Met, one of our contributions to 3D standards development is our reference asset initiative. Leveraging established objective scene-referred 2D photographic examples, combined with colorimetric measurements and 3D representation data sets will help serve as a quality benchmark for ongoing studies. Supporting the development and standardization of core technologies will enhance interoperability and provide greater access to digital content.
While there has been progress on standardization with an emphasis on web delivery, the current focus is on standards for high-fidelity content creation, archiving and interoperability.
Looking ahead, our department aims to unify the tools, techniques, and communities of 2D and 3D imaging through the development of shared standards. Beyond the immediate benefits of enhanced quality and access, we believe it is vital to provide the next generation of creative professionals with opportunities to advance technology in meaningful ways. We are excited about the potential for 3D cultural heritage imaging to become as productive and accessible as traditional 2D photography, paving the way for innovative approaches in preserving and sharing our cultural heritage.
SIGGRAPH 2024
SIGGRAPH (an acronym for Special Interest Group on Computer Graphics and Interactive Techniques) is a global non-profit organization and annual conference that focuses on computer graphics. This year I attended to gauge progress.
The most important takeaway from Siggraph over the past two years has been the focus on Universal Scene Description. While there were major announcements and packed sessions last year, one could characterise the vibe in 2023 as curiosity. People heard about OpenUSD, but thought it was some new-fangled version of glTF. The general sentiment was “oh no, yet another file format”. My team listened in, but post-conference struggled to connect OpenUSD with our daily work. If a couple tools in the pipeline do not offer support, we are forced to stick with legacy formats and inherent inconveniences. What a difference a year makes! This year every single OpenUSD session overflowed to the halls and speaker after speaker shared stories of practical progress and wonderful stories of industry collaboration. It was crystal clear that users and software developers completely understand that OpenUSD is critical for 3D to break out of its traditional vertical market silos to finally become broadly accessible-not just for web delivery, but for high-fidelity “Source of Truth” content creation and interchange critical for the cultural heritage community.
One of the most interesting sessions at Siggraph 2024 was the Metaverse Standards Forum meeting: The State of 3D Asset Interoperability using USD and glTF. After introductions by Neil Trevett, President of the Khronos Group and the Metaverse Standards Forum, and Guy Martin Founding Board Member & Steering Committee Vice Chair at AOUSD, speakers from Autodesk, Adobe, Nvidia and Cesium spoke about the future of interoperability. In the Q&A there was a question about USDZ and concerns that it was a secret closed format. It was publicly clarified that USDZ is in fact part of the AOUSD standardisation roadmap. It was also noted that Apple’s implementation of USDZ simply taps into existing OpenUSD capabilities much like the work of other companies that embraced the technology early on.
You might wonder what changed between last year and this year? We can talk about standards until the cows come home, but in the end it's all about adoption and critical mass. Since Siggraph 2023 announcements, Adobe started to roll out deep support for OpenUSD across its product offerings. Adobe also open sourced a series of file format conversion tools to move between legacy formats and OpenUSD. They also announced a prerelease program, for a 3D viewer that leverages state of the art technology in a lightweight cross-platform program similar to a PDF viewer. Thomas recently posted a survey about what 3D viewer people used. I expect that once this application is broadly available, it will be far easier to share 3D content with predictable quality-another step towards open, seamless 3D delivery. Just within our institution a universal 3D viewer on every desktop would be a game changer.
In the past year Blender added extended native support which made OpenUSD accessible to a wider community. Each iteration of Blender has added more OpenUSD support and frankly improved support and performance for people using Apple Silicon hardware. Sadly there are a few tools that lack OpenUSD support, and this is still a problem for cultural heritage users. Epic Games has been active in USD (for Unreal Engine) and is an AOUSD member, but has not included USD support in Reality Capture (yet). Geomagic Wrap is another program that we use that does not yet support OpenUSD. While we can certainly adopt newer tools that support AOUSD, it will take time to bring them online across our team. We hope that by the next Siggraph companies sitting on the side-lines will take advantage of the Adobe Open Source file format plug-ins to give us the ability to develop a modern OpenUSD-based 3D pipeline. Companies need to understand that users need open standards and interoperability and support for OpenUSD is a competitive advantage.
A common thread across all conversations during Siggraph is that standards and open source development requires time and dedication. Open standards and open source solutions are not at odds with commercial software, both are necessary to drive technology forward. The cultural heritage imaging community plays a key role in shaping the dialog around objectively verifiable color accuracy, spatial resolution and interoperability and even lighting techniques. Our decades of work to define “source of truth” standards for 2D photography of collection objects provides a perfect reference environment for 3D standards development. Considering that almost every 3D digitization device and even the headsets we use rely upon core 2D photographic principles, its critical to be active in shaping the conversation as formerly separate imaging techniques and user communities converge and mature. Building upon our participation in the ASWF USD working group, we are planning to join the AOUSD and look forward to hearing from users around the world to join the conversation. Aside from improving the state of the art for 2D and 3D technology through standards development, our work today will ideally help pave the way for the next generation of imaging professionals.
Looking Ahead
Cultural heritage imaging does not exist in isolation. The challenges we face are often the same as those tackled by other fields—and, in many cases, others have already found solutions. No vertical 3D user community has all of the answers to the complex challenges related to archiving and interchange of high fidelity 3D content. In my conversations with leaders in the computer graphics industry, it is evident that there is a strong shared commitment to advancing the technology; the same dedication that drives our efforts in cultural heritage. When different creative communities come together, remarkable things begin to happen.
One of the most inspiring experiences has been witnessing like-minded experts collaborate on the ASWF OpenUSD forum. A standout moment last year was Thomas’s presentation on cultural heritage and IIIF, which served as an outstanding example of bringing diverse communities together. I look forward to a follow-up presentation from the ASWF to the IIIF community because the goals of the efforts are inherently well aligned. Events like Siggraph offer valuable opportunities to meet face-to-face with leaders in the computer graphics community, and we are eager to continue bringing cultural heritage imaging users together around emerging standards to help shape the future of the field.
 | A guest post by
|