This post demonstrates several tools to optimise a 3D asset for publishing online, AR, & VR. It doesn’t go into huge detail for each tool, but hopefully it gives you a starting point for your own investigations!

I think that a lot of folks in the heritage sector are publishing interactive 3D data online that is a little too heavy for ‘regular people’ to enjoy. There, I said it 😹.

Just to be clear I am referring to interactive 3D data: for the record I think it is great to offer academics and creatives open access to high resolution data sets. For most people in the world, however, interacting with an optimised 3D file will be a much better experience: faster to download, better performance on all devices.

I’ve said as much in many keynotes and presentations over the years, but I still note the tension between professionals wanting to demonstrate their work at the highest resolution, but also wanting to reach wider audiences.

An added bonus of optimising your 3D data for the web is that it is then often suitable for AR, VR, and game engine applications. In this way you’re facilitating more use cases for the fruits of your digitisation work. Nice.

So how much should you optimise a given 3D asset?

My answer is always the same: as much as you can while still meeting the needs of your chosen audience. I always think of 3D as ‘just another kind of image’ and I don’t think anyone regularly expects an end user to load (for comparison) a TIFF image that weighs 100s of megabytes in their web browser: we convert images to nice, tiny, web friendly JPGs. Let’s do the same for our 3D! 💪

Time. To. Optimise.

Scroll on and you will find short tutorials showing you how to optimise the same 3D asset using 5 different tools. 👇

Some notes before we begin:

• We’ll going from a 3D model of 1,000,000 face geometry and an 8192x8192 PNG colour texture map (link) to a model of 50,000 faces + 4096x4096 colour texture map (links to output 3D files from each tool can be found in each article subsection below).

• For the sake of simplicity, I’m not worried about generating normal maps for now (🙀 scandalous, I know).

• The example 3D model (single mesh, single texture) used in these demos is pretty straight forward, things can get more complicated with some of the workflows depending on file formats, animated meshes, multiple textures, etc. etc.

• Of course you can output an optimised 3D model directly from e.g. your photogrammetry software, but these tutorials are for those occasions when a 3D model already exists and you don’t have access to the source project data.

• As a baseline, here is a link to a version of the 3D model optimised to 50,000 faces + 4k PNG texture using the original RealityCapture photogrammetry project.

OK, now on with the demonstrations!

Meshlab

Quick Guide:

1. Import.

2. Apply filter.

3. Export.

4. Resize texture map (I used https://onlinepngtools.com/resize-png, renamed it and replaced it in the source directory.)

5. In my experience, the image texture became disassociated from the .obj/.mtl and I had to re-link in my final 3D viewer.

6. Done!

💾 Link to output 3D file on Sketchfab.

Summary

Price: FREE!
Get Meshlab.
Time it took: 5 minutes.
Difficulty: 1/5
User control: 2/5
Output file size: 6mb

Blender

Quick Guide:

1. Import mesh, scale to approximate dimensions in metres.

2. Duplicated mesh.

3. Decimated Mesh > 50,000 faces (ration of 0.05).

4. Removed Material, created new Material.

5. Create new image 4096x4096, assign to material diffuse slot.

6. Add to shader editor.

7. Select high res mesh, shift-select low res mesh, select texture to bake to.

8. Bake using Cycles render:

   • Bake type diffuse.

   • Influence contributions: colour only

   • Enable Selected to active

     • Extrusion (how far outside of the low res mesh the bake will look for data) 0.02m

9. Connect baked texture in shader editor to check result.

10. Export mesh & texture map.

11. Done!

💾 Link to the output 3D model on Sketchfab.

Summary

Price: FREE!
Get Blender.
Time it took: 15 minutes.
Time it took to figure out: Hours 😅
Difficulty: 4/5
User control: 5/5
Output file size: 11MB

RapidPipeline

Quick Guide:

1. Sign up, log in.

2. Upload asset.

3. Choose / edit a 3D processing preset.

4. Hit ‘Run 3D Processor’

5. Done!

💾 Link to optimised file hosted on rapidpipeline.com.

Summary

Price: Starts at $15 /month billed annually, FREE trial available.
Get RapidPipeline
Time it took: 10 minutes.
Difficulty: 1/5
User control: 5/5
Output file size: 7MB

Needle Cloud (Beta)

Quick Guide

1. Sign up, log in.

2. Upload 3D asset.

3. Done!

Needle Cloud automatically produces a number of 3D outputs derived from your input file:

• A converted GLB.

• An optimised GLB.

• 2x progressive loading GLBs (multiple levels of detail for meshes and textures)

💾 Link to 3D file hosted on Needle Cloud

⚠️ I may be doing something wrong, but from my experience, optimised output files were not compatible with Sketchfab, Windows 3D Viewer, or Blender due to their using glTF extensions unsupported by those tools.

Summary

Price: FREE for evaluation purposes, artistic expression or non-commercial use. Commercial plans start at €17/month billed annually.
Get Needle Cloud.
Time it took: 10 minutes.
Difficulty: 1/5
User control: 1/5
Output file size: 4.5 - 11MB

Polycam

Quick Guide

1. Sign up, log in.

2. Hit ‘+ Create’ button, choose ‘Upload 3D Asset’

3. Select your files, upload.

4. Open uploaded asset, select ‘Remesh’.

5. Adjust remeshing parameters, hit ‘Remesh’.

6. Done!

💾 Link to optimised 3D file hosted on sketchfab.com.

Summary

Price: FREE plan available. Pro plan starts at $24.99/month billed annually.
Get Polycam.
Time it took: 15 minutes.
Difficulty: 1/5
User control: 3/5
Output filesize: 41MB

Conclusion

Happily, there are lots of ways to achieve 3D model simplification and several of them completely free. Which tools did I miss which you use regularly for this kind of task?

You may remember that I paid a visit to a local historical site and made some 3D captures of the in-progress reconstruction of an Iron Age roundhouse. Well I’ve been back to the site and during better weather (and with a bit more planning) I captured a more complete scan of the exterior. Then I had some fun…

Subscribe now

A whole new scan.

After my last capture, I realised that capturing the complete roundhouse from the ground would be tricky without the ability to elevate my camera. As I don’t own a drone I had to resort to something a little less high tech: a camera on a pole.

I own this microphone boom pole from a previous life as a sound recordist, and as my Canon G7X camera isn’t too heavy (302 grams) I was able to hoist the device 5 metres into the air fairly easily. Even so, things were still a little bit wobbly.

I could trigger the shutter from my smartphone using the Canon Camera Connect app over a direct wifi connection, which also gives you a preview of the viewfinder. It’s all pretty basic, but it does the job!

All in all I captured 587 images in just under one hour of shooting, then processed the set in Metashape (I had issues aligning using RealityCapture), finally making some basic cropping edits in Blender.

Bonus: a quick splat

The iOS 3D scanning app Scaniverse, recently added support for capturing gaussian splats. The resulting 3D scene is a little noisy with lots of ‘floaters’ from the sky but still pretty neat IMO, for under 2 minutes of capture time.

Time for some fun.

While capturing the roundhouse and processing the 3D model is rewarding in itself, the possibilities to use and re-use the data are even more exciting (for a nerd like me).

Reaching online audiences with Sketchfab

You can explore and download the output mesh model as well as a point cloud on Sketchfab.

If you can think of a better place to publish 3D data online, I am ready to hear it :) Of course Sketchfab is no replacement for a proper data repository (…and the core product has not been updated in some time …and the impending transformation into Fab is a bit of an unknown for the cultural heritage community) but, honestly, the more I explore self hosting of 3D, the more impressed I am by the product that the Sketchfab team has built.

Bonus: taking the roundhouse to the beach.

The roundhouse is located inland facing north, away from the sea ...so using Sketchfab’s AR view, I took it to the beach to enjoy the sunset view 🌊😌🌅.

Exploring visualisation with Twinmotion

Twinmotion is a piece of visualization and rendering software based on Unreal Engine - it has all the power of that game engine’s rendering capabilities, but with a much more non-game-developer-friendly user interface and user experience.

Primarily intended for use in the architectural industry, Twinmotion has an impressive array of tools for importing 3D models, adding landscapes, scenery, vegetation, setting lighting, weather conditions, and more.

Once you get the hang of using the app it is very easy to create some great looking images and video renders: I created the scenes shown in the video above in about an hour or two.

After exporting my rendered videos, I added some sound suitable recordings from my personal Freesound library for a little more ambiance. While none of what I created is backed by academic research, I hope I have demonstrated the possibilities of adding some life and context to a 3D scan.

Geolocation with Cesium

Cesium is a platform for geospatial 3D, one of the biggest players in fact and (as far as I can see) the most easy to use for creating online content. While there is a whole lot more to the platform than I will write here, the main affordance of Cesium for me is to plonk a 3D model somewhere on a 3D globe, and share a link.

I used Cesium Stories to create the demo here. This free service lets you import 3D data (which Cesium then automatically optimises, compresses, and tiles), place it precisely on a 3D globe, and create a step by step realtime 3D tour for your end user.

You can try the simple experience shown in the video yourself here.

Interactive AR with Adobe Aero

Aero is part of the Adobe suite of 3D tools, designed for authoring interactive augmented reality content. I use italics there because this differs quite a bit from the simple Sketchfab AR demo shown earlier in this article.

I’ve only just started using Aero, but my impression is that it is quite a capable app with a fairly intuitive user interface and ‘behaviours’ node system for no-code programming of interactivity.

In my first few minutes using the app I was able to load two versions of my roundhouse scans (one before thatching, one after) and program a tap interaction that would fade between the two scans. Duplicating the project meant I could publish it both anchored to a generic horizontal surface, as well as anchored to an image (I used this image from Wikimedia Commons for my test), which worked pretty well on a secondary phone screen.

When loading the the scenes on my Android phone I was prompted to use Adobe Aero via ‘app on demand’, which meant I did not need to install anything to view it (on my iOS device I already have the Aero mobile app installed). Pretty neat!

Manipulating in VR with Spatial.io

Spatial.io bills itself as “Spatial is a 3D gaming hub where creators craft and share immersive social experiences for web, mobile, and VR”. I knew I wanted to try loading my roundhouse scans and manipulating them (picking them up, scaling them, moving around them, etc.) in VR on my Quest 3, but I have yet to find a simple (and free) “3D model viewer” app.

I remembered Spatial from my days at Sketchfab (a 3D model import integration exists), and figured I’d give it a go. As well as creating and publishing your own spaces and content, you can explore other user generated content via a search/discovery system in app. It’s all a bit like Mozilla Hubs (R.I.P), but a bit more game-ified.

I was able to search for my roundhouse model on Sketchfab, import it (with attribution), ‘hold’, scale, and move the model using hand tracking and view it from different angles by moving my head about the place.

There’s lots to like about the tech going on here (for the most part it just works™), but I also received an alarming warning when I tried to connect my Google Drive: “This app is blocked. This app tried to access sensitive info in your Google Account. To keep your account safe, Google blocked this access.” So… I’m not sure if I can confidently recommend this app at the moment.

What’s next?

These are just a few ways that it’s possible to use and re-use a piece of 3D data within the context of education and engagement and hope this post has given you some ideas for your own work.

I really enjoy exploring new platforms and services to share with you so do let me know what you think!

One of the people I met was Dr Hayden Scott-Pratt who, among other things, is leading a project to reconstruct an Iron Age roundhouse at Hengistbury Head in Dorset. The project is called Living in the Round.

Hayden is exploring the use cases for 3D digitisation in relation to archaeological work and, as the site of the site of the Living in the Round project is not too far from me, I agreed to pop down for a chat about the topic.

As it turned out, Hayden and a band of volunteers were quite busy doing the actual reconstruction work on the roundhouse when I arrived, so there wasn't much time to talk. The work they were doing was time sensitive as the thatchers were arriving the following week and the roof frame was not yet finished. I didn’t want to hold things up so I agreed to meet Hayden to talk about 3D another time.

As I was there anyway, I thought it might be fun to capture some 3D data—aside from the roundhouse, there are lots of other interesting things at the site one could capture. If I could, I thought I could also try to digitise the thatch-less roundhouse as it was, as a quick record of the building process.

Here are some of the constraints I was working with:

• everything was completely unplanned

• I only had my smartphone with me (an iPhone 15)

• it was raining the whole time

• I had a a limited amount of time (~30 mins to capture the roundhouse)

• I could only capture images from ground level

As you can imagine I was not hoping to capture research grade data, rather experimenting with simple and easy to use workflows to see what was possible.

While I was waiting…

I had to wait until the volunteers and Hayden took their lunch break before trying to capture the roundhouse so in the meantime I grabbed a couple LiDAR captures using Scaniverse, and a couple of videos that I would process with Luma AI.

Luma AI

Luma AI takes video as its input and converts the data into a NeRF, Gaussian Splat, and surface 3D model. The capture process is very easy: just shoot a video of your subject making sure that all the features you want to reconstruct are visible at some point in the sequence. You then upload your video to Luma AI’s servers where it will be processed into several formats which can be viewed on lumalabs.ai, embedded elsewhere, or downloaded.

I captured a brazier (56 second video), a bit of pottery (48s video), and the roundhouse itself (62s video):

You can view the gaussian splat scenes over on lumalabs.ai: brazier, pottery, roundhouse.

Scaniverse

Scaniverse is one of the earliest 3D scanning apps to appear for LiDAR enabled iPhones and while it does not provide especially high fidelity 3D outputs, it is great for making quick scans on the go. Compared to themore laborious “shoot-move-shoot…” process of capturing a photogrammetry image set, using Scaniverse feels very fluid and fun. The textured surface 3D model outputs can be uploaded directly to Sketchfab from the app, or published on Scaniverse’s own platform.

I grabbed two scans with this app: one of a pottery station and another of a flint knapping station. Each scene took approximately 3 minutes to capture:

You can explore these scenes in 3D on Sketchfab.

Capturing the roundhouse

Once the roundhouse itself was clear of volunteers, I sprang into action and started snapping photos of it with my iPhone. Although it had been raining a little the whole time I was on site, it was only at this point that a proper downpour began—typical! 🌧️

To start with, I began by circling crab-wise around the roundhouse, snapping photos while holding my phone at 3 different heights: arms in the air, chest height, squatting down. After circling once, I moved in a bit closer and took a second round. I then entered the roundhouse and captured some images facing into the centre of the structure.

It was by no means a smooth or easy process:

• it was pouring with rain as I took the photos. I could barely see what was on the phone screen and water droplets kept causing false touch events.

• I ran out of storage space on my phone before gathering what I would consider a complete image set for the capture.

• The roof beams as imaged from within the roundhouse are woefully underexposed due to the phone compensating for me pointing it at the sky.

• I didn’t capture every nook and cranny of the roundhouse.

I captured 1070 JPG images in approximately 30 minutes with the default iPhone camera on full auto before the volunteers returned to their work. You can get an idea of the camera positions from the screenshots below with each blue rectangle representing the capture location of an individual photo:

Reconstructing the Roundhouse in 3D

I knew my dataset was not ideal and the time available for me to work on this project is limited, so my plan was always to just throw my images at some photogrammetry software and see what happened. I didn’t pre-process my images in any way (adjusting exposure etc.) and I didn’t mask the subject from the background either (e.g. to ignore the sky).

Here’s how a few pieces of software performed:

Polycam

Polycam is a cloud based service used to produce surface 3D models, gaussian splats, 360 images, floorplans and more. I had high hopes that Polycam would be able to produce a decent gaussian splat, but alas the processing failed multiple times 🥲. Polycam was also unable to align the entire image set and only produced a partial 3D mesh, but it is quite cool to be able to capture a 360 background image for the scene all in the same app:

As I like the simplicity of using a cloud based service, I also thought I’d try a couple of other apps that I haven’t used before or used in a while 👇

Pixpro

I’d never really heard of Pixpro being used in the context of cultural heritage before, but I thought I’d give it a go as they offer a free 14 day trial ($600/yr thereafter). Their desktop app can be used to process data locally, but can also be used to upload image sets for cloud based processing which is what I did in this case.

Despite having never used the app before I found the experience pretty intuitive and the 3D outputs were not bad at all. Pixpro aligned pretty much all the images and there was a neat geo-located preview of my image set, drawn from the image EXIF data:

The mesh wasn’t too bad either:

You can explore the output mesh model in 3D on Sketchfab.

Autodesk Recap Photo

There was a time when I really enjoyed using Autodesk photogrammetry software, way back in the time of 123D Catch, Memento, and Remake. I figured I should have a peep and try the latest incarnation of their photo reconstruction tech which comes as part of Autodesk Recap, and grabbed a free one month trial (£390/year thereafter).

First off installing the app was a pain because to run Recap, you also need to install Autodesk’s Connect app, and sign up to Autodesk Drive as well which took some time. The joke was on me anyway, as even if you have a free trial of Recap, you need a paid subscription to Drive to upload your photos so I was unable to run my images through this software. Autodesk even has a help centre doc which essentially states “if you want a free trial, you need to buy a paid subscription“🤷

Oh, well—onwards to my tried and tested local processing options! 👇

RealityCapture

RealityCapture is a market leader in desktop photogrammetry and LiDAR data processing, and often one of my go to apps for processing an image set. I used RealityCapture to process my 1000+ image set of NHM London’s Hintze Hall, so thought it would be ideal in this case, too

The software was able to align several subgroups of images (e.g. groups of 50 or 500 images) into separate components but I couldn’t easily get it to produce a unified 3D model combining all the images. No doubt if I had the time to add in a good few control points manually (e.g. give the software some clues as to how to stitch the sub-components together) I could produce a more complete scene, but I just don’t have time at the moment.

Metashape

Metashape is another market leader in the desktop photogrammetry / LiDAR processing space, and managed to aligned all my images 🎉 and I was able to go on an produce a pointcloud and meshed model. The whole process took a good few hours on my laptop, and the output surface 3D model (as expected) shows quite a bit of noise especially around the apex of the roundhouse roof. All things considered though, a decent output.

In the past I have used Metashape to align images, then exported a set of undistorted images and camera positions as a bundle.out file and then processed these using RealityCapture which generally processes data faster, and sometimes produces better 3D. However, Agisoft removed the easy to use ‘Undistort Images’ button a while ago and while it is still possible do this with a workaround, I didn’t have the time to try it out this time

You can explore the output pointcloud and mesh model in 3D on Sketchfab.

Edit: Bonus Gaussian Splat

As I was putting together the latest edition of The Spatial Heritage Review I discovered—thanks to this post by Studio Duckbill—an app called Postshot by Jawset which is currently in beta testing. It’s an app for local NeRF training and gaussian splat processing.

After Polycam and Luma AI failing to process my 1000+ image set into a splat, I wanted to have one last go at it. Lo and behold, Postshot created a really nice splat after about 3 hours of processing:

I was able to export the splat as a .PLY file.

I’m still not sure how to publish a 3D splat online without hosting it oneself, but I did use Playcanvas’ Supersplat to edit the resulting splat, mostly to remove the sky and background, as well as orient the splat correctly. Supersplat also lets me convert the PLY file into a .SPLAT file as well as a compressed PLY:

In Conclusion

So, can you digitise an iron age roundhouse with your phone? Yes, I think so!

Did I do a good job of it? No, not really.

To improve my process and outputs if I tried this again, I think I would:

• visit on a dry day 😅

• spend some time planning my captures

• bring a telescopic camera pole to capture the roundhouse from above

• empty my camera storage, shoot images in RAW

• spend more time preparing my input image data before processing

• try and capture a more complete gaussian splat with luma ai

Update: Thatching

I popped back for the rest of my chat with Hayden and his colleague Mark yesterday. It was much sunnier which was great for the folks from the Ancient Technology Centre and volunteers who were beginning to thatch the roundhouse.

I grabbed another quick video (120 seconds long), and processed it with Postshot using 1000 out of the available 4000 frames. Here’s the result cleaned up a bit in and viewed in Supersplat:

If do you like this, then please consider subscribing to my free monthly newsletter The Spatial Heritage Review, a monthly round up of things happening at the intersection of cultural heritage + digital 3D:

Subscribe now

I recently made this 3D scene to promote the launch of The Spatial Heritage Review. While it is a fairly simply scene, it still required several steps and different digital tools to achieve. I figured it might be of interest to readers to learn how to do something similar, so please read on for a step by step guide to my process.

You’ll learn about:

• Searching for and downloading free 3D assets from Sketchfab.

• Automatically optimising 3D assets for augmented reality (AR) with Rapid Compact.

• Importing 3D assets into Blender, assembling a scene, and adding some 3D text.

• Uploading the final scene to Sketchfab and making it ready to publish online.

• Recording a video of the scene in AR on a smartphone.

Open access 3D models on Sketchfab

While I am biased from my time working at Sketchfab, I really don’t know of an easier to use 3D publication platform . Additionally, it is home to pretty much the most extensive community of cultural heritage 3D content you’ll find online.

Video summary above, more detailed description of steps below.

If you’re not familiar with it, Sketchfab is a community platform for publishing and sharing 3D models online. It’s a great place to publish your own 3D models, as well as find and develop a following for your creative 3D outputs.

Many creators on Sketchfab choose to make their 3D models freely downloadable so that other people may re-use and remix their work - so you can also think of Sketchfab as a vast library of free 3D assets. You can join Sketchfab for free.

Let’s take a look at how to browse and search Sketchfab for what we’re looking for.

Find High Quality Open Access 3D Models on Sketchfab

Start by navigating to sketchfab.com and login.

Now, there is a lot of 3D content available on Sketchfab, so you’ll do well to use the available features on the site to narrow down what you you see on screen.

• Start with the search bar. Even if you’re not inputting a search term, this is the best place to begin. For our use case, we’re looking for cultural heritage content so let’s dial in that model category. This will return a list of every model in that category, for all time, sorted by likes.

• We’re specifically looking for downloadable models, so we could check the ‘Downloadable’ box but this filter also includes paid-to-download content and we only want free 3D models, specifically ones that use the Creative Commons CC-BY or Free Standard licenses. Check those options from the ‘Licenses’ drop down.

• From here we can scroll and browse until we find something we like the look of or, if we want to focus our options further, we can add a search term in the bar at the top.

Download the format(s) you need.

Once you find a 3D model we like the look of, click on it to open the model page.

Before downloading we can explore the model by rotating/zooming in 3D and using Sketchfab’s Model Inspector to see if it suits our needs. If all looks good, let’s hit that download link!

Sketchfab offers several file download options:

• The original file format that the user uploaded

  • Also any additional files the user uploaded, e.g. a higher resolution.

• USDZ*

• glTF*

• GLB*

  • This format also includes sub options for different texturemap sizes.

*These formats are auto-converted from the original upload by sketchfab.com. If you’re not what these formats are, the Sketchfab Help Center has you covered.

For this use case, let’s download the GLB version with the highest resolution texture. This format bundles up the geometry, material, and texture information into a single file (unlike, for example an OBJ or glTF package) and I’ve found that this format—aside from being convenient—plays well with RapidCompact in the following steps of our process.

At this point I also recommend copying the handily available attribution text that Sketchfab supplies in the download popup. We’ll need that later.

Repeat the above for as many models that you want to assemble in your scene. Bare in mind, however, that the more individual 3D models to try to combine in a single scene, the more you’ll need to optimise each individual asset for final download and viewing efficiency.

Speaking of which…

Automatically optimizing the 3D assets with RapidCompact

Optimising 3D assets can be tedious and time consuming, especially when you are working with 3D data that you did not create yourself. I’ve found RapidCompact to be a quick and easy to use solution.

Video summary above, more detailed description of steps below.

Note: it was at this stage that I switched screen recording software and did not realise the lower part of the screen was cropped off 🤦

RapidCompact is a cloud based utility for optimizing 3D assets for different use cases, automatically reducing face counts, texture counts and sizes, and converting formats from pretty much any 3D surface file format to another. RapidCompact offers a pretty generous free tier.

Uploading the file.

Log in and navigate to the upload page. Drag the GLB file you downloaded from Sketchfab from your OS folder onto the upload zone and hit ‘Upload’.

Choosing an optimization preset.

Once the file has finished uploading, it will show in your model management page. Check the box at the end of the listing, then move your attention to the ‘Optimization Settings’ section to the right.

There are a plethora of optimisation presets available (or you can even create your own), but I won’t go into details here - for the purposes of this guide, select one of the ‘Lens Studio’ presets. If you are combining a larger number of assets, I suggest use the ‘Low’ setting, for smaller groupings of assets you could opt for ‘Medium’ or ‘High’

The answer to “What face count and texture map dimensions should I use?” depends a lot on your subject, input data, and use case. My general advice for live/interactive use cases is to simplify your 3D data a much as possible: so the file is as light as possible while still being recognisable as the original subject.

Downloading the optimised file.

The optimisation process shouldn’t take too long and it will depend on the complexity of your input data and the output 3D model will show up on your Optimised Models page.

It is possible to download the optimised model from the preview card, but I recommend that you open the 3D preview to check that you’re happy with the optimisation before saving locally.

If you’re happy, hit the download button and choose the GLB file; if you’re not happy, go back to a previous step in the process and try different optimisation options.

If you’re optimisation repeatedly fails or if the output looks terrible, there may be something funky about the input data. For example, I’ve noticed that trying to automatically optimise a 3D surface model derived from CT scan data—which often has lots of hidden internal geometry—can throw a spanner in the works.

Repeat the optimisation and download steps for each 3D model that you want to add to your AR scene, then you’re ready to start assembling the scene in Blender!

Assembling a 3D scene in Blender

There are many premium 3D editors out there, but I’ve found that Blender has everything I’ve ever needed for working with cultural heritage 3D content.

Video summary above, more detailed description of steps below.

Note: my screen recording software has cut off part of the screen at the bottom🤦

Blender is a free, open source, fully featured 3D editor - you can download the latest version here. In this section of the tutorial I will be assuming a little prior knowledge in the steps below, but not a huge amount.

If you’re a complete newb to Blender there are extensive official tutorials and I’ve always found it easy enough to find even the most niche community tutorial with a quick web search for “How do I do this particular thing in Blender?”

Importing and placing your 3D models

Importing

Open up a fresh, blank scene in Blender and head to File > Import > glTF 2.0 (.glb / .gltf), navigate to the saved files you downloaded from RapidCompact, and import each asset.

At this point, and if you grabbed a variety of 3D models from different Sketchfab users, you will likely notice that the models are all different scales and import to locations far from the scene origin. This is part of life.

Setting the model origin

One workflow I often employ is to center each 3D object to the center of my scene and then begin artistic arrangement from there. To reset a 3D model to the center of your scene:

1. Left click select your model.

2. Hit F3 to open search panel.

3. Search for ‘Set Origin’.

4. Select ‘Origin to 3D Cursor’.

5. Hit F3 to open search panel.

6. This time, search for and select ‘Geometry to Origin’.

With everything in the middle we can start laying out our AR scene. If you are having trouble selecting assets that are overlapping, remember you can always select them in the Outliner panel in the top right.

Translating, rotating, & scaling the models

Once you’re ready to begin placing your 3D assets in their final positions, use the Grab tool (shortcut: g-key), Rotate tool (r-key), and Scale tool (s-key) as needed. You can limit any of these operations to a particular axis by hitting the x, y, or z key after you hut the tool keyboard shortcut. Left click or hit Enter/Return to commit to your updates, or if everything gets out of control hit Esc to cancel the operation or Ctrl-z to Undo (Ctrl-Shift-z to Redo).

Sometimes I find it helpful to switch to an orthographic scene view e.g. Front (Numpad-1), Right (Numpad-3), and Top (Numpad-7) before performing one of the above operations to aid exact placement

Note: the lowest part of any model will set the ‘ground’ for your scene when viewed in AR so make sure that everything aligns or that you are happy for some objects to be floating while some lie on the ground.

Adding a 3D text object in Blender and Exporting the Scene

Video summary above, more detailed description of steps below.

Note: my screen recording software has cut off part of the screen at the bottom🤦

1. Hit Shift-A to open the Add menu and select Text.

2. The text object will be added lying ‘flat’ on the ground of your scene, let’s stand it up: Left-click select the text object > hit R-key (rotate tool) > X-key (lock rotation to the X-axis) > type ‘90’ to rotate it 90 degrees so that it ‘stands up’ > Left-click or hit Return/Enter

3. With the text object selected, hit Tab to enter into edit mode: now you can delete and type in characters, spaces, new lines, etc. as normal.

4. Hit Tab again to get back to Object Mode.

5. Locate the text options in the right panel, it is denoted by a small ‘a’.

   1. Font: hit the file icon and navigate to a font file on your system, assign a font to the Regular slot. You should see your text update accordingly.

   2. Paragraph: adjust horizontal and vertical alignment as you wish.

   3. Geometry: set Extrude as you like, 0.1 or 0.05 should be adequate for the default text size. Optional: set the Bevel to a Depth to 0.01.

6. When you are ready to commit to your 3D text (you will no longer be able to edit the text after this step) Left-click select the text object, then Right-click it and select Convert To > Mesh.

7. We need to do a little mesh clean up now:

   1. Add and apply a Weld modifier using default settings. This will unify all faces of individual letters in to a single entity.

   2. Add a Decimate Modifier and adjust the Ratio until juuuust before you see the mesh begin to deform, then apply it. For the text “The Spatial Heritage Review”, I was able to reduce the face count from 45,000 to 7,300.

8. Finally, add a new Material to the mesh and give it a meaningful name.

Export.

Refer to the video above, more detailed steps below.

1. File > External Data > Pack Resources
   This step will pull all related texture files into the .blend file. Not always necessary but I’ve picked it up as a habit.

2. File > Export > glTF 2.0 (.glb/.gltf) to export your scene as a glb file.

Upload your 3D scene to Sketchfab & Setting Model Properties

Video summary above, more detailed description of steps below. Note: my screen recording software has cut off part of the screen at the bottom🤦

Log back into your Sketchfab account, drag and drop your finished scene .glb onto any page to begin upload.

Model Properties

1. Give your scene a sensible title, description, and tags and set the appropriate category. Be sure to add the appropriate attribution credits to the model description!

2. Enable App Free AR (Premium).

3. Save.

3D Settings

Video summary above, more detailed description of steps below. Note: my screen recording software has cut off part of the screen at the bottom🤦

1. Open your model’s 3D Settings

2. General: Adjust the camera field of view (FOV) to something like 20°, this can help with legibility of your 3D text in the web view.

3. Materials: Review the materials for each object in your scene - do they look right? Adjust as necessary. Turn up the emission on the text if you want a bright, flat look.

4. Lighting: Adjust dynamic and environment lighting as desired, add a ground shadow.

5. Post-processing: adjust the post-processing filters. At a minimum, add some SSAO, a little sharpness, maybe a vignette, and a bit of contrast.

6. Add annotations for each model that requires attribution, and any more for your use case.

7. Set the scene scale and user starting position in AR/VR settings.

8. Save settings.

9. Publish your scene!

Capturing a video of the 3D Scene in AR

On a compatible device, open the Sketchfab model page for your scene, and tap the [AR] button in the bottom right of the 3D viewer. This will download the relevant AR file format (glTF for Android, USDZ for iOS).

Once the model loads on your device, tap a surface somewhere in your immediate environment to place the 3D model in AR.

On higher end Apple devices that have LiDAR sensors, you will likely find that surface tracking is a little better.

Both Android and iOS include built in ways to record the AR experience, normally by holding down the shutter button for the duration of time you wish to catpture a video. If you’re struggling or unable to keep your thumb stuck to the shutter button, consider using the built in screen recorder or downloading a free app.

Depending on your use case, you may want to capture as smooth a video as possible - in my experience (and maybe it’s just me!) I’ve found walking around a 3D model in AR would cause the video to judder at each footstep. Consider using your devices built in stabilisation post processing (available in the Photos app on Android) or using a camera gimbal during capture. I use a cheapish gimbal from Hohem.

All done!

I hope this tutorial has given you some insight into sourcing free 3D models, automatically optimising them, building a scene, and publishing it online for web and AR viewing.

Please leave feedback and let me know what other tutorials you’d like to see in the comments and if you like this kind of content, please consider subscribing to my newsletter below!

Subscribe now

Attribution for 3D models used in this tutorial:

"Athena #3DST8" (https://skfb.ly/AEw8) by Digitage is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/).

"The Parade Shield of King Erik XIV of Sweden" (https://skfb.ly/6Rp9Y) by The Royal Armoury (Livrustkammaren) is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/).

"Wolf skull" (https://skfb.ly/6XFYr) by Virtual Museums of Małopolska is dedicated to the public domain via CC0 1.0 DEED (https://creativecommons.org/publicdomain/zero/1.0/).

"Italian Casket with Mythological Scenes, 1530" (https://skfb.ly/66QKA) by laurashea is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/).

If you’re interested in creating your own gaussian splats, I can recommend trying out Luma AI (web, cloud processing), Jawset Postshot (desktop, local processing), and Scaniverse (iOS app, local processing).

I’ve enjoyed capturing things in 3D using photogrammetry and phone based LiDAR for a some time, but recently I have a renewed enthusiasm for making myself look suspicious and weird in public: I love capturing plants and flowers as gaussian splats 💚

As always, I suggest you don’t just watch the video but also click or tap over to experience the 3D scenes interactively (links below). Doing so is not only a more fun experience (imo), it will also give you a better understanding of how this form of 3D capture might be useful for in your own work:

• Braya Alpina

• Common Gorse

• Platycladus orientalis

• Woolly Burdock

• Pin Oak Leaves

• Cotoneaster

Full disclosure: some of these plant names may be incorrect as, apart from common gorse which is an easy one, I had to use reverse image search to find out the names of which plants I had actually captured 😅 This i, one of the things I like about my hobby, though: I see something interesting looking or something that just looks nice, I make a capture of it and then before I publish it online, I am compelled to discover at least some rudimentary information about my subject. Because without context, it’s just a bunch of colourful pixels, no?

What are 3D Gaussian Splats?

It’s definitely a weird name. Gaussian splats (“splats'“ or “gsplats” as some people shorten it to) are the proverbial new kid on the 3D rendering block, despite NeRFs just moving into the neighbourhood too.

If you’re not familiar with gaussian splatting I can really recommend this blog post by Aras Pranckevičius, or this video by Bad Decisions Studio.

There are many ways to process input imagery or video into a gaussian splat, currently my favourites are Luma AI and Polycam. You can view splats on both of these cloud platforms via the web or a respective app, and there is also an rudimentary viewer/editor for splats for the online WebGL game engine PlayCanvas called Super Splat. There are also various plugins to import splats into the Unreal Engine and Unity game engines, and Felix Herbst of Needle has also demonstrated some neat editing and authoring capabilities.

If you know another platform or service for working with gsplats, why not drop a link in the comments?

Some things I like about capturing gaussian splats are:

• Capturing these scenes with a photogrammetry workflow just isn’t possible, so it’s fun to be able to add a new set of capture subjects to my list.

• The capture process is incredibly easy: capture a video on your phone, literally just wave it slowly around the subject (you can upload a .zip of images to Luma AI instead of a video if you like though). Most of the above captures I share above were made with less than a minute of video.

• All processing is done in the cloud: just upload the input data then sit back and wait for the notification that the capture has completed processing.

What are 3D gaussian splats good for?

At the time of writing, gaussian splats are mostly useful to enjoy as standalone pieces of 3D media, or in derivative form e.g. video or gifs or still renders.

Practical workflows for creating experiences with gsplats are no doubt in development, but right now you will need some coding chops to build a richer interactive experience on top of a splat.

Some very basic features I’d love to see made available for use with gaussian splats are:

• Web based editing tools (orientation, cropping, scaling, simplification, etc.).

• Annotation tools

• Better embed options

Regardless of these missing features and considering how easy it is to create gaussian splats - and here I refer to the capture process, not access to technology which is a different matter - I’d really love to see someone build some kind of crowd sourced digital plan identification guide that leverages 3D media.

Long story short: I think gaussian splats are pretty good and could be great once their utility increases. What do you think? Could be a useful medium for in the cultural heritage industry to pursue? Then again, the sector is still wrestling with how to make the most of ‘regular’ 3D yet …so maybe we should tackle that first 😉

If you enjoyed this post, please consider checking out my newsletter about 3D + cultural heritage The Spatial Heritage Review. Subscribe for free to receive new posts and support my work.

Subscribe now