3D Scanning Reflective Objects With Photogrammetry

By James Busby

One question I see rising time and time again is “how do I scan shiny or reflective objects with photogrammetry” The answer is, you can't, it's impossible to scan a / transparent / reflective or very shiny objects using photogrammetry without some kind of surface preparation or complicated projection technique. Photogrammetry works by defining exact positions of reference points on a given surface. So an object like the glass bowl we are going to use for this example will have little to no surface detail for the software to pick up on. Anything that is there will be confused by reflections and refractions within the surface of the glass. A few people have expressed the opinion that using a polarising filter on the camera lens will help in some way, whilst it will help to remove unwanted reflections the surface will still contain little or no detail and therefore will not scan well. In this tutorial, I'm going to show you a fast and relatively non-destructive way to scan almost any reflective surface.

There is only really one way around the problem and that is to coat the surface with something that will scan. I realise that the techniques in this tutorial might not be suitable for all objects such as high-value items that the owner might not take to kindly to having sprayed with matting powder or historical artefacts. But is the only way to really get good scan data from such an object using photogrammetry. For this example, I've chosen the hardest possible subject. A highly polished transparent glass bowl, a photogrammetry nightmare if ever there was one.

So the first stage is to prepare the tools and equipment that you will need.

Camera

Any camera will do but if you want the best results a DSLR with manual exposure ISO and aperture control is best, failing that a point and shoot with manual controls. For this tutorial, I'm using a Canon 100d with a 35mm prime lens.

3D Coating spray

Available from David this is very fine grain powder that will stick to most surfaces but more importantly can easily be removed after the scan. There are other coating sprays out there but we have found this is the easiest to apply and importantly remove later. Link to spray here

Black water based paint

For this example, I'm using "Snazaroo classic colours" black face paint as it's PH neutral and is very easily removed. Link to face paint

Toothbrush

Any type will do as long as it has bristles left and you never intend to brush your teeth with it ever again.

Software

You will need a copy of either AGI photoscan, Reality Capture or if your budget doesn't stretch to those Autodesk Remake has a free version. For this tutorial I'm going to be using Reality capture as it's my software of choice at the moment but the same principals apply to all the above.

Method

Step 1 Prepare the surface for scanning

The first thing we're going to do is prepare the surface of the bowl for scanning, as we discussed earlier the only way to capture an object like this is to transform the surface from something featureless and reflective to something a little more conducive to scanning. I.e loads of features with a matt surface.

Place the object on a piece of wood / thick card or a surface that you can easily pick up and move around without disturbing the subject. The reason for this is we are about to spray it with the coating powder and in all likelihood, you're not going to want to do that in the same location that you capture the images. I would recommend doing this in a large well-ventilated room or outside in the shade where this is very little wind.
Use the 3D coating spray to apply an even surface to the entirety of the object making sure not to miss any overhanging areas, nooks or crannies.
Leave the spray to dry for approximately 10 minutes, at first, it might appear that surface has not turned matt but as the spray dries into a powder it will become increasingly matt. You should end up with something similar to the image below

Now for the fun part, take your toothbrush and dip it in a cup of water to make is nice and wet, then rub the bristles onto the surface of the facepaint until they have picked up a good amount of black paint. Now just as you learned in primary school brush your thumb over the top of the toothbrush whilst pointing it at the surface of the object. You will immediately see 10000's of tiny little splatters of paint appearing on the surface, do this for the entire object. This is the detail that we need in order to reconstruct a good 3d scan so remember the more dots the better the scan. You should end up with something that resembles the images below.

Wait for the facepaint to dry, at least 5 minutes and we are done. Both the facepaint and the coating spray can easily be washed off after the scan.

Step 2 Camera and location setup

Make sure your camera is set to the fastest possible shutter speed with the lowest possible ISO for this example I'm using F8, ISO 200 and shutter speed of 1/100th of a second. If you're worried about motion blur you can use a tripod however if you are shooting outside be aware of the weather and changing light. The setting I'm using here i.e 1/100th of a second is not really fast enough to get blur-free handheld shots and with F8 there will be quite a shallow depth of field. I'm doing this because I want to show what can be achieved with not only lower quality cameras abut also relatively low-quality images not everyone is going to get it right the first time and I figured I'd make it hard for myself. If you want really sharp images I would say 1/200th of a second minimum shutter speed and minimum aperture of F11 this will, however, mean bumping the ISO up to around 800 so be aware of noise.
For this example I'm going to shoot the images outside, we do have a studio but I want this tutorial to be relevant to everyone not just those with access to a professional lighting setup. When shooting it's important to choose a day when the sky is overcast or somewhere where there is no direct sunlight, shooting on a bright day in the shade can work as I have done here. Below is an example of an image taken from this shoot.

Step 3 Shooting

Move around the subject taking pictures in a cylindrical motion. The technique I'm using here is a very simple tried and tested process. I'm going to pick 8 points around the object, so every 45 degrees or so. Starting down low on my knees at the first point I take 3 shots from the same position moving the camera about 10 cm left then up between each shot. Then I move around the object about 45 degrees and do the same again. once you I've completed the lower ring of shots I move to a crouch so that the next ring will be slightly higher and do the same again remembering each time to take 3 images with roughly 10cm offsets on each one.
I can't express the importance of overlapping images enough, for every shot that you take there should be at least 1/3 of the same area viewable in the last shot. When taking the three shots from the same position you should only offset the actual framing by only a few centimetres.
For this example, I shot around 345 images In RAW format, however for the purposes of this tutorial I have converted them to JPG because not everyone has a camera that shoots RAW.

Step 4 Converting images to geometry

Ok so that's the scanning part over, pretty simple really. What I'm going to do now is show you how to build clean and combine the scans you just shot in order to create a high-resolution scan of our reflective object.

Im going to be using Reality Capture, you can use AGI photoscan or any of the other solutions out there but this is our software of choice but the steps are more or less the same for AGI, Align, Build, Export.
The first thing we're going to do is load all the images into Reality Capture (RC for short) Simply hit the "Align Images" button and depending on the speed of your computer within 10 minutes you will have a sparse point cloud as shown below. This determines the positions of the cameras in preparation for the high detail build. As you can see here all 344 of my images aligned perfectly.
Once the alignment is complete we can click the "High Detail" button to begin the dense point cloud generation. This can take a long time, our 20 core Xeon machine took about 2.5 hours to generate this mesh so please be patient with it. some slower machines could take up to 10 hours. The best thing to do is to leave it going overnight and wake up to a nice mesh. You can see the dense point cloud in the image below
Because this point cloud is about a zillion polygons we need simplify it before we can export. Click the "simplify" in the reconstruction tab and enter the desired polygon count. For this example, I'm using 15 million polygons. It's quite a large mesh but I can easily load it into ZBrush and decimate it further from there. Try experimenting with the level of simplification to find a setting that works best for you. The image below shows the simplified point cloud that we will be exporting.
Hit the "Export mesh" button in the reconstruction tab and chose a location to save the file. I'm exporting as OBJ as its compatible with Zbrush.

Step 5 Zbrush Cleanup

Because I shot this outside with a fairly low shutter speed 100/th of a second and only F8 there is quite a lot of out of focus sections of the images and not all the shots were perfectly in focus due to shaky hands. This is one of the problems shooting outside without a tripod. Anyway, the result of this is that there is an amount of high-frequency surface noise covering the entire object. I'm not going to go into great detail about how to remove this in Zbrush as this will be the topic of another tutorial. But if you follow a few simple steps it's fairly easy to do.

Decimate the mesh down to around 200k polygons, this will increase the size of the triangles whilst still maintaining the shape of the mesh the upshot of this is that it does a fairly good job of removing any high frequency noise.
Once the decimation is complete duplicate the layer and Subdivide the model 2 times and then perform a Zremesh on the duplicate layer with the slider set to around 20.
Subdivide the new Zremeshed object 5 times and use the projection tool to project the details from the first decimated subdivided mesh back onto the new Zremeshed object.
Go in with a small smooth brush and carefully remove any remaining areas of noise, the Clay buildup tool is especially helpful for filling out areas that might have suffered from occlusion.
Finally were going to do one more decimation on the cleaned re-meshed scan so that it can be exported to your rendering software of choice, for me that's Modo. Im goiI'm to use a setting of around 750 which will give me a 750,000 polygon object, small enough to easily load into modo but large enough to retain all the details. We could go down the route of exporting the low res mesh with a displacement and normal map, but for the purposes of this tutorial Im going to with the fastest and simplest technique.

Step 6 Rendering

I'm going to be rendering this model in Modo, for the simple reason that its quick and easy to get amazing results with very little effort. For this render scene, I also scanned some oranges using the same techniques described before but without the surface coating, and orange has enough surface detail to scan easily.

The first thing is to import the model file>import. I positioned the oranges in Zbrush and imported them into the scene the same way.
Next, I loaded up the preset browser and imported the "Studio lighting sweep curve" for the background.
For the lighting, I simply deleted the scene light and loaded the Kitchen 01 HDR from the presets panel.
To create the glass I simply dragged the "Glass flint heaviest" surface from the preset browser onto the object. make sure double sided is switched on in the material properties panel.
For the orange surface, I used the standard shader with a slight blue tint to the specularity and the colour map as the bump map, very simple but effective in this situation.