Lytro

Lytro Illum: Getting in-depth about depth by Brenda Hartshorn

The most interesting thing about what the Lytro Illum can do that has an advantage over traditional photography, is that you can play with the depth in the scene.  This feature has the effect of allowing the viewer to feel more immersed in the scene by having the added 3D information of experiencing the illusion of depth off a 2D screen.  For the most part, the Illum's software is able to detect the depth of objects in the scene correctly.  However, there are a few conditions that confuse the processing which will result in some artifacts that will need to be manually corrected by pulling the depth map into Photoshop (or any other image editing software.)  In this post, I will discuss some challenging situations that the software has trouble with and cover how to correct the depth maps for processing.

Note: In the images shown below, areas shaded in blue are what the software is choosing as the near depth and the orange tinted areas are what is being interpreted as the far depth.


Texture & Patterns

Main Street, Chinese Camp, CA

Main Street, Chinese Camp, CA

As you can see from the depth map above, the software had a hard time choosing the correct depths for this busy scene.  The part of the wall that had been stuccoed over is so much smoother and brighter than the surrounding that this area became processed as being a different depth than the bricks.

Contrast: Low & High

Stalactites in the Black Chasm Cavern National Landmark.

Stalactites in the Black Chasm Cavern National Landmark.

Since the texture and tone of the stalactites are very similar to each other, it had trouble processing the correct depth in the upper left corner of the image for part of the far stalactite that borders the near one.  You can also see that the edge detection for this object type is a bit poor since the tip and edges of the formations are being mapped as part of the far depth.  The bottom right corner of the image is also not creating the correct depth map either but for different reasons.  If there are parts of the image that lack detail and either reach near pure black or pure white, the software is unable to estimate the depth correctly.  This area will become a depth artifact when processing and will either appear to be part of your foreground or background.

Blue marble rock formation near Volcano, CA.

Blue marble rock formation near Volcano, CA.

At the moment there does not appear to be a contestant rule for which the software chooses.  In the stalactite image, it incorrectly made part of the blackness of the cave a near depth.  While in contrast the image to the right, the software chose to make the mossy rock part of the far depth.

The software also has trouble with areas that are near white. The image below is a good example, as the bright parts of the clouds are being interpreted as being near.

Hatch Creek near Don Pedro Reservoir.

Hatch Creek near Don Pedro Reservoir.

Fixing the Depth Map

For users that have image editing software, Lytro has enabled a feature that allows them to easily correct the depth map.

  1. After selecting the image that you would like to fix, go to File>Export and select Depth Map from the drop down menu.  Select where you would like your file to be exported, and click Export.  You can then open the files in an image editor that allows you to manually paint the correct depth tone, like Photoshop.
  2. Open both of the files that are created in your image editor and use the color image as a layer on top of the gray scale depth image.  Photoshop may pop-up with an warning saying "The target document has a different depth than the source document.  This may result in lower than expected quality.  Are you sure you want to proceed?"  Just ignore this and click Yes as you are just using the image as a reference layer so you can better see what you are editing.  
  3. In your Layers pallet, drag the opacity of your reference layer down to about 20% or to a percent where you can see both layers.  For the depth map, the lighter tone represents the far depth and darker tone represents the near depth.  
  4. Select the Depth Map (Background layer) and the Eye dropper tool.  Set the Eye dropper tool to only sample the Current Layer from the drop down menu.  Use the Eye dropper to select an area with the same depth of the area that you would like to fix. 
  5. Now switch to the Paint Brush to correct the desired area.  You can resample depth area colors by holding Alt/Option to temporarily switch back and forth between the Eye dropper and Paint brush.  If you have large areas that need to be corrected, it can be easier to use the selection tools on the Reference layer image and then switch back to the Depth layer to paint in the desired area more precisely.  For this, you will want to turn the opacity of the Reference layer image back up to 100% temporarily while making the selection.
  6. Once you have addressed all problem areas, you can then delete the Reference image layer and save.
  7. You may now go back to the Lytro Desktop app.  Navigate to File>Import.  Under the Import drop down menu choose Selected Depth Maps, click Browse to find the file you would like to update, and then click Import.  After that it will just take a couple minutes to update the processing on your lightfield.

Skull: Lightfield and 3D model by Brenda Hartshorn

While accompanying a crew for doing some Native American archaeology exploration at Hatch Creek, one of the people from the party found a canine skull.  Initially, given the location of being out in the middle of Sierra Foothills (about a 3 hour hike from the nearest small town and an hour drive from the city of Modesto), the most logical explanation would have been that the skulls was from a coyote.  However, under further examination with the aid of using California Academy of Science's online 3D Skull exhibit, we were able to identify that the skull was actually from a Pitbull or breed close to it.  Since there are ranches in the area, it is possible that this dog came from one of them.  There did not appear to be an evidence of a mountain lion attack, however only the skull was found.

 

Although it was sad finding the skull of a dog and our survey of the area was unsuccessful for identifying any undocumented Miwok village site locations for the State of California, it was a great day to have been out in nature and use some technology I had helped make while working at Google.  The Skulls exhibit that California Academy of Science did in collaboration with Google's 3D scanning technology was one of the projects I was most excited to be involved in while working there.  I can now also be proud that I have used the technology as a scientific reference for identification. I hope to see this technology grow as a common tool for reference that is widely available to students, hobbyists, and professionals doing research.

Seeing the skull as a lightfield using the Lytro Illum has an added realism experience which is very nice.  Although, since I am unable to turn the skull around with a single lightfield, using it as a reference tool isn't as good as viewing the 3D model.  However, the lightfield is nice being able to see the higher resolution texture on the surface of the skull and be able to get a sense of the terrain in which it was found.  Therefore, the lightfield could serve as a nice supporting piece of material for an exhibit for adding context to the subject.

Lightfields at Pinnacles NP by Brenda Hartshorn

Took my Lytro Illum camera hiking and captured some lightfield of rock formations and a talus cave at Pinnacles National Park.  Although the park is home to the California condor and Townsend big-eared bats, I was not fortunate enough to see any.

The rock formation once part of the Neenach Volcano and sat on the San Andreas Fault.  The volcano was split in two as the Pacific Plate slid north nearly 200 miles.  The remain part that sits on the North American Plate is located near Lancaster, not to be confused with the Trona Pinnacles near Ridgecrest.  The Neenach formation is not open to the public as it is on private land.