Category Archives: post processing

August 4, 2009

Panoramas vs. Photosynth (Part 3): Technical Characteristics

By Mok Oh

Photographically capturing the WORLD!

Photographically capturing the WORLD!

This is Part 3 of this series.  (At least I didn’t pull a Lucas and start with part 4.)

Let’s compare technical characteristics/requirements as I’ve mentioned in Part 1 (and pls read Part 2 as well).

  • scalable
  • distributable
  • maintainable
  • extensible

Again, there may be more and these are not orthogonal or exclusive of each other.

Scalability

Now, remember that the context in which I’m comparing these two “methods” are in trying to photographicaly capture the entire world at a human-level POV!  So imagine an online experience where you can go to a website (e.g. EveryScape and Google) and be able to walk around just like you were there.  Yep. BIG idea.

So, being able to scalably capture, store, distribute, share, etc. the whole world is tantamount.  If you can’t do this, then game over man.

Companies like EveryScape, Google, Earthmine, Mapjack, Immersive Media and bunch others found a way to (cost) effectively drive around cities with car-mounted cameras.  Especially EveryScape and Google have done this scalably in multiple cities all round the world with thousands of miles of coverage.  (I’m sure there are others but I haven’t seen this much quantity of their content published yet.)  I think this is proof enough for me to say that panoramic images can scalably cover the world.

Photosynth has not quite done this yet.  I’ve seen pretty extensive number of photographs used to represent a landmark or an area, but I have not yet seen an entire city done this way yet.  There are lots of brilliant minds at this, I’m sure, and it does feel feasbile.  But if content publication is the standard…

Panorama 1, Photosynth 0.

Distributability

By this, I mean folks online can easily view and experience the content.  Again, going to everyscape.com or maps.google.com is proof enough.  Using Flash (and Flash did “change the world” in this sense) or Silverlight, users can experience the content, and the backend seems to have been implemented well.

Oh BTW, SeaDragon‘s f’in brilliant!

Panorama 2, Photosynth 1.

Maintainability

We live in a dynamic world.  Things change all around us.  Tomorrow, a Starbucks could turn into a Dunkin Donuts (yes, I’m from the east coast).  By maintainability, I mean that these changes in the real world could easily be reflected in the mirror world online.

In any type of changes in the real world, we (EveryScape) have a “self healing” backend, so only real work is photo acquisition.  Assuming all other car-mounted systems are similar, this is technically solved.

For Photosynth, it seems like a similar approach will work.  Although there may be some ownership issues with Photosynth (if crowd sourced), it feels quite easy to make this assumption of maintainability.

Panorama 3, Photosynth 2.

Extensibility

Panorama 4, Photosynth 3.

Overview of Technical Characteristics

It seems like the main tech difference between Panoramas and Photosynth is the scalability. One main issue with Photosynth is the image registration / pose estimation problem and how scalable this can be.  Basically, for each image added to the synth, features are detected, then corresponded to the rest of the point cloud, then a relative camera extrinsics are computed.  (Apologies for the tech lingo.)  I’m not fully convinced that this is the way to go when scaling up to what I want (da world!).  Perhaps supplementing the image with GPS and other sensors is a good way to solve this.  BUT, if the philosophy for Photosynth is still automation, consumer cameras, and crowd sourcing, I’m not sure I quite believe in scalability (yet).

Is scalability issue overcome-able for Photosynth?  I think yes.  Just need to see it to believe it.

6 comments   |  tags: , , , , , , , | posted in 2.5D, mirror worlds, panorama, Photography, post processing

July 23, 2009

HDR Part 2: Exposure Fusion

By Mok Oh

This blog is the second part of the previous blog on high dynamic range imagery.

Exposure Fusion (a.k.a. Enfuse) does not use HDR.  But it is related in a sense that it uses multiple exposures to create a nice “fused” image.  (So technically, “part 2″ is a bit misleading.)

Exposure Fusion was a paper by Mertens, Kautz, and Van Reeth in 2007, and you can learn more about the work here.  This technique basically bypasses HDR creation all together to create a wonderfully fused image.

Let’s just briefly discuss some issues with HDR (I will discuss some benefits of HDR in the next blog).  HDR “assembly” takes quite a bit of processing time and the file sizes bloat up big time — which also means longer time to load to any programs like Photoshop to do anything to it.  From there, you typically end up tone mapping the image anyway.  And don’t get some folks started on the pain-in-the-ass-ness of tone mapping.  Yeah, it generally sucks when you end up doing a lot of them by hand.

Exposure Fusion basically says, “that’s bullsh!t!” There’s no need to convert a bunch of files to something you won’t use, then have to convert again, only to spend the next 2 hours tweaking some parameters you don’t understand, that was named by some ivory-tower researchers (sorry guys ;-) ). Exposure fusion just creates a wonderfully “fused” image from your multiple-exposure set, which is the part I really like.

So, gettin’down to the brass tax, if you have a hard time going from HDR, then back to LDR using some tone mapping operator that doesn’t understand you, then use Enfuse.  It’s one of the most consistent way to create an image from multiple exposures.  And, it’ll save you time and lots of disk space.

One caveat is that Enfuse is a command line tool.  If you don’t like that, you can find some GUI wrapper programs out there (e.g. Bracketeer).

5 comments   |  tags: , , , , , , , | posted in Cameras, HDR, Photography, post processing

July 21, 2009

High Dynamic Range Imagery (part 1) — What the Heck Is It?

By Mok Oh

So, you’ve heard some folks talk about “high-dynamic range imagery” or HDR, and you think you sort of understand it, or not really?  Well then, I hope to de-mystify it for you in a series of blogs.

In my previous blog, I talked (or more like bitched) about why cameras suck; and one of my reasons was that they lacked sufficient dynamic range in capturing light.

“…, let’s talk about dynamic range.  This is the whole problem of the images above.  We’re only stuck with 0-255 per RGB channels.  This means that we need to describe the brightness of what we see — from dark shadows to sunlight — within the integer range of 0 to 255.  Even RAWs don’t cover it since the dynamic range needed to describe what we see could be 0-1,000,000.  Yes, cameras suck.  There are high-dynamic range imagery, and I will talk more about that soon.”

As shown above, there are series of photos that describe this problem.  In the beach shot 1, you see that the exposure was very long, so most pixels are washed out, but you can still see some contrast in the dark parts of the palm trees as well as the dark shadows on the sand ridges.

As the series of images get darker, e.g. beach shot 2 and beach shot 3, you can see the scenery much better, but the bright area around the sun is still too washed out.  By beach shot 6, we can see some outline of the sun better but everything else is now too dark.

Somewhere in this series of variously-exposed images lies the “right” answer for the composite image — this is the dynamic range problem.  Our eyes can see much better contrast than any of these camera shots (also because we can dynamically adapt better too, but that’s some other blog).  You can imagine manually photoshopping these images to get the solution image you want.  A more automated way is to create a single high-dynamic range image from this series of images, then tone map it.

Putting aside the technical lingo bullsh!t, I hope I’ve convinced you that there is a way to combine these images somehow to get the final image you want. (And I won’t bore you with the tech details either — if you must know, let me know pls.)

There are nice software products to do just this: Photoshop, Photomatix, and Enfuse.  There are more, but these are the ones I like.  (If you have your favorite, please comment and share!)

Beach shot "solution" using Photoshop

Beach shot "combined solution" using Photoshop

Beach shot "combined solution" using Photomatix

Beach shot "combined solution" using Photomatix

I’m not showing Enfuse just yet since it really isn’t HDR.  But I’m gonna stop right here for now, since the blog is getting too long.  I will talk more about Enfuse and more of HDRI-related issues in part 2 of this series.

13 comments   |  tags: , , , , , , | posted in Cameras, Photography, post processing

July 18, 2009

Arc de Triomphe Photography with My iPhone

By Mok Oh

One way to get more resolution or field of view is to create a panorama — take more photos and put them together.  My previous two posts have been about this, and am following up with a few more examples of Arc de Triomphe in Paris.

As I’ve mentioned before, I used AutoStitch on my iPhone 3G S.  Much of the panoramas were an experimentation of adding some time and positional elements, which resulted in pretty cool stitched photos.

To get what I call the time element, I stayed in the same place a few minutes waiting for dynamic elements of the scene to change — e.g. cars, people, clouds.  By doing this, things that are static remain more solid and things that move have a ghost-like quality to them.

To get what I call the positional elements, I tried to focus on a feature as I walked along a path.  In these examples, I focused on the Arc while moving towards it.  This tends to create an impressionist-painting-like effect.

4 comments   |  tags: , , , , , , , , , , , | posted in Mobile, panorama, Photography, post processing

July 13, 2009

AutoStitch iPhone App

By Mok Oh

I wanted an app on the iPhone that stitched mosaics/panoramas on the iPhone without having to off-load anything.  So, I searched the App store for “panoramas” and found a bunch, and tested them.  I found three apps that I liked:

  1. AutoStitch
  2. Pano
  3. PanoLab

My clear favorite was AutoStitch, since it seemed to be the easiest to use.  All I had to do was to select a bunch of photos that I took for a panorama, throw them in a “bucket,” and then click “stitch.”  After crunching away, it created some pretty awesome stitched mosaics.  All photos that you see posted here are stitched using AutoStich.  Definitely try it out.

3 comments   |  tags: , , , , | posted in panorama, Photography, post processing

June 30, 2009

Mobile Reality Demo

By Mok Oh

I was privileged enough to be a part of a panel in Mobile Reality at the Where 2.0 Conference this year, chaired by Brady Forrest.  Here’s the short description of the panel:

“An emerging class of smartphones including location-based services and persistent data connections are lenses by which we can effectively view data layers atop physical space. What was once only available from tethered desktop computers is now possible from pocket-sized companion devices that travel with us. We are seeing examples of this in their earliest incarnations – social networking, gaming, reference and commerce.

Opposed to looking far into the future, this panel looks at examples of this technology in use and available today to consumers on a variety of smartphone platforms, including the Apple iPhone and Google Android. Panelists will provide short demonstrations of this technology, followed by a topic discussion and Q&A.”

The reason for sharing this is to show you EveryScape’s initiatives towards mobility.  I believe EveryScape has one of the coolest and most useful visual platforms around (in my unbiased opinion), and you can see a glimpse of what’s being worked on in the video below (starting at the 2:3o mark).

2 comments   |  tags: , , , , | posted in 3D, Mobile, panorama, Photography, post processing

June 30, 2009

The Era of Post Processing (part deux)

By Mok Oh

I’ve opened up a can full of thoughts and ideas with my last blog on Post Processing.  So, imagine this:  What if you can create a 3D model out of any image?  What would you do with it?  How would you use it and for what?

This is work I did at MIT with Max Chen and Fredo Durand many years ago.

Leave a comment   |  tags: , , , | posted in Photography, post processing

June 26, 2009

3D Photoshop-ery!

By Mok Oh

Recently I wrote a blog about panoramic photography and about post processing.  I found this related video above which was a part of my research in collaboration with Max Chen and Fredo Durand at MIT around 2000.  This was the panorama that I took of the Omni Parker House Hotel mentioned in this post.  This was created from a single panoramic image.  I will certainly write more about this topic, but I want to drive home the fact that post processing of photographs is not limited to pixels and colors — geometry is certainly a part of it.  Case and point — we live in a three-dimensional space and photos are projections of that captured light.

2 comments   |  tags: , , , , | posted in panorama, Photography, post processing

June 25, 2009

The Post-Processing Era of Photography

By Mok Oh

The art in photography used to depend heavily on how and when the photographer captured the light into the film during the photo shoot. I remember counting down the number of shots I had left in my 35mm film, painstakingly developing my black and whites, and using the negatives to create the positives. From acquisition to seeing the image, it took forever. And only after seeing the images (which may be days after the shoot) could you know what you’ve done wrong.

These days, the art in photography depend heavily on the science of post processing. (Can you say Photoshop?) Digital SLRs can pack in tens of mega pixels, at multiple frames per second, at a higher dynamic range (i.e. RAWs), and your instant-gratification desires are gratified. When in doubt, take more photos. And to get that perfect image, you “Photoshop” them or post process using your favorite tool or technique. If you capture enough light, you can (theoretically anyway) recreate and reproject the rays of light from the past to the present.

The quality of photography became directly related to the quantity of photography. I don’t think this is bad at all. The medium has changed — it has become more complex and more expressive, I think (I’m definitely not a “purist”).

So, what type of post processing am I talking about? Take a look at this video above let me know what you think — it was created from a single photograph.

19 comments   |  tags: , , , | posted in panorama, Photography, post processing

Follow

Get every new post delivered to your Inbox.