Monthly Archives: June 2009

Mobile Reality Demo

I was privileged enough to be a part of a panel in Mobile Reality at the Where 2.0 Conference this year, chaired by Brady Forrest.  Here’s the short description of the panel:

“An emerging class of smartphones including location-based services and persistent data connections are lenses by which we can effectively view data layers atop physical space. What was once only available from tethered desktop computers is now possible from pocket-sized companion devices that travel with us. We are seeing examples of this in their earliest incarnations – social networking, gaming, reference and commerce.

Opposed to looking far into the future, this panel looks at examples of this technology in use and available today to consumers on a variety of smartphone platforms, including the Apple iPhone and Google Android. Panelists will provide short demonstrations of this technology, followed by a topic discussion and Q&A.”

The reason for sharing this is to show you EveryScape’s initiatives towards mobility.  I believe EveryScape has one of the coolest and most useful visual platforms around (in my unbiased opinion), and you can see a glimpse of what’s being worked on in the video below (starting at the 2:3o mark).


The Era of Post Processing (part deux)

I’ve opened up a can full of thoughts and ideas with my last blog on Post Processing.  So, imagine this:  What if you can create a 3D model out of any image?  What would you do with it?  How would you use it and for what?

This is work I did at MIT with Max Chen and Fredo Durand many years ago.


VideoScapes: Storytelling with Panoramas

VideoScape Authoring Tool

EveryScape's VideoScape Authoring Tool

Videos have an inherent aptitude and tendency to want to tell a story. Its nature is to capture light and sound along a flow of time.  Photos on the other hand typically capture light a slice in time.  A single picture can tell a story (since it’s worth a thousand words).  But often times, a series of photos are used in a slide show to tell a story.

So, the obvious question is — What about panoramas?  Of course, a single panorama can tell a story, but it always helps if there’s a time component involved.

EveryScape Inc. created what we call VideoScapes.  VideoScapes are authored and edited in a WYSIWYG way to tell a story using a series of connected panoramas in space.  It’s quite cool in my unbiased opinion.  :)   I believe it is a powerful and easy way to create dynamic videos and stories.  You can pause at any given time, look around, and press play to continue.

Some cool example links below.

Audio and background music are coming soon.


On Top of the Leaning Tower of Pisa

On Top of the Leaning Tower of Pisa

On Top of the Leaning Tower of Pi

Here is a panorama I took a few years ago.  From my last blog, where I commented on the state of the publishing and sharing of panoramas, I finally gave up on trying to find a website that does everything I want and ended up publishing my own using a hosting service.  Either click on the image above or here.

I think the image is powerful and wonderful, and as close to being there with the current state of consumer technology.

Comments welcome!


Thoughts about Panoramic Photography (part 2)

The Leaning Tower of Pisa, Italy.

A Panorama of the Leaning Tower of Pisa, Italy.

FRUSTRATION is the word that comes to mind today.  I spent most of my Saturday trying to figure out if there is an easy way to publish and share my panoramas (as a common user, not a techie).  As anyone would do, I first searched for my “YouTube for Panoramas” sites and found a bunch.  After getting accounts and trying most of these webistes, I found the whole experience difficult and frustrating.  Perhaps my bar was too high in comparing them to YouTube or any other video serving websites.  Perhaps things are different for panoramas.  Perhaps it’s too much to ask…

On second thought, NO.  I shouldn’t have to lower my expectations for panoramas — they’ve been around as long as photography and videos!  I think the problem is that the panoramic community is not rising up to the challenge of making panoramic medium as common as videos or photographs.

Here’s what I want at a high level:

  • Acquisition – Taking panoramas should be as easy as point-and-shooting with a digital camera.  I do believe there are ways to simplify this.
  • Post Processing – Stitching, image processing, tone mapping, etc. should be easy as a drag-and-drop, and have access to “pro” features for those who want/need finer control.  I do believe this part has made much progress thanks to many innovative minds.
  • File Formats – To a common user, this should be as understandable as a “JPEG.”  My mom knows what a JPEG is, but she doesn’t know what an equirectangular JPEG is.
  • Viewing – What happens when you double click on a JPEG?  I want that for my panos.
  • PublishingYouTube, Flickr, Picasa
  • SharingYouTube, Flickr, Picasa

I absolutely don’t think this is problematic.  It can be done.  In fact, much of the work has already been done.  So, why haven’t this happened yet?  Is there not enough demand?  Not enough cash, incentives or investments in this field?  Are there no business models or economics to make this a fruitful endeavor?

My guess is that we, as panorama community, need to make the process — from Acquisition all the way to Sharing — easy as 1-2-3 so that my mom can do it.  Let’s not make it into some esoteric art form that only a few can do.


3D Photoshop-ery!

Recently I wrote a blog about panoramic photography and about post processing.  I found this related video above which was a part of my research in collaboration with Max Chen and Fredo Durand at MIT around 2000.  This was the panorama that I took of the Omni Parker House Hotel mentioned in this post.  This was created from a single panoramic image.  I will certainly write more about this topic, but I want to drive home the fact that post processing of photographs is not limited to pixels and colors — geometry is certainly a part of it.  Case and point — we live in a three-dimensional space and photos are projections of that captured light.


The Post-Processing Era of Photography

The art in photography used to depend heavily on how and when the photographer captured the light into the film during the photo shoot. I remember counting down the number of shots I had left in my 35mm film, painstakingly developing my black and whites, and using the negatives to create the positives. From acquisition to seeing the image, it took forever. And only after seeing the images (which may be days after the shoot) could you know what you’ve done wrong.

These days, the art in photography depend heavily on the science of post processing. (Can you say Photoshop?) Digital SLRs can pack in tens of mega pixels, at multiple frames per second, at a higher dynamic range (i.e. RAWs), and your instant-gratification desires are gratified. When in doubt, take more photos. And to get that perfect image, you “Photoshop” them or post process using your favorite tool or technique. If you capture enough light, you can (theoretically anyway) recreate and reproject the rays of light from the past to the present.

The quality of photography became directly related to the quantity of photography. I don’t think this is bad at all. The medium has changed — it has become more complex and more expressive, I think (I’m definitely not a “purist”).

So, what type of post processing am I talking about? Take a look at this video above let me know what you think — it was created from a single photograph.


Thoughts about Panoramic Photography (part 1)

Leaning Tower of Pisa pano

Panoramic images are powerful.  If a picture is worth a thousand words, panoramas are worth an order of magnitude more (IMHO).  And yet, panos still are not prevalent (and perhaps not relevant enough).  Herein lies the premise of this blog.

I’ve been taking digital panoramic photography for about a decade now (tried analog panoramic photography many years longer a la David Hockney), pretty close to the time when digital cameras started to become more consumer friendly.  I remember in either 1999 or 2000 during grad school, going to the Omni Parker House Hotel in Boston around 1 AM, and talking my way in to try to take some panoramic imagery of their lobby for my research.  (Had to ask a few times to finally get a “yes” from someone… anyone ;-) ). By the time I was done with a couple of (high-dynamic range) panoramas, it was pas 3 AM.  I was using a heavy Bogen tripod, Peace River Studios 3Sixty pan head, and a digital SLR (I think it was a digital Nikon D1).  These all belonged to MIT (I was a student and couldn’t afford these wonderful equipment). The panorama I took was multi-rowed and multi-exposured.  So it took over an hour to take a single panoramic image.

Panoramic stitching programs didn’t exist much then, especially for HDRs (high-dynamic range imagery — I will talk more about these in some other post).  I used a stitching program written by a bunch of folks in the MIT Computer Graphics Lab.  It literally took hours (perhaps close to a day) of number crunching to correspond, warp and stitch each photograph.  And often times, the algorithm would fail — and I’d only find out after all that waiting via visual inspection.  Basically, the whole experience sucked (by today’s standards).

And finally to view the panorama, I used a quite expensive SGI machine (totally expensive and overrated) using OpenGL and some viewer I had to code up.  Ugh.  What a pain in the ass.

Today, I can take a panorama in less than a minute, and it would then take me a few minutes to stitch and display on a web browser.  What a difference a decade can make thanks to  convergence and improvements in various technologies!

And yet, panoramic photography still has not gone “main stream.”  Everyone who has a digital camera can (potentially) take a panoramic imagery, and there are a lot of digital cameras out there.  Tens or even hundreds of billions of digital images are online which can extrapolate to potentially billions of photos taken every day throughout the world.  BUT how many panoramic photographs are taken by consumers in comparison?  My guess would be about 1 percent of 1 percent.  It’s hard to answer this question without further research (hence the “part 1″ of this blog), but I do have some thoughts as to why:

  • Hardware — it’s gotta be as simple as a point-and-shoot.
  • Extra hardware — we don’t always carry around a rotating head and a tripod (Well, not everybody).  It has to be as simple as carrying a camera.
  • Software — nobody wants to spend their weekend stitching panoramas.  Well, almost nobody.  Either the “stitching” step needs to disappear all together or be as simple as clicking a button.
  • Viewing — to view a panorama, a special viewer is needed.  Every computer pretty much has an image viewer.  Every computer should have a panorama viewer.

If these are requirements for panoramic photography to be prevalent as consumer photography, then perhaps the bar’s too high.  Dunno yet.  More thoughts about this later.


Hello (Visual) World!

It's a Small World

The World is shrinking… or so people say. I would argue that it’s not. I think we just started to uncover the complexities of our World. For example, my (current) mission is to try to figure out how we can photographically capture the world, not just for the outside (e.g. aerial photos), but also for the insides — to create a visual platform on which people can virtually and visually navigate our World for various purposes. Oh, and there are many constraints to this problem — cash, resources, usability, scalability, deployability, social networkability, crowd sourceability, privacy issues, APIs, resolution — for starters. (Check out everyscape.com.)

Is it possible?  Abso-f’ing-lutely.  Is it probable?  More yes than maybe.  Stay tuned.


Follow

Get every new post delivered to your Inbox.