Tag Archives: panorama

September 24, 2009

Commoditization of Car-Mounted Immersive Imagery

By Mok Oh
PixKorea Car

PixKorea Car. They use digital SLRs with fisheye lenses. The mechanical rig can change heights. Cool.

Sorry it’s been a while since my last post.  I’ve been busy with family, work, my dog ate my home work, had to wash my hair…

I was invited to give a talk at a conference in Seoul, Korea called National Spatial Data Infrastructure Expo 2009 (Sep. 9-11).  I spoke about “How to Paint the World,” which stressed on the importance of a framework for capturing, processing, storing and distributing photorealistic, immersive, interactive content of our world for various applications (e.g. local search).  Ok, that sounded fancier than it actually was (or perhaps more boring than it was?).

But that isn’t the gist of this blog.  I wanted to write about how pleasantly surprised I was to see so many street-level, car-mounted camera aquisition systems in the show floor of the expo.  I think I saw at least 5 companies doing that when I walked around half the show floor, with various configs and cameras.

Car-mounted ground-level immersive imagery

This car uses Point Grey's Lady Bug (red on top) as well as digital SLRs on the bottom. Not exactly sure why.

IMG_1727

This car uses Point Grey's Lady Bug and two GPS's to determine orientation. When asked how well that worked, the answer was ambiguous.

In general, I am seeing a bunch of companies being formed that have a car-mounted system for street-level panoramic acquisition around the world.  I’m glad to see this, since it feels like another step towards this content type being useful and in demand.  But ultimately, street content will be commoditized (even before it can be monetized — but that’s whole other topic).

So, what does this mean?  Well, it means the consumers win in the long run.  It also means that the competition will hopefully improve the image quality of this exterior content (really lead by Google Street View).  Further differentiation and innovation needed to win in the competitive market will push the innovative minds to do a lot more than just display panoramas — enabling mashups, UGCs, improving extensibility and maintainability, encoding a whole lot more geo info, getting INTERIORS (a-hem!) etc. will be necessary for survival.  As I said, this should all be good for the consumers, if it pans out this way.  Yay.

It’s still a bit early to tell who, how, what will win or lose.  And somewhat surprisingly (to a US-centric person), Google is not winning else where around the world.  Yay.

2 comments   |  tags: , , , , , | posted in Acquisition, Cameras

August 10, 2009

Tribe Sourcing, Crowd Sourcing, and Automation

By Mok Oh
Floating Drones!

Floating Drones!

UPDATE: A friend of mine told me about an article on the New York Times about a floating drone the army’s working on!  Automatic photo acquisition technology is closer than we think.

Cab Sourcing

Cab Sourcing the Panoramic Photography Data Collection

When you are mass producing something repeatedly, you absolutely want automation as much as possible. Machines typically produce less error and are more consistent than humans for very specific tasks. But when you can’t, for one reason or another, going for either crowd sourcing or tribe sourcing makes a heap of sense. This doesn’t mean that automation has no place — it means automation takes on a different role.

Let’s be a bit more specific.

In my context, I’m talking about taking ba-zillitons of photographs from a human perspective (as opposed to a satellite perspective) and making sense of it for users. There are companies like Google, EveryScape, Microsoft, Tele Atlas, and NavTeq, that go around photographing the world for online use. Let’s focus on photographic data collection as a “case study” for this blog.

So the right set of questions in this context may be:  Can we automate the picture-taking process? If not, can we tap into the crowd?  How about creating a tribe?

Star Wars Darth Maul Droids

Star Wars Darth Maul Droids

Well, automating the photography of the entire world would be tough. One future solution could be to create lots of robots that walk, drive, or fly around acquiring and geotagging pictures.  NERD ALERT!  Remember Star Wars Episode 1, where Darth Maul sent out floating droids in Tatooween to find Princess Amidala? Something like that.  Unfortunately, we don’t have these droids yet.  (Can someone get on that???)

So, when automation isn’t possible, the next question is: Can we crowd source?  I’m not certain if we can crowd source this yet either, since car-mounted camera systems aren’t something we can buy at Best Buy. I don’t think it’s that far off either.  We may see panoramic cameras mounted on taxis — what I call cab sourcing — for instance. There are a few logistical, business-related, and technical issues that needs to be solved before this can happen, but why not?

Microsoft’s Photosynth harnesses the power of the crowd to make sense of a real place, but it’s yet to be seen that this technology can conquer the world.  (In fact, I’m looking forward to a new research publication coming out this September).

Google Street View Car

Google Street View Car

Tribe sourcing is the viable solution for now — find leadership, enable and incentivize the tribe to go out and photograph the world according to some plan. Google has the cash to create (quite wonderfully equipped) cars and have folks drive around. EveryScape’s found a cost-effective solution for this to tribe source (a.k.a. The Ambassador Program).

Until automation can happen, photographic data collection happens with tribes (or crowds).  Automation plays a role in that pictures are taken, geotagged, oriented, stitched, and processed automatically.

Leave a comment   |  tags: , , , , , , , , , | posted in 2.5D, maps, mirror worlds, panorama

August 6, 2009

Why Peleng Lens Sucks for Panoramas

By Mok Oh

When taking images for panoramas, a lot of folks use circular fisheye lenses — the wide field of view means less pictures to cover the full 360 x 180 degrees, so you can acquire less and stitch faster.

I played around with various lenses out in the market for panoramic photography trying to balance price vs. quality.   One of my favorite is Sigma 8mm, but unfortunately, Sigma is really hiking up their prices — what used to cost around ~$500, is now up to ~$900 (MSRP $1230!).  Can you say “exploitation?” Something’s going on.  (If anyone knows what’s up here, pls comment!)

Naturally, I looked for an alternative and found Peleng 8mm at less than half the price!  But in testing it out, it ended up being not acceptable for professional panoramic photography.

Peleng 8mm Fisheye Lens

Peleng 8mm Fisheye Lens

Here is my main reason for saying this is unacceptable — two words: lens flare.

Peleng 8mm Lens Test

Peleng 8mm Lens Test

Notice the flares on the bottom right (from light source on upper left) and bottom left (from light source on upper right) of this image.  The resulting panorama is below:

Peleng 8mm Lens Test: Stitched Pano

Peleng 8mm Lens Test: Stitched Pano

Now notice the flares creeping into the stitched panorama above on the bottom left.  (Pls click on the image to see a higher resolution panorama.)  Although this issue can be “solved” by cropping tighter or Photoshopping, this then messes up enough things to a point I would rather use a better (but more expensive) lens.

Other things like the lens not fitting in snugly to the camera, having a hard time taking the lens off, manual controls, etc. make me think that it’s not worth the savings for the risk of taking bad pictures for our customers.

Peleng, if you are reading, please let us know when you’ve improved and resolved some problems.  Until then, we are sticking to Sigmas.

Anyone out there have a similar experience?

1 comment   |  tags: , , , , , | posted in Cameras, panorama

August 4, 2009

Panoramas vs. Photosynth (Part 3): Technical Characteristics

By Mok Oh

Photographically capturing the WORLD!

Photographically capturing the WORLD!

This is Part 3 of this series.  (At least I didn’t pull a Lucas and start with part 4.)

Let’s compare technical characteristics/requirements as I’ve mentioned in Part 1 (and pls read Part 2 as well).

  • scalable
  • distributable
  • maintainable
  • extensible

Again, there may be more and these are not orthogonal or exclusive of each other.

Scalability

Now, remember that the context in which I’m comparing these two “methods” are in trying to photographicaly capture the entire world at a human-level POV!  So imagine an online experience where you can go to a website (e.g. EveryScape and Google) and be able to walk around just like you were there.  Yep. BIG idea.

So, being able to scalably capture, store, distribute, share, etc. the whole world is tantamount.  If you can’t do this, then game over man.

Companies like EveryScape, Google, Earthmine, Mapjack, Immersive Media and bunch others found a way to (cost) effectively drive around cities with car-mounted cameras.  Especially EveryScape and Google have done this scalably in multiple cities all round the world with thousands of miles of coverage.  (I’m sure there are others but I haven’t seen this much quantity of their content published yet.)  I think this is proof enough for me to say that panoramic images can scalably cover the world.

Photosynth has not quite done this yet.  I’ve seen pretty extensive number of photographs used to represent a landmark or an area, but I have not yet seen an entire city done this way yet.  There are lots of brilliant minds at this, I’m sure, and it does feel feasbile.  But if content publication is the standard…

Panorama 1, Photosynth 0.

Distributability

By this, I mean folks online can easily view and experience the content.  Again, going to everyscape.com or maps.google.com is proof enough.  Using Flash (and Flash did “change the world” in this sense) or Silverlight, users can experience the content, and the backend seems to have been implemented well.

Oh BTW, SeaDragon‘s f’in brilliant!

Panorama 2, Photosynth 1.

Maintainability

We live in a dynamic world.  Things change all around us.  Tomorrow, a Starbucks could turn into a Dunkin Donuts (yes, I’m from the east coast).  By maintainability, I mean that these changes in the real world could easily be reflected in the mirror world online.

In any type of changes in the real world, we (EveryScape) have a “self healing” backend, so only real work is photo acquisition.  Assuming all other car-mounted systems are similar, this is technically solved.

For Photosynth, it seems like a similar approach will work.  Although there may be some ownership issues with Photosynth (if crowd sourced), it feels quite easy to make this assumption of maintainability.

Panorama 3, Photosynth 2.

Extensibility

Panorama 4, Photosynth 3.

Overview of Technical Characteristics

It seems like the main tech difference between Panoramas and Photosynth is the scalability. One main issue with Photosynth is the image registration / pose estimation problem and how scalable this can be.  Basically, for each image added to the synth, features are detected, then corresponded to the rest of the point cloud, then a relative camera extrinsics are computed.  (Apologies for the tech lingo.)  I’m not fully convinced that this is the way to go when scaling up to what I want (da world!).  Perhaps supplementing the image with GPS and other sensors is a good way to solve this.  BUT, if the philosophy for Photosynth is still automation, consumer cameras, and crowd sourcing, I’m not sure I quite believe in scalability (yet).

Is scalability issue overcome-able for Photosynth?  I think yes.  Just need to see it to believe it.

6 comments   |  tags: , , , , , , , | posted in 2.5D, mirror worlds, panorama, Photography, post processing

August 3, 2009

Panoramas vs. Photosynth (Part 2): What Are They?

By Mok Oh

This is a continuation from Panoramas vs. Photosynth Part 1.  In Part 1, we discussed some characteristics that we may use to determine which one is better.  But before we delve into that, let me describe a bit more about these methods of capturing our world.

Panoramas

Panoramas assume that pictures are taken from a common point in space (i.e. pivoting around the entrance pupil).  That’s why a panoramic heads (shown below) are important in minimizing errors caused by parallax, so we can stitch a nice panorama without much visual artifacts.  Also, due to their immersive nature, many images are necessary to take a full 360 x 180 degree panorama.  Many people resort to using wide-angle or fisheye lens to lessen the quantity of images necessary to cover all view directions.

Panoramic tripod head

Panoramic tripod head

Panoramas are amazing in their immersive nature — you really do feel like you are there.  And the continuity of the experience is just fantastic in describing the scene captured.  But the constraints on the acquisition process make it more difficult to capture than just taking regular pictures.

Another limitation is that the users cannot move around in space.  But of course, I will argue that EveryScape has solved that.

Photosynth

Photosynth, on the other hand, has no such constraints when acquiring images.  You don’t have to be anal about the “entrance pupil,” or the “no-parallax point,” or the “pivot point” or all that BS.  Although technically rational, these constraints really do suck for the picture takers.  (That’s why there are gazillions more regular photographers than panoramic photographers.)  In taking pictures for Photosynth, there just needs to be sufficient visual overlap between them, such that the computer algorithm can automatically try to determine where the pictures were taken from.  This also means that you can move around in space!

But often times, using just pictures do suck in a sense that the users don’t quite feel immersive.  Immersivity is one of the qualities I mentioned in Part 1.  I was talking to a friend of mine a while back (@billwarner), and he said to me, “Don’t break reality.”  In many ways, if you “break reality,” then you don’t gain as much confidence from your users about the space you are describing.

So, What Next?

There are pros and cons for both these methods and in the next part, we will start to compare them and grade them if we can.

3 comments   |  tags: , , , , | posted in 2.5D, panorama, Photography

August 2, 2009

Panoramas vs. Photosynth (Part 1)

By Mok Oh
Photosynth example

Photosynth example

In this series, I’ll compare panoramas vs. Photosynth — methods that could be used to create a photographic platform of our world.  Well, actually more like initiate a conversation in comparing these two methods in representing our world (aka Mirror Worlds and the Metaverse Roadmap).  I’ll assume people have a decent understanding of what these are.  If not, please let me know and I can certainly explain.

So, how can we tell which one’s better?  Only time will tell.  Photosynth has a pretty huge backing and push from Microsoft, and personally I’m a huge fan.  On the other hand, panoramas have a big push as well from companies like EveryScape and Google.

One way to at least start a conversation about which one’s better is to list a set of requirements (i.e. “things you want”), weigh which one is more important, and rate how well either one does as a weighted sum (yes, I’m a f!@# nerd).

So what are these requirements?  Here’s what I think for starters:

Technical:

  • scalable
  • distributable
  • maintainable
  • extensible

Qualitative:

  • believable
  • photorealisitic
  • interactive
  • immersive

Web 2.0:

  • annotatable
  • searchable
  • sharable
  • personalization-able

Operational:

  • cost
  • time
  • resources needed

Business:

  • how do you make $ from this?
  • how do you get more users?
  • what are some applications?

UI/UX:

  • how intuitive and simple to use is it?
  • can my parents use it?
  • how helpful is it?

I’ve mentioned some of them in my talk at ETech 2009 and Where 2.0 2008.  The list may vary depending on specific applications, but for general platforms it’s a decent list, IMO.

Did I miss anything?  Please let me know, and stay tuned for the next set of blogs in this series.

4 comments   |  tags: , , , , , , | posted in 2.5D, panorama, Platforms

July 28, 2009

Panoramic Equipment

By Mok Oh

I’ve been taking panoramic images  for over 10 years, and I’ve been using various gears for taking them — cameras, lenses, rotating heads, tripods, and GPS.  Curious about what I use now?  Here’s my list.

Lens

My lens of choice is Sigma 8mm fisheye lens.  In general, I prefer the fisheye lens since the field of view is very wide, i.e. need to take less amount of pictures to cover the full 360 x 180 degrees; i.e. faster.

The optics is quite good, and we’ve had very few of them fail.  There’s some chromatic aberration, but typically stitching software takes care of that.

Sigma 8mm Fish Eye Lens

Sigma 8mm Fisheye Lens

Camera

My camera brand of choice is Canon.  We’ve tried Nikons but they failed a lot more for us under extreme conditions (ask me if you’re curious).  I currently use Canon T1i, which has a 1080p video recording capability. Awesome camera.

Canon T1i

Canon T1i

Because the T1i is not a full frame digital SLR, when used with the Sigma 8mm, the circular fisheye image is cropped.  But I actually prefer the crop for better “resolution” of the scene.

Panoramic Tripod Head

My choice for panoramic tripod head is Nodal Ninja R1.  It’s light, compact, sturdy, and precise. Also, because the mount attaches to the ring-mounted lens (see images below), you don’t have to worry about messing up the focus of the lens — I initially had some trepidation about this, but not any more.

Nodal Ninja R1 Ring-Mounted Camera

Nodal Ninja R1 Ring-Mounted Camera

So there.  What do you use to take your panoramas?  Care to share?

7 comments   |  tags: , , , , , , , , | posted in Cameras, panorama, Photography

July 16, 2009

Paris, the Visual City

By Mok Oh
More Paris

Paris shot from inside a car

More often than not, I find that a typical field of view of a camera is not enough.  Using my iPhone 3GS, I cannot help but create panoramas and mosaics to get a wider field of view or to tell a “story.”

I am visiting Paris on a biz trip and below are some examples of mosaics I shot using AutoStitch app on my iPhone.  If you have not been to Paris yet, two words for you:  You must.  It is such a visual city with incredible amounts of beautiful sites all around.

Rue du Nil, Paris.

Ghosts walking on Rue du Nil, Paris.

Preparing for some car-mounted photography in Paris

Preparing for some car-mounted photography in Paris

Dinner in Paris

Dinner in Paris

Paris

Paris

7 comments   |  tags: , , , , , , | posted in panorama, Photography

July 13, 2009

AutoStitch iPhone App

By Mok Oh

I wanted an app on the iPhone that stitched mosaics/panoramas on the iPhone without having to off-load anything.  So, I searched the App store for “panoramas” and found a bunch, and tested them.  I found three apps that I liked:

  1. AutoStitch
  2. Pano
  3. PanoLab

My clear favorite was AutoStitch, since it seemed to be the easiest to use.  All I had to do was to select a bunch of photos that I took for a panorama, throw them in a “bucket,” and then click “stitch.”  After crunching away, it created some pretty awesome stitched mosaics.  All photos that you see posted here are stitched using AutoStich.  Definitely try it out.

3 comments   |  tags: , , , , | posted in panorama, Photography, post processing

July 9, 2009

What About the Inside?

By Mok Oh

About a year ago in 2008, I had the privilege to speak in front of the Where 2.0 2008 audience.  The topic was titled, “What About the Inside?” (Online video link here.)  I’m bringing this up again since the question is still valid — in some sense much more valid now than before.  What was the context?  It was a question meant not only for the Where 2.0 audience, but also meant for the GIS, mapping, GPS, 3D/2D imaging, government/civic, architectural and urban planning, and any community of people who were doing any work that gathers information and interacts with interior places and spaces we live.

The premise was that we have an unnatural bias towards mapping the outside spaces more so than the inside.  Perhaps it has some historical significance, where outside spaces were the final frontier.  Well, that’s no longer true, since much of the earth’s outdoors has been mapped, explored, marked.  GPS, satellite and aerial imaging, smart software algorithms (e.g. image registration), and the Moore’s Law definitely changed the landscape of how we can gather information for the outside.  Now we even have ground-level imagery from Google, EveryScape, etc., and we can zoom around places virtually.

But what about the INSIDE spaces?  I surely don’t want to stop at just the entrance to a place I want to visit.  I want to go INSIDE!  Perhaps the right question is: How can I experience of the inside spaces of every interior space I’d like to visit?  Yeah, it’s a big question.  And in this context, by “I,” I mean for every single person who uses the internet.  There are definitely issues such as privacy, no GPS, no aerial photos, multiple floors/levels, etc. that question existing technologies and solutions, since they don’t seem to “map” that well for the interiors.

Please check out my Where 2.0 2008 presentation video and I would love to hear your thoughts.  And below is the clip of an EveryScape eye-candy commercial.

5 comments   |  tags: , , , | posted in maps, panorama, Photography

Follow

Get every new post delivered to your Inbox.