Panoramas vs. Photosynth: Qualitative Comparison August 13, 2009
Posted by Mok Oh in 2.5D, panorama.Tags: EveryScape, Google, Microsoft, panoramas, photosynth, qualitative comparison
trackback
On Top of the Leaning Tower of Pisa
This is Part 4 of the Panorama vs. Photosynth comparison. You can find the other parts here:
In this section, we discuss some of the qualitative characteristic comparison between panoramas and photosynths.
Before I move on, though, many have asked, “How about combining both?” And the answer is an enthusiastic Abso-f’in-lutely! I think that is the ultimate goal for many. I really do think there is a “best of both worlds” solution. In fact, I would argue that it is imperative to combine both to capture the world. With that said, let’s move on.
I think it makes sense to think about the qualitative aspects of this comparison in these steps: Acquisition, processing, viewing, user experience (UX) and sharing.
Acquisition
Man, oh man.. Photosynth soooo has the right idea here. Let users take photos with their whatever cameras, put it into a “bucket” (a.k.a. the computer), and let it sort it out for me. Only little know-how of overlapping the pictures is necessary to get your synths going. HOLY SHIT this is powerful. I saw Noah Snavely give this talk at SIGGRAPH 2006 and was freakin’ blown away. The idea of using UGC photos to make sense this was mind blowing for me.
On the other hand, panoramas are not as easy to acquire. I’ve been taking panoramas for about a decade now, and it still kinda sucks how much work it involves the user.
Panorama 0, Photosynth 1
Processing
Processing for photosynth is quite easy. You go to Photosynth.net, upload your photos, and wait a few minutes. It’s all web based so no other software bit is necessary.
For panoramas, you need to have a stitching software, e.g. PTGui, PanoTools, EasyPano. Some are free, and most are cheap. These software have become quite easy to use over the years — you just have to load the photos, press a button to stitch, and wait a few minutes.
Here’s the dicey part. In photosynth, there’s no easy way to control the processing. If you took some pictures and there’s not a sufficient overlap or the computer can’t find common-enough similarities between the photos, then you’re kinda screwed. You can’t explicitly tell the computer, “Hey stupid, this picture belongs here, and that belongs there.” Often the computer algorithm gets it wrong, and only thing you can do is to live with it or remove the photo (which can subsequently mess other things up).
In panoramas, the user can have finer controls by explicitly giving it guidance. But this notion of what we call “correspondence” is not always an intuitive thing for some.
User control vs. simplicity.. Hm. Which one wins? Call it a draw.
Panorama 0, Photosynth 1 (same score as before).
Viewing
Viewing wise, I would give panoramas the advantage, just because the “standard” viewer now uses flash, which I think is a huge f’in deal! Photosynth uses Silverlight. Flash has about 97% penetration, Silverlight has about 33%. This means that Photosynth works in only about third of the computers. That sucks. Even when you have the Microsoft brand, very few people wanna download any plugins. Users expect things to just work .
Panorama 1, Photosynth 1.
UX
UX wise, this is definitely subjective. I have my biases with panoramas. And with Google really spreading the panoramic viewing experience with their Street View, panorama viewers and user interface and experiences are becoming more common. Besides, panoramas feel a lot more natural when looking around.
Photosynth has a weird user experience for me. I cannot move around a space the way I want to — you are limited to, in many ways, where the photos are. It’s somewhat frustrating, since it really feels like a 3D environment (it actually is), but the limitations of movement (and that annoying auto-snapping to pics) seem to misset my expectations.
Due to my bias, I’m not scoring this part.
Panorama 1, Photosynth 1.
Sharing
Sharing is pretty easy for both (albeit somewhat limited).
Panorama 2, Photosynth 2.
Overall
It looks like a tie using this simplistic scoring system. What do you think?
Mok, very thoughtful comparison — thanks. But if you really want to “capture the world,” I’m not sure that panoramas and Photosynth are the only two choices, or the best ones. To me, the limitation of both approaches is that you’re at the mercy of the camera’s point-of-view, especially with a panorama but even with Photosynth, as you point out. In other words, you can only see things from the exact places where the photographer stood. I’m not convinced that Everyscape or Street View effectively solve this problem. An obvious third alternative would be the route that Google Earth / Microsoft Virtual Earth / Second Life have started to mark out: build a computational 3D model of everything (using maps or architectural data or LIDAR measuremenst or whatever), then texture it with photos from the real world. See my 2007 Technology Review article “Second Earth” (http://www.technologyreview.com/Infotech/18911/, now behind a subscription wall, unfortunately) for a lengthy run-through of this concept.
Tx for the insightful comment, Wade. I think that’s another good idea for a blog – comparing 3D with 2.5D.
(http://allthingsv.com/2009/07/08/2d-3d-2-5d/)
I agree with you that there are other technologies that have the potential to create the “Second Earth” in 3D sense. But they either suck at ground level (e.g. Google Earth, Virtual Earth), or are not scalable yet (LiDAR, manual tools).
I’m quite certain that 3D will finally make its way into a scalable, distributable, visually believable “Second Earth.” Just not true yet, and I don’t believe it would be true for anther 5-10 years.
[...] Roush made a great comment in Panoramas vs. Photosynth Part 4, not to forget that there are forces in 3D happening as well. This blog is inspired by Wade. [...]
Greetings, Mok,
Fascinating series.
I’m a huge fan of Photosynth, but I have one small quarrel with you when you say “It’s all web based so no other software bit is necessary.”. The synther is definitely a standalone program and is Windows only. Your statement would make it sound more like http://seadragon.com where the conversion really does happen in the cloud. I actually consider this something of a dangerous weakness compared to what Google is doing with Panoramio and Picasa web album photos and Street View.
The trade-off with Google’s method is that you never know when they may or may not decide to recalculate the synth for any given location that you’ve geotagged your photos at so that it includes your photos. The pain point with Photosynth is that currently neighboring synths or overlapping synths do not yet link to each other. I am curious at how that will play out when they introduce the proper Bing Maps integration that they’re working on.
As far as Wade’s objection, above, that the automatic generation of polygonal models from photography is somehow a third and different approach, separate from what Photosynth is up to, I really do disagree. The point cloud is not, I feel, the final product of Photosynth’s calculations. The current point cloud can certainly be beefed up using proper stereo computer vision algorithms far past what is currently seen from the image features that are generated with the 2 megapixel versions of our photos that the current version of Photosynth uses for reconstruction to a reasonable voxel representation of objects.
Past dense point clouds is, as pointed to above, clearly the dream of vector/polygonal models derived from the point clouds as achieved by products like iModeller or D Sculptor. I look at this as providing the clear path forward as far as populating digital globes with flora. Perhaps living in cities makes people consider 3D buildings to be of prime importance but to me the greenery is mandatory to have a beautiful world worth walking around in. This is where Photosynth shines. I’ve seen the automatic tree placement that has been attempted in Virtual Earth 3D and while I am glad that someone on that team recognises the need, the results are just pitifully inept. Photosynth, by contrast, is fully and easily capable of generating a pointcloud of a tree trunk and branches that is immediately recognisable if you are familiar with the tree in question. Placing automatically generated leaves of an identified species of tree on an accurate trunk and branches is far more acceptable than slamming down a default tree model that attempts to match the height or diameter of what is seen in Bird’s Eye View or satellite imagery.
If you’ve seen someone make a synth purely for the point cloud before, it truly is able to be viewed from angles which photos were not taken from. Granted, object faces that were never seen in any of the shots that made the synth may be void of texture and it is certainly not photorealistic, but the represented portions still provide a strong approximation of what a tree or sculpture would look like from the air or below the ground.
That’s probably enough from me for today. =] Here’s to a digital globe rendered in CryEngine 5.
Hi Nate,
Wow, that’s a long response. Will try to address all your thoughts here.
By “stand alone” program, for me that means it’s something we (the consumers) can download and use. I’m not aware of any stand alone progs for photosynth. Perhaps they have an enterprise version that one can buy with hardware and such, but that’s definitely not a $79 panorama stitching program.
BUT, if there is a stand alone photosynth we can download, I’m all for that. Pls share! tx.
NICE LINK on the Bing site. Definitely looking forward to that as well.
If you have some URLs to share about the voxel representation, I would absolutely love to see that. Again, I don’t believe voxel representation is internet ready, but I would rather be wrong.
Also, photosynth can produce some point cloud of a tree (as per your example), but in the end, the point cloud representation needs to be “solidified.” Converting a cloud into tree polys robustly could be problematic. Splatting the points might work, but the point clouds would not be dense enough typically. So, anyway, again i’d love to see something that works at internet scale. Just haven’t yet.
Tx for your response! What line of work are you in, BTW?
Hello Mok,
Great blog. I want to comment on this post by referencing another article you wrote a while back explaining why Amazon’s A9 Block View and Microsoft’s Streetside failed (http://allthingsv.com/2009/07/28/a9-and-streetside-why-did-they-fail/). In your own words, “One could argue that panoramic imagery is more immersive and experiential.”
If this is the case, it explains perfectly well why panoramic photos such as those provided by Google Street View are preferred to static images like Photosynth’s. You are absolutely right when you say that users feel limited in their movements around a space using Photosynth. There’s no need to avoid being subjective; I think you’re absolutely correct. The immersiveness is a big deal, and I think it’ll lead to a big reason why panoramic photos will win in popularity in the near future.
I also disagree that panoramic photos are difficult to process. What about Google Street View? That’s something that’s only possible with panaramic photography. It would be nearly impossible to create panoramic views of every spot in a giant city using Photosynth.
Sheil
Hello Sheil!
Thanks for your insight and your readership. I am hoping that we are right. I’m assuming you’ve taken panos before?
To me, it’s really quite hard to decide which one I prefer. Photosynth’s biggest appeal, and advantage, is that it truly create a 3D virtual environment using nothing but a handful, or bucketful, of spare photos or even pictures! That to me is much more in prospect and promise in the long run! Imagine being able to create entire worlds, with nothing but pictures of it!
Now, on the other hand, panorama is still king in terms of its ability to create a truly interactive photograph. It takes what makes photography so great, and without taking anything away from that, makes it a two way experience b/w the user and program. Viewers are no longer limited by where they can and can’t look, like in traditional phtography. The biggest trump card panorama has against Photosynth is that even when you can form a whole 360 degree view from photographs, it still retain the artisticality found in traditional photography since it doesn’t distort the photos so much they look like a sloppily joined bunch of photographs.
So all in all, I’ll still be rooting for Photosynth in the long run, since this here is a system that can be improved and eventually catch on to the quality of a panorama, while panorama, as beautiful and awe inspiring as it is, is at its end of the road. To further improve panorama, there’s really nothing left to do but to integrate some system similar to photosynth, which is quite counter-productive to this comparison and discussion.