Panoramas vs. 3D (Part 1): Introduction August 16, 2009
Posted by Mok Oh in 2.5D, 3D, Photography, mirror worlds, panorama.Tags: 3D, 2.5D, EveryScape, panoramas, Google Earth, Microsoft Bing
2 comments
Google Earth St. Patrick's Cathedral, New York, NY, at a "bird's eye" view.
Wade Roush made a great comment in Panoramas vs. Photosynth Part 4, not to forget that there are forces in 3D happening as well. This blog is inspired by Wade. Thanks, Wade!
Now, imagine an online web representation of our world in 3D… You type in maps.bing.com to get to a place you are interested in. You find a map, and you zoom down close to see the street. Then you click on the “3D” viewing mode. You zoom in some more. Then you land on the street… a photorealistic, real-time 3D street. You can smoothly walk around, not just in the streets but on the side walk as well. Hell, you can fly if you wanted to. You find the store you like, and you navigate inside. When you do, you are greeted by the store owner avatar saying hello. And it’s actually a person behind the avatar. You ask your questions about the store, if they have what you are looking for. You walk around to browse what they have. Then you walk out and zoom to your favorite restaurant to see how crowded the place really is currently… And cut. (Or wake up.)
I think this scenario, or something similar, has a real possibility in the future. Assuming this can happen, then the question is, how far off are we? It feels to me at least a decade away. Most likely more.
Now let’s get back to reality and talk about what we have today. We have Google Earth and Microsoft Bing that have a 3D representation of our world. There are wonderful other technologies, like C3 Technologies, but they have not yet proven to be scalable (by “proven,” I mean published and has significant coverage). It really is amazing to see cities like New York in full 3D glory, BUT from a “bird’s eye” view. At a thousand feet above ground, these cities look amazingly real.
At a ground level, not so much. (See below for a comparison.)
Photograph of St. Patrick's Cathedral, New York, NY
Google Earth St. Patrick's Cathedral, New York, NY
So, another relevant question is: What is it going to take to create a believable 3D on the web at the ground level? I think quite a bit. (To be discussed in the following blogs.)
At a bird’s eye point of view, the level of detail required to make a person believe what she’s seeing is “real” is much less so than at a ground level. Up there, the buildings are more or less boxes with photo textures. (Don’t get me wrong — the feat accomplished by Google Earth and Microsoft Bing are incredible.) But at a ground level, I would argue that there is an order of magnitude more 3D shit you gotta model to convince people that you are actually at 43rd and 5th in New York City, or Time Square or Champs Elysees. You have all the street level things that require to be represented realistically — people, cars, trees, news stands, lamp posts, signs, etc. All these have to be there just like they are in real life to convince folks that they’re really there.
So, for the foreseeable future, we have panoramas and UGC photographs to represent our real world, aka mirror world, a la Google Street View and EveryScape. These are what I call 2.5D representations, since they are not quite 3D, but more than 2D.
This series of my blog sounds like fun to write. Want more? Please let me know.
Much Ado About Augmented Reality? July 26, 2009
Posted by Mok Oh in 3D, Mobile, augmented reality.Tags: 3D, augmented reality, iPhone, Mobile, real time
9 comments
Augmented Reality Heads-Up Display View
My question to you is, do you believe in Augmented Reality?
In many ways, the notion of augmented reality is similar to 3D in a sense that expectations have been set high, but it hasn’t quite delivered. We expected HUD in cars that help us with directions and more; we expected augmented reality glasses that help us with what we are seeing — and we expected this to happen a decade ago. Hm…
Recently, there’s been a lot of talk about iPhone 3.1 that will enable real-time overlays for applications like augmented reality stuff. Definitely looking forward to what will happen there, and what type of innovations will be pushed from this community.
Personally, I think there’s a lot more innovation in technology and UI/UX for AR to succeed. Image recognition algorithms are not robust enough for general scenes (like in our every-day lives), but when supplemented with GPS, compass, accelerometers, and other sensors just might work and be useful.
What do you think?
I Want My 3D! July 5, 2009
Posted by Mok Oh in 3D.Tags: 3D, flash, Jurassic Park, viewer
8 comments
3D Operating System from Jurassic Park
Remember in Jurassic Park (yes, the first one) when the little girl character sees the computer system and says something like, “This is Unix! I can do this!” and she navigates the database in 3D? (Screenshot above). Well, to me, that was pretty darn cool. I thought then, wow, this is how the future of OS’s will be like. That was around the time first web browsers began to surface as well and the future looked bright for 3D.
Fast forward to present day. Why aren’t we browsing the web on 3D? Why aren’t the Operating Systems more 3D? Where the hell is Virtual Reality that I was promised? WTF happened??? I WANT MY 3D!!!
I’ve been in the 3D world (both in academic and industry) long enough to have an opinion, and I would be very interested if someone’s done some survey or deep analysis on this. Let’s go back and check with my last blog on 3D for a framework on this discussion.
Content authoring in 3D is hard.
To incorporate 3D into feature-length films cost millions of dollars and tremendous amounts of 3D brain power (lots of expensive PhDs). 3D games also take a long time to create, lots of $ and resources, and take some very smart and experienced folks and tools. I don’t think 3D authoring will get that much easier any time soon, but as long as some people are spending enough $ and putting enough resources and brain power into this problem, shouldn’t there be enough to bring this to the mainstream?
Content distribution for 3D is hard, but getting better
We used to need to buy a set of CDs or DVDs to get the 3D content (let’s say a game) into our computers way back when. These days, pretty much everything digital is delivered via the internet — even 3D content. Google Earth and Virtual Earth both have tremendous amounts of images and 3D content, but not all need to be delivered at once. In 3D lingo, level of detail algorithms enable “on demand” content delivery. For example, if you are using Google Earth or Maps, the right images are delivered to your viewer depending on your zoom level. But still, it takes quite a bit to load a 3D city in Google Earth or Virtual Earth.
Need a 3D viewer on the web browser
No one downloads a plugin. Well, very few. And without a “standard” 3D viewer on the browser it’s still hard to see 3D content. I think Flash will change everything. 3D can be displayed using Flash 10, and although it’s got some performance limitations so far, it can bring the 3D experience to your browsers now, and much improved versions for the future. Very exciting.
3D User Experirnce (UX) is getting better
I think that one of the detrimental things that happened to 3D was that the user experience of 3D were initially done by engineers (and therefore usable only by engineers). Yeah, sure there are engineers with design chops, but that’s for some other blog. 3D UX in my opinion was hard, and I still think it’s too hard. I’ve seen a recent 3D demo by one of the big players and even he had a hard time navigating the 3D scene. I think we can get there by getting more designers in the loop (and engineers out of the loop 
. Think Apple.
Thoughts about 3D (part 1) — The Framework July 4, 2009
Posted by Mok Oh in 3D.Tags: 3D, application, authoring distribution, file format, framework, player, UI, UX, viewer
1 comment so far
The 3D Axis -- Yep, 'Y' is up.
In the past few blogs, I’ve focused on the state of panoramas as visual medium. There’s still quite a bit to talk about there, but I’d like to shift gears to discussing 3D. 3D is a big topic and there are lots to discuss, so I will break down this blog series into specific topics within 3D — hence the “part 1.”
Let’s first discuss a framework from which we can breakdown 3D into various sub components:
- 3D content authoring – This refers to the content authoring; how they are made (e.g. tools to automation), file formats
- 3D distribution – Here, we’ll talk about how the content gets to the users, file formats, algorithms, etc.
- 3D viewer/player – Once the data reaches the user, a viewer or a player is necessary to see and interact with the content
- 3D application – Finally, there should be a reason for creation of the 3D content; we discuss UI, UX
So, moving forward, we can categorize our discussions into one (or more) of these buckets to give us a good basis for a decent framework. There may will be more, and this is not an exhaustive list, but it’s a good start (and I ain’t waiting for this to be right to blog).
Mobile Reality Demo June 30, 2009
Posted by Mok Oh in 3D, Mobile, Photography, panorama, post processing.Tags: 3D, augmented reality, Mobile, panorama, Photography
2 comments
I was privileged enough to be a part of a panel in Mobile Reality at the Where 2.0 Conference this year, chaired by Brady Forrest. Here’s the short description of the panel:
“An emerging class of smartphones including location-based services and persistent data connections are lenses by which we can effectively view data layers atop physical space. What was once only available from tethered desktop computers is now possible from pocket-sized companion devices that travel with us. We are seeing examples of this in their earliest incarnations – social networking, gaming, reference and commerce.
Opposed to looking far into the future, this panel looks at examples of this technology in use and available today to consumers on a variety of smartphone platforms, including the Apple iPhone and Google Android. Panelists will provide short demonstrations of this technology, followed by a topic discussion and Q&A.”
The reason for sharing this is to show you EveryScape’s initiatives towards mobility. I believe EveryScape has one of the coolest and most useful visual platforms around (in my unbiased opinion), and you can see a glimpse of what’s being worked on in the video below (starting at the 2:3o mark).
