Making 2D photos in to 3D

How do you take a 2D photograph and create a 3D scene out of it? When you take a 2D photograph the depth information is inherently loss. However when we see 2D photograph our brains can actually still perceive and estimate depth information, for example, a tree is in front and house is about 10 feet in the back and mountain is very far even though the photograph doesn't have any depth information in it at all. Using machine learning it is possible to partially recover and estimate depth information just like our brains does. For this, the team at Stanford has achieved very convincing results: first they divide the photograph in multiple segments and then applies several image processing filters for each segment to extract information such as blurriness, texture variation, shading etc. Then they create some training shots by sending out a robot toy car with laser light to get some sample scenes and actual depth information in the scene (they also use computer generated scenes to train the model). Then they feed this training data to algorithm based on Marcov Random Field to train the model. Each photograph is nothing but a connected graph of these segments as nodes and MRF fits quite well to operate on this graph using properties and connection of node as input and depth as output. Try out some photos you have taken at their website.

On unrelated note, there is an effect called Hitchcock Zoom where camera moves closer (or away) but in a way so that the subject size remains same by reducing (or increasing) zoom. As this is very counter intuitive for our visual system it provides for an interesting effect :).

Fast Asymmetric Generalized Hebbian Algorithm

I can get sucked in to challenges very easily especially when that involves Artificial Intelligence or statistical analysis. The challenge that has occupied my interests these days is the one that was put up by Netflix. It’s easy to describe: They give you 100 million data points for a triplet (Customer, Movie, Rating) and you have to predict the rating for given (customer, movie) pairs. If the average of squared errors of your predictions is below certain value you get a million dollar prize.

The problem is nothing new in the field. The researchers have been developing techniques for this class of problems since centuries – often without anticipating rewards. Any such material reward would be embarrassingly insignificant compared to the real prize - understanding the most unique and powerful thing in existence that we are aware of: The intelligence.

What makes Netflix prize an interesting challenge, however, is that it’s very well defined and several researchers are trying out their tools of trade so it also provides quantitative measurement for comparison. There are some consensus that Netflix have set the bar just high enough that no one would ever be able to achieve the lowest required RMS. But that shouldn’t stop you to enjoy the game and push extremes to new boundaries.

So how am I doing this? I started out brushing up on all existing techniques: Neural networks with linear elements, back propagation, principle component analysis and SVD, logistic regression (still many more to go: Bayesian networks, Markov decision process, SOMs and Recurrent networks). It’s one thing to read about these algorithms from text books and other to actually put in the practice to solve real world problems efficiently. The difficulty using these techniques straight from textbook (without domain specific enhancements) is that they suck when your data set is huge (matrix with 8.6 billion elements) and that there is no real generalized algorithms to determine several parameters such as learning rate, number of units and so on effectively.

The one algorithm that swept me away among all of these is called Generalized Hebbian Algorithm (GHA) - which probably is the most practical algorithm out there for linear problems since 1985 but is not described in even latest well known textbooks! This algorithm can deal with essentially infinite data that available serially, it will use only required amount of memory to hold eigenvectors and perform SVD starting with most significant eigenvalue!

In any case, I'm making my implementation of highly optimized version of this neural network algorithm available with source code.

Again limitation of GHA (and AGHA) is that they work best on linear problems only.

Earth: The Largest Neural Model

The individuals in population and their interaction with each other very closely models neural activities in brain. The entire neural network can be looked upon as smaller fractal version of human population, with lot of details vanished. It occurs to me that an alien might prefer to look upon Earth as the planet hosting one huge "brain" with individual humans essentially just forming "cells" for this brain.

Trading Geek Dinner For Self-Theories Of Intelligence

I'd to ditch the grand New York geek dinner to attend a talk on how the Brain makes memories at New York Academy of Sciences. This is my current absolute favorite subject to spend all my free time so I HAD to be there. The talk also turned out to be very energetic, fun and fast paced by Jennifer Mangels of Columbia University. The first part of the talk was about the role of the hippocampus in forming long term memories and her research. What they tried to do was to record EEG signals from different areas of the neocortex when a person tries to memorize something and recall it back later. Her research attempts to imperially prove that different areas of the brain must strongly participate together to have hippocampus realize the importance of the incoming information and form the contextual long term memories. While I really disgust at how everyone in neuroscience these days just do some EEGs and fMRIs and run around to write conclusions, above theory fits well with Jeff Hawkin's On Intelligence.

However more interesting was the second part of the talk: The Self Theory of Intelligence. This is very interesting. In 1970s, Carol Dweck did some research on human motivation in school children and noted that some students intrinsically tend to persist in the face of failure while others quit as soon as the going gets rough. After more investigations, she discovered that student's beliefs about the nature of intelligence had a strong connection with the way they approach challenging intellectual tasks: Students who view their intelligence as an unchangeable internal characteristic tend to shy away from academic challenges, whereas students who believe that their intelligence can be increased through effort and persistence seek them out.

Students who hold an "entity" theory of intelligence agree with statements such as "Your intelligence is something about you that you can't change very much." Since they believe their intelligence is fixed, these students place high value on success. They worry that failure-or even having to work very hard at something-will be perceived as evidence of their low intelligence. Therefore, they make academic choices that maximize the possibility that they will perform well. For example, a student may opt to take a lower-level course because it will be easier to earn an A. In contrast, students who have an "incremental" theory of intelligence are not threatened by failure. Because they believe that their intelligence can be increased through effort and persistence, these students set mastery goals and seek academic challenges that they believe will help them to grow intellectually (Dweck, 1999b).

Dr. Dweck's research on the impact of praise suggests that many teachers and parents may be unwittingly leading students to accept an entity view of intelligence. By praising students for their intelligence, rather than effort, many adults are sending the message that success and failure depend on something beyond the students' control. Comments such as "You got a great score on your math test, Jimmy! You are such a smart boy!" are interpreted by students as "If success means that I am smart, then failure must mean that I am dumb." When these students perform well they have high self-esteem, but this crashes as soon as they hit an academic stumbling block. Students who are praised for their effort are much more likely to view intelligence as being malleable, and their self-esteem remains stable regardless of how hard they may have to work to succeed at a task. (More at her page)

Dr.Mangels then showed some videos demonstrating how EEG patterns differ in these two types of people. This is very significant. It essentially implies that you can device a helmet for a person to wear and after few EEG recordings I would be able to tell if person is in one group or another! Think, interviews would be so different ;).