A brief history of statistical inference

In the beginning of 19th century, the majority of the statisticians believed in parametric methods and embraced maximum likelihood estimation. For a given problem, the standard workflow was to first identify an appropriate parametric model and then apply maximum likelihood estimation to find the parameter for the model. This approach worked very well for a while. Concurrently, several statisticians were interested in non-parametric methods. They proved that the empirically distribution converges to the true distribution universally and exponentially fast. This led to a general approach for statistical inference. However, this general approach was not widely appreciated and mostly considered as purely technical achievements. The field of statistical inference was dominated by parametric methods. This situation remained the case until the invention of computers, which leads to information explosion.

With increasing amount and complexity of data, statisticians began to realize the disadvantages of parametric methods. They identified the curse of dimensionality. They found that maximum likelihood estimation is not always the best approach for statistical inference. In order to apply statistical inference to challenging problems, they later move to the idea of empirical risk minimization. That is, instead of finding a parameter that best explains the data, one finds a function that results in the minimum empirical lost, called learning machine. However, unlike maximum likelihood estimation, statisticians was unable to prove the consistency and convergence of methods based on empirical risk minimization.

The proof only came later with the introduction of VC dimension. It was shown that both the necessary and sufficient conditions of consistency and convergence of methods based on empirical risk minimization depend on the capacity of the set of functions implemented by the learning machine. It is necessary and sufficient that the set of functions has a finite VC dimension. With this proof, statisticians are more satisfied now and move forward to solve more challenging problems.

fast MAP inference on xRF (2012)

After almost two weeks of travel, I'm finally back from my CVPR+new England trip : ) I guess that I should start writing about the interesting trend I observed in the papers published in major vision related conferences recently.

The first trend that I would like to talk about is "fast MAP inference on xRF model".

*Philipp Krähenbühl and Vladlen Koltun, "Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials",  NIPS'11

*Yimeng Zhang, and Tsuhan Chen. “Efficient Inference for Fully Connected CRFs with Stationary” , CVPR'12

*Min Sun, Murali Telaprolu, Honglak Lee, and Silvio Savarese. "An Efficient Branch-and-Bound Algorithm for Optimal Human Pose Estimation", CVPR'12

The first two papers present methods to solve MAP inference on fully connected CRF approximately very efficiently using efficient high-dimensional filtering algorithms. Yimeng et al.'s method can even handle pair-wise potentials other than Gaussian kernel as long as they are stationary.

The last work is actually my own work (XD) which introduces an exact inference algorithm to solve xRF efficiently which has large number of states per node (e.g., object detection and human pose estimation). The work shares similar high-level idea with the first two papers, where efficient data structure and search algorithm are used to significantly speed-up the inference.

In general, vision researches nowadays really need to be familiar with tools and algorithms maybe introduced in other fields enough so that they can use them to solve specific types of interesting vision task.


More observation coming up....

The First Post

We are graduate students from Taiwan working in the general area of computer vision, graphics, learning and computational photography. We hope this blog could serve as a platform for sharing news, resources, interesting works and personal perspectives. 

While the original platform on Facebook, Computer Vision Taiwanese Group, has been a great place for sharing resources/news, it suffers from the deficiency of many functionalities such post archives, discussions, and tag annotations. 

Please feel free to leave comments or suggestions. Also, if you are interested in being a blogger, please us know.