Mairal et al. Online dictionary learning for sparse coding. ICML 2009.
The following figures are all from this paper.
The paper goal is to propose a online learning method to learn sparse coding dictionary, the basis set which can represent specific data through linear combination, for large scale data.
Algorithm
The algorithm is showed above. Assuming the training set composed of i.i.d. samples of a distribution, the method is minimizing the cost function, the difference between linear combination of basis set and real data, by stochastic gradient discent to find decomposing coefficient $latex \alpha $ and dictionary $latex D$ iteratively.
Their algorithm above for updating the dictionary uses block-coordinate descent.
My comment
The work is solid because they present not only the experiment result but also the sophisticated theoretical proof showing the convergence of the online learning method; however, the assumption, the training set composed of i.i.d. samples of a distribution, may be suspicious because it is not alway satisfied in real world.
Go Top