DIFFERENTIAL LEARNING AND ICA

Differential learning relies on the differentiated values of nodes, whereas the conventional learning depends on the values themselves of nodes. In this paper, I elucidate the differential learning in the framework maximum likelihood learning of linear generative model with latent variables obeying random walk. I apply the idea of differential learning to the problem independent component analysis(ICA), which leads to differential ICA. Algorithm derivation using the natural gradient and local stability analysis are provided. Usefulness of the algorithm is emphasized in the case of blind separation of temporally correlated sources and is demonstrated through a simple numerical example.