Also note that while IT is promising to provide bounds in ML problems, search for an optimal yet practical solution under such bounds is probably out of the scope of IT which unfortunately a majority of ML research is devoted to as well. Even with such disability of IT application to ML, IT provides an information-theoretic interpretation of ML problems, besides the limits abound and achievablity mentioned in the first paragraph, which in turn help redesign the learning and inference process in ML problems (e.g., [2]).
References
[1] T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, 1991.
[2] T.T. Nguyen, and J. Choi. Layer-wise Learning of Stochastic Neural Networks with Information Bottleneck. In arXiv 1712.01272,
[2] T.T. Nguyen, and J. Choi. Layer-wise Learning of Stochastic Neural Networks with Information Bottleneck. In arXiv 1712.01272,
No comments:
Post a Comment