Convexified Convolutional Neural Networks
[edit]
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:40444053, 2017.
Abstract
We describe the class of convexified convolutional neural networks (CCNNs), which capture the parameter sharing of convolutional neural networks in a convex manner. By representing the nonlinear convolutional filters as vectors in a reproducing kernel Hilbert space, the CNN parameters can be represented as a lowrank matrix, which can be relaxed to obtain a convex optimization problem. For learning twolayer convolutional neural networks, we prove that the generalization error obtained by a convexified CNN converges to that of the best possible CNN. For learning deeper networks, we train CCNNs in a layerwise manner. Empirically, CCNNs achieve competitive or better performance than CNNs trained by backpropagation, SVMs, fullyconnected neural networks, stacked denoising autoencoders, and other baseline methods.
Related Material


