Nonlinear Weighted Finite Automata
[edit]
Proceedings of the TwentyFirst International Conference on Artificial Intelligence and Statistics, PMLR 84:679688, 2018.
Abstract
Weighted finite automata (WFA) can expressively model functions defined over strings but are inherently linear models.Given the recent successes of nonlinear models in machine learning, it is natural to wonder whether extending WFA to the nonlinearsetting would be beneficial.In this paper, we propose a novel model of neural network based nonlinear WFA model (NLWFA) along with a learning algorithm. Our learning algorithm is inspired by the spectral learning algorithm for WFA and relies on a nonlinear decomposition of the socalled Hankel matrix, by means of an autoencoder network. The expressive power of NLWFA and the proposed learning algorithm are assessed on both synthetic and real world data, showing that NLWFA can infer complex grammatical structures from data.
Related Material


