Speaker Diarization as a Fully Online Bandit Learning Problem in MiniVox

Baihan Lin, Xinxin Zhang
Proceedings of The 13th Asian Conference on Machine Learning, PMLR 157:1660-1674, 2021.

Abstract

We propose a novel machine learning framework to conduct real-time multi-speaker diarization and recognition without prior registration and pretraining in a fully online learning setting. Our contributions are two-fold. First, we propose a new benchmark to evaluate the rarely studied fully online speaker diarization problem. We build upon existing datasets of real world utterances to automatically curate MiniVox, an experimental environment which generates infinite configurations of continuous multi-speaker speech stream. Second, we consider the practical problem of online learning with episodically revealed rewards and introduce a solution based on semi-supervised and self-supervised learning methods. Additionally, we provide a workable web-based recognition system which interactively handles the cold start problem of new user’s addition by transferring representations of old arms to new ones with an extendable contextual bandit. We demonstrate that our proposed method obtains robust performance in the online MiniVox framework given either cepstrum-based representations or deep neural network embeddings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v157-lin21c, title = {Speaker Diarization as a Fully Online Bandit Learning Problem in MiniVox}, author = {Lin, Baihan and Zhang, Xinxin}, booktitle = {Proceedings of The 13th Asian Conference on Machine Learning}, pages = {1660--1674}, year = {2021}, editor = {Balasubramanian, Vineeth N. and Tsang, Ivor}, volume = {157}, series = {Proceedings of Machine Learning Research}, month = {17--19 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v157/lin21c/lin21c.pdf}, url = {https://proceedings.mlr.press/v157/lin21c.html}, abstract = {We propose a novel machine learning framework to conduct real-time multi-speaker diarization and recognition without prior registration and pretraining in a fully online learning setting. Our contributions are two-fold. First, we propose a new benchmark to evaluate the rarely studied fully online speaker diarization problem. We build upon existing datasets of real world utterances to automatically curate MiniVox, an experimental environment which generates infinite configurations of continuous multi-speaker speech stream. Second, we consider the practical problem of online learning with episodically revealed rewards and introduce a solution based on semi-supervised and self-supervised learning methods. Additionally, we provide a workable web-based recognition system which interactively handles the cold start problem of new user’s addition by transferring representations of old arms to new ones with an extendable contextual bandit. We demonstrate that our proposed method obtains robust performance in the online MiniVox framework given either cepstrum-based representations or deep neural network embeddings.} }
Endnote
%0 Conference Paper %T Speaker Diarization as a Fully Online Bandit Learning Problem in MiniVox %A Baihan Lin %A Xinxin Zhang %B Proceedings of The 13th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Vineeth N. Balasubramanian %E Ivor Tsang %F pmlr-v157-lin21c %I PMLR %P 1660--1674 %U https://proceedings.mlr.press/v157/lin21c.html %V 157 %X We propose a novel machine learning framework to conduct real-time multi-speaker diarization and recognition without prior registration and pretraining in a fully online learning setting. Our contributions are two-fold. First, we propose a new benchmark to evaluate the rarely studied fully online speaker diarization problem. We build upon existing datasets of real world utterances to automatically curate MiniVox, an experimental environment which generates infinite configurations of continuous multi-speaker speech stream. Second, we consider the practical problem of online learning with episodically revealed rewards and introduce a solution based on semi-supervised and self-supervised learning methods. Additionally, we provide a workable web-based recognition system which interactively handles the cold start problem of new user’s addition by transferring representations of old arms to new ones with an extendable contextual bandit. We demonstrate that our proposed method obtains robust performance in the online MiniVox framework given either cepstrum-based representations or deep neural network embeddings.
APA
Lin, B. & Zhang, X.. (2021). Speaker Diarization as a Fully Online Bandit Learning Problem in MiniVox. Proceedings of The 13th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 157:1660-1674 Available from https://proceedings.mlr.press/v157/lin21c.html.

Related Material