Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecture

Sangjun Park, Jinyeong Bak
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:39587-39615, 2024.

Abstract

Making neural networks remember over the long term has been a longstanding issue. Although several external memory techniques have been introduced, most focus on retaining recent information in the short term. Regardless of its importance, information tends to be fatefully forgotten over time. We present Memoria, a memory system for artificial neural networks, drawing inspiration from humans and applying various neuroscientific and psychological theories. The experimental results prove the effectiveness of Memoria in the diverse tasks of sorting, language modeling, and classification, surpassing conventional techniques. Engram analysis reveals that Memoria exhibits the primacy, recency, and temporal contiguity effects which are characteristics of human memory.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-park24a, title = {Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecture}, author = {Park, Sangjun and Bak, Jinyeong}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {39587--39615}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/park24a/park24a.pdf}, url = {https://proceedings.mlr.press/v235/park24a.html}, abstract = {Making neural networks remember over the long term has been a longstanding issue. Although several external memory techniques have been introduced, most focus on retaining recent information in the short term. Regardless of its importance, information tends to be fatefully forgotten over time. We present Memoria, a memory system for artificial neural networks, drawing inspiration from humans and applying various neuroscientific and psychological theories. The experimental results prove the effectiveness of Memoria in the diverse tasks of sorting, language modeling, and classification, surpassing conventional techniques. Engram analysis reveals that Memoria exhibits the primacy, recency, and temporal contiguity effects which are characteristics of human memory.} }
Endnote
%0 Conference Paper %T Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecture %A Sangjun Park %A Jinyeong Bak %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-park24a %I PMLR %P 39587--39615 %U https://proceedings.mlr.press/v235/park24a.html %V 235 %X Making neural networks remember over the long term has been a longstanding issue. Although several external memory techniques have been introduced, most focus on retaining recent information in the short term. Regardless of its importance, information tends to be fatefully forgotten over time. We present Memoria, a memory system for artificial neural networks, drawing inspiration from humans and applying various neuroscientific and psychological theories. The experimental results prove the effectiveness of Memoria in the diverse tasks of sorting, language modeling, and classification, surpassing conventional techniques. Engram analysis reveals that Memoria exhibits the primacy, recency, and temporal contiguity effects which are characteristics of human memory.
APA
Park, S. & Bak, J.. (2024). Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecture. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:39587-39615 Available from https://proceedings.mlr.press/v235/park24a.html.

Related Material