Holistic Representations for Memorization and Inference

Yunpu Ma, Marcel Hildebrandt, Volker Tresp, Stephan Baier
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:402-412, 2018.

Abstract

In this paper we introduce a novel holographic memory model for the distributed storage of complex association patterns and apply it to knowledge graphs. In a knowledge graph, a la- belled link connects a subject node with an ob- ject node, jointly forming a subject-predicate- objects triple. In the presented work, nodes and links have initial random representations, plus holistic representations derived from the initial representations of nodes and links in their local neighbourhoods. A memory trace is represented in the same vector space as the holistic representations themselves. To reduce the interference between stored information, it is required that the initial random vectors should be pairwise quasi-orthogonal. We show that pairwise quasi-orthogonality can be im- proved by drawing vectors from heavy-tailed distributions, e.g., a Cauchy distribution, and, thus, memory capacity of holistic representa- tions can significantly be improved. Further- more, we show that, in combination with a simple neural network, the presented holistic representation approach is superior to other methods for link predictions on knowledge graphs.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-ma18a, title = {Holistic Representations for Memorization and Inference}, author = {Ma, Yunpu and Hildebrandt, Marcel and Tresp, Volker and Baier, Stephan}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {402--412}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/ma18a/ma18a.pdf}, url = {https://proceedings.mlr.press/r16/ma18a.html}, abstract = {In this paper we introduce a novel holographic memory model for the distributed storage of complex association patterns and apply it to knowledge graphs. In a knowledge graph, a la- belled link connects a subject node with an ob- ject node, jointly forming a subject-predicate- objects triple. In the presented work, nodes and links have initial random representations, plus holistic representations derived from the initial representations of nodes and links in their local neighbourhoods. A memory trace is represented in the same vector space as the holistic representations themselves. To reduce the interference between stored information, it is required that the initial random vectors should be pairwise quasi-orthogonal. We show that pairwise quasi-orthogonality can be im- proved by drawing vectors from heavy-tailed distributions, e.g., a Cauchy distribution, and, thus, memory capacity of holistic representa- tions can significantly be improved. Further- more, we show that, in combination with a simple neural network, the presented holistic representation approach is superior to other methods for link predictions on knowledge graphs.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Holistic Representations for Memorization and Inference %A Yunpu Ma %A Marcel Hildebrandt %A Volker Tresp %A Stephan Baier %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-ma18a %I PMLR %P 402--412 %U https://proceedings.mlr.press/r16/ma18a.html %V R16 %X In this paper we introduce a novel holographic memory model for the distributed storage of complex association patterns and apply it to knowledge graphs. In a knowledge graph, a la- belled link connects a subject node with an ob- ject node, jointly forming a subject-predicate- objects triple. In the presented work, nodes and links have initial random representations, plus holistic representations derived from the initial representations of nodes and links in their local neighbourhoods. A memory trace is represented in the same vector space as the holistic representations themselves. To reduce the interference between stored information, it is required that the initial random vectors should be pairwise quasi-orthogonal. We show that pairwise quasi-orthogonality can be im- proved by drawing vectors from heavy-tailed distributions, e.g., a Cauchy distribution, and, thus, memory capacity of holistic representa- tions can significantly be improved. Further- more, we show that, in combination with a simple neural network, the presented holistic representation approach is superior to other methods for link predictions on knowledge graphs. %Z Reissued by PMLR on 04 October 2026.
APA
Ma, Y., Hildebrandt, M., Tresp, V. & Baier, S.. (2018). Holistic Representations for Memorization and Inference. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:402-412 Available from https://proceedings.mlr.press/r16/ma18a.html. Reissued by PMLR on 04 October 2026.

Related Material