TabTreeFormer: Tabular Data Generation Using Hybrid Tree-Transformer

Jiayu Li, Bingyin Zhao, Zilong Zhao, Uzair Javaid, Biplab Sikdar
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:3637-3645, 2026.

Abstract

Transformers have shown impressive results in tabular data generation. However, they lack domain-specific inductive biases which are critical for preserving the intrinsic characteristics of tabular data. They also suffer from poor scalability and efficiency due to quadratic computational complexity. In this paper, we propose TabTreeFormer, a hybrid transformer architecture that integrates inductive biases of tree-based models (e.g., non-smoothness and non-rotational invariance) to effectively handle the discrete and weakly correlated features in tabular datasets. To improve numerical fidelity and capture multimodal distributions, we introduce a novel tokenizer that learns token sequences based on the complexity of tabular values. This reduces vocabulary size and sequence length, yielding more compact and efficient representations without sacrificing performance. We evaluate TabTreeFormer on nine diverse datasets, benchmarking against eight generative models. We show that TabTreeFormer consistently outperforms baselines in utility, fidelity, and privacy metrics with competitive efficiency. Notably, in scenarios prioritizing data utility over privacy and efficiency, the best variant of TabTreeFormer delivers a 44% performance gain relative to its baseline variant. Our code is available at: \url{https://github.com/li-jiayu-ljy/tabtreeformer.}

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-li26h, title = { TabTreeFormer: Tabular Data Generation Using Hybrid Tree-Transformer }, author = {Li, Jiayu and Zhao, Bingyin and Zhao, Zilong and Javaid, Uzair and Sikdar, Biplab}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {3637--3645}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/li26h/li26h.pdf}, url = {https://proceedings.mlr.press/v300/li26h.html}, abstract = { Transformers have shown impressive results in tabular data generation. However, they lack domain-specific inductive biases which are critical for preserving the intrinsic characteristics of tabular data. They also suffer from poor scalability and efficiency due to quadratic computational complexity. In this paper, we propose TabTreeFormer, a hybrid transformer architecture that integrates inductive biases of tree-based models (e.g., non-smoothness and non-rotational invariance) to effectively handle the discrete and weakly correlated features in tabular datasets. To improve numerical fidelity and capture multimodal distributions, we introduce a novel tokenizer that learns token sequences based on the complexity of tabular values. This reduces vocabulary size and sequence length, yielding more compact and efficient representations without sacrificing performance. We evaluate TabTreeFormer on nine diverse datasets, benchmarking against eight generative models. We show that TabTreeFormer consistently outperforms baselines in utility, fidelity, and privacy metrics with competitive efficiency. Notably, in scenarios prioritizing data utility over privacy and efficiency, the best variant of TabTreeFormer delivers a 44% performance gain relative to its baseline variant. Our code is available at: \url{https://github.com/li-jiayu-ljy/tabtreeformer.} } }
Endnote
%0 Conference Paper %T TabTreeFormer: Tabular Data Generation Using Hybrid Tree-Transformer %A Jiayu Li %A Bingyin Zhao %A Zilong Zhao %A Uzair Javaid %A Biplab Sikdar %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-li26h %I PMLR %P 3637--3645 %U https://proceedings.mlr.press/v300/li26h.html %V 300 %X Transformers have shown impressive results in tabular data generation. However, they lack domain-specific inductive biases which are critical for preserving the intrinsic characteristics of tabular data. They also suffer from poor scalability and efficiency due to quadratic computational complexity. In this paper, we propose TabTreeFormer, a hybrid transformer architecture that integrates inductive biases of tree-based models (e.g., non-smoothness and non-rotational invariance) to effectively handle the discrete and weakly correlated features in tabular datasets. To improve numerical fidelity and capture multimodal distributions, we introduce a novel tokenizer that learns token sequences based on the complexity of tabular values. This reduces vocabulary size and sequence length, yielding more compact and efficient representations without sacrificing performance. We evaluate TabTreeFormer on nine diverse datasets, benchmarking against eight generative models. We show that TabTreeFormer consistently outperforms baselines in utility, fidelity, and privacy metrics with competitive efficiency. Notably, in scenarios prioritizing data utility over privacy and efficiency, the best variant of TabTreeFormer delivers a 44% performance gain relative to its baseline variant. Our code is available at: \url{https://github.com/li-jiayu-ljy/tabtreeformer.}
APA
Li, J., Zhao, B., Zhao, Z., Javaid, U. & Sikdar, B.. (2026). TabTreeFormer: Tabular Data Generation Using Hybrid Tree-Transformer . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:3637-3645 Available from https://proceedings.mlr.press/v300/li26h.html.

Related Material