Abstract
The Dirichlet process is one of the most widely used priors in Bayesian clustering. This process allows for a nonparametric estimation of the number of clusters when partitioning datasets. The “rich-get-richer” property is a key feature of this process, and transcribes that the a priori probability for a cluster to get selected dependent linearly on its population.
In this paper, we show that such hypothesis is not necessarily optimal. We derive the Powered Dirichlet Process as a generalization of the Dirichlet-Multinomial distribution as an answer to this problem. We then derive some of its fundamental properties (expected number of clusters, convergence). Unlike state-of-the-art efforts in this direction, this new formulation allows for direct control of the importance of the “rich-get-richer” prior. We confront our proposition to several simulated and real-world datasets, and confirm that our formulation allows for significantly better results in both cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Codes and datasets available at https://github.com/GaelPouxMedard/PDPs.
- 2.
References
Airoldi, E., Blei, D., Fienberg, S., Xing, E.: Mixed membership stochastic blockmodels. J. Mach. Learn. Res. 9, 1991–1992 (2008)
Arratia, R., Barbour, A.D., Tavaré, S.: Poisson process approximations for the Ewens sampling formula. Ann. Appl. Probab. 2(3), 519–535 (1992)
Betancourt, B., Zanella, G., Miller, J.W., Wallach, H., Zaidi, A., Steorts, R.C.: Flexible models for microclustering with application to entity resolution, vol. 29 (2016). https://proceedings.neurips.cc/paper/2016/file/670e8a43b246801ca1eaca97b3e19189-Paper.pdf
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Blei, D., Frazier, P.: Distance dependent Chinese restaurant processes. J. Mach. Learn. Res. 12, 2461–2488 (2011)
Cobo-López, S., Godoy-Lorite A., Duch, J.: Optimal prediction of decisions and model selection in social dilemmas using block models. EPJ Data Sci. 7(48) (2018)
Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. Ann. Stat. 1(2), 209–230 (1973)
Frigyik, A.B., Kapila, A., Gupta, M.R.: Introduction to the Dirichlet distribution and related processes (2010)
Ghosh, S., Raptis, M., Sigal, L., Sudderth, E.B.: Nonparametric clustering with distance dependent hierarchies. In: UAI 2014, pp. 260–269 (2014)
Godoy-Lorite, A., Guimerà, R., Moore, C., Sales-Pardo, M.: Accurate and scalable social recommendation using mixed-membership stochastic block models. PNAS 113(50), 14207–14212 (2016)
Goldwater, S., Griffiths, T.L., Johnson, M.: Producing power-law distributions and damping word frequencies with two-stage language models. JMLR 12(68) (2011)
Guimera, R., Llorente, A., Sales-Pardo, M.: Predicting human preferences using the block structure of complex social networks. PLOS One 7(9) (2012)
Guimerá, R., Sales-Pardo, M.: A network inference method for large-scale unsupervised identification of novel drug-drug interactions. PLoS Comput. Biol. (2013)
Hanson, J.W., Ortman, S.G., Lobo, J.: Urbanism and the division of labour in the roman empire. J. R. Soc. Interface 14(136), 20170367 (2017)
Ishwaran, H., James, L.: Generalized weighted Chinese restaurant processes for species sampling mixture models. Statistica Sinica 13, 1211–1235 (2003)
Jensen, S., Liu, J.: Bayesian clustering of transcription factor binding motifs. J. Am. Stat. Assoc. 103, 188–200 (2008)
Jordan, M.: Dirchlet processes, Chinese restaurant processes and all that. In: ICML (2005)
Lee, C.J., Sang, H.: Why the rich get richer? On the balancedness of random partition models. In: Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., Sabato, S. (eds.) Proceedings of the 39th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 162, pp. 12521–12541. PMLR, 17–23 July 2022
Lijoi, A., Mena, R.H., Prünster, I.: Controlling the reinforcement in Bayesian non-parametric mixture models. J. Roy. Stat. Soc. Ser. B (Stat. Methodol.) 69(4), 715–740 (2007). https://doi.org/10.1111/j.1467-9868.2007.00609.x. https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9868.2007.00609.x
McDowell, I.C., Manandhar, D., Vockley, C.M., Schmid, A.K., Reddy, T.E., Engelhardt, B.E.: Clustering gene expression time series data using an infinite Gaussian process mixture model. PLoS Comput. Biol. 14(1), e1005896 (2018)
Pitman, J., Yor, M.: The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Ann. Probab. 25(2), 855–900 (1997)
Poux-Médard, G., Velcin, J., Loudcher, S.: Interactions in information spread: quantification and interpretation using stochastic block models. arXiv (2020)
Qin, Z.S., McCue, L.A., Thompson, W., Mayerhofer, L., Lawrence, C.E., Liu, J.S.: Identification of co-regulated genes through Bayesian clustering of predicted regulatory binding sites. Nat. Biotechnol. 21, 435–439 (2003)
Rasmussen, C.E.: The infinite gaussian mixture model. In: NIPS 1999, pp. 554–560. MIT Press (1999)
Socher, R., Maas, A., Manning, C.: Spectral Chinese restaurant processes: nonparametric clustering based on similarities. In: JMLR - Proceedings, vol. 15, pp. 698–706 (2011)
Steorts, R.C.: Entity resolution with empirically motivated priors 10, 849 (2015)
Steorts, R.C., Hall, R., Fienberg, S.E.: SMERED: a Bayesian approach to graphical record linkage and de-duplication, vol. 33, pp. 922–930 (2014)
Sudderth, E., Jordan, M.: Shared segmentation of natural scenes using dependent Pitman-Yor processes. In: NIPS, vol. 21 (2009)
Teh, Y., Gorur, D.: Indian buffet processes with power-law behavior, vol. 22 (2009)
Wallach, H., Jensen, S., Dicker, L., Heller, K.: An alternative prior process for nonparametric Bayesian clustering. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 892–899. JMLR (2010)
Welling, M.: Flexible priors for infinite mixture models. In: Workshop on Learning with Non-parametric Bayesian Methods (2006)
Xu, W., Li, Y., Qiang, J.: Dynamic clustering for short text stream based on Dirichlet process. Appl. Intell. (2021). https://doi.org/10.1007/s10489-021-02263-z
Yin, J., Wang, J.: A dirichlet multinomial mixture model-based approach for short text clustering. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2014, pp. 233–242. Association for Computing Machinery, New York, NY, USA (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Poux-Médard, G., Velcin, J., Loudcher, S. (2023). Powered Dirichlet Process - Controlling the “Rich-Get-Richer” Assumption in Bayesian Clustering. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14169. Springer, Cham. https://doi.org/10.1007/978-3-031-43412-9_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-43412-9_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43411-2
Online ISBN: 978-3-031-43412-9
eBook Packages: Computer ScienceComputer Science (R0)
