Dynamic Weight Distribution Method of Loss Function Based on Category Theory

Abstract

Imbalanced datasets are prevalent in real life, which have a skewed data distribution. In the research of imbalanced datasets, the inherent patterns of minority class affect the generalization performance of Machine Learning models. The re-weighting/re-sampling methods are used to address the issue of imbalanced datasets. However, the above methods are formulated based on empirical and heuristic rules. A theoretical framework is required in the presentation of the re-weighting/re-sampling methods. Category Theory is used as a framework to analyze the training process of Machine Learning models on imbalanced data. Based on the analysis, a method named Dynamic Weight Adjustment (DWA) is proposed to improve the generalization performance of models on imbalanced data. The imbalanced datasets for binary classification and multi-classification problems are used to verify the effectiveness of the DWA. The results of the DWA are the best in both imbalanced datasets for binary classification and multi-classification problems.