Abstract
To resolve conflicts in multi-source data, truth discovery has been widely studied. However, existing methods ignore the uneven distribution of time-series data that evolves. The coverage of objects by sources may exhibit a long-tail phenomenon. Additionally, the distribution of objects across sources is often uneven, with some hot objects being claimed by multiple sources, while only a few sources claim other cold objects. Due to insufficient data support, the reliability evaluation of small sources and the truth estimation of cold objects are often unreasonable. Furthermore, unpredictably, the roles of big and small sources may be interchanged over time, and objects classified as hot and cold may switch places. Dynamic changes in data distribution further increase the complexity of estimation. To tackle this problem, we propose a novel dynamic truth discovery method(TDCITP). Based on data fluctuation, TDCITP adaptively chooses the stability method or the fluctuation method for truth estimation and source weight estimation. Confidence interval estimation is conducted for both source weight and truth estimation to distinguish small sources and cold objects. Experimental results demonstrate that our method outperforms state-of-the-art methods in terms of accuracy and efficiency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
D.P.Bertsekas: Non-linear programming. Athena Scientific, 2 edition (1999)
Fan, D., Sun, H., Yao, J., Zhang, K., Yan, X., Sun, Z.: Well production forecasting based on arima-lstm model considering manual operations. Energy 220, 119708 (2021)
Fang, X.S., Sheng, Q.Z., Wang, X., Chu, D., Ngu, A.H.: Smartvote: a full-fledged graph-based model for multi-valued truth discovery. WWW 22(4), 1855–1885 (2019)
Li, Q., Li, Y., Gao, J., Su, L., Zhao, B., Demirbas, M., Fan, W., Han, J.: A confidence-aware approach for truth discovery on long-tail data. VLDB 8(4), 425–436 (2014)
Li, Q., Li, Y., Gao, J., Zhao, B., Fan, W., Han, J.: Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation. In: SIGMOD. pp. 1187–1198 (2014)
Li, T., Gu, Y., Zhou, X., Ma, Q., Yu, G.: An effective and efficient truth discovery framework over data streams. In: EDBT. pp. 180–191. Springer (2017)
Li, X., Dong, X.L., Lyons, K., Meng, W., Srivastava, D.: Truth finding on the deep web: Is the problem solved? PVLDB (2015)
Li, Y., Gao, J., Meng, C., Li, Q., Su, L., Zhao, B., Fan, W., Han, J.: A survey on truth discovery. SIGKDD 17(2), 1–16 (2016)
Li, Y., Li, Q., Gao, J., Su, L., Zhao, B., Fan, W., Han, J.: On the discovery of evolving truth. In: SIGKDD. pp. 675–684 (2015)
Meir, R., Amir, O., Ben-Porat, O., Shabat, T.B., Cohensius, G., Xia, L.: Frustratingly easy truth discovery. In: AAAI. vol. 37, pp. 6074–6083 (2023)
Xiao, H., Gao, J., Li, Q., Ma, F., Su, L., Feng, Y., Zhang, A.: Towards confidence in the truth: A bootstrapping based truth discovery approach. In: SIGKDD. pp. 1935–1944 (2016)
Xiao, H., Gao, J., Li, Q., Ma, F., Su, L., Feng, Y., Zhang, A.: Towards confidence interval estimation in truth discovery. TKDE 31(3), 575–588 (2018)
Yang, J., Tay, W.P.: An unsupervised bayesian neural network for truth discovery in social networks. TKDE 34(11), 5182–5195 (2022)
Yang, Y., Bai, Q., Liu, Q.: Dynamic source weight computation for truth inference over data streams. In: AAMAS. pp. 277–285 (2019)
Yao, L., Su, L., Li, Q., Li, Y., Ma, F., Gao, J., Zhang, A.: Online truth discovery on time series data. In: SDM. pp. 162–170. SIAM (2018)
Yin, X., Han, J., Yu, P.S.: Truth discovery with multiple conflicting information providers on the web. In: SIGKDD. pp. 1048–1052 (2007)
Yu, X., Shao, Z., Dong, K., Chen, C.: A novel truth discovery approach for health recommendation systems. TCE (2023)
Zhang, D.Y., Badilla, J., Zhang, Y., Wang, D.: Towards reliable missing truth discovery in online social media sensing applications. In: ASONAM. pp. 143–150. IEEE (2018)
Acknowledgement
This work was supported by Shanghai Science and Tech- nology Commission (No. 22YF1401100), Fundamental Research Funds for the Central Universities (No. 22D111210), and National Science Fund for Young Scholars (No. 62202095, No. 62102058).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Fang, X., Chen, H., Du, X., Wei, Z., Zheng, Y., Sun, G. (2024). Dynamic Confidence-aware Truth Discovery on Unevenly Distributed Data Streams. In: Onizuka, M., et al. Database Systems for Advanced Applications. DASFAA 2024. Lecture Notes in Computer Science, vol 14854. Springer, Singapore. https://doi.org/10.1007/978-981-97-5569-1_35
Download citation
DOI: https://doi.org/10.1007/978-981-97-5569-1_35
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5568-4
Online ISBN: 978-981-97-5569-1
eBook Packages: Computer ScienceComputer Science (R0)