Skip to main content
Springer Nature Link
Log in

Conductive particle detection via deep learning for ACF bonding in TFT-LCD manufacturing

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

The inspection of conductive particles after Anisotropic Conductive Film (ACF) bonding is a common and crucial step in the TFT-LCD manufacturing process since the number of high-quality conductive particles is a key indicator of ACF bonding quality. However, manual inspection under microscope is a time-consuming, tedious, and error-prone. Therefore, there is an urgent demand in industry for the automatic conductive particle inspection system. It is challenging for automatic conductive particle quality inspection due to the existence of complex background noise and diversified particle appearance, including shape, size, clustering and overlapping, etc. As a result, it lacks an effective automatic detection method to handle all the complex particle patterns. In this paper, we propose a U-shaped deep residual neural network (i.e., U-ResNet), which can learn features of particle from massively labeled data. The experimental results show that the proposed method achieves high detection accuracy and recall rate, which exceedingly outperforms the previous work. Also, our system is very efficient and can work in real time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828.

    Article  Google Scholar 

  • Cao, K., & Jain, A. K. (2019). Automated latent fingerprint recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(4), 788–800.

    Article  Google Scholar 

  • Drozdzal, M., Vorontsov, E., Chartrand, G., Kadoury, S., & Pal, C. (2016). The importance of skip connections in biomedical image segmentation. Deep learning and data labeling for medical applications (pp. 179–187). Cham: Springer.

    Chapter  Google Scholar 

  • Ferguson, M., Ak, R., Lee, Y. T. T., & Law, K. H. (2017). Automatic localization of casting defects with convolutional neural networks. In 2017 IEEE international conference on big data (big data) (pp. 1726–1735). IEEE.

  • Gonzalez, R. C., & Woo, R. E. (2017). Digital image processing (4th ed.). London: Pearson.

    Google Scholar 

  • Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. Cambridge: MIT Press.

    Google Scholar 

  • Graves, A., Mohamed, A. R., & Hinton, G. (2013). Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing. IEEE. arXiv:1303.5778.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2015a). Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2015b). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision (pp. 1026–1034).

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).

  • Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504–507.

    Article  Google Scholar 

  • Jia, L., Sheng, X., Xiong, Z., Wang, Z., & Ding, H. (2014). Particle on bump (POB) technique for ultra-fine pitch chip on glass (COG) applications by conductive particles and adhesives. Microelectronics Reliability, 54(4), 825–832.

    Article  Google Scholar 

  • Kingma, D. P., & Ba, J. L. (2015). Adam: A method for stochastic optimization. In International conference on learning representations (pp. 1–13).

  • Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, & K. Q. Weinberger (Eds.), Advances in neural information processing systems 25 (pp. 1097–1105). New York: Curran Associates.

    Google Scholar 

  • Kwon, O., Kim, H. G., Ham, M. J., Kim, W., Kim, G. H., Cho, J. H., Kim, N. I., & Kim, K. (2018). A deep neural network for classification of melt-pool images in metal additive manufacturing. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-018-1451-6.

  • LeCun, Y. (1988). A theoretical framework for back-propagation. In D. Touretzky, G. Hinton, and T. Sejnowski (Eds.), Proceedings of the 1988 connectionist models summer school, pages 21–28, CMU, Pittsburgh, Pa, 1988. Morgan Kaufmann.

  • LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.

    Article  Google Scholar 

  • Lin, C. S., Huang, K. H., Lin, T. C., Shei, H. J., & Tien, Chuen-Lin. (2011). An automatic inspection method for the fracture conditions of anisotropic conductive film in the tft-lcd assembly process. International Journal of Optomechatronics, 5(3), 286–298.

    Article  Google Scholar 

  • Lin, H., Li, B., Wang, X., Shu, Y., & Niu, S. (2018). Automated defect inspection of led chip using deep convolutional neural network. Journal of Intelligent Manufacturing, 30(6), 2525–2534.

    Article  Google Scholar 

  • Long, J., Shelhamer, E., & Darrell, T. (2015a). Fully convolutional networks for semantic segmentation. In 2015 IEEE conference on computer vision and pattern recognition (CVPR), (pp. 3431–3440).

  • Long, J., Shelhamer, E., & Darrell, T. (2015b). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431–3440).

  • Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781.

  • Ni, G., Liu, L., Xiaohui, D., Zhang, J., Liu, J., & Liu, Y. (2017). Accurate aoi inspection of resistance in lcd anisotropic conductive film bonding using differential interference contrast. Optik-International Journal for Light and Electron Optics, 130, 786–796.

    Article  Google Scholar 

  • Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. . In N. Navab, J. Hornegger, W. M. Wells, & A. F. Frangi (Eds.), International conference on medical image computing and computer-assisted intervention (pp. 234–241). Berlin: Springer.

  • Russakovsky, O., Deng, J., Hao, S., Krause, J., Satheesh, S., & Ma, S. (2015). Imagenet large scale visual recognition challenge. International Journal of Computer Vision (IJCV), 115(3), 211–252.

    Article  Google Scholar 

  • Sezgin, M., & Sankur, B. (2004). Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic imaging, 13(1), 146–166.

    Article  Google Scholar 

  • Sheng, X., Jia, L., Xiong, Z., Wang, Z., & Ding, H. (2013). Acf-cog interconnection conductivity inspection system using conductive area. Microelectronics Reliability, 53(4), 622–628.

    Article  Google Scholar 

  • Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In 2015 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1–9).

  • Tabernik, D., Šela, S., Skvarč, J., & Skočaj, D. (2019). Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-019-01476-x.

  • Tran, N., Zhang, X., Xin, L., Shan, B., & Li, M. (2017). De novo peptide sequencing by deep learning. Proceedings of the National Academy of Sciences, 114(31), 8247–8252.

    Article  Google Scholar 

  • Wang, T., Qiao, M., Zhang, M., Yang, Y., & Snoussi, H. (2018). Data-driven prognostic method based on self-supervised learning approaches for fault detection. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-018-1431-x

  • Wirges, S., Hartenbach, F., & Stiller, C. (2018). Evidential occupancy grid map augmentation using deep learning. In 2018 IEEE intelligent vehicles symposium (IV) (pp. 668–673). arXiv:1801.05297.

  • Yen, Y. W., & Lee, C. Y. (2011). ACF particle distribution in COG process. Microelectronics Reliability, 51(3), 676–684.

    Article  Google Scholar 

  • Yu-ye, C., Ke, X., Zhen-xiong, G., Jun-jie, H., Chang, L., & Song-yan, C. (2017). Detection of conducting particles bonding in the circuit of liquid crystal display. Chinese Journal of Liquid Crystals and Displays, 32(7), 553–559.

    Article  Google Scholar 

  • Zan, T., Liu, Z., Wang, H., Wang, M., & Gao, X. (2019). Control chart pattern recognition using the convolutional neural network. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-019-01473-0

  • Zhang, Z., Liu, Q., & Wang, Y. (2018). Road extraction by deep residual u-net. IEEE Geoscience and Remote Sensing Letters, 15(5), 749–753.

    Article  Google Scholar 

  • Zhang, Z., Luo, P., Loy, C. C., & Tang, X. (2016). Learning deep representation for face alignment with auxiliary attributes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(5), 918–930.

    Article  Google Scholar 

  • Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017). Pyramid scene parsing network. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2881–2890).

  • Zhao, Z., Li, Y., Liu, C., & Gao, J. (2019). On-line part deformation prediction based on deep learning. Journal of Intelligent Manufacturing (pp. 1–14).

Download references

Acknowledgements

Eryun Liu and Zhiyu Xiang’s research were supported by the National Natural Science Foundation of China under Grant No. U1709214 and by the Fundamental Research Funds for the Central Universities.

Author information

Authors and Affiliations

  1. Zhejiang Provincial Key Laboratory of Information Processing, Communication and Networking, College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou, 310027, Zhejiang, China

    Eryun Liu, Kangping Chen & Zhiyu Xiang

  2. Tencent AI Healthcare, Shenzhen, China

    Jun Zhang

Authors
  1. Eryun Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Kangping Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Zhiyu Xiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jun Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Eryun Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, E., Chen, K., Xiang, Z. et al. Conductive particle detection via deep learning for ACF bonding in TFT-LCD manufacturing. J Intell Manuf 31, 1037–1049 (2020). https://doi.org/10.1007/s10845-019-01494-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-019-01494-9

Keywords