1. Home
  2. Issues
  3. Volume 37, Issue 1 (2026)
  4. A Case Study of Artificial Neural Networ ...

Informatica


A Case Study of Artificial Neural Network Compression Methods for Resource-Constrained Multi-Label Classification
Volume 37, Issue 1 (2026), pp. 87–107
Pub. online: 11 February 2026      Type: Research Article      Open accessOpen Access
Received
1 July 2025
Accepted
1 February 2026
Published
11 February 2026

Abstract

Proliferation of wearable healthcare devices has created the need to deliver artificial intelligence applications for these resource-constrained devices to achieve faster, localized decision-making, by bringing computation closer to the data sources, for improved responsiveness and privacy. This contribution presents the results of an experimental evaluation of artificial neural network compression techniques, including quantization, structured pruning, and knowledge distillation, applied to multi-label classification of electrocardiogram (ECG) signals. The experiments were carried out on the PTB-XL dataset using three deep learning models, i.e. an LSTM-based recurrent neural network, a 1D convolutional neural network, and a 1D residual neural network. The results show how the compression methods impact model quality and highlight opportunities to reduce model size and accelerate inference, thereby enabling effective deployment on resource-constrained, edge devices.

References

 
Berkaya, S.K., Uysal, A.K., Gunal, E.S., Ergin, S., Gunal, S., Gulmezoglu, M.B. (2018). A survey on ECG analysis. Biomedical Signal Processing and Control, 43, 216–235.
 
Boulif, A., Ananou, B., Ouladsine, M., Delliaux, S. (2024). Focal-based deep learning model for automatic arrhythmia diagnosis. In: International Conference on Computational Science. Springer, pp. 355–370.
 
Butterworth, S., (1930). On the theory of filter amplifiers. Wireless Engineer, 7(6), 536–541.
 
Chang, X.Q., Chew, A.F., Choong, B.C.M., Wang, S., Han, R., He, W., Xiaolin, L., Panicker, R.C., John, D. (2022). Atrial fibrillation detection using weight-pruned, log-quantised convolutional neural networks. In: 2022 IEEE 13th Latin America Symposium on Circuits and System (LASCAS). IEEE, pp. 1–4.
 
Cheng, H., Zhang, M., Shi, J.Q. (2024). A survey on deep neural network pruning: taxonomy, comparison, analysis, and recommendations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 10558–10578.
 
Dantas, P.V., Da Silva, W.S., Cordeiro, L.C., Carvalho, C.B. (2024). A comprehensive review of model compression techniques in machine learning. Applied Intelligence, 54(22), 11804–11844.
 
Dong, Z., Gao, Y., Huang, Q., Wawrzynek, J., So, H.K., Keutzer, K. (2021). Hao: Hardware-aware neural architecture optimization for efficient inference. In: 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, pp. 50–59.
 
Elman, J.L. (1990). Finding structure in time. Cognitive Science, 14(2), 179–211.
 
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874.
 
Fernández, A., García, S., Galar, M., Prati, R.C., Krawczyk, B., Herrera, F. (2018). Performance measures. In: Learning from Imbalanced Data Sets. Springer International Publishing, pp. 47–61.
 
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M.W., Keutzer, K. (2022). A survey of quantization methods for efficient neural network inference. In: Low-Power Computer Vision. Chapman and Hall/CRC, pp. 291–326.
 
Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning. MIT Press.
 
Gou, J., Yu, B., Maybank, S.J., Tao, D. (2021). Knowledge distillation: a survey. International Journal of Computer Vision, 129(6), 1789–1819.
 
He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778.
 
Hinton, G., Vinyals, O., Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv:1503.02531.
 
Hochreiter, S., Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.
 
Hohman, F., Kery, M.B., Ren, D., Moritz, D. (2024). Model compression in practice: lessons learned from practitioners creating on-device machine learning experiences. In: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pp. 1–18.
 
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., Bengio, Y. (2016). Binarized neural networks. In: NIPS’16: Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 4114–412.
 
Khan, F., Yu, X., Yuan, Z., Rehman, A.U. (2023). ECG classification using 1-D convolutional deep residual neural network. PLOS One, 18(4), 0284791.
 
Khan Mamun, M.M.R., Elfouly, T. (2023). AI-enabled electrocardiogram analysis for disease diagnosis. Applied System Innovation, 6(5), 95.
 
Kullback, S., Leibler, R.A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22(1), 79–86.
 
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.
 
Lee, K.-S., Park, H.-J., Kim, J.E., Kim, H.J., Chon, S., Kim, S., Jang, J., Kim, J.-K., Jang, S., Gil, Y., Son, H.S. (2022). Compressed deep learning to classify arrhythmia in an embedded wearable device. Sensors, 22(5), 1776.
 
Li, C., Wang, J., Wang, S., Zhang, Y. (2024). A review of IoT applications in healthcare. Neurocomputing, 565, 127017.
 
Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P. (2017). Pruning filters for efficient ConvNets. In: International Conference on Learning Representations.
 
Li, J., Alvarez, R. (2021). On the Quantization of Recurrent Neural Networks. arXiv:2101.05453.
 
Li, Z., Li, H., Meng, L. (2023). Model compression for deep neural networks: a survey. Computers, 12(3), 60.
 
Liberis, E., Dudziak, Ł., Lane, N.D. (2021). μnas: Constrained neural architecture search for microcontrollers. In: Proceedings of the 1st Workshop on Machine Learning and Systems, pp. 70–79.
 
Merdjanovska, E., Rashkovska, A. (2022). Comprehensive survey of computational ECG analysis: databases, methods and applications. Expert Systems with Applications, 203, 117206.
 
Mirvis, D.M., Goldberger, A.L. (2001). Electrocardiography. Heart Disease, 1, 82–128.
 
Safdar, M.F., Nowak, R.M., Pałka, P. (2024). Pre-processing techniques and artificial intelligence algorithms for electrocardiogram (ECG) signals analysis: a comprehensive review. Computers in Biology and Medicine, 170, 107908.
 
Selvan, R., Schön, J., Dam, E.B. (2023). Operating critical machine learning models in resource constrained regimes. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 325–335.
 
Sepahvand, M., Abdali-Mohammadi, F. (2022). A novel method for reducing arrhythmia classification from 12-lead ECG signals to single-lead ECG with minimal loss of accuracy through teacher-student knowledge distillation. Information Sciences, 593, 64–77.
 
Strodthoff, N., Wagner, P., Schaeffter, T., Samek, W. (2020). Deep learning for ECG analysis: benchmarks and insights from PTB-XL. IEEE Journal of Biomedical and Health Informatics, 25(5), 1519–1528.
 
Wagner, P., Strodthoff, N., Bousseljot, R.-D., Kreiseler, D., Lunze, F.I., Samek, W., Schaeffter, T. (2020). PTB-XL, a large publicly available electrocardiography dataset. Scientific Data, 7, 154.
 
Wang, Z., Yan, W., Oates, T. (2017). Time series classification from scratch with deep neural networks: a strong baseline. In: 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, pp. 1578–1585.
 
Wu, H., Judd, P., Zhang, X., Isaev, M., Micikevicius, P. (2020). Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation. arXiv:2004.09602.
 
Yang, P., Xie, M.-K., Zong, C.-C., Feng, L., Niu, G., Sugiyama, M., Huang, S.-J. (2023). Multi-label knowledge distillation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17271–17280.

Biographies

Hołda Przemysław
https://orcid.org/0000-0001-5425-7440
przemyslaw.holda@ibspan.waw.pl

P. Hołda is a researcher at the Systems Research Institute of the Polish Academy of Sciences. He received his BSE and MSE degrees in computer science from Warsaw University of Technology. His primary research interests are modern artificial intelligence methods and agent systems.

Wasielewska-Michniewska Katarzyna
https://orcid.org/0000-0002-3763-2373
katarzyna.wasielewska@ibspan.waw.pl

K. Wasielewska-Michniewska is an assistant professor at the Systems Research Institute of the Polish Academy of Sciences. She received her MSE in computer science from Warsaw University of Technology, PhD in computer science from the Polish Academy of Sciences. Her primary research interests are semantic technology and artificial intelligence.

Ganzha Maria
https://orcid.org/0000-0001-7714-4844
maria.ganzha@ibspan.waw.pl

M. Ganzha is an associate professor at Warsaw University of Technology. She received her MA and PhD in mathematics from Moscow University, DSc in computer science from the Polish Academy of Sciences. Her primary research interests are agent-based technology, semantic technology, and machine learning.

Paprzycki Marcin
https://orcid.org/0000-0002-8069-2152
marcin.paprzycki@ibspan.waw.pl

M. Paprzycki is an associate professor at the Systems Research Institute of the Polish Academy of Sciences. He received his MS in mathematics from Adam Mickiewicz University, PhD in mathematics from Southern Methodist University, DSc in mathematics from the Bulgarian Academy of Sciences. His primary research interests are parallel and distributed computing, Internet of Things, semantic technology, and agent systems.


Full article Related articles PDF XML
Full article Related articles PDF XML

Copyright
© 2026 Vilnius University
Open access article under the CC BY license.

Keywords
artificial neural networks deep learning quantization structured pruning knowledge distillation ECG

Funding
Work of Przemysław Hołda and Katarzyna Wasielewska-Michniewska was funded by the European Commission under the Horizon Europe project aerOS, grant No. 101069732.

Metrics
since January 2020
148

Article info
views

74

Full article
views

74

PDF
downloads

11

XML
downloads


Share


RSS