Compressed Amharic Text: A Prediction by Partial Match Context-Modeling Algorithm

Abstract

Amharic is one of the most widely spoken and written languages in Ethiopia, establishing a growing presence in the digital realm. The language serves as the official working language in Ethiopia, employing its distinctive writing system, Fidel, which descends from Ge’ez characters. Amharic symbols are represented by 16 bits in Universal Transformation Format (UTF-8), a standard encoding scheme that ensures compatibility across different platforms and devices. However, this representation utilizes more bits than necessary, as a prior study demonstrated that the entropy of written Amharic language can be as low as 1.074 bits/symbol if the first-order and higher-order statistical dependencies between successive symbols in the language are well-captured. The entropy provides a lower bound on the average number of bits needed to represent a symbol without loss of information, directly impacting the achievable compression rate. This paper proposes using the Prediction by Partial Match (PPM) context-modeling algorithm to efficiently compress Amharic text. PPM is well-suited for textual data, and it adaptively uses a combination of Markov models of different orders to capture first-order and higher-order statistical dependencies between symbols in a text. We used two versions of the PPM algorithm, PPMC and PPMD, to encode eight Amharic text files. Our results show that the best order for efficient encoding is order-3. With this order, we achieved an average of 84.2% reduction in file size, which has a compression rate of 3.3 bits/symbol. This compression rate closely approximates the estimated entropy of the Amharic language (1.074 bits/symbol), demonstrating that PPM effectively compresses Amharic text by exploiting the language’s inherent statistical structure. The reduced bitrate translates to significant bandwidth and energy savings in digital communication systems, as well as reduced storage space requirements.