<?xml version="1.0" encoding="utf-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.0 20120330//EN" "JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">INFORMATICA</journal-id>
<journal-title-group><journal-title>Informatica</journal-title></journal-title-group>
<issn pub-type="epub">1822-8844</issn><issn pub-type="ppub">0868-4952</issn><issn-l>0868-4952</issn-l>
<publisher>
<publisher-name>Vilnius University</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">INFO1159</article-id>
<article-id pub-id-type="doi">10.15388/Informatica.2017.152</article-id>
<article-categories><subj-group subj-group-type="heading">
<subject>Research Article</subject></subj-group></article-categories>
<title-group>
<article-title>Coding Algorithm for Grayscale Images – Design of Piecewise Uniform Quantizer with Golomb–Rice Code and Novel Analytical Model for Performance Analysis</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name><surname>Simić</surname><given-names>Nikola</given-names></name><email xlink:href="simicnikola90@gmail.com">simicnikola90@gmail.com</email><xref ref-type="aff" rid="j_info1159_aff_001">1</xref><bio>
<p><bold>N. Simić</bold> was born in Niš, Serbia, in 1990. He received the BS and MS degrees from the Faculty of Electronic Engineering, University of Niš, Serbia, in 2013 and 2014, respectively, and he was awarded as the best graduated student of his generation. He is currently a student of doctoral studies at the same faculty. His current research interests include image coding and intelligent watermarking. Mr. Simić has been a reviewer of <italic>Multimedia Systems International Journal</italic>. He is an author and co-author of about 20 papers (8 of them in peer-reviewed international journals).</p></bio>
</contrib>
<contrib contrib-type="author">
<name><surname>Perić</surname><given-names>Zoran H.</given-names></name><email xlink:href="zoran.peric@elfak.ni.ac.rs">zoran.peric@elfak.ni.ac.rs</email><xref ref-type="aff" rid="j_info1159_aff_001">1</xref><bio>
<p><bold>Z.H. Perić</bold> was born in Niš, Serbia, in 1964. He received the BS, MS and PhD degrees from the Faculty of Electronic Engineering, University of Niš, Serbia, in 1989, 1994 and 1999, respectively. He is a full-time professor at Department of Telecommunications, Faculty of Electronic Engineering, University of Niš. His current research interests include the information theory and signal processing. He is an author and co-author of over 200 papers. Dr. Perić has been a reviewer in a number of journals, including <italic>IEEE Transactions on Information Theory</italic>, <italic>IEEE Transactions on Signal Processing</italic>, <italic>IEEE Transactions on Communications, Compel</italic>, <italic>Informatica</italic>, <italic>Information Technology and Control</italic>, <italic>Expert Systems with Applications and Digital Signal Processing</italic>.</p></bio>
</contrib>
<contrib contrib-type="author">
<name><surname>Savić</surname><given-names>Milan S.</given-names></name><email xlink:href="malimuzicar@gmail.com">malimuzicar@gmail.com</email><xref ref-type="aff" rid="j_info1159_aff_002">2</xref><xref ref-type="corresp" rid="cor1">∗</xref><bio>
<p><bold>M.S. Savić</bold> was born in Vranje, Serbia, in 1984. He received the MSc and PhD degrees in computer science from the Faculty of Electronic Engineering, University of Niš, Serbia, in 2008 and 2012, respectively. Milan Savić was an assistant research professor at Mathematical Institute of Serbian Academy of Sciences and Arts and currently he is an assistant professor at Faculty of Natural Science and Mathematics, University of Pristina. His current research interests include source coding and quantization of speech signals and images. He is an author and co-author of about 25 papers (13 of them in peer-reviewed international journals). Dr. Savić is a member of the Editorial Board of <italic>Canadian Journal Computer and Information Sciences</italic>.</p></bio>
</contrib>
<aff id="j_info1159_aff_001"><label>1</label>Faculty of Electronic Engineering, <institution>University of Niš</institution>, Aleksandra Medvedeva 14, 18000 Niš, <country>Serbia</country></aff>
<aff id="j_info1159_aff_002"><label>2</label>Faculty of Natural Science and Mathematics, <institution>University of Pristina</institution>, Ive Lole Ribara 29, 38220 Kosovska Mitrovica, <country>Serbia</country></aff>
</contrib-group>
<author-notes>
<corresp id="cor1"><label>∗</label>Corresponding author.</corresp>
</author-notes>
<pub-date pub-type="ppub"><year>2017</year></pub-date><pub-date pub-type="epub"><day>1</day><month>1</month><year>2017</year></pub-date><volume>28</volume><issue>4</issue><fpage>703</fpage><lpage>724</lpage><history><date date-type="received"><month>6</month><year>2016</year></date><date date-type="accepted"><month>5</month><year>2017</year></date></history>
<permissions><copyright-statement>© 2017 Vilnius University</copyright-statement><copyright-year>2017</copyright-year>
<license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/4.0/">
<license-p>Open access article under the <ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/">CC BY</ext-link> license.</license-p></license></permissions>
<abstract>
<p>Scalar quantizer selection for processing a signal with a unit variance is a difficult problem, while both selection and quantizer design for the range of variances is even tougher and to the authors’ best knowledge, it is not theoretically solved. Furthermore, performance estimation of various image processing algorithms is unjustifiably neglected and there are only a few analytical models that follow experimental analysis. In this paper, we analyse application of piecewise uniform quantizer with Golomb-Rice coding in modified block truncation coding algorithm for grayscale image compression, propose design improvements and provide a novel analytical model for performance analysis. Besides the nature of input signal, required compression rate and processing delay of the observed system have a strong influence on quantizer design. Consequently, the impact of quantizer range choice is analysed using a discrete designing variance and it was exploited to improve overall quantizer performance, whereas variable-length coding is applied in order to reduce quantizer’s fixed bit-rate. The analytical model for performance analysis is proposed by introducing Inverse Gaussian distribution and it is obtained by discussing a number of images, providing general closed-form solutions for peak-signal-to-noise ratio and the total average bit-rate estimation. The proposed quantizer design ensures better performance in comparison to the other similar methods for grayscale image compression, including linear prediction of pixel intensity and edge-based adaptation, whereas analytical model for performance analysis provides matching with the experimental results within the range of 1 dB for <italic>PSQNR</italic> and 0.2 bpp for the total average bit-rate.</p>
</abstract>
<kwd-group>
<label>Key words</label>
<kwd>analytical model</kwd>
<kwd>Golomb–Rice coding</kwd>
<kwd>image compression</kwd>
<kwd>inverse Gaussian distribution</kwd>
<kwd>piecewise uniform quantizer</kwd>
<kwd>number of pixel change</kwd>
</kwd-group>
</article-meta>
</front>
<body>
<sec id="j_info1159_s_001">
<label>1</label>
<title>Introduction</title>
<p>Image processing incorporates a variety of manipulations designed in order to provide desired information or specific image quality that is application depended. Although both optical and analog image processing have a great importance in the image acquisition processes, digital image processing has a dominant role in a number of applications such as intelligent watermarking, video telephony or pattern recognition. With high-growing market, these applications became very important and high quality of images is required. The final objective is to achieve as high as possible image quality at the given bit-rate. On the other hand, age of information sets new hardware requirements with the rapid growing, setting limited data storage, low processing delay or hardware simplicity as important demands. As a result, digital image compression has a key role in development of future systems.</p>
<p>Block truncation coding (BTC) is the image compression method proposed by Delp and Mitchell (<xref ref-type="bibr" rid="j_info1159_ref_007">1979</xref>). Its quite simple design and capability to introduce other various techniques into the algorithm made it very popular and a common tool for solving different problems. Currently, it is commonly used to compress a video to be stored in frame memory for display devices such as LCDs (Kim <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_012">2016</xref>) and it is widely used for LCD overdrives (Kim and Lee, <xref ref-type="bibr" rid="j_info1159_ref_010">2016</xref>). Furthermore, modern watermarking techniques are very often based on block truncation coding – in Chang <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_005">2015</xref>) it is proposed high capacity reversible data hiding scheme based on residual histogram shifting for compressed images of block truncation coding whereas high capacity data hiding scheme for error-diffused block truncation coding is presented by Guo and Liu (<xref ref-type="bibr" rid="j_info1159_ref_008">2012</xref>). Next, SVD-based tamper detection and self-recovery algorithm using active watermarking and block partitioning is discussed in Dadkhah <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_006">2014</xref>). Moreover, BTC method can be successfully applied to the systems that deal with both grayscale and colour images (Lema and Mitchell, <xref ref-type="bibr" rid="j_info1159_ref_013">1984</xref>; Chang <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_004">2008</xref>).</p>
<p>Also, colour image quantization has been researched in recent years and some methods based on artificial bee colony algorithm (Ozturk <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_017">2014</xref>) or designed for specific feature extraction (Ponti <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_022">2016</xref>) were proposed. Besides that, current problems require application of various techniques to biometric systems, pattern recognition, computer vision or medical imaging making the image processing algorithms research very important (Barcellos <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_003">2015</xref>; Alomari <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_002">2016</xref>). In this paper, we are focused on piecewise-uniform quantizer designing and its application to the algorithm which processes grayscale images.</p>
<p>Issues that originate during image acquisition and image-processing steps that were applied to the image during digital image processing, such as those involving contrast and dynamic range mismatch, can be successfully depicted using a histogram (Salomon, <xref ref-type="bibr" rid="j_info1159_ref_023">2007</xref>). Histograms are frequently used to determine whether an image is making effective use of its intensity range by analysing the size, shape and form of the histogram’s distribution. In Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>) a model for <italic>PSQNR</italic> (peak signal-to-quantization-noise ratio) calculation which exploits weighting function for averaging was presented. However, this function depends on the input dataset and closed-form solutions are not provided. In this paper, we will consider a histogram of variances which is focused on distinction between a pixel value and a mean value of all pixels in a block that the pixel belongs to. This kind of histogram will be researched in order to make an analytical model with closed-form solutions for performance estimation of the proposed system. Besides <italic>PSQNR</italic>, estimation of the total average bit-rate will be provided due to a great influence of applied Golomb–Rice coding as a lossless data compression method (Perić <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_019">2010a</xref>, <xref ref-type="bibr" rid="j_info1159_ref_020">2010b</xref>). The histogram will be modelled by introducing Inverse Gaussian function in order to provide general closed-form solutions.</p>
<p>Besides providing a novel analytical model, emphasize of the paper is to propose an improved quantizer design, in order to improve overall system performance of block truncation coding. Improvements are done by applying Golomb–Rice binary coding of the piecewise uniform quantizer’s output signal as well as by determining its optimal support range, discussing the influence of the discrete designing variance. Variable-length coding is already presented in both software and hardware solutions in a number of schemes that are exploited in image processing techniques, including the state-of-the-art solutions (Sayood, <xref ref-type="bibr" rid="j_info1159_ref_027">2006</xref>; Salomon, <xref ref-type="bibr" rid="j_info1159_ref_023">2007</xref>). JPEG-LS exploits various schemes for adaptive coding of residuals such as arithmetic coding and the Merhav–Seroussi–Weinberger scheme (Ali and Manzur, <xref ref-type="bibr" rid="j_info1159_ref_001">2015</xref>). Moreover, some hardware solutions are already proposed, making Golomb–Rice coding suitable for both software and hardware applications (Kim <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_011">2011</xref>). Next, hybrid model for compression of Laplacian source, discussed in Perić <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_019">2010a</xref>), exploits Golomb–Rice output levels coding. However, the authors discussed hybrid model only for the unit variance, which is not suitable for image compression applications. Furthermore, aforementioned hybrid quantizer deals with continual signals whereas image compression system that we discuss considers discrete signals. In this paper, we propose Golomb–Rice coding application for designing core quantizer encoders, such as those applied in block truncation coding, and we discuss its performance for medium bit-rates that provide very high image quality. The obtained closed-form solutions depend on the input signal variance as well as discrete designing variance. Furthermore, this way designed quantizer encoder can be applied also in the other different and more complex image processing algorithms. In the end, it will be shown that this way improved simple block truncation coding system outperforms other similar techniques that incorporate linear prediction (Savic <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_026">2015</xref>) and forward edge-based adaptation (Mathews and Nair, <xref ref-type="bibr" rid="j_info1159_ref_016">2015</xref>). The system which incorporates linear prediction of the neighbouring pixel values (Savic <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_026">2015</xref>), exploits advantages of both uniform and piecewise uniform quantizers to perform additional data transfer in order to improve previously predicted values. However, this way applied dual-mode quantization is performed only on about 70% of the pixels, as the prediction provides satisfactory results at 30% of blocks (size <inline-formula id="j_info1159_ineq_001"><alternatives><mml:math>
<mml:mn>4</mml:mn>
<mml:mo>×</mml:mo>
<mml:mn>4</mml:mn></mml:math><tex-math><![CDATA[$4\times 4$]]></tex-math></alternatives></inline-formula>). However, we have to note that even though the proposed quantizer design provides high image quality, few existing techniques that incorporate vector instead of scalar quantization (Simic <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_028">2017</xref>) or transformation coding, such as wavelet or curvelet (Li <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_014">2010</xref>, <xref ref-type="bibr" rid="j_info1159_ref_015">2013</xref>), provide higher compression ratio but also demand higher processing delay and have significantly higher complexity.</p>
<p>The paper is organized as follows. Section <xref rid="j_info1159_s_002">2</xref> describes the proposed coding algorithm and piecewise uniform quantizer design with Golomb–Rice coding. Next, Section <xref rid="j_info1159_s_005">3</xref> proposes a novel analytical model with closed-form solutions, based on Inverse Gaussian function introduction. Finally, experimental results and comparison with other similar models are shown in Section <xref rid="j_info1159_s_006">4</xref>.</p>
</sec>
<sec id="j_info1159_s_002">
<label>2</label>
<title>Coding Algorithm and Piecewise Uniform Quantization</title>
<p>In the case when it is necessary to operate with discrete data, quantization process is very often done in two steps. The first one deals with the received continual signal where continual samples have to be quantized with the fixed uniform quantizer <inline-formula id="j_info1159_ineq_002"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{0}}$]]></tex-math></alternatives></inline-formula>, described with <inline-formula id="j_info1159_ineq_003"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${N_{0}}$]]></tex-math></alternatives></inline-formula> quantization levels. This way obtained samples, <inline-formula id="j_info1159_ineq_004"><alternatives><mml:math>
<mml:mi mathvariant="italic">X</mml:mi>
<mml:mo>=</mml:mo>
<mml:mo fence="true" stretchy="false">{</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mo>…</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msub>
<mml:mo fence="true" stretchy="false">}</mml:mo></mml:math><tex-math><![CDATA[$X=\{{x_{1}},{x_{2}},\dots ,{x_{{N_{0}}}}\}$]]></tex-math></alternatives></inline-formula>, are further quantized with quantizer <inline-formula id="j_info1159_ineq_005"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{1}}$]]></tex-math></alternatives></inline-formula> in order to perform additional data compression. Usually, quantizer <inline-formula id="j_info1159_ineq_006"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{0}}$]]></tex-math></alternatives></inline-formula> performs quantization with the high number of quantization levels. For grayscale images <inline-formula id="j_info1159_ineq_007"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mn>512</mml:mn></mml:math><tex-math><![CDATA[${N_{0}}=512$]]></tex-math></alternatives></inline-formula> since that block truncation coding algorithm is based on quantization of distinction between a pixel value and the mean value of all pixels in a block that pixel belongs to, where <inline-formula id="j_info1159_ineq_008"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mo>−</mml:mo>
<mml:mn>255</mml:mn></mml:math><tex-math><![CDATA[${x_{1}}=-255$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_009"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mn>255</mml:mn></mml:math><tex-math><![CDATA[${x_{{N_{0}}}}=255$]]></tex-math></alternatives></inline-formula> (pixels of an image can take integer values from 0 to 255). On the other hand, quantizer <inline-formula id="j_info1159_ineq_010"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{1}}$]]></tex-math></alternatives></inline-formula> should deal with a low or medium number of quantization levels, in order to make a compromise between reconstruction quality and the total average bit-rate. Nowadays, algorithms that discuss low number of quantization levels, providing lower image quality, are much more researched (Simic <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_028">2017</xref>). Thus, in this paper we will discuss piecewise uniform quantizer <inline-formula id="j_info1159_ineq_011"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{1}}$]]></tex-math></alternatives></inline-formula> with <inline-formula id="j_info1159_ineq_012"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>16</mml:mn></mml:math><tex-math><![CDATA[$N=16$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_013"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>32</mml:mn></mml:math><tex-math><![CDATA[$N=32$]]></tex-math></alternatives></inline-formula> quantization levels, i.e. with medium number of levels.</p>
<p>It is well-known from literature that Laplacian information source provides excellent modelling of distinction between pixel value and the mean value of all pixels in a block, i.e. provides satisfactory matching between block truncation coding algorithm and reality (Jayant and Noll, <xref ref-type="bibr" rid="j_info1159_ref_009">1984</xref>). Consequently, we will suppose that information source has Laplacian distribution with a memoryless property and mean value equal to zero. It is defined with: 
<disp-formula id="j_info1159_eq_001">
<label>(1)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">p</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:mo stretchy="false">|</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo stretchy="false">|</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ p(x)=\frac{1}{\sqrt{2}\sigma }\exp \bigg(-\frac{\sqrt{2}|x|}{\sigma }\bigg).\]]]></tex-math></alternatives>
</disp-formula>
</p>
<p>The second step of quantization process provides quantization of discrete output samples from quantizer <inline-formula id="j_info1159_ineq_014"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{0}}$]]></tex-math></alternatives></inline-formula> by using <italic>N</italic> quantization levels, where <inline-formula id="j_info1159_ineq_015"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo mathvariant="normal">&lt;</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[$N<{N_{0}}$]]></tex-math></alternatives></inline-formula>, providing additional data compression. Probabilities of these discrete input levels, considering Laplacian source, are defined with: 
<disp-formula id="j_info1159_eq_002">
<label>(2)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">P</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:msubsup>
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∫</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>+</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msubsup>
<mml:mi mathvariant="italic">p</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>=</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo>−</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>+</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ P({x_{i}})={\int _{{x_{i}}}^{{x_{i+1}}}}p(x)\hspace{0.1667em}\mathrm{d}x=\frac{1}{2}\bigg(\exp \bigg(-\frac{\sqrt{2}{x_{i}}}{\sigma }\bigg)-\exp \bigg(-\frac{\sqrt{2}{x_{i+1}}}{\sigma }\bigg)\bigg).\]]]></tex-math></alternatives>
</disp-formula>
</p>
<p>Design of the piecewise uniform quantizer <inline-formula id="j_info1159_ineq_016"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{1}}$]]></tex-math></alternatives></inline-formula> will be done as follows. Firstly, a piecewise uniform quantizer for the unit variance is designed. Its threshold values are obtained by using optimal compandor, whose compressor function is defined by Jayant and Noll (<xref ref-type="bibr" rid="j_info1159_ref_009">1984</xref>): 
<disp-formula id="j_info1159_eq_003">
<label>(3)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">c</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>+</mml:mo>
<mml:mn>2</mml:mn><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msubsup>
<mml:mrow>
<mml:mo largeop="false" movablelimits="false">∫</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mo>−</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
</mml:msubsup>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">p</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mn>3</mml:mn>
</mml:mrow>
</mml:msup>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">t</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:msubsup>
<mml:mrow>
<mml:mo largeop="false" movablelimits="false">∫</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mo>−</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msubsup>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">p</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mn>3</mml:mn>
</mml:mrow>
</mml:msup>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">t</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ c(x)=-1+2\frac{{\textstyle\textstyle\int _{-{t_{\max }}}^{x}}{p^{1/3}}(t)\hspace{0.1667em}\mathrm{d}t}{{\textstyle\textstyle\int _{-{t_{\max }}}^{{t_{\max }}}}{p^{1/3}}(t)\hspace{0.1667em}\mathrm{d}t},\]]]></tex-math></alternatives>
</disp-formula> 
where <inline-formula id="j_info1159_ineq_017"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${t_{\max }}$]]></tex-math></alternatives></inline-formula> denotes maximal signal amplitude. Boundaries between segments can be calculated with: <disp-formula-group id="j_info1159_dg_001">
<disp-formula id="j_info1159_eq_004">
<label>(4)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">φ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo>=</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>3</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo movablelimits="false">log</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>2</mml:mn>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mi mathvariant="italic">M</mml:mi>
<mml:mo>+</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>2</mml:mn>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mi mathvariant="italic">M</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="1.19em" minsize="1.19em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
</mml:mrow>
<mml:mrow>
<mml:mn>3</mml:mn>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo maxsize="1.19em" minsize="1.19em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mspace width="1em"/>
<mml:mn>0</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mn>2</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {\varphi _{i}^{\sigma =1}}=\frac{3}{\sqrt{2}}\log \bigg(\frac{2iM+(N-2iM)\exp \big(-\frac{\sqrt{2}}{3}{t_{\max }^{\sigma =1}}(N)\big)}{N}\bigg),\hspace{1em}0\leqslant i\leqslant L/2,\]]]></tex-math></alternatives>
</disp-formula>
<disp-formula id="j_info1159_eq_005">
<label>(5)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">φ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo>=</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>3</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo movablelimits="false">log</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>2</mml:mn>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mi mathvariant="italic">M</mml:mi>
<mml:mo>+</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mn>2</mml:mn>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mi mathvariant="italic">M</mml:mi>
<mml:mo>−</mml:mo>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="1.19em" minsize="1.19em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
</mml:mrow>
<mml:mrow>
<mml:mn>3</mml:mn>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo maxsize="1.19em" minsize="1.19em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mspace width="1em"/>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mn>2</mml:mn>
<mml:mo mathvariant="normal">&lt;</mml:mo>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {\varphi _{i}^{\sigma =1}}=\frac{3}{\sqrt{2}}\log \bigg(\frac{N}{2N-2iM+(2iM-N)\exp \big(-\frac{\sqrt{2}}{3}{t_{\max }^{\sigma =1}}(N)\big)}\bigg),\hspace{1em}L/2<i\leqslant L.\]]]></tex-math></alternatives>
</disp-formula>
</disp-formula-group> In previous expressions, <inline-formula id="j_info1159_ineq_018"><alternatives><mml:math>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup></mml:math><tex-math><![CDATA[${t_{\max }^{\sigma =1}}$]]></tex-math></alternatives></inline-formula> represents the optimal maximal signal amplitude for the unit variance and its values are taken from (Perić <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_018">2009</xref>). The value depends on the number of quantization levels – <inline-formula id="j_info1159_ineq_019"><alternatives><mml:math>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo>=</mml:mo>
<mml:mn>6.01</mml:mn></mml:math><tex-math><![CDATA[${t_{\max }^{\sigma =1}}=6.01$]]></tex-math></alternatives></inline-formula> for <inline-formula id="j_info1159_ineq_020"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>16</mml:mn></mml:math><tex-math><![CDATA[$N=16$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_021"><alternatives><mml:math>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo>=</mml:mo>
<mml:mn>7.91</mml:mn></mml:math><tex-math><![CDATA[${t_{\max }^{\sigma =1}}=7.91$]]></tex-math></alternatives></inline-formula> for <inline-formula id="j_info1159_ineq_022"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>32</mml:mn></mml:math><tex-math><![CDATA[$N=32$]]></tex-math></alternatives></inline-formula>. Furthermore, the number of uniform output levels in each segment is denoted with <italic>M</italic> and it can be calculated as <inline-formula id="j_info1159_ineq_023"><alternatives><mml:math>
<mml:mi mathvariant="italic">M</mml:mi>
<mml:mo>=</mml:mo>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi></mml:math><tex-math><![CDATA[$M=N/L$]]></tex-math></alternatives></inline-formula>.</p>
<p>The width of each region cell within a segment is equal to: 
<disp-formula id="j_info1159_eq_006">
<label>(6)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="normal">Δ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">φ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo>−</mml:mo>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">φ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">M</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {\Delta _{i}}=\frac{{\varphi _{i}^{\sigma =1}}-{\varphi _{i-1}^{\sigma =1}}}{M}.\]]]></tex-math></alternatives>
</disp-formula> 
Furthermore, decision thresholds of the piecewise uniform quantizer, designed for the unit variance, can be obtained as: 
<disp-formula id="j_info1159_eq_007">
<label>(7)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">ω</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">opt</mml:mi>
</mml:mrow>
</mml:msubsup>
<mml:mo>=</mml:mo>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">φ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo>+</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="normal">Δ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mspace width="1em"/>
<mml:mn>0</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>;</mml:mo>
<mml:mspace width="2.5pt"/>
<mml:mn>1</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">M</mml:mi>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {\omega _{i,j}^{\mathrm{opt}}}={\varphi _{i}^{\sigma =1}}+j\cdot {\Delta _{i}},\hspace{1em}0\leqslant i\leqslant L;\hspace{2.5pt}1\leqslant j\leqslant M.\]]]></tex-math></alternatives>
</disp-formula> 
Finally, the output levels are defined with: 
<disp-formula id="j_info1159_eq_008">
<label>(8)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">y</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">opt</mml:mi>
</mml:mrow>
</mml:msubsup>
<mml:mo>=</mml:mo>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">φ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo>+</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>2</mml:mn>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="normal">Δ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mspace width="1em"/>
<mml:mn>0</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>;</mml:mo>
<mml:mspace width="2.5pt"/>
<mml:mn>1</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">M</mml:mi>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {y_{i,j}^{\mathrm{opt}}}={\varphi _{i}^{\sigma =1}}+\bigg(\frac{2j-1}{2}\bigg){\Delta _{i}},\hspace{1em}0\leqslant i\leqslant L;\hspace{2.5pt}1\leqslant j\leqslant M.\]]]></tex-math></alternatives>
</disp-formula> 
In Eqs. (<xref rid="j_info1159_eq_004">4</xref>)–(<xref rid="j_info1159_eq_008">8</xref>) <italic>i</italic> denotes a segment while <italic>j</italic> denotes the output level within the observed segment. Due to the fact that compressor function <inline-formula id="j_info1159_ineq_024"><alternatives><mml:math>
<mml:mi mathvariant="italic">c</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$c(x)$]]></tex-math></alternatives></inline-formula>, defined with Eq. (<xref rid="j_info1159_eq_003">3</xref>), maps the input range to <inline-formula id="j_info1159_ineq_025"><alternatives><mml:math>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mo>−</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$(-{t_{\max }},{t_{\max }})$]]></tex-math></alternatives></inline-formula> wherein <inline-formula id="j_info1159_ineq_026"><alternatives><mml:math>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo stretchy="false">≪</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${t_{\max }^{\sigma =1}}\ll {x_{{N_{0}}}}$]]></tex-math></alternatives></inline-formula>, denormalization will be done by using a discrete designing variance <inline-formula id="j_info1159_ineq_027"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover></mml:math><tex-math><![CDATA[$\hat{\sigma }$]]></tex-math></alternatives></inline-formula> as follows: <disp-formula-group id="j_info1159_dg_002">
<disp-formula id="j_info1159_eq_009">
<label>(9)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">ω</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">ω</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">opt</mml:mi>
</mml:mrow>
</mml:msubsup>
<mml:mo>·</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mspace width="1em"/>
<mml:mn>0</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>;</mml:mo>
<mml:mspace width="2.5pt"/>
<mml:mn>1</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">M</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {\omega _{i,j}}={\omega _{i,j}^{\mathrm{opt}}}\cdot \hat{\sigma },\hspace{1em}0\leqslant i\leqslant L;\hspace{2.5pt}1\leqslant j\leqslant M,\]]]></tex-math></alternatives>
</disp-formula>
<disp-formula id="j_info1159_eq_010">
<label>(10)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">y</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">y</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">opt</mml:mi>
</mml:mrow>
</mml:msubsup>
<mml:mo>·</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mspace width="1em"/>
<mml:mn>0</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>;</mml:mo>
<mml:mspace width="2.5pt"/>
<mml:mn>1</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">M</mml:mi>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {y_{i,j}}={y_{i,j}^{\mathrm{opt}}}\cdot \hat{\sigma },\hspace{1em}0\leqslant i\leqslant L;\hspace{2.5pt}1\leqslant j\leqslant M.\]]]></tex-math></alternatives>
</disp-formula>
</disp-formula-group> Finally, quantization levels could be written in a simpler form using one index, as: 
<disp-formula id="j_info1159_eq_011">
<label>(11)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">y</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">y</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mspace width="1em"/>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mo>…</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {y_{k}}={y_{i,j}},\hspace{1em}k=1,\dots ,N,\]]]></tex-math></alternatives>
</disp-formula> 
where index <italic>k</italic>, that corresponds to indices <italic>i</italic> and <italic>j</italic>, is obtained as <inline-formula id="j_info1159_ineq_028"><alternatives><mml:math>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mo>=</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mi mathvariant="italic">M</mml:mi>
<mml:mo>+</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi></mml:math><tex-math><![CDATA[$k=(i-1)M+j$]]></tex-math></alternatives></inline-formula>.</p>
<sec id="j_info1159_s_003">
<label>2.1</label>
<title>Golomb–Rice Coding</title>
<p>After quantization with <inline-formula id="j_info1159_ineq_029"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{0}}$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_030"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{1}}$]]></tex-math></alternatives></inline-formula>, signal is encoded with a binary Golomb–Rice encoder at the output of designed quantizer encoder. In order to apply Golomb–Rice code described with parameter <inline-formula id="j_info1159_ineq_031"><alternatives><mml:math>
<mml:mi mathvariant="italic">m</mml:mi>
<mml:mo>=</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[$m={2^{k}}$]]></tex-math></alternatives></inline-formula>, <italic>N</italic> quantization levels have to be divided into <italic>S</italic> segments. Taking into account symmetry of the quantizer <inline-formula id="j_info1159_ineq_032"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{1}}$]]></tex-math></alternatives></inline-formula>, we will consider that <inline-formula id="j_info1159_ineq_033"><alternatives><mml:math>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>=</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mn>2</mml:mn></mml:math><tex-math><![CDATA[$S=L/2$]]></tex-math></alternatives></inline-formula>, i.e. <italic>S</italic> is equal to the number of segments in the positive range of quantizer <inline-formula id="j_info1159_ineq_034"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{1}}$]]></tex-math></alternatives></inline-formula>, in order to simplify encoding design. This way, each Golomb–Rice segment will cover the corresponding symmetric ranges from the quantizer <inline-formula id="j_info1159_ineq_035"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{1}}$]]></tex-math></alternatives></inline-formula> in both positive and negative range. Segments are indexed with <inline-formula id="j_info1159_ineq_036"><alternatives><mml:math>
<mml:mn>0</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mo>…</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn></mml:math><tex-math><![CDATA[$0,1,\dots ,S-1$]]></tex-math></alternatives></inline-formula>. The total number of quantization levels has to be equal to: 
<disp-formula id="j_info1159_eq_012">
<label>(12)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>·</mml:mo>
<mml:mi mathvariant="italic">m</mml:mi>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ N=S\cdot m.\]]]></tex-math></alternatives>
</disp-formula> 
When organizing encoding process in this way, it can be seen that <italic>m</italic> represents the number of output levels in each segment of variable length coding. Finally, the codeword of the <italic>j</italic>th segment (<inline-formula id="j_info1159_ineq_037"><alternatives><mml:math>
<mml:mn>0</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn></mml:math><tex-math><![CDATA[$0\leqslant j\leqslant S-1$]]></tex-math></alternatives></inline-formula>) is formed as <inline-formula id="j_info1159_ineq_038"><alternatives><mml:math>
<mml:munder>
<mml:mrow>
<mml:munder accentunder="false">
<mml:mrow>
<mml:mi mathvariant="italic">X</mml:mi>
<mml:mi mathvariant="italic">X</mml:mi>
</mml:mrow>
<mml:mo>︸</mml:mo></mml:munder>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:munder>
<mml:mn>0</mml:mn>
<mml:munder>
<mml:mrow>
<mml:munder accentunder="false">
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>…</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mo>︸</mml:mo></mml:munder>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:munder></mml:math><tex-math><![CDATA[$\underset{j}{\underbrace{XX}}0\underset{k}{\underbrace{x\dots x}}$]]></tex-math></alternatives></inline-formula> and its length is (Perić <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_019">2010a</xref>): 
<disp-formula id="j_info1159_eq_013">
<label>(13)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">l</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>+</mml:mo>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mo>+</mml:mo>
<mml:mn>1</mml:mn>
<mml:mspace width="2.5pt"/>
<mml:mo fence="true" stretchy="false">[</mml:mo>
<mml:mtext>bits</mml:mtext>
<mml:mo fence="true" stretchy="false">]</mml:mo>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {l_{j}}=j+k+1\hspace{2.5pt}[\text{bits}].\]]]></tex-math></alternatives>
</disp-formula>
</p>
</sec>
<sec id="j_info1159_s_004">
<label>2.2</label>
<title>Algorithm for Image Processing</title>
<p>In this section, we will explain the application of the previously described quantizer on grayscale image processing. The algorithm is performed from left to right and from top to bottom and it consists of the following steps: 
<list>
<list-item id="j_info1159_li_001">
<label>1.</label>
<p>The image is divided into a set of non-overlapping blocks, whose size is <inline-formula id="j_info1159_ineq_039"><alternatives><mml:math>
<mml:mi mathvariant="italic">n</mml:mi>
<mml:mo>×</mml:mo>
<mml:mi mathvariant="italic">n</mml:mi></mml:math><tex-math><![CDATA[$n\times n$]]></tex-math></alternatives></inline-formula> (we will use <inline-formula id="j_info1159_ineq_040"><alternatives><mml:math>
<mml:mi mathvariant="italic">n</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>4</mml:mn></mml:math><tex-math><![CDATA[$n=4$]]></tex-math></alternatives></inline-formula>). Each block is processed separately.</p>
</list-item>
<list-item id="j_info1159_li_002">
<label>2.</label>
<p>The average pixel value <inline-formula id="j_info1159_ineq_041"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${x_{\mathrm{av}}}$]]></tex-math></alternatives></inline-formula> of the block is found. Next, this value is quantized (with uniform quantizers using 6 bits) and obtained quantized average value <inline-formula id="j_info1159_ineq_042"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\hat{x}_{\mathrm{av}}}$]]></tex-math></alternatives></inline-formula> is transmitted to the receiver (Savic <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_024">2010</xref>).</p>
</list-item>
<list-item id="j_info1159_li_003">
<label>3.</label>
<p>The difference block of size <inline-formula id="j_info1159_ineq_043"><alternatives><mml:math>
<mml:mi mathvariant="italic">n</mml:mi>
<mml:mo>×</mml:mo>
<mml:mi mathvariant="italic">n</mml:mi></mml:math><tex-math><![CDATA[$n\times n$]]></tex-math></alternatives></inline-formula> is formed. Its elements are obtained as the difference of the pixel values of the current block <inline-formula id="j_info1159_ineq_044"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${x_{i,j}}$]]></tex-math></alternatives></inline-formula> and its quantized mean value <inline-formula id="j_info1159_ineq_045"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\hat{x}_{\mathrm{av}}}$]]></tex-math></alternatives></inline-formula>, and they are denoted with <inline-formula id="j_info1159_ineq_046"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${d_{i,j}}$]]></tex-math></alternatives></inline-formula>, <inline-formula id="j_info1159_ineq_047"><alternatives><mml:math>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mo>…</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">n</mml:mi></mml:math><tex-math><![CDATA[$i=1,\dots ,n$]]></tex-math></alternatives></inline-formula>; <inline-formula id="j_info1159_ineq_048"><alternatives><mml:math>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mo>…</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">n</mml:mi></mml:math><tex-math><![CDATA[$j=1,\dots ,n$]]></tex-math></alternatives></inline-formula>: 
<disp-formula id="j_info1159_eq_014">
<label>(14)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>−</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {d_{i,j}}={x_{i,j}}-{\hat{x}_{\mathrm{av}}}.\]]]></tex-math></alternatives>
</disp-formula>
</p>
</list-item>
<list-item id="j_info1159_li_004">
<label>4.</label>
<p>Quantization of elements <inline-formula id="j_info1159_ineq_049"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${d_{i,j}}$]]></tex-math></alternatives></inline-formula> is done using the proposed quantizer <inline-formula id="j_info1159_ineq_050"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Q_{1}}$]]></tex-math></alternatives></inline-formula>.</p>
</list-item>
<list-item id="j_info1159_li_005">
<label>5.</label>
<p>Quantized elements denoted with <inline-formula id="j_info1159_ineq_051"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\hat{d}_{i,j}}$]]></tex-math></alternatives></inline-formula> are binary coded with Golomb–Rice code and transmitted to the receiver.</p>
</list-item>
<list-item id="j_info1159_li_006">
<label>6.</label>
<p>In the receiver, after the reception of <inline-formula id="j_info1159_ineq_052"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\hat{x}_{\mathrm{av}}}$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_053"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\hat{d}_{i,j}}$]]></tex-math></alternatives></inline-formula>, the reconstruction of all pixels of the original image is done as: 
<disp-formula id="j_info1159_eq_015">
<label>(15)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>+</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {\hat{x}_{i,j}}={\hat{d}_{i,j}}+{\hat{x}_{\mathrm{av}}}.\]]]></tex-math></alternatives>
</disp-formula>
</p>
</list-item>
<list-item id="j_info1159_li_007">
<label>7.</label>
<p>Go to the step 2 until all blocks are processed.</p>
</list-item>
</list>
</p>
</sec>
</sec>
<sec id="j_info1159_s_005">
<label>3</label>
<title>Analytical Model for Performance Analysis</title>
<p>Besides analysis of the piecewise uniform quantizer design and its optimization, the main goal of this paper is to propose a novel analytical model for grayscale image compression performance estimation. The model considers the proposed quantizer design and modified block truncation coding algorithm described in Section <xref rid="j_info1159_s_004">2.2</xref> and it is based on a modelling of histogram of variances whereas distinction between a pixel and the mean pixel value of all pixels in a block that pixel belongs to, is considered as the input signal. The aforementioned histogram is obtained by processing ten standard test grayscale images (Baboon, Bridge, Couple, Jet, Cart, Lena, Pepper, Ship, Street and Church).</p>
<p>In order to analyse and make a model of such histogram, we have researched various distributions such as Rayleigh, Lognormal, Birnbaum-Saunders and Laplacian. However, the best results we have achieved were by using general Inverse Gaussian distribution that is defined with: 
<disp-formula id="j_info1159_eq_016">
<label>(16)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">f</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">λ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
<mml:mi mathvariant="italic">π</mml:mi>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>3</mml:mn>
</mml:mrow>
</mml:msup>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msup>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mo>−</mml:mo>
<mml:mi mathvariant="italic">λ</mml:mi>
<mml:msup>
<mml:mrow>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>−</mml:mo>
<mml:mi mathvariant="italic">μ</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msup>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">μ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msup>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ f(x)={\bigg(\frac{\lambda }{2\pi {x^{3}}}\bigg)^{1/2}}\exp \bigg(\frac{-\lambda {(x-\mu )^{2}}}{2{\mu ^{2}}x}\bigg).\]]]></tex-math></alternatives>
</disp-formula> 
Its support range is <inline-formula id="j_info1159_ineq_054"><alternatives><mml:math>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo stretchy="false">∈</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mn>0</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mo>+</mml:mo>
<mml:mi>∞</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$x\in (0,+\infty )$]]></tex-math></alternatives></inline-formula> whereas the mean parameter is (<inline-formula id="j_info1159_ineq_055"><alternatives><mml:math>
<mml:mi mathvariant="italic">μ</mml:mi>
<mml:mo mathvariant="normal">&gt;</mml:mo>
<mml:mn>0</mml:mn></mml:math><tex-math><![CDATA[$\mu >0$]]></tex-math></alternatives></inline-formula>) and <italic>λ</italic> is the shape parameter <inline-formula id="j_info1159_ineq_056"><alternatives><mml:math>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">λ</mml:mi>
<mml:mo mathvariant="normal">&gt;</mml:mo>
<mml:mn>0</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$(\lambda >0)$]]></tex-math></alternatives></inline-formula>. The parameters of Inverse Gaussian distribution are estimated by using maximum likelihood estimation method (MLE) which represents one of the standard methods and probably the most versatile one. However, the parameters could be estimated also by using other methods such as method of moments or least squares estimation method. If we suppose that <inline-formula id="j_info1159_ineq_057"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mo>…</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${x_{1}},{x_{2}},\dots ,{x_{n}}$]]></tex-math></alternatives></inline-formula> are independent and identically distributed observations, likelihood function is defined with: 
<disp-formula id="j_info1159_eq_017">
<label>(17)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">θ</mml:mi>
<mml:mo>;</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mo>…</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∏</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:munderover>
<mml:mi mathvariant="italic">f</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">θ</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ L(\theta ;{x_{1}},{x_{2}},\dots ,{x_{n}})={\prod \limits_{i=1}^{n}}f({x_{i}},\theta ),\]]]></tex-math></alternatives>
</disp-formula> 
where <italic>θ</italic> is a vector of parameters. After substituting Eq. (<xref rid="j_info1159_eq_016">16</xref>) in Eq. (<xref rid="j_info1159_eq_017">17</xref>), likelihood function for Inverse Gaussian distribution is obtained: 
<disp-formula id="j_info1159_eq_018">
<label>(18)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">μ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">λ</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">λ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
<mml:mi mathvariant="italic">π</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
</mml:mrow>
</mml:msup>
<mml:msup>
<mml:mrow>
<mml:mo maxsize="2.45em" minsize="2.45em" fence="true" mathvariant="normal">(</mml:mo>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∏</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:munderover><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">X</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>3</mml:mn>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.45em" minsize="2.45em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
</mml:mrow>
</mml:msup>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="2.45em" minsize="2.45em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
<mml:mi mathvariant="italic">λ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">μ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">λ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">μ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msup>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:munderover>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">X</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">λ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:munderover><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">X</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.45em" minsize="2.45em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ L(\mu ,\lambda )={\bigg(\frac{\lambda }{2\pi }\bigg)^{\frac{n}{2}}}{\Bigg({\prod \limits_{i=1}^{n}}\frac{1}{{X_{i}^{3}}}\Bigg)^{\frac{1}{2}}}\exp \Bigg(\frac{n\lambda }{\mu }-\frac{\lambda }{2{\mu ^{2}}}{\sum \limits_{i=1}^{n}}{X_{i}}-\frac{\lambda }{2}{\sum \limits_{i=1}^{n}}\frac{1}{{X_{i}}}\Bigg).\]]]></tex-math></alternatives>
</disp-formula> 
Finally, assuming that the log-likelihood function <inline-formula id="j_info1159_ineq_058"><alternatives><mml:math>
<mml:mo movablelimits="false">log</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">μ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">λ</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$\log (L(\mu ,\lambda ))$]]></tex-math></alternatives></inline-formula> is differentiable, unknown parameters are estimated as: <disp-formula-group id="j_info1159_dg_003">
<disp-formula id="j_info1159_eq_019">
<label>(19)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mo movablelimits="false">log</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">μ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">λ</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="italic">μ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo>=</mml:mo>
<mml:mn>0</mml:mn>
<mml:mspace width="1em"/>
<mml:mo stretchy="false">⇒</mml:mo>
<mml:mspace width="1em"/><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">μ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:munderover>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">X</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ \frac{\mathrm{d}\log L(\mu ,\lambda )}{\mathrm{d}\hspace{0.1667em}\mu }=0\hspace{1em}\Rightarrow \hspace{1em}\hat{\mu }=\frac{1}{n}{\sum \limits_{i=1}^{n}}{X_{i}},\]]]></tex-math></alternatives>
</disp-formula>
<disp-formula id="j_info1159_eq_020">
<label>(20)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mo movablelimits="false">log</mml:mo>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">μ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">λ</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="italic">λ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo>=</mml:mo>
<mml:mn>0</mml:mn>
<mml:mspace width="1em"/>
<mml:mo stretchy="false">⇒</mml:mo>
<mml:mspace width="1em"/><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">λ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:mo maxsize="2.45em" minsize="2.45em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:munderover>
<mml:msup>
<mml:mrow>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">X</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">μ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo maxsize="2.45em" minsize="2.45em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msup>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ \frac{\mathrm{d}\log L(\mu ,\lambda )}{\mathrm{d}\hspace{0.1667em}\lambda }=0\hspace{1em}\Rightarrow \hspace{1em}\hat{\lambda }=\Bigg(\frac{1}{n}{\sum \limits_{i=1}^{n}}{\bigg(\frac{1}{{X_{i}}}-\frac{1}{\hat{\mu }}\bigg)\Bigg)^{-1}}.\]]]></tex-math></alternatives>
</disp-formula>
</disp-formula-group></p>
<p>In order to transform Inverse Gaussian function to a probability density function (Fig. <xref rid="j_info1159_fig_001">1</xref>), unknown parameters will be estimated analogously to Eq. (<xref rid="j_info1159_eq_019">19</xref>) and Eq. (<xref rid="j_info1159_eq_020">20</xref>) as: <disp-formula-group id="j_info1159_dg_004">
<disp-formula id="j_info1159_eq_021">
<label>(21)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">μ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">g</mml:mi>
</mml:mrow>
</mml:munderover>
<mml:mi mathvariant="italic">h</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ \hat{\mu }={\sum \limits_{i=1}^{g}}h({\sigma _{i}}){\sigma _{i}},\]]]></tex-math></alternatives>
</disp-formula>
<disp-formula id="j_info1159_eq_022">
<label>(22)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">λ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:mo maxsize="2.45em" minsize="2.45em" fence="true" mathvariant="normal">(</mml:mo>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">g</mml:mi>
</mml:mrow>
</mml:munderover>
<mml:mi mathvariant="italic">h</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">μ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo maxsize="2.45em" minsize="2.45em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msup>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ \hat{\lambda }=\Bigg({\sum \limits_{i=1}^{g}}h({\sigma _{i}}){\bigg(\frac{1}{{\sigma _{i}}}-\frac{1}{\hat{\mu }}\bigg)\Bigg)^{-1}},\]]]></tex-math></alternatives>
</disp-formula>
</disp-formula-group> where <italic>g</italic> represents the number of considered discrete variances of the input signal <inline-formula id="j_info1159_ineq_059"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\sigma _{i}}$]]></tex-math></alternatives></inline-formula> whereas <inline-formula id="j_info1159_ineq_060"><alternatives><mml:math>
<mml:mi mathvariant="italic">h</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$h({\sigma _{i}})$]]></tex-math></alternatives></inline-formula> represents weighting function of the observed set of images. In the end, we have obtained <inline-formula id="j_info1159_ineq_061"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">μ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:mn>11</mml:mn></mml:math><tex-math><![CDATA[$\hat{\mu }=11$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_062"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">λ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:mn>8.4</mml:mn></mml:math><tex-math><![CDATA[$\hat{\lambda }=8.4$]]></tex-math></alternatives></inline-formula> for such set of test images by using described method. Figure <xref rid="j_info1159_fig_001">1</xref> shows both histogram of variances and corresponding Inverse Gaussian model for parameters calculated using Eqs. (<xref rid="j_info1159_eq_021">21</xref>)–(<xref rid="j_info1159_eq_022">22</xref>).</p>
<fig id="j_info1159_fig_001">
<label>Fig. 1</label>
<caption>
<p>Histogram of variances and Inverse Gaussian model.</p>
</caption>
<graphic xlink:href="info1159_g001.jpg"/>
</fig>
<p>In order to calculate the total average bit-rate of the proposed image compression algorithm, we propose two models. The first one is more precise and it is based on the proposed piecewise uniform quantizer design (M1 model) – all discrete thresholds and quantization levels are exploited in the same way as for <italic>PSQNR</italic> calculation. The other model is based on the optimal companding technique and it includes some additional error, but the obtained analytical expressions are much simpler and it is not necessary to memorize thresholds and quantization levels values (M2 model).</p>
<p>M1 model uses piecewise uniform quantizer with compressor function described with Eq. (<xref rid="j_info1159_eq_003">3</xref>) while Golomb–Rice coding is performed as it was described in Section <xref rid="j_info1159_s_003">2.1</xref>. Since that Laplacian source is considered at the entrance, compressor function can be defined as: 
<disp-formula id="j_info1159_eq_023">
<label>(23)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">c</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
<mml:mo>−</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="1.19em" minsize="1.19em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>3</mml:mn><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="1.19em" minsize="1.19em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
<mml:mo>−</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="1.19em" minsize="1.19em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:mn>3</mml:mn><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="1.19em" minsize="1.19em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mspace width="1em"/>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>⩾</mml:mo>
<mml:mn>0</mml:mn>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {c_{1}}(x)=\frac{1-\exp \big(-\frac{\sqrt{2}x}{3\hat{\sigma }}\big)}{1-\exp \big(-\frac{\sqrt{2}{t_{\max }}}{3\hat{\sigma }}\big)},\hspace{1em}x\geqslant 0.\]]]></tex-math></alternatives>
</disp-formula>
</p>
<p>The input signal will be coded using codewords of length <inline-formula id="j_info1159_ineq_063"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">l</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${l_{j}}$]]></tex-math></alternatives></inline-formula> (Eq. (<xref rid="j_info1159_eq_013">13</xref>)) with probabilities: <disp-formula-group id="j_info1159_dg_005">
<disp-formula id="j_info1159_eq_024">
<label>(24)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mn>2</mml:mn>
<mml:msubsup>
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∫</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mn mathvariant="normal">0</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msubsup>
<mml:mi mathvariant="italic">p</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>−</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {P_{0}}(\sigma ,\hat{\sigma })=2{\int _{\mathrm{0}}^{{d_{1}}}}p(x)\hspace{0.1667em}\mathrm{d}x=1-\exp \bigg(-\frac{\sqrt{2}\cdot {d_{1}}}{\sigma }\bigg),\]]]></tex-math></alternatives>
</disp-formula>
<disp-formula id="j_info1159_eq_025">
<label>(25)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mn>2</mml:mn>
<mml:msubsup>
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∫</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>+</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msubsup>
<mml:mi mathvariant="italic">p</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>=</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo>−</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>+</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
<mml:mtr>
<mml:mtd class="align-odd">
<mml:mspace width="1em"/>
<mml:mn>1</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>2</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[\begin{array}{l}\displaystyle {P_{j}}(\sigma ,\hat{\sigma })=2{\int _{{d_{j}}}^{{d_{j+1}}}}p(x)\hspace{0.1667em}\mathrm{d}x=\exp \bigg(-\frac{\sqrt{2}\cdot {d_{j}}}{\sigma }\bigg)-\exp \bigg(-\frac{\sqrt{2}\cdot {d_{j+1}}}{\sigma }\bigg),\\ {} \displaystyle \hspace{1em}1\leqslant j\leqslant S-2,\end{array}\]]]></tex-math></alternatives>
</disp-formula>
<disp-formula id="j_info1159_eq_026">
<label>(26)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mn>2</mml:mn>
<mml:msubsup>
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∫</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msubsup>
<mml:mi mathvariant="italic">p</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>=</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo>−</mml:mo>
<mml:mo movablelimits="false">exp</mml:mo>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {P_{S-1}}(\sigma ,\hat{\sigma })=2{\int _{{d_{S-1}}}^{{d_{S}}}}p(x)\hspace{0.1667em}\mathrm{d}x=\exp \bigg(-\frac{\sqrt{2}\cdot {d_{S-1}}}{\sigma }\bigg)-\exp \bigg(-\frac{\sqrt{2}\cdot {d_{S}}}{\sigma }\bigg).\]]]></tex-math></alternatives>
</disp-formula>
</disp-formula-group> In Eqs. (<xref rid="j_info1159_eq_024">24</xref>)–(<xref rid="j_info1159_eq_026">26</xref>) and Eqs. (<xref rid="j_info1159_eq_030">30</xref>)–(<xref rid="j_info1159_eq_032">32</xref>) with <inline-formula id="j_info1159_ineq_064"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${d_{i}}$]]></tex-math></alternatives></inline-formula> (<inline-formula id="j_info1159_ineq_065"><alternatives><mml:math>
<mml:mn>1</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn></mml:math><tex-math><![CDATA[$1\leqslant i\leqslant S-1$]]></tex-math></alternatives></inline-formula>) are denoted discrete threshold values and it is valid <inline-formula id="j_info1159_ineq_066"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">ω</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">M</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${d_{i}}(\hat{\sigma })={\omega _{i-1,M}}$]]></tex-math></alternatives></inline-formula>, whereas <inline-formula id="j_info1159_ineq_067"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mn>255</mml:mn></mml:math><tex-math><![CDATA[${d_{S}}=255$]]></tex-math></alternatives></inline-formula>. Furthermore, <inline-formula id="j_info1159_ineq_068"><alternatives><mml:math>
<mml:mi mathvariant="italic">p</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$p(x)$]]></tex-math></alternatives></inline-formula> is Laplacian distribution defined with Eq. (<xref rid="j_info1159_eq_001">1</xref>). Finally, the average bit-rate for the single variance is calculated with: 
<disp-formula id="j_info1159_eq_027">
<label>(27)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">GR</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>0</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:munderover>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">l</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {R_{\mathrm{GR}}}={\sum \limits_{j=0}^{S-1}}{l_{j}}\cdot {P_{j}}(\sigma ,\hat{\sigma }),\]]]></tex-math></alternatives>
</disp-formula> 
where <italic>σ</italic> represents the variance of distinction between a pixel value and mean value of all pixels in a block that pixel belongs to and it represents the input signal of the quantizer. Figure <xref rid="j_info1159_fig_002">2</xref> shows the total average bit-rate dependence on the input signal variance. By observing Fig. <xref rid="j_info1159_fig_002">2</xref>, it can be clearly seen that the average bit-rate is strongly dependent on the input signal variance. Consequently, for considering the wider set of signals, the weighting function has to be included. As a result, the total average bit-rate of the model M1 is obtained by using weighting function for averaging and taking into account the required number of bits for transmitting <inline-formula id="j_info1159_ineq_069"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${x_{\mathrm{av}}}$]]></tex-math></alternatives></inline-formula> (Section <xref rid="j_info1159_s_004">2.2</xref>) as: 
<disp-formula id="j_info1159_eq_028">
<label>(28)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">M</mml:mi>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mn>255</mml:mn>
</mml:mrow>
</mml:munderover>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>0</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:munderover>
<mml:mi mathvariant="italic">w</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">l</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>+</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">r</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {R_{\mathrm{M}1}}={\sum \limits_{i=1}^{255}}{\sum \limits_{j=0}^{S-1}}w({\sigma _{i}})\cdot {l_{j}}\cdot {P_{j}}({\sigma _{i}},\hat{\sigma })+{r_{\mathrm{av}}}.\]]]></tex-math></alternatives>
</disp-formula> 
In previous equation, probabilities of segments <inline-formula id="j_info1159_ineq_070"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[${P_{j}}({\sigma _{i}},\hat{\sigma })$]]></tex-math></alternatives></inline-formula> are defined with Eqs. (<xref rid="j_info1159_eq_024">24</xref>)–(<xref rid="j_info1159_eq_026">26</xref>) where thresholds are obtained using compressor function from Eq. (<xref rid="j_info1159_eq_023">23</xref>). Moreover, <inline-formula id="j_info1159_ineq_071"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">r</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${r_{\mathrm{av}}}$]]></tex-math></alternatives></inline-formula> denotes the required number of bits for transmitting <inline-formula id="j_info1159_ineq_072"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${x_{\mathrm{av}}}$]]></tex-math></alternatives></inline-formula> and it is equal to 0.375 [bpp]. For comparison, instead of <inline-formula id="j_info1159_ineq_073"><alternatives><mml:math>
<mml:mi mathvariant="italic">w</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$w({\sigma _{i}})$]]></tex-math></alternatives></inline-formula> which represents Inverse Gaussian distribution, it will be used <inline-formula id="j_info1159_ineq_074"><alternatives><mml:math>
<mml:mi mathvariant="italic">h</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$h({\sigma _{i}})$]]></tex-math></alternatives></inline-formula>, i.e. original histogram. The same averaging will be performed in both Eq. (<xref rid="j_info1159_eq_033">33</xref>) and Eq. (<xref rid="j_info1159_eq_035">35</xref>) in order to obtain the total average bit-rate and <italic>PSQNR</italic> of the M2 model, respectively.</p>
<fig id="j_info1159_fig_002">
<label>Fig. 2</label>
<caption>
<p><inline-formula id="j_info1159_ineq_075"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">GR</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${R_{\mathrm{GR}}}$]]></tex-math></alternatives></inline-formula> dependence on the input signal variance for various values of <inline-formula id="j_info1159_ineq_076"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover></mml:math><tex-math><![CDATA[$\hat{\sigma }$]]></tex-math></alternatives></inline-formula> (M1 model).</p>
</caption>
<graphic xlink:href="info1159_g002.jpg"/>
</fig>
<p>The second model (M2) is based on the optimal companding quantizer application. The compressor function of such quantizer is defined with: 
<disp-formula id="j_info1159_eq_029">
<label>(29)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">c</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>−</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">e</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:msqrt>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msqrt>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>3</mml:mn><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
</mml:mrow>
</mml:msup>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mspace width="1em"/>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>⩾</mml:mo>
<mml:mn>0</mml:mn>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {c_{2}}(x)=1-{e^{-\frac{\sqrt{2}x}{3\hat{\sigma }}}},\hspace{1em}x\geqslant 0.\]]]></tex-math></alternatives>
</disp-formula> 
Probabilities of segments defined with Eq. (<xref rid="j_info1159_eq_029">29</xref>) are: <disp-formula-group id="j_info1159_dg_006">
<disp-formula id="j_info1159_eq_030">
<label>(30)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mn>2</mml:mn>
<mml:msubsup>
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∫</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mn mathvariant="normal">0</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msubsup>
<mml:mi mathvariant="italic">p</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>−</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>−</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">m</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:mn>3</mml:mn><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
</mml:mrow>
</mml:msup>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {P_{0}}(\sigma ,\hat{\sigma })=2{\int _{\mathrm{0}}^{{d_{1}}}}p(x)\hspace{0.1667em}\mathrm{d}x=1-{\bigg(1-\frac{m}{N}\bigg)^{\frac{3\hat{\sigma }}{\sigma }}},\]]]></tex-math></alternatives>
</disp-formula>
<disp-formula id="j_info1159_eq_031">
<label>(31)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mspace width="0.1667em"/><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mn>2</mml:mn>
<mml:msubsup>
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∫</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>+</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msubsup>
<mml:mi mathvariant="italic">p</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>=</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>−</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">m</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:mn>3</mml:mn><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
</mml:mrow>
</mml:msup>
<mml:mo>−</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>−</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>+</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">m</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:mn>3</mml:mn><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
</mml:mrow>
</mml:msup>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
<mml:mtr>
<mml:mtd class="align-odd">
<mml:mspace width="1em"/>
<mml:mn>1</mml:mn>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>⩽</mml:mo>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>2</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[\begin{array}{l}\displaystyle {P_{j}}(\sigma ,\hspace{0.1667em}\hat{\sigma })=2{\int _{{d_{j}}}^{{d_{j+1}}}}p(x)\hspace{0.1667em}\mathrm{d}x={\bigg(1-j\frac{m}{N}\bigg)^{\frac{3\hat{\sigma }}{\sigma }}}-{\bigg(1-(j+1)\frac{m}{N}\bigg)^{\frac{3\hat{\sigma }}{\sigma }}},\\ {} \displaystyle \hspace{1em}1\leqslant j\leqslant S-2,\end{array}\]]]></tex-math></alternatives>
</disp-formula>
<disp-formula id="j_info1159_eq_032">
<label>(32)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true" columnalign="left">
<mml:mtr>
<mml:mtd class="align-odd">
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">σ</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mn>2</mml:mn>
<mml:msubsup>
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∫</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">d</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msubsup>
<mml:mi mathvariant="italic">p</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">d</mml:mi>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>=</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>−</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mi mathvariant="italic">m</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mstyle displaystyle="false">
<mml:mfrac>
<mml:mrow>
<mml:mn>3</mml:mn><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
</mml:mrow>
</mml:msup>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {P_{S-1}}(\sigma ,\hat{\sigma })=2{\int _{{d_{S-1}}}^{{d_{S}}}}p(x)\hspace{0.1667em}\mathrm{d}x={\bigg(1-(S-1)\frac{m}{n}\bigg)^{\frac{3\hat{\sigma }}{\sigma }}}.\]]]></tex-math></alternatives>
</disp-formula>
</disp-formula-group> Finally, the total average bit-rate of the model M2 is obtained by using weighting function for averaging and taking into account the required number of bits <inline-formula id="j_info1159_ineq_077"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">r</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${r_{\mathrm{av}}}$]]></tex-math></alternatives></inline-formula> for transmitting <inline-formula id="j_info1159_ineq_078"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${x_{\mathrm{av}}}$]]></tex-math></alternatives></inline-formula> (Section <xref rid="j_info1159_s_004">2.2</xref>) as: 
<disp-formula id="j_info1159_eq_033">
<label>(33)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">M</mml:mi>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mn>255</mml:mn>
</mml:mrow>
</mml:munderover>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>0</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">S</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:munderover>
<mml:mi mathvariant="italic">w</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">l</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>·</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo>+</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">r</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">av</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {R_{\mathrm{M}2}}={\sum \limits_{i=1}^{255}}{\sum \limits_{j=0}^{S-1}}w({\sigma _{i}})\cdot {l_{j}}\cdot {P_{j}}({\sigma _{i}},\hat{\sigma })+{r_{\mathrm{av}}}.\]]]></tex-math></alternatives>
</disp-formula> 
In Eq. (<xref rid="j_info1159_eq_033">33</xref>), unlike Eq. (<xref rid="j_info1159_eq_028">28</xref>) probabilities of segments <inline-formula id="j_info1159_ineq_079"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">P</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[${P_{j}}({\sigma _{i}},\hat{\sigma })$]]></tex-math></alternatives></inline-formula> are defined with Eqs. (<xref rid="j_info1159_eq_030">30</xref>)–(<xref rid="j_info1159_eq_032">32</xref>) where thresholds are obtained using compressor function from Eq. (<xref rid="j_info1159_eq_029">29</xref>).</p>
<p>Because of the fact that the proposed model describes a kind of a lossy image compression method, some information will be lost irreversibly during the quantization process. In order to measure reconstructed signal quality, we estimate distortion (<italic>D</italic>) that represents a standard measure. It can be calculated with Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>): 
<disp-formula id="j_info1159_eq_034">
<label>(34)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">D</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>2</mml:mn>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:munderover>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">j</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:munderover>
<mml:msup>
<mml:mrow>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>−</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">y</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msup>
<mml:mi mathvariant="italic">P</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ D=2{\sum \limits_{k=1}^{N/2}}{\sum \limits_{j=1}^{{q_{k}}}}{({t_{k,j}}-{y_{k}})^{2}}P({t_{k,j}}),\]]]></tex-math></alternatives>
</disp-formula> 
where <inline-formula id="j_info1159_ineq_080"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">t</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mi mathvariant="italic">j</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">∈</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Z</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${t_{k,j}}\in {Z_{k}}$]]></tex-math></alternatives></inline-formula> are input levels for which the quantization level is <inline-formula id="j_info1159_ineq_081"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">y</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${y_{k}}$]]></tex-math></alternatives></inline-formula>, whereas parameter <inline-formula id="j_info1159_ineq_082"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${q_{k}}$]]></tex-math></alternatives></inline-formula> denotes the number of input levels mapped with <inline-formula id="j_info1159_ineq_083"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">y</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${y_{k}}$]]></tex-math></alternatives></inline-formula>, where <inline-formula id="j_info1159_ineq_084"><alternatives><mml:math>
<mml:msubsup>
<mml:mrow>
<mml:mo largeop="false" movablelimits="false">∑</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mn>2</mml:mn></mml:math><tex-math><![CDATA[${\textstyle\sum _{k=1}^{N/2}}{q_{k}}={N_{0}}/2$]]></tex-math></alternatives></inline-formula>. <inline-formula id="j_info1159_ineq_085"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Z</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${Z_{k}}$]]></tex-math></alternatives></inline-formula> are non-overlapping and non-negative subsets of the set <italic>X</italic> where <inline-formula id="j_info1159_ineq_086"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Z</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mo fence="true" stretchy="false">{</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mo>…</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">q</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:msub>
<mml:mo fence="true" stretchy="false">}</mml:mo></mml:math><tex-math><![CDATA[${Z_{k}}=\{{x_{k1}},\dots ,{x_{k{q_{k}}}}\}$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_087"><alternatives><mml:math>
<mml:msubsup>
<mml:mrow>
<mml:mo largeop="false" movablelimits="false">⋃</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Z</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">k</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">X</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo>+</mml:mo>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\textstyle\bigcup _{k=1}^{N/2}}{Z_{k}}={X^{+}}$]]></tex-math></alternatives></inline-formula>, whereas with <inline-formula id="j_info1159_ineq_088"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">X</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo>+</mml:mo>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${X^{+}}$]]></tex-math></alternatives></inline-formula> is denoted the subset which consists of all non-negative elements of the set <italic>X</italic>. Furthermore, probabilities of discrete input levels are defined with Eq. (<xref rid="j_info1159_eq_002">2</xref>). Finally, taking previous consideration into account, including weighting averaging for observed test grayscale images, reconstructed image quality calculated by using the proposed analytical model will be denoted with <inline-formula id="j_info1159_ineq_089"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">wav</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\mathit{PSQNR}_{\mathrm{wav}}}$]]></tex-math></alternatives></inline-formula>. This measure will be calculated as: 
<disp-formula id="j_info1159_eq_035">
<label>(35)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">wav</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:munderover accentunder="false" accent="false">
<mml:mrow>
<mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mn>255</mml:mn>
</mml:mrow>
</mml:munderover>
<mml:mi mathvariant="italic">w</mml:mi>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">i</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo>
<mml:mi mathvariant="normal">PSQNR</mml:mi>
<mml:mspace width="2.5pt"/>
<mml:mo fence="true" stretchy="false">[</mml:mo>
<mml:mi mathvariant="normal">dB</mml:mi>
<mml:mo fence="true" stretchy="false">]</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ {\mathit{PSQNR}_{\mathrm{wav}}}={\sum \limits_{{\sigma _{i}}=1}^{255}}w({\sigma _{i}})\mathrm{PSQNR}\hspace{2.5pt}[\mathrm{dB}],\]]]></tex-math></alternatives>
</disp-formula> 
where 
<disp-formula id="j_info1159_eq_036">
<label>(36)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>10</mml:mn>
<mml:msub>
<mml:mrow>
<mml:mo movablelimits="false">log</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mn>10</mml:mn>
</mml:mrow>
</mml:msub><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">D</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mspace width="2.5pt"/>
<mml:mo fence="true" stretchy="false">[</mml:mo>
<mml:mi mathvariant="normal">dB</mml:mi>
<mml:mo fence="true" stretchy="false">]</mml:mo>
<mml:mo>.</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ \mathit{PSQNR}=10{\log _{10}}\frac{{x_{\max }^{2}}}{D}\hspace{2.5pt}[\mathrm{dB}].\]]]></tex-math></alternatives>
</disp-formula> 
In previous equation <inline-formula id="j_info1159_ineq_090"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${x_{\max }}$]]></tex-math></alternatives></inline-formula> represents the maximal theoretical pixel value of an image (for grayscale images <inline-formula id="j_info1159_ineq_091"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mn>255</mml:mn></mml:math><tex-math><![CDATA[${x_{\max }}=255$]]></tex-math></alternatives></inline-formula>).</p>
</sec>
<sec id="j_info1159_s_006">
<label>4</label>
<title>Numerical Results and Discussion</title>
<fig id="j_info1159_fig_003">
<label>Fig. 3</label>
<caption>
<p>Standard test grayscale images: (a) Baboon; (b) Bridge; (c) Couple; (d) Jet; (e) Cart; (f) Lena; (g) Pepper; (h) Ship; (i) Street; (j) Church.</p>
</caption>
<graphic xlink:href="info1159_g003.jpg"/>
</fig>
<p>This section provides experimental results of applying the proposed design and comparison with the results obtained by using described analytical model, as well as comparison with the other algorithms. The experiments are done for the set of ten standard test grayscale images (Baboon, Bridge, Couple, Jet, Cart, Lena, Pepper, Ship, Street and Church) of resolution <inline-formula id="j_info1159_ineq_092"><alternatives><mml:math>
<mml:mi mathvariant="italic">n</mml:mi>
<mml:mo>×</mml:mo>
<mml:mi mathvariant="italic">n</mml:mi></mml:math><tex-math><![CDATA[$n\times n$]]></tex-math></alternatives></inline-formula> pixels <inline-formula id="j_info1159_ineq_093"><alternatives><mml:math>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">n</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>512</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$(n=512)$]]></tex-math></alternatives></inline-formula>. These ten images are shown in Fig. <xref rid="j_info1159_fig_003">3</xref>. Unlike theoretical calculations, experimentally measured reconstructed image quality using <italic>PSQNR</italic> can be obtained as: 
<disp-formula id="j_info1159_eq_037">
<label>(37)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>10</mml:mn>
<mml:msub>
<mml:mrow>
<mml:mo movablelimits="false">log</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mn>10</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">(</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:msubsup>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo movablelimits="false">max</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">MSE</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle>
<mml:mo maxsize="2.03em" minsize="2.03em" fence="true" mathvariant="normal">)</mml:mo>
<mml:mspace width="2.5pt"/>
<mml:mo fence="true" stretchy="false">[</mml:mo>
<mml:mi mathvariant="normal">dB</mml:mi>
<mml:mo fence="true" stretchy="false">]</mml:mo>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ \mathit{PSQNR}=10{\log _{10}}\bigg(\frac{{x_{\max }^{2}}}{\mathit{MSE}}\bigg)\hspace{2.5pt}[\mathrm{dB}],\]]]></tex-math></alternatives>
</disp-formula> 
where <italic>MSE</italic> is mean squared error between original and reconstructed images and it is defined with: 
<disp-formula id="j_info1159_eq_038">
<label>(38)</label><alternatives><mml:math display="block">
<mml:mtable displaystyle="true">
<mml:mtr>
<mml:mtd>
<mml:mi mathvariant="italic">MSE</mml:mi>
<mml:mo>=</mml:mo><mml:mstyle displaystyle="true">
<mml:mfrac>
<mml:mrow>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">n</mml:mi>
<mml:mo>×</mml:mo>
<mml:mi mathvariant="italic">n</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mstyle><mml:mstyle displaystyle="true">
<mml:mo largeop="true" movablelimits="false">∑</mml:mo></mml:mstyle>
<mml:msup>
<mml:mrow>
<mml:mo maxsize="1.19em" minsize="1.19em" fence="true" mathvariant="normal">(</mml:mo>
<mml:mi mathvariant="italic">x</mml:mi>
<mml:mo>−</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">x</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mo>∗</mml:mo>
</mml:mrow>
</mml:msup>
<mml:mo maxsize="1.19em" minsize="1.19em" fence="true" mathvariant="normal">)</mml:mo>
</mml:mrow>
<mml:mrow>
<mml:mn>2</mml:mn>
</mml:mrow>
</mml:msup>
<mml:mo mathvariant="normal">,</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable></mml:math><tex-math><![CDATA[\[ \mathit{MSE}=\frac{1}{n\times n}\sum {\big(x-{x^{\ast }}\big)^{2}},\]]]></tex-math></alternatives>
</disp-formula> 
where summation is done for all pixels of an image.</p>
<p>First of all, we provide a detailed analysis for the case when the number of quantization levels is <inline-formula id="j_info1159_ineq_094"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>16</mml:mn></mml:math><tex-math><![CDATA[$N=16$]]></tex-math></alternatives></inline-formula> and the number of segments is <inline-formula id="j_info1159_ineq_095"><alternatives><mml:math>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>8</mml:mn></mml:math><tex-math><![CDATA[$L=8$]]></tex-math></alternatives></inline-formula>. This case is chosen since it has low complexity and Golomb–Rice coding has great influence <inline-formula id="j_info1159_ineq_096"><alternatives><mml:math>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mi mathvariant="italic">k</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>2</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$(k=2)$]]></tex-math></alternatives></inline-formula> that will be discussed later.</p>
<p>Figure <xref rid="j_info1159_fig_004">4</xref> shows theoretical comparison of <italic>PSQNR</italic> values, obtained by using the exact histogram of variances and the proposed closed-form solutions based on Inverse Gaussian distribution.</p>
<fig id="j_info1159_fig_004">
<label>Fig. 4</label>
<caption>
<p><italic>PSQNR</italic> – analytical model.</p>
</caption>
<graphic xlink:href="info1159_g004.jpg"/>
</fig>
<p>By observing Fig. <xref rid="j_info1159_fig_004">4</xref>, it can be seen that with increasing the value of discrete designing variance <inline-formula id="j_info1159_ineq_097"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover></mml:math><tex-math><![CDATA[$\hat{\sigma }$]]></tex-math></alternatives></inline-formula>, matching between the proposed closed-form solutions and analytical model that exploits the exact histogram is getting better and almost equal. However, a mismatch that can be noticed occurs due to variance mismatch within two quantization steps (Perić <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_021">2015</xref>) as well as it occurs to a non-ideal modelling using Inverse Gaussian distribution for low variances. Finally, the highest <italic>PSQNR</italic> is provided for such value of <inline-formula id="j_info1159_ineq_098"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover></mml:math><tex-math><![CDATA[$\hat{\sigma }$]]></tex-math></alternatives></inline-formula> that is almost equal to the corresponding mean value of the modelled input signal (<inline-formula id="j_info1159_ineq_099"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">μ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:mn>11</mml:mn></mml:math><tex-math><![CDATA[$\hat{\mu }=11$]]></tex-math></alternatives></inline-formula>), justifying the term “designing variance” for parameter <inline-formula id="j_info1159_ineq_100"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover></mml:math><tex-math><![CDATA[$\hat{\sigma }$]]></tex-math></alternatives></inline-formula> (Perić <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_021">2015</xref>).</p>
<p>By applying Eq. (<xref rid="j_info1159_eq_028">28</xref>) and Eq. (<xref rid="j_info1159_eq_033">33</xref>) for the observed case (<inline-formula id="j_info1159_ineq_101"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>16</mml:mn></mml:math><tex-math><![CDATA[$N=16$]]></tex-math></alternatives></inline-formula>, <inline-formula id="j_info1159_ineq_102"><alternatives><mml:math>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>8</mml:mn></mml:math><tex-math><![CDATA[$L=8$]]></tex-math></alternatives></inline-formula>), we have obtained theoretical values for the total average bit-rate. These results are shown in Fig. <xref rid="j_info1159_fig_005">5</xref>, and the performance of the proposed analytical model is compared with the values obtained by using the exact histogram values. By observing Fig. <xref rid="j_info1159_fig_005">5</xref>, it can be seen that the proposed closed-form expressions achieve great matching with the corresponding values obtained by using the exact histogram. Also, it can be noticed that the values obtained by using piecewise uniform quantizer for calculations (M1 model) predict higher bit-rates in comparison to the corresponding values obtained by using optimal compandor (M2 model). Later, Table <xref rid="j_info1159_tab_001">1</xref> will show that M1 model is more precise.</p>
<fig id="j_info1159_fig_005">
<label>Fig. 5</label>
<caption>
<p>The total average bit-rate – analytical model.</p>
</caption>
<graphic xlink:href="info1159_g005.jpg"/>
</fig>
<table-wrap id="j_info1159_tab_001">
<label>Table 1</label>
<caption>
<p>Comparison of experimental and theoretical results – the total average bit-rate and <italic>PSQNR</italic>.</p>
</caption>
<table>
<thead>
<tr>
<td style="vertical-align: middle; text-align: left; border-top: solid thin"/>
<td colspan="5" style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><italic>The total average bit-rate</italic> [bpp]</td>
<td colspan="3" style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><italic>PSQNR</italic> [dB]</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td colspan="2" style="vertical-align: top; text-align: left; border-bottom: solid thin">M1 model</td>
<td colspan="2" style="vertical-align: top; text-align: left; border-bottom: solid thin">M2 model</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">Experiments</td>
<td colspan="2" style="vertical-align: top; text-align: left; border-bottom: solid thin">Theoretical model</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">Experiments</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_103"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover></mml:math><tex-math><![CDATA[$\hat{\sigma }$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_104"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">hist</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{hist}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_105"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inv</mml:mi>
<mml:mo>.</mml:mo>
<mml:mspace width="2.5pt"/>
<mml:mi mathvariant="normal">Gaus</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{Inv}.\hspace{2.5pt}\mathrm{Gaus}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_106"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">hist</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{hist}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_107"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inv</mml:mi>
<mml:mo>.</mml:mo>
<mml:mspace width="2.5pt"/>
<mml:mi mathvariant="normal">Gaus</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{Inv}.\hspace{2.5pt}\mathrm{Gaus}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_108"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_109"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">hist</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{hist}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_110"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inv</mml:mi>
<mml:mo>.</mml:mo>
<mml:mspace width="2.5pt"/>
<mml:mi mathvariant="normal">Gauss</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{Inv}.\hspace{2.5pt}\mathrm{Gauss}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_111"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula></td>
</tr>
</thead>
<tbody>
<tr>
<td style="vertical-align: top; text-align: left">12</td>
<td style="vertical-align: top; text-align: left">3.8720</td>
<td style="vertical-align: top; text-align: left">3.8643</td>
<td style="vertical-align: top; text-align: left">3.7985</td>
<td style="vertical-align: top; text-align: left">3.7924</td>
<td style="vertical-align: top; text-align: left">3.9257</td>
<td style="vertical-align: top; text-align: left">42.8945</td>
<td style="vertical-align: top; text-align: left">43.8275</td>
<td style="vertical-align: top; text-align: left">40.1194</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left">14</td>
<td style="vertical-align: top; text-align: left">3.8044</td>
<td style="vertical-align: top; text-align: left">3.7986</td>
<td style="vertical-align: top; text-align: left">3.7311</td>
<td style="vertical-align: top; text-align: left">3.7276</td>
<td style="vertical-align: top; text-align: left">3.8987</td>
<td style="vertical-align: top; text-align: left">42.9537</td>
<td style="vertical-align: top; text-align: left">43.8193</td>
<td style="vertical-align: top; text-align: left">40.8063</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left">15</td>
<td style="vertical-align: top; text-align: left">3.7547</td>
<td style="vertical-align: top; text-align: left">3.7501</td>
<td style="vertical-align: top; text-align: left">3.7031</td>
<td style="vertical-align: top; text-align: left">3.7009</td>
<td style="vertical-align: top; text-align: left">3.8694</td>
<td style="vertical-align: top; text-align: left">41.6060</td>
<td style="vertical-align: top; text-align: left">42.3193</td>
<td style="vertical-align: top; text-align: left">41.3739</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left">16</td>
<td style="vertical-align: top; text-align: left">3.7171</td>
<td style="vertical-align: top; text-align: left">3.7135</td>
<td style="vertical-align: top; text-align: left">3.6781</td>
<td style="vertical-align: top; text-align: left">3.6772</td>
<td style="vertical-align: top; text-align: left">3.8360</td>
<td style="vertical-align: top; text-align: left">41.6322</td>
<td style="vertical-align: top; text-align: left">42.0689</td>
<td style="vertical-align: top; text-align: left">41.0867</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left">24</td>
<td style="vertical-align: top; text-align: left">3.5793</td>
<td style="vertical-align: top; text-align: left">3.5842</td>
<td style="vertical-align: top; text-align: left">3.5471</td>
<td style="vertical-align: top; text-align: left">3.5546</td>
<td style="vertical-align: top; text-align: left">3.6572</td>
<td style="vertical-align: top; text-align: left">40.0148</td>
<td style="vertical-align: top; text-align: left">39.9896</td>
<td style="vertical-align: top; text-align: left">40.9050</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">30</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">3.5121</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">3.5220</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">3.4931</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">3.5050</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">3.5733</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">38.3763</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">38.2929</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">39.6773</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>In order to compare obtained results with existing models, firstly we provide comparison with the results published in Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>). The corresponding theoretical results from the aforementioned paper, for fixed piecewise uniform quantizer application, are <inline-formula id="j_info1159_ineq_112"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">inf</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mn>35.32</mml:mn></mml:math><tex-math><![CDATA[${\mathit{PSQNR}_{\mathrm{inf}}}=35.32$]]></tex-math></alternatives></inline-formula> dB and <inline-formula id="j_info1159_ineq_113"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">inf</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mn>4.375</mml:mn></mml:math><tex-math><![CDATA[${R_{\mathrm{inf}}}=4.375$]]></tex-math></alternatives></inline-formula> bpp. Considering M1 model and taking into account that <italic>PSQNR</italic> values increase/decrease for 5.5 dB by changing the bit-rate for 1 bit for medium bit-rates, the proposed system achieves theoretical gain of <italic>PSQNR/R</italic> ratio in comparison to the corresponding design presented in Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>) and it is shown in Fig. <xref rid="j_info1159_fig_006">6</xref>.</p>
<p>It can be clearly seen that the proposed modelling provides gain estimation very close to the one obtained by using the exact histogram (<inline-formula id="j_info1159_ineq_114"><alternatives><mml:math>
<mml:mi mathvariant="normal">Δ</mml:mi>
<mml:mo mathvariant="normal">&lt;</mml:mo>
<mml:mn>1</mml:mn></mml:math><tex-math><![CDATA[$\Delta <1$]]></tex-math></alternatives></inline-formula> dB) for the values of discrete variance <inline-formula id="j_info1159_ineq_115"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal">&gt;</mml:mo>
<mml:mn>12</mml:mn></mml:math><tex-math><![CDATA[$\hat{\sigma }>12$]]></tex-math></alternatives></inline-formula>. Moreover, the difference decreases by increasing <inline-formula id="j_info1159_ineq_116"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover></mml:math><tex-math><![CDATA[$\hat{\sigma }$]]></tex-math></alternatives></inline-formula> and the best performance is achieved in the range <inline-formula id="j_info1159_ineq_117"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo stretchy="false">∈</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mn>14</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>16</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$\hat{\sigma }\in (14,16)$]]></tex-math></alternatives></inline-formula>. It should be noted that for <inline-formula id="j_info1159_ineq_118"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo mathvariant="normal">&gt;</mml:mo>
<mml:mn>22</mml:mn></mml:math><tex-math><![CDATA[$\hat{\sigma }>22$]]></tex-math></alternatives></inline-formula> difference is lower than 0.25 dB, that is almost negligible.</p>
<fig id="j_info1159_fig_006">
<label>Fig. 6</label>
<caption>
<p>Theoretical gain – comparison with Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>) <inline-formula id="j_info1159_ineq_119"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>16</mml:mn></mml:math><tex-math><![CDATA[$N=16$]]></tex-math></alternatives></inline-formula>, <inline-formula id="j_info1159_ineq_120"><alternatives><mml:math>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>8</mml:mn></mml:math><tex-math><![CDATA[$L=8$]]></tex-math></alternatives></inline-formula>.</p>
</caption>
<graphic xlink:href="info1159_g006.jpg"/>
</fig>
<p>Consequently, taking into account that the best performance is predicted in the range <inline-formula id="j_info1159_ineq_121"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo stretchy="false">∈</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mn>14</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>16</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$\hat{\sigma }\in (14,16)$]]></tex-math></alternatives></inline-formula> we have decided to provide experimental results by using the proposed algorithm for <inline-formula id="j_info1159_ineq_122"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo stretchy="false">∈</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mn>12</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>14</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>15</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>16</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>24</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>30</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$\hat{\sigma }\in (12,14,15,16,24,30)$]]></tex-math></alternatives></inline-formula> and these results and corresponding comparison with the theoretical models are shown in Table <xref rid="j_info1159_tab_001">1</xref>.</p>
<p>By observing the results shown in Table <xref rid="j_info1159_tab_001">1</xref>, it can be concluded that the total average bit-rate decreases by increasing discrete designing variance <inline-formula id="j_info1159_ineq_123"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover></mml:math><tex-math><![CDATA[$\hat{\sigma }$]]></tex-math></alternatives></inline-formula>. It is evident that theoretical results obtained by using the proposed solutions (<inline-formula id="j_info1159_ineq_124"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inv</mml:mi>
<mml:mo>.</mml:mo>
<mml:mspace width="2.5pt"/>
<mml:mi mathvariant="normal">Gaus</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{Inv}.\hspace{2.5pt}\mathrm{Gaus}}}$]]></tex-math></alternatives></inline-formula>) achieve performance very close to the one obtained by using the exact histogram (<inline-formula id="j_info1159_ineq_125"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">hist</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{hist}}}$]]></tex-math></alternatives></inline-formula>) as well as very close to the experimental results (<inline-formula id="j_info1159_ineq_126"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula>). However, the results obtained by using M1 model are closer to the experimental results as it could be expected because of its higher complexity. Next, the results tell us that for <inline-formula id="j_info1159_ineq_127"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo stretchy="false">∈</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mn>15</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>16</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>24</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$\hat{\sigma }\in (15,16,24)$]]></tex-math></alternatives></inline-formula> the absolute errors between the experimental results <inline-formula id="j_info1159_ineq_128"><alternatives><mml:math>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="italic">ex</mml:mi>
</mml:mrow>
</mml:msup>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$({\mathit{PSQNR}^{\mathit{ex}}})$]]></tex-math></alternatives></inline-formula> and the proposed closed-form solutions <inline-formula id="j_info1159_ineq_129"><alternatives><mml:math>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inv</mml:mi>
<mml:mo>.</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">Gauss</mml:mi>
</mml:mrow>
</mml:msup>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$({\mathit{PSQNR}^{\mathrm{Inv}.\hspace{0.1667em}\mathrm{Gauss}}})$]]></tex-math></alternatives></inline-formula> are less than 1 dB. Furthermore, we can conclude that theoretical results follow the changes of experimental results. This means that the proposed model provides satisfactory performance estimation within the range <inline-formula id="j_info1159_ineq_130"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo stretchy="false">∈</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mn>15</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>30</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$\hat{\sigma }\in (15,30)$]]></tex-math></alternatives></inline-formula>. Taking into account results from Fig. <xref rid="j_info1159_fig_006">6</xref>, it can be concluded that the best system performance is achieved for <inline-formula id="j_info1159_ineq_131"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:mn>15</mml:mn></mml:math><tex-math><![CDATA[$\hat{\sigma }=15$]]></tex-math></alternatives></inline-formula>. Moreover, since for <inline-formula id="j_info1159_ineq_132"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo stretchy="false">∈</mml:mo>
<mml:mo mathvariant="normal" fence="true" stretchy="false">(</mml:mo>
<mml:mn>15</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>16</mml:mn>
<mml:mo mathvariant="normal">,</mml:mo>
<mml:mn>24</mml:mn>
<mml:mo mathvariant="normal" fence="true" stretchy="false">)</mml:mo></mml:math><tex-math><![CDATA[$\hat{\sigma }\in (15,16,24)$]]></tex-math></alternatives></inline-formula> higher accuracy is provided, for all the other numbers of quantization segments <italic>L</italic> and quantization levels <italic>N</italic>, we will provide analysis that incorporates those three values.</p>
<p>Table <xref rid="j_info1159_tab_002">2</xref> shows theoretical and experimental performance for the various values of system parameters.</p>
<table-wrap id="j_info1159_tab_002">
<label>Table 2</label>
<caption>
<p>The total average bit-rate and <italic>PSQNR</italic> for various values of <italic>N</italic> and <italic>L</italic>.</p>
</caption>
<table>
<thead>
<tr>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><italic>N</italic></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><italic>L</italic></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_133"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover></mml:math><tex-math><![CDATA[$\hat{\sigma }$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_134"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">hist</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{hist}}}$]]></tex-math></alternatives></inline-formula> [bpp]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_135"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inv</mml:mi>
<mml:mo>.</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">Gauss</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{Inv}.\hspace{0.1667em}\mathrm{Gauss}}}$]]></tex-math></alternatives></inline-formula> [bpp]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_136"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula> [bpp]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_137"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">hist</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{hist}}}$]]></tex-math></alternatives></inline-formula> [dB]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_138"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inv</mml:mi>
<mml:mo>.</mml:mo>
<mml:mspace width="0.1667em"/>
<mml:mi mathvariant="normal">Gauss</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{Inv}.\hspace{0.1667em}\mathrm{Gauss}}}$]]></tex-math></alternatives></inline-formula> [dB]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_139"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula> [dB]</td>
</tr>
</thead>
<tbody>
<tr>
<td style="vertical-align: top; text-align: left">16</td>
<td style="vertical-align: top; text-align: left">4</td>
<td style="vertical-align: top; text-align: left">15</td>
<td style="vertical-align: top; text-align: left">4.6771</td>
<td style="vertical-align: top; text-align: left">4.6838</td>
<td style="vertical-align: top; text-align: left">4.4271</td>
<td style="vertical-align: top; text-align: left">40.8748</td>
<td style="vertical-align: top; text-align: left">41.5134</td>
<td style="vertical-align: top; text-align: left">40.8935</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">24</td>
<td style="vertical-align: top; text-align: left">4.5268</td>
<td style="vertical-align: top; text-align: left">4.5427</td>
<td style="vertical-align: top; text-align: left">4.3802</td>
<td style="vertical-align: top; text-align: left">37.6488</td>
<td style="vertical-align: top; text-align: left">38.0756</td>
<td style="vertical-align: top; text-align: left">39.1120</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">30</td>
<td style="vertical-align: top; text-align: left">4.4748</td>
<td style="vertical-align: top; text-align: left">4.4952</td>
<td style="vertical-align: top; text-align: left">4.3654</td>
<td style="vertical-align: top; text-align: left">36.1054</td>
<td style="vertical-align: top; text-align: left">36.4777</td>
<td style="vertical-align: top; text-align: left">37.8268</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left">32</td>
<td style="vertical-align: top; text-align: left">4</td>
<td style="vertical-align: top; text-align: left">15</td>
<td style="vertical-align: top; text-align: left">5.6553</td>
<td style="vertical-align: top; text-align: left">5.6720</td>
<td style="vertical-align: top; text-align: left">5.4179</td>
<td style="vertical-align: top; text-align: left">45.6322</td>
<td style="vertical-align: top; text-align: left">46.3163</td>
<td style="vertical-align: top; text-align: left">45.4106</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">24</td>
<td style="vertical-align: top; text-align: left">5.5130</td>
<td style="vertical-align: top; text-align: left">5.5379</td>
<td style="vertical-align: top; text-align: left">5.3710</td>
<td style="vertical-align: top; text-align: left">43.3062</td>
<td style="vertical-align: top; text-align: left">43.8258</td>
<td style="vertical-align: top; text-align: left">44.2079</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">30</td>
<td style="vertical-align: top; text-align: left">5.4549</td>
<td style="vertical-align: top; text-align: left">5.4850</td>
<td style="vertical-align: top; text-align: left">5.3563</td>
<td style="vertical-align: top; text-align: left">42.5318</td>
<td style="vertical-align: top; text-align: left">43.0339</td>
<td style="vertical-align: top; text-align: left">43.0973</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">8</td>
<td style="vertical-align: top; text-align: left">15</td>
<td style="vertical-align: top; text-align: left">4.7340</td>
<td style="vertical-align: top; text-align: left">4.7393</td>
<td style="vertical-align: top; text-align: left">4.6939</td>
<td style="vertical-align: top; text-align: left">46.6803</td>
<td style="vertical-align: top; text-align: left">47.4487</td>
<td style="vertical-align: top; text-align: left">46.3190</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">24</td>
<td style="vertical-align: top; text-align: left">4.5525</td>
<td style="vertical-align: top; text-align: left">4.5670</td>
<td style="vertical-align: top; text-align: left">4.5379</td>
<td style="vertical-align: top; text-align: left">45.4434</td>
<td style="vertical-align: top; text-align: left">45.9896</td>
<td style="vertical-align: top; text-align: left">45.2847</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">30</td>
<td style="vertical-align: top; text-align: left">4.4987</td>
<td style="vertical-align: top; text-align: left">4.5177</td>
<td style="vertical-align: top; text-align: left">4.4839</td>
<td style="vertical-align: top; text-align: left">43.7051</td>
<td style="vertical-align: top; text-align: left">44.1817</td>
<td style="vertical-align: top; text-align: left">45.3434</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">16</td>
<td style="vertical-align: top; text-align: left">15</td>
<td style="vertical-align: top; text-align: left">4.4131</td>
<td style="vertical-align: top; text-align: left">4.3769</td>
<td style="vertical-align: top; text-align: left">4.3364</td>
<td style="vertical-align: top; text-align: left">46.7983</td>
<td style="vertical-align: top; text-align: left">47.5733</td>
<td style="vertical-align: top; text-align: left">46.6716</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">24</td>
<td style="vertical-align: top; text-align: left">4.0102</td>
<td style="vertical-align: top; text-align: left">3.9907</td>
<td style="vertical-align: top; text-align: left">3.9568</td>
<td style="vertical-align: top; text-align: left">45.5497</td>
<td style="vertical-align: top; text-align: left">46.0716</td>
<td style="vertical-align: top; text-align: left">45.8032</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"/>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"/>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">30</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">3.8550</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">3.8423</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">3.8180</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">43.8754</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">44.3203</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">45.8192</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>As it was expected, <italic>PSQNR</italic> value increases by increasing the number of quantization levels. Moreover, for fixed number of quantization levels, <italic>PSQNR</italic> value increases by increasing the number of segments <italic>L</italic>. In order to compare compression quality for aforementioned quantizers, the total average bit-rate depending on the number of quantization levels and the number of quantization segments is also shown.</p>
<sec id="j_info1159_s_007">
<label>4.1</label>
<title>Comparison of the Obtained Results with Other Models</title>
<p>This section provides comparison of the obtained results with the corresponding one presented in other papers (Savic <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>; Mathews and Nair, <xref ref-type="bibr" rid="j_info1159_ref_016">2015</xref>; Savic <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_026">2015</xref>). The results are compared with the similar techniques as well as with the other that incorporates adaptation and linear prediction. In Table <xref rid="j_info1159_tab_003">3</xref> are shown the average <italic>PSQNR</italic> and the total average bit-rate values for three standard test grayscale images (Lena, Street and Boat) processed by the proposed algorithm and they are compared with the corresponding results published in Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>), denoted with <inline-formula id="j_info1159_ineq_140"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inf</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{Inf}}}$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_141"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inf</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{Inf}}}$]]></tex-math></alternatives></inline-formula>.</p>
<p>In Table <xref rid="j_info1159_tab_003">3</xref>, with <inline-formula id="j_info1159_ineq_142"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">Gain</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">d</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{Gain}^{\mathrm{d}}}$]]></tex-math></alternatives></inline-formula> is denoted experimental gain of the proposed model produced by the proposed quantizer design implementation, whereas <inline-formula id="j_info1159_ineq_143"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">Gain</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">tot</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{Gain}^{\mathrm{tot}}}$]]></tex-math></alternatives></inline-formula> is the total gain that is obtained by both different quantizer design and variable length Golomb–Rice coding, i.e. achieved by the proposed encoder design. It should be noted that Golomb–Rice coding is not suitable in the case when <inline-formula id="j_info1159_ineq_144"><alternatives><mml:math>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>4</mml:mn></mml:math><tex-math><![CDATA[$L=4$]]></tex-math></alternatives></inline-formula> since the total average bit-rate is higher than the bit-rate required for fixed quantizers (Savic <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>). However, it can be clearly seen that the proposed encoder design provides gain for the all observed system parameters and that the gain reaches 11.56 dB.</p>
<table-wrap id="j_info1159_tab_003">
<label>Table 3</label>
<caption>
<p>The comparison of the system performance with the results published in Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>) for three standard test grayscale images (Lena, Street and Boat).</p>
</caption>
<table>
<thead>
<tr>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><italic>N</italic></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><italic>L</italic></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_145"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover></mml:math><tex-math><![CDATA[$\hat{\sigma }$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_146"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula> [dB]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_147"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula> [bpp]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_148"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inf</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{Inf}}}$]]></tex-math></alternatives></inline-formula> [dB]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_149"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Inf</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{Inf}}}$]]></tex-math></alternatives></inline-formula> [bpp]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_150"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">Gain</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">d</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{Gain}^{\mathrm{d}}}$]]></tex-math></alternatives></inline-formula> [dB]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_151"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">Gain</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">total</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{Gain}^{\mathrm{total}}}$]]></tex-math></alternatives></inline-formula> [dB]</td>
</tr>
</thead>
<tbody>
<tr>
<td style="vertical-align: top; text-align: left">16</td>
<td style="vertical-align: top; text-align: left">4</td>
<td style="vertical-align: top; text-align: left">15</td>
<td style="vertical-align: top; text-align: left">42.045</td>
<td style="vertical-align: top; text-align: left">4.504</td>
<td style="vertical-align: top; text-align: left">33.037</td>
<td style="vertical-align: top; text-align: left">4.375</td>
<td style="vertical-align: top; text-align: left">9.008</td>
<td style="vertical-align: top; text-align: left">8.299</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">24</td>
<td style="vertical-align: top; text-align: left">39.963</td>
<td style="vertical-align: top; text-align: left">4.435</td>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">6.926</td>
<td style="vertical-align: top; text-align: left">6.596</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">30</td>
<td style="vertical-align: top; text-align: left">38.468</td>
<td style="vertical-align: top; text-align: left">4.412</td>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">5.431</td>
<td style="vertical-align: top; text-align: left">5.228</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">8</td>
<td style="vertical-align: top; text-align: left">15</td>
<td style="vertical-align: top; text-align: left">42.831</td>
<td style="vertical-align: top; text-align: left">3.869</td>
<td style="vertical-align: top; text-align: left">35.817</td>
<td style="vertical-align: top; text-align: left">4.375</td>
<td style="vertical-align: top; text-align: left">7.014</td>
<td style="vertical-align: top; text-align: left">9.797</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">24</td>
<td style="vertical-align: top; text-align: left">41.895</td>
<td style="vertical-align: top; text-align: left">3.657</td>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">6.078</td>
<td style="vertical-align: top; text-align: left">10.027</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">30</td>
<td style="vertical-align: top; text-align: left">40.443</td>
<td style="vertical-align: top; text-align: left">3.573</td>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">4.626</td>
<td style="vertical-align: top; text-align: left">9.037</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left">32</td>
<td style="vertical-align: top; text-align: left">4</td>
<td style="vertical-align: top; text-align: left">15</td>
<td style="vertical-align: top; text-align: left">46.556</td>
<td style="vertical-align: top; text-align: left">5.496</td>
<td style="vertical-align: top; text-align: left">40.058</td>
<td style="vertical-align: top; text-align: left">5.375</td>
<td style="vertical-align: top; text-align: left">6.498</td>
<td style="vertical-align: top; text-align: left">5.833</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">24</td>
<td style="vertical-align: top; text-align: left">45.311</td>
<td style="vertical-align: top; text-align: left">5.432</td>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">5.253</td>
<td style="vertical-align: top; text-align: left">4.940</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">30</td>
<td style="vertical-align: top; text-align: left">44.065</td>
<td style="vertical-align: top; text-align: left">5.408</td>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">4.007</td>
<td style="vertical-align: top; text-align: left">3.826</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">8</td>
<td style="vertical-align: top; text-align: left">15</td>
<td style="vertical-align: top; text-align: left">47.778</td>
<td style="vertical-align: top; text-align: left">4.704</td>
<td style="vertical-align: top; text-align: left">42.380</td>
<td style="vertical-align: top; text-align: left">5.375</td>
<td style="vertical-align: top; text-align: left">5.398</td>
<td style="vertical-align: top; text-align: left">9.089</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">24</td>
<td style="vertical-align: top; text-align: left">46.177</td>
<td style="vertical-align: top; text-align: left">4.490</td>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">3.797</td>
<td style="vertical-align: top; text-align: left">8.665</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">30</td>
<td style="vertical-align: top; text-align: left">46.225</td>
<td style="vertical-align: top; text-align: left">4.490</td>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">3.845</td>
<td style="vertical-align: top; text-align: left">8.713</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">16</td>
<td style="vertical-align: top; text-align: left">15</td>
<td style="vertical-align: top; text-align: left">48.214</td>
<td style="vertical-align: top; text-align: left">4.631</td>
<td style="vertical-align: top; text-align: left">42.640</td>
<td style="vertical-align: top; text-align: left">5.375</td>
<td style="vertical-align: top; text-align: left">5.574</td>
<td style="vertical-align: top; text-align: left">9.666</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">24</td>
<td style="vertical-align: top; text-align: left">46.502</td>
<td style="vertical-align: top; text-align: left">4.162</td>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left"/>
<td style="vertical-align: top; text-align: left">3.862</td>
<td style="vertical-align: top; text-align: left">10.534</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"/>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"/>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">30</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">46.494</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">3.974</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"/>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin"/>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">3.854</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">11.560</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Unlike conventionally designed classic block truncation coding models, there are a lot of modifications designed to provide specific performance. In Mathews and Nair (<xref ref-type="bibr" rid="j_info1159_ref_016">2015</xref>) an adaptive block truncation technique, that uses edge-based quantization in order to improve compression ratio and preserve reconstructed image quality was proposed. The main idea of this kind of adaptation is to adjust quantization levels for block processing depending on whether the observed block contains an edge or not. The authors have designed the system that preserves the edge information, since artifacts near edges are a common phenomenon and edges are considered as a very important feature of an image. From Mathews and Nair (<xref ref-type="bibr" rid="j_info1159_ref_016">2015</xref>) it can be seen that their model achieves <inline-formula id="j_info1159_ineq_152"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>36.9919</mml:mn></mml:math><tex-math><![CDATA[$\mathit{PSQNR}=36.9919$]]></tex-math></alternatives></inline-formula> dB and compression ratio <inline-formula id="j_info1159_ineq_153"><alternatives><mml:math>
<mml:mi mathvariant="italic">CR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>3.1087</mml:mn></mml:math><tex-math><![CDATA[$\mathit{CR}=3.1087$]]></tex-math></alternatives></inline-formula> for the standard test grayscale image Lena whereas <inline-formula id="j_info1159_ineq_154"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>36.3085</mml:mn></mml:math><tex-math><![CDATA[$\mathit{PSQNR}=36.3085$]]></tex-math></alternatives></inline-formula> dB and compression ratio <inline-formula id="j_info1159_ineq_155"><alternatives><mml:math>
<mml:mi mathvariant="italic">CR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>2.8586</mml:mn></mml:math><tex-math><![CDATA[$\mathit{CR}=2.8586$]]></tex-math></alternatives></inline-formula> for the standard test grayscale image Street is achieved. If we calculate the total average bit-rate from those <inline-formula id="j_info1159_ineq_156"><alternatives><mml:math>
<mml:mi mathvariant="italic">CR</mml:mi></mml:math><tex-math><![CDATA[$\mathit{CR}$]]></tex-math></alternatives></inline-formula> values, we get <inline-formula id="j_info1159_ineq_157"><alternatives><mml:math>
<mml:mi mathvariant="italic">R</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>2.5734</mml:mn></mml:math><tex-math><![CDATA[$R=2.5734$]]></tex-math></alternatives></inline-formula> bpp for Lena and <inline-formula id="j_info1159_ineq_158"><alternatives><mml:math>
<mml:mi mathvariant="italic">R</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>2.7986</mml:mn></mml:math><tex-math><![CDATA[$R=2.7986$]]></tex-math></alternatives></inline-formula> bpp for Street. On the other hand, by processing these images with the proposed algorithm for the case <inline-formula id="j_info1159_ineq_159"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>32</mml:mn></mml:math><tex-math><![CDATA[$N=32$]]></tex-math></alternatives></inline-formula>, <inline-formula id="j_info1159_ineq_160"><alternatives><mml:math>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>16</mml:mn></mml:math><tex-math><![CDATA[$L=16$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_161"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:mn>30</mml:mn></mml:math><tex-math><![CDATA[$\hat{\sigma }=30$]]></tex-math></alternatives></inline-formula> (the highest obtained gain) we obtain the following performance – Lena: <inline-formula id="j_info1159_ineq_162"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>46.624</mml:mn></mml:math><tex-math><![CDATA[$\mathit{PSQNR}=46.624$]]></tex-math></alternatives></inline-formula> dB and <inline-formula id="j_info1159_ineq_163"><alternatives><mml:math>
<mml:mi mathvariant="italic">R</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>3.721</mml:mn></mml:math><tex-math><![CDATA[$R=3.721$]]></tex-math></alternatives></inline-formula> bpp; Street: <inline-formula id="j_info1159_ineq_164"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>46.258</mml:mn></mml:math><tex-math><![CDATA[$\mathit{PSQNR}=46.258$]]></tex-math></alternatives></inline-formula> dB, <inline-formula id="j_info1159_ineq_165"><alternatives><mml:math>
<mml:mi mathvariant="italic">R</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>3.852</mml:mn></mml:math><tex-math><![CDATA[$R=3.852$]]></tex-math></alternatives></inline-formula> bpp. By comparing these results with the corresponding one published in Mathews and Nair (<xref ref-type="bibr" rid="j_info1159_ref_016">2015</xref>), we can conclude that our model achieves higher <inline-formula id="j_info1159_ineq_166"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi></mml:math><tex-math><![CDATA[$\mathit{PSQNR}$]]></tex-math></alternatives></inline-formula> but also higher average bit-rate. In order to compare the <inline-formula id="j_info1159_ineq_167"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mi mathvariant="italic">R</mml:mi></mml:math><tex-math><![CDATA[$\mathit{PSQNR}/R$]]></tex-math></alternatives></inline-formula> ratio, it should be noted that <inline-formula id="j_info1159_ineq_168"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi></mml:math><tex-math><![CDATA[$\mathit{PSQNR}$]]></tex-math></alternatives></inline-formula> value increases/decreases for 5.5 dB by changing the total average bit-rate for 1 bpp (Savic <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>). This way, by transforming the results from Mathews and Nair (<xref ref-type="bibr" rid="j_info1159_ref_016">2015</xref>) to the same bit-rates as ours, we get corresponding performance: Lena – <inline-formula id="j_info1159_ineq_169"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>43.3037</mml:mn></mml:math><tex-math><![CDATA[$\mathit{PSQNR}=43.3037$]]></tex-math></alternatives></inline-formula> dB, <inline-formula id="j_info1159_ineq_170"><alternatives><mml:math>
<mml:mi mathvariant="italic">R</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>3.721</mml:mn></mml:math><tex-math><![CDATA[$R=3.721$]]></tex-math></alternatives></inline-formula> bpp; Street – <inline-formula id="j_info1159_ineq_171"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>42.1022</mml:mn></mml:math><tex-math><![CDATA[$\mathit{PSQNR}=42.1022$]]></tex-math></alternatives></inline-formula> dB, <inline-formula id="j_info1159_ineq_172"><alternatives><mml:math>
<mml:mi mathvariant="italic">R</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>3.852</mml:mn></mml:math><tex-math><![CDATA[$R=3.852$]]></tex-math></alternatives></inline-formula> bpp. By comparing these transformed results with the corresponding one from the model that we propose, it can be concluded that the proposed design achieves gain for image Lena <inline-formula id="j_info1159_ineq_173"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Gain</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Lena</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mn>3.3203</mml:mn></mml:math><tex-math><![CDATA[${\mathit{Gain}_{\mathrm{Lena}}}=3.3203$]]></tex-math></alternatives></inline-formula> dB and for image Street <inline-formula id="j_info1159_ineq_174"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Gain</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">Street</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mn>4.1558</mml:mn></mml:math><tex-math><![CDATA[${\mathit{Gain}_{\mathrm{Street}}}=4.1558$]]></tex-math></alternatives></inline-formula> dB.</p>
<p>Next, we provide comparison to the results published in Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_026">2015</xref>), where an application of linear prediction and dual-mode quantization in classic BTC algorithm is presented. The obtained results in Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_026">2015</xref>) for <inline-formula id="j_info1159_ineq_175"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>16</mml:mn></mml:math><tex-math><![CDATA[$N=16$]]></tex-math></alternatives></inline-formula>, which is the most suitable configuration since linear prediction provides better performance for lower bit-rates, is <inline-formula id="j_info1159_ineq_176"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>32.83</mml:mn></mml:math><tex-math><![CDATA[$\mathit{PSQNR}=32.83$]]></tex-math></alternatives></inline-formula> dB and <inline-formula id="j_info1159_ineq_177"><alternatives><mml:math>
<mml:mi mathvariant="italic">R</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>2.71</mml:mn></mml:math><tex-math><![CDATA[$R=2.71$]]></tex-math></alternatives></inline-formula> bpp. On the other hand, from Table <xref rid="j_info1159_tab_003">3</xref> it can be seen that for <inline-formula id="j_info1159_ineq_178"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>16</mml:mn></mml:math><tex-math><![CDATA[$N=16$]]></tex-math></alternatives></inline-formula>, <inline-formula id="j_info1159_ineq_179"><alternatives><mml:math>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>8</mml:mn></mml:math><tex-math><![CDATA[$L=8$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_180"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:mn>15</mml:mn></mml:math><tex-math><![CDATA[$\hat{\sigma }=15$]]></tex-math></alternatives></inline-formula> we have <inline-formula id="j_info1159_ineq_181"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>42.831</mml:mn></mml:math><tex-math><![CDATA[$\mathit{PSQNR}=42.831$]]></tex-math></alternatives></inline-formula> dB and <inline-formula id="j_info1159_ineq_182"><alternatives><mml:math>
<mml:mi mathvariant="italic">R</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>3.8694</mml:mn></mml:math><tex-math><![CDATA[$R=3.8694$]]></tex-math></alternatives></inline-formula> bpp. By transforming the results from Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_026">2015</xref>), taking into account aforementioned <inline-formula id="j_info1159_ineq_183"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo mathvariant="normal" stretchy="false">/</mml:mo>
<mml:mi mathvariant="italic">R</mml:mi></mml:math><tex-math><![CDATA[$\mathit{PSQNR}/R$]]></tex-math></alternatives></inline-formula> rule, we have obtained <inline-formula id="j_info1159_ineq_184"><alternatives><mml:math>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>39.2067</mml:mn></mml:math><tex-math><![CDATA[$\mathit{PSQNR}=39.2067$]]></tex-math></alternatives></inline-formula> dB and <inline-formula id="j_info1159_ineq_185"><alternatives><mml:math>
<mml:mi mathvariant="italic">R</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>3.8694</mml:mn></mml:math><tex-math><![CDATA[$R=3.8694$]]></tex-math></alternatives></inline-formula> bpp. Finally, we can conclude that the proposed model achieves gain of <inline-formula id="j_info1159_ineq_186"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="italic">Gain</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">eswa</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mn>42.83139.2067</mml:mn>
<mml:mo>=</mml:mo>
<mml:mn>3.6243</mml:mn></mml:math><tex-math><![CDATA[${\mathit{Gain}_{\mathrm{eswa}}}=42.83139.2067=3.6243$]]></tex-math></alternatives></inline-formula> dB in comparison to the model that incorporates linear prediction from Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_026">2015</xref>).</p>
<p>In the end, we provide a comparison of the obtained theoretical results obtained for the considered set of ten standard test grayscale images (Tables <xref rid="j_info1159_tab_001">1</xref> and <xref rid="j_info1159_tab_002">2</xref>) with the experimental results for standard test grayscale images that do not belong to the set used for model making. This way we want to show generality of the proposed analytical model and closed-form expressions. Experiments are done for 4 standard test grayscale images – Parrot, Clown, Fruits and Girl. These images are shown in Fig. <xref rid="j_info1159_fig_007">7</xref>.</p>
<fig id="j_info1159_fig_007">
<label>Fig. 7</label>
<caption>
<p>Standard test grayscale images: (a) Parrot; (b) Clown; (c) Fruits; (d) Girl.</p>
</caption>
<graphic xlink:href="info1159_g007.jpg"/>
</fig>
<p>In Table <xref rid="j_info1159_tab_004">4</xref> obtained average experimental results (denoted with <inline-formula id="j_info1159_ineq_187"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_188"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula>) and corresponding theoretical results obtained by using the proposed analytical model (denoted with <inline-formula id="j_info1159_ineq_189"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">th</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{th}}}$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_190"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">th</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{th}}}$]]></tex-math></alternatives></inline-formula>) are shown.</p>
<table-wrap id="j_info1159_tab_004">
<label>Table 4</label>
<caption>
<p>System performance for standard test grayscale images from Fig. <xref rid="j_info1159_fig_007">7</xref>. (<inline-formula id="j_info1159_ineq_191"><alternatives><mml:math>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>8</mml:mn></mml:math><tex-math><![CDATA[$L=8$]]></tex-math></alternatives></inline-formula>; <inline-formula id="j_info1159_ineq_192"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:mn>15</mml:mn></mml:math><tex-math><![CDATA[$\hat{\sigma }=15$]]></tex-math></alternatives></inline-formula>).</p>
</caption>
<table>
<thead>
<tr>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><italic>N</italic></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_193"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_194"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">ex</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{ex}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_195"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">PSQNR</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">th</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${\mathit{PSQNR}^{\mathrm{th}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_196"><alternatives><mml:math>
<mml:msup>
<mml:mrow>
<mml:mi mathvariant="italic">R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">th</mml:mi>
</mml:mrow>
</mml:msup></mml:math><tex-math><![CDATA[${R^{\mathrm{th}}}$]]></tex-math></alternatives></inline-formula></td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_197"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="normal">Δ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">PSQNR</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\Delta _{\mathrm{PSQNR}}}$]]></tex-math></alternatives></inline-formula> [dB]</td>
<td style="vertical-align: top; text-align: left; border-top: solid thin; border-bottom: solid thin"><inline-formula id="j_info1159_ineq_198"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="normal">Δ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">R</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\Delta _{\mathrm{R}}}$]]></tex-math></alternatives></inline-formula> [bpp]</td>
</tr>
</thead>
<tbody>
<tr>
<td style="vertical-align: top; text-align: left">16</td>
<td style="vertical-align: top; text-align: left">42.4828</td>
<td style="vertical-align: top; text-align: left">3.7235</td>
<td style="vertical-align: top; text-align: left">42.3193</td>
<td style="vertical-align: top; text-align: left">3.7501</td>
<td style="vertical-align: top; text-align: left">0.1635</td>
<td style="vertical-align: top; text-align: left">0.0266</td>
</tr>
<tr>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">32</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">47.1250</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">4.7145</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">47.4487</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">4.7393</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">0.3237</td>
<td style="vertical-align: top; text-align: left; border-bottom: solid thin">0.0248</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>In Table <xref rid="j_info1159_tab_004">4</xref>, with <inline-formula id="j_info1159_ineq_199"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="normal">Δ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">PSQNR</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\Delta _{\mathrm{PSQNR}}}$]]></tex-math></alternatives></inline-formula> and <inline-formula id="j_info1159_ineq_200"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="normal">Δ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">R</mml:mi>
</mml:mrow>
</mml:msub></mml:math><tex-math><![CDATA[${\Delta _{\mathrm{R}}}$]]></tex-math></alternatives></inline-formula> are denoted the absolute error rates between corresponding experimental and theoretical results. By observing Table <xref rid="j_info1159_tab_004">4</xref>, it can be unambiguously seen that the proposed analytical model predicts system performance very well (<inline-formula id="j_info1159_ineq_201"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="normal">Δ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">PSQNR</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">≪</mml:mo>
<mml:mn>1</mml:mn></mml:math><tex-math><![CDATA[${\Delta _{\mathrm{PSQNR}}}\ll 1$]]></tex-math></alternatives></inline-formula> dB, <inline-formula id="j_info1159_ineq_202"><alternatives><mml:math>
<mml:msub>
<mml:mrow>
<mml:mi mathvariant="normal">Δ</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:mi mathvariant="normal">R</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">≪</mml:mo>
<mml:mn>0.2</mml:mn></mml:math><tex-math><![CDATA[${\Delta _{\mathrm{R}}}\ll 0.2$]]></tex-math></alternatives></inline-formula> bpp) even for the wider set of standard test grayscale images, whose statistical properties were not taken into consideration and model designing. The reconstructed images obtained by processing with the proposed algorithm are shown in Fig. <xref rid="j_info1159_fig_008">8</xref>. It can be noticed that the visual difference between reconstructed and original images almost does not exist and that there is not any noticeable geometric irregularity such as block effect.</p>
<fig id="j_info1159_fig_008">
<label>Fig. 8</label>
<caption>
<p>Reconstructed standard test grayscale images from Fig. <xref rid="j_info1159_fig_007">7</xref> (<inline-formula id="j_info1159_ineq_203"><alternatives><mml:math>
<mml:mi mathvariant="italic">N</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>16</mml:mn></mml:math><tex-math><![CDATA[$N=16$]]></tex-math></alternatives></inline-formula>; <inline-formula id="j_info1159_ineq_204"><alternatives><mml:math>
<mml:mi mathvariant="italic">L</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>8</mml:mn></mml:math><tex-math><![CDATA[$L=8$]]></tex-math></alternatives></inline-formula>; <inline-formula id="j_info1159_ineq_205"><alternatives><mml:math><mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="italic">σ</mml:mi>
</mml:mrow>
<mml:mo stretchy="false">ˆ</mml:mo></mml:mover>
<mml:mo>=</mml:mo>
<mml:mn>15</mml:mn></mml:math><tex-math><![CDATA[$\hat{\sigma }=15$]]></tex-math></alternatives></inline-formula>): (a) Parrot; (b) Clown; (c) Fruits; (d) Girl.</p>
</caption>
<graphic xlink:href="info1159_g008.jpg"/>
</fig>
</sec>
</sec>
<sec id="j_info1159_s_008">
<label>5</label>
<title>Conclusions</title>
<p>BTC algorithm as one of the core techniques for image processing has been modified and upgraded in a number of papers. Also, there are various modifications that are application dependent and they provide satisfactory results in comparison to the other techniques. However, the deficiency of analytical models for performance analysis motivated us to provide a model as well as an improved system design for grayscale image compression. In this study, we have developed the analytical model based on application of Inverse Gaussian distribution. The proposed closed-form solutions for <italic>PSQNR</italic> and the total average bit-rate provide matching with the experimental results for various values of system parameters within the range of 1 dB and 0.2 bpp, respectively.</p>
<p>Furthermore, the experimental results show that the proposed design provides better performance of reconstructed images in comparison to the other solutions based on the fixed piecewise uniform quantizer. Firstly, comparing with Savic <italic>et al.</italic> (<xref ref-type="bibr" rid="j_info1159_ref_025">2012</xref>), it was shown that experimental gain is provided for various system configurations and it is between 3.826 dB and 11.56 dB. Moreover, the proposed application of Golomb–Rice encoding provides gain even comparing to the other techniques such as adaptation and linear prediction. Comparing the achieved results with the corresponding one from Mathews and Nair (<xref ref-type="bibr" rid="j_info1159_ref_016">2015</xref>) for two standard test images Lena and Street, we have shown that the proposed system provides gain up to 4.1558 dB. This way, we have concluded that the proposed design achieves better <italic>PSQNR/R</italic> ratio in comparison to edge-based adaptation. Also, we have shown that the proposed design provides gain of 3.6243 dB in comparison to the algorithm which apply linear prediction scheme (Savic <italic>et al.</italic>, <xref ref-type="bibr" rid="j_info1159_ref_026">2015</xref>). The proposed method has very low complexity and it provides much lower processing time. In future work, we will intend to research possible application of the proposed quantizer to the other algorithms that incorporate transform coding as well as to provide analytical models for different image processing algorithms due to its deficiency.</p>
</sec>
</body>
<back>
<ack id="j_info1159_ack_001">
<title>Acknowledgements</title>
<p>This work is supported by Serbian Ministry of Education and Science through Mathematical Institute of Serbian Academy of Sciences and Arts (project III44006) and by Serbian Ministry of Education, Science and Technological Development (project TR32035).</p></ack>
<ref-list id="j_info1159_reflist_001">
<title>References</title>
<ref id="j_info1159_ref_001">
<mixed-citation publication-type="journal"><string-name><surname>Ali</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Manzur</surname>, <given-names>M.</given-names></string-name> (<year>2015</year>). <article-title>Symbol coding of Laplacian distributed prediction residuals</article-title>. <source>Digital Signal Processing</source>, <volume>44</volume>, <fpage>76</fpage>–<lpage>87</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_002">
<mixed-citation publication-type="journal"><string-name><surname>Alomari</surname>, <given-names>Y.M.</given-names></string-name>, <string-name><surname>Abdullah</surname>, <given-names>S.N.H.S.</given-names></string-name>, <string-name><surname>Zin</surname>, <given-names>R.R.M.</given-names></string-name>, <string-name><surname>Omar</surname>, <given-names>K.</given-names></string-name> (<year>2016</year>). <article-title>Iterative randomized irregular circular algorithm for proliferation rate estimation in brain tumor Ki-67 histology images</article-title>. <source>Expert Systems with Applications</source>, <volume>48</volume>, <fpage>111</fpage>–<lpage>129</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_003">
<mixed-citation publication-type="journal"><string-name><surname>Barcellos</surname>, <given-names>P.</given-names></string-name>, <string-name><surname>Bouvie</surname>, <given-names>C.</given-names></string-name>, <string-name><surname>Escouto</surname>, <given-names>F.L.</given-names></string-name>, <string-name><surname>Scharcanski</surname>, <given-names>J.</given-names></string-name> (<year>2015</year>). <article-title>A novel video based system for detecting and counting vehicles at user-defined virtual loops</article-title>. <source>Expert Systems with Applications</source>, <volume>42</volume>, <fpage>1845</fpage>–<lpage>1856</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_004">
<mixed-citation publication-type="journal"><string-name><surname>Chang</surname>, <given-names>C.-C.</given-names></string-name>, <string-name><surname>Lin</surname>, <given-names>C.-Y.</given-names></string-name>, <string-name><surname>Fan</surname>, <given-names>Y.-H.</given-names></string-name> (<year>2008</year>). <article-title>Lossless data hiding for color images based on block truncation coding</article-title>. <source>Pattern Recognition</source>, <volume>417</volume>, <fpage>2347</fpage>–<lpage>2357</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_005">
<mixed-citation publication-type="journal"><string-name><surname>Chang</surname>, <given-names>I.-C.</given-names></string-name>, <string-name><surname>Hu</surname>, <given-names>Y.-C.</given-names></string-name>, <string-name><surname>Chen</surname>, <given-names>W.-L.</given-names></string-name>, <string-name><surname>Lo</surname>, <given-names>C.-C.</given-names></string-name> (<year>2015</year>). <article-title>High capacity reversible data hiding scheme based on residual histogram shifting for block truncation coding</article-title>. <source>Signal Processing</source>, <volume>108</volume>, <fpage>376</fpage>–<lpage>388</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_006">
<mixed-citation publication-type="journal"><string-name><surname>Dadkhah</surname>, <given-names>S.</given-names></string-name>, <string-name><surname>Manaf</surname>, <given-names>A.A.</given-names></string-name>, <string-name><surname>Hori</surname>, <given-names>Y.</given-names></string-name>, <string-name><surname>Hassanien</surname>, <given-names>A.E.</given-names></string-name>, <string-name><surname>Sadeghi</surname>, <given-names>S.</given-names></string-name> (<year>2014</year>). <article-title>An effective SVD-based image tampering detection and self-recovery using active watermarking</article-title>. <source>Signal Processing: Image Communication</source>, <volume>29</volume>, <fpage>1197</fpage>–<lpage>1210</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_007">
<mixed-citation publication-type="journal"><string-name><surname>Delp</surname>, <given-names>E.J.</given-names></string-name>, <string-name><surname>Mitchell</surname>, <given-names>O.R.</given-names></string-name> (<year>1979</year>). <article-title>Image compression using block truncation coding</article-title>. <source>IEEE Transactions on Communications</source>, <volume>27</volume>(<issue>9</issue>), <fpage>1335</fpage>–<lpage>1342</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_008">
<mixed-citation publication-type="journal"><string-name><surname>Guo</surname>, <given-names>J.-M.</given-names></string-name>, <string-name><surname>Liu</surname>, <given-names>Y.-F.</given-names></string-name> (<year>2012</year>). <article-title>High capacity data hiding for error-diffused block truncation coding</article-title>. <source>IEEE Transactions on Image Processing</source>, <volume>21</volume>(<issue>12</issue>), <fpage>4808</fpage>–<lpage>4818</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_009">
<mixed-citation publication-type="book"><string-name><surname>Jayant</surname>, <given-names>N.S.</given-names></string-name>, <string-name><surname>Noll</surname>, <given-names>P.</given-names></string-name> (<year>1984</year>). <source>Digital Coding of Waveforms</source>. <publisher-name>Prentice-Hal</publisher-name>, <publisher-loc>New Jersey</publisher-loc>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_010">
<mixed-citation publication-type="journal"><string-name><surname>Kim</surname>, <given-names>S.</given-names></string-name>, <string-name><surname>Lee</surname>, <given-names>H.-J.</given-names></string-name> (<year>2016</year>). <article-title>RGBW image compression by low-complexity adaptive multi-level block truncation coding</article-title>. <source>IEEE Transactions on Consumer Electronics</source>, <volume>62</volume>(<issue>4</issue>), <fpage>412</fpage>–<lpage>419</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_011">
<mixed-citation publication-type="journal"><string-name><surname>Kim</surname>, <given-names>H.-S.</given-names></string-name>, <string-name><surname>Lee</surname>, <given-names>J.</given-names></string-name>, <string-name><surname>Kim</surname>, <given-names>H.</given-names></string-name>, <string-name><surname>Kang</surname>, <given-names>S.</given-names></string-name>, <string-name><surname>Park</surname>, <given-names>W.C.</given-names></string-name> (<year>2011</year>). <article-title>A lossless color image compression architecture using a parallel Golomb-Rice CODEC</article-title>. <source>IEEE Transactions on Circuits and Systems for Video Technology</source>, <volume>21</volume>(<issue>11</issue>), <fpage>1581</fpage>–<lpage>1587</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_012">
<mixed-citation publication-type="journal"><string-name><surname>Kim</surname>, <given-names>S.</given-names></string-name>, <string-name><surname>Lee</surname>, <given-names>D.</given-names></string-name>, <string-name><surname>Kim</surname>, <given-names>J.-S.</given-names></string-name>, <string-name><surname>Lee</surname>, <given-names>H.-J.</given-names></string-name> (<year>2016</year>). <article-title>A block truncation coding algorithm and hardware implementation targeting 1/12 compression for LCD overdrive</article-title>. <source>Journal of Display Technology</source>, <volume>12</volume>(<issue>4</issue>), <fpage>376</fpage>–<lpage>389</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_013">
<mixed-citation publication-type="journal"><string-name><surname>Lema</surname>, <given-names>M.D.</given-names></string-name>, <string-name><surname>Mitchell</surname>, <given-names>O.R.</given-names></string-name> (<year>1984</year>). <article-title>Absolute moment block truncation coding and its application to color images</article-title>. <source>IEEE Transactions on Communications</source>, <volume>32</volume>, <fpage>1148</fpage>–<lpage>1157</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_014">
<mixed-citation publication-type="journal"><string-name><surname>Li</surname>, <given-names>Y.</given-names></string-name>, <string-name><surname>Yang</surname>, <given-names>Q.</given-names></string-name>, <string-name><surname>Jiao</surname>, <given-names>R.</given-names></string-name> (<year>2010</year>). <article-title>Image compression scheme based on curvelet transform and support vector machine</article-title>. <source>Expert Systems with Applications</source>, <volume>37</volume>, <fpage>3063</fpage>–<lpage>3069</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_015">
<mixed-citation publication-type="journal"><string-name><surname>Li</surname>, <given-names>Y.</given-names></string-name>, <string-name><surname>Wang</surname>, <given-names>Y.</given-names></string-name>, <string-name><surname>Yang</surname>, <given-names>Q.</given-names></string-name> (<year>2013</year>). <article-title>Curvelet based image compression via core vector machine</article-title>. <source>Optik International Journal for Light and Electron Optics</source>, <volume>124</volume>(<issue>21</issue>), <fpage>4859</fpage>–<lpage>4866</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_016">
<mixed-citation publication-type="journal"><string-name><surname>Mathews</surname>, <given-names>J.</given-names></string-name>, <string-name><surname>Nair</surname>, <given-names>M.S.</given-names></string-name> (<year>2015</year>). <article-title>Adaptive block truncation coding technique using edge-based quantization approach</article-title>. <source>Computers &amp; Electrical Engineering</source>, <volume>43</volume>, <fpage>169</fpage>–<lpage>179</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_017">
<mixed-citation publication-type="journal"><string-name><surname>Ozturk</surname>, <given-names>C.</given-names></string-name>, <string-name><surname>Hancer</surname>, <given-names>E.</given-names></string-name>, <string-name><surname>Karaboga</surname>, <given-names>D.</given-names></string-name> (<year>2014</year>). <article-title>Color image quantization: a short review and an application with artificial Bee colony algorithm</article-title>. <source>Informatica</source>, <volume>25</volume>(<issue>3</issue>), <fpage>485</fpage>–<lpage>503</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_018">
<mixed-citation publication-type="journal"><string-name><surname>Perić</surname>, <given-names>Z.</given-names></string-name>, <string-name><surname>Petković</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Dinčić</surname>, <given-names>M.</given-names></string-name> (<year>2009</year>). <article-title>Simple compression algorithm for memoryless Laplacian source based on the optimal companding technique</article-title>. <source>Informatica</source>, <volume>20</volume>(<issue>1</issue>), <fpage>99</fpage>–<lpage>114</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_019">
<mixed-citation publication-type="journal"><string-name><surname>Perić</surname>, <given-names>Z.</given-names></string-name>, <string-name><surname>Dincic</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Petkovic</surname>, <given-names>M.</given-names></string-name> (<year>2010</year>a). <article-title>Design of a hybrid quantizer with variable length code</article-title>. <source>Fundamenta Informaticae</source>, <volume>98</volume>(<issue>2–3</issue>), <fpage>233</fpage>–<lpage>256</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_020">
<mixed-citation publication-type="journal"><string-name><surname>Perić</surname>, <given-names>Z.</given-names></string-name>, <string-name><surname>Savić</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Dinčić</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Denić</surname>, <given-names>D.</given-names></string-name>, <string-name><surname>Praščević</surname>, <given-names>M.</given-names></string-name> (<year>2010</year>b). <article-title>Forward adaptation of novel semilogarithmic quantizer and lossless coder for speech signals compression</article-title>. <source>Informatica</source>, <volume>21</volume>(<issue>3</issue>), <fpage>375</fpage>–<lpage>391</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_021">
<mixed-citation publication-type="journal"><string-name><surname>Perić</surname>, <given-names>Z.</given-names></string-name>, <string-name><surname>Simić</surname>, <given-names>N.</given-names></string-name>, <string-name><surname>Savić</surname>, <given-names>M.</given-names></string-name> (<year>2015</year>). <article-title>Analysis and design of two stage mismatch quantizer for Laplacian source</article-title>. <source>Elektronika ir Elektrotechnika</source>, <volume>21</volume>(<issue>3</issue>), <fpage>49</fpage>–<lpage>53</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_022">
<mixed-citation publication-type="journal"><string-name><surname>Ponti</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Nazare</surname>, <given-names>T.S.</given-names></string-name>, <string-name><surname>Thume</surname>, <given-names>G.S.</given-names></string-name> (<year>2016</year>). <article-title>Image quantization as a dimensionality reduction procedure in color and texture feature extraction</article-title>. <source>Neurocomputing</source>, <volume>173</volume>(<issue>2</issue>), <fpage>385</fpage>–<lpage>396</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_023">
<mixed-citation publication-type="book"><string-name><surname>Salomon</surname>, <given-names>D.</given-names></string-name> (<year>2007</year>). <source>Data Compression – The Complete Reference</source>. <edition>4</edition>th ed. <publisher-name>Springer</publisher-name>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_024">
<mixed-citation publication-type="journal"><string-name><surname>Savic</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Peric</surname>, <given-names>Z.</given-names></string-name>, <string-name><surname>Dincic</surname>, <given-names>M.</given-names></string-name> (<year>2010</year>). <article-title>Design of forward adaptive uniform quantizer for discrete input samples for Laplacian source</article-title>. <source>Electronics and Electrical Engineering</source>, <volume>9</volume>(<issue>105</issue>), <fpage>73</fpage>–<lpage>76</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_025">
<mixed-citation publication-type="journal"><string-name><surname>Savic</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Peric</surname>, <given-names>Z.</given-names></string-name>, <string-name><surname>Dincic</surname>, <given-names>M.</given-names></string-name> (<year>2012</year>). <article-title>Coding algorithm for grayscale images based on piecewise uniform quantizers</article-title>. <source>Informatica</source>, <volume>23</volume>(<issue>1</issue>), <fpage>125</fpage>–<lpage>140</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_026">
<mixed-citation publication-type="journal"><string-name><surname>Savic</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Peric</surname></string-name>, <string-name><surname>Simic N</surname>, <given-names>Z.</given-names></string-name> (<year>2015</year>). <article-title>Coding algorithm for grayscale images based on linear prediction and dual-mode quantization</article-title>. <source>Expert Systems with Applications</source>, <volume>42</volume>, <fpage>7285</fpage>–<lpage>7291</lpage>.</mixed-citation>
</ref>
<ref id="j_info1159_ref_027">
<mixed-citation publication-type="book"><string-name><surname>Sayood</surname>, <given-names>K.</given-names></string-name> (<year>2006</year>). <source>Introduction to Data Compression</source>. <edition>3</edition>rd ed. <publisher-name>Elsevier Inc.</publisher-name></mixed-citation>
</ref>
<ref id="j_info1159_ref_028">
<mixed-citation publication-type="journal"><string-name><surname>Simic</surname>, <given-names>N.</given-names></string-name>, <string-name><surname>Peric</surname>, <given-names>Z.H.</given-names></string-name>, <string-name><surname>Savic</surname>, <given-names>M.S.</given-names></string-name> (<year>2017</year>). <article-title>Image coding algorithm based on Hadamard transform and simple vector quantization</article-title>. <source>Multimedia Tools and Applications.</source> <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1007/s11042-017-4513-4" xlink:type="simple">https://doi.org/10.1007/s11042-017-4513-4</ext-link>.</mixed-citation>
</ref>
</ref-list>
</back>
</article>