Grammar-based code: Difference between revisions

Content deleted Content added

Inline

Latest revision as of 22:09, 8 August 2023

Straight-line grammar (with start symbol ß) for the second sentence of the United States Declaration of Independence. Each blue character denotes a nonterminal symbol; they were obtained from a gzip-compression of the sentence.

Grammar-based codes or Grammar-based compression are compression algorithms based on the idea of constructing a context-free grammar (CFG) for the string to be compressed. Examples include universal lossless data compression algorithms.^[1] To compress a data sequence $x=x_{1}\cdots x_{n}$ , a grammar-based code transforms $x$ into a context-free grammar $G$ . The problem of finding a smallest grammar for an input sequence (smallest grammar problem) is known to be NP-hard,^[2] so many grammar-transform algorithms are proposed from theoretical and practical viewpoints. Generally, the produced grammar $G$ is further compressed by statistical encoders like arithmetic coding.

Examples and characteristics

The class of grammar-based codes is very broad. It includes block codes, the multilevel pattern matching (MPM) algorithm,^[3] variations of the incremental parsing Lempel-Ziv code,^[4] and many other new universal lossless compression algorithms. Grammar-based codes are universal in the sense that they can achieve asymptotically the entropy rate of any stationary, ergodic source with a finite alphabet.

Practical algorithms

The compression programs of the following are available from external links.

Sequitur^[5] is a classical grammar compression algorithm that sequentially translates an input text into a CFG, and then the produced CFG is encoded by an arithmetic coder.
Re-Pair^[6] is a greedy algorithm using the strategy of most-frequent-first substitution. The compressive performance is powerful, although the main memory space requirement is very large.
GLZA,^[7] which constructs a grammar that may be reducible, i.e., contain repeats, where the entropy-coding cost of "spelling out" the repeats is less than the cost creating and entropy-coding a rule to capture them. (In general, the compression-optimal SLG is not irreducible, and the Smallest Grammar Problem is different from the actual SLG compression problem.)

References

^ Kieffer, J. C.; Yang, E.-H. (2000), "Grammar-based codes: A new class of universal lossless source codes", IEEE Trans. Inf. Theory, 46 (3): 737–754, doi:10.1109/18.841160
^ Charikar, M.; Lehman, E.; Liu, D.; Panigrahy, R.; Prabharakan, M.; Sahai, A.; Shelat, A. (2005), "The Smallest Grammar Problem", IEEE Trans. Inf. Theory, 51 (7): 2554–2576, doi:10.1109/tit.2005.850116, S2CID 6900082
^ Kieffer, J. C.; Yang, E.-H.; Nelson, G.; Cosman, P. (2000), "Universal lossless compression via multilevel pattern matching", IEEE Trans. Inf. Theory, 46 (4): 1227–1245, doi:10.1109/18.850665, S2CID 8191526
^ Ziv, J.; Lempel, A. (1978), "Compression of individual sequences via variable rate coding", IEEE Trans. Inf. Theory, 24 (5): 530–536, doi:10.1109/TIT.1978.1055934, hdl:10338.dmlcz/142945
^ Nevill-Manning, C. G.; Witten, I. H. (1997), "Identifying Hierarchical Structure in Sequences: A linear-time algorithm", Journal of Artificial Intelligence Research, 7 (4): 67–82, arXiv:cs/9709102, doi:10.1613/jair.374, hdl:10289/1186, S2CID 2957960
^ Larsson, N. J.; Moffat, A. (2000), "Offline Dictionary-Based Compression" (PDF), Proceedings of the IEEE, 88 (11): 1722–1732, doi:10.1109/5.892708
^ Conrad, Kennon J.; Wilson, Paul R. (2016). "Grammatical Ziv-Lempel Compression: Achieving PPM-Class Text Compression Ratios with LZ-Class Decompression Speed". 2016 Data Compression Conference (DCC). p. 586. doi:10.1109/DCC.2016.119. ISBN 978-1-5090-1853-6. S2CID 3116024.

External links

GLZA discussion and paper
Description of grammar-based codes with example
Sequitur codes Archived 2008-10-13 at the Wayback Machine
Re-Pair codes
Re-Pair codes a version of Gonzalo Navarro.
GrammarViz 2.0 - implementation of Sequitur, Re-Pair, and parallel Re-Pair in Java.

[1] Kieffer, J. C.; Yang, E.-H. (2000), "Grammar-based codes: A new class of universal lossless source codes", IEEE Trans. Inf. Theory, 46 (3): 737–754, doi:10.1109/18.841160

[2] Charikar, M.; Lehman, E.; Liu, D.; Panigrahy, R.; Prabharakan, M.; Sahai, A.; Shelat, A. (2005), "The Smallest Grammar Problem", IEEE Trans. Inf. Theory, 51 (7): 2554–2576, doi:10.1109/tit.2005.850116, S2CID 6900082

[3] Kieffer, J. C.; Yang, E.-H.; Nelson, G.; Cosman, P. (2000), "Universal lossless compression via multilevel pattern matching", IEEE Trans. Inf. Theory, 46 (4): 1227–1245, doi:10.1109/18.850665, S2CID 8191526

[4] Ziv, J.; Lempel, A. (1978), "Compression of individual sequences via variable rate coding", IEEE Trans. Inf. Theory, 24 (5): 530–536, doi:10.1109/TIT.1978.1055934, hdl:10338.dmlcz/142945

[5] Nevill-Manning, C. G.; Witten, I. H. (1997), "Identifying Hierarchical Structure in Sequences: A linear-time algorithm", Journal of Artificial Intelligence Research, 7 (4): 67–82, arXiv:cs/9709102, doi:10.1613/jair.374, hdl:10289/1186, S2CID 2957960

[6] Larsson, N. J.; Moffat, A. (2000), "Offline Dictionary-Based Compression" (PDF), Proceedings of the IEEE, 88 (11): 1722–1732, doi:10.1109/5.892708

[7] Conrad, Kennon J.; Wilson, Paul R. (2016). "Grammatical Ziv-Lempel Compression: Achieving PPM-Class Text Compression Ratios with LZ-Class Decompression Speed". 2016 Data Compression Conference (DCC). p. 586. doi:10.1109/DCC.2016.119. ISBN 978-1-5090-1853-6. S2CID 3116024.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

@@ Line 1: / Line 1: @@
+{{Short description|Lossless data compression algorithm}}
-{{Wikify|date=August 2007}}
+[[File:IndpGrm.gif|thumb|600px|[[Straight-line grammar]] (with start symbol ß) for the second sentence of the [[United States Declaration of Independence]]. Each blue character denotes a [[nonterminal symbol]]; they were obtained from a [[gzip]]-compression of the sentence.]]
+'''Grammar-based codes''' or '''Grammar-based compression''' are [[Data compression|compression]] algorithms based on the idea of constructing a [[context-free grammar]] (CFG) for the string to be compressed. Examples include universal [[lossless data compression]] algorithms.<ref>{{Citation | last1 = Kieffer  | first1 =  J. C.  | last2 = Yang | first2 = E.-H. | title = Grammar-based codes: A new class of universal lossless source codes | journal = IEEE Trans. Inf. Theory | volume = 46 | pages = 737–754 | year = 2000 | doi = 10.1109/18.841160 | issue = 3 }}</ref> To compress a data sequence <math>x = x_1 \cdots x_n</math>, a grammar-based code transforms <math>x</math> into a context-free grammar <math>G</math>.
+The problem of finding a smallest grammar for an input sequence ([[smallest grammar problem]]) is known to be NP-hard,<ref>{{Citation | last1 = Charikar  | first1 =  M.  | last2 = Lehman | first2 = E. | last3 = Liu | first3 = D. | last4 = Panigrahy | first4 = R. | last5 = Prabharakan | first5 = M. | last6 = Sahai | first6 = A. | last7 = Shelat | first7 = A. |  title = The Smallest Grammar Problem | journal = IEEE Trans. Inf. Theory | volume = 51 | pages = 2554–2576 | year = 2005 | issue = 7 | doi=10.1109/tit.2005.850116| s2cid = 6900082 }}</ref> so many grammar-transform algorithms are proposed from theoretical and practical viewpoints.
+Generally, the produced grammar <math>G</math> is further compressed by statistical encoders like [[arithmetic coding]].
+== Examples and characteristics ==
-'''Grammar-based codes''' are [[Data_compression|compression]] algorithms based on the idea of constructing a [[Context-free_grammar|context-free grammar]] for the string to be compressed. Examples include universal [[lossless data compression]] algorithms proposed in Kieffer and Yang 2000, and [[SEQUITUR_algorithm|SEQUITUR]](http://sequitur.info/), among others. To compress a data sequence<math>x = x_1 \cdots x_n</math>, a grammar-based code first transforms <math>x</math> into a context-free grammar <math>G</math>, and then uses an [[arithmetic coding]] algorithm to compress the grammar <math>G</math>. For a detailed description of grammar-based codes and context-free grammars, the reader is referred to Kieffer and Yang 2000.
+The class of grammar-based codes is very broad. It includes [[block code]]s, the multilevel pattern matching (MPM) algorithm,<ref>{{Citation | last1 = Kieffer | first1 = J. C. | last2 = Yang | first2 = E.-H.  | last3 = Nelson | first3 = G. | last4 = Cosman | first4 = P. | title = Universal lossless compression via multilevel pattern matching | journal = IEEE Trans. Inf. Theory | volume = 46 | pages = 1227–1245 | year = 2000 | doi = 10.1109/18.850665 | issue = 4 | s2cid = 8191526 | url = https://escholarship.org/uc/item/39k54514 }}</ref> variations of the incremental parsing [[LZ77 and LZ78|Lempel-Ziv code]],<ref>{{Citation | last1 = Ziv | first1 = J. | last2 = Lempel | first2 = A. | title = Compression of individual sequences via variable rate coding | journal = IEEE Trans. Inf. Theory | volume = 24 | pages = 530–536 | year = 1978 | doi = 10.1109/TIT.1978.1055934 | issue = 5 | hdl = 10338.dmlcz/142945 | hdl-access = free }}</ref> and many other new universal lossless compression algorithms.
+Grammar-based codes are universal in the sense that they can achieve asymptotically the [[entropy rate]] of any stationary, [[ergodic]] source with a finite alphabet.
-== Examples ==
+== Practical algorithms ==
+The compression programs of the following are available from external links.
-The class of grammar-based codes is very broad. It includes block codes, variations of the incremental parsing Lempel-Ziv code Ziv and Lempel 1978 ([[LZ77 and LZ78 (algorithms)|LZ78]]), the multilevel pattern matching (MPM) algorithm Kieffer et al. 2000, and many other new universal lossless compression algorithms.
+* [[SEQUITUR algorithm|Sequitur]]<ref>{{Citation | last1 = Nevill-Manning  | first1 =  C. G.  | last2 = Witten | first2 = I. H. | title = Identifying Hierarchical Structure in Sequences: A linear-time algorithm | journal =  Journal of Artificial Intelligence Research | volume = 7 | pages = 67–82 | year = 1997 | doi = 10.1613/jair.374| issue = 4 | hdl=10289/1186| arxiv = cs/9709102 | s2cid = 2957960 }}</ref> is a classical grammar compression algorithm that sequentially translates an input text into a CFG, and then the produced CFG is encoded by an arithmetic coder.
-== Coding theorems ==
+* [[Re-Pair]]<ref>{{Citation | last1 = Larsson  | first1 =  N. J.  | last2 = Moffat | first2 = A. | title = Offline Dictionary-Based Compression | journal =  Proceedings of the IEEE| volume = 88 | pages = 1722–1732| year = 2000 | issue = 11 | doi=10.1109/5.892708| url = http://www.davidsalomon.name/DC4advertis/BPE.pdf }}</ref> is a greedy algorithm using the strategy of most-frequent-first substitution. The compressive performance is powerful, although the main memory space requirement is very large.
-Grammar-based codes are universal in the sense that they can achieve asymptotically the entropy rate of any stationary, ergodic source with a finite alphabet.
+* [[GLZA]],<ref>{{Cite book| last1 = Conrad  | first1 =  Kennon J. | last2 = Wilson | first2 = Paul R. | title = 2016 Data Compression Conference (DCC) | chapter = Grammatical Ziv-Lempel Compression: Achieving PPM-Class Text Compression Ratios with LZ-Class Decompression Speed | pages = 586 | year = 2016 | doi=10.1109/DCC.2016.119| isbn = 978-1-5090-1853-6 | s2cid = 3116024 }}</ref> which constructs a grammar that may be reducible, i.e., contain repeats, where the entropy-coding cost of "spelling out" the repeats is less than the cost creating and entropy-coding a rule to capture them. (In general, the compression-optimal SLG is not irreducible, and the Smallest Grammar Problem is different from the actual SLG compression problem.)
+==See also==
+* [[Dictionary coder]]
+*[[Grammar induction]]
+* [[Straight-line grammar]]
 == References ==
+{{reflist|30em}}
-*Kieffer-Yang 2000. J. C. Kieffer and E.-H. Yang, "Grammar-based codes: A new class of universal lossless source codes," IEEE Trans. Inform. Theory, vol. 46, pp. 737&ndash;754, 2000.
+==External links==
-*Ziv and Lempel 1978. J. Ziv and A. Lempel, "Compression of individual sequences via variable rate coding," IEEE Trans. Inform. Theory, vol. 24, pp. 530&ndash;536, 2000.
+*[https://encode.su/threads/2427-GLZA GLZA discussion and paper]
+*[https://www.mit.edu/~6.454/www_fall_2002/emin/summary.pdf Description of grammar-based codes with example]
+*[http://sequitur.info Sequitur codes] {{Webarchive|url=https://web.archive.org/web/20081013024323/http://www.sequitur.info/ |date=2008-10-13 }}
+*[http://www.cbrc.jp/~rwan/en/restore.html Re-Pair codes]
+*[http://www.dcc.uchile.cl/~gnavarro/software/index.html Re-Pair codes] a version of Gonzalo Navarro.
+*[https://github.com/GrammarViz2/grammarviz2_src GrammarViz 2.0] - implementation of Sequitur, Re-Pair, and parallel Re-Pair in Java.
+{{Compression methods}}
-*Kieffer et al. 2000. J. C. Kieffer, E.-H. Yang, G. Nelson, and P. Cosman, "Universal lossless compression via multilevel pattern matching," IEEE Trans. Inform. Theory, vol. 46, pp. 1227&ndash;1245, 2000.
-*[http://www.mit.edu/~6.454/www_fall_2002/emin/summary.pdf Great description of grammar-based codes with easy to follow example]
 [[Category:Data compression]]
 [[Category:Coding theory]]
 [[Category:Information theory]]
-{{compu-stub}}
-[[pl:Kodowanie gramatykowe]]