An overhead reduction technique for mega-state compression schemes

Abraham Bookstein; Shmuel T. Klein; Timo Raita

doi:10.1016/s0306-4573(97)00034-4

An overhead reduction technique for mega-state compression schemes

Abraham Bookstein, Shmuel T. Klein, Timo Raita

Department of Mathematics

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

Many of the most effective compression methods involve complicated models. Unfortunately, as model complexity increases, so does the cost of storing the model itself. This paper examines a method to reduce the amount of storage needed to represent a Markov model with an extended alphabet, by applying a clustering scheme that brings together similar states. Experiments run on a variety of large natural language texts show that much of the overhead of storing the model can be saved at the cost of a very small loss of compression efficiency.

Original language	English
Pages (from-to)	745-760
Number of pages	16
Journal	Information Processing and Management
Volume	33
Issue number	6
DOIs	https://doi.org/10.1016/s0306-4573(97)00034-4
State	Published - Nov 1997

Bibliographical note

Funding Information:
* The work of the first author (AB) was supported, in part, by NSF Grant IRI-9307895-A01. The author gratefully acknowledges this support. We also wish to acknowledge support given by the Academy of Finland to TR, t To whom all correspondence should be addressed: tel: (773) 702-8268, fax: (773) 702-9861, [email protected]; [email protected], and [email protected]

Access to Document

10.1016/s0306-4573(97)00034-4

Cite this

@article{2ca62599053545cea22bc277e40b3761,

title = "An overhead reduction technique for mega-state compression schemes",

abstract = "Many of the most effective compression methods involve complicated models. Unfortunately, as model complexity increases, so does the cost of storing the model itself. This paper examines a method to reduce the amount of storage needed to represent a Markov model with an extended alphabet, by applying a clustering scheme that brings together similar states. Experiments run on a variety of large natural language texts show that much of the overhead of storing the model can be saved at the cost of a very small loss of compression efficiency.",

author = "Abraham Bookstein and Klein, {Shmuel T.} and Timo Raita",

note = "Funding Information: * The work of the first author (AB) was supported, in part, by NSF Grant IRI-9307895-A01. The author gratefully acknowledges this support. We also wish to acknowledge support given by the Academy of Finland to TR, t To whom all correspondence should be addressed: tel: (773) 702-8268, fax: (773) 702-9861, [email protected]; [email protected], and [email protected]",

year = "1997",

month = nov,

doi = "10.1016/s0306-4573(97)00034-4",

language = "אנגלית",

volume = "33",

pages = "745--760",

journal = "Information Processing and Management",

issn = "0306-4573",

publisher = "Elsevier Ltd.",

number = "6",

}

TY - JOUR

T1 - An overhead reduction technique for mega-state compression schemes

AU - Bookstein, Abraham

AU - Klein, Shmuel T.

AU - Raita, Timo

N1 - Funding Information: * The work of the first author (AB) was supported, in part, by NSF Grant IRI-9307895-A01. The author gratefully acknowledges this support. We also wish to acknowledge support given by the Academy of Finland to TR, t To whom all correspondence should be addressed: tel: (773) 702-8268, fax: (773) 702-9861, [email protected]; [email protected], and [email protected]

PY - 1997/11

Y1 - 1997/11

N2 - Many of the most effective compression methods involve complicated models. Unfortunately, as model complexity increases, so does the cost of storing the model itself. This paper examines a method to reduce the amount of storage needed to represent a Markov model with an extended alphabet, by applying a clustering scheme that brings together similar states. Experiments run on a variety of large natural language texts show that much of the overhead of storing the model can be saved at the cost of a very small loss of compression efficiency.

AB - Many of the most effective compression methods involve complicated models. Unfortunately, as model complexity increases, so does the cost of storing the model itself. This paper examines a method to reduce the amount of storage needed to represent a Markov model with an extended alphabet, by applying a clustering scheme that brings together similar states. Experiments run on a variety of large natural language texts show that much of the overhead of storing the model can be saved at the cost of a very small loss of compression efficiency.

UR - http://www.scopus.com/inward/record.url?scp=0031276750&partnerID=8YFLogxK

U2 - 10.1016/s0306-4573(97)00034-4

DO - 10.1016/s0306-4573(97)00034-4

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

AN - SCOPUS:0031276750

SN - 0306-4573

VL - 33

SP - 745

EP - 760

JO - Information Processing and Management

JF - Information Processing and Management

IS - 6

ER -

An overhead reduction technique for mega-state compression schemes

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this