• Journal of the Institute of Electronics Engineers of Korea TE
• /
• v.37 no.3
• /
• pp.37-44
• /
• 2000

In this paper, an efficient hardware architecture of Lempel-Ziv compressor for real time data compression is proposed. The accumulated shift operations in the Lempel-Ziv algorithm are the major problem, because many shift operations are needed to prepare a dictionary buffer and matching symbols. A new efficient architecture for the fast processing of Lempel-Ziv algorithm is presented in this paper. In this architecture, the optimization technique for dictionary size, a new comparing method of multi symbol and a rotational FIFO structure are used to control shift operations easily. For the functional verification, this architecture was modeled by C programming language, and its operation was verified by running on commercial DSP processor. Also, the design of overall architecture in VHDL was synthesized on commercial FPGA chip. The result of critical path analysis shows that this architecture runs well at the input bit rate of 256kbps with 33MHz clock frequency.

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.28 no.2
• /
• pp.265-271
• /
• 2010

This paper discusses a new software package, DEM_Comp, developed for effectively compressing large digital elevation model (DEM) data sets based on Lempel-Ziv-Welch (LZW) compression and Huffman coding. DEM_Comp was developed using the $C^{++}$ language running on a Windows-series operating system. DEM_Comp was also tested on various test sites with different territorial attributes, and the results were evaluated. Recently, a high-resolution version of the DEM has been obtained using new equipment and the related technologies of LiDAR (LIght Detection And Radar) and SAR (Synthetic Aperture Radar). DEM compression is useful because it helps reduce the disk space or transmission bandwidth. Generally, data compression is divided into two processes: i) analyzing the relationships in the data and ii) deciding on the compression and storage methods. DEM_Comp was developed using a three-step compression algorithm applying a DEM with a regular grid, Lempel-Ziv compression, and Huffman coding. When pre-processing alone was used on high- and low-relief terrain, the efficiency was approximately 83%, but after completing all three steps of the algorithm, this increased to 97%. Compared with general commercial compression software, these results show approximately 14% better performance. DEM_Comp as developed in this research features a more efficient way of distributing, storing, and managing large high-resolution DEMs.

• Journal of KIISE:Software and Applications
• /
• v.31 no.12
• /
• pp.1628-1636
• /
• 2004

Data compression techniques have been studied for decades because they saved space and time to reduce costs. The Lempel-Ziv 77 (LZ77) is a dictionary-based, lossless compression algorithm. The dictionary size of the LZ77 algorithm is fixed, and the performance of the algorithm is highly dependent on its dictionary size. In this paper, we suggest a dynamic LZ77 algorithm that changes its dictionary size during compression, and also we suggest a genetic algorithm to evolve the dictionary-resizing strategies. The suggested algorithm outperformed the original version up to about 16%.

• Spatial Information Research
• /
• v.19 no.1
• /
• pp.107-116
• /
• 2011

Due to development of environment of wireless Internet, location based services on the basis of spatial data have been increased such as real time traffic information as well as CNS(Car Navigation System) to provide mobile user with route guidance to the destination. However, the current application adopting the file-based system has limitation of managing and storing the huge amount of spatial data. In order to supplement this challenge, research which is capable of managing large amounts of spatial data based on embedded database system is surely demanded. For this reason, this study aims to suggest the lossless compression technique by using the hybrid approach of BytePacking and Lempel-Ziv which can be applicable in DBMS so as to save a mass spatial data efficiently. We apply the proposed compression technique to actual the Seoul and Inchcon metropolitan area and compared the existing method with suggested one using the same data through analyzing the query processing duration until the reconstruction. As a result of comparison, we have come to the conclusion that suggested technique is far more performance on spatial data demanding high location accuracy than the previous techniques.

• Journal of Information Processing Systems
• /
• v.12 no.2
• /
• pp.322-331
• /
• 2016

Image compression is an essential technique for saving time and storage space for the gigantic amount of data generated by images. This paper introduces an adaptive source-mapping scheme that greatly improves bit-level lossless grayscale image compression. In the proposed mapping scheme, the frequency of occurrence of each symbol in the original image is computed. According to their corresponding frequencies, these symbols are sorted in descending order. Based on this order, each symbol is replaced by an 8-bit weighted fixed-length code. This replacement will generate an equivalent binary source with an increased length of successive identical symbols (0s or 1s). Different experiments using Lempel-Ziv lossless image compression algorithms have been conducted on the generated binary source. Results show that the newly proposed mapping scheme achieves some dramatic improvements in regards to compression ratios.

• The Journal of the Korea Contents Association
• /
• v.10 no.12
• /
• pp.70-76
• /
• 2010

This paper Extended ELZW(EBCDIC Lempel Ziv Welch) algorithm uses 2 byte prefix field for pointer of a table and 1 byte suffix field for repeat counter. where, a prefix field uses a pointer(index) of compression table and a suffix field uses a counter of overlapping or recursion text data in compression table. To increase compression ratio, after construction of compression table, table data are properly packed as different bit string in accordance with a alphabet, Hangeul, and pointer respectively. Therefore, proposed ELZW algorithm is superior to 1byte LZW algorithm as 5.22 percent and superior to 2byte LZW algorithm as 8.96 percent.

• Proceedings of the Korean Information Science Society Conference
• /
• 1999.10a
• /
• pp.658-660
• /
• 1999

Ziv-Lempel 코딩 방식은 문자열이 반복해서 나올 때 뒤에 나오는 문자열을 앞에 나온 문자열에 대한 포인터로 대칭시킴으로써 압축을 한다. 따라서 이 방식을 위해서는 앞서 나온 문자열을 유지하는 사전과 문자열 매칭이 필수적이다. 그래서 이 두 가지에 효율적인 자료구조인 접미사 트리를 Ziv-Lempel 코딩 방식에 적용시키려고, 그 이후에 Fiala, Greene와 Larsson은 각각 McCreight와 Ukkonen의 접미사 트리 생성 알고리즘을 LZ77 코딩에 이용하였다. 접미사 트리를 이용한 Zv-Lempel 코딩에는 만들어진 사전, 즉 접미사 트리와 앞으로 압축될 문자열과의 가장 긴 매치는 찾는 과정이 있다. 이는 단순히 접미사 트리의 루트부터 차례로 검색해 나가도 되지만 이렇게 했을 때 걸리는 시간은 노드에서 자식을 찾는데 걸리는 분기 결정 시간에 의해 좌우된다. 즉 분기에 성형 시간 이상이 걸리면 가장 긴 매치를 찾는데도 역시 선형 시간 이상이 걸린다. 게다가 이 방법은 자기 중복(self-overlapping)의 이점을 살릴 수가 없다. Rodeh, Pratt와 Even은 McCreight의 생성 알고리즘을 이용할 때 가장 긴 매치를 바로 찾을 수 있다는 것을 발견했다. 그러나 Ukkonend의 알고리즘에 대해서는 아직 이러한 방법이 알려지지 않았다. 본 논문에서는 Ukkonen의 알고리즘에 몇가지 작업을 추가하여 전체적으로 선형시간안에 가장 긴 매치를 찾는 방법을 소개한다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.14 no.12
• /
• pp.4648-4663
• /
• 2020

This paper proposes the Lempel-Ziv 4(LZ4) compression accelerator optimized for scale-out servers in data centers. In order to reduce CPU loads caused by compression, we propose an accelerator solution and implement the accelerator on an Field Programmable Gate Array(FPGA) as heterogeneous computing. The LZ4 compression hardware accelerator is a fully pipelined architecture and applies 16 dictionaries to enhance the parallelism for high throughput compressor. Our hardware accelerator is based on the 20-stage pipeline and dictionary architecture, highly customized to LZ4 compression algorithm and parallel hardware implementation. Proposing dictionary architecture allows achieving high throughput by comparing input sequences in multiple dictionaries simultaneously compared to a single dictionary. The experimental results provide the high throughput with intensively optimized in the FPGA. Additionally, we compare our implementation to CPU implementation results of LZ4 to provide insights on FPGA-based data centers. The proposed accelerator achieves the compression throughput of 639MB/s with fine parallelism to be deployed into scale-out servers. This approach enables the low power Intel Atom processor to realize the Hadoop storage along with the compression accelerator.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.23 no.7
• /
• pp.1817-1823
• /
• 1998

V.42bis, a type of LZW(Lempel-Ziv-Welch) code, is well-known as theinter national standard is asynchronous data compression. In this paper, we analyze several undesirable phenomena arising from the application of v.42bis to high speed data transmission, and we propose a modified technique to overcome them. the proposed technique determines the proper size of the dictionary, one of important factors affecting the compression ratio, and improves the method of dictionary generation for a higher compression ratio. Furthermore, we analyze the problem of excessive mode changes and solve it to a certain degree by adjusting the threshold for mode change. By doing this, we can achieve smiller variation of the compression ratio in time. This improvement chtributes to easier and better design and control of the buffer in high speed data transmission.

• Journal of the Korean Institute of Telematics and Electronics C
• /
• v.34C no.3
• /
• pp.23-36
• /
• 1997

In tis paper, sthe new data model for the hardware implementation of lempel-ziv compression algorithm was proposed. Traditional model generates the codeword which consists of 3 bytes, the last symbol, the position and the matched length. MSB (most significant bit) of the last symbol is the comparession flag and the remaining seven bits represent the character. We confined the value of the matched length to 128 instead of 256, which can be coded with seven bits only. In the proposed model, the codeword consists of 2 bytes, the merged symbol and the position. MSB of the merged symbol is the comression flag. The remaining seven bits represent the character or the matched length according to the value of the compression flag. The proposed model reduces the compression ratio by 5% compared with the traditional model. The proposed model can be adopted to the existing hardware architectures. The incremental factors of the compression ratio are also analyzed in this paper.