• Title/Summary/Keyword: fine grained

Search Result 718, Processing Time 0.029 seconds

Structured Pruning for Efficient Transformer Model compression (효율적인 Transformer 모델 경량화를 위한 구조화된 프루닝)

  • Eunji Yoo;Youngjoo Lee
    • Transactions on Semiconductor Engineering
    • /
    • v.1 no.1
    • /
    • pp.23-30
    • /
    • 2023
  • With the recent development of Generative AI technology by IT giants, the size of the transformer model is increasing exponentially over trillion won. In order to continuously enable these AI services, it is essential to reduce the weight of the model. In this paper, we find a hardware-friendly structured pruning pattern and propose a lightweight method of the transformer model. Since compression proceeds by utilizing the characteristics of the model algorithm, the size of the model can be reduced and performance can be maintained as much as possible. Experiments show that the structured pruning proposed when pruning GPT-2 and BERT language models shows almost similar performance to fine-grained pruning even in highly sparse regions. This approach reduces model parameters by 80% and allows hardware acceleration in structured form with 0.003% accuracy loss compared to fine-tuned pruning.

A Case Study on Local Erosion Characteristics Evaluation of the Inchon Coast (인천 해안지역 지반의 국지적 침식특성 평가)

  • Kwak, Ki-Seok;Lee, Ju-Hyung;Park, Jae-Hyeon;Chung, Moon-Kyung
    • Proceedings of the Korean Geotechical Society Conference
    • /
    • 2006.03a
    • /
    • pp.455-465
    • /
    • 2006
  • In this study, the effect of scour was evaluated by regional and geotechnical characteristics and back data were accumulate for the design against scour through the local erosion characteristics evaluation of the Inchon coast. The erosion characteristics for the undisturbed soil samples collected near the main locations at the Incheon 2nd bridge, the Hwangyeong bridge, and a coast road in Songdo, are determined quantitatively through the scour rate tests. On the basic soil properties test, the bed around the Inchon coast chiefly consists of fined grained soils, and the soil samples were classified as silty clay(ML) or clay(CL) under the Unified Soil Classification System. On the scour rate test, the critical shear stress increases when the undrained shear strength increases as of the general trend of fine grained soils, and the average scour rate for the maximum velocity by 100 year flood is 173mm/hr at the Incheon 2nd bridge, 67mm/hr at the Hwangyeong bridge and 10mm/hr at a coast road in Songdo, respectively. Comparing to the scour rate of coarse grained soil, that of the bed around the Inchon coast is turned out to be very low. Therefore, the relative ability of the bed around the Inchon coast to resist erosion is assumed to be very high.

  • PDF

Sediment Characteristics of Waste Disposal Sites in the Southwestern UUeung Basin, the East Sea (동해 울릉분지 남서해역 해양투기장의 퇴적물 특성)

  • Chun, Jong-Hwa;Huh, Sik;Han, Sang-Joon;Shin, Dong-Hyeok;Cheong, Dae-Kyo;Hong, Ki-Hoon;Kim, Suk-Hyun
    • The Sea:JOURNAL OF THE KOREAN SOCIETY OF OCEANOGRAPHY
    • /
    • v.4 no.4
    • /
    • pp.312-322
    • /
    • 1999
  • We have studied both submarine morphology and sediment characteristics of waste disposal sites in the southwestern Ulleung Basin, East Sea, as part of a marine environmental preservation program. The Jung waste disposal site in the outer shelf is characterized by the thick accumulation of coarse-grained palimpsest sediments and fine-grained sediments from various sources. The Byung waste disposal site in the continental slope is generally characterized by hemipelagic muds with intermittent sandy sediments originated from the outer shelf and upper slope. The hemipelagic sediments, draping the seafloor, consist of fluidized muds. The core sediments show numerous bioturbation structures which cause vertical mixing of sediments. The surface sediments can be divided into four sand types (S-1, S-2, S-3, and S-4) and two mud types (M-1 and M-2) based on relative contents of reworked coarse-grained palimpsest sediments and fine-grained sediments. sorting and heavy mineral contents. The sands are probably relict sediments reworked during high-energy conditions such as typoon or storm. On the other hand, the muds were originated from various sources such as recent input from the Nakdong River, reworked fine-grained sediment from the shelf or suspended particulate matter from the East Sea Warm Current.

  • PDF

Effective Design Pattern and Enterprise Architecture Design Techniques in EJB Environment (EJB기반의 효율적인 설계 패턴 및 엔터프라이즈 아키텍처 설계 기법)

  • 민현기;김수동
    • Journal of KIISE:Software and Applications
    • /
    • v.30 no.11
    • /
    • pp.1025-1036
    • /
    • 2003
  • In industry, it is a current trend that systems are developed by using Enterprise JavaBeans(EJB) technology for reducing the cost and the time. Thus, the architecture of EJB is getting more essential to enhance reusability, extensibility and portability of system. However little has been studied in the realm of the practical software architectures for EJB. The architecture has just bean studied in abstract level, but not in concrete level providing the method to substantiate it using the practical J2EE techniques. Just using the EJB technology doesn't guarantee the reusability of the artifacts because EJB specification provides the characteristics and architecture for only fine grained components as session and entity bean. In this paper, we propose the enterprise software architecture for the systems based on EJB and the concrete techniques for implementing that. Also, design patterns of modeling efficient enterprise architecture are represented. By analyzing both the strengths and the weaknesses of suggested design patterns, EJB design patterns which are suitable for each layer of enterprise architecture will be identified. Through the component which design patterns are applied, the architecture can support the optimized relationship between the components. Five techniques for designing components from fine grained to coarse grained based on EJB technology, and architecture design techniques including transaction and assembling techniques are proposed.

Phase identification and degree of orientation measurements far fine-grained rock forming minerals using micro-area X-ray diffractometer -$Al_{2}SiO_{5}$ Polymorphs- (미소부 X-선 회절분석기를 이용한 미립조암광물의 상동정 및 배향도 측정 -$Al_{2}SiO_{5}$ 3상다형-)

  • 박찬수;김형식
    • The Journal of the Petrological Society of Korea
    • /
    • v.9 no.4
    • /
    • pp.205-210
    • /
    • 2000
  • Measurements of phase identification and degree of orientation for fine-grained (about 0.3 mm in diameter) minerals in rock samples performed by micro-area X-ray diffractometer.$Al_{2}SiO_{5}$ polymorphs (andalusite, kyanite and sillimanite) were chosen for the measurements and target minerals were existed on thin sections. Micro-area X-ray diffractometer is composed of 3(${\omega}\;{\chi}\;{\phi}$)-circle oscillating goniometer and position sensitive proportional counter (PSPC). $CuK_{\alpha}$ radiation was used as X-ray source and a pin hole ($50\;\mu\textrm{m}$$ in diameter) collimator was selected to focus radiation X-ray onto the target minerals. Phase identification and diffracted X-ray peak indexing were carried out by 3(${\omega}\;{\chi}\;{\phi}$)-circle oscillation measurement. Then, 2(${\omega}\;{\phi}$)-circle oscillation measurement was made for the purpose of searching the prevailing lattice plane of the minerals on thin section surface. Finally, for a selected peak by 2-circle oscillation measurement, X-ray pole figure measurement was executed for the purpose of check the degree of orientation of the single lattice direction and examine its pole distribution. As a result of 3-circle oscillation measurement, it was possible that phase identification among $Al_{2}SiO_{5}$ polymorphs. And from the results of 2-circle oscillation measurement and X-ray pole figure measurement, we recognized that poles of andalusite (122), kyanite (200) and sillimanite (310) lattice plances were well developed with direction normal to each mineral surface plane respectively. Therfore, the measurements used with micro-area X-ray diffractometer in this study will be a useful tool of phase identification and degree of orientation measurement for fine-grained rock forming minerals.

  • PDF

Rheological Models for Describing Fine-laden Debris Flows: Grain-size Effect (세립토 위주의 토석류에 관한 유변학적 모델: 입자크기 효과)

  • Jeong, Sueng-Won
    • Journal of the Korean Geotechnical Society
    • /
    • v.27 no.6
    • /
    • pp.49-61
    • /
    • 2011
  • This paper presents the applicability of rheological models for describing fine-laden debris flows and analyzes the flow characteristics as a function of grain size. Two types of soil samples were used: (1) clayey soils - Mediterranean Sea clays and (2) silty soils - iron ore tailings from Newfoundland, Canada. Clayey soil samples show a typical shear thinning behavior but silty soil samples exhibit the transition from shear thinning to the Bingham fluid as shear rate is increased. It may be due to the fact that the determination of yield stress and plastic viscosity is strongly dependent upon interstructrual interaction and strength evolution between soil particles. So grain size effect produces different flow curves. For modeling debris flows that are mainly composed of fine-grained sediments (<0.075 mm), we need the yield stress and plastic viscosity to mimic the flow patterns like shape of deposition, thickness, length of debris flow, and so on. These values correlate with the liquidity index. Thus one can estimate the debris flow mobility if one can measure the physical properties.