• Title/Summary/Keyword: Unit type

Search Result 3,375, Processing Time 0.033 seconds

A study on the rock mass classification in boreholes for a tunnel design using machine learning algorithms (머신러닝 기법을 활용한 터널 설계 시 시추공 내 암반분류에 관한 연구)

  • Lee, Je-Kyum;Choi, Won-Hyuk;Kim, Yangkyun;Lee, Sean Seungwon
    • Journal of Korean Tunnelling and Underground Space Association
    • /
    • v.23 no.6
    • /
    • pp.469-484
    • /
    • 2021
  • Rock mass classification results have a great influence on construction schedule and budget as well as tunnel stability in tunnel design. A total of 3,526 tunnels have been constructed in Korea and the associated techniques in tunnel design and construction have been continuously developed, however, not many studies have been performed on how to assess rock mass quality and grade more accurately. Thus, numerous cases show big differences in the results according to inspectors' experience and judgement. Hence, this study aims to suggest a more reliable rock mass classification (RMR) model using machine learning algorithms, which is surging in availability, through the analyses based on various rock and rock mass information collected from boring investigations. For this, 11 learning parameters (depth, rock type, RQD, electrical resistivity, UCS, Vp, Vs, Young's modulus, unit weight, Poisson's ratio, RMR) from 13 local tunnel cases were selected, 337 learning data sets as well as 60 test data sets were prepared, and 6 machine learning algorithms (DT, SVM, ANN, PCA & ANN, RF, XGBoost) were tested for various hyperparameters for each algorithm. The results show that the mean absolute errors in RMR value from five algorithms except Decision Tree were less than 8 and a Support Vector Machine model is the best model. The applicability of the model, established through this study, was confirmed and this prediction model can be applied for more reliable rock mass classification when additional various data is continuously cumulated.

The effect of Type 2 diabetes management using a smartphone-based blood glucose management training program (모바일 자가혈당관리 교육프로그램을 이용한 2형 당뇨병 관리 효과 분석)

  • Lee, Jung-Hwa;Jung, Jin-Hee;Sim, Kang-Hee;Choi, Hee-Sun;Lee, Jeong-Rim;Kang, Yang-Gyo;Song, Bok-Rye
    • Journal of Industrial Convergence
    • /
    • v.20 no.9
    • /
    • pp.59-70
    • /
    • 2022
  • Background: Diabetes education is an important factor in blood glucose control. Reinforced education is necessary for effective diabetes education. However, it is difficult to provide reinforced diabetes education within Korea's medical environment. Therefore, we want to analyze the effect of continuous diabetes education using mobile health care that can effectively provide repeated education without having to face the patient. Methods: This study is a multicenter, randomized, controlled, pre-post design study conducted to analyze the effect of a continuous diabetes education method. A total of 109 people were registered at five hospitals in south Korea, and they were randomly assigned to the app group (34 people) who received real-time coaching and repetitive training, the logbook group (37 people) who received face-to-face training after writing a blood glucose logbook, and the general group (38 people) who received a one-time diabetes education. The study was conducted for a total of 24 weeks. Twenty-one patients withdrew their consent and failed to perform an HbA1c. A final 88 patients were analyzed. The difference in HbA1c, Self-management behavior, and Quality of life before and after education was analyzed. Results: The study involved 51 (58%) male subjects, mean age was 55.8 years and mean duration of diabetes was 7.6 years. After 24 weeks of intervention, there was no significant difference in self-care behavior and quality of life between the three groups, but the HbA1c of the app group significantly decreased after education compared to the logbook group and the general group (F=4.62, p=.013). Conclusion: It can be seen through the app group that receiving real-time education is more effective in improving blood glucose management and continuous diabetes education is important.

Reliability Analysis on Stability of Armor Units for Foundation Mound of Composite Breakwaters (혼성제 기초 마운드의 피복재 안정성에 대한 신뢰성 해석)

  • Cheol-Eung Lee
    • Journal of Korean Society of Coastal and Ocean Engineers
    • /
    • v.35 no.2
    • /
    • pp.23-32
    • /
    • 2023
  • Probabilistic and deterministic analyses are implemented for the armor units of rubble foundation mound of composite breakwaters which is needed to protect the upright section against the scour of foundation mounds. By a little modification and incorporation of the previous empirical formulas that has commonly been applied to design the armor units of foundation mound, a new type formula of stability number has been suggested which is capable of taking into account slopes of foundation mounds, damage ratios of armor units, and incident wave numbers. The new proposed formula becomes mathematically identical with the previous empirical formula under the same conditions used in the developing process. Deterministic design have first been carried out to evaluate the minimum weights of armor units for several conditions associated with a typical section of composite breakwater. When the slopes of foundation mound become steepening and the incident wave numbers are increasing, the bigger armor units more than those from the previous empirical formula should be required. The opposite trends however are shown if the damage ratios is much more allowed. Meanwhile, the reliability analysis, which is one of probabilistic models, has been performed in order to quantitatively verify how the armor unit resulted from the deterministic design is stable. It has been confirmed that 1.2% of annual encounter probability of failure has been evaluated under the condition of 1% damage ratio of armor units for the design wave of 50 years return period. By additionally calculating the influence factors of the related random variables on the failure probability due to those uncertainties, it has been found that Hudson's stability coefficient, significant wave height, and water depth above foundation mound have sequentially been given the impacts on failure regardless of the incident wave angles. Finally, sensitivity analysis has been interpreted with respect to the variations of random variables which are implicitly involved in the formula of stability number for armor units of foundation mound. Then, the probability of failure have been rapidly decreased as the water depth above foundation mound are deepening. However, it has been shown that the probability of failure have been increased according as the berm width of foundation mound are widening and wave periods become shortening.

Analysis of the Pre-service Chemistry Teachers' Cognition of the Nature of Model in the Design and Development Process of Models Using Technology: Focusing on Boyle's Law (테크놀로지를 활용한 모델의 설계와 개발 과정에서 나타난 예비화학교사의 모델의 본성에 대한 인식 분석: 보일 법칙을 중심으로)

  • Na-Jin Jeong;Seoung-Hey Paik
    • Journal of the Korean Chemical Society
    • /
    • v.67 no.5
    • /
    • pp.378-392
    • /
    • 2023
  • The purpose of this study is to analyze the pre-service chemistry teachers' cognition of the nature of model in process of designing and developing models using technology. For this purpose, 19 pre-service chemistry teachers' in the 3rd grade of a education college located in the central region observe experimental phenomena related to Boyle's law presented in the 7th grade science textbook and researchers required the design and development of a model related to the observed experimental results using technology. Based on previous studies, the nature of model were classified into two aspect: 'Representational aspect' and 'Explanatory aspect'. The 'Representational aspect' was classified into 'Representation', 'Abstraction', and 'Simplification', and the 'Explanatory aspect' was classified into 'Analysis', 'Interpretation', 'Reasoning', 'Explanation', and 'Quantification'. The pre-service chemistry teachers' cognition were analyzed by the classification. As a result of the study, the 'Representation' of the 'expressive aspect' was uniformized in the form of space that changes in volume, and the pressure was expressed as the Brightness inside the cylinder or frequency of color change of particles for 'Abstraction'. In the case of 'Simplification', the particle collision was expressed as a perfectly elastic collision, but there was a group that could not simply indicate the type of particle. In the 'Explanatory aspect', in the case of 'Analysis', volume was classified as a manipulated variable, and in the case of 'Interpretation', most groups analyzed the change in pressure through the collision of gas particles. However, the cognition involved in 'Reasoning' was not observed much. In the case of 'Explanation', there were groups that did not succeed in explanation because the area where the particles collided was not set or incorrectly set, and in the case of 'Quantification', there was a group that formulated the number of collisions per unit time, and on the contrary, there was a group that could not quantify the number of collisions because they could not be expressed in numbers.

Development of Tree Carbon Calculator to Support Landscape Design for the Carbon Reduction (탄소저감설계 지원을 위한 수목 탄소계산기 개발 및 적용)

  • Ha, Jee-Ah;Park, Jae-Min
    • Journal of the Korean Institute of Landscape Architecture
    • /
    • v.51 no.1
    • /
    • pp.42-55
    • /
    • 2023
  • A methodology to predict the carbon performance of newly created urban greening plans is required as policies based on quantifying carbon performance are rapidly being introduced in the face of the climate crisis caused by global warming. This study developed a tree carbon calculator that can be used for carbon reduction designs in landscaping and attempted to verify its effectiveness in landscape design. For practical operability, MS Excel was selected as a format, and carbon absorption and storage by tree type and size were extracted from 93 representative species to reflect plant design characteristics. The database, including tree unit prices, was established to reflect cost limitations. A plantation experimental design to verify the performance of the tree carbon calculator was conducted by simulating the design of parks in the central region for four landscape design, and the causal relationship was analyzed by conducting semi-structured interviews before and after. As a result, carbon absorption and carbon storage in the design using the tree carbon calculator were about 17-82% and about 14-85% higher, respectively, compared to not using it. It was confirmed that the reason for the increase in carbon performance efficiency was that additional planting was actively carried out within a given budget, along with the replacement of excellent carbon performance species. Pre-interviews revealed that designers distrusted data and the burdens caused by new programs before using the arboreal carbon calculator but tended to change positively because of its usefulness and ease of use. In order to implement carbon reduction design in the landscaping field, it is necessary to develop it into a carbon calculator for trees and landscaping performance. This study is expected to present a useful direction for ntroducing carbon reduction designs based on quantitative data in landscape design.

Fish Community Structure and Biodiversity of the Korean Peninsula Estuaries (한반도 하구의 어류군집 구조 및 다양성)

  • Park, Sang-Hyeon;Baek, Seung-Ho;Kim, Jeong-Hui;Kim, Dong-Hwan;Jang, Min-Ho;Won, Doo-Hee;Park, Bae-Kyung;Moon, Jeong-Suk
    • Korean Journal of Ecology and Environment
    • /
    • v.55 no.1
    • /
    • pp.35-48
    • /
    • 2022
  • Fish assemblage of total 325 of Korean peninsula estuaries were surveyed to analyze the characteristics of community structure and diversity by sea areas for three years from 2016 to 2018. The scale (stream width) of Korean estuaries were various (14~3,356 m), and 68.9% of all estuaries showed salinity of less than 2 psu. Total 149 species classified into 52 families of fish were identified, and the dominant and sub-dominant species were Tribolodon hakonensis (relative abundance, RA, 12.5%) and Mugil cephalus (RA, 9.5%), respectively. The estuary of the Korean Peninsula had different physical and chemical habitat environments depending on the sea area, and accordingly, fish community structure also showed statistically significant differences (PERMANOVA, Pseudo-F=26.69, P=0.001). In addition, the NMDS (nonmetric multidimensional scaling) results showed the patterns that indicating fish community difference by sea areas, even though low community similarity within sea area (SIMPER, 21.79~26.39%). The estuaries of east sea areas were distinguished from the others in the aspects of which, the higher importance of migratory fishes and endangered species, and that of brackish species were characterized at south sea estuaries. However, the estuaries of west sea showed higher importance of species that have a relation with freshwater (primary freshwater species, exotic species), which is the result that associating with the lower salinity of west sea estuaries because of the high ratio of closed estuaries(78.2%). The SIMPER analysis, scoring the contribution rates of species to community similarity, also showed results corresponding to the tendency of different fish community structures according to each sea area. So far, In Korea, most studies on fish communities in estuaries have been conducted in a single estuary unit, which made it difficult to understand the characteristics of estuaries at the national level, which are prerequisite for policy establishment. In present study, we are providing fish community structure characteristics of Korean estuaries in a national scale, including diversity index, habitat salinity ranges of major species, distribution of migratory species. We are expecting that our results could be utilized as baseline information for establishing management policies or further study of Korean estuaries.

A preliminary study on the village landscape in Baengpo Bay, Haenam Peninsula - Around the Bronze Age - (해남반도 백포만일대 취락경관에 대한 시론 - 청동기시대를 중심으로 -)

  • KIM Jinyoung
    • Korean Journal of Heritage: History & Science
    • /
    • v.56 no.3
    • /
    • pp.62-74
    • /
    • 2023
  • Much attention has been focused on the Baekpoman area due to the archaeological achievements of the past, but studies on prehistoric times when villages began to form is insufficient, and the Bronze Age village landscape was examined in order to supplement this. In the area of Baekpo Bay, the natural geographical limit connected to the inland was culturally confirmed by the distribution density of dolmens, and the generality of the Bronze Age settlement was confirmed with the Hwangsan-ri settlement. Bunto Village in Hwangsan-ri represents a farming-based village in the Baekpo Bay area, and the residential group and the tomb group are located on the same hill, and it is composed of three individual residential groups, and the village landscape had attached buildings used as warehouses and storage facilities. In the area of Baekpo Bay, it spread in the Tamjin River basin and the Yeongsan River basin where Songgukri culture and dolmen culture were integrated, and the density distribution of the villages was considered to correspond to the distribution density of dolmens. In order to examine the landscape of village distribution, the classification of Sochon-Jungchon-Daechon was applied, and it was classified as Sochon, a sub-unit constituting the village, in that the number of settlements constituting the village in the Bronze Age was mostly less than five. There are numerical differences between Jungchon and Daechon, and the distribution pattern does not necessarily coincide with the hierarchy. The three individual residential groups of Bunto Village in Hwangsan-ri are Jungchon composed of complex communities of blood relatives with each family community, and a stabilized village landscape was created in the Gusancheon area. In the area of Baekpo Bay, Bronze Age villages formed a landscape in which small villages were scattered around the rivers and formed a single-layered relationship. Dolmens (tombs) were formed between the villages and villages, and seem to have coexisted. Sochondeul is a family community based on agriculture, and it is believed that self-sufficient stabilized rural villages that live by acquiring various wild resources in rivers, mountains, and the sea formed a landscape.

Depth Control and Sweeping Depth Stability of the Midwater Trawl (중층트롤의 깊이바꿈과 소해심도의 안정성)

  • 장지원
    • Journal of the Korean Society of Fisheries and Ocean Technology
    • /
    • v.9 no.1
    • /
    • pp.1-18
    • /
    • 1973
  • For regulating the depth of midwater trawl nets towed at the optimum constant speed, the changes in the shape of warps caused by adding a weight on an arbitrary point of the warp of catenary shape is studied. The shape of a warp may be approximated by a catenary. The resultant inferences under this assumption were experimented. Accordingly feasibilities for the application of the result of this study to the midwater trawl nets were also discussed. A series of experiments for basic midwater trawl gear models in water tank and a couple of experiments of a commercial scale gears at sea which involve the properly designed depth control devices having a variable attitude horizontal wing were carried out. The results are summarized as follows: 1. According to the dimension analysis the depth y of a midwater trawl net is introduced by $$y=kLf(\frac{W_r}{R_r},\;\frac{W_o}{R_o},\;\frac{W_n}{R_n})$$) where k is a constant, L the warp length, f the function, and $W_r,\;W_o$ and $W_n$ the apparent weights of warp, otter board and the net, respectively, 2. When a boat is towing a body of apparent weight $W_n$ and its drag $D_n$ by means of a warp whose length L and apparent weight $W_r$ per unit length, the depth y of the body is given by the following equation, provided that the shape of a warp is a catenary and drag of the warp is neglected in comparison with the drag of the body: $$y=\frac{1}{W_r}\{\sqrt{{D_n^2}+{(W_n+W_rL)^2}}-\sqrt{{D_n^2+W_n}^2\}$$ 3. The changes ${\Delta}y$ of the depth of the midwater trawl net caused by changing the warp length or adding a weight ${\Delta}W_n$_n to the net, are given by the following equations: $${\Delta}y{\approx}\frac{W_n+W_{r}L}{\sqrt{D_n^2+(W_n+W_{r}L)^2}}{\Delta}L$$ $${\Delta}y{\approx}\frac{1}{W_r}\{\frac{W_n+W_rL}{\sqrt{D_n^2+(W_n+W_{r}L)^2}}-{\frac{W_n}{\sqrt{D_n^2+W_n^2}}\}{\Delta}W_n$$ 4. A change ${\Delta}y$ of the depth of the midwater trawl net by adding a weight $W_s$ to an arbitrary point of the warp takes an equation of the form $${\Delta}y=\frac{1}{W_r}\{(T_{ur}'-T_{ur})-T_u'-T_u)\}$$ Where $$T_{ur}^l=\sqrt{T_u^2+(W_s+W_{r}L)^2+2T_u(W_s+W_{r}L)sin{\theta}_u$$ $$T_{ur}=\sqrt{T_u^2+(W_{r}L)^2+2T_uW_{r}L\;sin{\theta}_u$$ $$T_{u}^l=\sqrt{T_u^2+W_s^2+2T_uW_{s}\;sin{\theta}_u$$ and $T_u$ represents the tension at the point on the warp, ${\theta}_u$ the angle between the direction of $T_u$ and horizontal axis, $T_u^2$ the tension at that point when a weights $W_s$ adds to the point where $T_u$ is acted on. 5. If otter boards were constructed lighter and adequate weights were added at their bottom to stabilize them, even they were the same shapes as those of bottom trawls, they were definitely applicable to the midwater trawl gears as the result of the experiments. 6. As the results of water tank tests the relationship between net height of H cm velocity of v m/sec, and that between hydrodynamic resistance of R kg and the velocity of a model net as shown in figure 6 are respectively given by $$H=8+\frac{10}{0.4+v}$$ $$R=3+9v^2$$ 7. It was found that the cross-wing type depth control devices were more stable in operation than that of the H-wing type as the results of the experiments at sea. 8. The hydrodynamic resistance of the net gear in midwater trawling is so large, and regarded as nearly the drag, that sweeping depth of the gear was very stable in spite of types of the depth control devices. 9. An area of the horizontal wing of the H-wing type depth control device was $1.2{\times}2.4m^2$. A midwater trawl net of 2 ton hydrodynamic resistance was connected to the devices and towed with the velocity of 2.3 kts. Under these conditions the depth change of about 20m of the trawl net was obtained by controlling an angle or attack of $30^{\circ}$.

  • PDF

Studies on Grain Filling and Quality Changes of Hard and Soft Wheat Grown under the Different Environmental Conditions (환경 변동에 따른 경ㆍ연질 소맥의 등숙 및 품질의 변화에 관한 연구)

  • Young-Soo Han
    • KOREAN JOURNAL OF CROP SCIENCE
    • /
    • v.17
    • /
    • pp.1-44
    • /
    • 1974
  • These studies were made at Suwon in 1972 and at Suwon, Iri, and Kwangju in 1973 to investigate grain filling process and variation of grain quality of NB 68513 and Caprock as hard red winter wheat, Suke #169 as soft red winter wheat variety and Yungkwang as semi-hard winter variety, grown under-three different fertilizer levels and seeding dates. Other experiments were conducted to find the effects of temperature, humidity and light intensity on the grain filling process and grain quality of Yungkwang and NB 68513 wheat varieties. These, experiments were conducted at Suwon in 1973 and 1974. 1. Grain filling process of wheat cultivars: 1) The frequency distribution of a grain weight shows that wider distribution of grain weight was associated with large grain groups rather than small grain group. In the large grain groups, the frequency was mostly concentrated near mean value, while the frequency was dispersed over the values in the small grain group. 2) The grain weight was more affected by the grain thickness and width than by grain length. 3) The grain weight during the ripening period was rapidly increased from 14 days after flowering to 35 days in Yungkwang and from 14 days after flowering to 28 days in NB 68513. The large grain group, Yungkwang was rather slowly increased and took a longer period in increase of endosperm ratio of grain than the small grain group, NB 68513. 4) In general, the 1, 000 grain weight was reduced under high temperature, low humidity, while it was increased under low temperature and high humidity condition, and under high temperature and humidity condition. The effect of shading on grain weight was greater in high temperature than in low temperature condition and no definite tendency was found in high humidity condition. 5) The effects of temperature, humidity and shading on 1, 000 grain weight were greater in large-grain group, Yungkwang than in small grain group, NB 68513. Highly significant positive correlation was found between 1, 000 grain weight and days to ripening. 6) The 1, 000 grain weight and test weight were increased more or less as the fertilizer levels applied were increased. However, the rate of increasing 1, 000 grain weight was low when fertilizer levels were increased from standard to double. The 1, 000 grain weight was high when planted early. Such tendency was greater in Suwon than in Kwangju or Iri area. 2. Milling quality: 7) The milling rate in a same group of varieties was higher under the condition of low temperature, high humidity and early maturing culture which were responsible for increasing 1, 000 grain weight. No definite relations were found along with locations. 8) In the varieties tested, the higher milling rate was found in large grain variety, Yungkwang, and the lowest milling rate was obtained from Suke # 169, the small grain variety. But the small grained hard wheat variety such as Caprock and NB 68513 showed higher milling rate compared with the soft wheat variety, Suke # 169. 9) There were no great differences of ash content due to location, fertilizer level and seeding date while remarkable differences due to variety were found. The ash content was high in the hard wheat varieties such as NB 68513, Caprock and low in soft wheat varieties such as Yungkwang and Suke # 169. 3. Protein content: 10) The protein content was increased under the condition of high temperature, low humidity and shading, which were responsible for reduction of 1, 000 grain weight. The varietal differences of protein content due to high temperature, low humidity and shading conditions were greater in Yungkwang than in NB 68513. 11) The high content of protein in grain within one to two weeks after flowering might be due to the high ratio of pericarp and embryo to endosperm. As grains ripen, the effects of embryo and pericarp on protein content were decreased, reducing protein content. However, the protein content was getting increased from three or four weeks after flowering, and maximized at seven weeks after flowering. The protein content of grain at three to four weeks after flowering increased as the increase of 1, 000 grain weight. But the protein content of matured grain appeared to be affected by daily temperature on calender rather than by duration of ripening period. 12) Highly significant positive correlation value was found between the grain protein content and flour protein content. 13) The protein content was increased under the high level of fertilizers and late seeding. The local differences of protein content were greater in Suwon than in Kwangju and Iri. 14) Protein content in the varieties tested were high in Yungkwang, NB 68513 and Caprock, and low in Suke # 169. However, variation in protein content due to the cultural methods was low in Suke # 169. 15) Protein yield per unit area was increased in accordance with increase of fertilizer levels and early maturing culture. However, nitrogen fertilizer was utilized rather effectively in early maturing culture and Yungkwang was the highest in protein yield per unit area. 4. Physio-chemical properties of wheat flour: 16) Sedimentation value was higher under the conditions of high temperature, low humidity and high levels of fertilizers than under the conditions of low temperature, high moisture and low levels of fertilizers. Such differences of sedimentation values were more apparent in NB 68513 and Caprock than Yungkwang and Suke # 169. The local difference of sedimentation value was greater in Suwon than in Kwangju and Iri. Even though the sedimentation value was highly correlated with protein content of grain, the high humidity was considered one of the factors affecting sedimentation value. 17) Changes of Pelshenke values due to the differences of cultural practices and locations were generally coincident with sedimentation values. 18) The mixing time required for mixogram was four to six minutes in NB 68513, five to seven minutes in Cap rock. The great variation of mixing time for Yungkwang and Suke # 169 due to location and planting conditions was found. The mixing height and area were high in hard wheat than in soft wheat. Variation of protein content due to cultural methods were inconsistent. However, the pattern of mixogram were very much same regardless the treatments applied. With this regard, it could be concluded that the mixogram is a kind of method expressing the specific character of the variety. 19) Even though the milling property of NB 68513 and Caprock was deteriorated under either high temperature and low humidity of high fertilizer levels and late seeding conditions, baking quality was better due to improved physio-chemical properties of flour. In contrast, early maturing culture deteriorated physio-chemical properties, milling property of grain and grain protein yield per unit area was increased. However, it might be concluded that the hard wheat production of NB 68513 and Caprock for baking purpose could be done better in Suwon than in Iri or Kwangju area. 5. Interrelationships between the physio-chemical characters of wheat flour: 20) Physio-chemical properties of flour didn't have direct relationship with milling rate and ash content. Low grain weight produced high protein content and better physio-chemical flour properties. 21) In hard wheat varieties like NB 68513 and Caprock, protein content was significantly correlated with sedimentation value, Pelshenke value and mixing height. However, gluten strength and baking quality were improved by the increased protein content. In Yungkwang and Suk # 169, protein content was correlated with sedimentation value, but no correlations were found with Pelshenke value and mixing height. Consequently, increase of protein content didn't improve the gluten strength in soft wheat. 22) The highly significant relationships between protein content and gluten strength and sedimentation . value, and between Pelshenke value, mixogram and gluten strength indicated that the determination of mixogram and Pelshenke value are useful for de terming soft and hard type of varieties. Determination of sedimentation value is considered useful method for quality evaluation of wheat grain under different cultural practices.

  • PDF

Design and Implementation of MongoDB-based Unstructured Log Processing System over Cloud Computing Environment (클라우드 환경에서 MongoDB 기반의 비정형 로그 처리 시스템 설계 및 구현)

  • Kim, Myoungjin;Han, Seungho;Cui, Yun;Lee, Hanku
    • Journal of Internet Computing and Services
    • /
    • v.14 no.6
    • /
    • pp.71-84
    • /
    • 2013
  • Log data, which record the multitude of information created when operating computer systems, are utilized in many processes, from carrying out computer system inspection and process optimization to providing customized user optimization. In this paper, we propose a MongoDB-based unstructured log processing system in a cloud environment for processing the massive amount of log data of banks. Most of the log data generated during banking operations come from handling a client's business. Therefore, in order to gather, store, categorize, and analyze the log data generated while processing the client's business, a separate log data processing system needs to be established. However, the realization of flexible storage expansion functions for processing a massive amount of unstructured log data and executing a considerable number of functions to categorize and analyze the stored unstructured log data is difficult in existing computer environments. Thus, in this study, we use cloud computing technology to realize a cloud-based log data processing system for processing unstructured log data that are difficult to process using the existing computing infrastructure's analysis tools and management system. The proposed system uses the IaaS (Infrastructure as a Service) cloud environment to provide a flexible expansion of computing resources and includes the ability to flexibly expand resources such as storage space and memory under conditions such as extended storage or rapid increase in log data. Moreover, to overcome the processing limits of the existing analysis tool when a real-time analysis of the aggregated unstructured log data is required, the proposed system includes a Hadoop-based analysis module for quick and reliable parallel-distributed processing of the massive amount of log data. Furthermore, because the HDFS (Hadoop Distributed File System) stores data by generating copies of the block units of the aggregated log data, the proposed system offers automatic restore functions for the system to continually operate after it recovers from a malfunction. Finally, by establishing a distributed database using the NoSQL-based Mongo DB, the proposed system provides methods of effectively processing unstructured log data. Relational databases such as the MySQL databases have complex schemas that are inappropriate for processing unstructured log data. Further, strict schemas like those of relational databases cannot expand nodes in the case wherein the stored data are distributed to various nodes when the amount of data rapidly increases. NoSQL does not provide the complex computations that relational databases may provide but can easily expand the database through node dispersion when the amount of data increases rapidly; it is a non-relational database with an appropriate structure for processing unstructured data. The data models of the NoSQL are usually classified as Key-Value, column-oriented, and document-oriented types. Of these, the representative document-oriented data model, MongoDB, which has a free schema structure, is used in the proposed system. MongoDB is introduced to the proposed system because it makes it easy to process unstructured log data through a flexible schema structure, facilitates flexible node expansion when the amount of data is rapidly increasing, and provides an Auto-Sharding function that automatically expands storage. The proposed system is composed of a log collector module, a log graph generator module, a MongoDB module, a Hadoop-based analysis module, and a MySQL module. When the log data generated over the entire client business process of each bank are sent to the cloud server, the log collector module collects and classifies data according to the type of log data and distributes it to the MongoDB module and the MySQL module. The log graph generator module generates the results of the log analysis of the MongoDB module, Hadoop-based analysis module, and the MySQL module per analysis time and type of the aggregated log data, and provides them to the user through a web interface. Log data that require a real-time log data analysis are stored in the MySQL module and provided real-time by the log graph generator module. The aggregated log data per unit time are stored in the MongoDB module and plotted in a graph according to the user's various analysis conditions. The aggregated log data in the MongoDB module are parallel-distributed and processed by the Hadoop-based analysis module. A comparative evaluation is carried out against a log data processing system that uses only MySQL for inserting log data and estimating query performance; this evaluation proves the proposed system's superiority. Moreover, an optimal chunk size is confirmed through the log data insert performance evaluation of MongoDB for various chunk sizes.