• Title/Summary/Keyword: Large Scale Data

Search Result 2,773, Processing Time 0.03 seconds

Performance evaluation of principal component analysis for clustering problems

  • Kim, Jae-Hwan;Yang, Tae-Min;Kim, Jung-Tae
    • Journal of Advanced Marine Engineering and Technology
    • /
    • v.40 no.8
    • /
    • pp.726-732
    • /
    • 2016
  • Clustering analysis is widely used in data mining to classify data into categories on the basis of their similarity. Through the decades, many clustering techniques have been developed, including hierarchical and non-hierarchical algorithms. In gene profiling problems, because of the large number of genes and the complexity of biological networks, dimensionality reduction techniques are critical exploratory tools for clustering analysis of gene expression data. Recently, clustering analysis of applying dimensionality reduction techniques was also proposed. PCA (principal component analysis) is a popular methd of dimensionality reduction techniques for clustering problems. However, previous studies analyzed the performance of PCA for only full data sets. In this paper, to specifically and robustly evaluate the performance of PCA for clustering analysis, we exploit an improved FCBF (fast correlation-based filter) of feature selection methods for supervised clustering data sets, and employ two well-known clustering algorithms: k-means and k-medoids. Computational results from supervised data sets show that the performance of PCA is very poor for large-scale features.

An Implementation of Large Scale JMS(Java Message System) for Transmission Time Minimization (JMS 메시지 송수신 시간의 최소화를 위한 대용량 메시지 송수신 플랫폼 구현)

  • Cho, Poong-Youn;Park, Jae-Won;Choi, Jae-Hyun;Lee, Nam-Yong
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.15 no.1
    • /
    • pp.29-37
    • /
    • 2009
  • Recently, message based data transmission plays an important role in modem computing systems. Especially JMS(Java Message Service) is one of the most popular messaging platform. However, because of its characteristics for maintaining reliability, if we want to use it for transmission of large scale messages in a distributed Internet environment by using a WAN connection which may not be robust enough, we need to employ a different method to minimize total transmission time of messages. We found the fact that the total time of message transmission heavily depends on size of a message. In order to achieve the ideal size of a message, we develope a novel mechanism and a system which finds the ideal size of a message and automatical1y control JMS applications for minimizing transmission time. Finally, we test the proposed mechanism and system using real-data in order to prove advantages and compared with the naive mechanism. As a conclusion, we showed that our proposed mechanism and system provide an effective way to reduce transmission time of large scale messages in distributed environment.

Large-Scale Transport of Air Pollutants in the East Asian Region: Satellite and Ground Observations (동아시아 지역에서 광역적 대기오염의 이동: 위성과 지상 관측)

  • Kim, Hak-Sung;Chung, Yong-Seung
    • Journal of the Korean earth science society
    • /
    • v.28 no.1
    • /
    • pp.123-135
    • /
    • 2007
  • Five episodes of the large-scale transport of air pollutants in East Asia and its inflow into the Korean Peninsula have been analyzed through satellite and ground observations. These episodes include regionally polluted continental airmass, which is created by the pollutants produced in the cities and the industrial regions in China, to land on or pass through the Korean Peninsula by way of the Yellow Sea. The analysis of the NOAA satellite observation data made it possible to create images by combining 3 channels of visible and infrared ray ranges and also to identify the distribution and the transport of the air pollution mass over the Yellow Sea. The ground observation data of the air pollutants gathered in Chongwon were found highly valuable in verifying the information in comparison with ones from the satellite. Especially, regarding the episodes of large-scale transport of the air pollutants, the difference of concentration between $PM_{10}\;and\;PM_{2.5}$ was found small with the increase of $PM_{2.5}$ value. The concentration of $PM_{10}$ in the episode of yellows and, however, was found much higher than that of $PM_{2.5}$. In the episode of 27 January 2006, the inflow of the regionally polluted continental air mass into the central and southwestern region of the Korean Peninsula was observed sequentially in the various ground observatories as well as by the satellite. The north-northwest airflow dissipated the clouds over from Mt. Halla in Jeju Island up to far downwind, reduced air pollution, and created von $K\acute{a}rm\acute{a}n$ vortex.

Comparison of Characteristics of Drone LiDAR for Construction of Geospatial Information in Large-scale Development Project Area (대규모 개발지역의 공간정보 구축을 위한 드론 라이다의 특징 비교)

  • Park, Joon-Kyu;Lee, Keun-Wang
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.21 no.1
    • /
    • pp.768-773
    • /
    • 2020
  • In large-scale land development for the rational use and management of national land resources, the use of geospatial information is essential for the efficient management of projects. Recently, drone LiDAR (Light Detection And Ranging) has attracted attention as an effective geospatial information construction technique for large-scale development areas, such as housing site construction and open-pit mines. Drone LiDAR can be classified into a method using SLAM (Simultaneous Localization And Mapping) technology and a GNSS (Global Navigation Satellite System)/IMU (Inertial Measurement Unit) method. On the other hand, there is a lack of analytical research on the application of drone LiDAR or the characteristics of each method. Therefore, in this study, data acquisition, processing, and analysis using SLAM and GNSS/IMU type drone LiDAR were performed, and the characteristics and utilization of each were evaluated. As a result, the height direction accuracy of drone LiDAR was -0.052~0.044m, which satisfies the allowable accuracy of geospatial information for mapping. In addition, the characteristics of each method were presented through a comparison of data acquisition and processing. Geospatial information constructed through drone LiDAR can be used in several ways, such as measuring the distance, area, and inclination. Based on such information, it is possible to evaluate the safety of large-scale development areas, and this method is expected to be utilized in the future.

Analysis of large-scale flood inundation area using optimal topographic factors (지형학적 인자를 이용한 광역 홍수범람 위험지역 분석)

  • Lee, Kyoungsang;Lee, Daeeop;Jung, Sungho;Lee, Giha
    • Journal of Korea Water Resources Association
    • /
    • v.51 no.6
    • /
    • pp.481-490
    • /
    • 2018
  • Recently, the spatiotemporal patterns of flood disasters have become more complex and unpredictable due to climate change. Flood hazard map including information on flood risk level has been widely used as an unstructured measure against flooding damages. In order to product a high-precision flood hazard map by combination of hydrologic and hydraulic modeling, huge digital information such as topography, geology, climate, landuse and various database related to social economic are required. However, in some areas, especially in developing countries, flood hazard mapping is difficult or impossible and its accuracy is insufficient because such data is lacking or inaccessible. Therefore, this study suggests a method to delineate large scale flood-prone area based on topographic factors produced by linear binary classifier and ROC (Receiver Operation Characteristics) using globally-available geographic data such as ASTER or SRTM. We applied the proposed methodology to five different countries: North Korea Bangladesh, Indonesia, Thailand and Myanmar. The results show that model performances on flood area detection ranges from 38% (Bangladesh) to 78% (Thailand). The flood-prone area detection based on the topographical factors has a great advantage in order to easily distinguish the large-scale inundation-potent area using only digital elevation model (DEM) for ungauged watersheds.

A Large Scale Distributed Presence Service System by SIP Message Control Session (SIP 메시지 제어 세션에 의한 대용량 분산 프레즌스 서비스 시스템)

  • Jang, Choonseo
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.11 no.5
    • /
    • pp.514-520
    • /
    • 2018
  • Presence service provides various information about users such as locations, status of on/offline and network access methods, and number of presence resources required by each users increases largely in mobile environment. Therefore an effective method which can reduce load of presence servers is needed. In this paper, a large scale distributed presence service system which can distribute effectively total presence system load of presence servers using message control session has been presented. This large scale distributed presence service system provides various presence information for massive volumes of users. In this study, a new message control session architecture which can dynamically distribute loads of the presence servers to multiple servers has been presented, and a new presence information data architecture for controlling load of the presence servers has been designed. In this architecture, each presence server can exchange current load level in real time to get variance of the total system load change according to user numbers, and can distribute system load to maintain load level of each server evenly. The performance of the proposed large scale distributed presence service system has been analysed by experiments. The results has been showed that average presence resource subscription processing time reduced from 42.6% to 73.6%, and average presence notification processing time reduced from 37.6% to 64.8%.

Revision of 22-year Records of Atmospheric Baseline CO2 in South Korea: Application of the WMO X2019 CO2 Scale and a New Baseline Selection Method (NIMS Filter) (지난 22년간 한반도 이산화탄소 배경농도 재산정 연구 - WMO/GAW 척도 변경과 NIMS 온실가스 배경농도 산출기법을 중심으로 -)

  • Seo, Wonick;Lee, Haeyoung;Kim, Yeon-Hee
    • Atmosphere
    • /
    • v.31 no.5
    • /
    • pp.593-606
    • /
    • 2021
  • The Korea Meteorological Administration/National Institute of Meteorological Sciences (KMA/NIMS) has monitored atmospheric CO2 at Anmyeondo (AMY) World Meteorological Organization (WMO) Global Atmosphere Watch Programme (GAW) regional station since 1999, and expanded its observations at Jeju Gosan Suwolbong station (JGS) in the South and at Ulleungdo-Dokdo stations in the East (ULD and DOK) since 2012. Due to a recent WMO CO2 scale update and a new filter (NIMS) to select baseline levels at each station, the 22 years of CO2 data are recalculated. After correction for the new CO2 scale, we confirmed that those corrected records are reasonable within the compatibility goal (±0.1 ppm of CO2) between KMA/NIMS and National Oceanic and Atmosphereic Administration (NOAA) flask-air measurements with the new scale. With the new NIMS filter, CO2 baseline levels are now more representative of the large-scale background compared to previous values, which contained large CO2 enhancements. Atmospheric CO2 observed in South Korea is 4 to 8 ppm greater than the global average while the amplitude of seasonal variation is similar (10~13 ppm) to the amplitude averaged over a comparable latitude zone (30°N-60°N). Variations in CO2 growth rate are also similar, increasing and decreasing similar to global values, as it reflects the net balance between terrestrial respiration and photosynthesis. In 2020, atmospheric CO2 continued increasing despite the COVID-19 pandemic. Even though fossil emission was reduced (around -7% globally), we still emitted large amounts of anthropogenic CO2. Overall, since CO2 has large natural variations and its source was derived from not only fossil fuel but also biomass burning, the small fossil emission reduction could not affect the atmospheric level directly.

Construction Site Safety Management System Using ZigBee Communication (지그비 통신을 이용한 건설 현장 안전 관리 시스템)

  • Lee, ChangHo;Kim, KangHee;Kim, JiWon;Choi, SangBang
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.54 no.3
    • /
    • pp.39-51
    • /
    • 2017
  • Recently, looking at construction sites with either large or small scale, accidents like collision, fall, etc. occur often. These accidents lead to not only damage of human lives but also serious economic loss. In case of large scale constructions sites, safety management systems are used to reduce industrial accidents. However in construction sites with small scale, those systems cannot be applied due to problems such as lack of compatability and high installation expense. In this case, just by putting on safety gears can also reduce industrial accidents. Therefore, in this paper, a safety management systems that can be used at both large and small scale construction sites is proposed. This safety management system consists of a smart module, a repeater and a gateway, and a monitoring system. The smart module, which is detachable, is attached to a safety helmet. This module transfers the current status of the user to the monitoring system through the repeater and the gateway. The repeater transfers the data received from the smart module to the gateway, and the gateway sends the data from the repeater to the monitoring system. The monitoring system shows the user status to the safety supervisor by displaying the data - temperature, height, intensity of illumination, images - received from the smart module. The safety supervisor can monitor the user status in real-time and take immediate action in case of emergency through this monitoring system.

Generation of Large-scale Map of Surface Sedimentary Facies in Intertidal Zone by Using UAV Data and Object-based Image Analysis (OBIA) (UAV 자료와 객체기반영상분석을 활용한 대축척 갯벌 표층 퇴적상 분류도 작성)

  • Kim, Kye-Lim;Ryu, Joo-Hyung
    • Korean Journal of Remote Sensing
    • /
    • v.36 no.2_2
    • /
    • pp.277-292
    • /
    • 2020
  • The purpose of this study is to propose the possibility of precise surface sedimentary facies classification and a more accurate classification method by generating the large-scale map of surface sedimentary facies based on UAV data and object-based image analysis (OBIA) for Hwang-do tidal flat in Cheonsu bay. The very high resolution UAV data extracted factors that affect the classification of surface sedimentary facies, such as RGB ortho imagery, Digital elevation model (DEM), and tidal channel density, and analyzed the principal components of surface sedimentary facies through statistical analysis methods. Based on principal components, input data to be used for classification of surface sedimentary facies were divided into three cases such as (1) visible band spectrum, (2) topographical elevation and tidal channel density, (3) visible band spectrum and topographical elevation, tidal channel density. The object-based image analysis classification method was applied to map the classification of surface sedimentary facies according to conditions of input data. The surface sedimentary facies could be classified into a total of six sedimentary facies following the folk classification criteria. In addition, the use of visible band spectrum, topographical elevation, and tidal channel density enabled the most effective classification of surface sedimentary facies with a total accuracy of 63.04% and the Kappa coefficient of 0.54.

Flow Characteristics in a Multistage Axial Turbine (다단 축류형 터빈의 유동 특성 해석)

  • Um InSik;Park Jun Young;Baek Je Hyun
    • 한국전산유체공학회:학술대회논문집
    • /
    • 2000.10a
    • /
    • pp.149-154
    • /
    • 2000
  • The flows through a turbomachinery tend to be extremely complex due to its inherent unsteady and viscous phenomena. A good analysis of the flows associated with rotor/stator interactions in turbomachinery will be great help in design stage. In this investigation, unsteady viscous flow structurts through one and half stage of UTRC large scale rotating axial turbine are analysed. The numerical data was compared with experimental data and showed good agreement.

  • PDF