• Title/Summary/Keyword: 병렬시스템

Search Result 2,500, Processing Time 0.034 seconds

Big Data-based Sensor Data Processing and Analysis for IoT Environment (IoT 환경을 위한 빅데이터 기반 센서 데이터 처리 및 분석)

  • Shin, Dong-Jin;Park, Ji-Hun;Kim, Ju-Ho;Kwak, Kwang-Jin;Park, Jeong-Min;Kim, Jeong-Joon
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.19 no.1
    • /
    • pp.117-126
    • /
    • 2019
  • The data generated in the IoT environment is very diverse. Especially, the development of the fourth industrial revolution has made it possible to increase the number of fixed and unstructured data generated in manufacturing facilities such as Smart Factory. With Big Data related solutions, it is possible to collect, store, process, analyze and visualize various large volumes of data quickly and accurately. Therefore, in this paper, we will directly generate data using Raspberry Pi used in IoT environment, and analyze using various Big Data solutions. Collected by using an Sqoop solution collected and stored in the database to the HDFS, and the process is to process the data by using the solutions available Hive parallel processing is associated with Hadoop. Finally, the analysis and visualization of the processed data via the R programming will be used universally to end verification.

Efficient Processing of Grouped Aggregation on Non-Uniformed Memory Access Architecture (비균등 메모리 접근 구조에서의 효율적인 그룹화 집단 연산의 처리)

  • Choe, Seongjun;Min, Jun-Ki
    • Database Research
    • /
    • v.34 no.3
    • /
    • pp.14-27
    • /
    • 2018
  • Recently, to alleviate the memory bottleneck problme occurred in Symmetric Multiprocessing (SMP) architecture, Non-Uniform Memory Access (NUMA) architecture was proposed. In addition, since an aggregation operator is an important operator providing properties and summary of data, the efficiency of the aggregation operator is crucial to overall performance of a system. Thus, in this paper, we propose an efficient aggregation processing technique on NUMA architecture. Our proposed technique consists of partition phase and merge phase. In the partition phase, the target relation is partitioned into several partial relations according to grouping attribute. Thus, since each thread can process aggregation operator on partial relation independently, we prevent the remote memory access during the merge phase. Furthermore, at the merge phase, we improve the performance of the aggregation processing by letting each thread compute aggregation with a local hash table as well as avoiding lock contention to merge aggregation results generated by all threads into one.

Efficient Self-supervised Learning Techniques for Lightweight Depth Completion (경량 깊이완성기술을 위한 효율적인 자기지도학습 기법 연구)

  • Park, Jae-Hyuck;Min, Kyoung-Wook;Choi, Jeong Dan
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.20 no.6
    • /
    • pp.313-330
    • /
    • 2021
  • In an autonomous driving system equipped with a camera and lidar, depth completion techniques enable dense depth estimation. In particular, using self-supervised learning it is possible to train the depth completion network even without ground truth. In actual autonomous driving, such depth completion should have very short latency as it is the input of other algorithms. So, rather than complicate the network structure to increase the accuracy like previous studies, this paper focuses on network latency. We design a U-Net type network with RegNet encoders optimized for GPU computation. Instead, this paper presents several techniques that can increase accuracy during the process of self-supervised learning. The proposed techniques increase the robustness to unreliable lidar inputs. Also, they improve the depth quality for edge and sky regions based on the semantic information extracted in advance. Our experiments confirm that our model is very lightweight (2.42 ms at 1280x480) but resistant to noise and has qualities close to the latest studies.

Interface Conversion to Extend Communication Cable of Ultrasonic Sensor (초음파 센서 통신선 연장을 위한 인터페이스 변환)

  • Seo, Dae-Il;Kwon, Byung-Hyuk;Kim, Sang-Jin
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.17 no.3
    • /
    • pp.467-472
    • /
    • 2022
  • The 3D ultrasonic anemometer transmits measured data by connecting PC and RS232C interface. Depending on the observation location, it is often necessary to extend the cable connecting the PC and the sensor. When installing on the test bed of the Air Meteorological Agency, the original AWM2919 cable was required to be extended because the distance between the PC container and the equipment installation site was more than 30 m. The cable was extended through a process such as extending the AWM2919 cable, converting the interface with the PC from RS232C to RS485, and testing the RS485 communication. After the equipment was installed with an extended cable, data were remotely collected and analyzed to confirm successful cable extension.

Development of long-term daily high-resolution gridded meteorological data based on deep learning (딥러닝에 기반한 우리나라 장기간 일 단위 고해상도 격자형 기상자료 생산)

  • Yookyung Jeong;Kyuhyun Byu
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2023.05a
    • /
    • pp.198-198
    • /
    • 2023
  • 유역 내 수자원 계획을 효율적으로 수립하기 위해서는 장기간에 걸친 수문 모델링 뿐만 아니라 미래 기후 시나리오에 따른 수문학적 기후변화 영향 분석도 중요하다. 이를 위해서는 관측 값에 기반한 고품질 및 고해상도 격자형 기상자료 생산이 필수적이다. 하지만, 우리나라는 종관기상관측시스템(ASOS)과 방재기상관측시스템(AWS)으로 이루어진 고밀도 관측 네트워크가 2000년 이후부터 이용 가능했기에 장기간 격자형 기상자료가 부족하다. 이를 보완하고자 본 연구는 가정적인 상황에 기반하여 만약 2000년 이전에도 현재와 동일한 고밀도 관측 네트워크가 존재했다면 산출 가능했을 장기간 일 단위 고해상도 격자형 기상자료를 생산하는 것을 목표로 한다. 구체적으로, 2000년을 기준으로 최근과 과거 기간의 격자형 기상자료를 딥러닝 알고리즘으로 모델링하여 과거 기간을 대상으로 기상자료(일 단위 기온, 강수량)의 공간적 변동성 및 특성을 재구성한다. 격자형 기상자료의 생산을 위해 우리나라의 고도에 기반하여 기상 인자들의 영향을 정량화 하는 보간법인 K-PRISM을 적용하여 고밀도 및 저밀도 관측 네트워크로 두 가지 격자형 기상자료를 생산한다. 생산한 격자형 기상자료 중 저밀도 관측 네트워크의 자료를 입력 자료로, 고밀도 관측 네트워크의 자료를 출력 자료로 선정하여 각 격자점에 대해 Long-Short Term Memory(LSTM) 알고리즘을 개발한다. 이 때, 멀티 그래픽 처리장치(GPU)에 기반한 병렬 처리를 통해 비용 효율적인 계산이 가능하도록 한다. 최종적으로 1973년부터 1999년까지의 저밀도 관측 네트워크의 격자형 기상자료를 입력 자료로 하여 해당 기간에 대한 고밀도 관측 네트워크의 격자형 기상자료를 생산한다. 개발된 대부분의 예측 모델 결과가 0.9 이상의 NSE 값을 나타낸다. 따라서, 본 연구에서 개발된 모델은 고품질의 장기간 기상자료를 효율적으로 정확도 높게 산출하며, 이는 향후 장기간 기후 추세 및 변동 분석에 중요 자료로 활용 가능하다.

  • PDF

Developing a Korean sentiment lexicon through BPE (BPE를 활용한 한국어 감정사전 제작)

  • Park, Ho-Min;Cheon, Min-Ah;Nam-Goong, Young;Choi, Min-Seok;Yoon, Ho;Kim, Jae-Kyun;Kim, Jae-Hoon
    • Annual Conference on Human and Language Technology
    • /
    • 2019.10a
    • /
    • pp.510-513
    • /
    • 2019
  • 감정분석은 텍스트에서 나타난 저자 혹은 발화자의 태도, 의견 등과 같은 주관적인 정보를 추출하는 기술이며, 여론 분석, 시장 동향 분석 등 다양한 분야에 두루 사용된다. 감정분석 방법은 사전 기반 방법, 기계학습 기반 방법 등이 있다. 본 논문은 사전 기반 감정분석에 필요한 한국어 감정사전 자동 구축 방법을 제안한다. 본 논문은 영어 감정사전으로부터 한국어 감정사전을 자동으로 구축하는 방법이며, 크게 세 단계로 구성된다. 첫 번째는 한영 병렬 말뭉치를 이용한 한영 이중언어 사전을 구축하는 단계이고, 두 번째는 한영 이중언어 사전을 통한 한영 이중언어 그래프를 생성하는 단계이며, 세 번째는 영어 단어의 감정값을 한국어 BPE의 감정값으로 전파하는 단계이다. 본 논문에서는 제안된 방법의 유효성을 보이기 위해 사전 기반 한국어 감정분석 시스템을 구축하여 평가하였으며, 그 결과 제안된 방법이 합리적인 방법임을 확인할 수 있었으며 향후 연구를 통해 개선한다면 질 좋은 한국어 감정사전을 효과적인 방법으로 구축할 수 있을 것이다.

  • PDF

Distributed AI Learning-based Proof-of-Work Consensus Algorithm (분산 인공지능 학습 기반 작업증명 합의알고리즘)

  • Won-Boo Chae;Jong-Sou Park
    • The Journal of Bigdata
    • /
    • v.7 no.1
    • /
    • pp.1-14
    • /
    • 2022
  • The proof-of-work consensus algorithm used by most blockchains is causing a massive waste of computing resources in the form of mining. A useful proof-of-work consensus algorithm has been studied to reduce the waste of computing resources in proof-of-work, but there are still resource waste and mining centralization problems when creating blocks. In this paper, the problem of resource waste in block generation was solved by replacing the relatively inefficient computation process for block generation with distributed artificial intelligence model learning. In addition, by providing fair rewards to nodes participating in the learning process, nodes with weak computing power were motivated to participate, and performance similar to the existing centralized AI learning method was maintained. To show the validity of the proposed methodology, we implemented a blockchain network capable of distributed AI learning and experimented with reward distribution through resource verification, and compared the results of the existing centralized learning method and the blockchain distributed AI learning method. In addition, as a future study, the thesis was concluded by suggesting problems and development directions that may occur when expanding the blockchain main network and artificial intelligence model.

Design of a High-Speed Data Packet Allocation Circuit for Network-on-Chip (NoC 용 고속 데이터 패킷 할당 회로 설계)

  • Kim, Jeonghyun;Lee, Jaesung
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2022.10a
    • /
    • pp.459-461
    • /
    • 2022
  • One of the big differences between Network-on-Chip (NoC) and the existing parallel processing system based on an off-chip network is that data packet routing is performed using a centralized control scheme. In such an environment, the best-effort packet routing problem becomes a real-time assignment problem in which data packet arriving time and processing time is the cost. In this paper, the Hungarian algorithm, a representative computational complexity reduction algorithm for the linear algebraic equation of the allocation problem, is implemented in the form of a hardware accelerator. As a result of logic synthesis using the TSMC 0.18um standard cell library, the area of the circuit designed through case analysis for the cost distribution is reduced by about 16% and the propagation delay of it is reduced by about 52%, compared to the circuit implementing the original operation sequence of the Hungarian algorithm.

  • PDF

The preparation of surface-modified granular activated carbon (GAC) to enhance Perfluorooctanoic acid (PFOA) removal and evaluation of adsorption behavior (입상 활성탄 표면 개질을 통한 과불화옥탄산 (PFOA) 제거 향상 및 특성 평가)

  • Jeongwoo Shin;Byungryul An
    • Journal of Korean Society of Water and Wastewater
    • /
    • v.37 no.4
    • /
    • pp.177-186
    • /
    • 2023
  • Perfluorooctanoic acid(PFOA) was one of widely used per- and poly substances(PFAS) in the industrial field and its concentration in the surface and groundwater was found with relatively high concentration compared to other PFAS. Since various processes have been introduced to remove the PFOA, adsorption using GAC is well known as a useful and effective process in water and wastewater treatment. Surface modification for GAC was carried out using Cu and Fe to enhance the adsorption capacity and four different adsorbents, such as GAC-Cu, GAC-Fe, GAC-Cu(OH)2, GAC-Fe(OH)3 were prepared and compared with GAC. According to SEM-EDS, the increase of Cu or Fe was confirmed after surface modification and higher weight was observed for Cu and Fe hydroxide(GAC-Cu(OH)2 and GAC-Fe(OH)3, respectively). BET analysis showed that the surface modification reduced specific surface area and total pore volumes. The highest removal efficiency(71.4%) was obtained in GAC-Cu which is improved by 17.9% whereas the use of Fe showed lower removal efficiency compared to GAC. PFOA removal was decreased with increase of solution pH indicating electrostatic interaction governs at low pH and its effect was decreased when the point of zero charges(pzc) was negatively increased with an increase of pH. The enhanced removal of PFOA was clearly observed in solution pH 7, confirming the Cu in the surface of GAC plays a role on the PFOA adsorption. The maximum uptake was calculated as 257 and 345 ㎍/g for GAC and GAC-Cu using Langmuir isotherm. 40% and 80% of removal were accomplished within 1 h and 48 h. According to R2, only the linear pseudo-second-order(pso) kinetic model showed 0.98 whereas the others obtained less than 0.870.

Development of Industrial Embedded System Platform (산업용 임베디드 시스템 플랫폼 개발)

  • Kim, Dae-Nam;Kim, Kyo-Sun
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.47 no.5
    • /
    • pp.50-60
    • /
    • 2010
  • For the last half a century, the personal computer and software industries have been prosperous due to the incessant evolution of computer systems. In the 21st century, the embedded system market has greatly increased as the market shifted to the mobile gadget field. While a lot of multimedia gadgets such as mobile phone, navigation system, PMP, etc. are pouring into the market, most industrial control systems still rely on 8-bit micro-controllers and simple application software techniques. Unfortunately, the technological barrier which requires additional investment and higher quality manpower to overcome, and the business risks which come from the uncertainty of the market growth and the competitiveness of the resulting products have prevented the companies in the industry from taking advantage of such fancy technologies. However, high performance, low-power and low-cost hardware and software platforms will enable their high-technology products to be developed and recognized by potential clients in the future. This paper presents such a platform for industrial embedded systems. The platform was designed based on Telechips TCC8300 multimedia processor which embedded a variety of parallel hardware for the implementation of multimedia functions. And open-source Embedded Linux, TinyX and GTK+ are used for implementation of GUI to minimize technology costs. In order to estimate the expected performance and power consumption, the performance improvement and the power consumption due to each of enabled hardware sub-systems including YUV2RGB frame converter are measured. An analytic model was devised to check the feasibility of a new application and trade off its performance and power consumption. The validity of the model has been confirmed by implementing a real target system. The cost can be further mitigated by using the hardware parts which are being used for mass production products mostly in the cell-phone market.