• Journal of Advanced Research in Ocean Engineering
• /
• v.3 no.1
• /
• pp.32-40
• /
• 2017

Recently, the amount of data to be processed and the complexity thereof have been increasing due to the development of information and communication technology, and industry's interest in such big data is increasing day by day. In the shipbuilding and offshore industry also, there is growing interest in the effective utilization of data, since various and vast amounts of data are being generated in the process of design, production, and operation. In order to effectively utilize big data in the shipbuilding and offshore industry, it is necessary to store and process large amounts of data. In this study, it was considered efficient to apply Hadoop and R, which are mostly used in big data related research. Hadoop is a framework for storing and processing big data. It provides the Hadoop Distributed File System (HDFS) for storing big data, and the MapReduce function for processing. Meanwhile, R provides various data analysis techniques through the language and environment for statistical calculation and graphics. While Hadoop makes it is easy to handle big data, it is difficult to finely process data; and although R has advanced analysis capability, it is difficult to use to process large data. This study proposes a big data platform based on Hadoop for applications in the shipbuilding and offshore industry. The proposed platform includes the existing data of the shipyard, and makes it possible to manage and process the data. To check the applicability of the platform, it is applied to estimate the weights of offshore structure topsides. In this study, we store data of existing FPSOs in Hadoop-based Hortonworks Data Platform (HDP), and perform regression analysis using RHadoop. We evaluate the effectiveness of large data processing by RHadoop by comparing the results of regression analysis and the processing time, with the results of using the conventional weight estimation program.

• Journal of KIISE
• /
• v.44 no.4
• /
• pp.383-391
• /
• 2017

As we enter a new era of Big Data, the amount of semantic data has rapidly increased. In order to derive meaningful information from this large semantic data, studies that utilize the SWRL(Semantic Web Rule Language) are being actively conducted. SWRL rules are based on data extracted from a user's empirical knowledge. However, conventional reasoning systems developed on single machines cannot process large scale data. Similarly, multi-node based reasoning systems have performance degradation problems due to network shuffling. Therefore, this paper overcomes the limitations of existing systems and proposes more efficient distributed inference methods. It also introduces data partitioning strategies to minimize network shuffling. In addition, it describes a method for optimizing the incremental reasoning process through data selection and determining the rule order. In order to evaluate the proposed methods, the experiments were conducted using WiseKB consisting of 200 million triples with 83 user defined rules and the overall reasoning task was completed in 32.7 minutes. Also, the experiment results using LUBM bench datasets showed that our approach could perform reasoning twice as fast as MapReduce based reasoning systems.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.6 no.11
• /
• pp.2827-2848
• /
• 2012

Previously, we described a social media cloud computing service environment (SMCCSE). This SMCCSE supports the development of social networking services (SNSs) that include audio, image, and video formats. A social media cloud computing PaaS platform, a core component in a SMCCSE, processes large amounts of social media in a parallel and distributed manner for supporting a reliable SNS. Here, we propose a Hadoop-based multimedia system for image and video transcoding processing, necessary functions of our PaaS platform. Our system consists of two modules, including an image transcoding module and a video transcoding module. We also design and implement the system by using a MapReduce framework running on a Hadoop Distributed File System (HDFS) and the media processing libraries Xuggler and JAI. In this way, our system exponentially reduces the encoding time for transcoding large amounts of image and video files into specific formats depending on user-requested options (such as resolution, bit rate, and frame rate). In order to evaluate system performance, we measure the total image and video transcoding time for image and video data sets, respectively, under various experimental conditions. In addition, we compare the video transcoding performance of our cloud-based approach with that of the traditional frame-level parallel processing-based approach. Based on experiments performed on a 28-node cluster, the proposed Hadoop-based multimedia transcoding system delivers excellent speed and quality.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.18 no.12
• /
• pp.2885-2890
• /
• 2014

Localization of a sensor network node using machine learning has been recently studied. It is easy for Support vector machines algorithm to implement in high level language enabling parallelism. Raspberrypi is a linux system which can be used as a sensor node. Pi can be used to construct IP based Hadoop clusters. In this paper, we realized Support vector machine using python language and built a sensor network cluster with 5 Pi's. We also established a Hadoop software framework to employ MapReduce mechanism. In our experiment, we implemented the test sensor network with a variety of parameters and examined based on proficiency, resource evaluation, and processing time. The experimentation showed that with more execution power and memory volume, Pi could be appropriate for a member node of the cluster, accomplishing precise classification for sensor localization using machine learning.

• Investigative Magnetic Resonance Imaging
• /
• v.25 no.3
• /
• pp.141-155
• /
• 2021

Purpose: To develop a 3D magnetic resonance fingerprinting (MRF) method for application in high resolution knee cartilage PD, T₁, T₂ mapping. Materials and Methods: A novel 3D acquisition trajectory with golden-angle rotating radial in k_xy direction and interleaved echo planar imaging (EPI) acquisition in the k_z direction was implemented in the MRF framework. A centric order was applied to the interleaved EPI acquisition to reduce Nyquist ghosting artifact due to field inhomogeneity. For the reconstruction, singular value decomposition (SVD) compression method was used to accelerate reconstruction time and conjugate gradient sensitivity-encoding (CG-SENSE) was performed to overcome low SNR of the high resolution data. Phantom experiments were performed to verify the proposed method. In vivo experiments were performed on 6 healthy volunteers and 2 early osteoarthritis (OA) patients. Results: In the phantom experiments, the T₁ and T₂ values of the proposed method were in good agreement with the spin-echo references. The results from the in vivo scans showed high quality proton density (PD), T₁, T₂ map with EPI echo train length (N_ETL = 4), acceleration factor in through plane (R_z = 5), and number of radial spokes (N_spk = 4). In patients, high T₂ values (50-60 ms) were seen in all transverse, sagittal, and coronal views and the damaged cartilage regions were in agreement with the hyper-intensity regions shown on conventional turbo spin-echo (TSE) images. Conclusion: The proposed 3D MRF method can acquire high resolution (0.5 mm³) quantitative maps in practical scan time (~ 7 min and 10 sec) with full coverage of the knee (FOV: 160 × 160 × 120 mm³).

• Journal of the Korean Institute of Intelligent Systems
• /
• v.23 no.5
• /
• pp.392-399
• /
• 2013

Due to the development of IT technology and the information age, a dependency of the network over the most of our lives have grown to a greater extent. Although it provides us to get various useful information and service, it also has negative effectiveness that can provide network intruder with vulnerable roots. In other words, we need to urgently cope with theses serious security problem causing service disableness or system connected to network obstacle with exploiting various packet information. Many experts in a field of security are making an effort to develop the various security solutions to respond against these threats, but existing solutions have a lot of problems such as lack of storage capacity and performance degradation along with the massive increase of packet data volume. Therefore we propose the packet analysis model to apply issuing Big Data technology in the field of security. That is, we used NoSQL which is technology of massive data storage to collect the packet data growing massive and implemented the packet analysis model based on K-means clustering using MapReudce which is distributed programming framework, and then we have shown its high performance by experimenting.

• Journal of KIISE
• /
• v.41 no.10
• /
• pp.762-773
• /
• 2014

As the web of data is increasingly producing large RDFS datasets, it becomes essential in building scalable reasoning engines over large triples. There have been many researches used expensive distributed framework, such as Hadoop, to reason over large RDFS triples. However, in many cases we are required to handle millions of triples. In such cases, it is not necessary to deploy expensive distributed systems because logic program based reasoners in a single machine can produce similar reasoning performances with that of distributed reasoner using Hadoop. In this paper, we propose a scalable RDFS reasoner using logical programming methods in a single machine and compare our empirical results with that of distributed systems. We show that our logic programming based reasoner using a single machine performs as similar as expensive distributed reasoner does up to 200 million RDFS triples. In addition, we designed a meta data structure by decomposing the ontology triples into separate sectors. Instead of loading all the triples into a single model, we selected an appropriate subset of the triples for each ontology reasoning rule. Unification makes it easy to handle conjunctive queries for RDFS schema reasoning, therefore, we have designed and implemented RDFS axioms using logic programming unifications and efficient conjunctive query handling mechanisms. The throughputs of our approach reached to 166K Triples/sec over LUBM1500 with 200 million triples. It is comparable to that of WebPIE, distributed reasoner using Hadoop and Map Reduce, which performs 185K Triples/sec. We show that it is unnecessary to use the distributed system up to 200 million triples and the performance of logic programming based reasoner in a single machine becomes comparable with that of expensive distributed reasoner which employs Hadoop framework.

• Journal of KIISE
• /
• v.42 no.3
• /
• pp.307-319
• /
• 2015

Current ontology studies use the Hadoop distributed storage framework to perform map-reduce algorithm-based reasoning for scalable ontologies. In this paper, however, we propose a novel approach for scalable Web Ontology Language (OWL) Horst Lite ontology reasoning, based on distributed cluster memories. Rule-based reasoning, which is frequently used for scalable ontologies, iteratively executes triple-format ontology rules, until the inferred data no longer exists. Therefore, when the scalable ontology reasoning is performed on computer hard drives, the ontology reasoner suffers from performance limitations. In order to overcome this drawback, we propose an approach that loads the ontologies into distributed cluster memories, using Spark (a memory-based distributed computing framework), which executes the ontology reasoning. In order to implement an appropriate OWL Horst Lite ontology reasoning system on Spark, our method divides the scalable ontologies into blocks, loads each block into the cluster nodes, and subsequently handles the data in the distributed memories. We used the Lehigh University Benchmark, which is used to evaluate ontology inference and search speed, to experimentally evaluate the methods suggested in this paper, which we applied to LUBM8000 (1.1 billion triples, 155 gigabytes). When compared with WebPIE, a representative mapreduce algorithm-based scalable ontology reasoner, the proposed approach showed a throughput improvement of 320% (62k/s) over WebPIE (19k/s).

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.33 no.2
• /
• pp.111-122
• /
• 2015

This study purposed on introducing a new automated solution for detecting railway tracks and reconstructing track models from the mobile laser scanning data. The proposed solution completes following procedures; the study initiated with detecting a potential railway region, called Region Of Interest (ROI), and approximating the orientation of railway track trajectory with the raw data. At next, the knowledge-based detection of railway tracks was performed for localizing track candidates in the first strip. In here, a strip -referring the local track search region- is generated in the orthogonal direction to the orientation of track trajectory. Lastly, an initial track model generated over the candidate points, which were detected by GMM-EM (Gaussian Mixture Model-Expectation & Maximization) -based clustering strip- wisely grows to capture all track points of interest and thus converted into geometric track model in the tracking by detection framework. Therefore, the proposed railway track tracking process includes following key features; it is able to reduce the complexity in detecting track points by using a hypothetical track model. Also, it enhances the efficiency of track modeling process by simultaneously capturing track points and modeling tracks that resulted in the minimization of data processing time and cost. The proposed method was developed using the C++ program language and was evaluated by the LiDAR data, which was acquired from MMS over an urban railway track area with a complex railway scene as well.

• Journal of the Korean Institute of Gas
• /
• v.16 no.1
• /
• pp.39-45
• /
• 2012

While the natural gas supply industry has continuously been growing, its potential hazard has also risen since the natural gas facilities essentially require installations that carry highly flammable and pressurized gas close to the populated areas, posing a serious consequence of significant property damage as well as human casualties in the event of accident. Therefore Quantitative Risk Assessment (QAR) has been recognized as a appropriate method to reduce the risk as far as possible, considering the reality of unachievable zero-risk. However, it is hard to perform effective QRA on hundreds of gas facilities because of insufficient number of expert and long-term analysis. In this paper, we suggest a conceptual QRA system framework to support more efficient risk analysis in gas supply facilities. In this system, the experts make questionnaires and internal calculation formula needed in accident frequency/consequence analysis of the facility through pre-analysis on the point of analysis, called incident point, and general users locate the point on the map and input the value required by the questionnaire to obtain the risk. Ultimately, this is suggested based on the idea that the specialization is available in QRA analysis process and the validity of the system is verified through actual system construction and application.