• Title/Summary/Keyword: computing infrastructure

Search Result 501, Processing Time 0.023 seconds

An Analytic solution for the Hadoop Configuration Combinatorial Puzzle based on General Factorial Design

  • Priya, R. Sathia;Prakash, A. John;Uthariaraj, V. Rhymend
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.11
    • /
    • pp.3619-3637
    • /
    • 2022
  • Big data analytics offers endless opportunities for operational enhancement by extracting valuable insights from complex voluminous data. Hadoop is a comprehensive technological suite which offers solutions for the large scale storage and computing needs of Big data. The performance of Hadoop is closely tied with its configuration settings which depends on the cluster capacity and the application profile. Since Hadoop has over 190 configuration parameters, tuning them to gain optimal application performance is a daunting challenge. Our approach is to extract a subset of impactful parameters from which the performance enhancing sub-optimal configuration is then narrowed down. This paper presents a statistical model to analyze the significance of the effect of Hadoop parameters on a variety of performance metrics. Our model decomposes the total observed performance variation and ascribes them to the main parameters, their interaction effects and noise factors. The method clearly segregates impactful parameters from the rest. The configuration setting determined by our methodology has reduced the Job completion time by 22%, resource utilization in terms of memory and CPU by 15% and 12% respectively, the number of killed Maps by 50% and Disk spillage by 23%. The proposed technique can be leveraged to ease the configuration tuning task of any Hadoop cluster despite the differences in the underlying infrastructure and the application running on it.

A Framework for Provisioning Internet of Things Context-aware Services (사물인터넷 기반의 상황인지 서비스를 위한 프레임워크 설계)

  • Cheun, Du Wan
    • Journal of Service Research and Studies
    • /
    • v.2 no.2
    • /
    • pp.91-98
    • /
    • 2012
  • The emergence of new types of embedded computing devices and developments in wireless networking are broadening the domain of computing from the work place and home office to other facets of everyday life. This trend is expected to lead to a proliferation of Internet of Things (IoT) environments, in which inexpensive and interconnected computing devices are capable of supporting users in a range of tasks. Context-aware computing is a key source to develop such smart services. However, there are many challenges to enable services to be smart; Heterogeneous Computing Environments, Resource Limitations, Large Amount of Data Produced, and Different Requirements for Context Interpretation. Because of these challenges, there are difficulties in providing smart service by utilizing IoT devices. Currently, many researchers are conducting researches on mobile-computing based smart service development and provisioning and network infrastructure for interconnected IoT devices. Still, there are some limitations on developing core technologies for IoT computing based smart service development. In order to remedy this situation, this thesis presents a reusable framework that provides unique and noble features which are required in developing advanced context-aware IoT applications.

  • PDF

K-Defense Cloud Computing System Design through Cloud Modeling and Analysis of Social Network Service Application (소셜 네트워크 서비스 어플리케이션의 클라우드 모델링 및 분석을 통한 국방 클라우드 컴퓨탱 시스템 설계)

  • Lee, Sung-Tae;Ryou, Hwang-Bin
    • Convergence Security Journal
    • /
    • v.13 no.1
    • /
    • pp.37-43
    • /
    • 2013
  • In 2010, the Ministry of National Defense decided to build a MegaCenter including the cloud computing technology by 2014, as part of the '2012 Information Service Plan', which is now underway. The Cloud computing system environment should be designed applying cloud computing technology and policy for an efficient infrastructure that many IT resources are available in the data center as a concentrated form. That is, the system should be designed in such a way that clouding services will be efficiently provided to meet the needs of users and there will not be unnecessary waste of resources. However, in order to build an optimal system, it should be possible to predict the service performance and the resource availability at the initial phase of system design. In this paper, using the CloudAnalyst simulator to predict availability of the K-defence cloud computing system service, conducts cloud modeling and analysis of the 'Facebook', one of the most famous social network service applications with most users in the world. An Optimal K-Defense cloud computing design model is proposed through simulation results.

A Proposal of the Usage Metering Functions on Cloud Computing-Based Building Information Modeling (BIM) and the Law for the Open BIM Ecosystem (열린 BIM 생태계 조성을 위한 클라우드 컴퓨팅 기반 BIM 서비스 환경의 사용량 측정 기술 및 법 규정 제안)

  • Kim, Byungkon;Kim, Jongsung
    • Journal of KIBIM
    • /
    • v.6 no.3
    • /
    • pp.49-56
    • /
    • 2016
  • As project opportunities for the Architecture, Engineering and Construction (AEC) industry have grown more complex and larger, the utilization of Building Information Modeling (BIM) technologies for three-dimensional (3D) design and simulation practices has been increasing significantly; the typical applications of the BIM technologies include clash detection and design alternative based on 3D planning, which have been expanded over to the technology of construction management in the AEC industry for virtual design and construction. As for now, commercial BIM software has been operated under a single-user environment, which is why initial costs for its introduction are very high. Cloud computing, one of the most promising next-generation Internet technologies, enables simple Internet devices to use services and resources provided with BIM software. Recently in Korea, studies to link between BIM and cloud computing technologies have been directed toward saving costs to build BIM-related infrastructure, and providing various BIM services for small- and medium-sized enterprises (SMEs). This study addressed development of the usage metering functions of BIM software under cloud computing architecture in order to archive and use BIM data and create an optimal revenue structure so that the BIM services may grow spontaneously, considering a demand for cloud resources. For the reason, we surveyed relevant cases, and then analyzed needs and requirements from AEC industry. Based on the relevant cases, customizing for cloud BIM and design for the development was performed. We also surveyed any related-law to support cloud computing-based BIM service. Finally, we proposed herein how to optimally design and develop the usage metering functions of cloud BIM software.

A Study on Embody e-Government at Home with Home Network Infrastructure (홈네트워크기반의 가정 내 전자정부 구현을 위한 연구)

  • Shin, Young Jin
    • Informatization Policy
    • /
    • v.17 no.2
    • /
    • pp.45-67
    • /
    • 2010
  • This study is to reconstruct the user-oriented e-government service based on home network infrastructure using ubiquitous computing technology. Until now, 31 e-government roadmap tasks have been promoted as main projects of e-government. This study explores what should be respective priorities for the suppliers and users of e-government if the services are provided in linkage with home network infrastructure. The result of survey showed us that the efficiency of administration and the improvement of the civil service, including citizens' online participation should be priorities. In order to reinforce home network infrastructure in implementing u-government, the government, at first, has to relax regulation, support businesses to develop related technologies, and construct safe infrastructure for citizens to use e-government services. Second, businesses should develop home network technologies that include various contents and construct home network systems that citizens can use easily. Third, citizens should have interests and participate in the government policies for home network services. Particularly, this study aims to suggest policies that consider positions of both the suppliers and users of e-government so that home network-based e-government can be implemented in the future.

  • PDF

IaC-VIMF: IaC-Based Virtual Infrastructure Mutagenesis Framework for Cyber Defense Training (IaC-VIMF: 사이버 공방훈련을 위한 IaC 기반 가상 인프라 변이 생성 프레임워크)

  • Joo-Young Roh;Se-Han Lee;Ki-Woong Park
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.33 no.3
    • /
    • pp.527-535
    • /
    • 2023
  • To develop experts capable of responding to cyber security incidents, numerous institutions have established cyber training facilities to cultivate security professionals equipped with effective defense strategies. However, these challenges such as limited resources, scenario-based content development, and cost constraints. To address these issues, this paper proposes a virtual infrastructure variation generation framework. It provides customized, diverse IT infrastructure environments for each organization, allowing cyber defense trainers to accumulate a wide range of experiences. By leveraging Infrastructure-as-Code (IaC) containers and employing Word2Vec, a natural language processing model, mutable code elements are extracted and trained, enabling the generation of new code and presenting novel container environments.

Changing information environment and academic libraries (정보환경의 변화와 대학도서관의 발전방향)

  • 강숙희
    • Journal of Korean Library and Information Science Society
    • /
    • v.24
    • /
    • pp.189-219
    • /
    • 1996
  • Information technology will inevitably and profoundly alter every aspect of university life. New information technologies are creating new patterns in the research and teaching, with networks enhancing information sharing and data transmission. The revolution in information technology requires that universities integrate information technology with their traditional missions of teaching, research, and public service. Enhanced mode of information access and changing information-seeking behaviors suggest the need for new library services. New technologies have opened up many new services options for libraries. The nature of and potential for new information services suggest that libraries should become better connected with other campus units. Some of these issues will no doubt be addressed by the a n.0, pplication of new technologies and by building campus information infrastructure. An important element of the needed change for enhancing the library information services is the integration of the libraries and computing centers. Libraries and computing centers deal with the same commodity-information and delivering information. For the sake of scholarship and research, the two must devise an integrated a n.0, pproach to delivering the common commodity. Computing centers have special skills in system development and telecommunications. Libraries have good human interface skills and skills for organizing knowledge. Together they could be a particularly productive team. Noting this, some universities have created new information service units which unify the present-day computing, library and telecommunication functions. Existing models range from true merging to situations where some cooperative activities are being conducted. The purpose of this study is to suggest the direction in developing of information services in academic libraries by examining the changing information environment and trends of integrating libraries and computing centers. This paper inquires into the changing information environment, the impact of imformation technology on universities and libraries, some casses of integration of academic libraries and computing centers, and effects of integration of two and suggests the developing direction of information services in academic libraries.

  • PDF

IT Systems Risk Analysis Using Object Oriented Asset Classification Model (객체지향 자산분류모델을 이용한 위험분석에 관한연구)

  • Lee, Hyeak-Ro;Ahn, Seong-Jin
    • Journal of Internet Computing and Services
    • /
    • v.9 no.4
    • /
    • pp.79-84
    • /
    • 2008
  • In these days, many organizations try to manage their assets in safe way due to fast change in information-communication environment. In Korea, risk analysis and vulnerability analysis for security improvement of critical asset is booming by enforcement of Act on security of information and communication infrastructure. It is obligate that each critical information infrastructure needs to get vulnerability analysis. In this paper, we proposed Object Oriented Asset Classification model for asset analysis and risk analysis.

  • PDF

Basic Prototype Design and Verification of Hadoop Cluster based on Private Cloud Infrastructure for SMB (중소기업을 위한 프라이빗 클라우드 인프라 기반 하둡 클러스터의 기본 프로토타입 설계 및 실증)

  • Cha, Byung-Rae;Kim, Hyeong-Gyun;Kim, Dae-Gue;Kim, Jong-Won;Kim, Yong-Il
    • Journal of Advanced Navigation Technology
    • /
    • v.17 no.2
    • /
    • pp.225-233
    • /
    • 2013
  • Recently, Cloud Computing and Big Data has become a buzzword in the field of IT. In this paper, as part of special efforts to support small businesses (SMB) in these situations, we designed the basic prototypes ver. 0.1, 0.2, and 0.5 for Hadoop cluster based on private cloud infrastructure and implemented the part of basic prototypes. And we verified the performances of the basic prototypes using ASA Dataset.

A NoSQL data management infrastructure for bridge monitoring

  • Jeong, Seongwoon;Zhang, Yilan;O'Connor, Sean;Lynch, Jerome P.;Sohn, Hoon;Law, Kincho H.
    • Smart Structures and Systems
    • /
    • v.17 no.4
    • /
    • pp.669-690
    • /
    • 2016
  • Advances in sensor technologies have led to the instrumentation of sensor networks for bridge monitoring and management. For a dense sensor network, enormous amount of sensor data are collected. The data need to be managed, processed, and interpreted. Data management issues are of prime importance for a bridge management system. This paper describes a data management infrastructure for bridge monitoring applications. Specifically, NoSQL database systems such as MongoDB and Apache Cassandra are employed to handle time-series data as well the unstructured bridge information model data. Standard XML-based modeling languages such as OpenBrIM and SensorML are adopted to manage semantically meaningful data and to support interoperability. Data interoperability and integration among different components of a bridge monitoring system that includes on-site computers, a central server, local computing platforms, and mobile devices are illustrated. The data management framework is demonstrated using the data collected from the wireless sensor network installed on the Telegraph Road Bridge, Monroe, MI.