• 제목/요약/키워드: Computer data processing

검색결과 4,319건 처리시간 0.033초

Scalable Big Data Pipeline for Video Stream Analytics Over Commodity Hardware

  • Ayub, Umer;Ahsan, Syed M.;Qureshi, Shavez M.
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제16권4호
    • /
    • pp.1146-1165
    • /
    • 2022
  • A huge amount of data in the form of videos and images is being produced owning to advancements in sensor technology. Use of low performance commodity hardware coupled with resource heavy image processing and analyzing approaches to infer and extract actionable insights from this data poses a bottleneck for timely decision making. Current approach of GPU assisted and cloud-based architecture video analysis techniques give significant performance gain, but its usage is constrained by financial considerations and extremely complex architecture level details. In this paper we propose a data pipeline system that uses open-source tools such as Apache Spark, Kafka and OpenCV running over commodity hardware for video stream processing and image processing in a distributed environment. Experimental results show that our proposed approach eliminates the need of GPU based hardware and cloud computing infrastructure to achieve efficient video steam processing for face detection with increased throughput, scalability and better performance.

Computer Security에 관한 소고 - 사고범죄예방을 중심으로 - (A Study on Computer Security and Controls)

  • 이종철
    • 산업경영시스템학회지
    • /
    • 제4권4호
    • /
    • pp.25-34
    • /
    • 1981
  • Recently there has been a marked increase in concern for security in computerized operations. The purposes of computer security controls are to protect against the unauthorized access to and modification of data processing resources, unauthorised access to and modification of data files and software, and the misuse of authorized activities. The controls relate to the physical security of the data processing department and of the areas within the data processing department : to the security of the data files, programs, and system software : and to the human interaction with the data files, programs, and system software. The controls that will be discussed in this paper include : I. Risk on the computer use. II. Methods of risk counter measure. III. Role of system auditing.

  • PDF

SAR 디스플레이 영상을 위한 무손실 압축 (LOSSLESS DATA COMPRESSION ON SAR DISPLAY IMAGES)

  • Lee, Tae-hee;Song, Woo-jin;Do, Dae-won;Kwon, Jun-chan;Yoon, Byung-woo
    • 대한전자공학회:학술대회논문집
    • /
    • 대한전자공학회 2001년도 제14회 신호처리 합동 학술대회 논문집
    • /
    • pp.117-120
    • /
    • 2001
  • Synthetic aperture radar (SAR) is a promising active remote sensing technique to obtain large terrain information of the earth in all-weather conditions. SAR is useful in many applications, including terrain mapping and geographic information system (GIS), which use SAR display images. Usually, these applications need the enormous data storage because they deal with wide terrain images with high resolution. So, compression technique is a useful approach to deal with SAR display images with limited storage. Because there is some indispensable data loss through the conversion of a complex SAR image to a display image, some applications, which need high-resolution images, cannot tolerate more data loss during compression. Therefore, lossless compression is appropriate to these applications. In this paper, we propose a novel lossless compression technique for a SAR display image using one-step predictor and block arithmetic coding.

  • PDF

Design an Indexing Structure System Based on Apache Hadoop in Wireless Sensor Network

  • Keo, Kongkea;Chung, Yeongjee
    • 한국정보처리학회:학술대회논문집
    • /
    • 한국정보처리학회 2013년도 춘계학술발표대회
    • /
    • pp.45-48
    • /
    • 2013
  • In this paper, we proposed an Indexing Structure System (ISS) based on Apache Hadoop in Wireless Sensor Network (WSN). Nowadays sensors data continuously keep growing that need to control. Data constantly update in order to provide the newest information to users. While data keep growing, data retrieving and storing are face some challenges. So by using the ISS, we can maximize processing quality and minimize data retrieving time. In order to design ISS, Indexing Types have to be defined depend on each sensor type. After identifying, each sensor goes through the Indexing Structure Processing (ISP) in order to be indexed. After ISP, indexed data are streaming and storing in Hadoop Distributed File System (HDFS) across a number of separate machines. Indexed data are split and run by MapReduce tasks. Data are sorted and grouped depend on sensor data object categories. Thus, while users send the requests, all the queries will be filter from sensor data object and managing the task by MapReduce processing framework.

데이터스페이스의 질의 처리를 위한 향상된 인덱싱 기법 (An Improved Indexing Method for Query Processing of Dataspaces)

  • 황욱광;이동욱;신숭선;백성하;배해영
    • 한국정보처리학회:학술대회논문집
    • /
    • 한국정보처리학회 2009년도 춘계학술발표대회
    • /
    • pp.317-320
    • /
    • 2009
  • Dataspaces are the collections of heterogeneous and partially unstructured data. It is difficult for the users to explore the data from varies data sources using a single schema. And the queries supposed should be allowed to specify varying degrees of structure, spanning keyword queries to more structure-aware queries. Utilizing give the model of heterogeneous data and the definitions of two mainly types of query on dataspaces, in this paper we propose an improved method which can suppose the flexibly query more efficiently.

A Study on Distributed System Construction and Numerical Calculation Using Raspberry Pi

  • Ko, Young-ho;Heo, Gyu-Seong;Lee, Sang-Hyun
    • International journal of advanced smart convergence
    • /
    • 제8권4호
    • /
    • pp.194-199
    • /
    • 2019
  • As the performance of the system increases, more parallelized data is being processed than single processing of data. Today's cpu structure has been developed to leverage multicore, and hence data processing methods are being developed to enable parallel processing. In recent years desktop cpu has increased multicore, data is growing exponentially, and there is also a growing need for data processing as artificial intelligence develops. This neural network of artificial intelligence consists of a matrix, making it advantageous for parallel processing. This paper aims to speed up the processing of the system by using raspberrypi to implement the cluster building and parallel processing system against the backdrop of the foregoing discussion. Raspberrypi is a credit card-sized single computer made by the raspberrypi Foundation in England, developed for education in schools and developing countries. It is cheap and easy to get the information you need because many people use it. Distributed processing systems should be supported by programs that connected multiple computers in parallel and operate on a built-in system. RaspberryPi is connected to switchhub, each connected raspberrypi communicates using the internal network, and internally implements parallel processing using the Message Passing Interface (MPI). Parallel processing programs can be programmed in python and can also use C or Fortran. The system was tested for parallel processing as a result of multiplying the two-dimensional arrangement of 10000 size by 0.1. Tests have shown a reduction in computational time and that parallelism can be reduced to the maximum number of cores in the system. The systems in this paper are manufactured on a Linux-based single computer and are thought to require testing on systems in different environments.

Spatial Clearinghouse Components for OpenGIS Data Providers

  • Oh, Byoung-Woo;Kim, Min-Soo;Lee, Jong-Hun
    • 대한원격탐사학회:학술대회논문집
    • /
    • 대한원격탐사학회 1999년도 Proceedings of International Symposium on Remote Sensing
    • /
    • pp.84-88
    • /
    • 1999
  • Recently, the necessity of accessing spatial data from remote computer via network has been increased as distributed spatial data have been increased due to their size and cost. Many methods have been used in recent years for transferring spatial data, such as socket, CORBA, HTTP, RPC, FTP, etc. In this paper, we propose spatial clearinghouse components to access distributed spatial data sources via CORBA and Internet. The spatial clearinghouse components are defined as OLE/COM components that enable users to access spatial data that meet their requests from remote computer. For reusability, we design the spatial clearinghouse with UML and implement it as a set of components. In order to enhance interoperability among different platforms in distributed computing environment, we adopt international standards and open architecture such as CORBA, HTTB, and OpenGIS Simple Features Specifications. There are two kinds of spatial clearinghouse: CORBA-based spatial clearinghouse and Internet-based spatial clearinghouse. The CORBA-based spatial clearinghouse supports COM-CORBA bridge to access spatial data from remote data providers that satisfy the OpenGIS Simple Features Specification for OLE/COM using COM and CORBA interfaces. The Internet-based spatial clearinghouse provides Web-service components to access spatial data from remote data providers using Web-browser.

  • PDF

장소인식멀티센서스마트 환경을위한 데이터 퓨전 모델 (Locality Aware Multi-Sensor Data Fusion Model for Smart Environments)

  • 와카스 나와즈;무하머디 파힘;이승룡;이영구
    • 한국정보처리학회:학술대회논문집
    • /
    • 한국정보처리학회 2011년도 춘계학술발표대회
    • /
    • pp.78-80
    • /
    • 2011
  • In the area of data fusion, dealing with heterogeneous data sources, numerous models have been proposed in last three decades to facilitate different application domains i.e. Department of Defense (DoD), monitoring of complex machinery, medical diagnosis and smart buildings. All of these models shared the theme of multiple levels processing to get more reliable and accurate information. In this paper, we consider five most widely acceptable fusion models (Intelligence Cycle, Joint Directors of Laboratories, Boyd control, Waterfall, Omnibus) applied to different areas for data fusion. When they are exposed to a real scenario, where large dataset from heterogeneous sources is utilize for object monitoring, then it may leads us to non-efficient and unreliable information for decision making. The proposed variation works better in terms of time and accuracy due to prior data diminution.

Big IoT Healthcare Data Analytics Framework Based on Fog and Cloud Computing

  • Alshammari, Hamoud;El-Ghany, Sameh Abd;Shehab, Abdulaziz
    • Journal of Information Processing Systems
    • /
    • 제16권6호
    • /
    • pp.1238-1249
    • /
    • 2020
  • Throughout the world, aging populations and doctor shortages have helped drive the increasing demand for smart healthcare systems. Recently, these systems have benefited from the evolution of the Internet of Things (IoT), big data, and machine learning. However, these advances result in the generation of large amounts of data, making healthcare data analysis a major issue. These data have a number of complex properties such as high-dimensionality, irregularity, and sparsity, which makes efficient processing difficult to implement. These challenges are met by big data analytics. In this paper, we propose an innovative analytic framework for big healthcare data that are collected either from IoT wearable devices or from archived patient medical images. The proposed method would efficiently address the data heterogeneity problem using middleware between heterogeneous data sources and MapReduce Hadoop clusters. Furthermore, the proposed framework enables the use of both fog computing and cloud platforms to handle the problems faced through online and offline data processing, data storage, and data classification. Additionally, it guarantees robust and secure knowledge of patient medical data.

A Review of Window Query Processing for Data Streams

  • Kim, Hyeon Gyu;Kim, Myoung Ho
    • Journal of Computing Science and Engineering
    • /
    • 제7권4호
    • /
    • pp.220-230
    • /
    • 2013
  • In recent years, progress in hardware technology has resulted in the possibility of monitoring many events in real time. The volume of incoming data may be so large, that monitoring all individual data might be intractable. Revisiting any particular record can also be impossible in this environment. Therefore, many database schemes, such as aggregation, join, frequent pattern mining, and indexing, become more challenging in this context. This paper surveys the previous efforts to resolve these issues in processing data streams. The emphasis is on specifying and processing sliding window queries, which are supported in many stream processing engines. We also review the related work on stream query processing, including synopsis structures, plan sharing, operator scheduling, load shedding, and disorder control.