• 제목/요약/키워드: Traffic Big Data

Search Result 240, Processing Time 0.023 seconds

Study on Imputation Methods of Missing Real-Time Traffic Data (실시간 누락 교통자료의 대체기법에 관한 연구)

  • Jang Jin-hwan;Ryu Seung-ki;Moon Hak-yong;Byun Sang-cheal
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.3 no.1 s.4
    • /
    • pp.45-52
    • /
    • 2004
  • There are many cities installing ITS(Intelligent Transportation Systems) and running TMC(Trafnc Management Center) to improve mobility and safety of roadway transportation by providing roadway information to drivers. There are many devices in ITS which collect real-time traffic data. We can obtain many valuable traffic data from the devices. But it's impossible to avoid missing traffic data for many reasons such as roadway condition, adversary weather, communication shutdown and problems of the devices itself. We couldn't do any secondary process such as travel time forecasting and other transportation related research due to the missing data. If we use the traffic data to produce AADT and DHV, essential data in roadway planning and design, We might get skewed data that could make big loss. Therefore, He study have explored some imputation techniques such as heuristic methods, regression model, EM algorithm and time-series analysis for the missing traffic volume data using some evaluating indices such as MAPE, RMSE, and Inequality coefficient. We could get the best result from time-series model generating 5.0$\%$, 0.03 and 110 as MAPE, Inequality coefficient and RMSE, respectively. Other techniques produce a little different results, but the results were very encouraging.

  • PDF

Design of Data Pipeline for Linkage the Intelligent Maritime Transport Information System (지능형 해상교통정보시스템 연계를 위한 데이터파이프라인 설계)

  • Jong-Hwa Baek;Kwang-Hyun Lim;Deuk-Jae Cho
    • Proceedings of the Korean Institute of Navigation and Port Research Conference
    • /
    • 2022.06a
    • /
    • pp.315-316
    • /
    • 2022
  • In order to reduce maritime accidents and promote maritime safety and the happiness of the sea people, the Ministry of Oceans and Fisheries has been providing Intelligent Maritime Traffic Information services to the public from the end of January 2021. Various information is generated and collected through this service, and research and development is underway to develop and verify a service algorithm by applying the collected information to data science to realize a safer and more efficient intelligent maritime traffic information service. In order to develop and implement this, a data pipeline system that connects the collected and stored data and can access, use, and store data from multiple systems smoothly is required. Therefore, in this study, a data pipeline that can be used in various systems such as a datascience based service algorithm development environment and an intelligent maritime transportation service test-bed was designed.

  • PDF

De-cloaking Malicious Activities in Smartphones Using HTTP Flow Mining

  • Su, Xin;Liu, Xuchong;Lin, Jiuchuang;He, Shiming;Fu, Zhangjie;Li, Wenjia
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.6
    • /
    • pp.3230-3253
    • /
    • 2017
  • Android malware steals users' private information, and embedded unsafe advertisement (ad) libraries, which execute unsafe code causing damage to users. The majority of such traffic is HTTP and is mixed with other normal traffic, which makes the detection of malware and unsafe ad libraries a challenging problem. To address this problem, this work describes a novel HTTP traffic flow mining approach to detect and categorize Android malware and unsafe ad library. This work designed AndroCollector, which can automatically execute the Android application (app) and collect the network traffic traces. From these traces, this work extracts HTTP traffic features along three important dimensions: quantitative, timing, and semantic and use these features for characterizing malware and unsafe ad libraries. Based on these HTTP traffic features, this work describes a supervised classification scheme for detecting malware and unsafe ad libraries. In addition, to help network operators, this work describes a fine-grained categorization method by generating fingerprints from HTTP request methods for each malware family and unsafe ad libraries. This work evaluated the scheme using HTTP traffic traces collected from 10778 Android apps. The experimental results show that the scheme can detect malware with 97% accuracy and unsafe ad libraries with 95% accuracy when tested on the popular third-party Android markets.

A Convergence Implementation of Realtime Traffic Shaping and IPS on Small Integrated Security Router for IDC (IDC용 소형 통합보안라우터의 실시간 트래픽쉐이핑과 IPS의 융합 구현)

  • Yang, SeungEui;Park, Kiyoung;Jung, HoeKyung
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.23 no.7
    • /
    • pp.861-868
    • /
    • 2019
  • Various server-based services such as big data, IoT and artificial intelligence have been made online. As a result, the demand for IDC to support stable server operation is increasing. IDC is a server-based facility with a stable line and power supply facility that manages 20 to 30 servers in an efficiently separated rack-level subnetwork. Here, we need a way to efficiently manage servers security, firewall, and traffic on a rack-by-rack basis. Including traffic shaping capabilities that control routers, firewalls, IPS, and line speeds, as well as VPN technology, a recent interest. If three or five kinds of commercial equipment are adopted to support this, it may be a great burden to the management cost as well as the introduction cost. Therefore, in this paper, we propose a method to implement the five functions in one rack-unit small integrated security router. In particular, IDC intends to integrate traffic shaping and IPS, which are essential technologies, and to propose the utility accordingly.

A Study on User Behavior Analysis for Deriving Smart City Service Needs (스마트시티 서비스 니즈 도출을 위한 사용자 행위 분석에 관한 연구)

  • An, Se-Yun;Kim, So-Yeon
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.7
    • /
    • pp.330-337
    • /
    • 2018
  • Recently, there has been a growing interest in user-centered smart city services. In this study, user behavior analysis was performed as a preliminary study for user - centered smart city service planning. In particular, we will use GIS based location analysis data and video ethonography methodology to derive smart city service direction and needs. In this study, the area of Daejeon Design District selected as the Smart City Test bed was selected as the survey area and the location analysis data of the traffic accident analysis system of the road traffic corporation and the fixed camera We observed user's behavior type and change with image data extracted through the technique. Location analysis data is classified according to the type of accident, and image data is classified into 11 subdivided types of user activities. The problems and specificities observed were analyzed. The user behavior characteristics investigated through this study are meaningful to provide a basis for suggesting user - centered smart city services in the future.

Methodology for Developing a Predictive Model for Highway Traffic Information Using LSTM (LSTM을 활용한 고속도로 교통정보 예측 모델 개발 방법론)

  • Yoseph Lee;Hyoung-suk Jin;Yejin Kim;Sung-ho Park;Ilsoo Yun
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.22 no.5
    • /
    • pp.1-18
    • /
    • 2023
  • With the recent developments in big data and deep learning, a variety of traffic information is collected widely and used for traffic operations. In particular, long short-term memory (LSTM) is used in the field of traffic information prediction with time series characteristics. Since trends, seasons, and cycles differ due to the nature of time series data input for an LSTM, a trial-and-error method based on characteristics of the data is essential for prediction models based on time series data in order to find hyperparameters. If a methodology is established to find suitable hyperparameters, it is possible to reduce the time spent in constructing high-accuracy models. Therefore, in this study, a traffic information prediction model is developed based on highway vehicle detection system (VDS) data and LSTM, and an impact assessment is conducted through changes in the LSTM evaluation indicators for each hyperparameter. In addition, a methodology for finding hyperparameters suitable for predicting highway traffic information in the transportation field is presented.

Job-aware Network Scheduling for Hadoop Cluster

  • Liu, Wen;Wang, Zhigang;Shen, Yanming
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.1
    • /
    • pp.237-252
    • /
    • 2017
  • In recent years, data centers have become the core infrastructure to deal with big data processing. For these big data applications, network transmission has become one of the most important factors affecting the performance. In order to improve network utilization and reduce job completion time, in this paper, by real-time monitoring from the application layer, we propose job-aware priority scheduling. Our approach takes the correlations of flows in the same job into account, and flows in the same job are assigned the same priority. Therefore, we expect that flows in the same job finish their transmissions at about the same time, avoiding lagging flows. To achieve load balancing, two approaches (Flow-based and Spray) using ECMP (Equal-Cost multi-path routing) are presented. We implemented our scheme using NS-2 simulator. In our evaluations, we emulate real network environment by setting background traffic, scheduling delay and link failures. The experimental results show that our approach can enhance the Hadoop job execution efficiency of the shuffle stage, significantly reduce the network transmission time of the highest priority job.

Analysis of Component Technology for Smart City Platform

  • Park, Chulsu;Cha, Jaesang
    • International Journal of Advanced Culture Technology
    • /
    • v.7 no.3
    • /
    • pp.143-148
    • /
    • 2019
  • In order to solve the urban problems caused by the increase of the urban population, the construction of smart city applying the latest technology is being carried out all over the world. In particular, we will create a smart city platform that utilizes data generated in the city to collect and store and analyze, thereby enhancing the city's continuous competitiveness and resilience and enhancing the quality of life of citizens. However, existing smart city platforms are not enough to construct a platform for smart city as a platform for solution elements such as IoT platform, big data platform, and AI platform. To complement this, we will reanalyze the existing overseas smart city platform and IoT platform in a comprehensive manner, combine the technical elements applied to it, and apply it to the future Korean smart city platform. This paper aims to investigate the trends of smart city platforms used in domestic and foreign countries and analyze the technology applied to smart city to study smart city platforms that solve various problems of the city such as environment, energy, safety, traffic, environment.

A study on the Effect of Quality Characteristics of M2M Big Data providing real-time Information on User Satisfaction (실시간 정보를 제공하는 M2M 빅데이터 품질특성이 사용자 만족에 미치는 영향에 대한 연구 - 버스기사의 교통정보 시스템 중심으로 -)

  • DongSik, Yang;DongJin, Park;YunJae, Lee
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.27 no.6
    • /
    • pp.25-40
    • /
    • 2022
  • This study is about how the quality of M2M big data that provides real-time information affects users. Recently, there are many difficulties in acquiring and managing data because data types such as variety, data volume, and data velocity are changing rapidly and diversified. This not only leads to a decrease in data quality but also it can give a negative impact when making decisions using data. Generally, the quality of data is defined as 'suitability for use', which means that data quality must meet the expectations of user needs. Therefore, data providers need activities to improve data quality for this purpose, and the key is to identify data quality dimensions in each field where data is used and provide data suitable for the level of user needs. In this study, the relationship between the quality area of real-time M2M data used in the traffic information system and user satisfaction was analyzed. Research models and hypotheses were established to analyze the effects between variables related to M2M big data. In order to test the hypothesis, a causal relationship between the major factors was identified by conducting a survey and analyzing the data users.

Study on the Development of Congestion Index for Expressway Service Areas Based on Floating Population Big Data (유동인구 빅데이터 기반 고속도로 휴게소 혼잡지표 개발 연구)

  • Kim, Hae;Lee, Hwan-Pil;Kwon, Cheolwoo;Park, Sungho;Park, Sangmin;Yun, Ilsoo
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.17 no.4
    • /
    • pp.99-111
    • /
    • 2018
  • Service areas in expressways are very important facilities in terms of efficient expressway operation and the convenience of users. It needs a traffic management strategy to inform drivers in advance about congestion in service areas so as to distribute users of service areas. But due to the lack of sensors and data on numbers of people in the service areas, congestion in service areas had not been measured and managed appropriately. In this study, a congestion index for service areas was developed using telecommunication floating population big data. Two alternative indices (i.e., density of service areas and floating population V/c of service areas) were developed. Finally, the floating population V/c of service areas was selected as a congestion index for service areas for reasons of the ease of understanding and comparison.