• Title/Summary/Keyword: traffic flows

Search Result 346, Processing Time 0.028 seconds

Job-aware Network Scheduling for Hadoop Cluster

  • Liu, Wen;Wang, Zhigang;Shen, Yanming
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.1
    • /
    • pp.237-252
    • /
    • 2017
  • In recent years, data centers have become the core infrastructure to deal with big data processing. For these big data applications, network transmission has become one of the most important factors affecting the performance. In order to improve network utilization and reduce job completion time, in this paper, by real-time monitoring from the application layer, we propose job-aware priority scheduling. Our approach takes the correlations of flows in the same job into account, and flows in the same job are assigned the same priority. Therefore, we expect that flows in the same job finish their transmissions at about the same time, avoiding lagging flows. To achieve load balancing, two approaches (Flow-based and Spray) using ECMP (Equal-Cost multi-path routing) are presented. We implemented our scheme using NS-2 simulator. In our evaluations, we emulate real network environment by setting background traffic, scheduling delay and link failures. The experimental results show that our approach can enhance the Hadoop job execution efficiency of the shuffle stage, significantly reduce the network transmission time of the highest priority job.

Effect of bogie fairings on the snow reduction of a high-speed train bogie under crosswinds using a discrete phase method

  • Gao, Guangjun;Zhang, Yani;Zhang, Jie;Xie, Fei;Zhang, Yan;Wang, Jiabin
    • Wind and Structures
    • /
    • v.27 no.4
    • /
    • pp.255-267
    • /
    • 2018
  • This paper investigated the wind-snow flow around the bogie region of a high-speed train under crosswinds using a coupled numerical method of the unsteady Realizable $k-{\varepsilon}$ turbulence model and discrete phase model (DPM). The flow features around the bogie region were discussed and the influence of bogie fairing height on the snow accumulation on the bogie was also analyzed. Here the high-speed train was running at a speed of 200 km/h in a natural environment with the crosswind speed of 15 m/s. The mesh resolution and methodology for CFD analysis were validated against wind tunnel experiments. The results show that large negative pressure occurs locally on the bottom of wheels, electric motors, gear covers, while the positive pressure occurs locally on those windward surfaces. The airflow travels through the complex bogie and flows towards the rear bogie plate, causing a backflow in the upper space of the bogie region. The snow particles mainly accumulate on the wheels, electric motors, windward sides of gear covers, side fairings and back plate of the bogie. Longer side fairings increase the snow accumulation on the bogie, especially on the back plate, side fairings and brake clamps. However, the fairing height shows little impact on snow accumulation on the upper region of the bogie. Compared to short side fairings, a full length side fairing model contributes to more than two times of snow accumulation on the brake clamps, and more than 20% on the whole bogie.

A Real-Time Network Traffic Anomaly Detection Scheme Using NetFlow Data (NetFlow 데이터를 이용한 실시간 네트워크 트래픽 어노멀리 검출 기법)

  • Kang Koo-Hong;Jang Jong-Soo;Kim Ki-Young
    • The KIPS Transactions:PartC
    • /
    • v.12C no.1 s.97
    • /
    • pp.19-28
    • /
    • 2005
  • Recently, it has been sharply increased the interests to detect the network traffic anomalies to help protect the computer network from unknown attacks. In this paper, we propose a new anomaly detection scheme using the simple linear regression analysis for the exported LetFlow data, such as bits per second and flows per second, from a border router at a campus network. In order to verify the proposed scheme, we apply it to a real campus network and compare the results with the Holt-Winters seasonal algorithm. In particular, we integrate it into the RRDtooi for detecting the anomalies in real time.

Resilient Reduced-State Resource Reservation

  • Csaszar Andras;Takacs Attila;Szabo Robert;Henk Tamas
    • Journal of Communications and Networks
    • /
    • v.7 no.4
    • /
    • pp.509-524
    • /
    • 2005
  • Due to the strict requirements of emerging applications, per-flow admission control is gaining increasing importance. One way to implement per-flow admission control is using an on­path resource reservation protocol, where the admission decision is made hop-by-hop after a new flow request arrives at the network boundary. The next-steps in signaling (NSIS) working group of the Internet engineering task force (IETF) is standardising such an on-path signaling protocol. One of the reservation methods considered by NSIS is reduced-state mode, which, suiting the differentiated service (DiffServ) concept, only allows per-class states in interior nodes of a domain. Although there are clear benefits of not dealing with per-flow states in interior nodes-like scalability and low complexity-, without per-flow states the handling of re-routed flows, e.g., after a failure, is a demanding and highly non-trivial task. To be applied in carrier-grade networks, the protocol needs to be resilient in this situation. In this article, we will explain the consequences of a route failover to resource reservation protocols: Severe congestion and incorrect admission decisions due to outdated reservation states. We will set requirements that handling solutions need to fulfill, and we propose extensions to reduced-state protocols accordingly. We show with a set of simulated scenarios that with the given solutions reduced-state protocols can handle re-routed flows practically as fast and robust as stateful protocols.

Multi-Agent for Traffic Simulation with Vehicle Dynamic Model I : Development of Traffic Environment (차량 동역학을 이용한 멀티에이전트 기반 교통시뮬레이션 개발 I : 교통 환경 개발)

  • 조기용;권성진;배철호;서명원
    • Transactions of the Korean Society of Automotive Engineers
    • /
    • v.12 no.5
    • /
    • pp.125-135
    • /
    • 2004
  • The validity of simulation has been well-established for decades in areas such as computer and communication system. Recently, the technique has become entrenched in specific areas such as transportation and traffic forecasting. Several methods have been proposed for investigating complex traffic flows. However, the dynamics of vehicles and their driver's characteristics, even though it is known that they are important factors for any traffic flow analysis, have never been considered sufficiently. In this paper, the traffic simulation using a multi-agent approach with considering vehicle dynamics is proposed. The multi-agent system is constructed with the traffic environment and the agents of vehicle and driver. The traffic environment consists of multi-lane roads, nodes, virtual lanes, and signals. To ensure the fast calculation, the agents are performed on the based of the rules to regulate their behaviors. The communication frameworks are proposed for the agents to share the information of vehicles' velocity and position. The model of a driver agent which controls a vehicle agent is described in the companion paper. The vehicle model contains the nonlinear subcomponents of engine, torque converter, automatic transmission, and wheels. The simulation has proceeded for an interrupted and uninterrupted flow model. The result has shown that the driver agent performs human-like behavior ranging from slow and careful to fast and aggressive driving behavior, and that the change of the traffic state is closely related with the distance and the signal delay between intersections. The system developed shows the effectiveness and the practical usefulness of the traffic simulation.

Ventilation Characteristics by Traffic Piston Effect in Underground Network-type Road Junction (네트워크형 지하도로 입체교차로 내의 교통환기력에 의한 환기 특성)

  • Kim, Nam-Young;Jo, Jong-Bok;Han, Hwataik
    • Korean Journal of Air-Conditioning and Refrigeration Engineering
    • /
    • v.27 no.7
    • /
    • pp.337-343
    • /
    • 2015
  • This paper investigates the ventilation characteristics in a two-dimensional underground network junction composed of four main lines interconnected by eight ramps. Simple one-dimensional models cannot be applied to network junctions since there are interferences of traffic piston effects in the main lines and at the ramps. A numerical algorithm was developed to analyze the pressure and airflow distributions iteratively. The Darcy-Weisbach equation was used to calculate the piston effects by traffic flows, and a Hardy Cross iteration was conducted for network analysis at the interconnected junction. The results show interesting ventilation characteristics and CO concentration distributions depending on system parameters such as vehicle speed, tunnel diameter, and other junction configurations.

One-dimensional CNN Model of Network Traffic Classification based on Transfer Learning

  • Lingyun Yang;Yuning Dong;Zaijian Wang;Feifei Gao
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.2
    • /
    • pp.420-437
    • /
    • 2024
  • There are some problems in network traffic classification (NTC), such as complicated statistical features and insufficient training samples, which may cause poor classification effect. A NTC architecture based on one-dimensional Convolutional Neural Network (CNN) and transfer learning is proposed to tackle these problems and improve the fine-grained classification performance. The key points of the proposed architecture include: (1) Model classification--by extracting normalized rate feature set from original data, plus existing statistical features to optimize the CNN NTC model. (2) To apply transfer learning in the classification to improve NTC performance. We collect two typical network flows data from Youku and YouTube, and verify the proposed method through extensive experiments. The results show that compared with existing methods, our method could improve the classification accuracy by around 3-5%for Youku, and by about 7 to 27% for YouTube.

CCTV-Aided Accident Detection System on Four Lane Highway with Calogero-Moser System (칼로게로 모제 시스템을 활용한 4차선 도로의 사고검지 폐쇄회로 카메라 시스템)

  • Lee, In Jeong
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.39C no.3
    • /
    • pp.255-263
    • /
    • 2014
  • Today, a number of CCTV on the highway is to observe the flow of traffics. There have been a number of studies where traffic data (e.g., the speed of vehicles and the amount of traffic on the road) are transferred back to the centralized server so that an appropriate action can be taken. This paper introduces a system that detects the changes of traffic flows caused by an accident or unexpected stopping (i.e., vehicle remains idle) by monitoring each lane separately. The traffic flows of each lane are level spacing curve that shows Wigner distribution for location vector. Applying calogero-moser system and Hamiltonian system, probability equation for each level-spacing curve is derived. The high level of modification of the signal means that the lane is in accident situation. This is different from previous studies in that it does more than looking for the signal from only one lane, now it is able to detect an accident in entire flow of traffic. In process of monitoring traffic flow of each lane, when camera recognizes a shadow of vehicle as a vehicle, it will affect the accident detecting capability. To prevent this from happening, the study introduces how to get rid of such shadow. The system using Basian network method is being compared for capability evaluation of the system of the study. As a result, the system of the study appeared to be better in performance in detecting the modification of traffic flow caused by idle vehicle.

Signal Optimization Model Considering Traffic Flows in General Traffic Networks (일반적인 네트워크에서의 신호최적화모형 개발 연구)

  • 신언교;김영찬
    • Journal of Korean Society of Transportation
    • /
    • v.17 no.2
    • /
    • pp.127-135
    • /
    • 1999
  • Most existing progression bandwidth models maximize the single or multi weighted sum of bandwidths in the both directions to improve traffic mobility on an arterial, but they cannot be applied to general networks. Even though a few models formulating a looped network problem cannot be applied to networks have not loops. Also they have some defects in optimizing phase sequences. Therefore, the objective of this study is to develope a mathematical formulation of the synchronization problem for a general traffic network. The goal is achieved successfully by introducing the signal phasing for each movement and expanding the mixed integer linear programming of MAXBAND. The experiments indicate that the proposed model can formulate the general traffic network problem mere efficiently than any other model. In conclusion, this model may optimize signal time to smooth progression in the general networks.

  • PDF

Opportunistic Routing for Bandwidth-Sensitive Traffic in Wireless Networks with Lossy Links

  • Zhao, Peng;Yang, Xinyu
    • Journal of Communications and Networks
    • /
    • v.18 no.5
    • /
    • pp.806-817
    • /
    • 2016
  • Opportunistic routing (OR) has been proposed as a viable approach to improve the performance of wireless multihop networks with lossy links. However, the exponential growth of the bandwidth-sensitive mobile traffic (e.g., mobile video streaming and online gaming) poses a great challenge to the performance of OR in term of bandwidth guarantee. To solve this problem, a novel mechanism is proposed to opportunistically forwarding data packets and provide bandwidth guarantee for the bandwidth-sensitive traffic. The proposal exploits the broadcast characteristic of wireless transmission and reduces the negative effect of wireless lossy links. First, the expected available bandwidth (EAB) and the expected transmission cost (ETC) under OR are estimated based on the local available bandwidth, link delivery probability, forwarding candidates, and prioritization policy. Then, the policies for determining and prioritizing the forwarding candidates is devised by considering the bandwidth and transmission cost. Finally, bandwidth-aware routing algorithm is proposed to opportunistically delivery data packets; meanwhile, admission control is applied to admit or reject traffic flows for bandwidth guarantee. Extensive simulation results show that our proposal consistently outperforms other existing opportunistic routing schemes in providing performance guarantee.