• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제10권4호
• /
• pp.1519-1534
• /
• 2016

In this paper, we study how to optimize the data shuffle phase by leveraging the flow relationship in datacenter networks (DCNs). In most of the clustering computer frameworks, the completion of a transfer (a group of flows that can enable a computation stage to start or complete) is determined by the flow completing last, so that limiting the rate of other flows (not the last one) appropriately can save bandwidth without impacting the performance of any transfer. Furthermore, for the flows enter network late, more bandwidth can be assigned to them to accelerate the completion of the entire transfer. Based on these characteristics, we propose the flow chasing algorithm (FCA) to optimize the completion of the entire transfer. We implement FCA on a real testbed. By evaluation, we find that FCA can not only reduce the completion time of data transfer by 6.24% on average, but also accelerate the completion of data shuffle phase and entire job.