[KSCI] Korea Science Citation Index Service

http://dx.doi.org/10.3837/tiis.2016.03.008

Multi-level Scheduling Algorithm Based on Storm

Wang, Jie (School of Software, Dalian University of Technology)
Hang, Siguang (School of Software, Dalian University of Technology)
Liu, Jiwei (School of Software, Dalian University of Technology)
Chen, Weihao (School of Software, Dalian University of Technology)
Hou, Gang (School of Software, Dalian University of Technology)

Publication Information

KSII Transactions on Internet and Information Systems (TIIS) / v.10, no.3, 2016 , pp. 1091-1110 More about this Journal

Abstract

Hybrid deployment under current cloud data centers is a combination of online and offline services, which improves the utilization of the cluster resources. However, the performance of the cluster is often affected by the online services in the hybrid deployment environment. To improve the response time of online service (e.g. search engine), an effective scheduling algorithm based on Storm is proposed. At the component level, the algorithm dispatches the component with more influence to the optimal performance node. Inside the component, a reasonable resource allocation strategy is used. By searching the compressed index first and then filtering the complete index, the execution speed of the component is improved with similar accuracy. Experiments show that our algorithm can guarantee search accuracy of 95.94%, while increasing the response speed by 68.03%.

Keywords

Mixed load; long tail delay; online services; Storm;

Citations & Related Records

Reference

1	H. Yang, A. Breslow, J. Mars and L. Tang. "Bubble-flux: Precise online qos management for increased utilization in warehouse scale computers". In Proc. of the 40th Annual International Symposium on Computer Architecture, pp. 607-618, June 23-27, 2013. Article (CrossRefLink)
2	Zhang X, Tune E, Hagmann R, R. Jnagal and V. Gokhale." CPI 2: CPU performance isolation for shared compute clusters". In Proc. of the 8th ACM European Conference on Computer Systems. pp. 379-391, April 15-17, 2013. Article (CrossRefLink)
3	J. A. Colmenares, G. Eads, S. Hofmeyr, S. Bird., M. Moretó, D. Chou ... and J. D. Kubiatowicz. "Tessellation: Refactoring the OS around explicit resource containers with continuous adaptation". In Proc. of the 50th Annual Design Automation Conference, pp. 1-10, May 29-June 7, 2013. Article (CrossRefLink)
4	B. Rhoden, K. Klues, D. Zhu and E. Brewer. "Improving per-node efficiency in the datacenter with new OS abstractions". In Proc. of the 2nd ACM Symposium on Cloud Computing, pp. 25-32, October 26-28, 2011. Article(CrossRefLink)
5	D. Xu, C. Wu, P. C. Yew. "On Mitigating Memory Bandwidth Contention through Bandwidth-Aware Scheduling" In Proc. of the 19th international conference on Parallel architectures and compilation techniques, pp.237-248, 2010. Article (CrossRefLink)
6	J. Ahn, C. Kim, J. Han, Y. R. Cho and J. Huh. "Dynamic virtual machine scheduling in clouds for architectural shared resources". In Proc. of the 4th USENIX conference on Hot Topics in Cloud Computing, pp. 1-5, June, 2012. Article (CrossRefLink)
7	C. Delimitrou, C. Kozyrakis. “Paragon: QoS-aware scheduling for heterogeneous datacenters”. Acm Sigarch Computer Architecture News, vol. 48, no. 4, pp.77-88, April, 2013. Article(CrossRefLink)
8	J. Dean, S. Ghemawat. “MapReduce: simplified data processing on large clusters”.Communications of the ACM, vol. 51, no.1, pp. 107-113, 2008. Article (CrossRefLink) DOI
9	C. Kozyrakis. "Resource efficient computing for warehouse-scale datacenters". In Proc. Of the Conference on Design, Automation and Test in Europe, pp. 1351-1356, March 18-22, 2013. Article (CrossRefLink)
10	J. Dean J, L. A. Barroso. “The tail at scale”. Communications of the ACM, vol. 56, no. 2, pp. 74-90, 2013. Article (CrossRefLink) DOI
11	H. Kasture, D. Sanchez. “Ubik: efficient cache sharing with strict qos for latency-critical workloads”. Association for Computing Machinery, vol. 42, pp.729-742, 2014. Article(CrossRefLink)
12	S. Govindan, J. Liu, A. Kansal and A. Sivasubramaniam. "Cuanta: quantifying effects of shared on-chip resource interference for consolidated virtual machines". In Proc. of the 2nd ACM Symposium on Cloud Computing, pp. 22-35, October 26-28, 2011. Article (CrossRefLink)
13	C. Stewart, A. Chakrabarti and R. Griffith. "Zoolander: Efficiently meeting very strict, low-latency SLOs". In Proc. of the 10th International Conference on Autonomic Computing, pp. 265-277, June, 2013. Article (CreossRefLink)
14	A. Vulimiri, O. Michel, P. Godfrey and S. Shenker. "More is less: reducing latency via redundancy". In Proc. of the 11th ACM Workshop on Hot Topics in Networks, pp. 13-18, October 29-30, 2012. Article(CrossRefLink)
15	G. Ananthanarayanan, A. Ghodsi, S. Shenker and I. Stoica. “Effective straggler mitigation: attack of the clones”. Proc Nsdi, vol. 21, no. 10, pp.185-198, April, 2013. Article (CrossRefLink)
16	V. Jalaparti, P. Bodik, S. Kandula, I. Menache, M. Rybalkin and C. Yan. "Speeding up distributed request-response workflows". In Proc. of the SIGCOMM 2013 and Best Papers of the Co-Located Workshops, pp. 219-230, August 12-16, 2013. Article (CrossRefLink)
17	R. Han, J. Wang, F. Ge, et al. "SARP: producing approximate results with small correctness losses for cloud interactive services". In Proc. of the 12th ACM International Conference on Computing Frontiers. ACM, 2015. Article (CrossRefLink)
18	J.Wilkes and C.Reiss. “Details of the ClusterData-2011-1 trace”. Article (CrossRefLink)
19	C. Reiss, A. Tumanov, G. R. Ganger, R. H. Katz and M. A. Kozuch. "Heterogeneity and dynamicity of clouds at scale: Google trace analysis" In Proc. of the Third ACM Symposium on Cloud Computing. p. 7, October 14-17, 2012. Article (CrossRefLink)