ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Experimental Performance Comparison of Dynamic Data Race Detection Techniques

  • Received : 2015.12.01
  • Accepted : 2016.11.15
  • Published : 2017.02.01
Download PDF
⟨ Previous Next ⟩

Abstract

Data races are one of the most difficult types of bugs in concurrent multithreaded systems. It requires significant time and cost to accurately detect bugs in complex large-scale programs. Although many race detection techniques have been proposed by various researchers, none of them are effective in all aspects. In this paper, we compare the performance of five recent dynamic race detection techniques: FastTrack, Acculock, Multilock-HB, SimpleLock+, and causally precedes (CP) detection. We experimentally demonstrate the strengths and weaknesses of these dynamic race detection techniques in terms of their detection capability, running time, and runtime overhead using 20 benchmark programs with different characteristics. The comparison results show that the detection capability of CP detection does not differ from that of FastTrack, and that SimpleLock+ generates the lowest overhead among the hybrid detection techniques (Acculock, SimpleLock+, and Multilock-HB) for all benchmark programs. SimpleLock+ is 1.2 times slower than FastTrack on average, but misses one true data race reported from Mutilock-HB on the large-scale benchmark programs.

Keywords

References

  1. N.G. Leveson and C.S. Turner, "An Investigation of the Therac-25 Accidents," Comput., vol. 26, no. 7, July 1993, pp. 18-41. https://doi.org/10.1109/MC.1993.274940
  2. K. Poulsen, Software Bug Contributed to Blackout, Accessed Dec. 30, 2015. http://www.securityfocus.com/news/8016
  3. R.H.B. Netzer and B.P. Miller, "What Are Race Conditions?: Some Issues and Formalizations," Lett. Programming Languages Syst., vol. 1, no. 1, Mar. 1992, pp. 74-88. https://doi.org/10.1145/130616.130623
  4. E. Pozniansky and A. Schuster, "Efficient On-the-fly Data Race Detection in Multithreaded C++ Programs," Symp. Principles Practice Parallel Programming, San Diego, CA, USA, June 11-13, 2003, pp. 179-190.
  5. S. Savage et al., "Eraser: A Dynamic Data Race Detector for Multithreaded Programs," Symp. Operating Syst. Principles, Saint-Malo, France, Oct. 5-8, 1997, pp. 27-37.
  6. D. Marino, M. Musuvathi, and S. Narayanasamy, "LiteRace: Effective Sampling for Lightweight Data-Race Detection," Conf. Programming Language Des. Implementation, Dublin, Ireland, June 15, 2009, pp. 34-143.
  7. Y. Yu, T. Rodeheffer, and W. Chen, "RaceTrack: Efficient Detection of Data Race Conditions Via Adaptive Tracking," Symp. Operating Syst. Principles, Brighton, UK, Oct, 23-26, 2005, pp. 221-234.
  8. E. Pozniansky and A. Schuster, "MultiRace: Efficient On the Fly Data Race Detection in Multithreaded C++ Programs," Concurrency Comput.: Practice Experience, vol. 19, no. 3, Mar. 2007, pp. 327-340. https://doi.org/10.1002/cpe.1064
  9. C. Flanagan and S.N. Freund, "FastTrack: Efficient and Precise Dynamic Race Detection," Conf. Programming Language Des. Implementation, Dublin, Ireland, June 15-21, 2009, pp. 121-133.
  10. M.D. Bond, K.E. Coons, and K.S. McKinley, "PACER: Proportional Detection of Data Races," Conf. Programming Language Des. Implementation, Toronto, Canada, June 5-10, 2010, pp. 255-268.
  11. K. Zhai et al., "CARISMA: A Context-Sensitive Approach to Race-Condition Sample-Instance Selection for Multithreaded Applications," Int. Symp. Softw. Testing Anal., Minneapolis, MN, USA, July 15-20, 2012, pp. 221-231.
  12. Y.W. Song and Y.H. Lee, "Efficient Data Race Detection for C/C++ Programs Using Dynamic Granularity," Int. Parallel Distrib. Process. Symp., Phoenix, AZ, USA, May 19-23, 2014, pp. 679-688.
  13. R. O'Callahan and J.D. Choi, "Hybrid Dynamic Data Race Detection," Symp. Principles Practice Parallel Programming, San Diego, CA, USA, June 11-13, 2003, pp. 167-178.
  14. X. Xie and J. Xue, "Acculock: Accurate and Efficient Detection of Data Races," Symp. Code Generation Optimization, Chamonix, France, Apr. 2-6, 2011, pp. 201-212.
  15. X. Xie, J. Xue, and J. Zhang, "Acculock: Accurate and Efficient Detection of Data Races," Softw. Practice Experience, vol. 43, no. 5, May 2013, pp. 543-576. https://doi.org/10.1002/spe.2121
  16. M.S. Yu and D.H. Bae, "SimpleLock+: Fast and Accurate Hybrid Data Race Detection," Comput. J., vol. 59, no. 6, 2016, pp. 793-809. https://doi.org/10.1093/comjnl/bxu119
  17. Y. Smaragdakis et al., "Sound Predictive Race Detection in Polynomial Time," Symp. Principles Programming Languages, Philadelphia, PA, USA, Jan. 25-27, 2012, pp. 387-400.
  18. J. Huang, P.O. Meredith, and G. Rosu, "Maximal Sound Predictive Race Detection with Control Flow Abstraction," Conf. Programming Language Des. Implementation, Edinburgh, Ireland, June 9-11, 2014, pp. 337-348.
  19. SPEC JVM98 benchmarks, Accessed Dec. 30, 2015. http://www.spec.org/osg/jvm98/
  20. L.A. Smith, J.M. Bull, and J. Obdrizalek, "A Parallel Java Grande Benchmark Suite," Conf. Supercomput., Denver, CO, USA, Nov. 10-16, 2001, p. 8.
  21. C. Flanagan and S.N. Freund, "The RoadRunner Dynamic Analysis Framework for Concurrent Programs," Workshop Program Anal. Softw. Tools Eng., Toronto, Canada, June 5-6, 2010, pp. 1-8.
  22. S.M. Blackburn et al., "The DaCapo Benchmarks: Java Benchmarking Development and Analysis," Conf. Object-Oriented Programming Syst., Languages, Appli., Portland, OR, USA, Oct. 22-26, 2006, pp. 169-190.
  23. M. Said et al., "Generating Data Race Witnesses by an SMT-Based Analysis," Int. Conf. NASA Formal Methods Symp., Pasadena, CA, USA, Apr. 18-20, 2011, pp. 313-327.
  24. C. Flanagan and S.N. Freund, "Type-Based Race Detection for Java," Conf. Programming Language Des. Implementation, Vancouver, Canada, June 18-21, 2000, pp. 219-232.
  25. C. Flanagan and M. Abadi, "Object Types Against Races," Conf. Concurrency Theory, Eindhoven, Netherlands, Aug. 24-27, 1999, pp. 288-303.
  26. C. Boyapati, R. Lee, and M. Rinard, "Ownership Types for Safe Programming: Preventing Data Races and Deadlocks," Conf. Object-Oriented Programming, Syst., Languages, Appli., Seattle, WA, USA, Nov. 4-8, 2002, pp. 211-230.
  27. C. Boyapati and M. Rinard, "A Parameterized Type System for Race-Free Java Programs," Conf. Object-Oriented Programming, Syst. Languages, Appli, Tampa Bay, FL, USA, Oct. 14-18, 2001, pp. 56-69.
  28. R.L. Bocchino et al., "A Type and Effect System for Deterministic Parallel Java," Conf. Object Oriented Programming Syst. Languages Appli., Orlando, FL, USA, Oct. 25-29, 2009, pp. 97-116.
  29. A. Greenhouse and J. Boyland, "An Object-Oriented Effects System," European Conf. Object-Oriented Programming, Lisbon, Portugal, June 14-18, 1999, pp. 205-229.
  30. J. Boyland, "The Interdependence of Effects and Uniqueness," Workshop Formal Techs. Java Programs, Budapest, Hungary, June 18, 2001.
  31. C. Von Praun and T.R. Gross, "Static Conflict Analysis for Multi-threaded Object-Oriented Programs," Conf. Programming Language Des. Implementation, San Diego, CA, USA, June 8-11, 2003, pp. 115-128.
  32. M. Naik, A. Aiken, and J. Whaley, "Effective Static Race Detection for Java," Conf. Programming Language Des. Implementation, Ottawa, Canada, June 10-16, 2006, pp. 308-319.
  33. M. Naik and A. Aiken, "Conditional Must Not Aliasing for Static Race Detection," Symp. Principles Programming Languages, Nice, France, Jan. 17-19, 2007, pp. 327-338.
  34. C. Radoi and D. Dig, "Practical Static Race Detection for Java Parallel Loops," Int. Symp. Softw. Testing Anal., Lugano, Switzerland, July 15-20, 2013, pp. 178-190.
  35. Y. Lin and D. Dig, "A Study and Toolkit of CHECK-THEN-ACT Idioms of Java Concurrent Collections," Softw. Testing, Verification Rel., vol. 25, no. 4, June 2015, pp. 397-425. https://doi.org/10.1002/stvr.1567
  36. J. Wilcox et al., "Array Shadow State Compression for Precise Dynamic Race Detection," Conf. Automated Softw. Eng., Lincoln, NE, USA, Nov. 9-13, 2015, pp. 155-165.
  37. B.P. Wood, L. Ceze, and D. Grossman, "Low-Level Detection of Language-Level Data Races with LARD," Conf. Archit. Support Programming Languages Operating Syst., Salt Lake, Canada, Mar. 1-5, 2014, pp. 671-686.