Journal of the Korean Society for Industrial and Applied Mathematics

Korean Society for Industrial and Applied Mathematics (한국산업응용수학회)
권호 표지
DOI QR코드

DOI QR Code

RECENT ADVANCES IN DOMAIN DECOMPOSITION METHODS FOR TOTAL VARIATION MINIMIZATION

  • Received : 2020.03.26
  • Accepted : 2020.06.01
  • Published : 2020.06.25
Download PDF
⟨ Previous Next ⟩

Abstract

Total variation minimization is standard in mathematical imaging and there have been numerous researches over the last decades. In order to process large-scale images in real-time, it is essential to design parallel algorithms that utilize distributed memory computers efficiently. The aim of this paper is to illustrate recent advances of domain decomposition methods for total variation minimization as parallel algorithms. Domain decomposition methods are suitable for parallel computation since they solve a large-scale problem by dividing it into smaller problems and treating them in parallel, and they already have been widely used in structural mechanics. Differently from problems arising in structural mechanics, energy functionals of total variation minimization problems are in general nonlinear, nonsmooth, and nonseparable. Hence, designing efficient domain decomposition methods for total variation minimization is a quite challenging issue. We describe various existing approaches on domain decomposition methods for total variation minimization in a unified view. We address how the direction of research on the subject has changed over the past few years, and suggest several interesting topics for further research.

Keywords

References

  1. A. TOSELLI AND O. WIDLUND, Domain Decomposition Methods-Algorithms and Theory, vol. 34, Springer, Berlin, 2005.
  2. A. QUARTERONI AND A. VALLI, Domain Decomposition Methods for Partial Differential Equations, Oxford University Press, New York, 1999.
  3. J. XU, Iterative methods by space decomposition and subspace correction, SIAM Rev., 34 (1992), pp. 581-613. https://doi.org/10.1137/1034116
  4. C. R. DOHRMANN, A preconditioner for substructuring based on constrained energy minimization, SIAM J. Sci. Comput., 25 (2003), pp. 246-258. https://doi.org/10.1137/S1064827502412887
  5. J. MANDEL, Balancing domain decomposition, Commun. Numer. Methods Engrg., 9 (1993), pp. 233-241. https://doi.org/10.1002/cnm.1640090307
  6. C. FARHAT, M. LESOINNE, AND K. PIERSON, A scalable dual-primal domain decomposition method, Numer. Linear Algebra Appl., 7 (2000), pp. 687-714. https://doi.org/10.1002/1099-1506(200010/12)7:7/8<687::AID-NLA219>3.0.CO;2-S
  7. C. FARHAT AND F.-X. ROUX, A method of finite element tearing and interconnecting and its parallel solution algorithm, Internat. J. Numer. Methods Engrg., 32 (1991), pp. 1205-1227. https://doi.org/10.1002/nme.1620320604
  8. C.-O. LEE AND E.-H. PARK, A dual iterative substructuring method with a penalty term, Numer. Math., 112 (2009), pp. 89-113. https://doi.org/10.1007/s00211-008-0202-6
  9. L. I. RUDIN, S. OSHER, AND E. FATEMI, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259-268. https://doi.org/10.1016/0167-2789(92)90242-F
  10. D. STRONG AND T. CHAN, Edge-preserving and scale-dependent properties of total variation regularization, Inverse Problems, 19 (2003), pp. S165-S187. https://doi.org/10.1088/0266-5611/19/6/059
  11. A. CHAMBOLLE, An algorithm for total variation minimization and applications, J. Math. Imaging Vision, 20 (2004), pp. 89-97. https://doi.org/10.1023/b:jmiv.0000011321.19549.88
  12. Y. WANG, J. YANG, W. YIN, AND Y. ZHANG, A new alternating minimization algorithm for total variation image reconstruction, SIAM J. Imaging Sci., 1 (2008), pp. 248-272. https://doi.org/10.1137/080724265
  13. A. BECK AND M. TEBOULLE, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183-202. https://doi.org/10.1137/080716542
  14. T. GOLDSTEIN AND S. OSHER, The split Bregman method for L1-regularized problems, SIAM J. Imaging Sci., 2 (2009), pp. 323-343. https://doi.org/10.1137/080725891
  15. A. CHAMBOLLE AND T. POCK, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vision, 40 (2011), pp. 120-145. https://doi.org/10.1007/s10851-010-0251-1
  16. E. ESSER, X. ZHANG, AND T. F. CHAN, A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science, SIAM J. Imaging Sci., 3 (2010), pp. 1015-1046. https://doi.org/10.1137/09076934X
  17. L. BADEA, Convergence rate of a Schwarz multilevel method for the constrained minimization of nonquadratic functionals, SIAM J. Numer. Anal., 44 (2006), pp. 449-477. https://doi.org/10.1137/S003614290342995X
  18. L. BADEA AND R. KRAUSE, One-and two-level Schwarz methods for variational inequalities of the second kind and their application to frictional contact, Numer. Math., 120 (2012), pp. 573-599. https://doi.org/10.1007/s00211-011-0423-y
  19. X.-C. TAI AND J. XU, Global and uniform convergence of subspace correction methods for some convex optimization problems, Math. Comp., 71 (2001), pp. 105-124. https://doi.org/10.1090/S0025-5718-01-01311-4
  20. M. FORNASIER, A. LANGER, AND C.-B. SCHONLIEB, A convergent overlapping domain decomposition method for total variation minimization, Numer. Math., 116 (2010), pp. 645-685. https://doi.org/10.1007/s00211-010-0314-7
  21. M. FORNASIER AND C.-B. SCHONLIEB, Subspace correction methods for total variation and l1-minimization, SIAM J. Numer. Anal., 47 (2009), pp. 3397-3428. https://doi.org/10.1137/070710779
  22. M. HINTERMULLER AND A. LANGER, Subspace correction methods for a class of nonsmooth and nonadditive convex variational problems with mixed $L^1$/$L^2$ data-fidelity in image processing, SIAM J. Imaging Sci., 6 (2013), pp. 2134-2173. https://doi.org/10.1137/120894130
  23. Y. DUAN AND X.-C. TAI, Domain decomposition methods with graph cuts algorithms for total variation minimization, Adv. Comput. Math., 36 (2012), pp. 175-199. https://doi.org/10.1007/s10444-011-9213-4
  24. A. LANGER, S. OSHER, AND C.-B. SCHONLIEB, Bregmanized domain decomposition for image restoration, J. Sci. Comput., 54 (2013), pp. 549-576. https://doi.org/10.1007/s10915-012-9603-x
  25. C.-O. LEE, J. H. LEE, H. WOO, AND S. YUN, Block decomposition methods for total variation by primal-dual stitching, J. Sci. Comput., 68 (2016), pp. 273-302. https://doi.org/10.1007/s10915-015-0138-9
  26. C.-O. LEE AND C. NAM, Primal domain decomposition methods for the total variation minimization, based on dual decomposition, SIAM J. Sci. Comput., 39 (2017), pp. B403-B423. https://doi.org/10.1137/15M1049919
  27. H. CHANG, X.-C. TAI, L.-L. WANG, AND D. YANG, Convergence rate of overlapping domain decomposition methods for the Rudin-Osher-Fatemi model based on a dual formulation, SIAM J. Imaging Sci., 8 (2015), pp. 564-591. https://doi.org/10.1137/140965016
  28. M. HINTERMULLER AND A. LANGER, Non-overlapping domain decomposition methods for dual total variation based image denoising, J. Sci. Comput., 62 (2015), pp. 456-481. https://doi.org/10.1007/s10915-014-9863-8
  29. C.-O. LEE AND J. PARK, Fast nonoverlapping block Jacobi method for the dual Rudin-Osher-Fatemi model, SIAM J. Imaging Sci., 12 (2019), pp. 2009-2034. https://doi.org/10.1137/18M122604X
  30. C.-O. LEE, E.-H. PARK, AND J. PARK, A finite element approach for the dual Rudin-Osher-Fatemi model and its nonoverlapping domain decomposition methods, SIAM J. Sci. Comput., 41 (2019), pp. B205-B228. https://doi.org/10.1137/18m1165499
  31. C.-O. LEE AND J. PARK A finite element nonoverlapping domain decomposition method with Lagrange multipliers for the dual total variation minimizations, J. Sci. Comput., 81 (2019), pp. 2331-2355. https://doi.org/10.1007/s10915-019-01085-z
  32. Y. DUAN, H. CHANG, AND X.-C. TAI, Convergent non-overlapping domain decomposition methods for variational image segmentation, J. Sci. Comput., 69 (2016), pp. 532-555. https://doi.org/10.1007/s10915-016-0207-8
  33. T. F. CHAN, S. ESEDOGLU, AND M. NIKOLOVA, Algorithms for finding global minimizers of image segmentation and denoising models, SIAM J. Appl. Math., 66 (2006), pp. 1632-1648. https://doi.org/10.1137/040615286
  34. T. F. CHAN AND L. A. VESE, Active contours without edges, IEEE Trans. Image Process., 10 (2001), pp. 266- 277. https://doi.org/10.1109/83.902291
  35. J. PARK, An overlapping domain decomposition framework without dual formulation for variational imaging problems. arXiv:2002.10070 [math.NA], 2019. To appear in Adv. Comput. Math.
  36. C.-O. LEE, C. NAM, AND J. PARK, Domain decomposition methods using dual conversion for the total variation minimization with $L^1$ fidelity term, J. Sci. Comput., 78 (2019), pp. 951-970. https://doi.org/10.1007/s10915-018-0791-x
  37. T. F. CHAN AND S. ESEDOGLU, Aspects of total variation regularized $L^1$ function approximation, SIAM J. Appl. Math., 65 (2005), pp. 1817-1837. https://doi.org/10.1137/040604297
  38. A. TIKHONOV, Solution of incorrectly formulated problems and the regularization method, Soviet Math. Dokl., 4 (1963), pp. 1035-1038.
  39. G. AUBERT AND P. KORNPROBST, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, Springer, New York, 2006.
  40. E. GIUSTI, Minimal Surfaces and Functions of Bounded Variation, Birkhauser, Boston, 1984.
  41. X. FENG AND A. PROHL, Analysis of total variation flow and its finite element approximations, ESAIM Math. Model. Numer. Anal., 37 (2003), pp. 533-556. https://doi.org/10.1051/m2an:2003041
  42. A. CHAMBOLLE AND T. POCK, An introduction to continuous optimization for imaging, Acta Numer., 25 (2016), pp. 161-319. https://doi.org/10.1017/S096249291600009X
  43. Y. MEYER, Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: the Fifteenth Dean Jacqueline B. Lewis Memorial Lectures, vol. 22, American Mathematical Society, Providence, 2001.
  44. M. NIKOLOVA, A variational approach to remove outliers and impulse noise, J. Math. Imaging Vision, 20 (2004), pp. 99-120. https://doi.org/10.1023/b:jmiv.0000011920.58935.9c
  45. R. T. ROCKAFELLAR, Convex Analysis, Princeton University Press, New Jersey, 2015.
  46. I. EKELAND AND R. TEMAM, Convex Analysis and Variational Problems, vol. 28, SIAM, Philadelphia, 1999.
  47. Y. DONG, M. HINTERMULLER, AND M. NERI, An efficient primal-dual method for $L^1$TV image restoration, SIAM J. Imaging Sci., 2 (2009), pp. 1168-1189. https://doi.org/10.1137/090758490
  48. K. KUNISCH AND M. HINTERMULLER, Total bounded variation regularization as a bilaterally constrained optimization problem, SIAM J. Appl. Math., 64 (2004), pp. 1311-1333. https://doi.org/10.1137/S0036139903422784
  49. L. BADEA, X.-C. TAI, AND J. WANG, Convergence rate analysis of a multiplicative Schwarz method for variational inequalities, SIAM J. Numer. Anal., 41 (2003), pp. 1052-1073. https://doi.org/10.1137/S0036142901393607
  50. X.-C. TAI, Rate of convergence for some constraint decomposition methods for nonlinear variational inequalities, Numer. Math., 93 (2003), pp. 755-786. https://doi.org/10.1007/s002110200404
  51. J. BOLTE, S. SABACH, AND M. TEBOULLE, Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program., Ser. A, 146 (2014), pp. 459-494.
  52. A. CHAMBOLLE, AND T POCK, A remark on accelerated block coordinate descent for computing the proximity operators of a sum of convex functions, SMAI J. Comp. Math., 1 (2015), pp. 29-54. https://doi.org/10.5802/smai-jcm.3
  53. R. SHEFI AND M. TEBOULLE, On the rate of convergence of the proximal alternating linearized minimization algorithm for convex problems, EURO J. Comput. Optim., 4 (2016), pp. 27-46. https://doi.org/10.1007/s13675-015-0048-5
  54. A. LANGER AND F. GASPOZ, Overlapping domain decomposition methods for total variation denoising, SIAM J. Numer. Anal., 57 (2019), pp. 1411-1444. https://doi.org/10.1137/18M1173782
  55. S. BARTELS, Total variation minimization with finite elements: convergence and iterative solution, SIAM J. Numer. Anal., 50 (2012), pp. 1162-1180. https://doi.org/10.1137/11083277X
  56. M. HERRMANN, R. HERZOG, S. SCHMIDT, J. VIDAL-NUNEZ, AND G. WACHSMUTH, Discrete total variation with finite elements and applications to imaging, J. Math. Imaging Vision, 61 (2019), pp. 411-431. https://doi.org/10.1007/s10851-018-0852-7
  57. P.-A. RAVIART AND J.-M. THOMAS, A mixed finite element method for 2-nd order elliptic problems, in Mathematical Aspects of Finite Element Methods, Springer, 1977, pp. 292-315.
  58. I. GOODFELLOW, Y. BENGIO, AND A. COURVILLE, Deep Learning, MIT Press, Cambridge, 2016.
  59. S. J. WRIGHT, Coordinate descent algorithms, Math. Program., 151 (2015), pp. 3-34. https://doi.org/10.1007/s10107-015-0892-3
  60. P. L. COMBETTES AND V. R. WAJS, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200. https://doi.org/10.1137/050626090
  61. Y. E. NESTEROV, A method for solving the convex programming problem with convergence rate O(1=$k^2$), Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543-547.
  62. B. HE, Y. YOU, AND X. YUAN, On the convergence of primal-dual hybrid gradient algorithm, SIAM J. Imaging Sci., 7 (2014), pp. 2526-2537. https://doi.org/10.1137/140963467
  63. B. HE AND X. YUAN, Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective, SIAM J. Imaging Sci., 5 (2012), pp. 119-149. https://doi.org/10.1137/100814494
  64. R. T. ROCKAFELLAR, Monotone operators and the proximal point algorithm, SIAM J. Control Optim., 14 (1976), pp. 877-898. https://doi.org/10.1137/0314056
  65. A. CHAMBOLLE, AND T POCK, On the ergodic convergence rates of a first-order primal-dual algorithm, Math. Program., Ser. A, 159 (2016), pp. 253-287. https://doi.org/10.1007/s10107-015-0957-3