Browse > Article
http://dx.doi.org/10.12941/jksiam.2020.24.161

RECENT ADVANCES IN DOMAIN DECOMPOSITION METHODS FOR TOTAL VARIATION MINIMIZATION  

LEE, CHANG-OCK (DEPARTMENT OF MATHEMATICAL SCIENCES, KAIST)
PARK, JONGHO (DEPARTMENT OF MATHEMATICAL SCIENCES, KAIST)
Publication Information
Journal of the Korean Society for Industrial and Applied Mathematics / v.24, no.2, 2020 , pp. 161-197 More about this Journal
Abstract
Total variation minimization is standard in mathematical imaging and there have been numerous researches over the last decades. In order to process large-scale images in real-time, it is essential to design parallel algorithms that utilize distributed memory computers efficiently. The aim of this paper is to illustrate recent advances of domain decomposition methods for total variation minimization as parallel algorithms. Domain decomposition methods are suitable for parallel computation since they solve a large-scale problem by dividing it into smaller problems and treating them in parallel, and they already have been widely used in structural mechanics. Differently from problems arising in structural mechanics, energy functionals of total variation minimization problems are in general nonlinear, nonsmooth, and nonseparable. Hence, designing efficient domain decomposition methods for total variation minimization is a quite challenging issue. We describe various existing approaches on domain decomposition methods for total variation minimization in a unified view. We address how the direction of research on the subject has changed over the past few years, and suggest several interesting topics for further research.
Keywords
domain decomposition methods; total variation; mathematical imaging; parallel computation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. TOSELLI AND O. WIDLUND, Domain Decomposition Methods-Algorithms and Theory, vol. 34, Springer, Berlin, 2005.
2 A. QUARTERONI AND A. VALLI, Domain Decomposition Methods for Partial Differential Equations, Oxford University Press, New York, 1999.
3 J. XU, Iterative methods by space decomposition and subspace correction, SIAM Rev., 34 (1992), pp. 581-613.   DOI
4 C. R. DOHRMANN, A preconditioner for substructuring based on constrained energy minimization, SIAM J. Sci. Comput., 25 (2003), pp. 246-258.   DOI
5 J. MANDEL, Balancing domain decomposition, Commun. Numer. Methods Engrg., 9 (1993), pp. 233-241.   DOI
6 C. FARHAT, M. LESOINNE, AND K. PIERSON, A scalable dual-primal domain decomposition method, Numer. Linear Algebra Appl., 7 (2000), pp. 687-714.   DOI
7 C. FARHAT AND F.-X. ROUX, A method of finite element tearing and interconnecting and its parallel solution algorithm, Internat. J. Numer. Methods Engrg., 32 (1991), pp. 1205-1227.   DOI
8 C.-O. LEE AND E.-H. PARK, A dual iterative substructuring method with a penalty term, Numer. Math., 112 (2009), pp. 89-113.   DOI
9 L. I. RUDIN, S. OSHER, AND E. FATEMI, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259-268.   DOI
10 D. STRONG AND T. CHAN, Edge-preserving and scale-dependent properties of total variation regularization, Inverse Problems, 19 (2003), pp. S165-S187.   DOI
11 A. CHAMBOLLE, An algorithm for total variation minimization and applications, J. Math. Imaging Vision, 20 (2004), pp. 89-97.   DOI
12 Y. WANG, J. YANG, W. YIN, AND Y. ZHANG, A new alternating minimization algorithm for total variation image reconstruction, SIAM J. Imaging Sci., 1 (2008), pp. 248-272.   DOI
13 A. BECK AND M. TEBOULLE, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183-202.   DOI
14 T. GOLDSTEIN AND S. OSHER, The split Bregman method for L1-regularized problems, SIAM J. Imaging Sci., 2 (2009), pp. 323-343.   DOI
15 L. BADEA AND R. KRAUSE, One-and two-level Schwarz methods for variational inequalities of the second kind and their application to frictional contact, Numer. Math., 120 (2012), pp. 573-599.   DOI
16 A. CHAMBOLLE AND T. POCK, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vision, 40 (2011), pp. 120-145.   DOI
17 E. ESSER, X. ZHANG, AND T. F. CHAN, A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science, SIAM J. Imaging Sci., 3 (2010), pp. 1015-1046.   DOI
18 L. BADEA, Convergence rate of a Schwarz multilevel method for the constrained minimization of nonquadratic functionals, SIAM J. Numer. Anal., 44 (2006), pp. 449-477.   DOI
19 X.-C. TAI AND J. XU, Global and uniform convergence of subspace correction methods for some convex optimization problems, Math. Comp., 71 (2001), pp. 105-124.   DOI
20 M. FORNASIER, A. LANGER, AND C.-B. SCHONLIEB, A convergent overlapping domain decomposition method for total variation minimization, Numer. Math., 116 (2010), pp. 645-685.   DOI
21 M. FORNASIER AND C.-B. SCHONLIEB, Subspace correction methods for total variation and l1-minimization, SIAM J. Numer. Anal., 47 (2009), pp. 3397-3428.   DOI
22 M. HINTERMULLER AND A. LANGER, Subspace correction methods for a class of nonsmooth and nonadditive convex variational problems with mixed $L^1$/$L^2$ data-fidelity in image processing, SIAM J. Imaging Sci., 6 (2013), pp. 2134-2173.   DOI
23 Y. DUAN AND X.-C. TAI, Domain decomposition methods with graph cuts algorithms for total variation minimization, Adv. Comput. Math., 36 (2012), pp. 175-199.   DOI
24 H. CHANG, X.-C. TAI, L.-L. WANG, AND D. YANG, Convergence rate of overlapping domain decomposition methods for the Rudin-Osher-Fatemi model based on a dual formulation, SIAM J. Imaging Sci., 8 (2015), pp. 564-591.   DOI
25 A. LANGER, S. OSHER, AND C.-B. SCHONLIEB, Bregmanized domain decomposition for image restoration, J. Sci. Comput., 54 (2013), pp. 549-576.   DOI
26 C.-O. LEE, J. H. LEE, H. WOO, AND S. YUN, Block decomposition methods for total variation by primal-dual stitching, J. Sci. Comput., 68 (2016), pp. 273-302.   DOI
27 C.-O. LEE AND C. NAM, Primal domain decomposition methods for the total variation minimization, based on dual decomposition, SIAM J. Sci. Comput., 39 (2017), pp. B403-B423.   DOI
28 M. HINTERMULLER AND A. LANGER, Non-overlapping domain decomposition methods for dual total variation based image denoising, J. Sci. Comput., 62 (2015), pp. 456-481.   DOI
29 C.-O. LEE AND J. PARK, Fast nonoverlapping block Jacobi method for the dual Rudin-Osher-Fatemi model, SIAM J. Imaging Sci., 12 (2019), pp. 2009-2034.   DOI
30 C.-O. LEE, E.-H. PARK, AND J. PARK, A finite element approach for the dual Rudin-Osher-Fatemi model and its nonoverlapping domain decomposition methods, SIAM J. Sci. Comput., 41 (2019), pp. B205-B228.   DOI
31 C.-O. LEE AND J. PARK A finite element nonoverlapping domain decomposition method with Lagrange multipliers for the dual total variation minimizations, J. Sci. Comput., 81 (2019), pp. 2331-2355.   DOI
32 Y. DUAN, H. CHANG, AND X.-C. TAI, Convergent non-overlapping domain decomposition methods for variational image segmentation, J. Sci. Comput., 69 (2016), pp. 532-555.   DOI
33 C.-O. LEE, C. NAM, AND J. PARK, Domain decomposition methods using dual conversion for the total variation minimization with $L^1$ fidelity term, J. Sci. Comput., 78 (2019), pp. 951-970.   DOI
34 T. F. CHAN, S. ESEDOGLU, AND M. NIKOLOVA, Algorithms for finding global minimizers of image segmentation and denoising models, SIAM J. Appl. Math., 66 (2006), pp. 1632-1648.   DOI
35 T. F. CHAN AND L. A. VESE, Active contours without edges, IEEE Trans. Image Process., 10 (2001), pp. 266- 277.   DOI
36 J. PARK, An overlapping domain decomposition framework without dual formulation for variational imaging problems. arXiv:2002.10070 [math.NA], 2019. To appear in Adv. Comput. Math.
37 T. F. CHAN AND S. ESEDOGLU, Aspects of total variation regularized $L^1$ function approximation, SIAM J. Appl. Math., 65 (2005), pp. 1817-1837.   DOI
38 A. TIKHONOV, Solution of incorrectly formulated problems and the regularization method, Soviet Math. Dokl., 4 (1963), pp. 1035-1038.
39 G. AUBERT AND P. KORNPROBST, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, Springer, New York, 2006.
40 E. GIUSTI, Minimal Surfaces and Functions of Bounded Variation, Birkhauser, Boston, 1984.
41 X. FENG AND A. PROHL, Analysis of total variation flow and its finite element approximations, ESAIM Math. Model. Numer. Anal., 37 (2003), pp. 533-556.   DOI
42 A. CHAMBOLLE AND T. POCK, An introduction to continuous optimization for imaging, Acta Numer., 25 (2016), pp. 161-319.   DOI
43 I. EKELAND AND R. TEMAM, Convex Analysis and Variational Problems, vol. 28, SIAM, Philadelphia, 1999.
44 Y. MEYER, Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: the Fifteenth Dean Jacqueline B. Lewis Memorial Lectures, vol. 22, American Mathematical Society, Providence, 2001.
45 M. NIKOLOVA, A variational approach to remove outliers and impulse noise, J. Math. Imaging Vision, 20 (2004), pp. 99-120.   DOI
46 R. T. ROCKAFELLAR, Convex Analysis, Princeton University Press, New Jersey, 2015.
47 Y. DONG, M. HINTERMULLER, AND M. NERI, An efficient primal-dual method for $L^1$TV image restoration, SIAM J. Imaging Sci., 2 (2009), pp. 1168-1189.   DOI
48 K. KUNISCH AND M. HINTERMULLER, Total bounded variation regularization as a bilaterally constrained optimization problem, SIAM J. Appl. Math., 64 (2004), pp. 1311-1333.   DOI
49 L. BADEA, X.-C. TAI, AND J. WANG, Convergence rate analysis of a multiplicative Schwarz method for variational inequalities, SIAM J. Numer. Anal., 41 (2003), pp. 1052-1073.   DOI
50 X.-C. TAI, Rate of convergence for some constraint decomposition methods for nonlinear variational inequalities, Numer. Math., 93 (2003), pp. 755-786.   DOI
51 J. BOLTE, S. SABACH, AND M. TEBOULLE, Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program., Ser. A, 146 (2014), pp. 459-494.
52 A. CHAMBOLLE, AND T POCK, A remark on accelerated block coordinate descent for computing the proximity operators of a sum of convex functions, SMAI J. Comp. Math., 1 (2015), pp. 29-54.   DOI
53 R. SHEFI AND M. TEBOULLE, On the rate of convergence of the proximal alternating linearized minimization algorithm for convex problems, EURO J. Comput. Optim., 4 (2016), pp. 27-46.   DOI
54 A. LANGER AND F. GASPOZ, Overlapping domain decomposition methods for total variation denoising, SIAM J. Numer. Anal., 57 (2019), pp. 1411-1444.   DOI
55 S. BARTELS, Total variation minimization with finite elements: convergence and iterative solution, SIAM J. Numer. Anal., 50 (2012), pp. 1162-1180.   DOI
56 M. HERRMANN, R. HERZOG, S. SCHMIDT, J. VIDAL-NUNEZ, AND G. WACHSMUTH, Discrete total variation with finite elements and applications to imaging, J. Math. Imaging Vision, 61 (2019), pp. 411-431.   DOI
57 P.-A. RAVIART AND J.-M. THOMAS, A mixed finite element method for 2-nd order elliptic problems, in Mathematical Aspects of Finite Element Methods, Springer, 1977, pp. 292-315.
58 I. GOODFELLOW, Y. BENGIO, AND A. COURVILLE, Deep Learning, MIT Press, Cambridge, 2016.
59 S. J. WRIGHT, Coordinate descent algorithms, Math. Program., 151 (2015), pp. 3-34.   DOI
60 P. L. COMBETTES AND V. R. WAJS, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.   DOI
61 Y. E. NESTEROV, A method for solving the convex programming problem with convergence rate O(1=$k^2$), Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543-547.
62 B. HE, Y. YOU, AND X. YUAN, On the convergence of primal-dual hybrid gradient algorithm, SIAM J. Imaging Sci., 7 (2014), pp. 2526-2537.   DOI
63 B. HE AND X. YUAN, Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective, SIAM J. Imaging Sci., 5 (2012), pp. 119-149.   DOI
64 R. T. ROCKAFELLAR, Monotone operators and the proximal point algorithm, SIAM J. Control Optim., 14 (1976), pp. 877-898.   DOI
65 A. CHAMBOLLE, AND T POCK, On the ergodic convergence rates of a first-order primal-dual algorithm, Math. Program., Ser. A, 159 (2016), pp. 253-287.   DOI