Browse > Article
http://dx.doi.org/10.7582/GGE.2020.23.3.00157

Hyperparameter Search for Facies Classification with Bayesian Optimization  

Choi, Yonguk (Dept. Energy & Resources Engineering, Chonnam National University)
Yoon, Daeung (Dept. Energy & Resources Engineering, Chonnam National University)
Choi, Junhwan (Dept. of Earth Resources and Environmental Engineering, Hanyang University)
Byun, Joongmoo (Dept. of Earth Resources and Environmental Engineering, Hanyang University)
Publication Information
Geophysics and Geophysical Exploration / v.23, no.3, 2020 , pp. 157-167 More about this Journal
Abstract
With the recent advancement of computer hardware and the contribution of open source libraries to facilitate access to artificial intelligence technology, the use of machine learning (ML) and deep learning (DL) technologies in various fields of exploration geophysics has increased. In addition, ML researchers have developed complex algorithms to improve the inference accuracy of various tasks such as image, video, voice, and natural language processing, and now they are expanding their interests into the field of automatic machine learning (AutoML). AutoML can be divided into three areas: feature engineering, architecture search, and hyperparameter search. Among them, this paper focuses on hyperparamter search with Bayesian optimization, and applies it to the problem of facies classification using seismic data and well logs. The effectiveness of the Bayesian optimization technique has been demonstrated using Vincent field data by comparing with the results of the random search technique.
Keywords
facies classification; Bayesian optimization; random search; autoML; k-fold cross validation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Yoon, D., Yeeh, Z., and Byun, J., 2020, Seismic Data Reconstruction Using Deep Bidirectional Long Short-Term Memory with Skip Connections, IEEE Geosci. Remote Sens. Lett., 1-5, doi: 10.1109/LGRS.2020.2993847.   DOI
2 Yoon, D., Kim, S., Kim, J., Park, G., Park, H., Byun, J., Suh, J., Lee, C., Jang, I., Jo, S., and Choi, Y., 2018, Introduction of Resource Engineering with Machine Learning, CIR press, 377-396 (in Korean).
3 Zoph, B., and Le, Q. V., 2016, Neural Architecture Search with Reinforcement Learning, arXiv preprint arXiv:1611.01578.
4 Araya-Polo, M., Jennings, J., Adler, A., and Dahlke, T., 2018, Deep-learning tomography, Lead Edge, 37(1), 58-66, doi:10.1190/tle37010058.1.   DOI
5 Baldwin, J. L., Bateman, R. M., and Wheatley, C. L., 1990, Application of a neural network to the problem of mineral identification from well logs, The Log Analyst, 31(05), 279-293.
6 Bergstra, J. S., Bardenet, R., Bengio, Y., and Kegl, B., 2011, Algorithms for hyper-parameter optimization, Adv. Neural. Inf. Process. Syst., 2546-2554.
7 Choi, J., Yoon, D., Lee, S., and Byun, J., 2019, Petrofacies characterization using best combination of multiple elastic properties, J. Pet. Sci. Eng., 181, doi: 10.1016/j.petrol.2019.06.025.
8 Choi, J., Kim, B., Kim, S., and Byun, J., 2017, Probabilistic facies analysis using 3D crossplot of stochastic forwardmodeling results, 87th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, 3077-3081, doi: 10.1190/segam2017-17790996.1.
9 Delfiner, P., Peyret, O., and Serra, O., 1987, Automatic determination of lithology from well logs, SPE Formation Evaluation, 2(03), 303-310, doi: 10.2118/13290-PA.   DOI
10 Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J., and Greenspan, H., 2018, Synthetic data augmentation using GAN for improved liver lesion classification, 2018 IEEE 15th Int. Symp. Biomed. Imaging, 289-293, doi: 10.1109/ISBI.2018.8363576.
11 Jones, D. R., 2001, A taxonomy of global optimization methods based on response surfaces, Journal of Global Optimization, 21(4), 345-383.   DOI
12 Kanter, J. M., and Veeramachaneni, K., 2015, Deep feature synthesis: Towards automating data science endeavors, 2015 IEEE Int. Conf. Data. Sci. Adv. Anal., 1-10, doi: 10.1109/DSAA.2015.7344858.
13 Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T. Y., 2017, LightGBM: A Highly Efficient Gradient Boosting Decision Tree, Adv. Neural Infor. Process. Syst., 30, 3149-3157.
14 Klein, A., Falkner, S., Bartels, S., Hennig, P., and Hutter, F., 2017, Fast Bayesian optimization of machine learning hyperparameters on large datasets, International Conference on Artificial Intelligence and Statistics (AISTATS 2017), 528-536, doi: 10.1214/17-EJS1335SI.
15 Lee, S., Choi, J., Yoon, D., and Byun, J., 2018, Automatic labeling strategy in semi-supervised seismic facies classification by integrating well logs and seismic data, 88th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, 14-19, doi: 10.1190/segam2018-2998604.1.
16 Li, H., Yang, W., and Yong, X., 2018, Deep learning for groundroll noise attenuation, 88th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, 14-19, doi: 10.1190/segam2018-2981295.1.
17 Liu, H., Simonyan, K., and Yang, Y., 2018, Darts: Differentiable architecture search, arXiv preprint arXiv: 1806.09055.
18 Mockus, J., 2012, Bayesian approach to global optimization: theory and applications, Springer Science & Business Media, 37.
19 Oh, S., Noh, K., Yoon, D., Seol, S. J., and Byun, J., 2018, Salt delineation from electromagnetic data using convolutional neural networks, IEEE Geosci. Remote Sens. Lett., 16(4), 519-523, doi: 10.1109/LGRS.2018.2877155.   DOI
20 Nguyen, H. P., Liu, J., and Zio, E., 2020, A long-term prediction approach based on long short-term memory neural networks with automatic parameter optimization by Treestructured Parzen Estimator and applied to time-series data of NPP steam generators. Appl. Soft Comput., 89, 106116, doi:10.1016/j.asoc.2020.106116.   DOI
21 Park, J., Yoon, D., Seol, S. J., and Byun, J., 2019, Reconstruction of seismic field data with convolutional U-Net considering the optimal training input data, 89th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, doi: 10.1190/segam2019-3216017.1.
22 Rashmi, K. V., and Gilad-Bachrach, R., 2015, DART: Dropouts meet Multiple Additive Regression Trees, Artificial Intelligence and Statistics, 489-497.
23 Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., and Talwalkar, A., 2018, Hyperband: A novel bandit-based approach to hyperparameter optimization, J. Mach. Learn. Res., 18, 1-52.
24 Snoek, J., Larochelle, H., and Adams, R. P., 2012, Practical Bayesian optimization of machine learning algorithms, Adv. Neural Infor. Process. Syst., 2951-2959.
25 Wolpert, D. H., and Macready, W. G., 1997, No free lunch theorems for optimization, IEEE Trans. Evol. Comput., 1(1), 67-82.   DOI
26 Wrona, T., Pan, I., Gawthorpe, R. L., and Fossen, H., 2018, Seismic facies analysis using machine learning, Geophysics, 83(5), O83-O95.   DOI