Evaluation of a Solar Flare Forecast Model with Cost/Loss Ratio

There are probabilistic forecast models for solar flare occurrence, which can be evaluated by various skill scores (e.g. accuracy, critical success index, heidek skill score, true skill score). Since these skill scores assume that two types of forecast errors (i.e. false alarm and miss) are equal or constant, which does not take into account different situations of users, they may be unrealistic. In this study, we make an evaluation of a probabilistic flare forecast model (Lee et al. 2012) which use sunspot groups and its area changes as a proxy of flux emergence. We calculate daily solar flare probabilities from 1996 to 2014 using this model. Overall frequencies are 61.08% (C), 22.83% (M), and 5.44% (X). The maximum probabilities computed by the model are 99.9% (C), 89.39% (M), and 25.45% (X), respectively. The skill scores are computed through contingency tables as a function of forecast probability, which corresponds to the maximum skill score depending on flare class and type of a skill score. For the critical success index widely used, the probability threshold values for contingency tables are 25% (C), 20% (M), and 4% (X). We use a value score with cost/loss ratio, relative importance between the two types of forecast errors. We find that the forecast model has an effective range of cost/loss ratio for each class flare: 0.15-0.83(C), 0.11-0.51(M), and 0.04-0.17(X), also depending on a lifetime of satellite. We expect that this study would provide a guideline to determine the probability threshold for space weather forecast.