ROC and CAP analysis as tools for evaluating the quality of financial models diagnosis
У статті досліджено інструменти оцінювання якості моделей фінансової діагностики та сутність характеристики понять CAP і ROC. Вказані інструменти дозволяють здійснити необхідний аналіз з єдиним показником - коефіцієнтом точності. Показано, що коефіцієнт точності це просто лінійне перетворення площини під ROC-кривою. Розкрито метод обчислення довірчих інтервалів для області під ROC-кривою, а також порядок результатів аналізу порівняння площини під ROC-кривою за двома моделями фінансової діагностики однакової вибірки.
This article explores the tools to assess the quality of financial models and diagnostic characteristics of the nature of concepts CAP and ROC. These tools allow you to carry out the necessary analysis with a single parameter - the coefficient accuracy. It is shown that the accuracy rate is a linear transformation of the plane under the ROC-curve. Reveals the method of calculating confidence intervals for the area under the ROC-curve and the procedure of the analysis comparing area under ROC-curve for the two models of financial diagnostics same sample.
The validation techniques currently used in practice are the concepts of Cumulative Accuracy Profiles and Accuracy Ratios, which deliver a single number to judge upon the quality of internal rating models. However, the reliability of such judgements is questionable if no confidence interval can be stated in addition to the Accuracy Ratio.
By using the concept of Receiver Operating Characteristics are derived in an analytical and consequently simple way. Besides, a relationship between this area and the Accuracy Ratio is proven, which demonstrates that the concepts derived for Receiver Operating Characteristic Curves can be applied to Cumulative Accuracy Profiles, too. Hence different rating models can be compared by using confidence intervals instead of single numbers, which allows a sound decision-making about the superiority of one model over another.
[Download PDF version]
Keywords: CAP-curve, ROC-curve, confidence intervals, CAP – Cumulated Accuracy Profile, ROC – Receiver Operating Characteristic, default, non-default. |
THEORY
Yevtushenko Maryna
postgraduate of Department of Corporate Finance and Controlling SHEI «Kyiv National Economic University named after Vadym Hetman»
|
