A FREE SEMINAR ON DEFAULT PREDICTORS IN AGRICULTURAL CREDIT SCORING: EVIDENCE FROM BANK OF AGRICULTURE DATA SOKOTO
A FREE SEMINAR ON DEFAULT PREDICTORS IN AGRICULTURAL CREDIT SCORING: EVIDENCE FROM BANK OF AGRICULTURE DATA SOKOTO
Abstract
Decisions of Whether to grant Credit or not to the applicants are extremely important for any financial institution since it stimulates huge financial losses generated from defaulters. So many banks use Judgmental decisions, i.e. credit analysts go through every application separately and other banks develop credit Scoring Systems or combination of both. Credit Scoring System uses many types Predictive statistical models. But in recent times, researchers have begun looking for alternative algorithms with better predictive accuracy regarding classification. Neural network can be suitable alternative. It is very clear from the classification outcomes of this research that neural network compares well with other statistical techniques. It gives slight better results than k-Nearest Neighbor Classifier, Logistic regression and Discriminant Analysis and performs slight poorer than Classification and Regression Tree (CART). However, it is noteworthy that a “Bad accepted” generates much high costs than a “Good rejected” and neural network acquires less amount of “Bad accepted” than the other predictive statistical models. So, neural network achieves less cost of misclassification for the dataset use in the research. Furthermore, in the final section of this research, an optimization algorithm (Genetic Algorithm) is proposed in order to obtain better classification accuracy through the configurations of the neural network architecture. On the contrary, it is important to note that the success of the predictive model largely depends on the predictor variables selection to be used as inputs of the model.
Keywords: predictive analysis, credit scoring, pattern recognition, Discriminant analysis, Logistic regression
1.1 Introduction
Predictive analysis encompasses a variety of statistical techniques from modeling, machine learning, data mining and game theory that analyzes current and historical facts to make predictions about future events.
Predictive analytics is used in actuarial science, financial services, insurance, telecommunications, retail, travel, health care, pharmaceuticals and other fields. One of the most well-known applications is credit scoring, which is used throughout financial services. Scoring models process a customers’ credit history, loan application , customers’ data, etc, in order to rank-order individuals by their likelihood of making future credit payments on time.
A credit score is a numerical expression based on a statistical analysis of a person’s credit files, to represent the creditworthiness of that person. A credit score is primarily based on credit report information typically sourced from credit bureaus. The credit scoring models are developed to categorize applicants as either accepted (good) or rejected (bad) credits with respect to their characteristics such as age, income and marital condition. Creditors accept the application provided that it is expected to repay the financial obligation, and vice versa. Creditors can construct the classification rules based on the data of the previous applicants.
The way credit scoring works is simple theoretically. According to the author Jentzsch (2007), the basic working procedure can be explained in the following way: the dependent variable (Y) represents credit risk (the probability of repayment). The independent variables (predictor variables or Xi) are used to explain the dependent variable. The list and the value of the independent variables are extracted from the “Application Form” generally, or sometimes from the credit report. Then the performance of a specific customer is decided based on the performance of the similar types of customers by using credit scoring system, credit scoring system awards points on every possible factor to calculate the probability of repayment. After these points are awarded, credit score comes up. Normally, the higher the achieved points, the lower the risk is.
1.2 Statement of the problem
Credit decisions are very crucial for any type of financial institution because it can determine huge financial losses provoked from defaulters. A number of banks use judgmental decisions, means credit analysts go through every application separately and other banks use credit scoring system or combination of both. This therefore, primarily depends on the type of the product. In the case of small amount of credits like agricultural credits given to farmers (especially in credit cards), banks try to pursue automated system (credit scoring). The system provides decision based on the pattern recognition that is known from the previous customer’s database. Many types of algorithms are used in credit scoring. But recently, professionals started looking for more accurate decisions alternatives, for example neural network. But there are few guidelines on this topic and its application in credit decisions.
Justification of the study
Classification plays a crucial role in business planning, especially in Credit Scoring. But, conventional methods suffer from several limitations, for example, many conventional methods assume linear relationship among the variables although there may be non-linear relationship in reality and non-linear models like the multiple regression models require model selection which is based on trial and error process (Leondes 2005). So, using neural network to forecast can be a very good alternative. The higher predictive power of neural network application can be attributed to their ability of reproducing human intelligence (Chaffey et al. 2009) and to their powerful pattern recognition capability (Zhang 2003).
Aim and Objectives
The aim is to compare the predictive ability of Artificial Neural network with the conventional statistical models for credit scoring. This aim can be achieved through the following objectives:
To build a linear discriminant model capable of identifying an applicant credit status
To build a logistic regression model capable of identifying an applicant credit status
To build a k-nearest Neighbor Classifier (model) capable of identifying an applicant credit status
To build a classification and regression tree (CART) model capable of identifying an applicant credit status
To build an artificial neural network model capable of identifying an applicant credit status
Provide the concepts and theories that should be reviewed and considered during the selection of the predictor variables for the development of any type of credit scoring system.
2.0Literature review
Shi et al., (2002) utilized Multiple Criteria Linear Programming (SAS Software) to classify credit card database in to two groups (good and bad) and three groups (good, normal and bad). Onget al., (2005) applied Genetic Programming to classify good and bad customers. Wang et al., (2005) used Fuzzy Support Vector Machine (Fuzzy SVM) on credit database; based on the assumption that one customer can’t be absolutely good or bad. And the study also included other algorithms like linear regression, Logistic Regression and BP- network. Lee et al., (2006) employed Classification and Regression Tree (CART) and Multivariate Adaptive Regression Splines (MARS) and identified better performance in comparison with the Discriminant Analysis, Logistic Regression, Neural Network and Support Vector Machine. Sexton et al., (2006) used GA-based algorithm, called Neural Network Simultaneous Optimization Algorithm on a credit dataset. The performance was satisfactory and the model was able to identify significant variables among the other existing variables.
Min et al. (2006) proposed methods for improving SVM performance in two aspects: feature subset selection and parameter optimization. Gestelet al., (2006) used LS-SVM for credit rating of banks, and compared the results with ordinary least squares, LR and multilayer perceptron (MLP). The results show the LS-SVM classifier’s accuracy is better than the three methods. Huang et al. (2007) designed hybrid SVM-based credit scoring models for assessing the credit scores of applicants. Zhang et al., (2008) developed a hybrid credit scoring model applying genetic programming and SVM. The accuracy of their hybrid model was better when compared with SVM, genetic programming, decision tree, LR, and back propagation neural networks.
Thomas, Edelman et al. (2004) states that there should be a clear, rational, explanatory relationship between each variable and credit performance
Siddiqi (2005) pointed to use variable selection algorithms (Chi Square, R Square) before grouping the characteristics
Abrahams and Zhang (2009) drew attention to the importance of the “Sequence” of the independent variables selection
Fabozziet al., (2006) attempted to summarize the variables that have been found very helpful for the prediction of the future credit performance based on a large sample of customers
Anderloniet al. (2006) focused on statistical analysis to select the best set of variables in predicting the credit performance
However, this research work is different from the reviewed literatures in the following ways:
That the predictor variables used are different from those used by the reviewed literatures.
That none of the reviewed literature built credit scoring models for Nigerian Banks. Hence the need to build the credit scoring models based on the predictor variables that are available in the application form of the Nigerian Banking system.
That the Statistical models used are linear discriminant analysis, Logistic regression, k- nearest neighbor analysis and classification and regression tree analysis.
That the artificial neural network is also used.
3.0 MATERIALS AND METHODS
3.1 DATA USED
A real world credit dataset is used in this research. The dataset is extracted from the application forms of Bank of Agriculture, Sokoto(BOA). Thedataset is referred to as “Credit Dataset”. After preparing or cleaning the dataset, it is used in the subsequent sections for conducting the analysis with Discriminant Analysis, k- Nearest Neighbor Classifier, Classification and Regression tree (CART), Logistic Regression and Neural Network.
3.2 METHOD
Let us assume that the population of loans consists of two groups or classes G and B that denote loans that (after being granted) will turn good and bad in the future, respectively.
Let probability that a randomly chosen customer belongs to group G and B be denoted as and. Let x be a vector of independent variables used in the process of deciding whether an applicant belongs to group G or B. Let the probability that an applicant with measurement vector x belongs to group G is p(G|x), and that of B is p(B|x). Let the probability p(x|G) indicate that a good applicant has measurement vector x. Similarly, for a bad applicant the probability is p(x|B). The task is to estimate probabilities from the set of given data about applicants which turn out to be good or bad and to find a rule for how to partition the space X of all measurement vectors in to the two groups andbased on these probabilities so that inwould be the measurement vectors of applicants who turn out to be good and vice versa.However, it is usually not possible to find perfect classification as it may happen that the same vector is given by two applicants where one is good and the other is bad. Therefore it is necessary to find a rule that will minimize the costs of the agency providing credit connected with the misclassification of applicants. Let us denote as the costs connected with misclassifying a good applicant as bad and as the costs connected with misclassifying a bad applicant as good (>). If applicants with x are assigned to class G the expected costs are p(B|x) and the expected loss for the whole sample is
wherep(x) is a probability that the measurement vector is equal to x. This is minimized when into group G such applicants are assigned who have their group of measurement vectors
Without loss of generality we can normalize misclassification costs to += 1. In that case the rule for classification is to assign an applicant with x to class G if p(G|x)and otherwise to class B.
3.2.1 Linear Discriminant analysis
The aim of Linear Discriminant Analysis (LDA) is to classify a heterogeneous population in to homogenous subsets and further the decision process on these subsets. We can assume that for each applicant there are a specific number of explanatory variables available. The idea is to look for such a linear combination of explanatory variables which separates most subsets from each other. In a simple case of two subsets, the goal is to find the linear combination of explanatory variables, which leaves the maximum distance between means of the two subsets.
In a general case we consider the distributions p(x|G) and p(x|B) which are multivariate normal distributions with the common variance. Then equation (2) reduces to
Where .
3.2.2 Logit Analysis
The distribution of the credit information data is usually non-normal, and this fact may theoretically pose a problem when conducting an LDA. One way to overcome the problems with non-normality of data is to use an extension of the LDA model that allows for some parametric distribution. In this case a suitable extension is a generalized linear model known as logit model. Given a vector of application characteristics x, the probability of default p is related to vector x by the relationship
3.2.3 k-nearest Neighbor Classifier
The k-nearest Neighbor Classifier serves as an example of the non-parametric statistical approach. This techniques assesses the similarities between the pattern identified in the training set and input pattern. One chooses a metric on the space of applicants and takes k-nearest neighbor (k-NN) of the input pattern that is nearest in some metric sense. A new applicant will be classified in the class to which majority of the neighbors belong (in the case when the costs of misclassification are equal) or according to rule explained by equation (1). This means that this method estimates the p(G|x) (or p(B|x)) probability by the proportion of G (or B) class points among the k-nearest neighbors to the point x to be classified.
A commonly used metric is the standard Euclidean norm given by
wherexand y are measurement vectors
However, when the variables are in different units or categorized, it is necessary to use some appropriate standardization of variables as well as to select some data-dependent version of the Euclidean metric such as:
WhereA is an × n matrix with n number of variables. As matrix A can depend on x we can define two types of metrics according to how A is selected: local metrics are those where A depends on x; global metrics are those where A is independent of x.
The choice of the number of nearest neighbors chosen (k) determines the bias/variance trade-off in the estimator. The k has to be much smaller than the smallest class. A simulation study by Enas and Choi (1986) suggested that k ≈ n2/8 or n3/8 is reasonable
3.2.4 Classification and Regression Trees (CART)
Classification and Regression Trees (CART) is a nonparametric method that is due to Breiman, Friedman, Olshen and Stone (1984). It is a flexible and potent technique; however, it is used in banking practice chiefly only as a supporting tool to accompany the parametric estimation methods described earlier. It serves, for example, in the process to select repressors’ or characteristics with the highest explanatory power. The CART method employs binary trees and classifies a dataset into a finite number of classes. It was originally developed as an instrument for dealing with binary responses and as such it is suitable for use in credit scoring where the default and non-default responses are contained in data. Similarly as with the methodologies reviewed earlier, we make the assumption of having a training set of measurement vectors
.
Along with information whether an individual j defaulted, or not (hence,is coded 1 or
The CART tree consists of several layers of nodes: the first layer consists of a root node; the last layer consists of leaf nodes. Because it is a binary tree, each node (except the leaves) is connected to two nodes in the next layer. The root node contains the entire training set; the other nodes contain subsets of this set. At each node, this subset is divided into 2 disjoint groups based on one specific characteristic from the measurement vector. The classification process is a recursive procedure that starts at the root node and at each further node (with exception of the leaves) one single characteristic and a splitting rule (or constant c) is selected. First, the best split is found for each characteristic. Then, among these characteristics the one with the best split is chosen.
3.2.5 Neural Networks
A neural network (NNW) is a mathematical representation inspired by the human brain and its ability to adapt on the basis of the inflow of new information. Mathematically, NNW is a non-linear optimization tool. Many various types of NNW have been specified in the literature. The NNW design called multilayer perceptron (MLP) is especially suitable for classification and is widely used in practice. The network consists of one input layer, one or more hidden layers and one output layer, each consisting of several neurons. Each neuron processes its inputs and generates one output value that is transmitted to the neurons in the subsequent layer. Each neuron in the input layer (indexed i = 1,…,n) delivers the value of one predictor (or the characteristics) from vector x. When considering default/non-default discrimination, one output neuron is satisfactory.
In each layer, the signal propagation is accomplished as follows. First, a weighted sum of inputs is calculated at each neuron: the output value of each neuron in the proceeding network layer times the respective weight of the connection with that neuron. A transfer function g(x) is then applied to this weighted sum to determine the neuron's output value. So, each neuron in the hidden layer (indexed j = 1,…, q) produces the so-called activation
The neurons in the output layer (indexed k = 1,…, m) behave in a manner similar to the neurons of the hidden layer to produce the output of the network:
Where andare weights
The most popular algorithm for training multilayer perceptrons is the back-propagation algorithm. As the name suggests, the error computed from the output layer is back-propagated through the network, and the weights are modified according to their contribution to the error function. Essentially, back-propagation performs a local gradient search, and hence its implementation; although not computationally demanding, it does not guarantee reaching a global minimum. For each individual, weights are modified in such a way that the error computed from the output layer is minimized.
The BP learning involves three stages: the feed-forward of the input training pattern, the calculation of the associated error, and the adjustment of the weights. After the network reaches a satisfactory level of performance, it learns the relationships between independent variables (applicant’s attributes) and dependent variable (credit class). The trained BP network can then be adopted as a scoring model to classify the credit as either good or bad by inserting the values of applicant’s attributes.
Figure 1: The BP-based credit scoring architecture
3.2.6 Wilks’ Lambda Test for significance of canonical correlation
Hypothesis canonical correlation:
: There is no relationship between the two sets of variables
: There is linear relationship between two set of variables
Test statistic:
H is the variance due to linear relationship
W+H is the total variance
Decision rule: Reject if P<0.05 otherwise accept at the 5% level of significance
3.2.7 Chi-square Test
Hypothesis for Chi-square Test:
: The two variables are independent
: The two variables are not independent
Test statistic:
Decision Rule:Reject if P<0.05 otherwise accept at the 5% level of significance.
3.2.8 Omnibus Chi-square Test
The omnibus Chi-square test is a log-likelihood ratio test for investigating the model coefficients in logistic regression. The test procedures are as follows:
Hypothesis for Omnibus Chi-square Test:
: The model coefficients are not statistically significant
: The model coefficients are statistically significant
Test statistic:
Decision Rule:Reject if p<0.05 otherwise acceptat the 5% level of significance i.e. significance of the logistic model.
3.2.9 Box M Test for the Equality of Covariance Matrices
Hypothesis for Box’s M Test:
: The two covariance matrices are equal
: The two covariance matrices are not equal
Test Statistic:
Decision Rule: Reject if P<0.05 otherwise acceptat the 5% level of significance.
3.2.10 Wald Test
The Wald test is used to test the statistical significance of each coefficient in the logistic model. A Wald test calculates a statistic which is:
This value is squared which yields a chi-square distribution and is used as a Wald test statistic.
Decision rule: Reject (the null hypothesis that the coefficient is equal to zero) when p-value of that coefficient is less than level of significance.
4.0. RESULTS AND DISCUSSIONS
4.1. Discriminant Analysis
4.1.1 Model Measurement
A discriminant model is identified as “Useful” if there is at least 25% more improvement achievable over the by chance Accuracy rate. Here, “By Chance Accuracy Rate” is 51% and 25% increase of this value equals to 63% and the cross validated accuracy rate is 95%.Hence, cross validated accuracy rate can be used to discriminate better. Moreover, Wilks' lambda is a measure of the usefulness of the model.
Table 1: SPSS Output: Model Test :Wilks' Lambda
Test of Function(s)
Wilks' Lambda
Chi-square
Df
Sig.
1
.276
377.437
10
.000
The Wilk’s Lambda Probability of 0.000<0.05, means that predictors significantly discriminate the groups. This provides the proportion of total variability not explained, i.e. 27.6% unexplained.
Table 2 Canonical correlation.
Function
Eigenvalue
% of Variance
Cumulative %
Canonical Correlation
1
2.626a
100.0
100.0
.851
A canonical correlation of 0.851 suggests that the model explains 72.42% of the variation in the grouping variable, i.e. whether an applicant is a creditworthy or not.
4.1.2. Importance of Independent Variables:
Table 3: Discriminant Models
APPLICANT ATTRIBUTES
FUNCTIONS
STRUCTURE MATRIX
STANDARDIZED DISCRIMINANT COEFFICIENTS
CANONICAL DISCRIMINANT COEFFICIENTS
AGE
0.046
0.261
0.023
SEX
0.219
0.208
0.516
MARITAL STATUS
0.053
0.007
0.022
JOB
0.819
0.957
1.273
PURPOSE
-0.038
-0.045
-0.038
CREDIT AMOUNT
-0.007
-0.280
0.000
ANNUAL EARNINGS
0.010
-0.138
0.000
REPAYMENT PLAN
0.137
-0.074
-0.228
APPLICATION TYPE
-0.355
-0.438
-1.202
APPLICATION PERIOD
0.109
0.100
0.247
CONSTANT
-2.438
The structure matrix shows the correlations of each variable with each discriminant function. Generally, just like factor loadings, 0.30 is seen as the cut-off between important and less important variables. The interpretation of the standardized discriminant function coefficients is like that in multiple regressions. The canonical discriminant function coefficients (unstandardized coefficients) are used to create the discriminant function (equation). It operates just like regression equation. There are specific characteristics determined by the discriminant model for the two groups (creditworthy applicants and non-creditworthy applicants). For example, on an average, a good customer is a junior civil servant and applying as an individual.
Table 4: SPSS Output: Group Statistics: Only the Good Group:
Status of the Credit Applicant
Mean
Std. Deviation
Valid N (list wise)
Unweight
Weighted
Good
Application type
1.0366
.18832
164
164.000
Job of the applicant
3.0488
.95805
164
164.000
Characteristics of the Creditworthy Applicants
On the other hand, the bad group holds some certain characteristics in contradictory with the good group. For example, on an average, a non-creditworthy applicant is self-reliant and is a group application.
Table 5: SPSS Output: Group Statistics: Only the Bad Group:
Status of the Credit Applicant
Mean
Std. Deviation
Valid N (listwise)
Unweighted
Weighted
Bad
Application type
1.4559
.49989
136
136.000
Job of the applicant
1.0515
.37158
136
136.000
Characteristics of the Non-creditworthy Applicant
4.2. Logistic Regression:
4.2.1 Measurement of Model Performance: The significance of the model can be checked in two ways. First one is the significance test.
Table 6: SPSS Output: Model Test: Omnibus Tests of Model Coefficients
Chi-square
Df
Sig.
Step 1
Step
310.784
10
.000
Block
310.784
10
.000
Model
310.784
10
.000
Checking Usefulness of the Derived Model
In this analysis, the probability of the model chi-square (310.784) is 0.000, less than the level of significance of .05. The null hypothesis that there is no difference between the model with only a constant and the model with independent variables is rejected. The second way of confirming the usefulness of the model is to compare the classification accuracy rate. The independent variables can be characterized as useful predictors distinguishing between the two groups of the dependent variable if the classification accuracy rate is substantially higher than the accuracy attainable by chance alone.
Table 7: SPSS Output At Step 0: Classification Table:
Observed
Predicted
Applicant status
Percentage Correct
Good
Bad
Step 0
Applicant status
Good
164
0
100.0
Bad
136
0
.0
Overall Percentage
54.7
a. Constant is included in the model.
b. The cut value is .500
Checking Usefulness of the Derived Model
Table 8: SPSS Output At Step 1: Classification Table:
Observed
Predicted
Applicant status
Percentage Correct
Good
Bad
Step 1
Applicant status
Good
156
8
95.1
Bad
4
132
97.1
Overall Percentage
96.0
a. The cut value is .500
Checking Usefulness of the Derived Model
4.2.2 Importance of Independent Variables: The significance test is the statistical evidence of the presence of a relationship between the dependent variable and each of the independent variables. The significance test is the Wald Statistic. Here, the null hypothesis is that the b coefficient for the particular independent variable is equal to zero
Table 9: SPSS Output: Significant Variables by logistic model
B
S.E.
Wald
df
Sig.
Exp(B)
Step 1a
Age
-.007
.027
.074
1
.786
.993
Sex
-.883
.700
1.591
1
.207
.414
Marital Status
-.279
1.114
.063
1
.802
.756
Job
-3.638
.571
40.571
1
.000*
.026
Purpose
.031
.233
.018
1
.895
1.031
Credit Amount
.000
.000
3.006
1
.083
1.000
Annual earnings
.000
.000
.813
1
.367
1.000
Repayment plan
.882
.790
1.245
1
.264
2.416
Application type
2.469
.907
7.407
1
.006*
11.805
Application period
-.738
.727
1.029
1
.310
.478
Constant
3.804
3.045
1.561
1
.211
44.894
.
Reject the null hypothesis if the probability of the Wald statistic is less than or equal to the level of significance of .05
The individual coefficient represents the change in the odds of the modeled category associated with a one-unit change in the independent variable. Here, in this analysis, the modeled group is the “Bad Group” because of having the highest numerical code of 2. Individual coefficients are expressed in log units and are not directly interpretable
.
4.3. k-Nearest Neighbor Analysis:
4.3.1. Model measurement: The accuracy of k-NN model depends largely on the choice of the k (the number of the nearest neighbor to examine)
Table 10: SPSS Output: Error Summary (k=8, training sample=211, testing sample=89).
Partition
Percentage incorrectly classified cases
Training
4.74%
Holdout
1.13%
4.3.2Importance of Independent Variable
Figure 2: SPSS Output: Significant Variables
4.4 Classification and Regression Tree (CART) Analysis
Figure 3: SPSS Output: Classification Tree
Significant Variable Identified By the Classification Tree Model
4.4. Artificial Neural Network:
4.4.1 Measurement of Model Performance:
The following model summary table displays information about the results of the neural network training. Here, cross entropy error is displayed because the output layer uses the softmax activation function. This is the error function that the network tries to minimize during training.Moreover, the percentage of incorrect prediction is equivalent to 3.8% in the training samples. So, percentage of correct prediction is nearer to 96.2% that is quite high.If any dependent variable has scale measurement level, then the average overall relative error (relative to the mean model) is displayed. On the other hand, if the defined dependent variables are categorical, then the average percentage of incorrect predictions is displayed
Table 11: SPSS Output: Model Summary:
Training
Cross Entropy Error
31.851
Percent Incorrect Predictions
3.8%
Stopping Rule Used
1 consecutive step(s) with no decrease in errora
Training Time
0:00:00.375
Testing
Cross Entropy Error
6.795
Percent Incorrect Predictions
1.1%
Dependent Variable: Status of the Credit Applicant
a. Error computations are based on the testing sample.
Checking Usefulness of the derived Model
4.4.2 Importance of Independent Variables:
The following table performs an analysis, which computes the importance and the normalized importance of each predictor in determining the neural network.The analysis is based on the training and testing samples. The importance of an independent variable is a measure of how much the network’s model-predicted value changes for different values of the independent variable. Moreover, the normalized importance is simply the importance values divided by the largest importance values and expressed as percentages
Table 12: SPSS Output: Independent Variable Importance:
Importance
Normalized Importance
Attribute 2
.084
26.8%
Attribute3
.011
3.6%
Attribute 4
.314
100.0%
Attribute 5
.061
19.5%
Attribute 8
.035
11.1%
Attribute 9
.117
37.1%
Attribute 10
.054
17.2%
Attribute 6
.135
42.9%
Attribute 1
.087
27.6%
Attribute 7
.102
32.5%
Important Variables Identified By the Neural Network Model
4.5 Comparison of the Model’s Predictive Ability
4.5.1 Discriminant Analysis:
Table 13: SPSS Output: Classification Results:
Attribute 11
Predicted Group Membership
Total
1.00
2.00
Original
Count
1.00
156
8
164
2.00
4
132
136
%
1.00
95.1
4.9
100.0
2.00
2.9
97.1
100.0
Cross-validateda
Count
1.00
155
9
164
2.00
6
130
136
%
1.00
94.5
5.5
100.0
2.00
4.4
95.6
100.0
Predictive Ability of the Discriminant Model
The discriminant model is able to classify 156 good applicants as “Good Group” out of 164 good applicants. Thus, it holds 95% classification accuracy for the good group. On the other hand, the same discriminant model is able to classify 132 bad applicants as “Bad Group” out of 136 bad applicants. Thus, it holds 97.1% classification accuracy for the bad group. Thus, the model is able to generate 96.0%classification accuracy in combined groups.
4.5.2 Logistic Regression:
Table 14: SPSS Output: Classification Results:
Observed
Predicted
Attribute 11
Percentage Correct
1.00
2.00
Step 1
Attribute 11
1.00
156
8
95.1
2.00
4
132
97.1
Overall Percentage
96.0
Predictive Ability of the Logistic Model
The logistic model is able to classify 156 good applicants as “Good Group” out of 164 good applicants. Thus, it holds 95.1% classification accuracy for the good group. On the other hand, the same logistic model is able to classify 132 bad applicants as “Bad Group” out of 136 bad applicants. Thus, it holds 97.1% classification accuracy for the bad group. Thus, the model is able to generate 96.0% classification accuracy for the both groups.
4.5.3 k-Nearest Neighbor Analysis:
Table 15: SPSS Output: Classification Results:
Predictive Ability of the k-Nearest Neighbor Classifier
The k-Nearest Neighbor Classifier model is able to classify 110 good applicants as “Good Group” out of 116 good applicants. Thus, it holds 94.83% classification accuracy for the good group. On the other hand, the same k-Nearest Neighbor Classifier model is able to classify 91 bad applicants as “Bad Group” out of 95 bad applicants. Thus, it holds 95.79% classification accuracy for the bad group. Thus, the model is able to generate 95.27%classification accuracy in combined groups.
4.5.4 Classification and Regression Tree Analysis:
Table 16: SPSS Output: Classification Results:
Observed
Predicted
1.00
2.00
Percent Correct
1.00
157
7
95.7%
2.00
3
133
97.8%
Overall Percentage
53.3%
46.7%
96.7%
Growing Method: CHAID
Dependent Variable: Status of the Credit Applicant
Predictive Ability of the Classification Tree
The Classification Tree analysis model is able to classify 157 good applicants as “Good Group” out of 164 good applicants. Thus, it holds 95.7% classification accuracy for the good group. On the other hand, the same Classification Tree analysis model is able to classify 133 bad applicants as “Bad Group” out of 136 bad applicants. Thus, it holds 97.8% classification accuracy for the bad group. Thus, the model is able to generate 96.7% classification accuracy in combined group.
4.5.5 Artificial Neural Network:
Table 17: SPSS Output: Classification Results:
Sample
Observed
Predicted
1.00
2.00
Percent Correct
Training
1.00
119
4
96.7%
2.00
4
81
95.3%
Overall Percent
59.1%
40.9%
96.2%
Testing
1.00
40
1
97.6%
2.00
0
51
100.0%
Overall Percent
43.5%
56.5%
98.9%
Dependent Variable: Status of the Credit Applicant
Predictive Ability of the Artificial Neural Network
The neural network model is able to classify 119 good applicants as “Good Group” out of 123 good applicants. Thus, it holds 96.7% classification accuracy for the good group. On the other hand, the same neural network model is able to classify 81 bad applicants as “Bad Group” out of 85 bad applicants. Thus, it holds 95.3% classification accuracy for the bad group. Thus, the model is able to generate 96.2% classification accuracy for the both groups. Here, the training sample is taken into account, because statistical models don’t use testing sample.
4.6. Optimization of Neural Network Performance & Future Research Scope
Genetic Algorithm can be used as an optimization method for improving NN performance.
There are two main issues about the performance of the neural network. First of all, it is important to determine its structure and secondly, it is also vital to specify the weights of the neural network that help to minimize the total errors (Deb, Poli et al. 2004). These are optimization issues. Evolutionary algorithm (genetic algorithm) is a kind of optimization technique that uses selection and recombination as the main instruments to deal with optimization problems (Kamruzzaman, Begg et al. 2009). For example, genetic algorithm is the main available method that can be used to find well suited network architecture for a given task or problem (Patel, Honavar et al. 2001). According to the genetic algorithm theory, all combinations of the parameters of the possible solutions of a given problem (in this analysis, all the possible combinations of the parameters of the neural network architecture) must be coded into a gene and by a process of the selection of the fittest (in this case, the best neural network architecture that provides better classification) only the best solutions are selected for the reproduction, and after each subsequent generation, new solutions (only selected if they provide better classification accuracy) are generated by means of the reproduction between solutions and their related mutations (Mira, Cabestanyet al. 2009).
Genetic algorithm is especially suitable for complex optimization problems. Many researchers tried to optimize the weights of the neural network using genetic algorithm alone (or, with backpropagation algorithm) and others attempted to find out a good network architecture or structure (the number of units and their interconnections) (Deb, Poliet al. 2004). Instruments or tools used by the genetic algorithm are selection, crossover and mutation (Larose, 2006).
Selection indicates the technique of selecting chromosomes that will be used for reproduction. The fitness function appraises each of the chromosome (candidate solutions), and the better (fitter) the chromosome, the more probability that it will be selected for the reproduction purpose. The central task of crossover is to perform recombination, means that the creation of the two new offspring by randomly choosing a locus and exchanging subsequences to the left and right of that locus between two chromosomes chosen during the selection process. Mutation randomly modifies the bits or digits at a particular locus in a chromosome, most of the time with very low likelihood. Mutation brings new information to the genetic pool and protects against finding too quickly to a local optimum. The concepts are from Larose (2006).
There are some unique benefits of using Genetic Algorithm as searching and optimization technique. For example, it is efficient, adaptive, and robust search technique, that is capable of generating optimal (or, near optimal) solutions (Pal and Wang 1996). For these characteristics and advantages, genetic algorithms applications in pattern recognition problems (which require robust, fast and close appropriate solutions) are perfect and almost natural.
Although this study is aimed at providing a comparative ability of the neural network in the classification of the credit card customers, its scope of coverage is limited by the selection and comparison of the statistical models with neural network model. Future research efforts can certainly go beyond this limitation in order to have a more comprehensive review of genetic algorithm related literatures and to use the same dataset for the optimization of the network.
4.7. Findings and Conclusion
Appropriate predictor variables selection is one of the conditions for successful credit scoring models development. This study reviews several considerations regarding the selection of the predictor variables. Moreover, using the Multilayer Perceptron Algorithm of Neural Network, network architecture is constructed for predicting the probability that a given customer will default on a loan. The model results are comparable to those obtained using commonly used techniques like Logistic Regression, CART Discriminant Analysis or k-NN Classifier, as described in the following table:
Table: 18: Predictive Models Comparison
Credit Dataset
Models
Good
Accepted
Good
Rejected
Bad
Accepted
Bad
Rejected
Success
Rate
Discriminant Analysis
156
8
132
4
96.0%
Logistic Regression
158
8
132
4
96.0%
k-NN Classifier
110
6
91
4
95.79%
CART
157
7
133
3
96.7%
Neural Network
119
51
81
4
96.3%
There are two noteworthy and interesting points about this table. First of all, it shows the predictive ability of each model. Here, the column 2 and 5 (“Good Accepted” and “Bad Rejected”) are the applicants that are classified correctly. Moreover, the column 3 and 4 (“Good Rejected” and “Bad Accepted”) are the applicants that are classified incorrectly. Furthermore, it shows that neural network gives slightly better results than discriminant analysis, Logistic regression and k-NN Classifier but, it gives slightly worst results than Classification and Regression Trees. It should be noted that it is not possible to draw a general conclusion that neural network holds better predictive ability than k-NN Classifier, Logistic regression and discriminant analysis, worst predictive ability than Classification and Regression Tree (CART) because this study covers only one dataset. On the other hand, statistical models can be used to further explore the nature of the relationship between the dependent and each independent variable, and statistical models are preferred than neural network if there is tiny difference in predictive ability of the neural network and the statistical models.
Secondly, the above provided table gives an idea about the cost of misclassification. In order to introduce the costs of misclassification it is assumed that a “Bad Accepted” generates much higher costs than a “Good Rejected”, because there is a chance to lose the whole amount of credit while accepting a “Bad” and only losing the interest payments while rejecting a “Good” (Schader, Gaul et al. 2009). Here, in this analysis, it is apparent that neural network (equals to 81) acquired less amount of “Bad Accepted” than discriminant analysis (equals to 132) logistic regression (equals to 132) Classification and Regression Tree (equals 133) and k-NN Classifier (equals 91). So, neural network achieves less cost of misclassification.
In the above table, statistical models contain the same number of cases for the model construction, but the NN and k-NN Classifier hold less number of cases because a portion of the cases is kept for the “Testing Sample” to prevent the network from overtraining/ over-fitting. Furthermore, statistical models don’t use anything like testing samples. So, the classification results are slightly incomparable because of the uneven number of cases used in all models. After re-running the analysis, without any testing sample in the NN, and k-NN Classifier with the equal number of sample sizes in all the models, it is found that "Bad Accepted" is decreased further to 3 from 81 for the NN. Because, neural network over trained itself. Moreover, overall percentage correct is increased to 98% from 96.3%. So, for the credit dataset, NN is giving good results always. But, for k-NN Classifier even though the whole sample is used for training the model, the “Bad Accepted” remained 4 but the overall percentage correct is increased to 96.34% from 96.3% which is very insignificant.
In the final section, Genetic Algorithm is proposed to obtain better classification accuracy through the configurations of the neural network architecture and weights. Moreover, it is suggested as future research scope that the same dataset can be used for neural network performance improvement and the results can be compared with before optimization results.
References
Abrahams, C.R. and Zhang, M. (2009) Credit Risk Assessment: The New Lending System for Borrowers, Lenders, and Investors, Wiley.
Agresti, A. (2002) Categorical Data Analysis, Wiley- Interscience
Altman, E. (1968): Financial Ratios, Discriminant Analysis and the prediction of Corporate Bankruptcy. Journal of Finance, 23,589-609
Anderloni, L., Braga, M.D. and Carluccio, E.M. (2006) New Frontiers in Banking Services: Emerging Needs and Tailored Products for Untapped Markets, Springer.
Beares, P.R., Beck, R.E. and Siegel, S.M. (2001) Consumer Lending, American Bankers Association.
Blattberg, R.C., Kim, B.D. and Neslin, S.A. (2008) Database Marketing: Analyzing and Managing Customers, Springer.
Bocij, P., Chaffey, D., Greasley, A. and Hickies, S. (2009) Business Information Systems: Technology, Development and Management for the E-business, FT Press.
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J. (1984): Classification and Regression Trees. Pacific Grove, CA: Wadsworth, 1984.
Campbell, T.S., Dietrich, J.K. (1983): The Determinants of Default on Insured Conventional Residential Mortgage Loans. Journal of Finance, 38,1569-1581
Charitou, A., Neophytou, E., Charalambous, C.(2004): Predicting Corporate Failure: Empirical Evidence for the UK. European Accounting Review, 13, 465-497.
Christine B., (2009) Logistic regression and its Application in Credit Scoring.
Chatterjee, S. Barcun, S. (1970): A Nonparametric Approach to Credit Screening. Journal of American Statistical Association, 65,50-154.
Colquitt, J. (2007) Credit Risk Management, McGraw-Hill.
Culp, C.L. (2001) The Risk Management Process: Business Strategy and Tactics, Wiley.
Davis, R.H., Edelman, D.B., and Gammerman, A.J. (1992). Machine learning algorithms for Credit-Card applications. IMA Journal of Mathematics Applied in Business and Industry, 4, 43-51.
Deb, K., Poli, R., Banzhaf, W., Beyer, H.G., Burke, E., Darwen, P., Dasgupta, D., Beyer, H.G., Foster, Harman, M., Holland, O., Lanzi, P.L., Spector, L., Tettamanzi, A. and Thierens, D.(2004) Genetic and Evolutionary Computation GECCO 2004, Springer.
Desai, V.S. Crook, J.N., and Overstreet, G.A. (1996). A comparison of neural networks and linear scoring models in the credit union environment. European Journal of Operation Research, 95(1), 24-37
Denison, D.G.T., Mallick, B.K., Smith, A.F.M. (1998): A Bayesian CART Algorithm. Biometrics, 85,363-377.
Devaney, S. (1994): The Usefulness of Financial Ratios as Predictors of Household Insolvency: Two Perceptives. Financial Counseling and planning, 5, 15-24.
Eisenbeis, R.A. (1977): Pitfalls in the Application of Discriminant Analysis in Business, Finance and Economics. Journal of Finance, 32, 875-900.
Esposito, F., Malerba, D., Semeraro, G.(1997): A comparative Analysis of Methods for Pruning Decision Trees. IEEE Transactions on pattern Analysis and Machine Intelligence, 19, 476-491.
Fabozzi, F.J., Davis, H.A. and Choudhry, M. (2006) Introduction to Structured Finance, Wiley.
Feldman, D., Gross, S. (2005): Mortgage Default: Classification Tree Analysis. Journal of Real Estate Finance and Economics, 30,2005 369-396.
Fisher, R.A. (1936): The use of Multiple Measurements in Taxonomic Problems. Annals of Eugenic, 7, 179-188.
Frydman, H., Altman, E., Kao, D.L. (1985): Introducing Recursive Partitioning for Financial Classification.
Gardner, M.J., Mills, D.L. (1989): Evaluating the likelihood of Default on Delinquency Loans: Financial Management, 18, 55-63.
Greenacre, M. and Blasius, J. (2006) Multiple Correspondence Analysis and Related Methods, Chapman and Hall/CRC.
Hand, D.J., Vinciotti, V. (2003): Choosing k for Two-class Nearest Neighbor Classifiers with Unbalanced Classes. Pattern Recognition Letters, 24, 1555-1562.
Henley, W.E., and Hand, D.J. (1996). A k-nearest neighbor Classifier for assessing Consumer Credit risk. Statistician, 44, 77-95.
Holmes, C.C., Adams, N.M. (2002): A Probabilistic Nearest Neighbor Method for Statistical Pattern Recognition. Journal of Royal Statistical Society. Series B,64,295-306.
Jorion, P. (2000) Value at Risk: The New Benchmark for Managing Financial Risk, McGraw-Hill.
Kamruzzaman, J., Begg, R.K. and Sarker, R.A. (2006) Artificial Neural Networks in Finance and Manufacturing, Idea Group Publishing.
Larose, D.T. (2006) Data Mining Methods and Models, Wiley-IEEE Press.
Lawrence, E., Arshadi, N. (1995): A MultinominalLogit Analysis of Problem Loan Resolution Choices in Banking. Journal of Money, Credit and Banking, 27, 202-216.
Leondes, C.T. (2005) Intelligent Knowledge- Based Systems: Business and Technology in the New Millennium, Kluwer Academic Publishers.
Liang, L., Desheng, W.(2005) An application of pattern recognition on scoring Chinese corporations financial conditions based on backpropagation neural network
Mays,E.(1998) Credit Risk Modeling: Design and Application, CRC.
Meyers, L.S. Gamst, G.C. and Guarino, A.J. (2005) Applied Multivariate Research: Design and Interpretation, Sage Publications.
Mira, J., Cabestany, J. and Prieto, A. (2009) New Trends in Neural Computation, Springer.
Mu, C.C., Shih, H.H. (2003) Credit Scoring and rejected instances reassigning through evolutionary computation techniques. Journal of Expert Systems with Applications 24, 433-441.
Pal, S.K. and Wang, P.P. (1996) Genetic Algorithms for Pattern Recognition, CRC
Patel, M., Honavar, V. and Balakrishnan, K.(2001) Advances in the Evolutionary Synthesis of Intelligent Agents, The MIT Press.
Plewa, F.J. and Friedlob, G.T. (1995) Understanding Cash Flow, Wiley.
Reichert, A.K., Cho, C.C. and Wagner, G.M. (1983). An examination of the Conceptual issues involved in developing Credit-Scoring Models. Journal of Business and Economics Statistics ,1,101-114.
Rosenberger, L.E., Nash, J and Graham A. (2009) The Deciding Factor: The Power of Analytics to make Every Decision a Winner, Jossey-Bass
Samsul M.I., Lin Z. Fei Li (2009) Application of Artificial Intelligence (Artificial Neural Network) to Assess Credit Risk. A Predictive Model For Credit Card Scoring.
Servigny, A.D. and Renault, O. (2004) The Standard and Poor’s Guide to Measuring and Managing Credit Risk, McGraw-Hill.
Siddiqi, N. (2005) Credit Risk Scorecards: Developing and Implementing Intelligent Credit Scoring, Wiley.
Thomas, Edelman, D. and Crook, J. (2002) Credit Scoring and its Applications, Society for Industrial Mathematics.
Thomas, L.C., Edelman, D.B. and Crook, J.N. (2004) Readings in Credit Scoring: Foundations, Development and aims, Oxford University Press.
West, D. (2000). Neural Network Credit Scoring Models. Computers and Operation Research, 27,1131-1152.
Zhang, G.P. (2000). Neural Networks for Classification: a survey. IEEE Transactions on Systems, Man, and Cybernetics-Part: Applications and Reviews, 30(4), 451-462.
Zhang, G.P. (2003) Neural Networks in Business Forecasting, Information Science Publishing
0 Response to "A FREE SEMINAR ON DEFAULT PREDICTORS IN AGRICULTURAL CREDIT SCORING: EVIDENCE FROM BANK OF AGRICULTURE DATA SOKOTO"
Post a Comment