Activity 15 Probabilistic Classification

Pattern recognition was performed in the previous activity using the Euclidean distance. Now, we are going to employ Linear Discriminant Analysis (LDA) for classifying members of a certain group. The image from the previous activity are reused for this purpose and is shown in Figure 1. LDA shows that groups can be separated by a linear combination of features that are used to describe the members.

Figure1. Image used for pattern recognition using LDA

Using the object features, the members are assigned to the group with highest conditional probability. This is known as Baye's rule, which helps minimize the total error of classification (TEC). Mathematically, it is expressed as the object belong to group i where

Thus, the probability that the member belongs to group i given a set of data x is determined.

To implement LDA, we form a matrix x consisting of rows of objects and columns of features. For this particular activity, I used the same objects (blue and red dinaosaurs) and same features as the previous activity, that is, r_values and area. From this matrix, the two classes of objects are separated (x1 and x2) with their corresponding features. To visualize this, refer to the equation below.

From x1 and x2, the mean features u1 and u1, and global mean vector are determined which are used to get the mean corrected data x10 and x2o. From these values, the pooled covariance matrix are calculated using the equation

where g is the number of groups ci(r,s) is the covariance of each group. Finally, the discriminant function is solved using the following formula:

The selection of an object to a particular group is by choosing the Fi of the class with higher value. Here, F1 correspond to the class of red dinosaurs while F2 to the blue dinosaurs. Table 1 shows the result after performing LDA.

Table 1. Results obtained after applying LDA

The percentage of correct prediction of the classification is 100%.
For this activity, I am able to apply LDA to classify objects belonging to a certain class. Since I met the objective, I give myself a grade of 10.

References:
http://people.revoledu.com/kardi/tutorial/LDA/
http://www.craftkitsandsupplies.com/images/Foam_Shapes/Foam_Dinosaur_Shapes.jpg