Question

Give practical examples to differentiate LDA and PCA

Answer 1

1) What is Principle Component Analysis(PCA)

PCA is a technique for feature extraction. so it combines our input variables in a specific way, then we can drop the least important variables while still retaining the most valuable parts of all the variables.

a) Step:

1) Calculate the covariance matrix X of data points.

2) Calculate eigenvectors and correspond eigenvalues.

3) Sort eigenvectors accordingly to their given value in decrease order.

4) Choose first k eigenvectors and that will be the new k dimensions.

5) Transform the original n-dimensional data points into k_dimensions

2) What is a Linear Discriminant Analysis(LDA):

LDA is a type of Linear combination, a mathematical process using various data items and applying a function to that site to separately analyze multiple classes of objects or items.

Step:

1) Compute the d-dimensional mean vector for the different classes from the dataset.

2) Compute the Scatter matrix(in between class and within the class scatter matrix)

3) Sort the Eigen Vector by decrease Eigen Value and choose k eigenvector with the largest eigenvalue to from a d*k dimensional matrix w (where every column represent an eigenvector)

4. Used d * k eigenvector matrix to transform the sample onto the new subspace.

This can be summarized by the matrix multiplication.

Y = X * W (where X is an n*d dimension matrix representing the n samples and you are transformed n * k dimensional samples in the new subspace.

Note data link: https://gist.github.com/tijptjik/9408623