Geek asks and answers

PCA, LDA and Naive Baysian

I am trying to apply linear discriminant analysis to a problem with two classes. As I understand it, the LDA assumes that both classes have the same covariance matrix, and then model probability as a Gaussian distribution in various ways.

Another classifier I tried is naive Bayesian. It ignores any correlation between predictor variables.

Now I don’t understand what happens when I apply a PCA to a dataset. By its very definition, the PCA algorithm rotates the data so that the covariance matrix is ​​diagonal. Nothing is lost in rotation, but since the covariance matrix is ​​now diagonal, will the naive Bayesian language not be as good as the LDA, or even better, since the LDA will have much more parameters for evaluation? However, to my knowledge, Naive Bayes is superior to the LDA with or without a PCA.

Indeed, the data is exactly the same as long as I use all the basic components that tell me that the result really should be what it is. However, the covariance matrix is ​​diagonal ... brain crisis

Can someone explain this to me? I hope that I have formulated my question quite clearly. Thank!

+5
source share
3 answers

PCA is useful for reducing dimensionality (for example, if the size of your exercise set is too small for the number of data measurements). But if you use all the basic components, the PCA will not improve the results of your linear classifier - if your classes were not linearly divided in the original data space, then rotating your coordinates through the PCA will not change that.

, PCA, , , . , ( , PCA), , (.. ). , , .

+6

, PCA , . , ( ), . , , , PCA +.

+1

@bogatron , Naive Bayes , .

p(x|1)p(y|1)p(z|1) = p(x|2)p(y|2)p(z|2)

,

P(M1)=P(M2)=0.5.

p(.|1) and p(.|2) .

0

More articles:


All Articles