Geek asks and answers

ROC / AUC calculation for MaxEnt and BIOMOD

Many thanks to Winchester for the kind help! I also saw a tutorial and it works for me! Over the past two days, I have studied the output of both MaxEnt and BIOMOD, and I think that the terms used in these two are still a bit confused.

From the Philips code, it seems that he used sampling points and a reference point to calculate the ROC, while in BIOMOD there is only a forecast from the points of presence and pseudo-declaration. which means that for the same data set I have the same amount of data about the presence / selection, but different data of the absence / background for the two models, respectively. And when I recount ROC, it usually does not agree with the values ​​indicated by the model itself.

I think that so far I have not received an assessment of the model regarding what was evaluated, and how to create an evaluation data set, i.e. mixing matrix, and how much of the data was selected as an estimate.

Thank you all for your kind reply! I am very sorry for the inconvenience. I added some more suggestions to the message for BIOMOD, to make it workable, as for MaxEnt, you can use training data.

Actually, the purpose of my message is to find someone who has experience with a presence / absence dataset and a presence-only dataset. I probably know how to handle them separately, but not completely.

I use both MaxEnt and several algorithms in BIOMOD to distribute my views, and I would like to build ROC / AUC on the same figure, has anyone done this before?

, MaxEnt ROC ROCR vcd, MaxEnt Philips:

   install.packages("ROCR", dependencies=TRUE)
   install.packages("vcd",  dependencies=TRUE)
   library(ROCR)
   library(vcd)
   library(boot)
   setwd("c:/maxent/tutorial/outputs")
   presence <- read.csv("bradypus_variegatus_samplePredictions.csv")
   background <- read.csv("bradypus_variegatus_backgroundPredictions.csv")
   pp <- presence$Logistic.prediction                # get the column of predictions
   testpp <- pp[presence$Test.or.train=="test"]       # select only test points
   trainpp <- pp[presence$Test.or.train=="train"]   # select only test points
   bb <- background$logistic
   combined <- c(testpp, bb)                             # combine into a single vector
   label <- c(rep(1,length(testpp)),rep(0,length(bb)))  # labels: 1=present, 0=random
   pred <- prediction(combined, label)                    # labeled predictions
   perf <- performance(pred, "tpr", "fpr")          # True / false positives, for ROC curve
   plot(perf, colorize=TRUE)                                # Show the ROC curve
   performance(pred, "auc")@y.values[[1]]            # Calculate the AUC

BIOMOD / , 1000 , . script, :

library(BIOMOD)
library(PresenceAbsence)

data(Sp.Env)

Initial.State(Response=Sp.Env[,12:13], Explanatory=Sp.Env[,4:10], 
IndependentResponse=NULL, IndependentExplanatory=NULL)

Models(GAM = TRUE, NbRunEval = 1, DataSplit = 80,
   Yweights=NULL, Roc=TRUE, Optimized.Threshold.Roc=TRUE, Kappa=F, TSS=F, KeepPredIndependent = FALSE, VarImport=0,
   NbRepPA=0, strategy="circles", coor=CoorXY, distance=2, nb.absences=1000)


load("pred/Pred_Sp277")

    data=cbind(Sp.Env[,1], Sp.Env[,13], Pred_Sp277[,3,1,1]/1000)

    plotroc <- roc.plot.calculate(data)


    plot(plotroc$threshold, plotroc$sensitivity, type="l", col="blue ")

    lines(plotroc$threshold, plotroc$specificity)
    lines(plotroc$threshold, (plotroc$specificity+plotroc$sensitivity)/2, col="red")

, ? , , , . , -, .

~

Marco

+3
3

, , , , MaxEnt BIOMOD ( ). , pROC - , ROC. , , roc AUC.

library(pROC)

#Create dummy data set for test observations
obs<-rep(0:1, each=50)
pred1<-c(runif(50,min=0,max=0.8),runif(50,min=0.3,max=0.6))
pred2<-c(runif(50,min=0,max=0.6),runif(50,min=0.4,max=0.9))

roc1<-roc(obs~pred1) # Calculate ROC for each method
roc2<-roc(obs~pred2) 

#Plot roc curves for each method

plot(roc1)
lines(roc2,col="red")

#Compare differences in area under ROC
roc.test(roc1,roc2,method="bootstrap",paired=TRUE)
+3

, PresenceAbsence. , ROC. , , y, , , . , ?

data(SIM3DATA)
plotroc <- roc.plot.calculate(SIM3DATA,which.model=2, xlab = NULL, ylab = NULL)
plot(plotroc$threshold, plotroc$sensitivity, type="l", col="blue ")   
lines(plotroc$threshold, plotroc$specificity)    
lines(plotroc$threshold, (plotroc$specificity+plotroc$sensitivity)/2, col="red")
lines(1 - plotroc$specificity, plotroc$sensitivity, lwd = 2, lty = 5)
+2

I am using the pROC package . It has many good features when it comes to building ROC and AUC in the same schedule. It is also very useful.

+1
source

More articles:


All Articles