Geek asks and answers

Quotes reading data in R

I have some data, and I'm trying to load it into the R . It is located in .csv files and I can view data in Excel and OpenOffice. (If you're curious, this is the 2011 survey results from the available Canadian election data here ).

Data is encoded in an unusual way. Typical line:

12002,Central Nova","Nova-Centre"," 1","River John",N,N,"",1,299,"Chisholm","","Matthew","Green Party","Parti Vert",N,N,11

There is Central-Nova at the end ", but not at the beginning. Therefore, in order to read the data, I suppressed quotes that worked fine for the first few files. i.e.

test<-read.csv("pollresults_resultatsbureau11001.csv",header = TRUE,sep=",",fileEncoding="latin1",as.is=TRUE,quote="")

Now here is the problem: in another file (for example, pollresults_resultatsbureau12002.csv) there is a data line like this:

12002,Central Nova","Nova-Centre"," 6-1","Pictou, Subd. A",N,N,"",0,168,"Parker","","David K.","NDP-New Democratic Party","NPD-Nouveau Parti democratique",N,N,28

Since I need to suppress quotes, writing "Pictou, Subd. A"allows R to split it into 2 variables. Data cannot be read because it wants to add a column halfway through the creation of a data block.

Excel and OpenOffice can open these files without problems. Somehow, Excel and OpenOffice know that quotation marks only matter if they are at the beginning of a variable.

Do you know which option I need to enable on R to get this data? I have> 300 files that I need to upload (each with ~ 1000 lines each), so manually fixing is not an option ...

I looked everywhere for a solution, but cannot find it.

+5
source share
2 answers

, , CSV .

# Deal with French properly
options(encoding="latin1")

# Set your working directory to where you have
#   unzipped all of your 308 CSV files
setwd("path/to/unzipped/files")

# Get the file names
temp <- list.files()

# Extract the 5-digit code which we can use as names
Codes <- gsub("pollresults_resultatsbureau|.csv", "", temp)

# Read all the files into a single list named "pollResults"
pollResults <- lapply(seq_along(temp), function(x) {
  T0 <- readLines(temp[x])
  T0[-1] <- gsub('^(.{6})(.*)$', '\\1\\"\\2', T0[-1])
  final <- read.csv(text = T0, header = TRUE)
  final
})
names(pollResults) <- Codes

-. 90- data.frame, , pollResults[[90]] pollResults[["24058"]] ( , , ).

, . , 308 CSV , , CSV " _".

invisible(lapply(seq_along(pollResults), function(x) {
  NewFilename <- paste("Corrected", temp[x], sep = "_")
  write.csv(pollResults[[x]], file = NewFilename, 
            quote = TRUE, row.names = FALSE)
}))

, !

+1

@AnandaMahto (. ).

-, - :

options(encoding="latin1")

, readLines():

temp <- readLines("pollresults_resultatsbureau13001.csv")

+. , 5 . , .

temp[-1] <- gsub('^(.{6})(.*)$', '\\1\\"\\2', temp[-1])

.

fileConn<-file("pollresults_resultatsbureau13001.csv") writeLines(temp,fileConn) close(fileConn)

, R:

data<-read.csv(file="pollresults_resultatsbureau13001.csv",header = TRUE,sep=",")

, ( , ), .

+1

More articles:


All Articles