Geek asks and answers

Matlab, read some 2d arrays from csv file

I have a csv file containing 2d arrays of 4 columns, but a different number of rows. For instance:

2, 354, 23, 101
3, 1023, 43, 454
1, 5463, 45, 7657

4, 543, 543, 654
3, 56, 7654, 344

...

I need to be able to import data so that I can perform operations on each data block, however csvread, dlmread and textscan ignore empty lines.

I can’t find a solution anywhere, how can this be done?

PS:

It may be worth noting that the files of the above format are actually a concatenation of many files containing only one data block (I don’t want to read from thousands of files each time), so the empty line between the blocks can be changed to any other separator / marker. This is only done using a python script.

EDIT: My decision is based / inspired by the petricor below

csvread textscan, . , , nan ( python script), . :

filename = 'data.csv';
fid = fopen(filename);
allData = cell2mat(textscan(fid,'%f %f %f %f','delimiter',','));
fclose(fid);

nanLines = find(isnan(allData(:,1)))';

iEnd = (nanLines - (1:length(nanLines)));
iStart = [1 (nanLines(1:end-1) - (0:length(nanLines)-2))];
nRows = iEnd - iStart + 1;

allData(nanLines,:)=[];

data = mat2cell(allData, nRows);

0.28s ( 103000 ). petrichor, .

+3
1
filename = 'data.txt';

%# Read all the data
allData = csvread(filename);

%# Compute the empty line indices
fid = fopen(filename);
lines = textscan(fid, '%s', 'Delimiter', '\n');
fclose(fid);
blankLines = find(cellfun('isempty', lines{1}))';

%# Find the indices to separate data into cells from the whole matrix
iEnd = [blankLines - (1:length(blankLines)) size(allData,1)];
iStart = [1 (blankLines - (0:length(blankLines)-1))];
nRows = iEnd - iStart + 1;

%# Put the data into cells
data = mat2cell(allData, nRows)

:

data = 

    [3x4 double]
    [2x4 double]
+1

More articles:


All Articles