Geek asks and answers

MATLAB, how to calculate distances between large coordinate sets without exceeding memory limits?

(using MATLAB) I have a large coordinate matrix and a large sparse adjacency matrix for which the coordinates are related to each other. I asked earlier about SO how to efficiently calculate these distances in this SO question , but I ran into memory problems , which are a much more serious problem.

I used this MATLAB function to calculate the distance matrix Dists = pdist2(fruchterman_graph(:,:),fruchterman_graph(:,:),'euclidean');, but it does not work on large networks for both speed and memory.

This is the code that only works on small graphs (not hundreds of thousands):

coordinate = kamada_graph;
    [n, d] = size(kamada_graph);
    assert(d == 2);
    resi = sparse(adj*  spdiags((1:n)',0,n,n));
    resj = sparse(spdiags((1:n)',0,n,n) * adj);
    res = sparse(n,n);
    f = find(adj);  
    res(f) = sqrt((coordinate(resi(f), 1) - coordinate(resj(f), 1)) .^ 2 +...
                          (coordinate(resi(f), 2) - coordinate(resj(f), 2)) .^ 2);

This creates large graphs

??? == > find , .
[i,j] = find(S) .
== > modularize_graphs 49 [f] = find(adj);

, :

[i,j] = find(ajd);    
res(i,j) = sqrt((coordinate(resi(i,j), 1) - coordinate(resj(i,j), 1)) .^ 2 +...
    (coordinate(resi(i,j), 2) - coordinate(resj(i,j), 2)) .^ 2);

(~ 500 ) :

??? .
HELP MEMORY .
== > modularize_graphs 50
    res(i,j) = sqrt((coordinate(resi(i,j), 1) - coordinate(resj(i,j), 1)) .^ 2 +...

, , (N,2) x, y, , , ?

- , , Adj.

+5
1

, :

%# assuming a square and symmetric adjacency matrix
nCoords = size(adjMat,1);

%# there are at most (n^2-n) distances 
%# if this runs out of memory already, there 
%# is a way to only store existing distances to save 
%# even more memory
distances = NaN((nCoords^2-nCoords)/2,1);

ct = 0;
for row = 1:nCoords
   for col = 1:row-1
     ct = ct+1;
     if adjacencyMatrix(row,col)
        distances(ct) = sum( (coordinate(row,:)-coordinate(col,:)).^2 );
     end
   end
end

distances = sqrt(distances);

( , resi resj, ).

[row,col]=find(adjacencyMatrix);

distances = sqrt(sum( (coordinate(row,:) - coordinate(col,:)).^2 ,2));

sparseDistanceMatrix = sparse(row,col,distances);
+3

More articles:


All Articles