[Octave-bug-tracker] [bug #47175] “dimension too large” error when broadcasting to sparse matrix in octave

[anonymous]

URL:
  <http://savannah.gnu.org/bugs/?47175>

                 Summary: “dimension too large” error when broadcasting to
sparse matrix in octave
                 Project: GNU Octave
            Submitted by: None
            Submitted on: Wed 17 Feb 2016 01:17:31 AM UTC
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Documentation
                  Status: None
             Assigned to: None
         Originator Name: Dan Parshall
        Originator Email: address@hidden
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 3.8.2
        Operating System: GNU/Linux

    _______________________________________________________

Details:

32-bit Octave has a limit on the maximum number of elements in an array. I
have recompiled from source (following the script at
https://github.com/calaba/octave-3.8.2-enable-64-ubuntu-14.04 ), and now have
64-bit indexing.

Nevertheless, when I attempt to perform elementwise multiplication using a
broadcast function, I get error: out of memory or dimension too large for
Octave's index type

My suspicion is that this is a bug with repmat, and that instead of producing
a sparse matrix it generates a full one.  But I don't know Octave's internals
well enough to be sure.

Minimal code to reproduce the problem:


function indexerror();
% both of these are formed without error
%    a = zeros (2^32, 1, 'int8');
%    b = zeros (1024*1024*1024*3, 1, 'int8');
%   sizemax    % returns 9223372036854775806

    nnz = 1000            % number of non-zero elements
    rowmax = 250000
    colmax = 100000

    irow = zeros(1,nnz);
    icol = zeros(1,nnz);
    for ind =1:nnz
        irow(ind) = round(rowmax/nnz*ind);
        icol(ind) = round(colmax/nnz*ind);
    end

    sparseMat = sparse(irow,icol,1,rowmax,colmax);

    % column vector to be broadcast
    broad = 1:rowmax;
    broad = broad(:);

    % this gives "dimension too large" error
    toobig = bsxfun(@times,sparseMat,rep);

    % so does this
    toobig2 = sparse(repmat(broad,1,size(sparseMat,2)));
    mult = sparse( sparseMat .* toobig2 );        % never made it this far
end



My current workaround is to do the multiplication inside of a loop, but it's
sloooooow:

    % loop over rows, instead of using bsxfun
    mult_loop = sparse([],[],[],rowmax,colmax);
    for ind =1:length(broad);
        mult_loop(ind,:) = broad(ind) * sparseMat(ind,:);
    end


Copying from my SO question:
http://stackoverflow.com/questions/35421967/dimension-too-large-error-when-broadcasting-to-sparse-matrix-in-octave




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47175>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/