Re: [Help-glpk] Out of memory, malloc error

[Anthony Presley]

On Wed, 2007-11-28 at 00:31 +0300, Andrew Makhorin wrote:
> > We have a (what I consider) to be fairly simple model, which performs
> > very well for small - medium data sets (between 500 and 2500 lines of
> > data).  However, we recently tried to use the mod file for a larger data
> > set (about 3200 lines of data).  Note: We're not using glpk to solve our
> > Integer Program, we use it to create an MPS file, which is then solved
> > by another solver, so we use the --check option.  Using this method, we
> > have an MPS file generated in under 20 seconds.
> 
> > Using a faster Core 2 Duo, 2GB of RAM, and 2GB of swap, glpk ran fine
> > until our last rule.  In the last rule, memory sky rocketed from 400MB
> > for the glpk process to more than 4GB, resulting in a system load of
> > over 20, and glpk eventually quit with a malloc() error.  It took over
> > 60 minutes to get to this point.
> 
> > So, I moved it to our production system, which has 4GB of RAM and 4GB of
> > swap, and the same thing occurred.  It took over 40 minutes to get to
> > this point.
> 
> > Our original data sets use about 200 - 300MB, in total.
> 
> > Any ideas on how we might go about speeding up the creation of the MPS
> > file .... and bringing down the memory footprint?
> 
> Most probably this happens due to huge amount of data, because
> internal model representation used in the MathProg translator has
> relatively high overhead expenses.
> 
> Is there an excessive swapping on the generation phase?
> 
> Which is the model size (estimated) for successful and unsuccessful
> runs, resp.?
>