Re: Commiting Large Files.

[Derek Robert Price]

Richard Horton wrote:

> For various reasons we're using CVSNT to maintain a repository which amongst
> other goodies contains large image files of laptop's (around 650MB). 
>
> I am aware that client.c reads the whole file (in send_modified) into
> contiguous memory and so this is causing the commit to fail with out of
> memory error in buffer.c.
>
> Is there a fix or work around for this? 
>
> The images can NOT be compressed further and spliting them would lead to
> problems in reassembling them (well not so much problems as an admin
> overhead we can ill afford).
>
> Any suggestions would be appreciated.
>  

There's not really a fix or workaround, other than not checking the 
files in.  I started rewriting some of the CVS server code to use mmap 
to open RCS archives for reading a year and a half ago or so.  The idea 
is to use the system's mmap function when available to let the system 
hand you what looks like a memory pointer but allow the system to do the 
paging for you.  In theory, the OS could be using the file itself as 
"swap" but I'm not sure it actually does.  Apache uses this method when 
reading files as the fastest and most efficient way to go.

You could use this as a starting point to opening all files that way. 
I'd like to see the patch when you were done.

Derek

--
               *8^)

Email: derek@ximbiot.com

Get CVS support at <http://ximbiot.com>!
--
The Gothic idea that we were to look backwards instead of forwards for the
improvement of the human mind, and to recur to the annals of our ancestors for
what is most perfect in government, in religion and in learning, is worthy of
those bigots in religion and government by whom it has been recommended, and
whose purposes it would answer.  But it is not an idea which this country will
endure.
                        - Thomas Jefferson to Joseph Priestley, 1800