Ben,
Portable (265 Mb .sav source file & delete all but 7 fields, then
save):
> > I should mention that the file was saved in
the > > 'portable' format (9Mb/90mins)
System (same 265 Mb .sav source file):
> > 2) Delete all but 8 fields, reduces .sav file to 4 Mb >
> 3) Save process took approx. 1hr 40 mins (8:40-10:20 pm)
Sorry - please disregard (I can't recall the full details, and probably not
comparable):
> > ... I've since saved the > > 'portable' file
in the 'system' format instead (3Mb/2 > > mins), and the 'system'
version also seems to load > > much more quickly.
Regards,
Dave.
----- Original Message -----
Sent: Tuesday, August 05, 2008 7:57
PM
Subject: Re: PSPP Windows/SET TEMP/File
save times
Now I am confused. You mentioned that portable files
were involved:
>
> I should mention that the file was saved in
the > > 'portable'
format (9Mb/90mins) - I've since saved the >
> 'portable' file in the 'system' format instead
(3Mb/2 > > mins), and
the 'system' version also seems to load >
> much more quickly.
Can you clarify which
actions, on which file formats, took what amount of time? If it is
clear what is slow, and how important that is, then it is clear what parts
of PSPP need work.
Thanks,
Ben.
"David Purdy" <address@hidden>
writes:
> Ben, > > Many thanks - just to note
that this involved a *system* rather than portable > format
save: > >> 2) Delete all but 8 fields, reduces .sav file
to 4 Mb >> 3) Save process took approx. 1hr 40 mins (8:40-10:20
pm) > Regards, > >
Dave. > > ----- Original Message
----- > From: Ben
Pfaff > To: David
Purdy > Cc: address@hidden >
Sent: Tuesday, August 05, 2008 6:12 PM >
Subject: Re: PSPP Windows/SET TEMP/File save times >
> Loading and, especially, saving to the
portable file format is > not well
optimized. It is difficult to do in a fast and
portable > manner, because numbers in portable
files are expressed in an > awkward manner that
is difficult to work with quickly. It's >
quite possible that this is the problem, especially given
that > the system file format worked much
faster. > > "David
Purdy" <address@hidden>
writes: > > >
Correction: a file save with SET TEMP defined in WinXP has produced
no > > material improvement in lapsed time
over a comparable file save without >
SET > > TEMP defined (>90 minutes to save
9 Mb; see earliest message): > >
> > 1) Source SPSS .sav file 417 fields x
510,121 records = 265 Mb > >
> > 2) Delete all but 8 fields, reduces .sav
file to 4 Mb > >
> > 3) Save process took approx. 1hr 40 mins
(8:40-10:20 pm) > >
> > PC = WinXP Pro SP3, 78 Gb hard disk
space available, 2048 Mb RAM (Dell > >
Dimension 8300 3.2GHz Pentium 4) > >
> > Regards, >
> > >
Dave. > > >
> ----- Original Message ----- > > From:
unknown-1 > > To: David
Purdy > > Cc: address@hidden >
> Sent: Sunday, August 03, 2008 9:37 PM >
> Subject: Re: 0.6.0-10 PSPP windows - Queries
(WinXP) > > >
> [cut] >
> > > 2) Quicker
'Save' - I've had some mixed results, maybe influenced
by > > working with a
large file (>500,000 records). After
trying >
something > > similar
to the very slow save reported earlier (after deleting
all > but >
> about 7 fields), I got the impression that the
file was being > created
much > > more
quickly. > > >
> Nice that is what I expected from using the temp
variable/directory. > >
> > [cut] >
> > > -----
Original Message ----- >
> From: David Purdy >
> To: address@hidden >
> Cc: Ben Pfaff
[PSPP] > > Sent:
Friday, August 01, 2008 11:08 AM >
> Subject: Re: 0.6.0-10 PSPP windows - Queries
(WinXP) > >
> >
Ben, > >
> > Many thanks for
the feedback. > >
> > > 1) Very
long file save time ? >
> >
> I was using the GUI version - the disk didn't
seem to be especially >
> active, nor PSPP using a lot of CPU time (I
think). I should >
mention > > that the
file was saved in the 'portable' format (9Mb/90mins) -
I've > since >
> saved the 'portable' file in the 'system' format
instead (3Mb/2 > mins),
and > > the 'system'
version also seems to load much more quickly. >
> >
> [cut] >
> >
> ----- Original Message
----- >
> From: Ben
Pfaff >
> To: David
Purdy >
> Cc: address@hidden >
> Sent: Friday, August 01,
2008 4:52 AM >
> Subject: Re: 0.6.0-10
PSPP windows - Queries (WinXP) >
> >
> "David Purdy"
writes: >
> >
> > 1) Very long file
save time ? - I'm using a large third party >
SPSS >
> .sav
file >
> > (417 fields x
510,121 records = 265 Mb), but am only >
interested in 7 >
>
fields. >
> > To ease the
processing load, I deleted the unwanted fields
in > PSPP, >
> but
the >
> > resultant .sav file
took over 90 minutes to save (9 Mb).
Are > there >
>
any >
> > settings that could
be adjusted to reduce the file save time ? >
> >
> Wow! That's a long
time. I don't know, off-hand, why it
would >
> take that long. Is
the disk active during that time? Is
PSPP >
> using a lot of CPU
time? Is this in the GUI or the
command-line >
> syntax
version? >
> >
> Increasing the workspace
to a value larger than the size of your >
> file might help, e.g. SET
WORKSPACE=300000. >
> >
>
[cut] > > >
>
_______________________________________________ >
> Pspp-users mailing list > > address@hidden >
> http://lists.gnu.org/mailman/listinfo/pspp-users >
> -- > "I knew it
was a bad sign when I drove through Hell (MI) on
Wednesday > and there was snow on the ground and
all the standing water had a > layer of ice on
the surface... Yes folks, Hell has frozen
over > and my cable modem *still* wasn't
working..." --Ed Glowacki >
-- "A computer is a state
machine. Threads are for people who cant [sic] program state
machines." --Alan Cox
|