Re: how can I keep gnus from Hiding All Threads when open a group

[Harry Putnam]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>> Anyway, I doubt this is the source of the slowdown -- asking Gnus to
>>> display 50,000 messages in a summary buffer is going to be slow no
>>> matter what. Turning off threading altogether should help some, but
>>> that's all I can say.

Harry replied:
>> I do realize that groups of that size are really really slow.
>> I have some experience in that area.
>>
>> However unless the `hiding all threads' message is misleading and
>> doesn't really have anything to do with what is happening, I can see
>> the wait growing longer by watching `hiding all threads'; 1000 ...
>> ...2000 ....3000 the waits between are relatively short but after about
>> 4000 the waits between ouput grow exponentially.  Getting up to 7/8
>> thousand the wait between 7 and 8000 is quite lengthy.  I haven't
>> timed it or I'd post the times.
>>
>> That's what gave me the idea of cutting out the thread hiding.... Also,
>> must of what I do in there means I have to SHOW threads anyway `T S'
>> so I have to undo the Hiding anyway.

[...]

> Okay, well that certainly makes it sound like thread hiding really is
> the source of the problem. It looks like setting
> `gnus-thread-hide-subtree' to nil will do what you want. But I'd

That change to `nil' seems not to have made much (if any )
difference.. In fact I still see that message about hiding threads and
progressivly longer waits.

> definitely recommend playing with `gnus-show-threads' and
> `gnus-fetch-old-headers' (and maybe try `gnus-build-sparse-threads'),
> and see which combination of settings gets you the fastest load.

Haven't tried messing with those yet.  As you might imagine and in
line with your comment below, doing so means lots of wait time chking
if a change in those vars helps. Plus I've kind of gotten past the
necessity.

> No matter what, I think the main recommendation would be to try to open
> group buffers with fewer messages! Is it really necessary to display
> tens of thousands of messages each time?

Ahh yes, and so I have moved past the need to open so many messages at
once, and thereby not facing the threading issues in such a massive
setting.

You see, it seems sort of required (opening many many msgs at once )to
initially get significant amounts of postings accumulated into the
agent to do what I'm really working on.. which is have a large
database of technical information from usenet on disk.  Some kinds of
searching just cannot be done over the internet.

For example; no online provider in there right mind can allow regex
searching.. it would tie them up if used to any extent.

(The effort to acquire postings is preparatory to try to learn how to
use the new search possibilities that yourself, Andrew C and others
have been working on currently as noted in several continuing threads
on `gmane.emacs.gnus.general')

The problem is that if I asked gnus to just download whatever is on
the gmane or enews.newsguy.com spools for a group..., in the most
interesting ones there may be several hundred thousand msgs going back
to early 2000, in some cases I've gotten clear into the late 90s.

So I set about acquiring around 50,000 or so where possible. Ended up
with 8 grps or so, but most are between 10 and 40 thousand with a few
above 70,000.  

Hence the spate of time spent opening many many headers to be able to
mark and download 50,000 or so in bunches of 5,000 or so, and over
time, so as not to get blackballed off gmane for abusive downloading.

Maybe some kind of trickery with agent categories could have made this
a bit or even a lot easier... I just set the predicate to `true' on
all agentized groups and did the downloading manually

I have 38 groups agentized over three servers .. quite minor league
compared to a real news spool...

I ended up with 1,033,195 posts but now only have to keep up with what
is newly posted.