Re: how can I keep gnus from Hiding All Threads when open a group

[Harry Putnam]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Harry Putnam <reader@newsguy.com> writes:
>
> [...]
>

>>> No matter what, I think the main recommendation would be to try to open
>>> group buffers with fewer messages! Is it really necessary to display
>>> tens of thousands of messages each time?
>
> [...]
>
>> (The effort to acquire postings is preparatory to try to learn how to
>> use the new search possibilities that yourself, Andrew C and others
>> have been working on currently as noted in several continuing threads
>> on `gmane.emacs.gnus.general')
>>
>> The problem is that if I asked gnus to just download whatever is on
>> the gmane or enews.newsguy.com spools for a group..., in the most
>> interesting ones there may be several hundred thousand msgs going back
>> to early 2000, in some cases I've gotten clear into the late 90s.
>>
>> So I set about acquiring around 50,000 or so where possible. Ended up
>> with 8 grps or so, but most are between 10 and 40 thousand with a few
>> above 70,000.  
>>
>> Hence the spate of time spent opening many many headers to be able to
>> mark and download 50,000 or so in bunches of 5,000 or so, and over
>> time, so as not to get blackballed off gmane for abusive downloading.
>>
>> Maybe some kind of trickery with agent categories could have made this
>> a bit or even a lot easier... I just set the predicate to `true' on
>> all agentized groups and did the downloading manually
>>
>> I have 38 groups agentized over three servers .. quite minor league
>> compared to a real news spool...
>>
>> I ended up with 1,033,195 posts but now only have to keep up with what
>> is newly posted.
>
> Yeesh... At this point it might make more sense just to install your own
> local LeafNode server and do nntp yourself, don't you think?

Yes, my thoughts exactly... I was just studying up on slrnpull.
Something I used a bit long ago... never really fiddled with
leafnode.. maybe its a little simpler?

The stumbling block with either is probably how to slurp the database
gnus has already generated.

But also, how would I fit news pulled that way, into the search
features being coded in gnus lately?

> I'm sorry I can't speak to your specific questions about the agent -- I
> only use the agent with my gmane backend (all the other backends are
> local), and there I just agentize the whole server, I haven't tried
> anything finer-grained than that.

I'm sorry to be bending your ear so extensively but not getting much
input on ... gnus.user

Perhaps if you knew the depths of my ignorance you might know you
probably can fill in a few things.

Do you know if there is supposed to be (in orginal intent) a
difference between having groups under an agentized server... and
having not just the server agentized but also specific groups
agentized under that server as well?

I do know that the list of agentized groups (by category) in
News/agent/lib/categories does not appear to include any of my gmane
groups that have not been agentized, even though the gmane server is
agentized.  Not sure one could even use a category if the group itself
was not agentized.

For example... With the server agentized...

Do you use either `J s' (gnus-agent-fetch-session)
  (Fetch all articles and headers that are eligible for fetching)

  OR
                   `J u (gnus-agent-fetch-groups N)
   (Put all new articles in the current groups into the Agent)
   
in group buffer, after a scan for new news, to download the new
messages?

The `J u' doc string is pretty easy to understand ... all NEW msgs in
CURRENT group

But for some reason Larsi (who writes really good documentation) or
someone else felt it was necessary to use a slightly different
description for `J s'

   'articles and headers that are eligible for fetching'

I'm guessing the description uses different language because the action
is different in some way... That is, not just `New' msgs but maybe
something more inclusive.  Maybe something like all un-downloaded msgs
that conform to a specific category, which might be something of a
disaster in groups with well over 100.000 msgs that have not been
downloaded yet.

I ask because the times I tried `J s' or `J u' (Even on just the
current group) it produces a massive lengthy read of something.. I'm
not sure if the visible count being shown is characters or lines but
it gets up to over 120,000 eventually but I've never let it
finish... I was afraid it was trying to start from the very oldest
posts on the gmane spool and start downloading from there.

Which might be a good way to loose access to the new gmane.

I'd like to know what is being read in that case above?

You say, you do nothing finer grained with the agent than an agentized
server.. so do you mean that you download all groups you have
subscribed to on that server?

Unless you only subscribe to a few groups, that seems like it would
become a very lot of messages at some point.

How do you handle the downloading? In terms of cmds used?

Do you do batching to keep up with new posts and actually go unplugged
at times?

I noticed in my one attempt (recently) at using the batch command
shown in the manual:

     You can run a complete batch command from the command line with the
     following incantation:

     #!/bin/sh
     emacs -batch -l ~/.emacs -l ~/.gnus.el -f gnus-agent-batch >/dev/null 2>&1

that gnus started downloading from groups under an agentized server
that were not themselves agentized as well as groups that were.

I ask all this because it is really time consuming and somewhat
difficult to set up real experimenting to discover the basic actions
one should expect when using the agent.