Re: #852 - stop google from indexing old docs

[Mark Polesky]

Graham Percival wrote:
> True, although it would be simple to have
> doc/stable/
> doc/devel/
> doc/archive/

That's better.

> Oh, we'll keep the stable docs.  I just don't think the
> old *unstable* docs are worth having online.

Is it really such a burden to keep the old docs on the
lilypond.org server?  Why the desire to remove them?  It's
really not a huge thing to me, so I don't want to argue too
much about it.  Just curious.

Also, I looked at the new robots.txt and I have one comment.
These lines are all unnecessary (redundant):

Disallow: /doc/v2.0/Documentation/user/out-www/lilypond/source
Disallow: /doc/v2.0/Documentation/user/out-www/source
Disallow: /doc/v2.2/Documentation/user/out-www/lilypond/source
Disallow: /doc/v2.2/Documentation/user/out-www/source
Disallow: /doc/v2.4/Documentation/user/out-www/lilypond/source
Disallow: /doc/v2.4/Documentation/user/out-www/source
Disallow: /doc/v2.5/Documentation/user/out-www/lilypond/source
Disallow: /doc/v2.5/Documentation/user/out-www/source
Disallow: /doc/v2.5/Documentation/user/out-www/music-glossary/source

...as long as the following lines are there:

Disallow: /doc/v2.0/
Disallow: /doc/v2.2/
Disallow: /doc/v2.4/
Disallow: /doc/v2.5/

- Mark