Re: Doc build hanging (with memory leak?)

[David Kastrup]

Joseph Rushton Wakeling <address@hidden> writes:

> On 14/05/12 09:46, David Kastrup wrote:
>> We don't have a canonical developer, one whose personal
>> branch/repository would be official for the project.
>
> GitHub and Launchpad both permit branches to be owned by groups as
> well as individuals.  I'm sure other DVCS-based code hosts do as well,
> but those are the two I'm most familiar with.

Does not change the problem of coordination between individuals.

>> We have automated regression testing taking a _lot_ of computation
>> time.  We have a history of our canonical repository _breaking_.  We
>> have a total dearth of high-level developers, and a number of
>> volunteers at lower level.
>
> What's the particular problem here -- is it that the regression tests
> are too heavy for users to run on their own machines prior to
> submitting a merge request, or is it that they can't be trusted to run
> them?

Both.

>> If you compare the complexity of the code base and infrastructure to
>> the available manpower and skill sets, the logical verdict is death
>> by bitrot.  The red tape is doing a surprisingly good job at keeping
>> the project from rusting in its tracks in spite of the large mismatch
>> between project complexity and active workers.  Partly because
>> sufficiently hardened red tape can be replaced by automatic
>> procedures.
>>
>> We have a fine-grained mesh of regression tests where _lots_ of stuff
>> gets caught before making it into the code base.
>
> Don't get me wrong.  I'm not arguing against regression tests or
> careful review -- I strongly support them.  My objection is that the
> way it's been implemented is unnecessary complicated:
>
>   -- you ask me to install a custom program, unique to LP, just in
>   order to upload my code to your testing site.  Why not let me upload
>   to a repo of my choice (GitHub, Gitorious, Bitbucket, Google Code
>   ...) and just submit to your testing system a repo location and
>   branch name from which it can pull?  Come to that, why can't I just
>   manually upload a patch file like the one I emailed to the list?  Or
>   create and push to a personal repo on the testing system?

Have you actually used Rietveld for reviewing code?  It does a job that
the mentioned tools don't.

>   -- this custom-built program can't readily handle git branches other
>   than master.  Yes, I can use the SHA1, but that's finnicky on the
>   user side, an unnecessary complication.

You can use any branch name for "git cl upload".  Using stuff other than
master has the disadvantage that the diff might not apply for the
centralized regression tests.

>   -- the custom-built program asks for my Google ID and password.  Can
>   I be sure that it handles them securely?

Read the source.  Or don't use it.  You can upload without using a
custom tool IIRC.  It will just cause a lot of unnecessary work, and you
are complaining about unnecessary work.

>> Yes, I have bursts of creativity where I bypass proper procedures for
>> well-chosen subsets of patches in order to avoid getting into a
>> tangle of diverging commits.  But that is an exception to the rule.
>> The "norm for DVCS-hosted software projects" is commit first,
>> question later.
>
> That's not my experience of DVCS-based projects.  It's perfectly
> possible to run a contributor's merge request through a test suite
> before accepting it, or to request them to ensure the test suite
> passes before submitting such a request. What I don't see is that it's
> necessary to entangle the technical means for submitting (and
> tracking) the merge request with the technical means for applying the
> test procedure -- or to require project-specific programs to do
> either.

I think you are not clear about the number of merge conflicts such a
style would cause when using LilyPond.  It is bad enough as it is.

>> We don't have the amount of qualified cleanup personnel to deal with
>> this style.
>
> I'm not proposing "commit first, question later".  I'm saying that the
> process of _submitting_ a patch or merge request should be trivial.

It is.  Have you actually tried it?

> There also has to be some common sense involved.  My _real_ objection
> here is that a tiny, spur-of-the-moment contribution done as a casual
> favour to the project was greeted with a request for me to go through
> a complicated submission procedure.

Small contributions tend to get registered by the bug squad and work
their way through the queue from there.  If you are interested in better
ability to oversee your contribution's progress, it makes sense looking
at the tools yourself.

> It's a trivial documentation patch and it's hardly onerous for a core
> developer to check that it's OK and just add it in.

That's what happens when something gets picked up by the bug squad and
entered as an issue.  But there are very few "core developers" to go
around, and their status of "core developer" is not because they tend to
idle around and wait for the chance of coaching spur-of-the-moment
contributions into the proper form.

There are reasons the show is run in the manner it is, and the reasons
are not just because everybody except yourself is an idiot and just
waited for you to tell them how to make everything simple and smooth.

I could be malicious and offer you the responsibility to make everything
better, but the fact is that I am not in the position to even make such
an offer, and a lot of hard work and discussions have gone into working
with the current system, and not all of the reasons are strictly
technical but are also done in order to utilize the distribution of
talent available for LilyPond best.

There is no sensible way around learning the ropes before attempting to
replace them.  Certainly people would welcome a transition to systems
better suited to git.  But your hand-waving proposals not even include a
substitute for the Rietveld vetting system (like, say, Gerrit) including
organizing the hosting for it, let alone adapting the existing workflow
and toolchains.

-- 
David Kastrup