Re: [Gnu-arch-users] tla file-locking (good idea or bad idea?)

[Tom Lord]

    > From: Jeremy Shaw <address@hidden>

    > I would like to add file-locking to tla. Are there any strong
    > objections?  Should it be straight forward? 

Yes.

    > [....] I would like to be able to do
    > something like:

    > tla lock-file debian/changelog

For the first pass, at least, please forget about having a tla command
for this.   Imagine instead:

        % tla-lock-file debian/changelog

Once that works we can talk about adding a command (or not).


    > To lock an individual file. This should prevent developers from
    > commiting changes that touch the changelog, but allow all other
    > changes.

That's mostly fine but consider how it is semi-inconsistent with the
arch model of an archive and raises some intricate semantic issues.

An archive can be shared, over a variety of transports, by many users.

There is an interface abstraction to archives, portable across all
transports, which supports a small number of simple operations.   File
locking simply does not map onto this abstraction in its current form.

The abstract interface to archives is designed so that it can be
implemented on top of very minimal approximations of a "dumb file
systems".  It is quite awkward (at best) to implement file locking on
a dumb file system.

I think you have underspecified the semantics of file-locking.  Are
locks automatically lost when the holder commits?   Are you sure that
that's really the right solution in general?

Can I only lock regular files and symlinks?  or can I also link
directories?  You need to come up with a good answer for that, as
well.

So (not reviewing your reasons for wanting this feature):

1) Make or take a separate facility for locking.

   What's locked?  Presumably either an arch version name or
   revisision name plus an inventory id (not a relative path).


2) Use pre- and post- commit hooks to enforce / cancel these locks.

   You may need some trivial tweaks to what's passed, in the
   environment, to hook scripts.  You may need some help parsing
   changesets.



Now, do you really want this or is there a simpler, better way?:

    > Here's why this could be useful -- We have an autobuilder that can
    > check out a project, build the .debs, and upload the packages and
    > source to our local 'debian' repository. When the autobuilder checks
    > out a package, it automatically generates a new debian/changelog entry
    > in its local copy before it starts the build. If the build is
    > successful, it will commit the debian/changelog. If the build fails,
    > then it will send an email, and leave the debian/changelog alone.

    > The problem is -- what if someone commits a change to the
    > debian/changelog while the autobuilder is building? When the
    > autobuilder finishes, it will get a conflict when it attempts to
    > commit the changelog. There any many 'solutions' to this problem, but
    > they all have various tradeoffs. The solution we like best is to be
    > able to lock just the debian/changelog because it has the following
    > properties:

But already, without file locks, arch will refuse to commit a tree
that is not up-to-date.   So you have the same net effect.


    > (1) The autobuilder can always commit the debian/changelog with out
    >     conflicts. (Some packages take 8+ hours to build -- having to
    >     throw out the whole build simply because the debian/changelog can
    >     not be commited is not acceptable).

    > (2) A developer can almost always commit. It is relatively rare for a
    >     developer to modify the debian/changelog. So, 99% of the time, it
    >     is fine for a developer to commit while the autobuilder is
    >     running. Using lock-revision would create long windows of time
    >     where a developer would like to commit a harmless change, but
    >     can't. A lock-file would only prevent commits the 1% of the time
    >     that it would cause trouble.

    > (3) A developer get immediate feedback as to whether their commit to
    >     debian/changelog is accepted or not.

    > (4) file-locks fit best with the existing autobuilder design. The
    >     current autobuilder is designed to be able to work with multiple
    >     different source control systems, but expects that they will all
    >     have a file-locking command. Perhaps a bad assumption -- but I
    >     didn't write it ;) However, as tla gains popularity, I think other
    >     people will find themselves in the same boat when they try to
    >     migrate their tools from cvs to tla.

    > I have not tried to convince you that file-locks are the best solution
    > to our problem -- that would require several more pages of
    > details. But, hopefully, I have given enough detail to explain why
    > someone might want to use file-locks.

I think you need to use _branches_, not file locks.   I think you need
to step back and take a fresh look at your production pipeline in
light of the many convenient features now available to you.

    > Rephrasing my original questions: (a) if I submitted a (well written,
    > properly implemented) patch would it be considered for inclusion 

All patches fitting that description are considered.

    > (b) does anyone already know, off the top of their head, that
    > implementing > file-locking is actually a very difficult task
    > that should not be attempted in the first place?

Although I've given you a recipe for doing it above, I strongly
suspect that there is a better way to organize your pipeline using
branches.  It's not something I would waste my time doing thinking
that it would be an improvement to arch.

-t