Re: CVS diff and unknown files.

[Sergei Organov]

Paul Sander <address@hidden> writes:
> On Jan 31, 2005, at 9:30 AM, address@hidden wrote:
[...]
> > Don't you wish to even try to understand the suggestion? The suggestion
> > is: invoke *the same triggers* both at 'cvs add' and 'cvs commit' time,
> > -- there is no need for separate triggers.
> 
> I understand it perfectly. What you don't seem to understand is that
> the set of possible actions taken at add time is different from those
> actions taken at commit time.

Triggers and actions taken aren't the same thing. I fail to see why the
same trigger can't take different actions.

> These two sets cannot simply be combined and run at both times.

You already said that add-time checks is subset of commit-time checks.
Just bypass some tests if the triggers are invoked at add-time.

> 
> >> That's not to say that we will *always* know at add time that the
> >> commit will fail; failures can occur due to problems in their content
> >> which are clearly not ready to check at add time.
> >
> > Well, if I understand correctly, your intentionally want to have weaker
> > checks at add-time than at commit-time. Instead, you can do it in the
> > commit-time trigger by skipping some of the tests if the file in
> > question is new one.
> 
> Oh, so what you're saying is that rather than making CVS differentiate
> between the two sets of triggers, you want the commitinfo script to do
> it instead. I suppose that's doable.

Exactly! I took too much time to explain this. Should be either my or
your fault (or both).

> But I'll counter with this: Why not combine commitinfo, loginfo, and
> taginfo the same way? My argument is that add-time triggers differ
> from commitinfo triggers in the same way that commitinfo triggers
> differ from those others: They run at different times for different
> purposes.

All the triggers you've mentioned have no common tasks to do. On the
other hand, add-time checks you have in mind are subset of commit-time
checks you have in mind. The subset is easily achieved by bypassing some
of checks.

I would have no objection to add-time-script if there were a separate
server operation called "add-file-to-the-working-copy", or
"check-if-file-name-is-ok-for-repository", but AFAIK there is currently
no such operations. I consider inventing new server operation just to
implement some policy that you have in mind and that could be
implemented using existing functionality anyway to be an overkill,
sorry.

> >>>> What's new is that we can cause "cvs add" to fail under certain
> >>>> conditions that we know in advance will cause commitinfo to refuse a
> >>>> file. To make sure that the condition is checked, we do it a second
> >>>> time before commitinfo, because some users insist on defeating it the
> >>>> first time.
> >>>
> >>> What functionality is lacking? "cvs -n ci" allows you to do it. All you
> >>> require is "cvs add" to *enforce* the checks. I have two levels of
> >>> objection here:
> >>>
> >>> 1. Enforcement by "cvs add" of the checks doesn't require introduction
> >>>    of separate "add triggers" as usual "commit triggers" could do the
> >>>    job as well.
> >>
> >> My argument is that it does not in fact "do the job as well." I've
> >> given a number of reasons why in earlier messages.
> >
> > No, you didn't. Or at least I somehow missed them. The fact that
> > currently CVS doesn't invoke any triggers at add-time has nothing to do
> > with my objection. The question is what is so different about add-time
> > triggers from the point of view of *server* so that it's not enough to
> > just invoke commit-time scripts at the add-time?
> 
> You need to get out of the server. The server's not the one we're
> trying to make happy here. It's the user that matters, and the tools
> must bend to the users, not the other way around.
> 
> From an implementation standpoint, there's no difference between the
> triggers, other than the conditions in which they fire and the files
> in which they're configured. Thus the argument you make in the above
> paragraph simply makes no sense.

It's you who needs to get out of the server, not me. You propose server
changes, not me. If the functionality has nothing to do with the server,
then get out of the server indeed.

> Triggers certainly don't have full control over the user's workspace.
> But they can decide what interactions with the server are acceptable
> to the server's admin. That is why they're good as enforcers.

That's why they are good as enforcers of things that happen on the
server. They (fortunately) have absolutely no control over the
user's workspace.

> Wrappers don't have that capability, because they have no way to be
> final authority to decide what is acceptable to the server.

Yes, wrappers don't have that capability, enforcing things on the
server, but unlike server-side triggers, they do have capabilities (to
the level your favorite OS allows you) over the user workspace.

CVS client commands are in fact just wrappers above the CVS
client/server protocol. If you believe wrappers don't have policy
enforcement capabilities, then your attempts to implement this
capability in CVS client commands suffer from the same deficiency,
sorry.

> >> If I do in fact understand you to be recommending that commitinfo be
> >> used at add time, then I must disagree. Triggers used at commit time
> >> to check the content of files (like Greg's RCS Id trigger, for
> >> example) are not appropriate to use at add time. Things like checking
> >> that the user has the right to add a file, or checking that the name
> >> of the file complies with policy, are legitimate to check at add time.
> >> They can also be checked at commit time, and the current proposal does
> >> that because the add-time triggers are optional.
> >
> > So the only thing you need to know in the commitinfo is what user is
> > trying to do and decide what checks are required based on this
> > information. If commitinfo currently doesn't have all the required
> > information, maybe it's better to fix it instead of inventing yet
> > another hack in the form of add-time triggers?
> 
> What you're really proposing is an alternative implementation for add-time
> triggers. Think about it.

It's how it looks from your point of view. From my point of view I
propose alternative implementation of the functionality you need without
inventing anything new, being add-time triggers or something else.

What you suggest is yet another reincarnation of commit-time triggers
"run at different time" where "time" is defined in terms of operations
on the working copy. Think about it.

> >> If users think the wrappers are in their way, they'll drop down to
> >> CVS to add the file, causing more significant breakage later.
> >
> > Either trust your users or arrange things so that it won't be possible
> > for them to invoke cvs commands directly. But even if you choose the
> > latter way, those that do think the wrappers are in their way will
> > probably find a way to bypass them anyway.
> 
> First, I've already related a horror in which users were trusted and
> failed. This is why we need safeguards on the circumstances in which
> they can perform certain actions.

They will fail no matter what you do then.

> Second, if a wrapper invokes CVS, then there's no way to effectively
> hide CVS from the users.

You like to bring wrong statements into discussion very much. If you
don't know a way, it doesn't necessarily mean there is none.

> This is why policy must be enforced below the level of the CVS command
> line.

CVS command line tools are no more than wrappers on top of CVS
client/server protocol. They can't enforce anything that the protocol
itself can't enforce. In turn, the protocol can only enforce what
clients can or can't do *to the server*.

You seem to be thinking that in the client/server model the server is a
boss. The reality is closer to the opposite. In fact it is client who is
boss. It commands what server should do, and then servers does (or
doesn't) what the boss have told him to do. There is no way for server
to force client to do (or not to do) something.

> It's also an important reason why the client/server paradigm is
> so powerful: It removes sensitive parts of the application from the
> user's domain, assuming the server runs on a machine other than the
> user's workstation.

If you indeed want to have server control over the user working
directories, it's server who must manage the directories, not user OS.
Otherwise server has no capabilities to enforce any policy on the user
working directories.

It's the fact that repository is under direct control of the server that
allows server to impose policies on the repository, it's the fact that
working directories aren't directly managed by the server that makes
your proposed design inherently broken, sorry.

> >> It's at that time that they finally learn the value of the process.
> >
> > If somebody prefers to learn on his own mistakes, -- let him do it
> > unless it doesn't break others work.
> 
> Do you really mean, "let him do it unless it breaks others work"? How
> do you prevent them from making mistakes that harm others?

By not allowing him to break the repository, obviously. That's the goal
of server-side policies, -- prevent users from breaking others work. You
just can't prevent them from doing their mistakes in their working copy,
for example, one can run 'rm -rf' in his working copy after two days of
active development.

> They won't know which mistakes are harmful until they try them.

Yes, they will learn anyway as soon as they try to commit their changes,
i.e., as soon as they try to break others work.

> That's exactly the point of enforcing policy: It allows the entire
> project to learn from the mistakes of others so that when
> inexperienced users repeat past mistakes, everybody is protected.

You can't force nobody to learn unless she wishes to learn. You can't
protect user from himself. Either the user wishes to know if she adheres
to policies, in which case she will run whatever commands you tell him
early, or the user doesn't wish to know, in which case you has very
little to do about it.

> > [...]
> >>>> BTW, the definition of "client/server" implies the presence of a
> >>>> network for communication between the two parts,
> >>>
> >>> Wrong. Client and server can work on the same computer without any
> >>> network. Moreover, client/server idiom is heavily used inside
> >>> "monolithic" programs as well. Basically, client/server only means that
> >>> there is some "server" that makes some operations on behalf of
> >>> "clients", how clients communicate with the server is a secondary issue.
> >>
> >> While your statement is true, the fact is that client and server
> >> programs typically operate on different machines. The fact that they
> >> do is what gives the client/server paradigm its power. Sure, I can
> >> write client/server programs that use loopback interfaces, Unix domain
> >> sockets, or even named pipes, but they offer no benefit because
> >> function calls are more efficient RPC calls.
> >
> > Don't you aware that client/server paradigm is widely used inside
> > "monolithic" programs as well? There are always things left to be
> > learned in this world.
> 
> Frankly, I don't care.  And it's not relevant to the topic at hand.

No, it's very relevant to the topic at hand. It's your ignorance of the
true meaning of the client/server model that makes you believe
everything is fine with your proposed design. The client/server model is
mostly about splitting of responsibilities, not about media through
which client and server communicate.

[...]
> > Well, you've asked "why would you possibly want to deliberately
> > proceed toward a dead end" and I tried to provide an example when
> > the end is not actually dead. The problem is that unless I add the
> > files, 'cvs diff' doesn't work on them, and if your proposal is
> > implemented, and I bypass the add-time checks, I have no way to
> > repeat them later without committing the files.
> 
> Okay, so when you say "I need the files to be added to the working
> copy anyway" what you really mean is that you an entry in the CVS
> metadata so that you can do other stuff that won't work without it.
> Fine, my proposed add-time trigger implementation allows for this.

*How do I repeat skipped checks later with your proposal?* I already
tired asking this same question again and again just to get no answer.

Well, this time I've found an answer below, the answer is "run 'cvs -n
commit'", but it doesn't fit into your model. That's what I'm
advocating, -- you don't in fact need anything else but "cvs -n commit"
to let user know ASAP about possible problems.

> Here's another potential benefit: Most projects have some criteria
> that submitted patches must have before they will be accepted. These
> criteria might include naming conventions (e.g. all files named in
> lower-case). If you intend to send a patch to the project, there's a
> greater probability that it will be accepted the first time if you
> subject yourself to some of their policies.

Then I need full commit-time checks, not your limited add-time checks.
Better spend your brain forces on improvements of the former than on
invention of the latter.

[...]
> >> See above.  The sets of add-time and commit-time triggers are
> >> different.
> >
> > For the policy you have in your mind they are different, for a policy
> > somebody else needs they are the same. Why do you need to invent new
> > thing, add-time triggers, if commit-time triggers can do the job
> > provided they have enough information to make the right decision?
> 
> In my experience, the two sets are different in the general case.
> In those limited situations where both sets are the same, install the
> triggers for add-time execution and don't bother with commit-time
> triggers (because add-time triggers also fire at commit time).

In your situation where add-time checks don't match commit-time checks,
just don't do some of the checks at add-time. However, if you still
insist on the "the sooner we tell user something is wrong, -- the
better", the checks must be the same.

> This is a complementary view of my proposal: Have a set of triggers
> that runs at add and commit times, and another set that runs only at
> commit time. It so happens that the second set is already implemented.

There is actually no sense to make them different if you indeed insist
on informing users about problems ASAP.

> Invoking commitinfo and passing an argument to distinguish between add
> time and commit time, which seems to be what you have in mind, is an
> alternative implementation.

Yes that's what I have in mind.

> I think mine is better, and it certainly has fewer backward
> compatibility issues.

Backward-compatible wrong design is not necessarily better than a sane
one even if it's not backward-compatible, though I think there is still
a way to keep backward compatibility in my case either.

[...]
> > Ah, so the sole purpose of invention of add-time triggers is in fact an
> > attempt to don't break compatibility with the current commit-time
> > scripts, right? If so, it's better to explicitly state it to avoid
> > misunderstanding similar to mine.
> 
> I thought that was exposed in an earlier message.  Sorry for not making it
> clear.

No, there was no even such a word, "compatibility" in your earlier
messages.

If I propose a way to implement my variant and make it
backward-compatible, will you agree that mine is better? My
understanding is that no, you won't. So compatibility is not in fact the
main point of our disagreement.

[...]
> > Seriously, think about what you're saying.  You want to do the following,
> 
> >>>> dramatically oversimplied:
> >>>>
> >>>> edit foo.h
> >>>> cvs add foo.h
> >>>> edit foo.h
> >>>> cc FOO.C
> >>>> cvs commit foo.h
> >>>>
> >>>> It's at this point you want to fail, after you've done all the
> >>>> work. In my opinion it's better all around you fail after two
> >>>> steps, not five.
> >>>
> >>> You've missed my point. I have nothing against step2 warns me about the
> >>> problems, maybe even by default, if I still have ability to suppress the
> >>> warnings. I'm strongly against step2 to refuse to add the file as I
> >>> believe it would just disturb instead of help: if step2 refuses to add
> >>> the file, it doesn't prevent me from doing steps 3 and 4. In fact it
> >>> even doesn't prevent me from invoking 'cvs commit foo.h', but instead of
> >>> getting comprehensive explanation from the server why my change isn't
> >>> accepted, I'll get less useful message "run cvs add first".
> >>
> >> Are you saying that if step 2 fails, you would proceed with steps 3
> >> and 4 and maybe attempt 5 without correcting the failure condition?
> >
> > I said what I said, your suggested _failure_ to add the file to the
> > working copy doesn't prevent me from doing the rest of steps. It means
> > that it's not any better than _warning_. And I even showed why it is
> > slightly worse.
> 
> Well, what do you expect if you ignore the warning in step 2?

With your proposal? I expect your "enforced policy" to somehow prevent me
from continuing my way to the "dead end". You claimed you will be able
to enforce your favorite policy on my working copy. You failed.

[...]

> > If all the user always did only right things, why in the world would you
> > need those add-time checks in the first place?
> 
> If users only did the right things all of the time (where "the right things"
> are defined here as being in compliance with all of the project's policies)
> then there would be no need at all for triggers.  The fact is that user's
> don't always do the right things, so safeguards are needed to coax them back
> onto the path of truth and righteousness.

And you've failed to force me back onto the path of truth with your
proposals due to the fact that you still fail to understand, -- server
has no control over user workspace, so server-side triggers can't force
any policy w.r.t. user workspace.

> >>>> Sure, you can change your behavior and move the add down to right
> >>>> before the commit, and that is consistent with the working styles
> >>>> of many people, but then they get what they ask for.
> >>>
> >>> So even with you proposals implemented you fail to actually impose
> >>> policies you are trying to impose with the proposals, -- too bad.
> >>>
> >>> That's another thing I'm trying to explain, -- you have no way to
> >>> actually impose policies on the client side. Thinking otherwise is no
> >>> more than self-delusion.
> >>
> >> Well, that's what I get for pandering to the crowd that thinks that
> >> they simply must be able to add a file while working offline. If an
> >> add-time connection to the server were a requirement (i.e. there were
> >> no feature to opt-out of add-time triggers) then this wouldn't be a
> >> problem. Don't blame me for trying to meet your requirements.
> >
> > I don't. I blame you for the failure to meet your own requirements. Even
> > if you make invocation of triggers by "cvs add" an absolute must, the
> > users will simply don't run "cvs add" for days until they need to commit
> > their changes due to your own expectations of your users behavior.
> 
> If they're breaching policy in their file additions, then because the
> changes will ultimately be rejected, it's not in their best interest
> to delay the discovery of the violations.

If it's not in their interest then warning is more appropriate than
failure to complete operation, I believe. My whole point here is that
failure to add the file doesn't prevent your users from misbehavior any
more than a warning, so why failure?

[...]

> >> and running commit-time triggers at add-time is not appropriate.
> >
> > Why? Because it doesn't match the policy you have in mind? Then change
> > you commit-time triggers so that they don't do some tests at add-time.
> 
> No, it's not because the triggers don't match the policy that I have
> in mind. It's because there are certain tests that simply are not
> appropriate to perform at add time. Scanning the contents of the new
> files, for example, are not appropriate.

No, it is appropriate. You said yourself many times that you need to
check everything ASAP. If the file has contents, why don't you check for
its validity?

> Software development processes that have higher maturity in the
> capability maturity model might gather metrics at commit time and not
> at add time. There are lots of things that might be done, and rather
> than trying to make them look the same to CVS while letting them
> resolve themselves really does nothing other than to make writing them
> harder.

I'm afraid you still have no clear idea about splitting of
responsibilities between client and server due to your limited
understanding of client/server model, so your attempt to makes things
better could well make them worse.

> >> Like I said in another post, after the commit has completed, the
> >> effects of the two implementations are identical.
> >
> >> What we're arguing about is what's the right way to make the journey
> >> to that point. My way has merit.
> >
> > Sorry, but the only real merit I guessed is backwards compatibility with
> > current commit scripts. Anything else?
> 
> Backward compatibility is a good one.

My solution could be made backward-compatible as well.

> The scripts will be easier to write if the two sets are kept separate
> and invoked separately, also.

No, they will be easier to write only when an administrator will try to
implement a policy similar to those you have in your mind where the
checks are different. As I believe the checks must be the same, I in
fact like it very much that implementing inherently broken policies is
slightly more difficult, -- it will force admins to think twice before
actually doing that.

> Mark also mentioned some good ones.

Sorry, I've somehow missed that. Care to repeat or give a reference?

[...]
> >> You've admitted that you're not familiar with the CVS design.
> >> Please don't argue issues of complexity in a vacuum.
> >
> > Yes, I'm not familiar with the CVS design, hopefully though it's not
> > that broken that introducing additional feature is less complex than
> > tuning an existing one to meet new requirements.
> 
> The two implementations are equally complex.

Resulting application complexity is what actually matters in the long
term, not the complexity of turning current implementation to the new
one.


[...]

> >> I hope this convinces you that appropriate checks should be made at
> >> appropriate times. Many commit-time checks are not appropriate to make
> >> at add time.
> >
> > No, you didn't. Why? Because when I try to stay on your point of view
> > "the sooner user is notified about the problems, -- the better", I
> > immediately see that the example with your add-time triggers fails to
> > notify the user at add-time that his file doesn't contain valid RCS Id.
> 
> That's because I don't believe that that specific test is appropriate
> to make until commit time. After all, in my examples, the user hasn't
> even edited the new file yet, so why would it have a valid RCS ID in
> it?

So the rule "the sooner, -- the better" doesn't apply anymore? Cool! It
makes me believe that the policies you are in favor of aren't carefully
thought about in the first place.

Anyway, if you believe the check for RCS ID shouldn't be done for empty
files, that is a reasonable thing, just don't check for RCS IDs in empty
files.

> It seems that you're inexperienced at that aspect of SCM process
> design. Over the course of the development effort, there are certain
> gateways and it's up to the policy makers to decide where they are and
> what the cost of admittance should be. Every project is different, so
> these qualities will differ. I'm describing mine, and yours are
> obviously different.
> 
> I'll call your approach the "theatre" approach where you buy your
> ticket at the first gateway, and once you're in you can go anywhere.
> I'll call mine the "tollway" approach where you pay as you go.
> 
> >>> You've just explained what I already understood. That's quibbling.
> >>> Are you going to answer to my direct question: how do I repeat the
> >>> "run add-time trigger" part? No way?
> >>
> >> How do you run commit time triggers? If a commit failed, you retry the
> >> commit, right?
> >
> > If I wish to check for commit to succeed I can do it as many times as I
> > like using 'cvs -n commit', right?
> >
> >> If an add-time trigger fails, fix the error and retry the add. How
> >> simple is that?
> >
> > The question is not what I do if add-time trigger fails, the question is
> 
> > how do I repeat the checks later if it succeeded?
> 
> Yes, you can run "cvs -n commit" as many times as you like.  Hopefully you
> won't have to run it at all.
> 
> 

[...]
> >>> And that's one of problems with your proposed add-time triggers
> >>> solution. The user needs a way to invoke later the checks "cvs add"
> >>> didn't do in this case. That's the scenario where user needs separate
> >>> "add file to working copy" and "check file for validity against
> >>> repository" actions. Every time I ask you how do I repeat the add-time
> >>> check, you don't give satisfactory answer.
[...]
> > I want to know how do I invoke the checks that have been skipped during
> > off-line work when I go on-line?
> 
> cvs [-n] commit

Cool! So that's what "cvs add" in fact didn't do when I worked off-line?
This fits perfectly into my model! That's exactly my point, -- if any,
"cvs add" should perform the same checks "cvs -n commit" does!

In your model though, "cvs add" does another set of checks, and you
still fail to tell me how do I repeat *exactly the same checks* later.

> >> Add-time triggers would work similarly. The scripts would be invoked
> >> by the "cvs add" command before the Entries file is edited. If it
> >> fails, the user is notified. To re-run the add-time trigger, the user
> >> attempts another add. But then there's the opt-out requirement, so the
> >> trigger can be defeated at the behest of the user.
> >
> > So, similar to commit, when working off-line I should invoke "cvs -n
> > add" to be able later, when I go on-line, to invoke "cvs [-n] add" once
> > again to invoke the checks? But the whole purpose of invoking "cvs add"
> > is to add files to the working copy, so "cvs -n add" has no sense at all
> > when I'm working off-line.
> 
> That is true. It would fail anyway because it won't add a file that
> already has metadata. You should use "cvs [-n] commit".

Cool! That's exactly my point! "cvs -n commit" is what should be
invoked to perform the checks, either explicitly by the user or
automatically by the "cvs add".

> >> So the add-time triggers run a second time when the "cvs commit"
> >> command is invoked, before the commitinfo scripts run. I would
> >> argue that they should run even before the commit message is
> >> edited. If it fails, the user is notified. To re-run the add-time
> >> trigger from this point, the user would attempt another commit.
> >>
> >>> Besides, I still don't see why add-time triggers != commit-time
> >>> triggers. I.e., provided they are run on server, how they are different
> 
> >>> from the point of view of the server?
> >>
> >> They're not. Triggers are triggers are triggers, and they're all
> >> treated the same way by the server. It's just a question of which ones
> >> run in what order at what times.
> >
> > So from the point of view of the server they run at different times? No,
> > I think they will be invoked by different server commands, right?
> 
> Correct, different sets of triggers run as the result of invoking different
> commands.
> 
> >>> Well, suppose you are designer and I'm a user then. I, the user, ask
> >>> you, the designer, to explain me why do you think I, the user, will
> >>> never need plain and simple "add new file to the working copy" user
> >>> operation.
> >>
> >> The answer is: You can add anything you want to your workspace, but if
> >> you intend to commit it then you must comply with the design. And I
> >> will tell you at the moment you declare your intent if I think you're
> >> not in compliance.
> >
> > Well, what is the command to just add the file to my working copy
> > without intent to commit it? Please don't tell me there is one as then
> > you agree that "add new file to the working copy" is a useful user
> > operation. Please don't tell me there is none as I need it indeed.
> 
> I have never disputed that "add new file to the working copy" is a
> useful operation.

Really?! What then did you mean here when you've answered YESSSSS here:

/begin quote
me> Ah, now I see. I suggest "add new file to the working copy" to be a
me> useful user operation, and you believe it is not? So the minimum
me> semantics of "cvs add" you agree with is something like "add new file to
me> the working copy but only after you make sure the file path is OK with
me> respect to the repository".

you> YESSSSS!!!!
/end quote

Don't you even care to understand questions before answering, or you've
already changed your opinion?


[...]
> >> I hope this message has cleared things up.
> >
> > Yes, unfortunately it seems it cleared things up to the understanding
> > that there are no chances to get to a suitable agreement.
> 
> I disagree!  :-)
> 
> I believe that what I described above as a complementary view of my proposal
> reconciles our differences.

No, I somehow still feel myself opposed to your proposal, sorry.

In fact I've said everything I could about the issue, so those who are
interested already noticed all my thoughts. That was my primary
intention, so I stop this discussion from my side.

-- 
Sergei.