Re: [RFC] Automatic 'make check' in CI

[Dan Eble]

On Nov 19, 2020, at 02:30, Jonas Hahnfeld <hahnjo@hahnjo.de> wrote:
> 
> Am Mittwoch, den 18.11.2020, 21:49 -0500 schrieb Dan Eble:
>> On Nov 18, 2020, at 15:25, Jonas Hahnfeld <hahnjo@hahnjo.de> wrote:
>>> 
>>> Until then, we need to rebase to adapt the MR commits to the
>>> available test-baseline.
>> 
>> Do you mean that we would need to rebase to master before any push that 
>> updates an MR?
> 
> At the very least if there were changes to regression tests added to
> master in the mean time and you don't want them to appear as in your
> run of "make check". So always doing this should be a safe default
> (even if this renders the "Compare with previous version" kind of
> useless; some compromise we have to make here…)

Hold on.  I understand that "Compare with previous version" becomes harder to 
read after a rebase, but is that the only reason you've been thinking of 
testing against the merge base?  I think I would prefer testing against the 
current version of master.  That will expose a patch that lags behind as soon 
as it goes into Patch::new rather than delaying until the submitter chooses to 
rebase it, which might be immediately before merging it when nobody is paying 
attention.

>> I could live with that.
>> 
>> Thanks for the work you've put into this.  It's going to be helpful.
> 
> So OK to start with the initial version and improve upon that in
> smaller steps?

OK with me.
— 
Dan