Re: [Ltib] PKG_KERNEL_LEAVSRC issues...

[Peter Barada]

On 12/21/2010 03:56 PM, Stuart Hughes wrote:
> Hi Peter,
>
> Peter Barada wrote:
> > On 12/20/2010 03:24 PM, Stuart Hughes wrote:
> > > Hi Peter,
> > >
> > > See inline:
> > >
> > > Peter Barada wrote:
> > > > On 12/20/2010 04:35 AM, address@hidden wrote:
> > > > > Hi Peter,
> > > > >
> > > > > I'm going to need to re-read this email as at first read I'm not
> > > > > exactly
> > > > > sure what you are running into.  If possible can you reduce it to 2
> > > > > sentences of; "I expected this... but observe this...."
> > > > Ok, to clear things up, I expect if I remove rpm/RPMS/arm/kernel*.rpm
> > > > and everything in rpm/BUILD that "./ltib" will "do the right thing",
> > > > namely %prep, %build but not %clean the kernel if I have a package
> > > > enabled that selects PKG_KERNEL_LEAVESRC.  Since PKG_*_LEAVESRC is a
> > > > transient config it fails when I try to build helloworld_mod.
> > > Okay I can see what you mean, that's a bug/design flaw.
> > >
> > > I think a simple fix maybe to just make LEAVESRC non-transient, but I
> > > worry about some unwanted side-effect.  I have a vague recollection that
> > > I was going to do this at one point and backed out.
> > >
> > > Maybe for now you could try for your use cases by deleting line 3149:
> > >
> > >           s,^(\w+LEAVESRC)=y,# $1 is not set,;
> > That would change the transient case - have to try to know for sure if
> > that alone will fix what I'm seeing.
> Try it and see if it does.
>
> On another note, I was trying to recall and I think the reason I made
> LEAVESRC transient was so that if someone checked in a defconfig it
> didn't cause a kernel rebuild on first install.  It's a long time ago
> now, so I'm a bit vague.  If this is right, another alternative would be
> to removed these when loading any defconfig.
But if that is done, then unless a configuration step is done to see 
that PGK_HELLOWORLD_MOD is enabled which selects PKG_KERNEL_LEAVESRC, 
then ltib won't know that the kernel source needs to not be cleaned, right?

> > > > > Just to shed some light on the design, normally .config will override
> > > > > defconfig, however if you touch defconfig ltib will notice this and
> > > > > use
> > > > > the updated defconfig.  The purpose of this is so that if you have an
> > > > > autobuilder and check-in an update to defconfig, the autobuilder
> > > > > will use
> > > > > the updated defconfig.
> > > > In this case I have not touched any of the config files...
> > > No, but I wanted you to understand that touching the defconfig file will
> > > have this effect in case you run into it or want to know how to trigger
> > > this behaviour.
> > Thanks - possibly useful but not sure, kinda like a hidden switch.
> Overall practice showed this to be the best compromise.  So this is a
> case where opinions may differ, but this behaviour works best IMHO.
Pardon me, on rereading you're right, touching defconfig should make it 
take precedence over defconfig.dev and .config.  I must have been 
confused with something else.

> > > > > Regarding "LEAVESRC" the idea is that this only needs to be
> > > > > transient as
> > > > > once a package is left unpacked, it will never be removed and so on
> > > > > subsequence builds the same sources remain there and just get
> > > > > re-built.
> > > > True, but being "paranoid" especially if I'm mucking with the config(s)
> > > > and package spec files, its easiest to force multiple package rebuilds
> > > > by deleting their resultant .rpm and rpm/BUILD directories and
> > > > executing
> > > > "./ltib".  Also my buildbot leaves its universe alone (i.e. all it does
> > > > is "svn update; ./ltib -m clean; ./ltib -b --preconfig
> > > > config/platform/<platform>/defconfig").
> > > >
> > > > I'm sure I'm missing something - how can I tell LTIB to remove a
> > > > package's source (that's left around due to LEAVESRC) - if its
> > > > exists in
> > > > rpm/BUILD, ltib cowardly refuses to remove it even if I update its
> > > > .spec
> > > > file. So I added a "rm -rf rpm/BUILD/* to the buildbot steps and ended
> > > > up w/o kernel source and helloworld_mod failing.
> > > The idea is once a package source is unpacked the only way it will get
> > > removed is manually (or by some external autobuild script).  The reason
> > > is that you may be working on that source (e.g. developing a kernel
> > > driver), the last thing you want is to lose those changes.  Touching the
> > >    .spec file just means that you want a rebuild, not that any sources
> > > need unpacking (unless they are not there already).
> > In my case where I have buildbot running on a slave that I can't log
> > into, if the kernel was left unpacked (because helloworld_mod needed
> > it), if I add a patch to the kernel, in the next build after an update
> > from my SVN, LTIB realizes *something* needs to be done but doesn't
> > remove the offending source in rpm/BUILD.  So I'm caught in a
> > problem.  If I blow away all the source in rpm/BUILD, LTIB doesn't
> > repopulate(or enough to build helloworld_mod).  If I leave the sources
> > in rpm/BUILD as is, then LTIB can't pick up new patches to any package
> > %prep'd in rpm/BUILD.  NEEDSRC is a way to fix this case.  I agree
> > that LEAVESRC is useful for transient "hey, leave this around so I can
> > look", but because its transient, .config doesn't have it left on
> > (i.e. its cleared out of .config, left in defconfig.dev) and unless
> > defconfig is explicitly used the behavior is inconsistent.
> See earlier; I think the transient nature can be removed and put into
> the section that parses the defconfig.  I think this would fix your
> problem as I believe on a fresh install, the static LEAVESRC=y would get
> removed, however when run through mconf (conf) the select in the kernel
> module would turn it on for you and unpack the sources.
Is mconf invoked in the -b case?  If not then I can see things being out 
of kilter.
> BTW: if you're actually doing kernel development, it's probably better
> to use a directory build and point at your kernel sources.  This way
> your external autobuild scripts can do a "git pull" before running LTIB
> to get the kernel source tree updated.
In my world I do kernel (and some package development) using SVN trees 
and appropriate .spec files, but then diff the kernel tree an create a 
patch for SVN using a .spec.in file to build it.  Its a bit messy but is 
needed since I use the buildslave to check that the LTIB tree I pull out 
has everything necessary to duplicate my internal SVN kernel/package 
trees (and this LTIB tree becomes what gets distributed to customers).  
Since the build can take quite from scratch I want to limit the building 
on the slave to the minimum to allow it to keep up with checkins as part 
of continuous integration testing.
> Can you think this through/try it out (by removing that line) and let me
> know how it goes.  Depending how that goes, and time permitting I'll try
> to get a fix in for this.
Sure, will do, I'll get back to you.  Thanks!
> Regards, Stuart


--
Peter Barada
address@hidden