Re: [Ltib] Need to remove package build source if that package .spec changed

[Stuart Hughes]

Hi Peter,

I started implementing this and again I found myself asking; what if
someone has edits they want in that directory and they get clobbered?
Whether tarball or svn (git, cvs etc) based packages, no one will thank
you if their precious work gets zapped.

The only safe way to do this would be to ensure prior to source removal
ensure that the sources have not been changed.  The problem with this is
that:

* This can take a long time as it needs a full unpack of the reference
and then a diff -q, which I guess could be okay/acceptable

* This can only work for tarball based package which is okay/acceptable

* This is not reliable unless the clean/distclean for the package
actually works. Which may/may not be acceptable

Let me know if:

a) You agree with my reasoning

b) You still think this is worth trying to do (it's more complicated
than the patch you sent)

Regards, Stuart

On 31/01/11 16:21, Stuart Hughes wrote:
> Hi Peter,
> 
> Okay, I think I understand the requirement now.  I won't rely on the svn
> tag in the name, I think that's too weak/restrictive (in case someone
> wants to use git or something else).  Instead I think the absence of a
> SOURCE: tag should be the clue to LTIB that this is a directory build
> (svn, cvs, git etc) and so clobbering is not allowed.
> 
> Would you like me to send the patch to the list for you to look at/try
> out before checking in?
> 
> Regards, Stuart
> 
> On 31/01/11 15:22, Peter Barada wrote:
>> On 01/31/2011 10:10 AM, Stuart Hughes wrote:
>>> Hi Peter,
>>>
>>> Okay, there are 2 separate use cases here;
>>>
>>> 1/ The tarball+patches builds
>>>
>>> 2/ The SVN (SCM) based directory builds.
>>>
>>> In the case of 2/ all that I've said before stands (e.g. clobber is a
>>> very bad idea).
>>>
>>> In the case of 1/ your patch would be reasonable, but I would also add a
>>> prompt to the user to make sure.  This would be bypassed if the --batch
>>> option was in force (e.g. for auto-builder).  My preference would be to
>>> limit this to only tarball based builds.  The detection I'd propose is
>>> checking for the existence of a source tag being present.  Does this
>>> sounds reasonable?
>> That sounds perfectly reasonable.
>>
>> However in the case of #2, --clobber can't happen for a SCM build since
>> in this case the patch has:
>>
>>     # If its a SCM (source code management) package, force prep
>>     my $svn_bld = 0;
>>     if ($tok->{version} eq 'svn') {
>>     $svn_bld = 1;
>>     *$dir_bld = 0*;
>>     }
>>
>> And later:
>>  
>>        if ($cf->{clobber} && *$dir_bld* && $unpack eq 'yes' && $spec_upd) {
>>         print "Clobber forces removal of
>> $cf->{rpmdir}/BUILD/$tok->{pkg_dir_name}\n";
>>         system_nb("rm -rf $cf->{rpmdir}/BUILD/$tok->{pkg_dir_name}");
>>         # force prep
>>         $dir_bld = 0;
>>         }
>>
>>
>>> Regards, Stuart
>>>
>>> On 31/01/11 14:23, Peter Barada wrote:
>>>> On 01/31/2011 05:13 AM, Stuart Hughes wrote:
>>>>> Hi Peter,
>>>>>
>>>>> I'm reluctant to add this as it breaks a golden rule that LTIB promises
>>>>> not to delete source code once unpacked.  What if some developer had put
>>>>> in some edits and forgotten and had the --clobber in a script?
>>>> Then they shoot themselves in the foot.  You can't make things completely 
>>>> foolproof as God will come along and create a more determined fool...
>>>>
>>>> Perhaps renaming the option to "---clobber" or "--KlObBeR" or even 
>>>> "--Remove-prepped-source-if-spec-file-updates" to make it stand out even 
>>>> more would help...
>>>>> Reading this, and your other email about SVN support, maybe there's
>>>>> another way.  Rather than removing the source, why not try to update it;
>>>>> this would prevent accidental clobbering and also allow updates.  This
>>>>> way different developers can pull in unrelated updates and stay in sync.
>>>> That is in the case where a packages is not kept as a tarball+patches but 
>>>> instead as a directory under some SCM.  In the case where developer A 
>>>> checks in a patch (and correspodning .spec file change to add the %patch), 
>>>> and devloper B updates his LTIB source, everything works fine unless the 
>>>> patch checked in is an update to a package that LTIB leaves checked out 
>>>> due to another package needing the source.
>>>>
>>>> As an example if the LTIB project developer A and B are working on has 
>>>> helloworld_mod enabled, then the kernel source will always be left 
>>>> prepped.  If Developer A adds a patch to the kernel (and updates the 
>>>> kernel .spec file), and developer B then updates his LTIB with the changes 
>>>> from developer A and builds, he will not build the change developer A 
>>>> checked in since the kernel source is already %prepped.   This can lead to 
>>>> confusion...
>>>>
>>>>> This can be done using this directory build mechanism.  Below is an
>>>>> example of what I mean, stripped back as a guide. There are related
>>>>> examples in dtc-dir-build.spec.in,
>>>>> dist/lfs-5.1/kernel/kernel26-dir-build.spec.in &
>>>>> dist/lfs-5.1/u-boot/u-boot-dir-build.spec.in ):
>>>>>
>>>>> Maybe this could do what you need?
>>>>>
>>>>> Regards, Stuart
>>>>>
>>>>> --- a spec file that manages it's own SCM commands ---
>>>>>
>>>>> %define pfx /opt/freescale/rootfs/%{_target_cpu}
>>>>> %define buildsubdir %{name}-%{version}
>>>>> ....
>>>>> Name            : some_name
>>>>> Version         : 1.1
>>>>> Release         : 1
>>>>> ....
>>>>> %Prep
>>>>> # only do the code checkout if the target directory doesn't exist
>>>>> if [ -e "%{buildsubdir}" ]
>>>>> then
>>>>>     exit 0
>>>>> fi
>>>>> mkdir %{buildsubdir}
>>>>> cd %{buildsubdir}
>>>>> git-clone git://some_url/some_project
>>>>> git checkout some_branch
>>>>>
>>>>> %build
>>>>> cd some_project
>>>>> git pull
>>>>> make
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 28/01/11 16:09, Peter Barada wrote:
>>>>>> Stuart,
>>>>>>
>>>>>> As discussed previously, if I have a build that leaves source
>>>>>> unpacked
>>>>> (because its used by another package as in the kernel source used by
>>>>> helloworld_mod), I ran into the problem that if I updated my LTIB tree
>>>>> from SVN (and another developer added a patch to the kernel), the
>>>>> current version of LTIB wouldn't clobber the kernel source from
>>>>> rpm/BUILD. This can lead to serious confusion as Developer A says "Hey,
>>>>> I checked in a patch to fix that bug in package X" and developer B pulls
>>>>> updates LTIB, does "./ltib" and responds "no you didn't"....
>>>>>> The attached patch adds "--clobber" which removes a packages source
>>>>> and then runs the %prep step if it detects that the spec file has been
>>>>> updated.
>>>>>
>>>>
>>
>>
>> -- 
>> Peter Barada
>> address@hidden
>>
> 
> _______________________________________________
> LTIB home page: http://ltib.org
> 
> Ltib mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/ltib
>