Re: [Ltib] ./ltib -m release # externally distributable error

[aaron]

I added that line to the script and it generates printouts as follows this
email.  Is it possible to have an error in the .spec file such that it
completes './ltib' but not './ltib -m trelease'?  Is it possible to create
an iso with packages added to a local LTIB tree but not uploaded to the
GPP?

I'm doing a project for a company right now which is related to some other
projects that use tools like the ELDK, so the iso format is a familiar
starting point for cross-platform development for this particular project.
 In terms of rebuilding a software project from scratch, a procedure can
be written for installing a common Linux distribution onto a machine,
installing the software of the binary iso installer, and rebuilding the
project from source, in this case with './ltib'.  I could also tar up the
ltib code in CVS, and create a script that copies the required tarfiles
and patches to the appropriate area, but I think a self contained iso file
is much better, especially if that's a part of the functionality of the
LTIB itself, which I assumed it was from the beginning of my development
based on reading the FAQ.

In attempting to create such a binary release, and with the mentality of
the avionics industry in mind, I found it a little awkward that I needed
to access the network for any aspect of the creation of this iso,
especially since I had already successfully completed many iterations of
building the project executing './litb'.  Of course I can read/edit the
perl scripts and see the printouts on the command line to know what's
happening, but I just think it would be nice to have a way to make a
non-distributable, private binary iso installer while having the network
interface off.

One problem I had to work around was a firewall at work, so doing 'cvs
...' was harder for me than normal.  That's why I was asking if there
could be a version, maybe similar to the trelease mode in combination with
the special tag localdir, in which no network connectivity was needed in
order to create the iso.  I guess I'm viewing the iso file more as a
convenient packager than as a medium for software distribution.

One limitation for a build with no network might be CVS itself.  We are
using Subversion here, and I was thinking of migrating to Git, but I was
trying to see if there was potentially a way to copy all of the files of
the LTIB under version control to the stage directory where the iso is
built.  With Subversion, you can do the following:

$ svn co http://software.server/svn/project
$ cd project

# Here maybe I edit some project files so they're different than the
# versions that are on the server.  Also, I disconnect the network.

$ svn export . stage-with-changes
$ svn export -r BASE . stage-without-changes

Now the directory "stage-with-changes" contains the version control files
as well as the edits that I made, and "stage-without-changes" contains the
version control files exactly as they are on the server, without the
changes.  I can do this without network connectivity.  It might be that
CVS can't do something like that, hence the need for the cvsmanifest
script (which also requires network connectivity at some point in time).

I admit that I haven't briefed myself on the legal structure of the LTIB,
but I'm assuming that you can use it to develop private applications, as
well as a focal point for public, networked development.  There might be
other reasons for wanting to build an iso without interfacing with the CVS
repository as well, such as needing to make small changes that other
developers wouldn't find useful or might find detrimental but which are
locally necessary.

Any input would be appreciated.


Aaron

----

$ ./ltib -m trelease

You are about to create an iso image from this working project.
This will include all sources and built rpms for the target platform,
as well as the LTIB source code.

*** TEST RELEASE, NOT TO BE DISTRIBUTED EXTERNALLY ***

Before doing this, you should have done the following:

 1. Make sure you have committed any changes
 2. Make sure your source code is up to date
 3. Configured ltib for the target
 4. Run ltib to build all the selected packages

Do you want to continue: Y|n ?
y
Please enter the tag name for this release
Giving the name 'HEAD' will use the head of the trunk, or current branch.
In test mode, only existing tags can be used (e.g use HEAD)
localdir

Checking for unsaved config changes:

*** TEST RELEASE, NOT TO DISTRIBUTED EXTERNALLY ***

Checking licensing and external availabity of sources
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The following constraints/checks are enforced:

1/ All source/patches in the default configuration must be distributable.

2/ Any sources/patches not in the default configuration that are not
distributable are not be copied to the ISO image.

3/ Any content copied to the ISO image has it's md5 checksum verified by
downloading a fresh copy of the md5sum file and using it to verify the
content it checksums.

4/ Files already present on local disk in the LPP will be used.  Any
missing files will be downloaded from either the GPP staging area
or the CPP (Clickthru) staging area.

                  ------------------------


Copying content from default configuration (mandatory)
---------------------------------------------------------

checking: toolchain
====================
test_remote_file: url=tc-fsl-x86lnx-ppc-uclibc-4.2.4-1.i386.rpm,
file=tc-fsl-x86lnx-ppc-uclibc-4.2.4-1.i386.rpm
Testing network connectivity
OK GPP_STAGE:

OK GPP_STAGE: tc-fsl-x86lnx-ppc-uclibc-4.2.4-1.i386.rpm
test_remote_file: url=tc-fsl-x86lnx-ppc-uclibc-4.2.4-1.src.rpm,
file=tc-fsl-x86lnx-ppc-uclibc-4.2.4-1.src.rpm
OK GPP_STAGE: tc-fsl-x86lnx-ppc-uclibc-4.2.4-1.src.rpm

checking: fake-provides
=========================
License: GPL

checking: kernel-2.6.13.4-mpc8349itx
======================================
License: GPL
test_remote_file: url=linux-2.6.13.tar.gz, file=linux-2.6.13.tar.gz
OK GPP_STAGE: linux-2.6.13.tar.gz
test_remote_file: url=patch-2.6.13.4.gz, file=patch-2.6.13.4.gz
OK GPP_STAGE: patch-2.6.13.4.gz
test_remote_file: url=kernel-2.6.13-mpc8349itx-all-20060713.patch,
file=kernel-2.6.13-mpc8349itx-all-20060713.patch
OK GPP_STAGE: kernel-2.6.13-mpc8349itx-all-20060713.patch
test_remote_file: url=kernel-2.6.13-83xx-PCI-coherency.patch,
file=kernel-2.6.13-83xx-PCI-coherency.patch
OK GPP_STAGE: kernel-2.6.13-83xx-PCI-coherency.patch
test_remote_file: url=mpc8349itx-linux26-20060320-gianfar-fixes.patch,
file=mpc8349itx-linux26-20060320-gianfar-fixes.patch
OK GPP_STAGE: mpc8349itx-linux26-20060320-gianfar-fixes.patch
test_remote_file: url=mpc8349itx-linux26-20060620-gianfar-opt.patch,
file=mpc8349itx-linux26-20060620-gianfar-opt.patch
OK GPP_STAGE: mpc8349itx-linux26-20060620-gianfar-opt.patch
test_remote_file: url=kernel-2.6.13-mpc8349itx-rtc-usb-20060914.patch,
file=kernel-2.6.13-mpc8349itx-rtc-usb-20060914.patch
OK GPP_STAGE: kernel-2.6.13-mpc8349itx-rtc-usb-20060914.patch
test_remote_file: url=kernel-2.6.13-gianfar-skb-recycle.patch,
file=kernel-2.6.13-gianfar-skb-recycle.patch
OK GPP_STAGE: kernel-2.6.13-gianfar-skb-recycle.patch
test_remote_file: url=kernel-2.6.13-mpc8349itx-qg8mcu-2007011a.patch,
file=kernel-2.6.13-mpc8349itx-qg8mcu-2007011a.patch
OK GPP_STAGE: kernel-2.6.13-mpc8349itx-qg8mcu-2007011a.patch
test_remote_file: url=kernel-2.6.13-gcc4.patch, file=kernel-2.6.13-gcc4.patch
OK GPP_STAGE: kernel-2.6.13-gcc4.patch

...
...
...

checking: SeaMAC
==================
License: GPL
test_remote_file: url=SeaMAC_0.98.1.tar.gz, file=SeaMAC_0.98.1.tar.gz
URL must be absolute at /usr/lib/perl5/vendor_perl/5.10.0/LWP/UserAgent.pm
line 206.
traceback:
 LWP::UserAgent::prepare_request:206
  (eval):249
   LWP::UserAgent::simple_request:248
    LWP::UserAgent::request:263
     Ltibutils::test_remote:345
      Ltibutils::test_remote_file:406
       main::release_copy:51
        main::release_copy_pkg:95
         main::release_copy_content:160
          main::release_main:346
           main::f_trelease:1228
            main:561

Exiting on error or interrupt
Please see | tee RELEASE_COPY_CONTENT for details


> Hi Aaaron,
>
> Either there's problem in the SeaMAC.spec, or some bug in Ltibutils.pm.
>
> It would probably be easiest to put an 'warn' in just after the start of
> test_remote file to see what URL is getting called out.  So it would
> look like this (around 400)
>
> sub test_remote_file
> {
>      my ($url, $cf) = @_;
>      warn("no url passed"), return unless $url;
>      my ($file) = $url =~ m-/?([^/]+)$-;
> warn "test_remote_file: url=$url, file=$file\n";
>      my $ua  = LWP::UserAgent->new;
>
> Regards, Stuart
>
> address@hidden wrote:
>> I'll write a few more details tomorrow, but I was able to create the
>> binary release iso that I wanted to.  I used the 'trelease' mode with
>> the
>> special tag 'localdir' and a MANIFEST file.  However, I still needed to
>> hack the file bin/Ltibutils.pm to build the iso, because I was getting
>> errors such as the following.  The SeaMAC package is one that I
>> introduced
>> to the build by creating a dist/lfs-5.1/SeaMAC/SeaMAC.spec file and
>> incorporating it into the configuration, etc.  Only after hacking the
>> file
>> (and I'm not a perl programmer) was I able to arrive at the iso.  Thanks
>> for the clue about the uClibc toolchain being distributable.  After
>> updating my ltib CVS files I got passed the toolchain check.
>>
>> ---
>>
>> $ ./ltib -m trelease
>> Updating lpp from local packages
>>
>> You are about to create an iso image from this working project.
>> This will include all sources and built rpms for the target platform,
>> as well as the LTIB source code.
>>
>> *** TEST RELEASE, NOT TO BE DISTRIBUTED EXTERNALLY ***
>>
>> Before doing this, you should have done the following:
>>
>>  1. Make sure you have committed any changes
>>  2. Make sure your source code is up to date
>>  3. Configured ltib for the target
>>  4. Run ltib to build all the selected packages
>>
>> Do you want to continue: Y|n ?
>> y
>> Please enter the tag name for this release
>> Giving the name 'HEAD' will use the head of the trunk, or current
>> branch.
>> In test mode, only existing tags can be used (e.g use HEAD)
>> localdir
>>
>> Checking for unsaved config changes:
>>
>> *** TEST RELEASE, NOT TO DISTRIBUTED EXTERNALLY ***
>>
>> Checking licensing and external availabity of sources
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>> The following constraints/checks are enforced:
>>
>> 1/ All source/patches in the default configuration must be
>> distributable.
>>
>> 2/ Any sources/patches not in the default configuration that are not
>> distributable are not be copied to the ISO image.
>>
>> 3/ Any content copied to the ISO image has it's md5 checksum verified by
>> downloading a fresh copy of the md5sum file and using it to verify the
>> content it checksums.
>>
>> 4/ Files already present on local disk in the LPP will be used.  Any
>> missing files will be downloaded from either the GPP staging area
>> or the CPP (Clickthru) staging area.
>>
>>                   ------------------------
>>
>>
>> Copying content from default configuration (mandatory)
>> ---------------------------------------------------------
>>
>> checking: toolchain
>> ====================
>> Testing network connectivity
>> OK GPP_STAGE:
>>
>> OK GPP_STAGE: tc-fsl-x86lnx-ppc-uclibc-4.2.4-1.i386.rpm
>> OK GPP_STAGE: tc-fsl-x86lnx-ppc-uclibc-4.2.4-1.src.rpm
>>
>> ...
>> ...
>> ...
>>
>> checking: SeaMAC
>> ==================
>> License: GPL
>> URL must be absolute at
>> /usr/lib/perl5/vendor_perl/5.10.0/LWP/UserAgent.pm
>> line 206.
>> traceback:
>>  LWP::UserAgent::prepare_request:206
>>   (eval):249
>>    LWP::UserAgent::simple_request:248
>>     LWP::UserAgent::request:263
>>      Ltibutils::test_remote:345
>>       Ltibutils::test_remote_file:404
>>        main::release_copy:51
>>         main::release_copy_pkg:95
>>          main::release_copy_content:160
>>           main::release_main:346
>>            main::f_trelease:1228
>>             main:561
>>
>> Exiting on error or interrupt
>> Please see | tee RELEASE_COPY_CONTENT for details
>>
>>
>>
>>> Hi Aaron,
>>>
>>> I re-read this again and there are a number of points/issues to comment
>>> on:
>>>
>>> 1/ You can release whatever the toolchain.  The problem you seem to
>>> have
>>> had in the first instance is that there was no network connection
>>> available and so the checks to see that the file to be copied is on the
>>> GPP failed.
>>>
>>> 2/ If your BSP is different from that in CVS and/or it contains content
>>> that is not on the GPP, then you can't make an 'official' release.
>>> There's a long story behind this to do with signing off approved
>>> content.  However, you have some options:
>>>
>>> a) use ./ltib -m trelease
>>> b) use the tag 'localdir' and a MANIFEST file
>>>
>>> In the case of a), you still (may) need a network connection, but
>>> errors
>>> are not fatal.  The ISO image will be marked as 'non-distributable'
>>>
>>> In the case of b) 'localdir' is a special tag that says read the
>>> file called MANIFEST and use that to copy as the ltib source (not the
>>> packages).  You can generate a static manifest file from CVS using:
>>> bin/cvsmanifest > MANIFEST.  However, I still think a network is needed
>>> to copy some of the package content if its missing from your LPP.  This
>>> is not really an option I want to support.
>>>
>>> 3/
>>>  >> Undefined subroutine &main::tag_repository called at
>>>  >> /var/backup/20091012-ltib/ltib/bin/Ltibrelease.pm line 353.
>>>  >> traceback:
>>>
>>> This is a bug, at some point the export of tag_repository got removed
>>> from Ltibutils.pm.  I've put it back.  Note that if you use the tagname
>>> 'HEAD' no CVS tagging is attempted.
>>>
>>>
>>> Recommendations:  if it's possible/useful get your packages into LTIB
>>> and then you can simply run:  ./ltib -m release and use the tag HEAD,
>>> this is the best way.  Failing that I'd go with ./ltib -m trelease
>>>
>>> Question: if you can say, what do you have in your LTIB tree that can't
>>> be put into the Savannah CVS?  Also why is making an ISO useful to you?
>>>   The problem with these is they can't be updated easily.
>>>
>>> Regards, Stuart
>>>
>>>
>>> Stuart Hughes wrote:
>>>> Hi Aaron,
>>>>
>>>> I'll look into this and get back to you,  all the content on the GPP
>>>> is
>>>> distributable so the problems you raise are bugs.
>>>>
>>>> Regards, Stuart
>>>>
>>>> address@hidden wrote:
>>>>> At some point in my usage of the LTIB tool I switched from using the
>>>>> glibc
>>>>> C library to uclibc to save space or because of compiler errors, I
>>>>> forget
>>>>> which.
>>>>>
>>>>> Now I'm trying to bundle up my ltib tree into a binary release using:
>>>>>
>>>>> ./ltib -m release
>>>>>
>>>>> I'm getting the following error:
>>>>>
>>>>> checking: toolchain
>>>>> ====================
>>>>> Testing network connectivity
>>>>> No network download connection available
>>>>>
>>>>> tc-fsl-x86lnx-ppc-uclibc-4.2.4-1.i386.rpm is not externally
>>>>> distributable
>>>>> Error: tc-fsl-x86lnx-ppc-uclibc-4.2.4-1.i386.rpm  required by
>>>>> defconfig
>>>>> traceback:
>>>>>  main::release_copy_content:167
>>>>>   main::release_main:353
>>>>>    main::f_release:1198
>>>>>     main:552
>>>>>
>>>>> If possible, I would like to create a binary iso release of my tree
>>>>> that
>>>>> includes the packages in the local package pool that are required to
>>>>> reproduce my images from source.  This iso file is not for
>>>>> distribution,
>>>>> but for internal records.  Do I need to switch back to using the
>>>>> glibc
>>>>> based cross-toolchain?  Also, it seems like this mode of ltib
>>>>> requires
>>>>> contact with the remote CVS repository, but I would like to be able
>>>>> to
>>>>> create this iso without network connectivity.
>>>>>
>>>>> In an attempt to produce any type of binary iso release, I started
>>>>> with
>>>>> a
>>>>> fresh copy of the ltib today and did the following:
>>>>>
>>>>> $ ./ltib --preconfig config/platform/mpc8349itx/defconfig
>>>>>
>>>>> (here I had to get around two things that stopped the build, one
>>>>> being
>>>>> the
>>>>> cuImage error, and the other de-selecting the samba package which had
>>>>> an
>>>>> error in the configuration (attempted execution of cross-compiled
>>>>> test
>>>>> code))
>>>>>
>>>>> $ cp config/platform/mpc8349itx/.config
>>>>> config/platform/mpc8349itx/defconfig
>>>>> $ cp config/platform/mpc8349itx/nas_linux-2.6.13.4-mpc8349itx.config
>>>>> config/platform/mpc8349itx/nas_linux-2.6.13.4-mpc8349itx.config.dev
>>>>> $ ./ltib -m release
>>>>>
>>>>> This last command errors with:
>>>>>
>>>>> ...
>>>>>
>>>>> checking: sparse
>>>>> ==================
>>>>> License: Open Software License
>>>>> OK GPP_STAGE: sparse-0.4.tar.gz
>>>>> OK GPP_STAGE: sparse-0.4-array-fix.patch
>>>>>
>>>>> checking: git
>>>>> ===============
>>>>> License: GPL
>>>>> OK GPP_STAGE: git-1.5.6.5.tar.gz
>>>>> OK GPP_STAGE: git-1.5.6.5-no-perl-install.patch
>>>>>
>>>>> checking: tunctl
>>>>> ==================
>>>>> License: GPL
>>>>> OK GPP_STAGE: tunctl-1.5.tar.gz
>>>>>
>>>>> checking: mux_server
>>>>> ======================
>>>>> License: LGPL
>>>>> OK GPP_STAGE: mux_server.c
>>>>> OK GPP_STAGE: mux_sever-1.0-fs-decl.patch
>>>>> Undefined subroutine &main::tag_repository called at
>>>>> /var/backup/20091012-ltib/ltib/bin/Ltibrelease.pm line 353.
>>>>> traceback:
>>>>>  main::release_main:353
>>>>>   main::f_release:1235
>>>>>    main:560
>>>>>
>>>>> I guess it would be nice if there were some method of producing the
>>>>> binary
>>>>> iso file without having to contact the remote CVS repository.
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> LTIB home page: http://ltib.org
>>>>>
>>>>> Ltib mailing list
>>>>> address@hidden
>>>>> http://lists.nongnu.org/mailman/listinfo/ltib
>>>>>
>>>>
>>>> _______________________________________________
>>>> LTIB home page: http://ltib.org
>>>>
>>>> Ltib mailing list
>>>> address@hidden
>>>> http://lists.nongnu.org/mailman/listinfo/ltib
>>>>
>>
>>
>>
>>
>> _______________________________________________
>> LTIB home page: http://ltib.org
>>
>> Ltib mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/ltib
>>
>