Re: [Libcdio-devel] [PATCH] UDF+ISO image extraction sample (and a plan for upcoming changes)

[Rocky Bernstein]

On Mon, Jan 23, 2012 at 12:04 PM, Pete Batard <address@hidden> wrote:

> On 2012.01.23 15:03, Rocky Bernstein wrote:
>
>> Ok. Since I think it important, unless there are other volunteers I'll
>> look
>> into adding tests for the very specific bugs that are encountered.
>>
>
> I very much appreciate that! And I'll try to help as much as I can.
>
> To give you an idea of what's in store, here are the 4 tests we would need
> to have an image for then:
> - multiple files that can be extracted sequentially (hopefully easy)
>

This was  what commit 462b417e was about. To get a failing test for this
all I had to do was create a UDF with two files (README and COPYING from
libcdio) using the command:

     mkisofs -R -J -udf -iso-level 3 -o /tmp/test-udf1.iso /tmp/udf-test1

Which adds less than 900k to the untarred distribution.


- at least one file with extended attributes and allocated descriptors that
> result in a File Entry that is greater in size than the last one used. I'm
> not exactly sure how the reuse is supposed to occur, and since it doesn't
> matter with my proposed fix, I haven't investigated, but this may be linked
> to the next test below, as the problem manifests itself with files that are
> larger than 1 GB apparently
>

When there is a specific patch for this, I'll start on a test. Otherwise,
I'd not be sure I would be creating the problem you are seeing.


> - at least one file that uses multiple extents (or whatever the UDF specs
> calls it). In the case of the Windows UDF, a new extent is used for every 1
> GB of data. Now, my tests with the Windows UDF show that the current
> libcdio handles it properly (good job!), but we'd still probably want that
> in an image.
>

I am committing only to testing bugs encountered that are you are fixing,
not bugs one can imagine encountering but haven't encountered.  Others who
are more interested in UDF support are welcome to add suggest other tests.
More below...



> - at least one file that is larger than 4 GB. Most likely 2 GB will do, as
> I think all the 32 bit issues I spotted were with signed values, but we
> might as well go for 4 to be safe.
>

Adding 2GB to the distribution for testing everyone gets is not acceptable.
Thus either the images need to be stored someplace where folks can download
them. Or have a way to create suitable images on the fly.  Either way,
probably something for a little later.


>
> Additionally, having some filenames in CJK for instance, to test/implement
> Unicode preservation would be welcome (I assume most of us will be testing
> with a Western locale).


Is this something that has been observed as being a bug?

I didn't say this as part of the TDD/BDD thing, but part of the "Agile
programming" philosophy is starting out with what you need right now, not
what you imagine you might need. Of course, it is also useful to try not to
code oneself in a corner.


>
>
>  And as I wrote before, having even slightly better UDF support may be just
>> enough to lure more people to handle all those other more thorny issues.
>>
>
> That's what I'm hoping as well, as I'd rather benefit from other people
> from fixing issues that I may end up experiencing otherwise ;)
>
>
>  I understand where you are coming from, but suggest even in those other
>> projects it may be a mistake. With testing there is a chicken-and-egg
>> bootstrap problem. Unless one is working in an environment like
>> Ruby-on-Rails where there already has been a lot of work done to support
>> testing (even to the extent of automatically cloning databases), there is
>> some initial hit one takes in setting up tests. But after doing that, the
>> testing becomes easier. And the pay-off in my view is worth the work done.
>>
>
> Agreed. That also has a lot to do on whether you are in for the short or
> the long term.
>
>
>  Here is something similar. You just wrote an example program. It was
>> probably easier to do because there were other example programs and the
>> build framework to cut and paste from.
>>
>
> Absolutely! Most of the extract.c sample is copy/paste of existing libcdio
> samples, and I'm very grateful for that. However, the main issue here is
> not sample accessibility but accessibility of data to test the samples
> against, in order to check for potential problems.
>
> My view, and this is also part of the reason I can't see myself spending
> much time crafting an UDF test image, is that even after Microsoft removes
> the Windows 8 preview UDF from public availability, an awful lot of libcdio
> users should still be able to get their hands on a Windows installation
> image to display the same (potential) symptoms. All the Windows 7 images I
> tested with do and I fully expect the release Windows 8 media to do as well.
>
> Of course, if no public Windows image is available from Microsoft or
> elsewhere, I'm not too happy about the idea of punishing libcdio users that
> work with Linux, Solaris, BSD, OSX, etc exclusively. But the sheer market
> penetration of Windows means that most of these users should be able to
> find someone who can provide one.
>
>
>  Experience has shown that the
>> example programs have been very helpful, although when I wrote the first
>> ones, it felt more as an afterthought.
>>
>
> Yeah. The other side of the equation, which I've seen with libusb, can
> also occur if, after finding that samples are lacking, you decide to write
> a generic one that tests a handful of features, instead of going with a
> myriad of samples that perform single but very elementary tests, and the
> maintainers of the project decide that having no sample at all is better
> than having one that does too much...
>
> As a matter of fact, the extract sample is designed to work for both
> ISO9660 and UDF images. While I'm aware that this could be split into an
> iso-extract.c and an udf-extract.c, I sure hope you're not going to ask for
> that, because (since it isn't that large) I think it'll be more beneficial
> for libcdio users this way.


extract.c  (even with my one line error-reporting fix) is still very short.
So it doesn't matter if it handles ISO9660 and UDF.


>
>  So again, perhaps if a framework for testing UDF images is started adding
>> to that for bugs that you want to fix won't be too much trouble. And there
>> may be some code in the testing that you can use on the other projects.
>>
>
> Ack. As I said, my main problem is the time I want to spend. Otherwise,
> I'm all for having a reliable testing framework as well.
>
> Regards,
>
> /Pete
>
>