[Libcdio-devel] [PATCH] UDF+ISO image extraction sample (and a plan for upcoming changes)

[Pete Batard]

Alright, now that I have straightened out a working UDF+ISO extraction 
in my application (or at least what looks like one so far), I am in a 
position to try to feed back a few fixes and improvements to libcdio.

Naturally, I don't expect all the modifications I have currently applied 
to the source to work in my app, to be transposable as is, especially as 
I took some liberties here and there. But at the very leats, I should be 
able to provide details on how I addressed the specific issues that I 
had, and we can try to take it from there.

Overall, even though I almost gave up on libcdio in despair a few days 
ago, not much seems to be needed to get UDF support going.

For people interested in testing the upcoming changes, the first thing 
we are going to need is a new sample that does image extraction, which 
is provided in the attached patch. It's heavily based on what the 
existing iso and udf samples already do for directory listing and file 
extraction, the main different being that it avoid using ftruncate 
because ultimately we would need an ftruncate64 (we'll be handling some 
files that are > 4GB in size) and this becomes a cross-platform 
headache, especially if one plans to eventually support MSVC compilers.
The current sample does not do timestamp/premissions preservation and 
does not deal with Unicode (unless the underlying calls support UTF-8, 
which wouldn't be the case on Windows), as this isn't really something 
we should have issues with. One I have sorted these in my app, I'll 
probably update the sample though.

Before I go ahead an apply it to mainline, I'd appreciate if you could 
review the sample to confirm that it looks OK. Unless I get a green 
light, I'm not planning to push anything to mainline for another day or two.

The second item we will need besides that sample, is an UDF image that 
can exhibit issues that need to be addressed. The one I would encourage 
everyone to use would be the "Windows 8 Developer Preview with developer 
tools English" since:
1. It is freely available from Microsoft
2. It is more than 4 GB in size (=> will test 32 bit limitations)
3. It also contains a file that is more than 4 GB in size (=> more 32 
bit test, as well as extended attributes use)
This image can be downloaded at: [1] (direct link [2]).

With the sample and the image file above, we should then be able to:

1. Make UDF extraction work on Linux 32 bit (note that 64 bit Linux may 
not exhibit the 32 bit problems I've seen, so I'd strongly encourage to 
use 32 bit Linux for tests)

2. Make UDF extraction work on MinGW32 as well as other platforms that 
don't have _FILE_OFFSET_BITS 64 or transparent large file support

3. Make UDF and ISO9660 image extraction work with MSVC on Windows


As you may guess, my objective is to bridge as much as the gap as 
possible between the libcdio source I use in my app (which is compiled 
with both MinGW and MSVC), and official. Eventually, I'd like to be able 
to use mainline files as is, to easily benefit from future libcdio 
fixes/updates.

Finally, because I am only interested in image extraction at the moment, 
and it should be enough to keep us busy for some time (especially if we 
add MSVC support), I'm going to be very restrictive in the scope of 
libcdio for this whole exercise. In effect, the following options are 
what I'll use on all platforms (which you may want to add to your 
autogen.sh for testing)

./configure --enable-maintainer-mode --disable-cddb \
  --disable-vcd-info --disable-cxx --disable-cpp-progs \
  --without-cd-drive --without-cd-info --without-cd-paranoia \
  --without-cdda-player --without-cd-read

With this sorted out, we are ready to start patching libcdio...

Regards,

/Pete

[1] http://msdn.microsoft.com/en-us/windows/apps/br229516
[2] 
http://wdp.dlws.microsoft.com/WDPDL/9B8DFDFF736C5B1DBF956B89D8A9D4FD925DACD2/WindowsDeveloperPreview-64bit-English-Developer.iso
From 4616028005dfedc6580baa75d5390a9696921ffc Mon Sep 17 00:00:00 2001
From: Pete Batard <address@hidden>
Date: Sun, 22 Jan 2012 02:00:37 +0000
Subject: [PATCH] Add ISO9660+UDF image extraction example

---
 example/Makefile.am |    5 +-
 example/extract.c   |  270 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 274 insertions(+), 1 deletions(-)
 create mode 100644 example/extract.c

diff --git a/example/Makefile.am b/example/Makefile.am
index de65b5b..995f619 100644
--- a/example/Makefile.am
+++ b/example/Makefile.am
@@ -26,7 +26,7 @@ paranoia_progs = paranoia paranoia2
 endif
 if BUILD_EXAMPLES
 noinst_PROGRAMS = audio cdchange cdtext device discid drives eject \
-                 isofile isofile2 isofuzzy isolist isolsn \
+                 extract isofile isofile2 isofuzzy isolist isolsn \
                  mmc1 mmc2 mmc2a mmc3 $(paranoia_progs) tracks \
                  sample3 sample4 udf1 udffile cdio-eject
 endif
@@ -54,6 +54,9 @@ drives_LDADD          = $(LIBCDIO_LIBS) $(LTLIBICONV)
 eject_DEPENDENCIES    = $(LIBCDIO_DEPS)
 eject_LDADD           = $(LIBCDIO_LIBS) $(LTLIBICONV)
 
+extract_DEPENDENCIES  = $(LIBISO9660_LIBS) $(LIBUDF_LIBS) $(LIBCDIO_DEPS)
+extract_LDADD         = $(LIBISO9660_LIBS) $(LIBUDF_LIBS) $(LIBCDIO_LIBS) 
$(LTLIBICONV)
+
 cdio_eject_DEPENDENCIES = $(LIBCDIO_DEPS)
 cdio_eject_LDADD        = $(LIBCDIO_LIBS) $(LTLIBICONV)
 
diff --git a/example/extract.c b/example/extract.c
new file mode 100644
index 0000000..30644db
--- /dev/null
+++ b/example/extract.c
@@ -0,0 +1,270 @@
+/*
+  Copyright (C) 2012 Pete Batard <address@hidden>
+  Based on samples copyright (c) 2003-2011 Rocky Bernstein <address@hidden>
+  
+  This program is free software: you can redistribute it and/or modify
+  it under the terms of the GNU General Public License as published by
+  the Free Software Foundation, either version 3 of the License, or
+  (at your option) any later version.
+
+  This program is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
+
+  You should have received a copy of the GNU General Public License
+  along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+/* Extract the full content of either an UDF or ISO9660
+   TODO: timestamp preservation, file permissions, Unicode
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <malloc.h>
+#include <errno.h>
+
+#include <cdio/cdio.h>
+#include <cdio/logging.h>
+#include <cdio/iso9660.h>
+#include <cdio/udf.h>
+
+#if defined(_WIN32)
+#include <direct.h>
+#else
+#include <sys/stat.h>
+#include <sys/types.h>
+#define _mkdir(a) mkdir(a, S_IRWXU)
+#endif
+
+#ifndef MIN
+#define MIN(a,b) (((a) < (b)) ? (a) : (b))
+#endif
+
+#define print_vd_info(title, fn)     \
+  if (fn(p_iso, &psz_str)) {         \
+    printf(title ": %s\n", psz_str); \
+  }                                  \
+  free(psz_str);                     \
+  psz_str = NULL;
+
+const char *psz_extract_dir;
+
+static int udf_extract_files(udf_t *p_udf, udf_dirent_t *p_udf_dirent, const 
char *psz_path)
+{
+  FILE *fd = NULL;
+  int i_length;
+  char* psz_fullpath;
+  const char* psz_basename;
+  udf_dirent_t *p_udf_dirent2;
+  uint8_t buf[UDF_BLOCKSIZE];
+  int64_t i_read, i_file_length;
+
+  if ((p_udf_dirent == NULL) || (psz_path == NULL))
+    return 1;
+
+  while (udf_readdir(p_udf_dirent)) {
+    psz_basename = udf_get_filename(p_udf_dirent);
+    i_length = 3 + strlen(psz_path) + strlen(psz_basename) + 
strlen(psz_extract_dir);
+    psz_fullpath = (char*)calloc(sizeof(char), i_length);
+    if (psz_fullpath == NULL) {
+      fprintf(stderr, "Error allocating file name\n");
+      goto out;
+    }
+    i_length = snprintf(psz_fullpath, i_length, "%s%s/%s", psz_extract_dir, 
psz_path, psz_basename);
+    if (i_length < 0) {
+      goto out;
+    }
+    printf("Extracting: %s\n", psz_fullpath);
+    if (udf_is_dir(p_udf_dirent)) {
+      _mkdir(psz_fullpath);
+      p_udf_dirent2 = udf_opendir(p_udf_dirent);
+      if (p_udf_dirent2 != NULL) {
+        if (udf_extract_files(p_udf, p_udf_dirent2, 
&psz_fullpath[strlen(psz_extract_dir)]))
+          goto out;
+      }
+    } else {
+      fd = fopen(psz_fullpath, "wb");
+      if (fd == NULL) {
+        fprintf(stderr, "  Unable to create file\n");
+        goto out;
+      }
+      i_file_length = udf_get_file_length(p_udf_dirent);
+      while (i_file_length > 0) {
+        memset(buf, 0, UDF_BLOCKSIZE);
+        i_read = udf_read_block(p_udf_dirent, buf, 1);
+        if (i_read < 0) {
+          fprintf(stderr, "  Error reading UDF file %s\n", 
&psz_fullpath[strlen(psz_extract_dir)]);
+          goto out;
+        }
+        fwrite(buf, (size_t)MIN(i_file_length, i_read), 1, fd);
+        if (ferror(fd)) {
+          fprintf(stderr, "  Error writing file\n");
+          goto out;
+        }
+        i_file_length -= i_read;
+      }
+      fclose(fd);
+      fd = NULL;
+    }
+    free(psz_fullpath);
+  }
+  return 0;
+
+out:
+  if (fd != NULL)
+    fclose(fd);
+  free(psz_fullpath);
+  return 1;
+}
+
+static int iso_extract_files(iso9660_t* p_iso, const char *psz_path)
+{
+  FILE *fd = NULL;
+  int i_length, r = 1;
+  char psz_fullpath[4096], *psz_basename;
+  const char *psz_iso_name = &psz_fullpath[strlen(psz_extract_dir)];
+  unsigned char buf[ISO_BLOCKSIZE];
+  CdioListNode_t* p_entnode;
+  iso9660_stat_t *p_statbuf;
+  CdioList_t* p_entlist;
+  size_t i;
+  lsn_t lsn;
+  int64_t i_file_length;
+
+  if ((p_iso == NULL) || (psz_path == NULL))
+    return 1;
+
+  i_length = snprintf(psz_fullpath, sizeof(psz_fullpath), "%s%s/", 
psz_extract_dir, psz_path);
+  if (i_length < 0)
+    return 1;
+  psz_basename = &psz_fullpath[i_length];
+
+  p_entlist = iso9660_ifs_readdir(p_iso, psz_path);
+  if (!p_entlist)
+    return 1;
+
+  _CDIO_LIST_FOREACH (p_entnode, p_entlist) {
+    p_statbuf = (iso9660_stat_t*) _cdio_list_node_data(p_entnode);
+    /* Eliminate . and .. entries */
+    if ( (strcmp(p_statbuf->filename, ".") == 0)
+      || (strcmp(p_statbuf->filename, "..") == 0) )
+      continue;
+    iso9660_name_translate(p_statbuf->filename, psz_basename);
+    if (p_statbuf->type == _STAT_DIR) {
+      _mkdir(psz_fullpath);
+      if (iso_extract_files(p_iso, psz_iso_name))
+        goto out;
+    } else {
+      printf("Extracting: %s\n", psz_fullpath);
+      fd = fopen(psz_fullpath, "wb");
+      if (fd == NULL) {
+        fprintf(stderr, "  Unable to create file\n");
+        goto out;
+      }
+      i_file_length = p_statbuf->size;
+      for (i = 0; i_file_length > 0; i++) {
+        memset(buf, 0, ISO_BLOCKSIZE);
+        lsn = p_statbuf->lsn + i;
+        if (iso9660_iso_seek_read(p_iso, buf, lsn, 1) != ISO_BLOCKSIZE) {
+          fprintf(stderr, "  Error reading ISO9660 file %s at LSN %lu\n",
+            psz_iso_name, (long unsigned int)lsn);
+          goto out;
+        }
+        fwrite(buf, (size_t)MIN(i_file_length, ISO_BLOCKSIZE), 1, fd);
+        if (ferror(fd)) {
+          fprintf(stderr, "  Error writing file\n");
+          goto out;
+        }
+        i_file_length -= ISO_BLOCKSIZE;
+      }
+      fclose(fd);
+      fd = NULL;
+    }
+  }
+  r = 0;
+
+out:
+  if (fd != NULL)
+    fclose(fd);
+  _cdio_list_free(p_entlist, true);
+  return r;
+}
+
+int main(int argc, char** argv)
+{
+  iso9660_t* p_iso = NULL;
+  udf_t* p_udf = NULL; 
+  udf_dirent_t* p_udf_root;
+  char *psz_str = NULL;
+  char vol_id[UDF_VOLID_SIZE] = "";
+  char volset_id[UDF_VOLSET_ID_SIZE+1] = "";
+  int r = 0;
+
+  cdio_loglevel_default = CDIO_LOG_DEBUG;
+  
+  if (argc < 3) {
+    fprintf(stderr, "Usage: extract <iso_image> <extraction_dir>\n");
+    return 1;
+  }
+
+  psz_extract_dir = argv[2];
+  if (_mkdir(psz_extract_dir) == 0) {
+    printf("Creating directory: %s\n", psz_extract_dir);
+  } else if (errno != EEXIST) {
+    fprintf(stderr, "Unable to create extraction directory %s\n", 
psz_extract_dir);
+    return 1;
+  }
+
+  /* First try to open as UDF - fallback to ISO if it failed */
+  p_udf = udf_open(argv[1]);
+  if (p_udf == NULL)
+    goto try_iso;
+
+  p_udf_root = udf_get_root(p_udf, true, 0);
+  if (p_udf_root == NULL) {
+    fprintf(stderr, "Couldn't locate UDF root directory\n");
+    goto out;
+  }
+  vol_id[0] = 0; volset_id[0] = 0;
+  
+  /* Show basic UDF Volume info */
+  if (udf_get_volume_id(p_udf, vol_id, sizeof(vol_id)) > 0)
+    fprintf(stderr, "Volume id: %s\n", vol_id);
+  if (udf_get_volume_id(p_udf, volset_id, sizeof(volset_id)) >0 ) {
+    volset_id[UDF_VOLSET_ID_SIZE]='\0';
+    fprintf(stderr, "Volume set id: %s\n", volset_id);
+  }
+  fprintf(stderr, "Partition number: %d\n", udf_get_part_number(p_udf));
+
+  /* Recursively extract files */
+  r = udf_extract_files(p_udf, p_udf_root, "");
+
+  goto out;
+
+try_iso:
+  p_iso = iso9660_open(argv[1]);
+  if (p_iso == NULL) {
+    fprintf(stderr, "Unable to open image '%s'.\n", argv[1]);
+    goto out;
+  }
+
+  /* Show basic ISO9660 info from the Primary Volume Descriptor. */
+  print_vd_info("Application", iso9660_ifs_get_application_id);
+  print_vd_info("Preparer   ", iso9660_ifs_get_preparer_id);
+  print_vd_info("Publisher  ", iso9660_ifs_get_publisher_id);
+  print_vd_info("System     ", iso9660_ifs_get_system_id);
+  print_vd_info("Volume     ", iso9660_ifs_get_volume_id);
+  print_vd_info("Volume Set ", iso9660_ifs_get_volumeset_id);
+
+  r = iso_extract_files(p_iso, "");
+
+out:
+  if (p_iso != NULL)
+    iso9660_close(p_iso);
+  if (p_udf != NULL)
+    udf_close(p_udf);
+
+  return r;
+}
-- 
1.7.8.msysgit.0