Re: [Surprise-dev] GNU Parted and Partition Surprise

[Jan Kratochvil]

On Sun, Jul 22, 2001 at 08:37:30PM +1000, Andrew Clausen wrote:
> On Sun, Jul 22, 2001 at 10:16:38AM +0200, Jan Kratochvil wrote:
...
> > The rebuild of the filesystem (only its metadata) is done in-place (no need 
> > for
> > some additional disk) by our General Converter (gconv). Such approach is
> > slower, of course, but we can do ANY change (such as changing a blocksize
> > etc.). For end user it is completely transparent (only a bit slower).
> 
> Ah, so THAT'S what it's doing!  I must admit, I found the Surprise source
> rather difficult to understand...

BTW have you really read "docs/sgml/{user,tech}.sgml" ?
I've uploaded it now for your pleasure to:

                http://vellum.cz/~short/priv/surprise/user.html
                http://vellum.cz/~short/priv/surprise/tech.html

You need several tools (docbook, m4 etc.) to rebuild it from the CVS.


> I was thinking about doing a similar thing in Parted, and I have thought a
> lot about the design, to the point were I could code it, but other things are
> a higher priority...
> 
> Anyway, things like "changing blocksize" are doable in Parted.  (In fact,
> for FAT, the only issue is figuring out a nice API interface... the low-level
> code already written and used for block size shrinking)

Yes, verything is "doable" in Parted but you need to write special 'module' for
each feature. IMO the whole filesystem rebuild takes it more simple and
straightforward (and slower).

For example filesystem conversions in Parted need N^2 (N=supported filesystem
types) modules while Surprise needs only 2*N modules (2='import' and 'export'
part of the filesystem module).

> BTW: is this a simpler alternative, to convert from FS type A to B,
> if both A and B are supported read/write by the kernel, and both A and B
> can be shrunk / grown, and one can move-the-start: just do:
> 
> * shrink A a bit
> * grow B a bit
> * copy a bit from A to B
> * repeat, until all is copied
> * remove A
> * grow B to full size of A
> 
> Is that going to be significantly slower than your General Converter?
> It could also be applied to things like changing the block size...

It has one big performance hit: You really need to move the data between A and
B. Surprise only rebuilds metadata so it should be much faster.

But I don't think that conversion speeds are too much important so yes, you can
do it by your proposed way - why not? When I was talking with my friend admin,
I was saying that Surprise is currently pretty slow as it doesn't yet have much
acceleration functions. He answered that it he does't matter how it is slow as
he really doesn't mangle his disks every day. The need for speed is maybe a
professional deformation of partitioning software hackers. :-)


> > So Parted (or even PowerQuest Partition Magic) is a good approach for fast
> > operations but it is too limited.
> 
> Parted's approach is also simpler / easier to maintain.  (Another reason
> I didn't try doing a generalize resize / whatever)

You have: shrink, grow, reblocksize, convert-to-X, convert-to-Y, ...
Everything as separate functionality - it is much more code I think than
'import' and 'export' approach. A much more code to debug. While you will be
adding more supported filesystems (and their conversions), the code base will
grow to unmaintainable sizes.

What do you see complicated on 'import' or 'export' filesystem parts? Note:
These parts don't solve anything about overlapping filesystems, this problem is
left for General Converter.


> Anyway: it would have been wise for you to talk to me when you discovered
> Parted.  I'm sure we've learned lots from our experiences, and have lots of
> tricks to tell each other ;)

There is one _problem_: This was a university team project and unfortunately it
isn't wised to just extend foreign project (Parted) - such 'extension' would
then be hard to advocate as OUR OWN PROJECT.

So when it is now written it would be nice to look at the code and designs
which are better in both of the projects and merge it somehow (we planned to
merge Parted to our more-general-approach codebase before our project got into
its current stagnation).

> Also, the partition handling code in parted should be shared, IMHO.
> It would be fairly easy to add a
> 
> ped_disk_convert (PedDisk* disk, PedDiskType* new_type)
> 
> (this is planned ;)   (Note: PedDisk represents a partition table)

/* You have to set the partition table type in a loop through all the
 * partitions of the disk as separate partitions of the same disk can be in
 * different partition tables:
 * PC partition table can have BSD partition table as its sub-space.
 * Such shared partition table we identify by UP_ID_PTBL_PCBSD.
 */

disk=final->fi_exp_disks+disk_no;
for (part_no=0; part_no<disk->di_count; part_no++) {
        disk->di_up[part_no]->up_entry_type = UP_ID_PTBL_MAC; /* for example */
        /* We want the partition enty fields to be recalculated: */
        disk->di_up[part_no]->up_entryvalid = 0;
}
commit_disks(final,NULL,0,ERR_PASS);

This is not planned, it is working.


> There's lots of duplication... (apart from anything else, us thinking
> about the same ideas twice)

It was partially a duplication but it was needed due to the damned school.


> Also, one question: is it safe to interrupt a "generic conversion" ?
> (eg: power failure)

Simply: NO.
But we differentiate between the 4 consequence phases of commit where only the
fourth phase is fatal to be interrupted. Phases 1-3 only modify free blocks
(and memory and swap) of the partitions.

Technical documentation
12. General Convertor
12.8 Used algorithm

* The fourth phase - Crazy permutating blocks
  This phase is implementation of swapping machine. This machine is simple to
  eliminate possible bugs in this critical section. And it is designed for
  pertient future support of journalling. The main task is to read swapping
  program and apply this swaps.


                                                Jan "Lace" Kratochvil