Re: [Evms] Questions about portability

[Andrew Clausen]

Christoph Hellwig wrote:
> > one single library != one single library
> 
> Hmm. I don't have to understand the philosophical background of this
> sentence? ;)

;-)

> > Yuck!  So, should the hints describe how to get the information as
> > well?!
> > (look in byte xxx of the partition, etc.)  Is /etc/lvms/align.<fstype>
> > Turing complete?
> 
> I doubt.
> 
> It should just be a plain key/value file
> 
> align:          4kb
> rounding:       32kb
> max-size:       4gb

"align" depends on the on-disk file system (not merely the file system
type).
For example, if you have an ext2 file system, that starts at sector 63
(512
byte unit), and the file system has a block size of 2k, then:

        new_start = old_start + X * block_size

max_size might be a function of certain properties of the file system,
like the block size.  (it is rather difficult to grow the block size,
and if the resizer can't handle it, then this is going to be a factor)
However, there is no way a front-end can know this, unless it asks
the resizer.

> > This is information that must be COMPUTED, by the resizer, from the
> > file system.
> >
> 
> Yes.  But unless the fs is totaly braindead this computations follow simple
> rules.

I disagree.  ext2 doesn't follow simple rules, and I don't think it's
braindead.  FAT is braindead, and it also doesn't follow simple rules :p

>  And I don't think we should overdesign the whole system just to
> support completly braindead filesystems.

I think we should!  The "overdesigned system" (libparted) is clean
enough
for me.  (Heck, people are always trying to make it uglier, but I don't
let them ;-)

I mean, why bother with partitioning, except for compatibility?
Partitions suck!

> These filesystem would usually be foreign fses and only occur on one partition
> type, so the hints can be added. (Yes, that's _horribly_ ugly, but less ugly
> than messing up the whole design).

I don't follow.  How do the hints work?

> > Fine.  We can separate libparted into separate source trees.
> > I don't see the point (yet), but maybe it may make sense in future.
> 
> E.g. the 'get fs blocksize' code is better placed in the kernel - there
> might be lots of other uses for it.

Maybe this is true, but the block size doesn't tell you anything about
the constraints.  It's the *resizer* that has the constraints.  Only the
resizer knows what it can and can't do.  My FAT resizer USED to be
unable
to shrink the cluster size, or move the start... but now it can.

> It's just the one tool for one thing design compared to 'put everything
> related to partition and fs handling in one lib'.
> 
> > You don't need to move the LV with LVM.  You can shrink the end, and
> > copy the start to the end, so to speak.
> >
> > However, moving the entire LV is a bad idea.  If it gets interrupted,
> > you can't undo the operation.  Doing journalling on disk will slow
> > you down by about a factor of 10, compared to a normal copy without
> > journaling (reason: all the seeks).  Besides, I don't see how
> > journalling is "clean", when you can do an in-place resize atomically
> > by other methods.
> 
> I can only tell how the current Linux LVM does it:
>   - it locks
>   - moves it to it's new location
>   - changes the remapping table
>   - it unlocks
> 
> The locks are currently global - but they could be made more finegraded.

That's my first suggestion, right?  (Which I don't object to ;-)

> > What if the program/library is modular?  I mean, couldn't you say the
> > same thing about a Linux distrubition:
> >
> >       "I don't like these Linux CDs that try to do everything!  I
> >       like buying Windows, and then MS Word separately..." etc.
> >
> 
> Actually it's the other way around:

Absolutely!  But I'm sure you'll here some people proposing that
(ridiculous) argument!  (I have!!!)  My point is: "Do one thing and do
it right." is easily taken out of context.  It's an extremely vague
statement, and I'm not sure how it is relevant here.  Each module
in libparted does one thing, and does it right.  I don't understand
how being a module as opposed to a program is relevant to a "Do
one thing and do it right" argument.

>         I don't like windows where there are a few programs trying
>         to handle everything, e.g. the office suits.
>         Instead I write my text using vi, run ispell over it, then
>         tex, I preview it using xdvi, and print it using dvips.

Agreed!

> > I mean, what's so special about the unit "program/library"?
> 
> A program is much more intuitive and simple.
> 
> (pointer: the same thing is currently discussed on lkml
> in the kORBit thread).

Comparison between programs and libraries:
(1) programs and libraries have interfaces
(2) programs are easy to interface with users, but difficult to
interface
with other programs (i.e. difficult to be used by other programs).  When
I say "difficult", I mean for non-trivial stuff.
(3) libraries are easy to interface with programs (i.e. used by other
programs), but difficult to interface with users

(4) and (5) follow from (2) and (3):

(4) libraries are easily wrapped by programs, to provide an interface to
users
(5) programs are difficult to wrap in a library interface

> > Besides, it's unfair to call libparted a single program/library, in the
> > sense that it's a monolithical structure.  It is fair to say it has a
> > single API, though.  (ATM, libparted is many libraries statically
> > linked)
> >
> > > I _much_ prefere the tradtitional UNIX way of doing things:
> > >
> > >         Do one thing and do it right.
> >
> > Each module "does one thing and does it right".
> >
> > I think you are taking this idea out of context.
> >
> > ("One thing" can equal "run the whole system" or "plot a pixel"
> > or "handle data storage".  The question is: at what level should
> > it be "one thing"?  A single function?  API?  module?  program?
> > library?  collection of libraries?  package?)
> 
> Actually this might be worth a discussion.
> I the traditional UNIX context one thing is a small, specific task, like
> do a filesystem check for one specific filesystem, do a filesystem check
> by invoking low-level fsck tool, check a text for spelling mistakes,
> render a text written in a markup language or give out one file on the tty.
> The level is usually one program.  These programs may of course share common
> functions (like append this string on another, give me information about one
> specific fs, print out a string, create a file) on a library basis.

Yep.

However, I think file systems and partitions (and maybe LVMs) have
Initimate Relationships TM.  Actually, they aren't very intimate (hence,
libparted is able to be very modular), but they are too intimate to be
separated completely (unfortunately).

I can see that the last point is contentious, but I don't see any
disadvantages to taking this attitude.

Here's another interesting option: libparted is structured such that
it would be very trivial to write resize.ext2 and resize.fat front
ends.  VERY TRIVIAL (such frontends would be about 30 lines of code).

Maybe, we should keep libparted as-is for partition table stuff,
but use the simpler front ends for LVM.

However, is this sufficient for conversion between LVM types,
or converting from file-systems-on-partitions to physical volumes
containing a logical volume with the old file system?

Maybe it is possible...

> > > I don't see any reason why a LVMS should be interested in that.
> > > If it has partition type handling built in anyway, it would just tell the
> > > low-level partition handling tool: write me a partition type for fs XYZ.
> >
> > Yeah, but the mapping from fs <-> partition type is neither surjective
> > nor injective.  i.e. you need more information to know what type the
> > partition is, than simple the fs.  If you ignore this, you end up
> > leaving Windows and MacOS unbootable (or Linux unbootable on Mac's,
> > due to their broken firmware).
> 
> Could you elaborate that a little more?
> I don't get the point yet.
> It might be just because I'm more interested in the actually data managment
> then playing with ugly details like partition tables ;)

In the case of FAT/Windows: if the file system type and the partition
type don't match, M$ chokes.  There are 12 different partition types
(numbers)
corresponding to different types of FAT partitions.  These correspond to
various permutations of:

(1) FAT12/16/32
(2) the size of the file system (some brain-damage, WRT small file
systems)
(3) whether the partition is "hidden"
(4) whether M$ should use LBA

(1) and (2) should be determined by the file system code.  You could
hack around this, by creating different file systems for fat12, fat16
and fat32, but this seems dubious to me.  (For example: progams
might get confused when the resizer automagically converts from fat16
to fat32, or whatever)

I think my Mac example was bogus... the partition types need to be
right,
but it is merely a function of the file system type.  I'm sure other
cases will pop-up... Sun disk labels for example, allow partitions
and the disk label metadata to overlap, so you have to be careful
about which file system you use, so as not to clobber the disk label.
(ext2, and new linux-swap don't use the first 1k of the partition)

> > > > Here's another: how much free space does the resizer need to resize
> > > > the partition to a diferent size?  This is resizer-specific (resizers
> > > > often need spare space to grow metadata, or for temporary "swap space",
> > > > etc.)
> > >
> > > I come from the sane LVM world where is is 0.
> >
> > Huh?  Where do you grow your group descriptor tables to?  If you want to
> > be atomic, then you also want to keep both and new metadata.  0 is
> > not only insane, it's not possible.
> 
> It looks like we are talking abot s.th. different:
> sure you need to allocate mores pace then the pure data space that will
> be available later.  But just like e.g. mkfs you will create the fs in
> it's 'vanilla' size and will have less avaible.

But, it's useful to know how much free space you need, before hand,
so you can plan ahead.  (Think: automatic partitioning)

> > > But for such things like constant number there exists yet another nice
> > > UNIX thingie:  ASCII config files.
> >
> > Can't be done.  Eg: for ext2, you would need to know if it has
> > sparse super blocks (to know how many group descriptor tables need
> > to be shrunk/grown, and by how much)
> 
> Why should the generic resize frontend want to know about that?

Automatic partitioning, and "planning ahead".  (Eg: figuring out how
you want to rearrange your partitions, and then clicking "go!".  It's
nicer to know before hand, if it's going to work)

> > > > Also, it cuts back the other way: there is useful stuff that resizers
> > > > (and other utilities) can share, if they link (dynamically, if you
> > > > want...) together.  libparted's error handling, for example.
> > >
> > > I don't care wether we have a small liblvms that implements things the
> > > tools _might_ share, but I don't want it to be too tightly coupled.
> >
> > I also don't like things "too tightly coupled".  Just, we seem to
> > have different ideas of "too tight".  BTW: I don't think your ideas
> > make the coupling any "looser", they just make the coupling occur
> > elsewhere.
> 
> One example: For whatever reason I just want to resize the LV, and
> not mess with the fs.  In my way I would just call the spezific
> lowlevel tool and be done - no additional overhead.
> With your lib you could either not do it - or you would have to
> add another frontend or at least a new command line switch.

Not true.  You already can do it!

Parted has a fake "raw" partition table, that has exactly one
partition that occupies the entire device.  So:

        # parted /dev/hda1 (or the LV device) resize 1 0 [new-size]

(where 1 = the fake "partition" number, and 0 = the new start)

Obviously, I could write a very simple front-end, to have
equivalent behaviour to resize.ext2, or whatever.

> Or I have writen a new test filesystem and write a little resizer
> for it - in my scheme the high-level tool would simply call it
> if it is in the right place - how would you integrate that?
> Corba? Messing around two nights with XS?

ROFL!  /me imagines how CORBA and libparted might work together...

libparted is easy to port to... (in fact, both the ext2 and fat
resizers were originally separate programs)  You could almost write
a sed script to do it!

> > BTW: I see this is beginning to turn into a holy war, hehe.  Please
> > don't take this stuff personally ;-)  I think these are important
> > issues, so I think we should continue the debate ;-)
> 
> Ok.

At least I'm arguing with a smart guy ;-)  Philosophical debates
aren't as much fun when they have implications... hehe

Andrew Clausen