Re: Interface for SCSI transactions ?

[olafBuddenhagen]

Hi,

On Fri, Sep 23, 2011 at 05:57:44PM +0200, Thomas Schmitt wrote:
> Olaf Buddenhagen:

> > Unless there is some practical difficulty, I *strongly* suggest to
> > describe the individual struct members with MIG, rather than doing
> > the marshalling (serialisation) by hand.
> 
> Regrettably MIG explicitely refuses to model struct entrails. mig.ps
> page 3: "There is no way to specify the fields of a C structure to
> MIG." (which is quite a disappointment)

Yes, it's not possible to pass a complete structure as a sigle
parameter, except by treating it as a flat byte (or int) array...

The *right* way of doing it is to list the elements of the structure as
individual RPC parameters.

> > This means the RPC will be really SCSI-specific. It is a bit ugly,
> > as it will add a meaningless entry for all other device classes;
> 
> As it is planned currently, it will be usable for any device oriented
> transaction, that takes a struct as input and delivers another struct
> as output.

Well, here I was talking about using explicit parameters (see above),
i.e. *not* having a generic device_transact() RPC-over-RPC.

I totally understand the idea of device_transact() -- and I listed a
number of arguments in favour and (mostly) against it...

> > the transaction RPC isn't actually likely ever to be used by
> > anything else anyways.
> 
> Not necessarily. Who ever has the plan to make a specialized driver in
> userspace, will at least find an existing RPC gateway.

No no no no! See my reply to Samuel -- I will violently protest any
attempt to use such an RPC-over-RPC call in actual Hurd userspace
interfaces :-)

Perhaps I should actually book that as another argument *against*
introducing device_transact(): making sure nobody EVER considers
implementing it in userspace...

> I meanwhile compared the transaction structs of Linux SG_IO, Linux
> CDROM_SEND_PACKET, FreeBSD CAM, and Solaris uscsi. Mostly Linux SG_IO
> is appealing. With a touch of CAM and slightly extended. So my structs
> are designed now, with much room for improvements in the communication
> with underlying SCSI or IDE drivers.

Great :-)

> Astounding that Linux SG_IO totally hides the auto sense mechanism,
> which issues a REQUEST SENSE command after the payload command yielded
> CHECK_CONDTION. CAM and uscsi both expose some info about this
> fetching of sense data. I plan to allow generous info about it
> (provided one can get it from the kernel).

Hm... I don't know anything about this stuff; but my intuition is that
if this *can* be hidden from users (without adverse effects), it
should... Is it actually useful to have this data?

> > (Doing manual marshalling is extra effort; and it will have to
> > change again when moving to userspace drivers -- while using a
> > proper hurdish RPC from the setout means it can be directly reused
> > for the userspace driver interface.)
> 
> Should i know the term "manual marshalling" ? (Google ... but still
> half clueless.)
> 
> Do you mean my plan to have serializer and de-serializer ?

Sorry, I meant to mention the other term, but somehow forgot :-(

Yes, marshalling is the correct term for the
serialisation/de-serialisation done for RPC parameters. It's the main
purpose of the RPC stubs generated by MIG -- and it's the major reason
for an interface generator being introduced in the first place... Which
is why I'm very sceptical of any interface that doesn't use MIG's
abilities here, instead doing the marshalling by hand.

> My hairs are raised by the idea to transfer plain struct memory
> through the RPC. Sure, it works as long as client and server run on
> the same architecture and were compiled in a compatible way ... but
> ... urgh !
> 
> So i think it is appropriate to prescribe an explizit representation
> layer for the structs and their components.

...such as MIG .defs? ;-)

If you really consider field sizes a problem, that's indeed one more
reason for passing each field as an explicit RPC parameter (using a
machine-independent type).

I wouldn't worry about it too much though. While Mach was originally
designed to allow network-transparent IPC between machines using
different architectures, the network IPC code doesn't even exit anymore
in the current gnumach tree. (And was broken before -- I wouldn't be
surprised to learn that it never fully worked in any Mach
implementation...) Indeed there are already a few RPCs in the Hurd
interaces that transfer structs as plain byte arrays... And after some
close consideration, I decided to do the same myself while porting KGI.

This only works for pure structs with fixed-sized fields of course. The
Linux sg_io structure seems to have a number of pointers to
variable-sized arrays -- passing at least these as individual RPC
parameters is the only way to avoid complex, error-prone and ugly manual
marshalling.

The more I see here, the more I'm convinced that device_transact() is a
really bad idea. The RPC-over-RPC approach might be acceptable for
device_get_status()/device_set_status(), which tend to be fairly simple
calls; but for something as complex as the SCSI transactions, it gets
really really ugly.

> Open questions concern the wiring from glue/block.c to ide and scsi,
> and the implementation of the counterpart functions in userspace.
> (Where to implement the serializer for the input struct , and the
> de-serializer for the output ? Where to define the function_code value
> ?)

At the peril of sounding like a broken record: I suggest using MIG for
the marshalling, so you no longer have to think about these questions
:-)

> > OTOH, machines able to run Mach natively are indeed more likely to
> > have an ATAPI writer (including mine), so it's probably more useful
> > to focus on these...
> 
> Really old CD burners were attached to really old SCSI controllers,
> back in the 1990s. I got my last SCSI burner in 2000. It died 2003.

Now that you mention it, it's slowly coming back from external
storage... ;-)

As CD burners generally break down after a couple years, we really don't
have to worry about "true" SCSI burners anymore. ATAPI burners are of
course also on the way out in favor of SATA -- but still the most likely
in a box running Hurd natively...

(I just remembered that I actually have TWO of those: in addition to the
ATAPI burner I bought recently for my ancient Pentium3 desktop machine,
my T40p notebook from 2003 or so has a DVD-reader/CD-writer
combo-drive.)

-antrik-