Re: Tablicious in Octave Forge [was: Re: [GSoC 2021] How should I do now with project Table datatype]

[Andrew Janke]

On 3/12/21 9:25 AM, Guillaume wrote:
> I think having a GSoC student working on the implementation of table
> would be great and I would skip the Forge packaging to focus on adding
> the functionality in core Octave (of course, reusing as much as possible
> of Tablicious).

Okay. I'll think about it. To be honest, I think implementing table
might be kind of a large project so it might be a while before it lands
in an Octave release, and I'd still like to get more user eyeballs on
Tablicious in the mean time.

(I don't expect anyone else to put time in to getting Tablicious on
Forge; that'd be just me.)

> Once the low level data structure is in place (a write-up on "proxy key"
> would be great),

I'll probably make documenting "proxy key" and related stuff my top
priority here, then.

> the burden of the implementation of all table related
> functions could be shared among all interested parties:
> https://www.mathworks.com/help/matlab/tables.html
> Would it also be possible to decouple projects? Why not implementing
> (part of) table without requiring strings or datetime or categorical or
> missing? I feel that if the foundations are there and robust, we can
> build the rest one bit at a time.

Yes, mostly: I think table needs missing/ismissing(), because that's a
fundamental part of the join semantics. Defining and implementing
missing/ismissing() is pretty easy.

The other types - datetime, string, categorical - are mostly separable.I
just threw them in to Tablicious because I think they are rather useful
types to stick in to columns in tables, and I wanted users to be able to
install just the one package instead of 3 or 4.

Except for how those types get displayed when a table is pretty-printed
or exported to Excel or CSV: there's no existing interface (in Octave or
in Matlab) for defining custom displays or conversions for types or
classes on a per-element basis when those types are placed inside a
composite type (as opposed to a custom display format for the overall
array, like disp() does).

That is, for datetimes in a table:

dt = datetime({'1/1/2001', '2/1/2001', '3/1/2001'}');
t = table(dt);
prettyprint(t)

I really want it to do this:

        dt
    ___________
    01-Jan-2001
    01-Feb-2001
    01-Mar-2001

and not this:

           dt
    _________________
    [1-by-1 datetime]
    [1-by-1 datetime]
    [1-by-1 datetime]

If the interface for defining that sort of display could be defined,
then string/categorical/datetime are totally decouple-able. (Or you
could just give up on a generic display customization interface and
write up special-case handling for each of string, categorical, and
datetime when they do appear in Octave. That might be the
maximally-Matlab-compatible approach anyway.)

Cheers,
Andrew