Re: [Texmacs-dev] Roadmap for GUI's and porting TeXmacs

[root]

I have been pretty silent lately but was investigating the same
problems. So here is my take.

  An aside: I think your new menu system is a step in the right
  direction even it seems not satisfactory to David. I have my own
  ideas on that (discussed with David on IRC) but I think they are too
  personal and we want something that has a general appeal so I don't
  care much about them.  And BTW, I like very much the light blue box
  that show embdding environement.  It was especially needed for
  editing tables with invisible borders for example. Better document
  navigation is a goal but less so urgent than the issue discussed
  in your mail.

The roadmap has to go beyond the GUI. We can't implement everything at
once but everything will eventually be tied together so we can't think
these things independently. I am sorry if I generalise too much when
you want a concrete roadmap. But the sooner things are thought out,
the less bumpy will be the ride later.

The good news is that a lot of it has been thought out for us and that
real GUI toolkits must solve these problems too and expose their
solutions as an API.  I have been amazed by the progress made in the
last ten years. I had to totally change my mind frame.
For example, I encourage you to read the "overviews"
entry in http://doc.trolltech.com/3.0/index.html (as I should have
first done).  In fact, this mail is partially a rehashing of the
concept exposed ther in the context of TeXmacs.

High Level Event Framework

  More generally, we need an powerful event model that also deals with
  timers and all kinds of inputs/outputs. Better, we need a higher
  level framework on top of that. In fact, I have been amazed how far
  TeXmacs has been able to go without these basic facilities.  It is
  nevertheless a major obstacle to further growth.

  So, when we speak of port to a toolkit, we must think further than a
  GUI toolkit. Anyway, any real toolkit has to deal with this problem
  somehow so if it does not expose it clearly, this is kind of scary.

  Personnaly, I am biased and see that with an indirection.  I am
  interested in perl and POE is good very high level "multitasking and
  networking framework for perl" that is designed to be pluggable into
  another framework (currently Tk and Qt). POE is a whole thriving
  ecosystem within Perl. See:
  http://search.cpan.org/search?mode=module&query=POE

Embedding

  We want TeXmacs to be abble to be embed and to be embedded.  The
  toolkit(s) we will choose has/have to support that embedding.  I
  really believed that TeXmacs will achieve fame thru applications
  that will use it as a widget. Then everyone will be ready to TeXmacs
  as a tool in its own right.  We eventually have to think of a
  convention for embedding non TeXmacs documnent in TeXmacs documents. 
  I am speaking here of the external format of a document:
  as it is saved to disk.  Also, we must be modal: many people will
  only read stuff in a TeXmacs widget: there are many readers for a
  writer.  We read a lot of documents a write only a few even if the
  later is more complex.  So, we need a read/only browsing mode.

Limited Resources

  We want to avoid multiple toolkit ports if not needed so we really
  wants a toolkit that buys us a side effetc a portability to other
  OSs.

Data Representation and other Extensions Languages

  As I said, I am biased and have a sincere hate of non infixed
  languages even if/because I have practicized many of then
  (PostScript, FORTH...) and want Perl in.  The recent events, the
  release of Perl 5.8 and the design of the upcoming perl6 and its
  parrot runtime confort me in that thought.

  We should be eventually able to support interpretors embedding
  (single and multiple).

  So, The Guile start-up code must be easily interpretable in another
  potential extension language. Reciprocally an extension language
  is interesting only if one can write easily such an intepretor.

  We must eventually have clear conventions for the representations of
  menus and TeXmacs GUI.  The tendancy in current toolkits (Gtk, Qt)
  is to read a XML representation of the GUI and build it using a
  widget factory.


Exposing an API

  Modern toolkit have widgets that support concepts like slots and
  object properties (like you youselve noted). We must think of the
  main TeXmacs widget that way.

  I think the best thing Joris can be do for TeXmacs in the short
  term, is to work on that.  I know that internals TeXmacs structure
  may change but an API is better than none. You don't want someone
  who feel TeXmacs cool and mess directly with the internals to make
  it his own thing (say for embedding perl :P for example). You want a
  maximum of people to hook to TeXmacs, you don't want them to mess
  with it or to make a mess of it or to come with a fork to support
  their own things.  Think TeXmacs as an object oriented library.

  A trivial example that is very involved when it should not be: I
  want to support a menu that opens buffer in a new window as well as in
  the current window. I have to mess with code that pee guile. No way!
  You must expose an API to your internal structures. They will change
  as your understanding of the issues will progress (no way around
  that) and this API will become an adaptater to next one.


The road toward success

  TeXmacs does not have the success it deserves. Too late to change
  the name, but borrowing the name of its conceptual parents makes
  it dificult  to stand on its own. TeXmacs is TeXmacs and is
  neither emacs or TeX.  What are the problems to overcome?

    1/ Perception: people think they don't need TeXmacs and that it
    will have adverse effects on the structuration of their documents,

     Power users just use (La)TeX and don't quite figure out the
     benefit of a wisiwig typesetter.  I am amazed at the popular myth
     initiated by the LateX book: wisiwig interface leads to less
     strutured documents. In fact, my understanding is that Joris is
     internally moving out of TeX because it is not structured enough
     and that intelligent interactivity like structured navigation
     force the TeXmacs designer to think more structured document
     formats. Am right or wrong on that one, Joris?

 2/ It is way to difficult to learn 

  I am not even speaking as a programmer here.

  In demos, many people are amazed at the facility Joris can built a
  complex strucuted document with mind and eye boggling maths but they
  can't reproduce that at home. It is incredibly frustrating.  The
  layout of the menus is only part of the problem. Wizars/druids will
  help there. Without a real toolkit, we can't even think to build
  them.

  TeXmacs will eventually be a widget and a library (the sooner, the
  better) and it may indirectly find sucess that way.  We rightly are
  concerned by what external tools can bring to us and at what
  cost. Licensing problems come to mind.  But Open Source/Free
  Software is really an ecosystem and must be thought so.

  For example, I am amazed how weak are the interactive text widget
  classes in Qt comparatively to the Tk Text widget for example. In Tk
  you can tie tags to text parts and bind properties (like font and
  colors) and event handlers to tags.  Perl programmers are addicted
  to such features and despite the poor look and perfmonce of TK, they
  seem not uninterested to move to Qt and Gtk or whatever.

  David is rightly bothered by the Trolltech license.  I am more
  thinking of what TeXmacs will bring to the Qt echosystem if it is
  available as a Qt widget. Or as Qt widgets, if we also want to
  mimick existing APIs like the Tk Text widget. It is OK to have
  "degenerated" version of the TeXmacs widget if that can lead more
  people toward TeXmacs.  Also curiously, when most widgets class
  dealing with text layout are read only/mode (for outputting
  document, not write them), TeXmacs is lacking a read-only mode to
  browse documents. Who do the most should be the best as doing the
  least.

Now some comment on Joris mail


On Thu, Jul 25, 2002 at 01:25:56PM +0200, Joris van der Hoeven wrote:
> 
> I think that it is time that we fix some very general strategy
> on how we will port TeXmacs to other platforms.
> Here are a few ideas.
> 
> 1. I absolutely want GUI independence.
>    There are several reasons for this:
> 
>    a) GUI's are instable: what is a nice GUI now maybe
>       a bad or unsupported GUI later on. I lost a lot of time
>       using the Openwin toolkit, which was standard in GNU/Linux
>       distributions 5 years ago, but which does not exist anymore....



> 
>    b) Licensing of GUI's may change: look what happened to Qt.
>       When Qt was proprietary, it was interesting to use Gtk instead.
>       Now that Qt is almost GPL, Gtk seems to be slowly dieing...
>       But Qt may rebecome proprietary!


I think that Free Software/Open Source is intimately tied to linux
that "owns" most source contributors. It seems that everything that
Sun touches is doomed. Gtk is loosing ground since Sun has encorsed
it.  Qt is progressively becoming the de-facto standard in linux (even
if it is thru the even higher level KDE but more unsound abd less
documented).  Gtk has not good documentation compared to Qt.

It is indeed possible that Qt rebecomes proprierary but we must start
by one port. Qt is the best bet.  TrollTech has a sound model. They
derive their technical sthrength from Linux but make money out of
Windows. So the less ideal for us license.

> 
>    c) GUI independence forces good modular design.
> 
>    On the negative side, GUI independence forces us to
>    use some kind of a "greatest common divisor" strategy,
>    so that we may be unable to exploit certain fancy features
>    of a given GUI. Nevertheless, this can also be considered
>    to be a feature, because unstable features will be
>    naturally discarded.

The problem is the main widget. The rest, you alreasy almost
abstracted as an API. I made an experiment to untie the main widget by
making TeXmacs a main only widget API to gain some understanding. It
was surprisingly easy.  You have a skewed view of the problem of the
other widgets because you had to implement and maintain them. Really
it should be a no brainer.

Once I had finished cooking two patches for the menus (more on that
later!), I will investigage the QXtWidget. It seems that we should be
able to first derive the TeXmacs main widget from it but that still
ties us to X so it is a short term solution.


> 
> 2. I have a strong preferance for splitting the GUI into two parts:
>    the low-level interface with the Window manager and
>    the toolkit with all kind of widgets. The reasons are:
> 
>    a) This modularity is already present,
>       so we remain compatible with ourselves.
> 
>    b) It makes it possible to port TeXmacs in a more gradual way.
> 
>    c) Even if we don't manage to port the widgets of a given toolkit,
>       as soon as we managed to port the first part, then *at least*
>       we can run TeXmacs on all supported architectures by the toolkit,
>       using our own widgets.
> 
>    From the external point of view, the toolkit should really
>    be a list of widget constructors (like in widget.hh).
>    This will enable us ultimately to construct the precise interface
>    in a very dynamic way using the Scheme extension language
>    (or anything else).

The best bet is on widget factories that are fed XML (both Qt and Gtk
do that) We will have to do that dynamically for menus. I don't like
much this format but this is the norm. It should be easier to adapt
from one XML format to another than from an API to another.

> 3. From a technical point of view, how do we find the greatest
>    common divisor of several GUI's? How to make them compatible?
>    For this, we need to analyze at least two or three existing
>    GUI's first, as I and David started to do with the list
>    from a previous mail. We also need to analyze the design
>    strategy behind the present GUI and where we may expect
>    problems to come from. Here are a few points:
> 
>    a) TeXmacs is event driven system with a medium level of typing.
>       In compairison with Qt for instance, we do not have
>       the type safeness of the SIGNAL/SLOT system.
>       However, our event system has the powerfull advantage that
>       we can write so called transparant or modifier widget.
>       For instance, imagine that you have an arbitrary widget w,
>       which we want to modify or give some additional structure,
>       like a background color, another size, an additional method,
>       a button-like structure, etc. Then we may do this in a way
>       to send all non treated events (regardless of what kind!)
>       by the modified widget to w.

Probably the main work is to translate the underling toolkit event
model to the TeXmacs one. This goes for d), g).

> 
>    b) Transparent menu entries. This is a particular application of a).
>       A menu entry may be an arbitrary widget in TeXmacs.
>       Also, the menus are fully dynamic.
> 
>    c) Our system is widget-based (well, some call that gadget-based).
>       It means that our widgets are not necessarily attached to windows.

This means that the coordiante space of the main gadget in TeXmacs is
translated compared to the toolkit that use real windows.

> 
>    d) Keyboard modifiers. TeXmacs will need a lot of keyboard modifiers
>       (Shift, Control, Alt, Meta/M$, AltGr, Hyper, User).
>       This is possible (and still; quite complicated) in X.
>       Is keyboard handling sufficiently flexible in the different GUI's?
> 
>    e) Fonts. We currently use almost only our own fonts,
>       which require clipped pixmap operations and invisible pixmaps.
>       Later on, we might want to use other fonts.

On my experiment, I don't even try to interface with the Qt widget API.

> 
>    f) Character encodings. We have a potentially universal encoding
>       scheme. 

I don't understant what you are talking about.

        It should be possible to make our scheme compatible
>       with Unicode, like in Qt. We have to be careful that something
>       similar can be done for other GUI's.


Unicode is most important. It is complex. The best tutorial document I
know about Unicode is
http://www.netthink.co.uk/downloads/unicode.ps.gz.  It is targetted to
a Perl crowd but there is no need to know Perl to read it.

BTW if I got it right and if my memory serves me well, the UTF8 encoding
can encode strings of integer whatever the size of the interger.

In fact, when I will start my Perl embedding, the string class will be
derived from the Perl one that supports very well TeXmacs since the
very recent 5.008. Before that the perl Unicode support was very incomplete.

> 
>    g) Keyboard and pointer grabbing. This has been particularly painful
>       for me when designing the GUI: a small mistake and
>       the application makes X hang.
> 
>    h) Graphics. All is done using the SI-based units.
>       Some changes are to be foreseen in order to make wysiwyg font handling
>       dpi-independent.

We want a toolkit that has a PostScript-like graphics model.
QPainter is nice.

> 
> 4. I am willing to make changes in the existing GUI if this is really 
> necessary
>    or if it leads to a fundamentally better design. However, as a general
>    strategy, we should first look whether a toolkit can be made compatible
>    with the existing conventions. Please throroughly argument proposed 
> changes!

I think your GUI is an incredible achievement and it made me learn
patterns, C++ and now guile (no less). but it just gets in the way today.
There are a lot of documented GUIs around while TeXmacs is really
a unique beast (albeit difficult to ride).

I think that the best things you can do to TeXmacs is to expose as
much of it as an API and make it a well documented library. 

> 
> OK, I'm getting a bit tired. More later...

So am I. But I think this discussion is worth it.

> 
> <Joris>
> 
> 
> _______________________________________________
> Texmacs-dev mailing list
> address@hidden
> http://mail.gnu.org/mailman/listinfo/texmacs-dev