Re: [PATCH v2] Add new entity \-- serving as markup separator/escape symbol

[Samuel Wales]

my deep apologies for the typo in john's name.  i meant of course John
Kitchin -- jkitchin.  i refer to his new style link syntax and his
proof of concept for cl style keyword args.  i still owe you email
replies.


On 7/29/22, Samuel Wales <samologist@gmail.com> wrote:
> i am not in a position to judge \-- but i like the idea of not having
> zws be used, and expect you have thought it out.
>
>
> just an idea: something approximately like this might work, or
> something like john kitchen's poc implementation of it might.  this is
> called extensible syntax.  one of the goals of es is to reduce the
> proliferation of org syntax and other stuff.
>
> es was proposed long ago, but i was unable to sufficiently follow up
> for unrelated reasons.  i have lots of replies and lots of further
> work on it but that's neither here nor there in this case.
>
> [other stuff includes but is not limited to increase reusability and
> reliability of code to implement things you want to do with syntax
> such as whether to show it, add a subfeature, export it variantly in
> different exporters, escape it, quote it, pretty-print it, etc.; allow
> user to do this so org is not burdened by it; etc.  terms to look up
> in the mailing list archives include extensible syntax, parsing risk,
> and id markers.]
>
>   $[emphasis :position beg :type bold :display "*"]bold text$[emphasis
> :position end :type bold :display "*"]
>
> alternatively:
>
>   $()...
>
> other than the basics, such as sexp, i do NOT care about the details
> of the $[] low level syntax in general OR the arglist details in this
> particular case.  those can change according to consensus or
> implementation needs etc.  instead, it is getting the concept across
> that matters to me.  one key thing about es is that when we want a new
> feature, we do not need new org syntax for that new feature.  OR for
> new subfeatures.  we just do something like this:
>
>   $[extended-timestamp :whatever yes :displays-as interval]
>
> or whatever.  this has nothing to do with bold emphasis.  it is an
> unrelated feature, using the same outer syntax.  another completely
> unrelated feature i'd strongly like, for emacs in general, is id
> markers.  that too can be done with this syntax.
>
> it looks verbose to 3rd party tools but is parseable by them.  this
> example displays as * to the user.  parseable as lisp sexp data using
> lisp tools.  it is meant to be vaguely reminiscent of a cl function
> call while still not likely to occur naturally.
>
> it would of course not be typed by the user directly but by some
> completion thing.
>
> i am not doing well so i am unlikely to be able to respond much or at
> all to queries.  please take it easy on me if this rubs you the wrong
> way.  it is just an idea and it does not have to be the answer.
>
> merely saying that once implemented, could solve this problem and ALSO
> later problems.  in fact, we discussed coloring of text using this
> syntax.  although with various understandings of it.  that's kinda
> similar to emphasis.
>
> On 7/29/22, Ihor Radchenko <yantar92@gmail.com> wrote:
>> Max Nikulin <manikulin@gmail.com> writes:
>>
>>>>> The good point in your patch is that \- is still work as shy hyphen
>>>>> (that, by the way, may be used in some cases instead of zero width
>>>>> space: *intra*\-word). On the other hand I have managed to find a case
>>>>> when your approach is not ideal:
>>>>>
>>>>> *\--scratch\--*
>>>>>
>>>>> <p>
>>>>> <b>&#x00ad;-scratch</b></p>
>>>>
>>>> Well. I think that it is impossible to use the same escape construct to
>>>> both force emphasis and escape it.
>>>
>>> Let's articulate the problem as follows: when some characters ("*". "/".
>>> etc.) besides used literally are overloaded with 2 additional roles that
>>> are start emphasis group and terminate emphasis group, in addition to
>>> lightweight markup heuristics, it is necessary to provide a way to
>>> disambiguate which of 3 roles is associated with particular character.
>>>
>>> "Activate" and "deactivate" characters or entities for emphasis markers
>>> are alternative and perhaps not so clear terms have used before.
>>>
>>> The advantage of zero width space is that "[:space:]" is part of
>>> PREMATCH and POSTMATCH (outer) regexps in
>>> `org-emphasis-regexp-components' and "[:space:]" is forbidden at the
>>> inner borders of emphasized span of text. The latter is mostly
>>> meaningful, however I am unsure if bold space has the same width as
>>> regular one, and space in fixed width font is certainly distinct.
>>>
>>> The problem with the "\--" entity is that it is not handled properly at
>>> the start of emphasis region. It neither disables emphasis nor parsed as
>>> complete entity, instead it becomes combination of "\-" shy hyphen and
>>> literal "-".
>>>
>>> Unsure if it can be solved consistently. Possible ways:
>>> - It addition to space-like (in respect to current regexp) entity add
>>> another one that acts as a part of word, but like "\--" stripped from
>>> output. Likely it should be accompanied by more changes in the parser
>>> and regexps.
>>> - Provide some new explicit syntax for literal character, start of
>>> emphasis group, end of emphasis group.
>>
>> The fact that \-- was not parsed in your example is because entities
>> cannot be directly followed by a letter (see 12.4 Special Symbols).
>>
>> You need
>>
>> *\--{}scratch\--*
>>
>> Concerning the 3 listed roles of the *_/+ markup, I propose to simplify
>> the problem a bit and not try to make \-- serve as a proper escape
>> symbol.
>> Instead, we can document the already existing quoting entities:
>>
>>  ("slash" "/" nil "/" "/" "/" "/")
>>  ("plus" "+" nil "+" "+" "+" "+")
>>  ("under" "\\_" nil "_" "_" "_" "_")
>>  ("equal" "=" nil "=" "=" "=" "=")
>>  ("star" "\\star" t "*" "*" "*" "⋆")
>>
>> Then, your example should better be written as
>>
>> \star{}scratch\star
>>
>> \-- may better work between markup, not inside.
>>
>>> Concerning zero width space workaround, I may be wrong, but Nicolas
>>> might consider using U+200B zero width space as the escape character for
>>> itself: single one is filtered out during export, double zero width
>>> space becomes single character. (I do not like this kind of "white
>>> space" programming language".)
>>
>> This is too complex, IMHO.
>> If desired, we can again go the entity road and introduce
>> \zws entity.
>>
>> Note that we already have
>>
>>  ("nbsp" "~" nil "&nbsp;" " " " " " ")
>>  ("ensp" "\\hspace*{.5em}" nil "&ensp;" " " " " " ")
>>  ("emsp" "\\hspace*{1em}" nil "&emsp;" " " " " " ")
>>  ("thinsp" "\\hspace*{.2em}" nil "&thinsp;" " " " " " ")
>>
>> Generally, it is a good idea to advertise entities in the manual.
>> Zero-width space is not only limited, it is impossible to use, e.g. in
>> tables when you want to quote "|". The only solution is using \vert or
>> \vbar entity.
>>
>>> Another question is whether U+2060 word
>>> joiner (or some other character) should be added either as alternative
>>> to zero width space or to allow =    verbatim    = fixed width text
>>> surrounded by fixed width spaces.
>>
>> This particular example is tricky.
>> If we put escape symbol _inside_ the verbatim, it is never possible to
>> know if the user intents to use that symbol literally or not.
>> But non-space before/after opening/closing markup char is hard-coded and
>> changing it is fragile.
>>
>> Instead of using some kind of "escape" symbol here, I suggest turning to
>> the idea about inline special blocks. We can introduce a more verbose
>> markup that will allow spaces inside at the beginning/end of the
>> contents.
>>
>> https://orgmode.org/list/87a6b8pbhg.fsf@posteo.net
>> Manuel Macías [ML:Org mode] (2022) About 'inline special blocks'
>>
>> Instead of using the tricky *bold text*, we may allow _*{bold text}*_ or
>> something similar, with _name{...}name_ being inline special block.
>>
>> Best,
>> Ihor
>>
>>
>
>
> --
> The Kafka Pandemic
>
> A blog about science, health, human rights, and misopathy:
> https://thekafkapandemic.blogspot.com
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com