Re: [tip] org-publish to work with (very) large books

[Juan Manuel Macías]

Ihor Radchenko writes:

> Yet, the information is surprisingly scattered. I was unable to find a
> single guide on the available possibilities. Mostly unanswered or
> partially answered questions from users.

Yes you're right. In addition, what I have been testing is not a panacea
either. In general, when it comes to long and complex documents, there
is no other solution than to arm yourself with patience, launch
asynchronous processes and dedicate yourself to doing another task while
LaTeX does its job. And, of course, trust that there are no errors. The
only advantage to debugging the code of a LaTeX document is how much you
learn about LaTeX and TeX in the process. But many times it is something
that can become frustrating, and the log file can be more cryptic than a
Sumerian inscription. The cause/effect relationship in LaTeX errors can
be the most surrealistic things in the world :-D.

Luckily, there's texstackexchange, where the LaTeX core and LaTeX
package developers themselves write, which is an endless source of
help...

> Do you have anything from LuaTeX in mind that could improve the current
> ox-latex pdf export when LuaTeX is used as the TeX engine?

I've thought about it sometimes, but haven't been able to find anything
concrete for Org. LuaLaTeX cares that a well-formed LaTeX document is
delivered to it, and Org already does that very well. In LuaTeX you can
insert lua code between La(TeX) code. For example:

\begin{luacode}
local x = "foo"
local y = "bar"
tex.print (x .. " and " .. y)
\end{luacode}

But in Org we have all the power of Babel: Org wins.

In LuaTeX you can write functions as pre-process filters, and associate
these functions with different callbacks. For example, there is a
callback_input_buffer, but we already have something like that in Org,
and with a larger scope and not limited to output to LaTeX.

In general, the advanced features of LuaTeX are more typographical and
micro-typographical in nature, and I guess they are of little use to
Org. For example, I recently wrote this function that highlights in red
the text that is in a language other than the main language of the
document (in my case, Spanish, langid 80). Act low-level on the line
node, just before LuaTeX does the line break to create the paragraph:

\directlua{
w = node.new("whatsit","pdf_literal")
w.data = "1 0 0 rg"
z = node.new("whatsit","pdf_literal")
z.data = "0 g"
function other_langs(h,c)
   for n in node.traverse_id(0,h) do
      for x in node.traverse_id(node.id("glyph"),n.head) do
         if x.lang < 80 or x.lang > 80 then
            local before, after = node.copy(w), node.copy(z)
            n.head = node.insert_before(n.head,x,before)
            n = node.insert_after(n,x,after)
         end
      end
   end
   return h
   end

luatexbase.add_to_callback('post_linebreak_filter', other_langs, 'other_langs')
}

According to the LuaTeX manual, "TeX’s nodes are represented in Lua as
userdata objects with a variable set of fields". What this function does
is simply manipulate the .lang field of the glyph nodes in an hlist node
(the line with its components).

Functions associated with the post_linebreak_filter callback are very
useful and productive, but from a purely typographical point of view.

At the pure LaTeX level, LuaLaTeX is not very different from LaTeX. Any
LaTeX document, generally speaking, can be compiled with LuaLaTeX, as
long as it is in utf8 and does not contain some pdfLaTeX- or
XelaTeX-specific commands. Today the compatibility between engines is
reasonably good, and more and more packages designed exclusively for
LuaTeX are uploaded to CTAN. The TeX ecosystem is notorious for its
slowness and conformism, but LuaTeX is meant to be the natural
replacement for pdfTeX. Sometime in the uncertain future :-)

Best regards,

Juan Manuel