GNOME Terminal 3.33.3 brings RTL/BiDi support

[Egmont Koblinger <egmont gmail com>]

Hi,

3.33.3 is a beautiful palindrome version number, reading it from left
to right or right to left doesn't make a difference.

Not so much with Arabic, Hebrew, Persian, Syriac, Urdu etc. scripts.
They need to go from right to left, and thus they often show up
backwards in terminals with their traditionally inherent left to right
characteristics.

I'm thrilled to announce that GNOME Terminal 3.33.3 (well, in fact VTE
0.57.3) introduces support for right to left (RTL) and bidirectional
(BiDi) text.


The work involved understanding the conflicting requirements,
examinimg the behavior of other terminals, studying an almost 30 year
old document on this topic; and based on these coming up with a
technical proposal [1] that resolves the conflicts and addresses all
the requirements, is reasonably simple and reasonably backwards
compatible, and builds on the existing Unicode Bidirectional Algorithm
and BiDi best practices. Needless to say, this design also had to be
implemented.

GNOME Terminal's new default is to shuffle the characters for the
display according to the BiDi algorithm. This fixes the behavior of
simple utilities where RTL text previously appeared in reverse order.

There are applications that require the terminal not to shuffle the
characters. Emacs's BiDi-aware text editing is one example, as it
shuffles the characters itself and sends them to the terminal in the
desired display order. This behavior of the terminal can be set with:
    printf "\e[8l"
and performing BiDi by the terminal can be re-enabled with:
    printf "\e[8h"

Other escape sequences are also available, e.g. to autodetect the text
direction, or switch to RTL paragraph direction. Consult
vte/perf/bidi.sh or the specification for details.


The current milestone is an extremely important one, but isn't the end
of the road.

There's one notable technical issue with the specification and the
current implementation: BiDi control characters at the beginning of a
logical string are lost. I'm planning to address this in the next
development cycle.

The current work adds BiDi to this platform, but what the user cares
about is the entire ecosystem, the overall user experience. To provide
the best experience, applications will need to use this platform
wisely, e.g. automatically turn off BiDi if that's what they need (the
user shouldn't need to issue those aforementioned commands manually).
It'll probably take years of ongoing improvements to get the most out
of the terminal's new features. Until then, such inconveniences as
having to manually set the desired mode remain, but at least now you
can set the desired mode and get nice BiDi experience, which is a
giant step forward. Enjoy it!


cheers,
egmont

[1] https://terminal-wg.pages.freedesktop.org/bidi/