Subj : Re: Need volonteers to test another patch
To : Nicholas Boel
From : Vitaliy Aksyonov
Date : Sat Mar 02 2024 10:42 pm
Hello Nicholas.
02 Mar 24 14:44, you wrote to me:
VA>> Most probably it was some combination which made it looking
VA>> almost correct. I think that screen may be the reason you saw
VA>> pseudo-graphics
NB> I'm no longer using screen or tmux, and this version (along with all
NB> previous versions since you made the ncurses change) work the same and
NB> show UTF-8 properly, except for what you describe below:
VA>> more or less correctly. Remember those line wraps? That happens
VA>> because GoldEd converts those symbols to UTF-8 first. All
VA>> pseudo-graphics symbols represented as 3 bytes. So that line
VA>> become 3 times longer in bytes. Then GoldEd tries to split
VA>> message to lines and it uses bytes! not symbols. That's why it
VA>> splits the line in the middle of those pseudo-graphics. Even
VA>> worse, it may tear apart one UTF-8 symbol to two lines and it
VA>> will be displayed incorrectly.
NB> Yeah, I've noticed most of this.. and thank you for your explanation.
NB> At least now I know why it is happening.
VA>> GoldEd cannot work correctly with multibyte sequences. And even
VA>> if it looks "correct", it's just because most English letters has
VA>> same codes in cp437 and UTF-8.
NB> Maybe simple ones, like german umlauts and whatnot. But cp437 doesn't
NB> have any Cyrillic, Greek, Japanese, Chinese, etc.
VA>> If you want to keep using UTF-8, I may only suggest to find
VA>> version, which "works" for you and stick to it.
NB> I already have!
VA>> Until full UTF-8 support implemented in GoldEd (if that ever
VA>> happen), don't expect it to work correctly, sorry.
NB> That's ok. I had it somewhat working for awhile, the latest reverts
NB> have changed that. I don't have an issue going back to a "lucky"
NB> version. ;)
VA>> It's your choice. Just be aware, that if it works - it's just
VA>> pure luck and don't expect it to last. Until we implement UTF-8
VA>> support. It may take years. Or never happen. It's not so easy to
VA>> do it with backward compatibility wih all older systems like DOS
VA>> or OS/2.
NB> I don't mind being lucky sometimes.
I played little bit more with the code and different settings and found what was the difference between those two versions.
First - you use "wide" ncurses - ncursesw. I use ncurses.
Second. GoldEd incorrectly initializes ncurses in reverted version (like it was before I started to make any changes in GoldEd code). ncurses documentation explicitly says that before call initscr(), locale has to be set with setlocale. Otherwise it will use incorrect settings. Most probably plain C locale. I kinda get picture very similar to yours.
Did you have an issue with non-English chars when scrolling the message? What I see in UFT-8 terminal now - most of unicode text displayed correctly and pseudo-graphic too. Only pseudo-graphics lines broken similar to yours. And when I scroll text down - it corrupts.
If you may try to add one line of code to latest master and try it - that would be helpful.
goldlib/gcui/gkbdbase.cpp
line 149, right before initscr add this:
setlocale(LC_ALL, "");
If that gives you expected result - I'll push this change to master.
