Update Scintilla to 5.3.7 and Lexilla to 5.2.7 #3551

kugel- · 2023-09-06T05:53:57Z

Notable changes since 5.1.5

multithreaded layouting and line wrapping. Neil says this can lead to dramatic improvements. Disabled by default, see SCI_SETLAYOUTTHREADS (i.e. we don't use it yet)
New APIs to support 64-bit document positions on Win32: SCI_GETSTYLEDTEXTFULL, SCI_GETTEXTRANGEFULL, SCI_FINDTEXTFULL, and SCI_FORMATRANGEFULL We should move should move to these APIs, as the predecessors will be deprecated sooner or later. Not sure if it affects GTK-backend on win32 as well?
Change bar (probably similar to the git-changebar plugin)

xiota · 2023-09-13T04:53:29Z

0001-Fix-assertion-failure-on-some-system-when-an-INDIC_S.patch.gz

See:

elextr · 2023-09-16T02:52:11Z

@kugel- works for me, thanks for doing yet another Scintilla update, and with patch that @xiota cherry picked from Scintilla to avoid UB.

eht16 · 2023-09-17T09:28:55Z

data/filedefs/filetypes.python.in

@@ -21,6 +21,7 @@ fstring=string_1
 fcharacter=character
 ftriple=string_2
 ftripledouble=string_2
+attribute=identifier_1ö


elextr · 2023-09-17T09:45:49Z

As @kugel- pointed out elsewhere, this replaces #3441. So is the GDscript change on #3441 needed here as well?

Edit: just read #3441 where @kugel- answers the above question, just consider this a reminder then :-)

elextr · 2023-09-22T22:23:57Z

@kugel- don't forget this, fixes crasher in geany/geany-plugins#1272

rdipardo · 2023-09-23T23:57:07Z

Handle new styles for strings in R (rawstrings, backticks, espaces

I would defer to @zufuliu's patch as being simpler and slightly optimizing the iteration count. IIUC, it's Geany's policy to only consume upstream patches anyway.

elextr · 2023-09-24T00:28:53Z

@rdipardo I don't understand, the commit by @kugel- is the mapping of lexer entities to styles within Geany, the patch by @zufuliu is to Scintilla, they are totally different and not interchangable?

rdipardo · 2023-09-24T05:49:17Z

@elextr, yes, I see now the commit only touched filedefs (after expanding the diffs that GitHub's device fingerprinting scripts decided to collapse for my diminutive screen).

At least the Prolog enthusiasts now have a mentioned thread to follow, in case they want to know what's blocking #3171.

rdipardo · 2023-09-24T05:58:49Z

Speaking of filedefs, has anyone had a chance to look at the suite of new properties the shell lexer acquired in recent versions? Most of them are for embedded variables, for example inside here docs:

The command substitution preference also became a set of enumerated options:


lexer.bash.command.substitution	Set how to highlight $() command substitution. 0 (the default) highlighted as backticks. 1 highlighted inside. 2 highlighted inside with extra scope tracking.
lexer.bash.nested.backticks	Set this property to 0 to disable nested backquoted command substitution.
lexer.bash.styling.inside.backticks	Set this property to 1 to highlight shell expansions inside backticks.
lexer.bash.styling.inside.heredoc	Set this property to 1 to highlight shell expansions inside here document.
lexer.bash.styling.inside.parameter	Set this property to 1 to highlight shell expansions inside ${} parameter expansion.
lexer.bash.styling.inside.string	Set this property to 1 to highlight shell expansions inside string.

elextr · 2023-09-24T06:30:00Z

You should be able to set options in the [lexer_proterties] section of the filetype file, so you can try all those out and see what they do.

kugel- · 2023-09-24T21:51:24Z

@kugel- don't forget this, fixes crasher in geany/geany-plugins#1272

Scintilla 5.3.7 was released including the fix. I'll bump to that version.

kugel- · 2023-09-26T06:06:33Z

@eht16 do you know what's wrong with the Windows CI?

kugel- · 2023-09-26T06:49:36Z

ScintillaOrg/lexilla#206 fixed after the 5.2.7 release. I did not backport the fix. I think R is not relevant enough to carry an additional change to Lexilla, or is it?

elextr · 2023-09-26T07:14:16Z

I think R is not relevant enough to carry an additional change to Lexilla, or is it?

I'm not sure "relevant" is well defined, thats a personal judgement depending if you use R :-). I would think only patches that fix crashes need to be carried outside Scintilla releases, thats the only reason I mentioned the one above.

eht16 · 2023-09-26T20:56:33Z

@eht16 do you know what's wrong with the Windows CI?

No :(.
I can succesfully build the image with normal Docker on a native Linux system. Either something changed on how the Gitlab runners are implemented or something in pacman/MSYS2 changed, still unsure what is causing this.
So far I didn't find any relevant information on the net. I'll try if it is enough to just create /etc/mtab.

kugel- · 2023-09-27T06:24:28Z

I added a commit to enable the change history since it was trivial. Seems like a nice, user-visible change to take along (gives a bit of a modern feeling).

Before saving

After saving

elextr · 2023-09-27T07:37:45Z

@eht16 what version of g++/libstdc++ is used for windows, nowhere does the version get printed in configure or build, grrrr? The CI build is now running but failing when it can't find std::mutex (and yes Neil did #include <mutex>

geany/scintilla/src/Editor.cxx

Line 29 in e4feb5c

#include <mutex>

). That is C++11 AFAIK so not sure why it fails unless you are using an ancient version of g++. I suspect Scintilla (not just Lexilla) now needs a real C++17 toolset.

kugel- · 2023-09-27T07:39:40Z

AFAIK scintilla needs C++17 since version 4?

elextr · 2023-09-27T07:47:16Z

Yes, but there were "C++17 compliant(ish)" and "really C++17 compliant" versions of stdlibs. Scintilla didn't start using threading and futures and other concurrency features until these latest few versions, and older libraries might not support them. At least thats the only reason I can think of that it gives the current errors. But as I said Mutex is C++11 so it really should work by now.

eht16 · 2023-09-28T05:18:17Z

@eht16 what version of g++/libstdc++ is used for windows, nowhere does the version get printed in configure or build, grrrr?

In https://github.com/geany/geany/actions/runs/6322193707/job/17167381804?pr=3551#step:9:23 we show basic build environment information. Adding the gcc/g++ version is easy, maybe also the libstdc++ version as well. Anything else you are missing?
From the log of the image build, gcc 10.2 and libstdc++6 10.2 is used.

Is gcc 10 fine in general?
We could upgrade the Mingw64 build image to Debian Bookworm (currently Bullseye) to get newer gcc versions but on my last attempt this caused weired errors and so I decided to postpone it to after the release.

The CI build is now running

It's running only because I added a workaround in geany/infrastructure#10 to the image build which worked and I accidentally pushed the resulting image to the registry, so it can now be used for other builds. This is OK for the moment but I would still like to know the background of the /etc/mtab problem.

elextr · 2023-09-28T07:07:28Z

Is gcc 10 fine in general?

On an old machine (note to self: upgrade it) linux g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 works here (this PR builds and runs fine).

I really don't know what is wrong with the mingw build at this point.

Edit:

Adding the gcc/g++ version is easy, maybe also the libstdc++ version as well.

👍 Not aware of anything else.

eht16 · 2023-10-01T11:07:31Z

Is gcc 10 fine in general?

On an old machine (note to self: upgrade it) linux g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 works here (this PR builds and runs fine).

I really don't know what is wrong with the mingw build at this point.

Me neither.
geany/infrastructure#10 fixes the first problem with the Docker mingw64 image build. Since this is done now, I will test this PR on a real Windows system to see if it compiles there (I guess yes).

Adding the gcc/g++ version is easy, maybe also the libstdc++ version as well.

👍 Not aware of anything else.

Will do later.

eht16 · 2023-10-03T13:14:29Z

The build failures are not because the used compiler is too old but it's missing features: https://sourceforge.net/p/mingw-w64/bugs/959/

#3568 changes the used cross compiler toolchain to its "posix" variant which has all necessary C++ features and so the build will succeed.

I added there some more information about the used compiler to the build log.

eht16 · 2023-10-03T13:26:41Z

@kugel- @elextr should we include the patch from https://groups.google.com/g/scintilla-interest/c/_tiE_nSaiG4/m/v0e8lzTjAQAJ here?

I'd say yes because the next Scintilla release might be after the upcoming Geany release and the issue might also affect Geany users.

kugel- · 2023-10-03T18:17:29Z

Apart from that (we can merge that patch individually if we agree on it), is there anything blocking?

The PR adds new string thanks to the change history feature so need to merge before the string freeze

eht16 · 2023-10-03T21:01:36Z

I'm fine with merge. The failed Windows CI build is handled seperately.

Only one minor thing, I guess we should bump plugin API version because GeanyIndentPrefs and Scintilla itself are part of the plugin API.

elextr · 2023-10-03T23:51:16Z

@eht16 did it build ok on "real" windows, preferably whatever is going to be used to make the release so we know that will be ok?

Otherwise I'm fine with it, it works for me on Linux.

eht16 · 2023-10-04T06:23:52Z

Yes, it builds fine on native Windows as there the compiler from MSYS2 is used. Sorry forgot to mention this, all the build problems are specific to the used cross compiler toolchain used in the CI builds.

kugel- · 2023-10-04T07:14:18Z

Fine. I'll merge and bump the plugin API separately.

Regarding https://groups.google.com/g/scintilla-interest/c/_tiE_nSaiG4/m/v0e8lzTjAQAJ, I think it's a bit too early to jump on that patch. The problem seems to an extreme edge case and the proposed solution is not yet released or even in Scintilla mainline.

elextr · 2023-10-04T07:19:45Z

The problem seems to an extreme edge case and the proposed solution is not yet released or even in Scintilla mainline.

Yes agreed, and if it is a problem the changebars can be turned off in prefs.

Notable changes since 5.1.5 - multithreaded layouting and line wrapping. Neil says this can lead to dramatic improvements. Disabled by default, see SCI_SETLAYOUTTHREADS (i.e. we don't use it yet) - New APIs to support 64-bit document positions on Win32: SCI_GETSTYLEDTEXTFULL, SCI_GETTEXTRANGEFULL, SCI_FINDTEXTFULL, and SCI_FORMATRANGEFULL We should move should move to these APIs, as the predecessors will be deprecated sooner or later. Not sure if it affects GTK-backend on win32 as well? - Change bar (will be enabled in a separate commit) Also update Lexilla to 5.2.7, changes since 5.1.4: - Tons of LexBash improvements - Also numerous improvements to languages (Ruby, Matlab, F# and many more)

"attribute" is for highlighting object attributes or methods, i.e. to style foo and bar differently in foo.bar. Also set lexer properties to allow scintilla using different styles. See ScintillaOrg/lexilla#49

%i, %I, %w and %s are newly recognized by Scintilla. See also: https://docs.ruby-lang.org/en/master/syntax/literals_rdoc.html#label-Percent+Literals

Note:With escapes there is still an issue, thus we disable it for now via lexer properties. See also: ScintillaOrg/lexilla#206

While at it, fix annotation style. Somehow an earlier lexilla import falsely used decorator in filetypes.gdscript.

We previously imported the README but did not keep it up-to-date. This was adding noise to the diff to pristine scintilla.

Makes Geany feel a litttle bit more modern. Can be enabled/disabled via Prefs. Default is to show the change history in the markers margin. Note: Scintilla tracks only its own change history and is not connected to any VCS, unlike the git-changebar plugin. Also, change history is lost when the document is closed and re-opened.

EditorPrefs was extended and scintilla was updated. Be helpful and allow plugins to detect this.

b4n · 2023-10-07T20:19:26Z

I added a commit to enable the change history since it was trivial. Seems like a nice, user-visible change to take along (gives a bit of a modern feeling).

I'm a tad late to the party (as usual), but I don't think it should be enabled by default. Here's a few reasons:

I've seen it do odd stuff (when I ^Zd a change on a line, it stayed green, yet didn't match the saved state), so I'm wondering if it's already stable enough… but maybe it's just because it matched one of the saved states of the file because I did save that previous state before!?
if you reload the file, everything becomes green…
if someone doesn't have the marker margin visible, background rendering is quite intrusive (especially after reloading the file 😉 )
I personally don't understand the point of the feature. Why would I want to see changes since the file was loaded? It's might be nice to see which lines aren't saved, but having everything since the file was loaded I don't understand the use case.
I think it's not entirely obvious what the feature is about if you don't know it. Meaning that if I just opened Geany, I could wonder what the heck was that thing.
Colors are a bit off with the default theme (or any for that matter), maybe we'd need some way to adjust that?

b4n · 2023-10-07T20:21:09Z

The problem seems to an extreme edge case

Not saying it's common, but AFAIR bulk search & replace will do that (potentially massively change lines from the end to the start)

elextr · 2023-10-08T01:55:26Z

I guess Neil added this because somebody wanted it. I can see that there are times when knowing what was changed in a session is useful. Since last save is maybe less useful since most IDEs have autosave and lots of people seem to like it.

But as @b4n points out there are operations that can produce unusable results when blindly recording lines edited since session start.

Other IDEs (the ones I tested) show diffs to git, not just the session, thats probably more useful. Maybe, depends how good the diff algorithm is, I'm sure we all have seen excessive github diffs for small changes.

Styling just needs "somebody" to add settings for the relevant marker and indicator types to the colour schemes files.

Anyway, its in Scintilla, we might as well make it available to users, the only question is by default or not. I don't have a strong feeling either way.

eht16 · 2023-10-08T09:41:20Z

After having used the change history for a few days now, I also think it might be better disabled by default so users should enable it explicitly if desired.

@kugel-
More importantly, we should document it. At the very least there should be a few lines describing the feature at https://github.com/geany/geany/blob/master/doc/geany.txt?plain=1#L2295.

I will take care of updated screenshots seperately.

eht16 · 2023-10-11T10:39:25Z

"Change history" docs will be added in #3593.

kugel- added this to the 1.39/2.0 milestone Sep 6, 2023

kugel- mentioned this pull request Sep 11, 2023

Fix keyword colorization on filetype change #3553

Merged

elextr mentioned this pull request Sep 13, 2023

Spell Check plugin crashes with color-variant-selector emojis geany/geany-plugins#1272

Open

elextr mentioned this pull request Sep 16, 2023

Update lexilla to 5.2.4 #3441

Closed

eht16 reviewed Sep 17, 2023

View reviewed changes

kugel- force-pushed the scintilla branch from 55aabfa to 1caf9e0 Compare September 22, 2023 19:16

kugel- changed the title ~~Update Scintilla to version 5.3.6~~ Update Scintilla to 5.3.7 and Lexilla to 5.2.7 Sep 26, 2023

eht16 mentioned this pull request Oct 3, 2023

CI: Use mingw posix crosscompiler #3568

Merged

elextr approved these changes Oct 4, 2023

View reviewed changes

kugel- mentioned this pull request Oct 4, 2023

Add more date formats; use ISO 8601 by default (yyyy-mm-dd) #3570

Open

kugel- added 5 commits October 4, 2023 21:47

Map new python "attribute" mapping to identifier_2, by default.

9aa597e

"attribute" is for highlighting object attributes or methods, i.e. to style foo and bar differently in foo.bar. Also set lexer properties to allow scintilla using different styles. See ScintillaOrg/lexilla#49

Map new ruby lex classes to existing styles, for now.

7f78ddf

%i, %I, %w and %s are newly recognized by Scintilla. See also: https://docs.ruby-lang.org/en/master/syntax/literals_rdoc.html#label-Percent+Literals

Handle new styles for strings in R (rawstrings, backticks, espaces)

0f04b24

Note:With escapes there is still an issue, thus we disable it for now via lexer properties. See also: ScintillaOrg/lexilla#206

Handle new nodepath style for gdscript

59d8aee

While at it, fix annotation style. Somehow an earlier lexilla import falsely used decorator in filetypes.gdscript.

kugel- force-pushed the scintilla branch from e4feb5c to 75d528d Compare October 4, 2023 19:57

kugel- added 3 commits October 4, 2023 22:42

Update scintilla/scintilla_changes.patch and update-scintilla.sh

c2a96e8

We previously imported the README but did not keep it up-to-date. This was adding noise to the diff to pristine scintilla.

Bump plugin API

35b244b

EditorPrefs was extended and scintilla was updated. Be helpful and allow plugins to detect this.

kugel- force-pushed the scintilla branch from 75d528d to 35b244b Compare October 4, 2023 20:42

kugel- merged commit 4af1965 into geany:master Oct 4, 2023
3 of 4 checks passed

b4n mentioned this pull request Oct 10, 2023

Do not enable Scintilla's Change History by default #3591

Merged

Update Scintilla to 5.3.7 and Lexilla to 5.2.7 #3551

Update Scintilla to 5.3.7 and Lexilla to 5.2.7 #3551

Conversation

kugel- commented Sep 6, 2023

xiota commented Sep 13, 2023

elextr commented Sep 16, 2023

eht16 Sep 17, 2023

Choose a reason for hiding this comment

elextr commented Sep 17, 2023 • edited Loading

elextr commented Sep 22, 2023

rdipardo commented Sep 23, 2023

elextr commented Sep 24, 2023

rdipardo commented Sep 24, 2023

rdipardo commented Sep 24, 2023

elextr commented Sep 24, 2023

kugel- commented Sep 24, 2023

kugel- commented Sep 26, 2023

kugel- commented Sep 26, 2023

elextr commented Sep 26, 2023

eht16 commented Sep 26, 2023 • edited Loading

kugel- commented Sep 27, 2023

elextr commented Sep 27, 2023 • edited Loading

kugel- commented Sep 27, 2023

elextr commented Sep 27, 2023

eht16 commented Sep 28, 2023

elextr commented Sep 28, 2023 • edited Loading

eht16 commented Oct 1, 2023

eht16 commented Oct 3, 2023

eht16 commented Oct 3, 2023

kugel- commented Oct 3, 2023

eht16 commented Oct 3, 2023

elextr commented Oct 3, 2023

eht16 commented Oct 4, 2023 via email

kugel- commented Oct 4, 2023

elextr commented Oct 4, 2023

b4n commented Oct 7, 2023

b4n commented Oct 7, 2023

elextr commented Oct 8, 2023

eht16 commented Oct 8, 2023

eht16 commented Oct 11, 2023

elextr commented Sep 17, 2023 •

edited

Loading

eht16 commented Sep 26, 2023 •

edited

Loading

elextr commented Sep 27, 2023 •

edited

Loading

elextr commented Sep 28, 2023 •

edited

Loading