Update to 10.39

ChangeLog

@@ -1,6 +1,69 @@
 Change Log for PCRE2
 --------------------
 
+Version 10.39 29-October-2021
+-----------------------------
+
+1. Fix incorrect detection of alternatives in first character search in JIT.
+
+2. Merged patch from @carenas (GitHub #28):
+
+  Visual Studio 2013 includes support for %zu and %td, so let newer
+  versions of it avoid the fallback, and while at it, make sure that
+  the first check is for DISABLE_PERCENT_ZT so it will be always
+  honoured if chosen.
+
+  prtdiff_t is signed, so use a signed type instead, and make sure
+  that an appropiate width is chosen if pointers are 64bit wide and
+  long is not (ex: Windows 64bit).
+
+  IMHO removing the cast (and therefore the positibilty of truncation)
+  make the code cleaner and the fallback is likely portable enough
+  with all 64-bit POSIX systems doing LP64 except for Windows.
+
+3. Merged patch from @carenas (GitHub #29) to update to Unicode 14.0.0.
+
+4. Merged patch from @carenas (GitHub #30):
+
+  * Cleanup: remove references to no longer used stdint.h
+
+  Since 19c50b9d (Unconditionally use inttypes.h instead of trying for stdint.h
+  (simplification) and remove the now unnecessary inclusion in
+  pcre2_internal.h., 2018-11-14), stdint.h is no longer used.
+
+  Remove checks for it in autotools and CMake and document better the expected
+  build failures for systems that might have stdint.h (C99) and not inttypes.h
+  (from POSIX), like old Windows.
+
+  * Cleanup: remove detection for inttypes.h which is a hard dependency
+
+  CMake checks for standard headers are not meant to be used for hard
+  dependencies, so will prevent a possible fallback to work.
+
+  Alternatively, the header could be checked to make the configuration fail
+  instead of breaking the build, but that was punted, as it was missing anyway
+  from autotools.
+
+5. Merged patch from @carenas (GitHub #32):
+
+  * jit: allow building with ancient MSVC versions
+
+  Visual Studio older than 2013 fails to build with JIT enabled, because it is
+  unable to parse non C89 compatible syntax, with mixed declarations and code.
+  While most recent compilers wouldn't even report this as a warning since it
+  is valid C99, it could be also made visible by adding to gcc/clang the
+  -Wdeclaration-after-statement flag at build time.
+
+  Move the code below the affected definitions.
+
+  * pcre2grep: avoid mixing declarations with code
+
+  Since d5a61ee8 (Patch to detect (and ignore) symlink loops in pcre2grep,
+  2021-08-28), code will fail to build in a strict C89 compiler.
+
+  Reformat slightly to make it C89 compatible again.
+
+
 Version 10.38 01-October-2021
 -----------------------------
 

NEWS

@@ -2,6 +2,18 @@ News about PCRE2 releases
 -------------------------
 
 
+Version 10.39 29-October-2021
+-----------------------------
+
+This release is happening soon after 10.38 because the bug fix is important.
+
+1. Fix incorrect detection of alternatives in first character search in JIT.
+
+2. Update to Unicode 14.0.0.
+
+3. Some code cleanups (see ChangeLog).
+
+
 Version 10.38 01-October-2021
 -----------------------------
 

NON-AUTOTOOLS-BUILD

@@ -343,10 +343,10 @@ cache can be deleted by selecting "File > Delete Cache".
 
 BUILDING PCRE2 ON WINDOWS WITH VISUAL STUDIO
 
-The code currently cannot be compiled without a stdint.h header, which is
-available only in relatively recent versions of Visual Studio. However, this
-portable and permissively-licensed implementation of the header worked without
-issue:
+The code currently cannot be compiled without an inttypes.h header, which is
+available only with Visual Studio 2013 or newer. However, this portable and
+permissively-licensed implementation of the stdint.h header could be used as an
+alternative:
 
   http://www.azillionmonkeys.com/qed/pstdint.h
 

README.md

@@ -2,7 +2,7 @@
 
 pcre2 Windows build with Visual Studio.
 
-This version is pcre2-10.38.
+This version is pcre2-10.39.
 
 To build, simply open the required solution file, and
 you know how to use Visual Studio, right?

README.pcre2

@@ -5,9 +5,8 @@ PCRE2 is a re-working of the original PCRE1 library to provide an entirely new
 API. Since its initial release in 2015, there has been further development of
 the code and it now differs from PCRE1 in more than just the API. There are new
 features, and the internals have been improved. The original PCRE1 library is
-now obsolete and should not be used in new projects. The latest release of
-PCRE2 is available in .tar.gz, tar.bz2, or .zip form from this GitHub
-repository:
+now obsolete and no longer maintained. The latest release of PCRE2 is available
+in .tar.gz, tar.bz2, or .zip form from this GitHub repository:
 
 https://github.com/PhilipHazel/pcre2/releases
 
@@ -377,12 +376,12 @@ library. They are also documented in the pcre2build man page.
 
 . The C99 standard defines formatting modifiers z and t for size_t and
   ptrdiff_t values, respectively. By default, PCRE2 uses these modifiers in
-  environments other than Microsoft Visual Studio when __STDC_VERSION__ is
-  defined and has a value greater than or equal to 199901L (indicating C99).
-  However, there is at least one environment that claims to be C99 but does not
-  support these modifiers. If --disable-percent-zt is specified, no use is made
-  of the z or t modifiers. Instead of %td or %zu, %lu is used, with a cast for
-  size_t values.
+  environments other than Microsoft Visual Studio versions earlier than 2013
+  when __STDC_VERSION__ is defined and has a value greater than or equal to
+  199901L (indicating C99). However, there is at least one environment that
+  claims to be C99 but does not support these modifiers. If
+  --disable-percent-zt is specified, no use is made of the z or t modifiers.
+  Instead of %td or %zu, %lu is used, with a cast for size_t values.
 
 . There is a special option called --enable-fuzz-support for use by people who
   want to run fuzzing tests on PCRE2. At present this applies only to the 8-bit
@@ -906,4 +905,4 @@ The distribution should contain the files listed below.
 Philip Hazel
 Email local part: Philip.Hazel
 Email domain: gmail.com
-Last updated: 27 August 2021
+Last updated: 29 October 2021

distfiles/download.url

@@ -1,2 +1,2 @@
-https://github.com/PhilipHazel/pcre2/releases/download/pcre2-10.38/pcre2-10.38.tar.bz2
-https://github.com/PhilipHazel/pcre2/releases/download/pcre2-10.38/pcre2-10.38.tar.bz2.sig
+https://github.com/PhilipHazel/pcre2/releases/download/pcre2-10.39/pcre2-10.39.tar.bz2
+https://github.com/PhilipHazel/pcre2/releases/download/pcre2-10.39/pcre2-10.39.tar.bz2.sig

include/pcre2.h

@@ -42,9 +42,9 @@ POSSIBILITY OF SUCH DAMAGE.
 /* The current PCRE version information. */
 
 #define PCRE2_MAJOR           10
-#define PCRE2_MINOR           38
+#define PCRE2_MINOR           39
 #define PCRE2_PRERELEASE      
-#define PCRE2_DATE            2021-10-01
+#define PCRE2_DATE            2021-10-29
 
 /* When an application links to a PCRE DLL in Windows, the symbols that are
 imported have to be identified as such. When building PCRE2, the appropriate
@@ -84,8 +84,8 @@ set, we ensure here that it has no effect. */
 /* Have to include limits.h, stdlib.h, and inttypes.h to ensure that size_t and
 uint8_t, UCHAR_MAX, etc are defined. Some systems that do have inttypes.h do
 not have stdint.h, which is why we use inttypes.h, which according to the C
-standard is a superset of stdint.h. If none of these headers are available,
-the relevant values must be provided by some other means. */
+standard is a superset of stdint.h. If inttypes.h is not available the build
+will break and the relevant values must be provided by some other means. */
 
 #include <limits.h>
 #include <stdlib.h>

src/pcre2_jit_compile.c

@@ -1251,10 +1251,13 @@ SLJIT_ASSERT(*cc == OP_ONCE || *cc == OP_BRA || *cc == OP_CBRA);
 SLJIT_ASSERT(*cc != OP_CBRA || common->optimized_cbracket[GET2(cc, 1 + LINK_SIZE)] != 0);
 SLJIT_ASSERT(start < EARLY_FAIL_ENHANCE_MAX);
 
+next_alt = cc + GET(cc, 1);
+if (*next_alt == OP_ALT)
+  fast_forward_allowed = FALSE;
+
 do
   {
   count = start;
-  next_alt = cc + GET(cc, 1);
   cc += 1 + LINK_SIZE + ((*cc == OP_CBRA) ? IMM2_SIZE : 0);
 
   while (TRUE)
@@ -1512,7 +1515,7 @@ do
         {
         count++;
 
-        if (fast_forward_allowed && *next_alt == OP_KET)
+        if (fast_forward_allowed)
           {
           common->fast_forward_bc_ptr = accelerated_start;
           common->private_data_ptrs[(accelerated_start + 1) - common->start] = ((*private_data_start) << 3) | type_skip;
@@ -1562,8 +1565,8 @@ do
   else if (result < count)
     result = count;
 
-  fast_forward_allowed = FALSE;
   cc = next_alt;
+  next_alt = cc + GET(cc, 1);
   }
 while (*cc == OP_ALT);
 
@@ -4198,20 +4201,15 @@ static void move_back(compiler_common *common, jump_list **backtracks, BOOL must
 TMP2 is not used. Otherwise TMP2 must contain the start of the subject buffer,
 and it is destroyed. Does not modify STR_PTR for invalid character sequences. */
 DEFINE_COMPILER;
-#if defined SUPPORT_UNICODE
-#if PCRE2_CODE_UNIT_WIDTH != 32
+
+#if defined SUPPORT_UNICODE && PCRE2_CODE_UNIT_WIDTH != 32
 struct sljit_jump *jump;
 #endif
-#if PCRE2_CODE_UNIT_WIDTH == 8
-struct sljit_label *label;
-#endif
-#endif
-
-SLJIT_UNUSED_ARG(backtracks);
-SLJIT_UNUSED_ARG(must_be_valid);
 
 #ifdef SUPPORT_UNICODE
 #if PCRE2_CODE_UNIT_WIDTH == 8
+struct sljit_label *label;
+
 if (common->utf)
   {
   if (!must_be_valid && common->invalid_utf)
@@ -4277,6 +4275,10 @@ if (common->invalid_utf && !must_be_valid)
   }
 #endif /* PCRE2_CODE_UNIT_WIDTH == [8|16|32] */
 #endif /* SUPPORT_UNICODE */
+
+SLJIT_UNUSED_ARG(backtracks);
+SLJIT_UNUSED_ARG(must_be_valid);
+
 OP2(SLJIT_SUB, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1));
 }
 
@@ -14130,9 +14132,9 @@ PCRE2_EXP_DEFN int PCRE2_CALL_CONVENTION
 pcre2_jit_compile(pcre2_code *code, uint32_t options)
 {
 pcre2_real_code *re = (pcre2_real_code *)code;
-#if defined(SUPPORT_JIT) && defined(_MSC_VER) && _MSC_VER < 1700
+#ifdef SUPPORT_JIT
 executable_functions *functions;
-static int executable_allocator_is_working;
+static int executable_allocator_is_working = 0;
 #endif
 
 if (code == NULL)
@@ -14168,13 +14170,7 @@ actions are needed:
 */
 
 #ifdef SUPPORT_JIT
-#if defined(_MSC_VER) && _MSC_VER < 1700
 functions = (executable_functions *)re->executable_jit;
-executable_allocator_is_working = 0;
-#else
-executable_functions *functions = (executable_functions *)re->executable_jit;
-static int executable_allocator_is_working = 0;
-#endif
 #endif
 
 if ((options & PCRE2_JIT_INVALID_UTF) != 0)

src/pcre2_jit_test.c

@@ -352,6 +352,7 @@ static struct regression_test_case regression_test_cases[] = {
 	{ MU, A, 0, 0, ".[ab]?.", "xx" },
 	{ MU, A, 0, 0, "_[ab]+_*a", "_aa" },
 	{ MU, A, 0, 0, "#(A+)#\\d+", "#A#A#0" },
+	{ MU, A, 0, 0, "(?P<size>\\d+)m|M", "4M" },
 
 	/* Bracket repeats with limit. */
 	{ MU, A, 0, 0, "(?:(ab){2}){5}M", "abababababababababababM" },