What's new in .NET 7 Preview 4 [WIP]

[leecow]

What's new in .NET 7 Preview 4

This issue is for teams to highlight work for the community that will release .NET 7 Preview 4

To add content, use a new conversation entry. The entry should include the team name and feature title as the first line as shown in the template below.

## Team Name: Feature title

[link to the tracking issue or epic item for the work]

Tell the story of the feature and anything the community should pay particular attention 
to be successful using the feature.

Preview 1: #7106
Preview 2: #7107
Preview 3: #7108
Preview 4: #7378
Preview 5: #7441
Preview 6: #7454
Preview 7: #7455

[tarekgh]

Observability

Introducing Activity.Current change event dotnet/runtime#67276

    public partial class Activity : IDisposable
    {
        public static event EventHandler<ActivityChangedEventArgs>? CurrentChanged;
    }

Usage Example

    Activity.CurrentChanged += CurrentChanged;

    void CurrentChanged(object? sender, ActivityChangedEventArgs e)
    {
        Console.WriteLine($"Activity.Current value changed from Activity: {e.Previous.OperationName} to Activity: {e.Current.OperationName}");
    }

Expose performant Activity properties enumerator methods dotnet/runtime#67207

The exposed methods can be used in the perf critical scenarios to enumerate the Activity Tags, Links, and Events properties without any extra allocations and with fast items access.

namespace System.Diagnostics
{
    partial class Activity
    {
        public Enumerator<KeyValuePair<string,object>> EnumerateTagObjects();
        public Enumerator<ActivityLink> EnumerateLinks();
        public Enumerator<ActivityEvent> EnumerateEvents();
    
        public struct Enumerator<T>
        {
            public readonly Enumerator<T> GetEnumerator();
            public readonly ref T Current;
            public bool MoveNext();
        }
    }
}

Usage Example

    Activity a = new Activity("Root");

    a.SetTag("key1", "value1");
    a.SetTag("key2", "value2");

    foreach (ref readonly KeyValuePair<string, object?> tag in a.EnumerateTagObjects())
    {
        Console.WriteLine($"{tag.Key}, {tag.Value}");
    }

[tarekgh]

Adding Microseconds and Nanoseconds to TimeStamp, DateTime, DateTimeOffset, and TimeOnly dotnet/runtime#23799

namespace System {
    public struct DateTime {
        public DateTime(int year, int month, int day, int hour, int minute, int second, int millisecond, int microsecond);
        public DateTime(int year, int month, int day, int hour, int minute, int second, int millisecond, int microsecond, System.DateTimeKind kind);
        public DateTime(int year, int month, int day, int hour, int minute, int second, int millisecond, int microsecond, System.Globalization.Calendar calendar);
        public int Microsecond { get; }
        public int Nanosecond { get; }
        public DateTime AddMicroseconds(double value);
    }
    public struct DateTimeOffset {
        public DateTimeOffset(int year, int month, int day, int hour, int minute, int second, int millisecond, int microsecond, System.TimeSpan offset);
        public DateTimeOffset(int year, int month, int day, int hour, int minute, int second, int millisecond, int microsecond, System.TimeSpan offset, System.Globalization.Calendar calendar);
        public int Microsecond { get; }
        public int Nanosecond { get; }
        public DateTimeOffset AddMicroseconds(double microseconds);
    }
    public struct TimeSpan {
        public const long TicksPerMicrosecond = 10L;
        public const long NanosecondsPerTick = 100L;
        public TimeSpan(int days, int hours, int minutes, int seconds, int milliseconds, int microseconds);
        public int Microseconds { get; }
        public int Nanoseconds { get; }
        public double TotalMicroseconds { get; }
        public double TotalNanoseconds { get; }
        public static TimeSpan FromMicroseconds(double microseconds);
    }
    public struct TimeOnly {
        public TimeOnly(int day, int hour, int minute, int second, int millisecond, int microsecond);
        public int Microsecond { get; }
        public int Nanosecond { get; }
    }
}

Thanks to @ChristopherHaws and @deeprobin helping in the design and implementation.

[joperezr]

More improvements and new APIs for System.Text.RegularExpressions

Augment Regex extensibility point for better perf and span-based matching
[API Proposal]: Add Regex.Enumerate(ReadOnlySpan) which is allocation free
Overhaul Regex's handling of RegexOptions.IgnoreCase

For preview 4 we are adding the remaining planned APIs in order to add span support into our Regex library. These APIs are amortized allocation-free. The main span-based APIs being added for Preview 4 are:

• Regex.IsMatch(ReadOnlySpan<char> input): Indicates whether the regular expression finds a match in the input span.
• Regex.Count(ReadOnlySpan<char> input): Searches an input string for all occurrences of a regular expression and returns the number of matches.
• Regex.EnumerateMatches(ReadOnlySpan<char> input): Searches an input span for all occurrences of a regular expression and returns a ValueMatchEnumerator to lazily iterate over the matches.

We have also done a lot of work that enhances Regex performance in general, with improvements like:

• Improving the handling of more common Regex sets
• Improving the performance in the logic that finds possible positions where a match could exist.
• Use spans in some of our internal types to avoid allocations where possible which makes the engine go faster.
• Improve the logic for when a loop can be made atomic.

Finally we have also made some improvements on the code that is generated by the Regex source generator in order to make the code more readable and easier to debug, as well as enable projects with multiple source generated regex patterns to share common code between them.

[maryamariyan]

Added metrics for Microsoft.Extensions.Caching

• Let consumers of MemoryCache access metrics runtime#50406

For preview 4 we added metrics support for IMemoryCache. The main APIs being added for Preview 4 are:

• MemoryCacheStatistics which holds cache hit/miss/estimated size and count for IMemoryCache
• GetCurrentStatistics: returns an instance of MemoryCacheStatistics, or null when TrackStatistics flag is not enabled. The library has a built-in implementation available for MemoryCache.

GetCurrentStatistics() API allows app developers to use either event counters or metrics APIs to track statistics for one or more memory cache. Note, using these APIs for getting statistics for multiple caches is possible but requires developer to write their own meter.

Issue dotnet/runtime#67770 tracks adding built-in meters into the library to help improve developer experience with monitoring caches.

How to use the APIs

GetCurrentStatistics() API (based on dotnet/runtime#50406) allows app developers to use either event counters or metrics APIs to track statistics for one or more memory caches with code snippets.

With IMemoryCache.GetCurrentStatistics(), the user now has support for the following use cases:

• One cache with either event counters or metrics APIs
• Multiple caches with metrics API

Using IMemoryCache.GetCurrentStatistics() for one memory cache

Use AddMemoryCache API to instantiate a single memory cache and via DI get it injected to enable them calling GetCurrentStatistics.

Sample usage/screenshot for event counter:

// when using `services.AddMemoryCache(options => options.TrackStatistics = true);` to instantiate

    [EventSource(Name = "Microsoft-Extensions-Caching-Memory")]
    internal sealed class CachingEventSource : EventSource
    {
        public CachingEventSource(IMemoryCache memoryCache) { _memoryCache = memoryCache; }
        protected override void OnEventCommand(EventCommandEventArgs command)
        {
            if (command.Command == EventCommand.Enable)
            {
                if (_cacheHitsCounter == null)
                {
                    _cacheHitsCounter = new PollingCounter("cache-hits", this, () =>
                        _memoryCache.GetCurrentStatistics().CacheHits)
                    {
                        DisplayName = "Cache hits",
                    };
                }
            }
        }
    }

Helps them view stats below with dotnet-counters tool:

Using IMemoryCache.GetCurrentStatistics() for multiple memory caches

In order to get stats for more than one memory cache in the app, the user may use metrics APIs in their own code, so long as they have a way of distinguishing their caches by name or ID:

sample usage/screenshot for multiple caches using metrics APIs

 Meter s_meter = new Meter("Microsoft.Extensions.Caching.Memory.MemoryCache", "1.0.0");
 var cacheHitsMetrics = s_meter.CreateObservableGauge<int>("cache-hits", GetCacheHits);

// metrics callback for cache hits
IEnumerable<Measurement<long>> GetCacheHits()
{
    return new Measurement<long>[]
    {
            // or measurements could be looped or read from a real queue somewhere:
            new Measurement<long>(mc1.GetCurrentStatistics().TotalHits, new KeyValuePair<string,object>("CacheName", "mc1")),
            new Measurement<long>(mc2.GetCurrentStatistics().TotalHits, new KeyValuePair<string,object>("CacheName", "mc2")),
            new Measurement<long>(mc3.GetCurrentStatistics().TotalHits, new KeyValuePair<string,object>("CacheName", "mc3")),
    };
}

Sample stats with dotnet-counters tool:

Each metrics would need to create its own observable gauge (one for hits, then misses, etc.) and each callback function for the gauge iterates through list of caches creating measurements.

This gist provides a small sample.

[eerhardt]

.NET Libraries: Nullable annotations for Microsoft.Extensions

dotnet/runtime#43605

We have finished annotating the Microsoft.Extensions.* libraries for nullability. In .NET 7 Preview 4, all Microsoft.Extensions.* libraries have been fully annotated for nullability.

This work wouldn't have been possible without @maxkoshevoi's multiple-month effort. Starting with the first PR in August 2021, all the way to the final PR in April 2022, this was a lot of work that is greatly appreciated by the .NET community.

[carlossanlop]

Adding new Tar APIs

Implement Tar APIs - dotnet/runtime#67883

We are adding the new System.Formats.Tar assembly, which contains cross-platform APIs that allow reading, writing, archiving and extracting Tar archives.

Usage examples

For the most common usage, extracting and archiving, we are offering these methods:

// Generates a tar archive where all the entry names are prefixed by the root directory 'SourceDirectory'
TarFile.CreateFromDirectory(sourceDirectoryName: "/home/dotnet/SourceDirectory/", destinationFileName: "/home/dotnet/destination.tar", includeBaseDirectory: true);

// Extracts the contents of a tar archive into the specified directory, but avoids overwriting anything found inside
TarFile.ExtractToDirectory(sourceFileName: "/home/dotnet/destination.tar", destinationDirectoryName: "/home/dotnet/DestinationDirectory/", overwriteFiles: false);

We also offer variants that allow extracting from a stream or archiving into a stream:

// Generates a tar archive where all the entry names are prefixed by the root directory 'SourceDirectory'
using MemoryStream archiveStream = new();
TarFile.CreateFromDirectory(sourceDirectoryName: @"D:\SourceDirectory\", destination: archiveStream, includeBaseDirectory: true);

// Extracts the contents of a stream tar archive into the specified directory, and avoids overwriting anything found inside
TarFile.ExtractToDirectory(source: archiveStream, destinationDirectoryName: @"D:\DestinationDirectory\", overwriteFiles: false);

Additionally, the entries of an archive can be traversed one by one using the reader:

// Opens an archive for reading individual entries, and closes the archive stream after the reader disposal
using FileStream archiveStream = File.OpenRead("/home/dotnet/SourceDirectory/source.tar");
using TarReader reader = new(archiveStream, leaveOpen: false);
TarEntry? entry;
while ((entry = reader.GetNextEntry()) != null)
{
    // Extracts an entry to the desired destination, and overwrites if an entry with the same name is found
    string destinationFileName = Path.Join("/home/dotnet/DestinationDirectory", entry.Name);
    entry.ExtractToFile(destinationFileName, overwrite: true);
}

And entries can also be written one by one into an archive stream using the writer:

using FileStream archiveStream = File.OpenWrite(@"D:\DestinationDirectory\destination.tar");
// Create the archive in the PAX format and write individual entries, closing the stream after the writer disposal
using TarWriter writer = new(archiveStream, TarFormat.Pax, leaveOpen: false);

// Add an entry from an existing file
writer.WriteEntry(fileName: @"D:\SourceDirectory\file.txt", entryName: "file.txt");

// Or write an entry from scratch
PaxTarEntry entry = new(entryType: TarEntryType.Directory, entryName: "directory");
writer.WriteEntry(entry);

We can also pair these new APIs with stream-based compression methods, like System.IO.Compression.GZipStream.

Extracting the contents from a compressed tar archive into the filesystem:

using FileStream compressedStream = File.OpenRead("/home/dotnet/SourceDirectory/compressed.tar.gz");
using GZipStream decompressor = new(compressedStream, CompressionMode.Decompress);
TarFile.ExtractToDirectory(source: decompressor, destinationDirectoryName: "/home/dotnet/DestinationDirectory/", overwriteFiles: false);

Reading individual entries from a compressed tar archive:

using FileStream compressedStream = File.OpenRead(@"D:\SourceDirectory\compressed.tar.gz");
using GZipStream decompressor = new(compressedStream, CompressionMode.Decompress);
using TarReader reader = new(decompressor, leaveOpen: true);
TarEntry? entry;

while ((entry = GetNextEntry(copyData: true)) != null)
{
    Console.WriteLine($"Entry type: {entry.EntryType}, entry name: {entry.Name}");
}

Writing the contents of a directory into a compressed tar archive:

using FileStream compressedStream = File.Create("/home/dotnet/DestinationDirectory/compressed.tar.gz");
using GZipStream compressor = new(compressedStream, CompressionMode.Compress);
TarFile.CreateFromDirectory(sourceDirectoryName: "/home/dotnet/SourceDirectory/", destination: compressor, includeBaseDirectory: true);

Or write individual entries into a compressed tar archive:

using FileStream compressedStream = File.Create(@"D:\DestinationDirectory\compressed.tar.gz");
using (GZipStream compressor = new(compressedStream, CompressionMode.Compress))
{
    using (TarWriter writer = new TarWriter(compressor, TarFormat.Pax, leaveOpen: true))
    {
        // Add an entry from an existing file
        writer.WriteEntry(fileName: @"D:\SourceDirectory\file.txt", entryName: "file.txt");

        // Or write an entry from scratch
        PaxTarEntry entry = new(entryType: TarEntryType.Directory, entryName: "directory");
        writer.WriteEntry(entry);
    }
}

APIs

namespace System.Formats.Tar
{
    public static class TarFile
    {
        public static void CreateFromDirectory(string sourceDirectoryName, string destinationFileName, bool includeBaseDirectory);
        public static void CreateFromDirectory(string sourceDirectoryName, Stream destination, bool includeBaseDirectory);
        public static void ExtractToDirectory(string sourceFileName, string destinationDirectoryName, bool overwriteFiles);
        public static void ExtractToDirectory(Stream source, string destinationDirectoryName, bool overwriteFiles);
    }
    public enum TarEntryType : byte
    {
        V7RegularFile = '\0',
        RegularFile = '0',
        HardLink = '1',
        SymbolicLink = '2',
        CharacterDevice = '3',
        BlockDevice = '4',
        Directory = '5',
        Fifo = '6',
        GlobalExtendedAttributes = 'g',
        ExtendedAttributes = 'x',
        ContiguousFile = '7',
        DirectoryList = 'D',
        LongLink = 'K',
        LongPath = 'L',
        MultiVolume = 'M',
        RenamedOrSymlinked = 'N',
        SparseFile = 'S',
        TapeVolume = 'T',
    }
    public enum TarFormat
    {
        Unknown = 0,
        V7 = 1,
        Ustar = 2,
        Pax = 3,
        Gnu = 4,
    }
    public sealed class TarReader : System.IDisposable
    {
        public TarReader(System.IO.Stream archiveStream, bool leaveOpen = false);
        public System.Formats.Tar.TarFormat Format { get; }
        public System.Collections.Generic.IReadOnlyDictionary<string, string>? GlobalExtendedAttributes { get; }
        public void Dispose();
        public System.Formats.Tar.TarEntry? GetNextEntry(bool copyData = false);
    }
    public sealed class TarWriter : System.IDisposable
    {
        public TarWriter(System.IO.Stream archiveStream, System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, string>>? globalExtendedAttributes = null, bool leaveOpen = false);
        public TarWriter(System.IO.Stream archiveStream, System.Formats.Tar.TarFormat archiveFormat, bool leaveOpen = false);
        public System.Formats.Tar.TarFormat Format { get; }
        public void Dispose();
        public void WriteEntry(string fileName, string? entryName);
        public void WriteEntry(System.Formats.Tar.TarEntry entry);
    }
    public abstract class TarEntry
    {
        internal TarEntry();
        public int Checksum { get; }
        public System.IO.Stream? DataStream { get; set; }
        public System.Formats.Tar.TarEntryType EntryType { get; }
        public int Gid { get; set; }
        public long Length { get; }
        public string LinkName { get; set; }
        public TarFileMode Mode { get; set; }
        public System.DateTimeOffset ModificationTime { get; set; }
        public string Name { get; set; }
        public int Uid { get; set; }
        public void ExtractToFile(string destinationFileName, bool overwrite);
        public override string ToString();
    }
    public sealed class V7TarEntry : TarEntry
    {
        public V7TarEntry(System.Formats.Tar.TarEntryType entryType, string entryName);
    }
    public abstract class PosixTarEntry : TarEntry
    {
        internal PosixTarEntry();
        public int DeviceMajor { get; set; }
        public int DeviceMinor { get; set; }
        public string GroupName { get; set; }
        public string UserName { get; set; }
        public override string ToString();
    }
    public sealed class UstarTarEntry : TarEntryPosix
    {
        public UstarTarEntry(System.Formats.Tar.TarEntryType entryType, string entryName);
    }
    public sealed class PaxTarEntry : TarEntryPosix
    {
        public PaxTarEntry(System.Formats.Tar.TarEntryType entryType, string entryName, System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, string>>? extendedAttributes);
        public System.Collections.Generic.IReadOnlyDictionary<string, string> ExtendedAttributes { get; }
    }
    public sealed class GnuTarEntry : TarEntryPosix
    {
        public GnuTarEntry(System.Formats.Tar.TarEntryType entryType, string entryName);
        public System.DateTimeOffset AccessTime { get; set; }
        public System.DateTimeOffset ChangeTime { get; set; }
    }

    [System.FlagsAttribute]
    public enum TarFileMode
    {
        None = 0,
        OtherExecute = 1,
        OtherWrite = 2,
        OtherRead = 4,
        GroupExecute = 8,
        GroupWrite = 16,
        GroupRead = 32,
        UserExecute = 64,
        UserWrite = 128,
        UserRead = 256,
        StickyBit = 512,
        GroupSpecial = 1024,
        UserSpecial = 2048,
    }
}

Notes

• We are not yet including any async APIs. They will be included in a future preview release.
• In the Ustar, PAX and GNU formats, we are not yet supporting the UName, GName, DevMajor and DevMinor fields.
• In the GNU format, we won't be supporting the rare entry types for writing, like sparse files, multi-volume, renamed or symlinked, tape volume.

[JulieLeeMSFT]

CodeGen

Community PRs (Many thanks to JIT community contributors!)

From @sandreenko

• Reenable crossgen2 promotion outside version bubble runtime#66983

From @SingleAccretion

• Only use first field maps for shared statics runtime#66558
• Do not generate TYP_STRUCT LCL_FLD nodes runtime#66251
• Memory containment for FIELD_LIST operands on x86 runtime#65803
• Use a simpler representation for array addresses after morph runtime#64581
• Account for global stores in hoisting properly runtime#67213
• Add some documentation on GT_CAST's semantics runtime#67208
• Fix missing side effect flags on HWI nodes runtime#67206
• Allow null-based ARR_ADDRs runtime#67486
• More XARCH PUTARG_STK CQ improvements runtime#67400

From @SkiFoD

• Draft for x ^ 0 opt (#65071) runtime#65072

Dynamic PGO

• Rely on PGO for isinst/castclass runtime#65922

Arm64

• ARM64 - Emitting msub instruction runtime#66621
• ARM64 - Optimizing a % b operations part 2 runtime#66407
• [arm64] JIT: Better addressing modes around bytes runtime#66902
• [arm64] Addressing modes for SIMD runtime#67490
• [arm64] Add RCPC ISA (8.3+) and use ldap for volatile reads runtime#67384

Loop Optimizations

• Enable multiple nested "no GC" region requests runtime#68105

General Optimizations

• JIT: Optimize unused array allocations runtime#67205
• Optimize ValueNumStore::GetVNFunc runtime#67395
• vm: Improve getStringLiteral for dynamic context runtime#68055

[AndyAyersMS]

On Stack Replacement (aka OSR)

On Stack Replacement allows the runtime to change the code executed by currently running methods in the middle of method execution, while those methods are active "on stack." It serves as a complement to tiered compilation.

dotnet/runtime#65675 enabled OSR by default on x64 and Arm64, and enabled quick jitting for methods with loops on those same platforms.

OSR allows long-running methods to switch to more optimized versions mid-execution, so the runtime can jit all methods quickly at first and then transition to more optimized versions when those methods are called frequently (via tiered compilation) or have long-running loops (via OSR).

Performance Impact

OSR improves startup time. Almost all methods are now initially jitted by the quick jit. We have seen 25% improvement in startup time in jitting-heavy applications like Avalonia “IL” spy, and the various TechEmpower benchmarks we track show 10-30% improvements in time to first request (see chart below: OSR was enabled by default on March 30).

OSR can also improve performance of applications, and in particular, applications using Dynamic PGO, as methods with loops are now better optimized. For example, the Array2 microbenchmark showed dramatic improvement when OSR was enabled.

Further technical details

See the OSR Design Document for details on how OSR works.

See OSR Next Steps for details on the work that went into enabling OSR, and possible future enhancements.

[leecow]

.NET 7 GA is available. Closing these pre-release issues.