From 67642b9de72180c44b6461b54e216fd657cdee36 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?S=C3=A9bastien=20Doeraene?= Date: Tue, 23 Apr 2024 17:22:36 +0200 Subject: [PATCH] Add the activity report for 2024 Q1 and the roadmap for 2024 Q2. --- projects.md | 2 +- records.md | 1 + records/2024-Q1-activity-report.md | 249 +++++++++++++++++++++++++++++ records/2024-Q1-roadmap.md | 2 +- records/2024-Q2-roadmap.md | 129 +++++++++++++++ 5 files changed, 381 insertions(+), 2 deletions(-) create mode 100644 records/2024-Q1-activity-report.md create mode 100644 records/2024-Q2-roadmap.md diff --git a/projects.md b/projects.md index 93c58df..4639cf3 100644 --- a/projects.md +++ b/projects.md @@ -1,3 +1,3 @@ --- -redirect_to: /records/2024-Q1-roadmap.html +redirect_to: /records/2024-Q2-roadmap.html --- diff --git a/records.md b/records.md index ff073d0..71c2d09 100644 --- a/records.md +++ b/records.md @@ -14,6 +14,7 @@ Every quarter, the Scala Center publishes an activity report detailing the work that has been achieved. You can find the roadmap for the current quarter in the [Projects page]({% link projects.md %}). +- [2024, Q1]({% link records/2024-Q1-activity-report.md %}) - [2023, Q4]({% link records/2023-Q4-activity-report.md %}) - [2023, Q3]({% link records/2023-Q3-activity-report.md %}) - [2023, Q2]({% link records/2023-Q2-activity-report.md %}) diff --git a/records/2024-Q1-activity-report.md b/records/2024-Q1-activity-report.md new file mode 100644 index 0000000..30fd71c --- /dev/null +++ b/records/2024-Q1-activity-report.md @@ -0,0 +1,249 @@ +--- +layout: contact +title: Scala Center Activity Report for 2024 Q1 +--- + +Scala Center team: +Darja Jovanovic, 100%; +Anatolii Kmetiuk, 100% until March 15; +Adrien Piquerez, 80%; +Jamie Thompson, 100%; +Sébastien Doeraene, 100%; +Guillaume Martres, 20%; +Valérie Meillaud: 30%. +VirtusLab team: Jędrzej Rochala, 100%. + +## At a Glance +{: .no_toc} + +* Table of Contents +{:toc} + + +## Language, Compiler, Standard Library + +### Maintainance of the Scala 3 Compiler + +For Scala 3. + +Every month, about 100 new issues are opened on [the Scala 3 repository](https://github.com/lampepfl/dotty/). +The project welcomes any help it can get in triaging, bug-fixing, PR reviewing, etc. + +Our goal is to solve long-standing issues while keeping up with new ones. +We also aim to get more people involved in working on the compiler to ensure the sustainability of the project. + +We contributed PRs for bug fixes in various areas. + +### Improving Performance of the Scala 3 compiler + +For Scala 3. + +#### Support for pipelined concurrent compilation + +We merged PRs [scala/scala3#18880] and [scala/scala3#20153], which from Scala 3.5.0 enable support for pipelined builds in sbt (pipelined Scala 2 builds are supported since 2020). +Pipelining enables multi-project builds to increase throughput by compiling upstream sub-projects in parallel with a downstream sub-project. +In the contributors forum, [early testers report](https://contributors.scala-lang.org/t/try-out-pipelined-builds-in-scala-3-5-nightly/6627) improvements from 10%-30% in build times. +The maximum potential for improvements really depends on each individual build layout. + +The pipeline compilation mode will act as the basis for further compilation modes that can increase parallelism in builds, such as an initial outline typing mode, currently incubating in PR [scala/scala3#19589]. + +We also improved the Java type checker in the compiler (PR [scala/scala3#19747]), which is necessary for pipeline builds which export Java definitions to TASTy format. + +#### Computing compilation metadata in parallel. + +Another way to potentially improve compilation times is move computation of various data formats used by Scala tooling to a separate thread. +We are currently supervising a student project at EPFL to compute Zinc metadata using TASTy Query -- which can be offloaded to a separate thread after the pickler phase is completed. +Typically, computation of Zinc metadata adds 10% time to the build of a single project. +We hope to reduce that overhead. + +### Scala.js-specific Minifier + +For Scala 2 and 3. + +During this quarter, we implemented various optimizations reducing code size of Scala.js projects. +The main ingredient was a global property renamer for the Scala.js linker, which we call the minifier. +The minifier is enabled by default when the Google Closure Compiler cannot be used, which is notably the case when emitting ECMAScript modules. + +Other optimizations apply in all configurations, and include: + +* purity analysis of constructors in order to better elide unused fields and modules +* dealiasing of `val`s in top-level objects when their right-hand-sides are constants, references to other objects, or references to fields of other objects +* "inlining" of local `val`s that are used only once + +We shipped all of these as part of [Scala.js 1.16.0](https://www.scala-js.org/news/2024/03/19/announcing-scalajs-1.16.0/). +Most feedback reports from users suggest converge on 35-40% code size reduction for real-world applications. + +### WebAssembly backend for Scala.js + +For Scala 2 and 3. + +In collaboration with VirtusLab, we have developed an initial version of a [WebAssembly backend for Scala.js](https://github.com/tanishiking/scala-wasm). +The implementation already passes the full test suite of Scala.js, with the exception of `@JSExport`ed members in Scala classes. + +The code produced is not yet optimized. +Code size is however comparable to the minified output for JavaScript. +This suggests that after optimizations, we will reach significantly smaller bundles. +We do not know yet what the performance profiles will look like. + +The next steps on this project are to merge the work into Scala.js core and ship it as part of the next release. +We will then look into enabling the Scala.js optimizer in the WebAssembly backend. + +### TASTy Reader for Scala 2 + +For Scala 2. + +We added support in Scala 2.13.13 (PR [scala/scala#10670][scala/scala#10670]) for TASTy produced by Scala 3.4. +This update integrates support for two major changes: + +* the ability to read from classpath entries that only contain TASTy files, such as when using pipeline compilation. +* the ability to read Java definitions from TASTy. + +### Scala Improvement Process + +For Scala 3. + +The [Scala Improvement Process](https://docs.scala-lang.org/sips/) coordinates the evolution of the language. +It ensures that the decisions are made by taking into account the needs of all the stakeholders of the language. + +Several SIPs were approved for Implementation in the past quarter. +The binary literals SIP made it all the way to final approval, and will ship in Scala 3.5.0. + +## Developer Experience + +### Debugger in Metals + +For Scala 2 and 3. + +We released the [4.x major version](https://github.com/scalacenter/scala-debug-adapter/releases/tag/v4.0.0) of the Scala Debug Adapter, which features pretty stack traces, run-time evaluation and hot-code replace: + +* In the pretty stack traces (scala 3 only) we print the fully typed Scala signatures instead of the erased Java ones. +* The run-time evaluator (Scala 2 and 3) is able to evaluate Java-like expressions using run-time information of the program. It speeds up the watches and the debug console, and allows to access members without casting. The compiler-based evaluation is still used as a fall-back to evaluate Scala expressions: overload resolution, pattern matching, lambdas, implicit resolution, macro expansion, etc. +* The hot-code replace feature (Scala 2 and 3) allows to re-compile code during the debug session and to load it. It changes the implementation of methods that are already loaded by the debugged JVM. + +This new version of the Scala Debug Adapter is available in Metals since version 1.3.0. + +### Presentation Compiler + +For Scala 3. + +We are always working on improving the IDE experience. +During this quarter, we fixed a number of issues in the presentation compiler, which powers autocompletion, signature hints, etc. + +We also invested some time into a significant refactoring in the autocompletion implementation. +That refactoring helped us bring two improvements: + +* introducing fuzzy search for completions, making them more resilient to user input, and +* improving the heuristic for completions of constructors and `apply` methods to classes and values. + +They will be shipped in Scala 3.3.4 in the LTS line, and 3.5.0 in the Next line. + +### sbt 2 + +For Scala 2 and Scala 3. + +We pursued the migration of sbt to Scala 3: + +* We cleaned up parts of the sbt codebase: [reducing abstraction in Execute](https://github.com/sbt/sbt/pull/7444), [using Tuple.Map instead of AList](https://github.com/sbt/sbt/pull/7456), [removing usage of Manifest](https://github.com/sbt/sbt/pull/7490). +* We fixed many of the pending tests: mostly in [#7522](https://github.com/sbt/sbt/pull/7522), but also in [#7492](https://github.com/sbt/sbt/pull/7492), [#7494](https://github.com/sbt/sbt/pull/7494), [#7517](https://github.com/sbt/sbt/pull/7517), [#7519](https://github.com/sbt/sbt/pull/7519), [#7521](https://github.com/sbt/sbt/pull/7521), and [#7538](https://github.com/sbt/sbt/pull/7538). + +We use the [#7462 meta issue](https://github.com/sbt/sbt/issues/7462) to track the progress of porting sbt to Scala 3. + +Additionally, we are supporting the development of remote caching in sbt 2 by reviewing pull requests ([#7464](https://github.com/sbt/sbt/pull/7464), [#7525](https://github.com/sbt/sbt/pull/7525)) and contributing ([#7539](https://github.com/sbt/sbt/pull/7539)). + +### Zed Editor Support for Scala + +For Scala 2 and 3. + +The Zed text editor become open source in early 2024, and gathered a lot of attention across the industry. +We investigated adding support for Scala with Metals as an LSP extension. +In the process, we identified and reported a bug in Zed's LSP client implementation (see issue [zed-industries/zed#7091][zed-industries/zed#7091]). +We also coordinated with other community members. +A Scala plugin has now been accepted to the Zed extension store. + +### Metals maintenance + +For Scala 2 and 3. + +We are generally working on improving the stability of Metals. +One major difficulty is that effectively reporting issues is difficult for users, notably in the presence of crashes. + +To get better insight into everyday issues, we are implementing a telemetry system inside Metals. +It will allow users to opt in to automatically sending reports in the event of crashes. +We expect that these additional data will help us make Metals more robust in the future. + +## Community and Contributor Experience + +For Scala 2 and Scala 3 throughout. + +### Ambassador Program + +We incubated the [Scala Ambassador program](https://github.com/scalacenter/ambassadors). +This is a program to foster a new generation of Scala community leaders, able to competently answer community questions. +Those people will also be empowered to organize local community events and otherwise help their local communities thrive. + +Currently, the program is in its early stages, as we welcome our first Ambassadors on an invitation basis. +The program is meant to recognize and support the independent work of active community leaders. +We publicly announced it at Scalar 2024 and [on our website](https://www.scala-lang.org/blog/2024/03/28/ambassadors-initiative.html). + +The ambassador program will strengthen the community, amplify the official story around Scala and increase activity in the local Scala communities. + +### Google Summer of Code + +Google Summer of Code has been a long-standing vehicle for the Scala Center to attract newcomers to the Scala OSS world. + +This year, the Scala Center was accepted again as an organization. +We received 15 good proposals from students, supported by 11 mentors from across the Scala community. +We are in the process of submitting the proposals to Google for ranking. +After we find out how many projects Google will accept for the implementation stage, we will mentor students and support other mentors throughout the program. + +### Scala.IO + +We organized a Scala Spree co-located with the Scala.IO conference. +We also gave a talk there about the Scala 3 Compiler Academy, which covers how to bootstrap and develop similar projects. + +### Scalar Conference + +At Scalar, Warsaw, we presented a talk about how to use type class derivation "Mirrors for operations, not data". +The talk raises awareness about the new type-class derivation feature of Scala 3. +It proposes new directions to take it in, such as synthesizing web services from descriptions. + +Our team also held a booth at Scalar. +At the booth, we answered questions from the community, collected feedback and worked to establish new corporate partnerships. + +### Scala Website Scams + +In February and March 2024, the Scala community was the target of an elaborate scam scheme. +We took immediate action to warn the community, shut down domain names used by the scammers, and support victims--to the extend possible. + +Among the most effective measures we took, we added warning banners to the official Scala websites. +Since the fraudulent sites copied major sections of our websites, many links take potential victims to the real site, where they see our warning. +We have had reports that this actually helped to prevent more people become victims to this scam. + +We extensively communicated about the scams and our response. +You can find more details in [an initial warning blog post](https://www.scala-lang.org/blog/2024/03/01/fake-scala-courses.html) and [an update on our response](https://www.scala-lang.org/blog/2024/03/18/scam-response.html). + +We have not received further complaints in April, which we hope means the end of this particular attack. + +### Scala in Embedded Systems programming + +We have been consulting with the Embedded Systems Laboratory at EPFL on a project to create a framework for developing FPGA accelerators in Scala, based upon the Chisel DSL. +We aim to promote Scala's abstraction capabilities to make it easier for hardware engineers to do their work. + +### Compiler Sprees + +We maintained our involvement in the [Scala 3 Compiler Academy Issue Spree](https://www.scala-lang.org/blog/2022/11/02/compiler-academy.html). +We collaborated with both new and experienced contributors on issues such as [dotty#19464](https://github.com/lampepfl/dotty/pull/19464) and [dotty#19463](https://github.com/lampepfl/dotty/pull/19463). + +Since its inception, the compiler spree has helped close [more then a hundred issues](https://github.com/lampepfl/dotty/issues?q=is%3Aissue+sort%3Aupdated-desc+label%3ASpree+is%3Aclosed) with the help of over 80 contributors. + +[scala/improvement-proposals#67]: https://github.com/scala/improvement-proposals/pull/67 +[scala/scala3#18880]: https://github.com/scala/scala3/pull/18880 +[scala/scala-lang#1604]: https://github.com/scala/scala-lang/pull/1604 +[scala/docs.scala-lang#2993]: https://github.com/scala/docs.scala-lang/pull/2993 +[zed-industries/zed#7091]: https://github.com/zed-industries/zed/issues/7091 +[scala/scala#10670]: https://github.com/scala/scala/pull/10670 +[pipeline-thread]: https://contributors.scala-lang.org/t/try-out-pipelined-builds-in-scala-3-5-nightly/6627 +[scala/scala3#19747]: https://github.com/scala/scala3/pull/19747 +[scala/scala3#19589]: https://github.com/scala/scala3/pull/19589 +[scala/scala3#20153]: https://github.com/scala/scala3/pull/20153 +[scala/scala3#19838]: https://github.com/scala/scala3/pull/19838 diff --git a/records/2024-Q1-roadmap.md b/records/2024-Q1-roadmap.md index ac579f5..e9a7cc4 100644 --- a/records/2024-Q1-roadmap.md +++ b/records/2024-Q1-roadmap.md @@ -8,7 +8,7 @@ We also post regular updates about our projects on the [Scala Contributors forum To have more information about our _completed_ projects, please see the [quarterly activity reports]({% link records.md %}). -## Roadmap for 2023 Q4 +## Roadmap for 2024 Q1 {: .no_toc} The following sections present our plan for the current quarter. diff --git a/records/2024-Q2-roadmap.md b/records/2024-Q2-roadmap.md new file mode 100644 index 0000000..f089427 --- /dev/null +++ b/records/2024-Q2-roadmap.md @@ -0,0 +1,129 @@ +--- +layout: contact +title: Scala Center Roadmap for 2024 Q2 +--- + +This page lists the projects that the Scala Center plans to work on during 2024 Q2. +We also post regular updates about our projects on the [Scala Contributors forum](https://contributors.scala-lang.org/c/scala-center/25). + +To have more information about our _completed_ projects, please see the [quarterly activity reports]({% link records.md %}). + +## Roadmap for 2024 Q2 +{: .no_toc} + +The following sections present our plan for the current quarter. +Every project description is followed by the concrete results we will deliver and their expected outcome on the Scala community. + +* Table of Contents +{:toc} + +### Language, Compiler, Standard Library + +Our mission is to reduce the number of bugs in the compiler implementation, to help the community to contribute to these tools, and to make sure they evolve in a way that takes into account the needs of the community. + +#### WebAssembly backend for Scala.js + +During the first quarter of this year, we implemented an initial WebAssembly backend for Scala.js in collaboration with VirtusLab. +We will now polish it and merge it into Scala.js core. +It will be shipped as part of an upcoming Scala.js version. + +Currently, the WebAssembly backend produces correct but unoptimized code. +Further work is necessary to enable the Scala.js optimizer with that backend. + +We will also investigate how we can target non-JavaScript WebAssembly hosts. +This will significantly further the reach of the Scala language to new ecosystems. + +#### Compiler Performance + +In this quarter, we will continue the work we have been doing to improve compiler performance. +In particular, we expect to continue our work on computing metadata in parallel with the main pipeline. + +#### Scala Improvement Process + +The [Scala Improvement Process](https://docs.scala-lang.org/sips/index.html) is a process for submitting changes to the Scala language. +It aims to evolve Scala openly and collaboratively. + +We hold regular committee meetings, which we both coordinate and participate in. + +#### Improvement Process for the library + +Following the successful launch of the Scala Improvement Process for Scala 3, we are looking to establish a similar process for the Scala Standard Library evolution. +This will be necessary once we effectively implement [SIP-51, Drop Forwards Binary Compatibility of the Scala 2.13 Standard Library](https://docs.scala-lang.org/sips/drop-stdlib-forwards-bin-compat.html). + +### Documentation and Education + +Our mission is to simplify the structure and content of the website, to create and maintain high-quality online educational content (including online courses), and to help the community to contribute to the website. + +#### Getting Started experience with scala-cli + +As soon as scala-cli gets officially released as the new Scala command ([SIP-46](https://docs.scala-lang.org/sips/scala-cli.html)), we will update the getting started experience to be based on it. +We will create a new learning path for beginners, recommending the new Scala command. +There will be emphasis on important Scala tooling, and how it helps the developer experience. + +#### Continue the fusion of Scala 2 and Scala 3 docs + +In the past year, we have progressively fused the documentation of Scala 2 and Scala 3. +In this quarter, we will continue this effort. + +#### Standard library API documentation + +Compared to other popular languages, the API documentation of our standard library is terse. +While that terseness is somewhat compensated by good collections guides, the Scaladoc remains an important source of documentation, as it is available right in our IDEs. + +We will start working on a library-wide review of the API documentation, and will make sure it contains comprehensive, standalone information. + +#### Tutorials + +We identified tutorials as a form of documentation that is underrepresented in the Scala ecosystem. +As part of the Scala Toolkit effort, we plan to write more tutorials to get things done in Scala. + +### Developer Experience + +Our mission is to make sure the tools Scala developers use to edit, analyze, navigate through, transform, compile, run, and debug Scala programs are as easy to use as possible, that they work reliably for everyone, and deliver a great developer experience. + +#### sbt 2.x + +As mentioned in our 2024 Q1 report, work on sbt 2.x is well under way. +One of the main early tasks consists in migrating the existing codebase to Scala 3. + +Many tests now pass after the migration, but much remains to be done. +Some macros, in particular, were left behind in the initial migration. +During this quarter, we will contribute to finishing the migration to Scala 3. + +In addition, we intend to work on the loading process. +We will investigate how we can make it lighter and faster. + +### Community and Contributor Experience + +Our mission is to create the best environment for the emergence of a strong Scala ecosystem made of high-quality, reliable, libraries that bring simple solutions to complex problems. + +#### Communication + +We will conduct actions aiming at communicating a positive image of Scala, and making people excited about it. + +- We will regularly share our achievements and engage the community on our projects _via_ our [LinkedIn](https://www.linkedin.com/company/scala-center/) page. +- We will speak at tech conferences and local meetups to encourage people to contribute to the Scala ecosystem, and to let non-Scala programmers know about Scala. + +### Maintenance Work + +We will also spend a small part of our time reviewing pull requests, triaging issues, and fix issues for the following projects, to make sure important points are addressed: + +- Scala 3 compiler +- Scala 2 TASTy reader +- Scala.js +- Scaladex +- Scalafix +- Scastie +- Coursier +- Bloop +- Metals +- sbt +- sbt-dependency-submission +- scala-debug-adapter +- tasty-query +- tasty-mima + +## Advisory Board Proposals +{: .no_toc} + +For reference, you can see [here](https://github.com/scalacenter/advisoryboard/tree/master/proposals) the list of Advisory Board proposals and their respective status.