diff --git a/guides/source/documents.yaml b/guides/source/documents.yaml index ae5a1d3df120a..bf29eb4015040 100644 --- a/guides/source/documents.yaml +++ b/guides/source/documents.yaml @@ -292,7 +292,6 @@ name: Threading and Code Execution in Rails url: threading_and_code_execution.html description: This guide describes the considerations needed and tools available when working directly with concurrency in a Rails application. - work_in_progress: true - name: Contributing documents: diff --git a/guides/source/threading_and_code_execution.md b/guides/source/threading_and_code_execution.md index 2337a2611b668..f8d5c5d07d33d 100644 --- a/guides/source/threading_and_code_execution.md +++ b/guides/source/threading_and_code_execution.md @@ -5,65 +5,78 @@ Threading and Code Execution in Rails After reading this guide, you will know: -* What code Rails will automatically execute concurrently -* How to integrate manual concurrency with Rails internals -* How to wrap all application code +* Where to find concurrent code execution in Rails +* How to integrate manual concurrency within Rails +* How to wrap application code using the Rails Executor * How to affect application reloading -------------------------------------------------------------------------------- -Automatic Concurrency ---------------------- +Automatic Concurrency in Rails +------------------------------ -Rails automatically allows various operations to be performed at the same time. +Rails automatically allows various operations to be performed at the same time +(concurrently) in order for an application to run more efficiently. In this +section, we will explore some of the ways this happens behind the scenes. When using a threaded web server, such as the default Puma, multiple HTTP -requests will be served simultaneously, with each request provided its own -controller instance. +requests will be served simultaneously, with a separate controller instance for +each request. -Threaded Active Job adapters, including the built-in Async, will likewise -execute several jobs at the same time. Action Cable channels are managed this -way too. +Threaded Active Job adapters, including the built-in Async adapter, will +likewise execute several jobs at the same time. Action Cable channels are +managed this way too. + +Asynchronous Active Record queries are also performed in the background, +allowing other processes to run on the main thread. These mechanisms all involve multiple threads, each managing work for a unique instance of some object (controller, job, channel), while sharing the global process space (such as classes and their configurations, and global variables). -As long as your code doesn't modify any of those shared things, it can mostly +As long as your code doesn't modify any of those shared resources, it can mostly ignore that other threads exist. -The rest of this guide describes the mechanisms Rails uses to make it "mostly -ignorable", and how extensions and applications with special needs can use them. +The rest of this guide describes the mechanisms Rails uses to make other threads +"mostly ignorable", and how extensions and applications with special +requirements can use these mechanisms. + +NOTE: You can read more about how to configure Rails' concurrency in the +[Framework Behavior](#framework-behavior) section. + -Executor --------- +### Threads vs Fibers -The Rails Executor separates application code from framework code: any time the -framework invokes code you've written in your application, it will be wrapped by -the Executor. +The `active_support.isolation_level` value in your `config/application.rb` file +provides you the option to define where Rails' internal state should be stored +while tasks are run. If you use a fiber-based server or job processor (e.g. +`falcon`), you should set this value to `:fiber`, otherwise it is best to set it +to `:thread`. -The Executor consists of two callbacks: `to_run` and `to_complete`. The Run -callback is called before the application code, and the Complete callback is -called after. +Wrapping Application Code +------------------------- -### Default Callbacks +### The Rails Executor -In a default Rails application, the Executor callbacks are used to: +The Rails Executor separates application code from framework code by wrapping +code that you've written, which is necessary when threads are being used. -* track which threads are in safe positions for autoloading and reloading -* enable and disable the Active Record query cache -* return acquired Active Record connections to the pool -* constrain internal cache lifetimes +#### Callbacks -Prior to Rails 5.0, some of these were handled by separate Rack middleware -classes (such as `ActiveRecord::ConnectionAdapters::ConnectionManagement`), or -directly wrapping code with methods like -`ActiveRecord::Base.connection_pool.with_connection`. The Executor replaces -these with a single more abstract interface. +The Executor consists of two callbacks: `to_run` and `to_complete`. The `to_run` +callback is called before the application code, and the `to_complete` callback +is called after. -### Wrapping Application Code +In a default Rails application, the Rails Executor callbacks are used to: + +* Track which threads are in safe positions for autoloading and reloading. +* Enable and disable the Active Record query cache. +* Return acquired Active Record connections to the pool. +* Constrain internal cache lifetimes. + +#### Code Execution If you're writing a library or component that will invoke application code, you -should wrap it with a call to the executor: +should wrap it with a call to the Executor: ```ruby Rails.application.executor.wrap do @@ -72,15 +85,17 @@ end ``` TIP: If you repeatedly invoke application code from a long-running process, you -may want to wrap using the [Reloader](#reloader) instead. +may want to wrap using the [Reloader](#the-reloader) instead. + Each thread should be wrapped before it runs application code, so if your application manually delegates work to other threads, such as via `Thread.new` -or Concurrent Ruby features that use thread pools, you should immediately wrap -the block: +or [Concurrent Ruby](https://github.com/ruby-concurrency/concurrent-ruby) +features that use thread pools, you should immediately wrap the block: ```ruby Thread.new do + # no code here Rails.application.executor.wrap do # your code here end @@ -88,14 +103,12 @@ end ``` NOTE: Concurrent Ruby uses a `ThreadPoolExecutor`, which it sometimes configures -with an `executor` option. Despite the name, it is unrelated. - -The Executor is safely re-entrant; if it is already active on the current -thread, `wrap` is a no-op. +with an `executor` option. Despite the name, it is _not_ related to the Rails +Executor. -If it's impractical to wrap the application code in a block (for -example, the Rack API makes this problematic), you can also use the `run!` / -`complete!` pair: +If it's impractical to wrap the application code in a block (for example, the +Rack API makes this problematic), you can also use the `run!` / `complete!` +pair: ```ruby Thread.new do @@ -106,19 +119,28 @@ ensure end ``` -### Concurrency +#### Running Mode + +The Executor will put the current thread into `running` mode in the [Load +Interlock](#load-interlock). This operation will block temporarily if another +thread is currently either autoloading a constant or unloading/reloading the +application. -The Executor will put the current thread into `running` mode in the [Reloading -Interlock](#reloading-interlock). This operation will block temporarily if another -thread is currently unloading/reloading the application. +### The Reloader -Reloader --------- +Like the Executor, the +[Reloader](https://api.rubyonrails.org/classes/ActiveSupport/Reloader.html) also +wraps application code. The Reloader is only suitable where a long-running +framework-level process repeatedly calls into application code, such as for a +web server or job queue. -Like the Executor, the Reloader also wraps application code. If the Executor is -not already active on the current thread, the Reloader will invoke it for you, -so you only need to call one. This also guarantees that everything the Reloader -does, including all its callback invocations, occurs wrapped inside the +NOTE: Rails automatically wraps web requests and Active Job workers, so you'll +rarely need to invoke the Reloader for yourself. Always consider whether the +Executor is a better fit for your use case. + +If the Executor is not already active on the current thread, the Reloader will +invoke it for you, so you only need to call one. This also guarantees that +everything the Reloader does, including its callbacks, occurs wrapped inside the Executor. ```ruby @@ -127,38 +149,37 @@ Rails.application.reloader.wrap do end ``` -The Reloader is only suitable where a long-running framework-level process -repeatedly calls into application code, such as for a web server or job queue. -Rails automatically wraps web requests and Active Job workers, so you'll rarely -need to invoke the Reloader for yourself. Always consider whether the Executor -is a better fit for your use case. - -### Callbacks +#### Callbacks Before entering the wrapped block, the Reloader will check whether the running -application needs to be reloaded -- for example, because a model's source file has -been modified. If it determines a reload is required, it will wait until it's -safe, and then do so, before continuing. When the application is configured to -always reload regardless of whether any changes are detected, the reload is -instead performed at the end of the block. +application needs to be reloaded (because a model's source file has been +modified, for example). If it determines a reload is required, it will wait +until it's safe, and then do so, before continuing. When the application is +configured to always reload regardless of whether any changes are detected, the +reload is instead performed at the end of the block. The Reloader also provides `to_run` and `to_complete` callbacks; they are invoked at the same points as those of the Executor, but only when the current execution has initiated an application reload. When no reload is deemed necessary, the Reloader will invoke the wrapped block with no other callbacks. -### Class Unload +#### Class Unload -The most significant part of the reloading process is the Class Unload, where +The most significant part of the reloading process is the "class unload", where all autoloaded classes are removed, ready to be loaded again. This will occur -immediately before either the Run or Complete callback, depending on the -`reload_classes_only_on_change` setting. +immediately before either the `to_run` or `to_complete` callback, depending on +the +[`reload_classes_only_on_change`](configuring.html#config-reload-classes-only-on-change) +setting. Often, additional reloading actions need to be performed either just before or -just after the Class Unload, so the Reloader also provides `before_class_unload` -and `after_class_unload` callbacks. +just after the "class unload", so the Reloader also provides +[`before_class_unload`](https://api.rubyonrails.org/classes/ActiveSupport/Reloader.html#method-c-before_class_unload) +and +[`after_class_unload`](https://api.rubyonrails.org/classes/ActiveSupport/Reloader.html#method-c-after_class_unload) +callbacks. -### Concurrency +#### Concurrency Only long-running "top level" processes should invoke the Reloader, because if it determines a reload is needed, it will block until all other threads have @@ -172,14 +193,16 @@ thread is mid-execution. Child threads should use the Executor instead. Framework Behavior ------------------ -The Rails framework components use these tools to manage their own concurrency -needs too. +The Rails framework components use the Executor and the Reloader to manage their +own concurrency needs too. -`ActionDispatch::Executor` and `ActionDispatch::Reloader` are Rack middlewares -that wrap requests with a supplied Executor or Reloader, respectively. They -are automatically included in the default application stack. The Reloader will -ensure any arriving HTTP request is served with a freshly-loaded copy of the -application if any code changes have occurred. +[`ActionDispatch::Executor`](https://api.rubyonrails.org/classes/ActionDispatch/Executor.html) +and +[`ActionDispatch::Reloader`](https://api.rubyonrails.org/classes/ActionDispatch/Reloader.html) +are Rack middlewares that wrap requests with a supplied Executor or Reloader, +respectively. They are automatically included in the default application stack. +The Reloader will ensure any arriving HTTP request is served with a +freshly-loaded copy of the application if any code changes have occurred. Active Job also wraps its job executions with the Reloader, loading the latest code to execute each job as it comes off the queue. @@ -190,18 +213,19 @@ WebSocket message. Only the message handler is wrapped, though; a long-running Cable connection does not prevent a reload that's triggered by a new incoming request or job. Instead, Action Cable uses the Reloader's `before_class_unload` callback to disconnect all its connections. When the client automatically -reconnects, it will be speaking to the new version of the code. +reconnects, it will be interacting with the new version of the code. The above are the entry points to the framework, so they are responsible for ensuring their respective threads are protected, and deciding whether a reload is necessary. Other components only need to use the Executor when they spawn additional threads. -### Configuration +### Reloader and Executor Configuration -The Reloader only checks for file changes when `config.enable_reloading` is -`true` and so is `config.reload_classes_only_on_change`. These are the defaults in the -`development` environment. +The Reloader only checks for file changes when +[`config.enable_reloading`](configuring.html#config-enable-reloading) and +[`config.reload_classes_only_on_change`](configuring.html#config-reload-classes-only-on-change) +are both `true`. These are the defaults in the `development` environment. When `config.enable_reloading` is `false` (in `production`, by default), the Reloader is only a pass-through to the Executor. @@ -209,11 +233,12 @@ Reloader is only a pass-through to the Executor. The Executor always has important work to do, like database connection management. When `config.enable_reloading` is `false` and `config.eager_load` is `true` (`production` defaults), no reloading will occur, so it does not need the -Reloading Interlock. With the default settings in the `development` environment, the -Executor will use the Reloading Interlock to ensure code reloading is performed safely. +[Load Interlock](#load-interlock). With the default settings in the +`development` environment, the Executor will use the Load Interlock to ensure +constants are only loaded when it is safe. -Reloading Interlock -------------------- +Load Interlock +-------------- The Reloading Interlock ensures that code reloading can be performed safely in a multi-threaded runtime environment. @@ -228,3 +253,86 @@ threads are currently running application code, and ensuring that reloading waits until no other threads are executing application code. +### `permit_concurrent_loads` + +The Executor automatically acquires a `running` lock for the duration of its +block, and autoload knows when to upgrade to a `load` lock, and switch back to +`running` again afterwards. + +Other blocking operations performed inside the Executor block (which includes +all application code), however, can needlessly retain the `running` lock. If +another thread encounters a constant that it must autoload, this can cause a +deadlock. + +For example, assuming `User` is not yet loaded, the following will deadlock: + +```ruby +Rails.application.executor.wrap do + th = Thread.new do + Rails.application.executor.wrap do + User # inner thread waits here; it cannot load + # User while another thread is running + end + end + + th.join # outer thread waits here, holding 'running' lock +end +``` + +To prevent this deadlock, the outer thread can `permit_concurrent_loads`. By +calling this method, the thread guarantees it will not dereference any +possibly-autoloaded constant inside the supplied block. The safest way to meet +that promise is to put it as close as possible to the blocking call: + +```ruby +Rails.application.executor.wrap do + th = Thread.new do + Rails.application.executor.wrap do + User # inner thread can acquire the 'load' lock, + # load User, and continue + end + end + + ActiveSupport::Dependencies.interlock.permit_concurrent_loads do + th.join # outer thread waits here, but has no lock + end +end +``` + +Another example, using Concurrent Ruby: + +```ruby +Rails.application.executor.wrap do + futures = 3.times.collect do |i| + Concurrent::Promises.future do + Rails.application.executor.wrap do + # do work here + end + end + end + + values = ActiveSupport::Dependencies.interlock.permit_concurrent_loads do + futures.collect(&:value) + end +end +``` + +### ActionDispatch::DebugLocks + +If your application is deadlocking and you think the Load Interlock may be +involved, you can temporarily add the ActionDispatch::DebugLocks middleware to +`config/application.rb`: + +```ruby +config.middleware.insert_before Rack::Sendfile, + ActionDispatch::DebugLocks +``` + +If you then restart the application and re-trigger the deadlock condition, +`/rails/locks` will show a summary of all threads currently known to the +interlock, which lock level they are holding or awaiting, and their current +backtrace. + +Generally a deadlock will be caused by the interlock conflicting with some other +external lock or blocking I/O call. Once you find it, you can wrap it with +`permit_concurrent_loads`.