Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a paragraph on recommended values on running Placement in production on K8s #4365

Merged
merged 2 commits into from
Oct 2, 2024

Conversation

elena-kolevska
Copy link
Contributor

Thank you for helping make the Dapr documentation better!

Please follow this checklist before submitting:

  • Commits are signed with Developer Certificate of Origin (DCO - learn more)
  • Read the contribution guide
  • Commands include options for Linux, MacOS, and Windows within codetabs
  • New file and folder names are globally unique
  • Page references use shortcodes instead of markdown or URL links
  • Images use HTML style and have alternative text
  • Places where multiple code/command options are given have codetabs

In addition, please fill out the following to help reviewers understand this pull request:

Description

Adds a paragraph on recommended values on running Placement in production on K8s

Issue reference

Signed-off-by: Elena Kolevska <elena@kolevska.com>
Copy link
Contributor

@alicejgibbons alicejgibbons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor changes, @msfussell can you review now?

@@ -260,6 +260,20 @@ Verify your production-ready deployment includes the following settings:

1. Dapr supports and is enabled to **scope components for certain applications**. This is not a required practice. [Learn more about component scopes]({{< ref "component-scopes.md" >}}).

## Recommended Placement service configuration

The [Placement service]({{< ref "placement.md" >}}) is a critical component in Dapr, responsible for actor placement table dissemination. It's recommended to configure the Placement service with the following values:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The [Placement service]({{< ref "placement.md" >}}) is a critical component in Dapr, responsible for actor placement table dissemination. It's recommended to configure the Placement service with the following values:
The [placement service]({{< ref "placement.md" >}}) is a critical component in Dapr when using the Actors API and is responsible for actor placement table dissemination. Placement tables are disseminated every time a new actor instance created and thus served by a new pod, or when pod membership of an existing actor instance changes. Details on this process found [here](https://docs.dapr.io/developing-applications/building-blocks/actors/actors-features-concepts/#actor-placement-service).
When running in production, it's recommended to configure the placement service with the following values:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elena-kolevska I added one sentence describing what the placement dissemination is, plz correct it if needed and add a link to the doc: https://docs.dapr.io/developing-applications/building-blocks/actors/actors-features-concepts/#actor-placement-service

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feedback

  1. We should remove the word "critical" which does not add anything to this sentence.
  2. I do not understand the need for this section in the sentence "and thus served by a new pod, or when pod membership of an existing actor instance changes" - what do mean by pod membership? This should simply be that when an actor is updated or a new instance added, this information is replicated to other placement service instances, yes?


The [Placement service]({{< ref "placement.md" >}}) is a critical component in Dapr, responsible for actor placement table dissemination. It's recommended to configure the Placement service with the following values:

1. **High availability**. Ensure that the Placement service is highly available and can survive individual node failures. Helm chart value: `dapr_placement.ha=true`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. **High availability**. Ensure that the Placement service is highly available and can survive individual node failures. Helm chart value: `dapr_placement.ha=true`
1. **High availability**. Ensure that the placement service is highly available (3 replicas) and can survive individual node failures. Helm chart value: `dapr_placement.ha=true`

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Placement service is a name so it is Placement service rather than placement service.

The [Placement service]({{< ref "placement.md" >}}) is a critical component in Dapr, responsible for actor placement table dissemination. It's recommended to configure the Placement service with the following values:

1. **High availability**. Ensure that the Placement service is highly available and can survive individual node failures. Helm chart value: `dapr_placement.ha=true`
2. **In memory logs**. Use in-memory Raft log store for faster writes. The tradeoff is more disseminations (and thus, network traffic) in an eventual Placement service pod failure. Helm chart value: `dapr_placement.cluster.forceInMemoryLog=true`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. **In memory logs**. Use in-memory Raft log store for faster writes. The tradeoff is more disseminations (and thus, network traffic) in an eventual Placement service pod failure. Helm chart value: `dapr_placement.cluster.forceInMemoryLog=true`
2. **In-memory logs**. Use in-memory Raft log store for faster writes. The tradeoff is more disseminations (and thus, network traffic) and an eventual placement service pod failure. Helm chart value: `dapr_placement.cluster.forceInMemoryLog=true`

1. **High availability**. Ensure that the Placement service is highly available and can survive individual node failures. Helm chart value: `dapr_placement.ha=true`
2. **In memory logs**. Use in-memory Raft log store for faster writes. The tradeoff is more disseminations (and thus, network traffic) in an eventual Placement service pod failure. Helm chart value: `dapr_placement.cluster.forceInMemoryLog=true`
3. **No metadata endpoint**. Disable the unauthenticated `/placement/state` endpoint which exposes placement table information for the Placement service. Helm chart value: `dapr_placement.metadataEnabled=false`
4. **Timeouts** Control the sensitivity of network connectivity between the Placement service and the sidecars using the below timeout values. You can adjust these values based on your network conditions as needed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
4. **Timeouts** Control the sensitivity of network connectivity between the Placement service and the sidecars using the below timeout values. You can adjust these values based on your network conditions as needed.
4. **Timeouts** Control the sensitivity of network connectivity between the placement service and Dapr sidecars using the below timeout values. Smart default values are set but you can adjust these based on your network conditions.

2. **In memory logs**. Use in-memory Raft log store for faster writes. The tradeoff is more disseminations (and thus, network traffic) in an eventual Placement service pod failure. Helm chart value: `dapr_placement.cluster.forceInMemoryLog=true`
3. **No metadata endpoint**. Disable the unauthenticated `/placement/state` endpoint which exposes placement table information for the Placement service. Helm chart value: `dapr_placement.metadataEnabled=false`
4. **Timeouts** Control the sensitivity of network connectivity between the Placement service and the sidecars using the below timeout values. You can adjust these values based on your network conditions as needed.
1. `dapr_placement.keepAliveTime` sets the interval at which the Placement service sends [keep alive](https://grpc.io/docs/guides/keepalive/) pings to daprd on the gRPC stream to check if the connection is still alive. Lower values will lead to shorter actor rebalancing time in case of pod loss/restart, but higher network traffic during normal operation. Accepts values between `1s` and `10s`. Default is `2s`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. `dapr_placement.keepAliveTime` sets the interval at which the Placement service sends [keep alive](https://grpc.io/docs/guides/keepalive/) pings to daprd on the gRPC stream to check if the connection is still alive. Lower values will lead to shorter actor rebalancing time in case of pod loss/restart, but higher network traffic during normal operation. Accepts values between `1s` and `10s`. Default is `2s`.
1. `dapr_placement.keepAliveTime` sets the interval at which the placement service sends [keep alive](https://grpc.io/docs/guides/keepalive/) pings to Dapr sidecars via gRPC stream to check if the connection is still alive. Lower values will lead to shorter actor rebalancing time in case of pod loss/restart, but higher network traffic during normal operation. Accepts values between `1s` and `10s`. Default is `2s`.

3. **No metadata endpoint**. Disable the unauthenticated `/placement/state` endpoint which exposes placement table information for the Placement service. Helm chart value: `dapr_placement.metadataEnabled=false`
4. **Timeouts** Control the sensitivity of network connectivity between the Placement service and the sidecars using the below timeout values. You can adjust these values based on your network conditions as needed.
1. `dapr_placement.keepAliveTime` sets the interval at which the Placement service sends [keep alive](https://grpc.io/docs/guides/keepalive/) pings to daprd on the gRPC stream to check if the connection is still alive. Lower values will lead to shorter actor rebalancing time in case of pod loss/restart, but higher network traffic during normal operation. Accepts values between `1s` and `10s`. Default is `2s`.
2. `dapr_placement.keepAliveTimeout` sets the timeout period for daprd to respond to the Placement service's [keep alive](https://grpc.io/docs/guides/keepalive/) pings before the Placement service closes the connection. Lower values will lead to shorter actor rebalancing time in case of pod loss/restart, but higher network traffic during normal operation. Accepts values between `1s` and `10s`. Default is `3s`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. `dapr_placement.keepAliveTimeout` sets the timeout period for daprd to respond to the Placement service's [keep alive](https://grpc.io/docs/guides/keepalive/) pings before the Placement service closes the connection. Lower values will lead to shorter actor rebalancing time in case of pod loss/restart, but higher network traffic during normal operation. Accepts values between `1s` and `10s`. Default is `3s`.
2. `dapr_placement.keepAliveTimeout` sets the timeout period for Dapr sidecars to respond to the placement service's [keep alive](https://grpc.io/docs/guides/keepalive/) pings before closing the connection. Lower values will lead to shorter actor rebalancing time in case of pod loss/restart, but higher network traffic during normal operation. Accepts values between `1s` and `10s`. Default is `3s`.

4. **Timeouts** Control the sensitivity of network connectivity between the Placement service and the sidecars using the below timeout values. You can adjust these values based on your network conditions as needed.
1. `dapr_placement.keepAliveTime` sets the interval at which the Placement service sends [keep alive](https://grpc.io/docs/guides/keepalive/) pings to daprd on the gRPC stream to check if the connection is still alive. Lower values will lead to shorter actor rebalancing time in case of pod loss/restart, but higher network traffic during normal operation. Accepts values between `1s` and `10s`. Default is `2s`.
2. `dapr_placement.keepAliveTimeout` sets the timeout period for daprd to respond to the Placement service's [keep alive](https://grpc.io/docs/guides/keepalive/) pings before the Placement service closes the connection. Lower values will lead to shorter actor rebalancing time in case of pod loss/restart, but higher network traffic during normal operation. Accepts values between `1s` and `10s`. Default is `3s`.
3. `dapr_placement.disseminateTimeout` sets the timeout period for dissemination to be delayed after actor membership change (usually related to pod restarts) so as to avoid excessive dissemination during multiple pod restarts. Higher values will reduce the frequency of dissemination, but delay the table dissemination. Accepts values between `1s` and `5s`. Default is `2s`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
3. `dapr_placement.disseminateTimeout` sets the timeout period for dissemination to be delayed after actor membership change (usually related to pod restarts) so as to avoid excessive dissemination during multiple pod restarts. Higher values will reduce the frequency of dissemination, but delay the table dissemination. Accepts values between `1s` and `5s`. Default is `2s`.
3. `dapr_placement.disseminateTimeout` sets the timeout period for dissemination to be delayed after actor pod membership change (usually related to pod restarts) so as to avoid excessive dissemination during multiple pod restarts. Higher values will reduce the frequency of dissemination, but delay the table dissemination which can lead to actor timeouts. Accepts values between `1s` and `5s`. Default is `2s`.

@@ -260,6 +260,20 @@ Verify your production-ready deployment includes the following settings:

1. Dapr supports and is enabled to **scope components for certain applications**. This is not a required practice. [Learn more about component scopes]({{< ref "component-scopes.md" >}}).

## Recommended Placement service configuration

The [Placement service]({{< ref "placement.md" >}}) is a critical component in Dapr, responsible for actor placement table dissemination. It's recommended to configure the Placement service with the following values:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The [Placement service]({{< ref "placement.md" >}}) is a critical component in Dapr, responsible for actor placement table dissemination. It's recommended to configure the Placement service with the following values:
The [Placement service]({{< ref "placement.md" >}}) is responsible for actor placement table dissemination when using the actors API. It's recommended to configure the Placement service with the following values:

Copy link
Member

@msfussell msfussell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments

@@ -260,6 +260,20 @@ Verify your production-ready deployment includes the following settings:

1. Dapr supports and is enabled to **scope components for certain applications**. This is not a required practice. [Learn more about component scopes]({{< ref "component-scopes.md" >}}).

## Recommended Placement service configuration

The [Placement service]({{< ref "placement.md" >}}) is a critical component in Dapr, responsible for actor placement table dissemination. It's recommended to configure the Placement service with the following values:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feedback

  1. We should remove the word "critical" which does not add anything to this sentence.
  2. I do not understand the need for this section in the sentence "and thus served by a new pod, or when pod membership of an existing actor instance changes" - what do mean by pod membership? This should simply be that when an actor is updated or a new instance added, this information is replicated to other placement service instances, yes?


The [Placement service]({{< ref "placement.md" >}}) is a critical component in Dapr, responsible for actor placement table dissemination. It's recommended to configure the Placement service with the following values:

1. **High availability**. Ensure that the Placement service is highly available and can survive individual node failures. Helm chart value: `dapr_placement.ha=true`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Placement service is a name so it is Placement service rather than placement service.

Signed-off-by: Elena Kolevska <elena@kolevska.com>
@elena-kolevska
Copy link
Contributor Author

Thanks for the review @msfussell and @alicejgibbons , I addressed your comments in the latest commit.

@yaron2 yaron2 merged commit 69ce373 into dapr:v1.15 Oct 2, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants