Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move caser from a global variable to a local variable #340

Merged
merged 3 commits into from
Dec 17, 2024

Conversation

kojisaikiAtSony
Copy link
Contributor

@kojisaikiAtSony kojisaikiAtSony commented Nov 25, 2024

(This is split change from #339)

Problem

Parallel "Publish" request to a single goaws instance will randomly corrupt DataType values like below:

  • S\u0000ring
  • \u0000\u0000\u0000\u0000\u0000\u0000
  • Sring\u0000
  • S\u0000tingg

Cause

We should keep enclosing Caser instance in a request thread.
https://pkg.go.dev/golang.org/x/text@v0.20.0/cases#Caser

A Caser may be stateful and should therefore not be shared between goroutines.

There is a similar issue with this problem.
golang/go#59520

How reproduce and confirm the fix

We faced this problem with our business application but you can reproduce the same issue with the below code.

var caser = cases.Title(language.AmericanEnglish)

var dataType = []string{"String", "Binary"}

func TestCaser(t *testing.T) {
	for r := 0; r < 100000; r++ {
		var wg sync.WaitGroup
		for i := 0; i < len(dataType); i++ {
			name := dataType[i]
			wg.Add(1)
			go func() {
				fmt.Println(caser.String(name))
				wg.Done()
			}()
		}
		wg.Wait()
	}
}

image

You can confirm the fix in this PR works with this code.

var (
	caser = cases.Title(language.AmericanEnglish)
	mu    sync.Mutex
)

var dataType = []string{"String", "Binary"}

func formatTitle(input string) string {
	mu.Lock()
	defer mu.Unlock()
	return caser.String(input)
}

func TestCaser(t *testing.T) {
	for r := 0; r < 100000; r++ {
		var wg sync.WaitGroup
		for _, input := range dataType {
			wg.Add(1)
			go func(input string) {
				defer wg.Done()
				fmt.Println(formatTitle(input))
			}(input)
		}
		wg.Wait()
	}
}

@kojisaikiAtSony
Copy link
Contributor Author

Hi @Admiral-Piett, we found the root cause of the strange behavior of caser we noted in #339. Could you please review this?

@Admiral-Piett
Copy link
Owner

@kojisaikiAtSony Awesome! Thank you so much for the detailed write up as well. That's SUPER helpful for me to figure out what I'm looking at. If you want to throw your nice mocked out test in there too for bonus, that would be cool, otherwise I'm ready to merge it as is. Up to you, so just let me know!

@kojisaikiAtSony
Copy link
Contributor Author

I added the mocked out test but it's not passed CI yet... please wait 🙇

@kojisaikiAtSony
Copy link
Contributor Author

kojisaikiAtSony commented Dec 13, 2024

Hi @Admiral-Piett , I added the test TestPublishRequest_SetAttributesFromForm_success_concurrent. If the caser instance moved to global again, the test may be failed with assertion error or panic randomly.

@Admiral-Piett Admiral-Piett merged commit 12016a3 into Admiral-Piett:master Dec 17, 2024
2 checks passed
@kojisaikiAtSony kojisaikiAtSony deleted the fixed-caserTitle branch December 18, 2024 04:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants