Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
I looked a little bit into why the `fastscan` package was slower than I expected. And it's because I accidentally left stuff in the lexer that is totally unused but is the primary source of allocations and memory usage (and thus also bad for latency since allocations aren't free). Once upon a time, in an older version of the lexer in protoparse (on which this fastscan lexer is based), this stuff was used to capture the actual raw text for a token. The new lexer in protocompile uses a completely different approach, to reduce the memory usage. But it's completely unused here. Removing it basically doubles the throughput of `fastscan` and causes it to use 1/3rd as many allocations and 1/4th as much memory. ``` -- before -- BenchmarkGoogleapisFastScan-10 5 210397558 ns/op 468123099 B/op 11217259 allocs/op BenchmarkGoogleapisFastScan-10 5 202723933 ns/op 468156548 B/op 11217275 allocs/op -- after -- BenchmarkGoogleapisFastScan-10 12 93760795 ns/op 111257610 B/op 3710000 allocs/op BenchmarkGoogleapisFastScan-10 12 94548743 ns/op 111259758 B/op 3710010 allocs/op ```
- Loading branch information