Repository navigation
load balancing: support creating additional connections when MAX_CONCURRENT_STREAM is reached #21386
Description
What version of gRPC and what language are you using?
1.24.3
What operating system (Linux, Windows,...) and version?
Linux - Ubuntu 14.04 (GCE instance)
What runtime / compiler are you using (e.g. python version or version of gcc)
Go - 1.13
What did you do?
If possible, provide a recipe for reproducing the error. Try being specific and include code snippets if helpful.
- Connect to remotebuildexecution endpoint (remotebuildexecution.googleapis.com:443) using github.com/bazelbuild/remote-apis-sdks (https://github.com/bazelbuild/remote-apis-sdks/blob/master/go/pkg/client/client.go#L185)
- Make ~500 concurrent requests to Execute() streaming API with "sleep 45" command using a single connection (as seen in the code for Dial() linked in (1) ). API proto - https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2/remote_execution.proto#L106 .
What did you expect to see?
All the 500 rpc's execute in parallel
What did you see instead?
Only 100 rpc's run concurrently, the remaining 400 block until the first 100 finishes running.
Anything else we should know about your project / environment?
- We previously reported a similar issue for the Java client at Use multiple connections to avoid the server's SETTINGS_MAX_CONCURRENT_STREAMS limit #11704 - the work around suggested does not work for us now since the DNS name resolution returns IPv6 addresses, which don't resolve on GCE instances (which is where our client would run). We get a
Immediate connect fail for 2a00:1450:400c:c06::93: Network is unreachableerror when we try to connect to IPv6 addresses. - The grpc-go client has a similar issue reported in its repository - Control MAX_CONCURRENT_STREAMS server-side and account for it on client-side grpc-go#2412. It also has a comment about the work involved in the Go client for the feature.
- Ideally we would like to avoid having to manage multiple connections (connection pool) in our client.
Also this issue is blocking our Go client (which we are writing for our API) from achieving high parallelism, which makes the client less useful to our customers.
Reacted by Martin Schröder, Nontawat Numor, Riccardo, Sam Batschelet, Kaijie Chen, fatedier, Ranjith KA, Patrick Freed, zatekusen, Tadeusz and 22 more
Activity
gkousik commentedon Dec 11, 2019
FYI @ola-rozenfeld @buchgr - In relation to #11704, I suppose if you run Bazel on GCE (where it doesn't connect to IPv6 addresses), you will also remote execution parallelism being constrained to a around ~200 concurrent requests.
dfawley commentedon Dec 11, 2019
This would have to be a feature provided by all languages, not just Go.
cc @ejona @markdroth
[-]Grpc-go client - support creating additional connections when MAX_CONCURRENT_STREAM is reached[/-][+]load balancing: support creating additional connections when MAX_CONCURRENT_STREAM is reached[/+]12 remaining items
ejona86 commentedon Nov 3, 2022
gRFC without implementation resources doesn't seem all that helpful. Maybe just externalize the internal doc (copy the contents to your personal account and share it)? That lets people determine if the solution would help them, gives them an idea of the work involved, and would be a guide for any would-be contributors that want to step up.
markdroth commentedon Nov 3, 2022
Didn't we say that the internal design we had been evaluating wasn't going to work in the xDS case? If so, I don't think we'd actually want to pursue that anyway.
patrickfreed commentedon Nov 3, 2022
Either one would be super helpful and much appreciated.
I think this feature is still very useful for the L4 load balancer case (#7957).
markdroth commentedon Nov 3, 2022
I do understand that, but we don't want to have to support two mechanisms for the same thing. Ultimately, we will need a mechanism that solves this problem for the xDS case, so if we introduce this mechanism now for the L4 case and then need to introduce a separate mechanism later for the xDS case, then we're stuck supporting two mechanisms for the same thing. I would prefer not to introduce a mechanism for this until we know that it will solve all the use cases that we care about.
patrickfreed commentedon Dec 8, 2022
I think it would still be valuable if the internal doc were released publicly, since it's possible the community could help brainstorm ideas for a solution that accommodates the xDS case too.
bogdan-patraucean commentedon Feb 12, 2024
Any updates on this?
vincentyl commentedon Feb 26, 2024
we're seeing the same issue in python as well
bgdnvk commentedon Apr 8, 2024
bump for visibility - really interested in this and found this issue from the docs page
grpc performance note:
tan-zhuo commentedon Jun 27, 2024
How's it going, buddy? Can this be resolved?
markdroth commentedon Oct 27, 2025
I've published a gRFC proposing connection scaling to handle MAX_CONCURRENT_STREAMS limits:
grpc/proposal#516
patsevanton commentedon Jan 26, 2026
Any updates?
markdroth commentedon Jan 26, 2026
We've reached agreement on the design. I'm waiting for one more review before I merge the gRFC, but I don't expect any significant changes before it gets merged.
I'm about to merge an implementation for C-core, which you'll be able to start testing it in the 1.79 release. The feature will still be guarded by an environment variable in that release, since we need it to pass interop tests before we remove that guard, and that will come in a later version. The other languages will follow, although there's no specific timeline identified yet.
clientv3: add context-keyed channel pool
markdroth commentedon Aug 18, 2026
This is now fully implemented. It will be enabled by default in the upcoming 1.84 release.