Download does not start immediately, a lot of network traffic (file is downloaded twice?) #18213
Comments
|
@marcel-idana Do you have a proxy/VPN? Does |
|
Thanks for your quick response! Weeks ago I tried to download files with I have two user accounts on my machine and changed user permissions with It also happen with the casks
Also the non cask formulae I also reinstalled Homebrew with no result. I still believe that my OS is in some odd state and i would be sorry if i wasted your time ✌🏼 |
|
I think this happens when upstream has no brew/Library/Homebrew/utils/curl.rb Line 232 in fafa618 Thus, essentially a double download. For example, in top 5 casks:
So far, only Microsoft and Google software. Anyway, should try to at least cache the GET to get ~2x speed up (maybe slightly less if any non- |
|
I also have same issue for some specific package, like 
|
|
I also encountered a similar issue, through network monitoring, there is download traffic, but the progress bar takes a very long time to appear, and the progress bar starts from 0. |
|
Taking another look at issue, the behavior may actually be a bug in macOS Specifically when using Using iTerm2 as an example: ❯ hyperfine '/usr/bin/curl --head --request GET https://iterm2.com/downloads/stable/iTerm2-3_5_5.zip' '/opt/homebrew/opt/curl/bin/curl --head --request GET https://iterm2.com/downloads/stable/iTerm2-3_5_5.zip'
Benchmark 1: /usr/bin/curl --head --request GET https://iterm2.com/downloads/stable/iTerm2-3_5_5.zip
Time (mean ± σ): 2.043 s ± 0.283 s [User: 0.198 s, System: 0.054 s]
Range (min … max): 1.718 s … 2.546 s 10 runs
Benchmark 2: /opt/homebrew/opt/curl/bin/curl --head --request GET https://iterm2.com/downloads/stable/iTerm2-3_5_5.zip
Time (mean ± σ): 97.7 ms ± 36.4 ms [User: 14.3 ms, System: 5.6 ms]
Range (min … max): 81.1 ms … 279.0 ms 28 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
/opt/homebrew/opt/curl/bin/curl --head --request GET https://iterm2.com/downloads/stable/iTerm2-3_5_5.zip ran
20.92 ± 8.31 times faster than /usr/bin/curl --head --request GET https://iterm2.com/downloads/stable/iTerm2-3_5_5.zip |
|
Just want to chime in and confirm that I, too, can see in my web server logs that
@cho-m In my logs I see it happening with multiple curl versions as far back as 7.64.1 on Catalina. |
Thanks for the information. I only tested 8.6.0 thru 8.10.1. The ones that I saw behavior were However, May need either @reitermarkus or @samford for context on original fallback scenario - #15117 I was experimenting with skipping fallback in #18508 on the download code path. Livecheck code path still needs it but it has a chance of timing out as it slows down the operation, e.g. ❯ hyperfine 'brew livecheck catlight' 'HOMEBREW_FORCE_BREWED_CURL=1 brew livecheck catlight'
Benchmark 1: brew livecheck catlight
Time (mean ± σ): 10.250 s ± 0.339 s [User: 2.270 s, System: 0.608 s]
Range (min … max): 9.622 s … 10.648 s 10 runs
Benchmark 2: HOMEBREW_FORCE_BREWED_CURL=1 brew livecheck catlight
Time (mean ± σ): 2.085 s ± 0.304 s [User: 0.658 s, System: 0.185 s]
Range (min … max): 1.838 s … 2.803 s 10 runs
Summary
HOMEBREW_FORCE_BREWED_CURL=1 brew livecheck catlight ran
4.92 ± 0.74 times faster than brew livecheck catlight |
The `curl --head --request GET` causes a full download to happen on `curl` prior to 8.10.0[^1] (ignoring `curl` 8.6.0 which returned a "Weird server reply" due to a bug[^2]) which causes poor UX due to slow Cask downloads that can take almost twice as long as it should. [^1]: #18213 [^2]: https://curl.se/mail/lib-2024-02/0000.html
|
The 8.10.0 fix was curl/curl@29610e5 |
Noting that 3 requests are expected, i.e.
In theory, the first 2 should be quick as only getting headers. We've confirmed no issue with curl 8.4.0 (#18508 (comment)) and I've tested 8.5.0 with no issue, so the impacted |
Is the point of the HEAD request to get information on the payload presence, size, etc.? If so, how about adding In my experience practically every httpd out there supports the Range/Byte-Range header these days. |
The `curl --head --request GET` causes a full download to happen on `curl` from 8.7.0 to 8.9.1[^1] which causes poor UX due to slow Cask downloads that can take almost twice as long as they should. [^1]: #18213
|
The requests are for Sadly without exact examples, it is hard to tell what works/breaks. For now, experimenting with downgrade to |
The `curl --head --request GET` causes a full download to happen on `curl` from 8.7.0 to 8.9.1[^1] which causes poor UX due to slow Cask downloads that can take almost twice as long as they should. [^1]: #18213
|
Workaround has been merged. If on stable |
brew doctoroutputVerification
brew doctoroutput" above saysYour system is ready to brew.and am still able to reproduce my issue.brew updatetwice and am still able to reproduce my issue.brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.brew configoutputWhat were you trying to do (and why)?
Install any homebrew cask. For example
rsudio:This occurred also with
appcleaner,webstorm,golandand more casks.What happened (include all command output)?
Installing a cask takes a lot of time, the download bar is not shown immediately and the installation causes a lot of network traffic. Maybe issue 10528 is related? Also maybe I have some odd OS state that triggers this.
In the shown output below a long pause with network activity occurs.
### LONG PAUSE WITH NETWORK ACTIVITY ###is not part of the output was inserted by me.Full unmodified output:
What did you expect to happen?
Cask installation takes less time and the download bar is shown immediately.
Step-by-step reproduction instructions (by running
brewcommands)The text was updated successfully, but these errors were encountered: