mirror of https://github.com/astral-sh/uv
Make fetching github releases faster (#11614)
I noticed that the latest two `sync-python-releases` jobs failed due to `httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)`. For the current python-build-standalone release, each request page (defaulting to 30 items per page) takes about 20 seconds and loads around 32MB of data. This extensive data load might be causing the request to frequently fail. In this PR, I reduced number of items per page to 10 and added `Accept-Encoding: gzip, deflate` to the request header. Now, it takes about 6 seconds to load, and the compressed response size has been reduced to 534KB. I hope this would addresses the request failure.
This commit is contained in:
parent
e8712800d1
commit
c57fb4dcef
|
|
@ -238,7 +238,9 @@ class CPythonFinder(Finder):
|
|||
# Collect all available Python downloads
|
||||
for page in range(1, pages + 1):
|
||||
logging.info("Fetching CPython release page %d", page)
|
||||
resp = await self.client.get(self.RELEASE_URL, params={"page": page})
|
||||
resp = await self.client.get(
|
||||
self.RELEASE_URL, params={"page": page, "per_page": 10}
|
||||
)
|
||||
resp.raise_for_status()
|
||||
rows = resp.json()
|
||||
if not rows:
|
||||
|
|
@ -595,7 +597,10 @@ async def find() -> None:
|
|||
"`GITHUB_TOKEN` env var not found, you may hit rate limits for GitHub API requests."
|
||||
)
|
||||
|
||||
headers = {"X-GitHub-Api-Version": "2022-11-28"}
|
||||
headers = {
|
||||
"X-GitHub-Api-Version": "2022-11-28",
|
||||
"Accept-Encoding": "gzip, deflate",
|
||||
}
|
||||
if token:
|
||||
headers["Authorization"] = "Bearer " + token
|
||||
client = httpx.AsyncClient(follow_redirects=True, headers=headers, timeout=15)
|
||||
|
|
|
|||
Loading…
Reference in New Issue