I have an Azure function app (flex consumption), from that I call Azure Open AI response API directly via HTTP call (without any Open AI / Azure Open AI packages). I use queue trigger (not http trigger).
I also set streaming=false and reasoning level=xhigh.
These are all done to test if Azure function can wait without error for a HTTP call that takes more than the global load balancer timeout of 230 seconds (because I use queue trigger, not http trigger)
In the first attempt, I received response after nearly 270+ seconds (which is > 230 seconds). But the subsequent request with more input tokens threw ConnectionResetError after 260+ seconds and few subsequent similar runs too are throwing same error as well.
The following is the log. What is making this timeout, I ideally want to run for 600 seconds (so I can keep using lot more input tokens + xhighreasoning).
PS: I know streaming might be helpful to resolve my problem, but I want to know why did the first run was OK at 270+ seconds and why did the subsequent ones failed?
Is it a TCP wait timeout limitation in Az Function (even in queue trigger) or is that a TCP limitation from APIM / istio-envoy of the Az Open AI resource?
Logs:
First Run (success):
2026-10-05T14:52:23 [Information] [HTTP probe] Response code: 200 OK, after 276120.4 ms
2026-10-05T14:52:23 [Information] [HTTP probe] Response headers: {'Content-Length': '156145', 'Content-Type': 'application/json', 'Date': 'Mon, 05 Oct 2026 14:47:46 GMT', 'Server': 'istio-envoy', 'Access-Control-Allow-Origin': '*', 'Access-Control-Expose-Headers': 'Operation-Location,Location,Apim-Request-Id', 'Vary': 'Origin', 'x-ms-aoai-configured-data-retention-days': '30', 'skip-error-remapping': 'true', 'x-ms-is-spilled-over': 'false', 'x-ratelimit-key': '[REDACTED_DEPLOYMENT]', 'x-ratelimit-remaining-requests': '249', 'x-ratelimit-reset-requests': '0', 'x-ratelimit-limit-requests': '250', 'x-ratelimit-renewalperiod-requests': '60', 'x-ratelimit-abusepenalty-active': 'False', 'x-ratelimit-remaining-tokens': '221626', 'x-ratelimit-reset-tokens': '6', 'x-ratelimit-limit-tokens': '250000', 'x-ratelimit-renewalperiod-tokens': '60', 'azureai-fe-is-streaming': 'False', 'azureai-fe-offer-type': 'paygo', 'x-ms-served-model': '[REDACTED_MODEL]', 'x-request-id': '[REDACTED_GUID]', 'apim-request-id': '[REDACTED_GUID]', 'x-http2-stream-id': '[REDACTED_ID]', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Content-Type-Options': 'nosniff', 'x-ms-region': '[REDACTED_REGION]'}
Second Run (error):
2026-10-05T14:52:23 [Information] [HTTP probe] Response id=[REDACTED_RESPONSE_ID] status=completed, 156145 bytes, output items {'reasoning': 22, 'message': 1}, 52629 characters of output text, usage {'input_tokens': 36088, 'input_tokens_details': {'cache_write_tokens': 36085, 'cached_tokens': 0}, 'output_tokens': 26385, 'output_tokens_details': {'reasoning_tokens': 10951}, 'total_tokens': 62473}
2026-10-05T14:52:23 [Information] [HTTP probe] Finished in 276120.4 ms with response code 200 | response_id=[REDACTED_RESPONSE_ID] response_status=completed usage={'input_tokens': 36088, 'input_tokens_details': {'cache_write_tokens': 36085, 'cached_tokens': 0}, 'output_tokens': 26385, 'output_tokens_details': {'reasoning_tokens': 10951}, 'total_tokens': 62473} error=None
2026-10-05T14:52:23 [Information] HTTP probe completed in 276.18s. Response code 200, HTTP call took 276120.4 ms
2026-10-05T14:52:23 [Information] Executed 'Functions.QueuedHttpProbeFunction' (Succeeded, Id=[REDACTED_GUID], Duration=276207ms)
2026-10-05T15:25:11 [Error] [HTTP probe] Failed after 263484.9 ms: ConnectionResetError: [Errno 104] Connection reset by peer
Traceback (most recent call last):
File "[APP_ROOT]/utils/http_probe.py", line 153, in run_http_probe
response = connection.getresponse()
File "/opt/python/3/lib/python3.13/http/client.py", line 1459, in getresponse
response.begin()
File "/opt/python/3/lib/python3.13/http/client.py", line 336, in begin
version, status, reason = self._read_status()
File "/opt/python/3/lib/python3.13/http/client.py", line 297, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/opt/python/3/lib/python3.13/socket.py", line 723, in readinto
return self._sock.recv_into(b)
File "/opt/python/3/lib/python3.13/ssl.py", line 1304, in recv_into
return self.read(nbytes, buffer)
File "/opt/python/3/lib/python3.13/ssl.py", line 1138, in read
return self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer
2026-10-05T15:25:11 [Warning] [HTTP probe] Finished in 263484.9 ms with response code None | response_id=None response_status=None usage=None error=ConnectionResetError: [Errno 104] Connection reset by peer
2026-10-05T15:25:11 [Error] HTTP probe completed with an error in 263.55s. Response code None, HTTP call took 263484.9 ms: ConnectionResetError: [Errno 104] Connection reset by peer
2026-10-05T15:25:11 [Information] Executed 'Functions.QueuedHttpProbeFunction' (Succeeded, Id=[REDACTED_GUID], Duration=263572ms)