Align request_timeout Behavior in Async and Non-Async APIs

[zzh1996]

In the non-Async API, the requests library is employed. Per the requests library documentation, the timeout accepts either a single float value (representing both connect and read timeouts) or a tuple (specifying connect and read timeouts separately).

Conversely, in the Async API, a single float for request_timeout represents the total timeout (as documented here). When given as a tuple, it corresponds to the connect and total timeouts. This inconsistency exists between the Async and non-Async APIs.

Prior to this pull request, the Async API lacked the ability to set a read timeout as in the non-Async API. This limitation was problematic because:

• A large total timeout value (e.g., 5 minutes) would force users to wait the entire time despite potentially not receiving new messages after the first 5 seconds (a very common issue in the GPT-4 API).
• A small total timeout value (e.g., 15 seconds) might be inadequate for the server to generate a substantial response.

This issue can be remedied by setting the read timeout (i.e., the number of seconds the client waits between bytes received from the server) to a reasonable value like 15 seconds.

[ShantanuNair]

@zzh1996 This would be great.

[ShantanuNair]

@jhallard What do you think about this?

[ShantanuNair]

@zzh1996 Apparently fixed in the v1.0.0 beta: #631 (comment)