Add robust redirect handling and retry logic

[recursivecodes]

Description

• Implement manual redirect handling with Location header parsing
• Add retry mechanism with exponential backoff for failed requests
• Increase timeout to 30s for slow WHIP servers
• Add comprehensive logging for debugging connection issues
• Handle both absolute and relative redirect URLs
• Prevent infinite redirect loops with configurable limits

Some WHIP servers will redirect for load balancing purposes. This fix will properly follow the redirects which must be manual in order to retain auth headers.

Related

No related issues.

Testing

Tested fix to publish from ESP32P4 to Amazon IVS stages. Confirmed working.

---

Checklist

Before submitting a Pull Request, please ensure the following:

• 🚨 This PR does not introduce breaking changes.
• All CI checks (GH Actions) pass.
• Documentation is updated as needed.
• Tests are updated or added as necessary.
• Code is well-commented, especially in complex areas.
• Git history is clean — commits are squashed to the minimum necessary.

[recursivecodes]

Any thoughts on this @TempoTian? This is crucial for Amazon IVS WebRTC support, since the WHIP requests are redirected multiple times and the auth header needs to be retained.

[recursivecodes]

Hello @TempoTian I just wanted to check on this PR. Do you have thoughts?

[TempoTian]

Thanks, I think the PR is quite good for http redirection, http_client have also implement redirection logic maybe reuse it is better.
I see some sample code like, I think it can fit your request also. For extract buffer or timeout control maybe provide an open_extra api with more detail setting better. How do you think about it?

 int post_len = esp_http_client_get_post_field(http->client, &buffer); _stream_redirect: if ((err = esp_http_client_open(http->client, post_len)) != ESP_GMF_ERR_OK) { ESP_LOGE(TAG, "Failed to open http stream"); return err; } if (post_len && buffer && wrlen == 0) { if (esp_http_client_write(http->client, buffer, post_len) <= 0) { ESP_LOGE(TAG, "Failed to write data to http stream"); return ESP_GMF_ERR_FAIL; } } int status_code = esp_http_client_get_status_code(http->client); if (status_code == 301 || status_code == 302) { esp_http_client_set_redirection(http->client); goto _stream_redirect; } 

[recursivecodes]

The problem with the built-in support for redirection is that it drops the auth header, which is why manual redirection is required (to add the auth header back to the request). Reusing the client should work I think.

[TempoTian]

Yes you are right, some re-direction also have cookie header and need carried when redirection also.
Generally browser resend with following header, I think resend cookie + auth (origin same) maybe enough for most users:

• Carries cookies (per cookie rules).
• Keeps Authorization only if same origin.
• Keeps safe request headers (UA, Accept, etc.).
• Updates or drops sensitive headers depending on origin and policy.
• May change method depending on status code.

[recursivecodes]

Do you need any updates from me, or will you work on merging?

[TempoTian]

I want to rewrite it use original http client code, after finished if possible could you please help to test the results. I will update once finished the code.

[TempoTian]

I have commited the code to 86d0c8f, you can check whether it fixed all your mentioned issue. If need any improvement, please let me know.

[recursivecodes]

@TempoTian i can confirm that the latest version has resolved all of my outstanding issues with SDP requests! thank you for this great fix!

[TempoTian]

@recursivecodes Thanks for your info.

[recursivecodes]

Thanks again, @TempoTian. I have published a blog post and demo repo on how to use this with Amazon IVS.

Blog post:

https://dev.to/recursivecodes/live-streaming-to-amazon-ivs-on-an-esp32-microcontroller-with-embedded-sensor-metadata-17mj

Repo:

https://github.com/recursivecodes/ivs-esp-whip-demo.git

[TempoTian]

Great! The demo looks amazing.
PS: Regarding the color issue, the ISP team is still working on it. Once the fix is released, you’ll be able to get better image quality simply by upgrading esp-video.

[recursivecodes]

Thanks @TempoTian - is there a ticket I can watch for status updates on the color issue?

Edit: I think I found the issue.