Skip to content

Conversation

CalebCourier
Copy link
Collaborator

This PR addresses #1163

To avoid orphaned spans being spawned from the Validator.validate calls executed in the ThreadPoolExecutor, we check if there is a current span and do not wrap the validate call in a new span if there isn't.
In conjunction with this, we now also wrap Validator.async_validate this does not experience the same issue as listed above because it itself is not run in the executor, rather it is the one running the executor.

This is not a long term solution for the root cause, but it does avoid the behaviour we see in the issue.

Other potential issues from the ThreadPoolExecutor are any validators that attempt to retrieve information from the ContextStore as it may not have the context that existed in the main thread. This would need to be tested and verified.

@CalebCourier CalebCourier requested a review from a team November 14, 2024 17:49
@dtam dtam merged commit 93a6a36 into main Nov 14, 2024
12 checks passed
@dtam dtam deleted the temp-fix-async-mlflow branch November 14, 2024 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants