[Tokenizer] Fix decode output with space in decode_token

[DrownFish19]

PR types

Bug fixes

PR changes

Others

Description

Fix decode output with space in decode_token.
For example, new_text = " 123"(space + 123), and prefix_text = " "(space),the clean_up_tokenization_spaces will make new_text to "123".

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 53.96%. Comparing base (56d293d) to head (0cd1c52).
Report is 316 commits behind head on develop.

Additional details and impacted files

@@ Coverage Diff @@ ## develop #9010 +/- ## =========================================== - Coverage 54.08% 53.96% -0.13%  =========================================== Files 650 652 +2 Lines 103915 104929 +1014 =========================================== + Hits 56200 56621 +421  - Misses 47715 48308 +593 

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ZHUI]

LGTM