BUG: read_sql wraps plain SQL strings in sqlalchemy.text() to allow % modulo and LIKE patterns (#35484) #62578
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Member
| @SergioGarcia00 Can you update the description linking to the relevant issue? |
Author
Oh, missed that sry. #35484 |
Author
| I dont really understand the errors I got by the CI, is there something I did wrong? |
Added type hints for function signatures and improved code clarity.
Updated the _sa_text_if_string function to include type hints and enhanced docstring for clarity. Modified the read_sql function to use the updated _sa_text_if_string function.
Refactor SQL tests to use SQLAlchemy select statements and remove unused imports.
Author
| I really dont get it, |
Reorder imports and clean up comments in test file.
Author
| pre-commit.ci autofix |
Author
| It looks like the failing job is due to an artifact/cache conflict (Failed to CreateArtifact: 409 Conflict), not the code itself. Could someone with permissions re-run the “Without PyArrow” job? |
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
The issue is coming from #35484
What does this PR do?
Fixes a long-standing issue where pd.read_sql fails to handle the percent character (%) when using SQLAlchemy engines.
Since pandas ≥ 1.1, the IO layer sets no_parameters=True for SQL execution, which causes plain string queries containing % to be misinterpreted as parameter placeholders.
As a result:
Queries like SELECT 1 % 2 raised UndefinedFunction errors.
Queries using LIKE 'Jo%' failed to match rows correctly.
This PR introduces a small helper _sa_text_if_string that wraps plain SQL strings in sqlalchemy.text() before execution.
This ensures that % is treated correctly, while SQLAlchemy select() objects continue to work as before.
Summary of changes
pandas/io/sql.py
Added _sa_text_if_string(stmt) helper.
Applied it inside SQLDatabase.read_query() before self.execute().
pandas/tests/io/sql/test_percent_patterns.py
Added new tests for:
SELECT 5 % 2 (modulo operator)
LIKE 'John%' (pattern matching)
SQLAlchemy selectable using (literal(7) % literal(3))
Other
Restored .gitignore to match upstream so only relevant files are modified.
How was this tested?
All pre-commit checks (ruff, codespell, etc.) pass locally.
pytest -q pandas/tests/io/sql/test_percent_patterns.py -ra → 3 tests passed.
Tested both on in-memory SQLite and PostgreSQL (PANDAS_TEST_POSTGRES_URI).
Notes for reviewers
The change only affects plain string queries.
SQLAlchemy expressions (select(), text(), etc.) remain unaffected.
A short “Bug Fixes” note will be added to doc/source/whatsnew/v3.x.y.rst.