Python

Understanding LiteralString in Python 3.11

Image generated by Jacob Ferus

The LiteralString is a new type in Python 3.11 that can be used as a type annotation or type hint. In essence, it tells you that a variable should be hardcoded.

Hardcoding refers to the insertion of data directly into code, rather than obtaining them dynamically from a process or user input. An example of a hardcoded variable would be the following:

MY_HARDCODED_VARIABLE = "hello world"

While this would not:

value = get_last_user_input()

But does it matter if a variable is hardcoded or not? In one situation it can make a significant difference: security. Using user-inputted strings in the same way as you would a string hardcoded by developers can lead to security vulnerabilities and malicious attacks. A good example of this is SQL injections, and here is one place where the LiteralString comes in handy.

Let’s illustrate this in psycopg, a Python database adapter for PostgreSQL. In this library, you can securely manage user input into SQL code while avoiding SQL injection and vulnerabilities. Still, if you don’t use the library correctly, you can leave yourself open to attacks. Let's see an example.

Note: in the examples, input() is used for demonstration purposes, in a real-world scenario, the vulnerable input would typically come from a remote source, e.g. input to an API.

from psycopg import connect

psql_connection_string = "..."

def get_query() -> str:
 return input()

if __name__ == '__main__':
 query = get_query()
 with connect(
 psql_connection_string
 ) as conn:
 with conn.cursor() as cur:
 cur.execute(query)

In the code, a connection is made to the database and then a query is executed using the cursor object. This query is entirely constructed from user input, and hence, clearly not safe.

Previously, no indication of danger would appear in this situation, but with the LiteralString and a type checker, it will! Do note though that the code will still run, thus it is important to check the warnings.

In recent versions of psycopg the LiteralString has been implemented. For this reason, the following appears in my IDE (VS Code with Pylance):

As can be seen, this security flaw is captured because the library has used the type hint of LiteralString instead of just str:

This way, in addition to the documentation, we can be notified when are using the library incorrectly. Let’s see another example.

In psycopg there is a SQL-object that can be used to combine various SQL- expressions in a safe manner. For instance, to insert a table name into an SQL expression you would do this:

from psycopg import sql

table_name = input()
sql_var_table = sql.SQL("SELECT * FROM {table_name}")
sql_expr = sql_var_table.format(table_name=sql.Identifier(table_name))

Now, despite the table coming from user input, it is managed in a way that the user cannot, for instance, delete rows from a table by injecting malicious code.

Still, if you’re sloppy, you might type this:

sql_expr = sql.SQL("SELECT * FROM {table_name}".format(table_name=table_name))

Here the string is first created using the user input and then the SQL-object is created with the user-input-affected string, causing a vulnerability. Once again though, if you have a type checking program running, this will be detected as this is not a LiteralString:

Do note that combining LiteralStrings, e.g., a + b still makes it a LiteralString.

It’s nice when libraries are using the LiteralString to make it easier for you to avoid mistakes, but you could also improve the security of your own projects and code with it. For instance, any time you are using hardcoded values that you do not want any user to touch, using the type hint of LiteralString will avoid possible mistakes in the future. Take an example of reading files from a folder:

from typing import LiteralString
import os

FILE_PATH = "./app/"

def load_file(file: LiteralString) -> str:
 path = os.path.join(FILE_PATH, file)
 with open(path, "r") as f:
 return f.read()

# good, filename is hardcoded
print(load_file("file.txt"))

# DANGER: user input is used
# e.g. user_file_input = "/../../path/to/secret_info.txt"
user_file_input = input()

load_file(user_file_input)

If this method is intended to be used by the developer only, then specifying the filename type as a LiteralString will enable the detection of problems by the IDE.

• Follow me on 𝕏: @jacobferus 🚀
• Check out Viralwalk — Awesome Websites from all over the internet 🙏

You may also enjoy:

Creating Financial Dashboards in Python with Solara