Skip to content

TCP connection not closing properly leading to dangling CLOSE_WAIT state and system overload #471

@anugrahasinha

Description

@anugrahasinha

Testcase 1

Python 3.10.15 (main, Sep 7 2024, 18:35:33) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from databricks import sql >>> sql.__version__ '3.1.2' >>> with sql.connect(server_hostname = "<HIDE>", http_path = "<HIDE>", access_token = "<HIDE>") as connection: ... with connection.cursor() as cursor: ... cursor.execute("select * from table") ... column_names = [desc[0] for desc in cursor.description] ... query_out = [dict(zip(column_names, row)) for row in cursor.fetchall()] ... ... <databricks.sql.client.Cursor object at 0x7f88551c14e0> >>> query_out [{"sometime" : "something}] On the sidelines, the network connection for this process (after with clause gets finished), remain in ESTABLISHED state and then moves to CLOSE_WAIT state indefinitely. Every 1.0s: netstat -apn | grep 1070099 asinha-vm-ubuntu-20-04-1: Wed Nov 20 02:43:13 2024 tcp 25 0 10.5.220.4:58078 so:me:ip:add:443 CLOSE_WAIT 1070099/python3 

Testcase 2

Python 3.10.15 (main, Sep 7 2024, 18:35:33) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from databricks import sql >>> sql.__version__ '3.1.2' >>> connection = sql.connect(server_hostname = "<HIDE>", http_path = "<HIDE>", access_token = "<HIDE>") >>> ## at this point a TCP connection got made as ESTABLISHED state ## >>> cursor = connection.cursor() >>> cursor.execute("select * from table") <databricks.sql.client.Cursor object at 0x7f9d65db26b0> >>> column_names = [desc[0] for desc in cursor.description] >>> query_out = [dict(zip(column_names, row)) for row in cursor.fetchall()] >>> cursor.close() >>> connection.close() >>> # even after close being called, the connection remains in ESTABLISHED state (for a long time) and then finally moves to CLOSE_WAIT state, which never changes. Every 1.0s: netstat -apn | grep 1074808 asinha-vm-ubuntu-20-04-1: Wed Nov 20 02:49:36 2024 tcp 25 0 10.5.220.4:53810 so:me:ip:add:443 CLOSE_WAIT 1074808/python3 

Connections should be closed properly, and all client resources should be freed properly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions