FEAT: Complex Datatype support-XML (#293)

### Work Item / Issue Reference <!-- IMPORTANT: Please follow the PR template guidelines below. For mssql-python maintainers: Insert your ADO Work Item ID below (e.g. AB#37452) For external contributors: Insert Github Issue number below (e.g. #149) Only one reference is required - either GitHub issue OR ADO Work Item. --> <!-- mssql-python maintainers: ADO Work Item --> > [AB#38821](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/38821) <!-- External contributors: GitHub Issue --> > GitHub Issue: #<ISSUE_NUMBER> ------------------------------------------------------------------- ### Summary <!-- Insert your summary of changes below. Minimum 10 characters required. --> This pull request adds comprehensive support for the SQL Server `XML` data type to the Python MSSQL driver, ensuring proper handling for insertion, retrieval, batching, and streaming of XML data. It also introduces a suite of tests to verify correct XML behavior, including edge cases like empty, large, and malformed XML values. ### MSSQL Driver Enhancements * Added support for the `SQL_SS_XML` data type throughout the driver, including binding, fetching, and row size calculations, so that XML columns are handled correctly during data operations. [[1]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R24) [[2]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R2529-R2534) [[3]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R2992) [[4]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R3195-R3198) [[5]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R3412) * Updated logic in `FetchMany_wrap` and `FetchAll_wrap` to treat XML columns as LOBs, enabling efficient streaming for large XML values. [[1]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1L3504-R3517) [[2]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1L3626-R3639) ### Test Coverage for XML Support * Added multiple tests in `tests/test_004_cursor.py` to verify XML handling, including basic insert/fetch, empty/null values, large XML streaming, batch inserts, and error handling for malformed XML input. <!-- ### PR Title Guide > For feature requests FEAT: (short-description) > For non-feature requests like test case updates, config updates , dependency updates etc CHORE: (short-description) > For Fix requests FIX: (short-description) > For doc update requests DOC: (short-description) > For Formatting, indentation, or styling update STYLE: (short-description) > For Refactor, without any feature changes REFACTOR: (short-description) > For release related changes, without any feature changes RELEASE: #<RELEASE_VERSION> (short-description) ### Contribution Guidelines External contributors: - Create a GitHub issue first: https://github.com/microsoft/mssql-python/issues/new - Link the GitHub issue in the "GitHub Issue" section above - Follow the PR title format and provide a meaningful summary mssql-python maintainers: - Create an ADO Work Item following internal processes - Link the ADO Work Item in the "ADO Work Item" section above - Follow the PR title format and provide a meaningful summary -->

Author: gargsaumya

mssql_python/pybind/ddbc_bindings.cpp

@@ -21,6 +21,7 @@
 #define SQL_SS_TIMESTAMPOFFSET (-155)
 #define SQL_C_SS_TIMESTAMPOFFSET (0x4001)
 #define MAX_DIGITS_IN_NUMERIC 64
+#define SQL_SS_XML (-152)
 
 #define STRINGIFY_FOR_CASE(x) \
  case x: \
@@ -2525,6 +2526,12 @@ SQLRETURN SQLGetData_wrap(SqlHandlePtr StatementHandle, SQLUSMALLINT colCount, p
 }
  break;
  }
+ case SQL_SS_XML:
+ {
+ LOG("Streaming XML for column {}", i);
+ row.append(FetchLobColumnData(hStmt, i, SQL_C_WCHAR, true, false));
+ break;
+ }
  case SQL_WCHAR:
  case SQL_WVARCHAR:
  case SQL_WLONGVARCHAR: {
@@ -3395,6 +3402,7 @@ size_t calculateRowSize(py::list& columnNames, SQLUSMALLINT numCols) {
  case SQL_LONGVARCHAR:
  rowSize += columnSize;
  break;
+ case SQL_SS_XML:
  case SQL_WCHAR:
  case SQL_WVARCHAR:
  case SQL_WLONGVARCHAR:
@@ -3499,7 +3507,7 @@ SQLRETURN FetchMany_wrap(SqlHandlePtr StatementHandle, py::list& rows, int fetch
 
  if ((dataType == SQL_WVARCHAR || dataType == SQL_WLONGVARCHAR || 
  dataType == SQL_VARCHAR || dataType == SQL_LONGVARCHAR ||
- dataType == SQL_VARBINARY || dataType == SQL_LONGVARBINARY) &&
+ dataType == SQL_VARBINARY || dataType == SQL_LONGVARBINARY || dataType == SQL_SS_XML) &&
  (columnSize == 0 || columnSize == SQL_NO_TOTAL || columnSize > SQL_MAX_LOB_SIZE)) {
  lobColumns.push_back(i + 1); // 1-based
  }
@@ -3621,7 +3629,7 @@ SQLRETURN FetchAll_wrap(SqlHandlePtr StatementHandle, py::list& rows) {
 
  if ((dataType == SQL_WVARCHAR || dataType == SQL_WLONGVARCHAR || 
  dataType == SQL_VARCHAR || dataType == SQL_LONGVARCHAR ||
- dataType == SQL_VARBINARY || dataType == SQL_LONGVARBINARY) &&
+ dataType == SQL_VARBINARY || dataType == SQL_LONGVARBINARY || dataType == SQL_SS_XML) &&
  (columnSize == 0 || columnSize == SQL_NO_TOTAL || columnSize > SQL_MAX_LOB_SIZE)) {
  lobColumns.push_back(i + 1); // 1-based
  }

tests/test_004_cursor.py

@@ -11409,6 +11409,90 @@ def test_datetime_string_parameter_binding(cursor, db_connection):
  drop_table_if_exists(cursor, table_name)
  db_connection.commit()
 
+SMALL_XML = "<root><item>1</item></root>"
+LARGE_XML = "<root>" + "".join(f"<item>{i}</item>" for i in range(10000)) + "</root>"
+EMPTY_XML = ""
+INVALID_XML = "<root><item></root>" # malformed
+
+def test_xml_basic_insert_fetch(cursor, db_connection):
+ """Test insert and fetch of a small XML value."""
+ try:
+ cursor.execute("CREATE TABLE #pytest_xml_basic (id INT PRIMARY KEY IDENTITY(1,1), xml_col XML NULL);")
+ db_connection.commit()
+
+ cursor.execute("INSERT INTO #pytest_xml_basic (xml_col) VALUES (?);", SMALL_XML)
+ db_connection.commit()
+
+ row = cursor.execute("SELECT xml_col FROM #pytest_xml_basic;").fetchone()
+ assert row[0] == SMALL_XML
+ finally:
+ cursor.execute("DROP TABLE IF EXISTS #pytest_xml_basic;")
+ db_connection.commit()
+
+
+def test_xml_empty_and_null(cursor, db_connection):
+ """Test insert and fetch of empty XML and NULL values."""
+ try:
+ cursor.execute("CREATE TABLE #pytest_xml_empty_null (id INT PRIMARY KEY IDENTITY(1,1), xml_col XML NULL);")
+ db_connection.commit()
+
+ cursor.execute("INSERT INTO #pytest_xml_empty_null (xml_col) VALUES (?);", EMPTY_XML)
+ cursor.execute("INSERT INTO #pytest_xml_empty_null (xml_col) VALUES (?);", None)
+ db_connection.commit()
+
+ rows = [r[0] for r in cursor.execute("SELECT xml_col FROM #pytest_xml_empty_null ORDER BY id;").fetchall()]
+ assert rows[0] == EMPTY_XML
+ assert rows[1] is None
+ finally:
+ cursor.execute("DROP TABLE IF EXISTS #pytest_xml_empty_null;")
+ db_connection.commit()
+
+
+def test_xml_large_insert(cursor, db_connection):
+ """Test insert and fetch of a large XML value to verify streaming/DAE."""
+ try:
+ cursor.execute("CREATE TABLE #pytest_xml_large (id INT PRIMARY KEY IDENTITY(1,1), xml_col XML NULL);")
+ db_connection.commit()
+
+ cursor.execute("INSERT INTO #pytest_xml_large (xml_col) VALUES (?);", LARGE_XML)
+ db_connection.commit()
+
+ row = cursor.execute("SELECT xml_col FROM #pytest_xml_large;").fetchone()
+ assert row[0] == LARGE_XML
+ finally:
+ cursor.execute("DROP TABLE IF EXISTS #pytest_xml_large;")
+ db_connection.commit()
+
+
+def test_xml_batch_insert(cursor, db_connection):
+ """Test batch insert (executemany) of multiple XML values."""
+ try:
+ cursor.execute("CREATE TABLE #pytest_xml_batch (id INT PRIMARY KEY IDENTITY(1,1), xml_col XML NULL);")
+ db_connection.commit()
+
+ xmls = [f"<root><item>{i}</item></root>" for i in range(5)]
+ cursor.executemany("INSERT INTO #pytest_xml_batch (xml_col) VALUES (?);", [(x,) for x in xmls])
+ db_connection.commit()
+
+ rows = [r[0] for r in cursor.execute("SELECT xml_col FROM #pytest_xml_batch ORDER BY id;").fetchall()]
+ assert rows == xmls
+ finally:
+ cursor.execute("DROP TABLE IF EXISTS #pytest_xml_batch;")
+ db_connection.commit()
+
+
+def test_xml_malformed_input(cursor, db_connection):
+ """Verify driver raises error for invalid XML input."""
+ try:
+ cursor.execute("CREATE TABLE #pytest_xml_invalid (id INT PRIMARY KEY IDENTITY(1,1), xml_col XML NULL);")
+ db_connection.commit()
+
+ with pytest.raises(Exception):
+ cursor.execute("INSERT INTO #pytest_xml_invalid (xml_col) VALUES (?);", INVALID_XML)
+ finally:
+ cursor.execute("DROP TABLE IF EXISTS #pytest_xml_invalid;")
+ db_connection.commit()
+
 def test_close(db_connection):
  """Test closing the cursor"""
  try: