- Notifications
You must be signed in to change notification settings - Fork 2
fix: prevent orphaned Power Query connections during worksheet loading #299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Root Cause: ListObjects.Add() with connection STRING creates orphaned connections with generic names (Connection, Connection1, etc.) Solution: - Use Connections.Add2() to create properly named connections first - Pass connection OBJECT to ListObjects.Add() to reuse the connection - For LoadToBoth: create TWO connections with distinct names: - Worksheet: 'Query - {name}' - Data Model: 'Query - {name} (Data Model)' - Updated pattern matching in Delete/Unload/List/GetLoadConfig Test Coverage: - 14 new worksheet cleanup tests - 9 lifecycle cleanup tests - All 74+ Power Query tests passing Fixes #298 Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
| public void Delete_OrphanedPowerQueryConnection_GenericName_Succeeds() | ||
| { | ||
| // Arrange - Use the test file that has orphaned connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); |
Check notice
Code scanning / CodeQL
Call to System.IO.Path.Combine Note test
Show autofix suggestion Hide autofix suggestion
Copilot Autofix
AI 6 days ago
The best fix for this issue is to replace Path.Combine with Path.Join on line 216. This makes path joining safer by ensuring none of the earlier arguments are dropped if any later ones are absolute paths (which could accidentally happen in future edits). Functionally, Path.Join does not behave differently from Path.Combine for all relative segments, so the change preserves behavior. The only code to change is the one line constructing the sourceFile path on line 216. This file is using Path without an explicit import, so we need to ensure a using directive for System.IO is available, but it’s likely already provided in the unseen snippet; if not, add it at the top.
-
Copy modified line R216
| @@ -213,7 +213,7 @@ | ||
| public void Delete_OrphanedPowerQueryConnection_GenericName_Succeeds() | ||
| { | ||
| // Arrange - Use the test file that has orphaned connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| var sourceFile = Path.Join(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| | ||
| if (!IOFile.Exists(sourceFile)) | ||
| { |
| return; | ||
| } | ||
| | ||
| var testFile = Path.Combine(_fixture.TempDir, $"OrphanedPQ_{Guid.NewGuid():N}.xlsx"); |
Check notice
Code scanning / CodeQL
Call to System.IO.Path.Combine Note test
Show autofix suggestion Hide autofix suggestion
Copilot Autofix
AI 6 days ago
The best way to fix this problem is to replace the call to Path.Combine with Path.Join. Path.Join always joins the arguments with path separators, and does not treat absolute paths specially, avoiding the risk present in Path.Combine. Specifically, change line 224 in tests/ExcelMcp.Core.Tests/Integration/Commands/Connection/ConnectionCommandsTests.Delete.cs from:
var testFile = Path.Combine(_fixture.TempDir, $"OrphanedPQ_{Guid.NewGuid():N}.xlsx");to
var testFile = Path.Join(_fixture.TempDir, $"OrphanedPQ_{Guid.NewGuid():N}.xlsx");No new imports or helper definitions are necessary, since the surrounding code already uses the Path class.
-
Copy modified line R224
| @@ -221,7 +221,7 @@ | ||
| return; | ||
| } | ||
| | ||
| var testFile = Path.Combine(_fixture.TempDir, $"OrphanedPQ_{Guid.NewGuid():N}.xlsx"); | ||
| var testFile = Path.Join(_fixture.TempDir, $"OrphanedPQ_{Guid.NewGuid():N}.xlsx"); | ||
| IOFile.Copy(sourceFile, testFile); | ||
| | ||
| using var batch = ExcelSession.BeginBatch(testFile); |
| public void Delete_OrphanedPowerQueryConnection_StandardNameMissingQuery_Succeeds() | ||
| { | ||
| // Arrange - Use the test file that has orphaned connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); |
Check notice
Code scanning / CodeQL
Call to System.IO.Path.Combine Note test
Show autofix suggestion Hide autofix suggestion
Copilot Autofix
AI 6 days ago
The problem can be fixed by replacing the use of Path.Combine with Path.Join for the flagged path-building calls. Path.Join combines path segments without dropping earlier segments if a later segment turns out to be absolute—making its semantics safer for this scenario. In the code snippet shown, update all lines using Path.Combine relevant to this code region (lines 216, 224, 251, and 259). No additional method changes or variable definitions are needed; ensure that System.IO.Path is imported (which is implicitly available unless excluded), as both methods come from the same namespace.
-
Copy modified line R216 -
Copy modified line R224 -
Copy modified line R251 -
Copy modified line R259
| @@ -213,7 +213,7 @@ | ||
| public void Delete_OrphanedPowerQueryConnection_GenericName_Succeeds() | ||
| { | ||
| // Arrange - Use the test file that has orphaned connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| var sourceFile = Path.Join(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| | ||
| if (!IOFile.Exists(sourceFile)) | ||
| { | ||
| @@ -221,7 +221,7 @@ | ||
| return; | ||
| } | ||
| | ||
| var testFile = Path.Combine(_fixture.TempDir, $"OrphanedPQ_{Guid.NewGuid():N}.xlsx"); | ||
| var testFile = Path.Join(_fixture.TempDir, $"OrphanedPQ_{Guid.NewGuid():N}.xlsx"); | ||
| IOFile.Copy(sourceFile, testFile); | ||
| | ||
| using var batch = ExcelSession.BeginBatch(testFile); | ||
| @@ -248,7 +248,7 @@ | ||
| public void Delete_OrphanedPowerQueryConnection_StandardNameMissingQuery_Succeeds() | ||
| { | ||
| // Arrange - Use the test file that has orphaned connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| var sourceFile = Path.Join(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| | ||
| if (!IOFile.Exists(sourceFile)) | ||
| { | ||
| @@ -256,7 +256,7 @@ | ||
| return; | ||
| } | ||
| | ||
| var testFile = Path.Combine(_fixture.TempDir, $"OrphanedPQ2_{Guid.NewGuid():N}.xlsx"); | ||
| var testFile = Path.Join(_fixture.TempDir, $"OrphanedPQ2_{Guid.NewGuid():N}.xlsx"); | ||
| IOFile.Copy(sourceFile, testFile); | ||
| | ||
| using var batch = ExcelSession.BeginBatch(testFile); |
| return; | ||
| } | ||
| | ||
| var testFile = Path.Combine(_fixture.TempDir, $"OrphanedPQ2_{Guid.NewGuid():N}.xlsx"); |
Check notice
Code scanning / CodeQL
Call to System.IO.Path.Combine Note test
Show autofix suggestion Hide autofix suggestion
Copilot Autofix
AI 6 days ago
To fix the problem, replace calls to Path.Combine that join potentially non-absolute directory paths with filenames with corresponding calls to Path.Join. In C# (since .NET Core 2.1), Path.Join is available and does not drop earlier segments if later arguments are absolute, ensuring path joining behaves predictably. Specifically, in tests/ExcelMcp.Core.Tests/Integration/Commands/Connection/ConnectionCommandsTests.Delete.cs, replace the call on line 259:
var testFile = Path.Combine(_fixture.TempDir, $"OrphanedPQ2_{Guid.NewGuid():N}.xlsx");with
var testFile = Path.Join(_fixture.TempDir, $"OrphanedPQ2_{Guid.NewGuid():N}.xlsx");No additional imports are needed if System.IO.Path is accessible (as it is in C# test files), so only the minimal change needs to be made to the line in question.
-
Copy modified line R259
| @@ -256,7 +256,7 @@ | ||
| return; | ||
| } | ||
| | ||
| var testFile = Path.Combine(_fixture.TempDir, $"OrphanedPQ2_{Guid.NewGuid():N}.xlsx"); | ||
| var testFile = Path.Join(_fixture.TempDir, $"OrphanedPQ2_{Guid.NewGuid():N}.xlsx"); | ||
| IOFile.Copy(sourceFile, testFile); | ||
| | ||
| using var batch = ExcelSession.BeginBatch(testFile); |
| public void Delete_ValidPowerQueryConnection_ThrowsWithRedirect() | ||
| { | ||
| // Arrange - Use the test file that has valid Power Query connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); |
Check notice
Code scanning / CodeQL
Call to System.IO.Path.Combine Note test
Show autofix suggestion Hide autofix suggestion
Copilot Autofix
AI 6 days ago
To fix the issue, replace the call to Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx") with Path.Join(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"), which is guaranteed to preserve all path segments regardless of input. This change should be made on line 286 of tests/ExcelMcp.Core.Tests/Integration/Commands/Connection/ConnectionCommandsTests.Delete.cs. You should ensure that System.IO.Path is available (no new using directives needed, as it's in the global namespace and already referenced by usage of Path.Combine). No other functional change is necessary.
-
Copy modified line R286
| @@ -283,7 +283,7 @@ | ||
| public void Delete_ValidPowerQueryConnection_ThrowsWithRedirect() | ||
| { | ||
| // Arrange - Use the test file that has valid Power Query connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| var sourceFile = Path.Join(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| | ||
| if (!IOFile.Exists(sourceFile)) | ||
| { |
| public void IsOrphanedPowerQueryConnection_ValidConnection_ReturnsFalse() | ||
| { | ||
| // Arrange - Use the test file that has valid Power Query connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); |
Check notice
Code scanning / CodeQL
Call to System.IO.Path.Combine Note test
Show autofix suggestion Hide autofix suggestion
Copilot Autofix
AI 6 days ago
To address this issue, replace the use of Path.Combine on line 364 with Path.Join. Path.Join is designed for straightforward path composition and will simply concatenate the path segments using the correct directory separator, without any "drop" behavior for absolute paths. Given the arguments are all well-controlled and relative (except for the first), this change is a drop-in replacement and does not affect existing logic.
Needed:
- Ensure
using System.IO;is present ifPath.Joinis not directly accessible, but in this context, it can be assumed or added if missing. - On line 364, replace
Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx")withPath.Join(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx").
No other code or method changes are required.
-
Copy modified line R364
| @@ -361,7 +361,7 @@ | ||
| public void IsOrphanedPowerQueryConnection_ValidConnection_ReturnsFalse() | ||
| { | ||
| // Arrange - Use the test file that has valid Power Query connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| var sourceFile = Path.Join(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| | ||
| if (!IOFile.Exists(sourceFile)) | ||
| { |
| return; | ||
| } | ||
| | ||
| var testFile = Path.Combine(_fixture.TempDir, $"IsValid_{Guid.NewGuid():N}.xlsx"); |
Check notice
Code scanning / CodeQL
Call to System.IO.Path.Combine Note test
Show autofix suggestion Hide autofix suggestion
Copilot Autofix
AI 6 days ago
The fix is to replace Path.Combine calls in the highlighted regions with Path.Join. Path.Join safely concatenates all arguments using the directory separator, without dropping earlier segments if a later one is absolute. This change should be applied to all instances flagged by CodeQL, specifically the ones shown on lines 372 and similarly on line 364 (which are structurally identical and should both be updated for consistency). No changes to functionality are expected, and there is no need for new imports since System.IO.Path is already in use in the file. Only the relevant Path.Combine calls need to be changed; all else remains the same.
-
Copy modified line R321 -
Copy modified line R329 -
Copy modified line R364 -
Copy modified line R372
| @@ -318,7 +318,7 @@ | ||
| public void IsOrphanedPowerQueryConnection_GenericNamedConnection_ReturnsTrue() | ||
| { | ||
| // Arrange - Use the test file that has orphaned connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| var sourceFile = Path.Join(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| | ||
| if (!IOFile.Exists(sourceFile)) | ||
| { | ||
| @@ -326,7 +326,7 @@ | ||
| return; | ||
| } | ||
| | ||
| var testFile = Path.Combine(_fixture.TempDir, $"IsOrphaned_{Guid.NewGuid():N}.xlsx"); | ||
| var testFile = Path.Join(_fixture.TempDir, $"IsOrphaned_{Guid.NewGuid():N}.xlsx"); | ||
| IOFile.Copy(sourceFile, testFile); | ||
| | ||
| using var batch = ExcelSession.BeginBatch(testFile); | ||
| @@ -361,7 +361,7 @@ | ||
| public void IsOrphanedPowerQueryConnection_ValidConnection_ReturnsFalse() | ||
| { | ||
| // Arrange - Use the test file that has valid Power Query connections | ||
| var sourceFile = Path.Combine(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| var sourceFile = Path.Join(AppContext.BaseDirectory, "TestData", "MSXI Baseline.xlsx"); | ||
| | ||
| if (!IOFile.Exists(sourceFile)) | ||
| { | ||
| @@ -369,7 +369,7 @@ | ||
| return; | ||
| } | ||
| | ||
| var testFile = Path.Combine(_fixture.TempDir, $"IsValid_{Guid.NewGuid():N}.xlsx"); | ||
| var testFile = Path.Join(_fixture.TempDir, $"IsValid_{Guid.NewGuid():N}.xlsx"); | ||
| IOFile.Copy(sourceFile, testFile); | ||
| | ||
| using var batch = ExcelSession.BeginBatch(testFile); |
| catch (Exception ex) | ||
| { | ||
| _output.WriteLine($"ListObjects.Add with connection object FAILED: {ex.Message}"); | ||
| | ||
| // Method 2: Try with connection string but specifying the connection name | ||
| ComUtilities.Release(ref listObject); | ||
| try | ||
| { | ||
| // Use the full connection string but the connection should already exist | ||
| string fullConnString = $"OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=Add2WorksheetTest;Extended Properties=\"\""; | ||
| listObject = listObjects.Add( | ||
| 0, // SourceType: 0 = xlSrcExternal | ||
| fullConnString, // Source: connection string | ||
| Type.Missing, // LinkSource | ||
| 1, // XlListObjectHasHeaders: xlYes | ||
| range // Destination | ||
| ); | ||
| _output.WriteLine("ListObjects.Add with connection string SUCCEEDED!"); | ||
| } | ||
| catch (Exception ex2) | ||
| { | ||
| _output.WriteLine($"ListObjects.Add with connection string ALSO FAILED: {ex2.Message}"); | ||
| } | ||
| } |
Check notice
Code scanning / CodeQL
Generic catch clause Note test
Show autofix suggestion Hide autofix suggestion
Copilot Autofix
AI 6 days ago
The best way to fix this problem is to narrow the catch clause from Exception to only those exception types expected from COM interop operations, most notably COMException. This prevents suppressing unrelated errors but still handles anticipated failures from the Excel API interface. Specifically, the change should be applied to the catch (Exception ex) block on line 1666 by replacing it with catch (COMException ex). This will require ensuring the file includes a using System.Runtime.InteropServices; directive, but as we see, line 8 already includes this import.
No other edits outside the shown code are needed, and no additional method definitions or variable changes are required.
-
Copy modified line R1666
| @@ -1663,7 +1663,7 @@ | ||
| ); | ||
| _output.WriteLine("ListObjects.Add with connection object SUCCEEDED!"); | ||
| } | ||
| catch (Exception ex) | ||
| catch (COMException ex) | ||
| { | ||
| _output.WriteLine($"ListObjects.Add with connection object FAILED: {ex.Message}"); | ||
| |
| catch (Exception ex2) | ||
| { | ||
| _output.WriteLine($"ListObjects.Add with connection string ALSO FAILED: {ex2.Message}"); | ||
| } |
Check notice
Code scanning / CodeQL
Generic catch clause Note test
Show autofix suggestion Hide autofix suggestion
Copilot Autofix
AI 6 days ago
To address the generic catch clause, replace all broad catch (Exception ex) and catch (Exception ex2) statements with specific catch blocks for exceptions that are likely to be thrown by COM interop and Excel COM calls, such as COMException, and possibly ArgumentException or others if warranted. In addition, add a fallback catch block for unexpected exception types which logs the full diagnostics. Edit lines 1666–1669 and 1685–1688 in the file, replacing them with more specific catches. You should add using System.Runtime.InteropServices; as needed (this is already present), and ensure all appropriate exceptions are handled explicitly.
-
Copy modified lines R1666-R1677 -
Copy modified line R1680 -
Copy modified lines R1695-R1702 -
Copy modified line R1705
| @@ -1663,9 +1663,21 @@ | ||
| ); | ||
| _output.WriteLine("ListObjects.Add with connection object SUCCEEDED!"); | ||
| } | ||
| catch (COMException comEx) | ||
| { | ||
| _output.WriteLine($"ListObjects.Add with connection object FAILED (COMException): {comEx.Message}"); | ||
| | ||
| // Method 2: Try with connection string but specifying the connection name | ||
| } | ||
| catch (ArgumentException argEx) | ||
| { | ||
| _output.WriteLine($"ListObjects.Add with connection object FAILED (ArgumentException): {argEx.Message}"); | ||
| | ||
| // Method 2: Try with connection string but specifying the connection name | ||
| } | ||
| catch (Exception ex) | ||
| { | ||
| _output.WriteLine($"ListObjects.Add with connection object FAILED: {ex.Message}"); | ||
| _output.WriteLine($"ListObjects.Add with connection object FAILED (unexpected): {ex}"); | ||
| | ||
| // Method 2: Try with connection string but specifying the connection name | ||
| ComUtilities.Release(ref listObject); | ||
| @@ -1682,9 +1692,17 @@ | ||
| ); | ||
| _output.WriteLine("ListObjects.Add with connection string SUCCEEDED!"); | ||
| } | ||
| catch (COMException comEx2) | ||
| { | ||
| _output.WriteLine($"ListObjects.Add with connection string ALSO FAILED (COMException): {comEx2.Message}"); | ||
| } | ||
| catch (ArgumentException argEx2) | ||
| { | ||
| _output.WriteLine($"ListObjects.Add with connection string ALSO FAILED (ArgumentException): {argEx2.Message}"); | ||
| } | ||
| catch (Exception ex2) | ||
| { | ||
| _output.WriteLine($"ListObjects.Add with connection string ALSO FAILED: {ex2.Message}"); | ||
| _output.WriteLine($"ListObjects.Add with connection string ALSO FAILED (unexpected): {ex2}"); | ||
| } | ||
| } | ||
| |
| catch (Exception ex) | ||
| { | ||
| _output.WriteLine($"ERROR: {ex.Message}"); | ||
| } |
Check notice
Code scanning / CodeQL
Generic catch clause Note test
Show autofix suggestion Hide autofix suggestion
Copilot Autofix
AI 6 days ago
To fix the problem, replace the overly broad catch (Exception ex) block with specific catch clauses for likely/expected exception types (COMException, etc.), and where truly unexpected exceptions occur, either let them bubble up or log and rethrow. In this diagnostic test context, the most expected failure is a COMException, so we should catch that explicitly and handle/log accordingly. If other exception types are expected (e.g., InvalidOperationException), those can also be caught and logged. For everything else, either don't catch, or optionally catch, log, and rethrow after logging. The necessary change is within the catch block from lines 1726-1729 of PowerQueryComApiBehaviorTests.cs.
No new methods or field definitions are required, but you must ensure that System.Runtime.InteropServices.COMException is accessible (it is, via the existing using System.Runtime.InteropServices; line).
-
Copy modified lines R1726-R1729 -
Copy modified lines R1732-R1733
| @@ -1723,9 +1723,14 @@ | ||
| } | ||
| } | ||
| } | ||
| catch (COMException comEx) | ||
| { | ||
| _output.WriteLine($"COM ERROR: {comEx.Message}"); | ||
| } | ||
| catch (Exception ex) | ||
| { | ||
| _output.WriteLine($"ERROR: {ex.Message}"); | ||
| _output.WriteLine($"UNEXPECTED ERROR: {ex.GetType().Name}: {ex.Message}"); | ||
| throw; | ||
| } | ||
| finally | ||
| { |
Summary
Fixes #298
Power Query load-to-worksheet operations were creating orphaned connections with generic names like
Connection,Connection1, etc. These connections accumulate over time and cannot be easily managed.Root Cause
When loading Power Query results to a worksheet, the code was using
ListObjects.Add()with a connection string, which caused Excel to create a new connection with a generic name instead of reusing/creating a properly named connection.Solution
Changed the implementation to:
Connections.Add2()to create properly named connections (Query - {queryName}) BEFORE callingListObjects.Add()ListObjects.Add()instead of a connection stringLoadToBothmode, create TWO properly named connections:Query - {queryName}Query - {queryName} (Data Model)Files Changed
Core Implementation:
PowerQueryCommands.LoadTo.cs- Use Add2 pattern for worksheet loadingPowerQueryCommands.Create.cs- Updated LoadToBoth case to use dual connectionsPowerQueryCommands.Lifecycle.cs- Updated pattern matching for new naming conventionPowerQueryHelpers.cs- Enhanced orphan detection logicTests:
PowerQueryCommandsTests.WorksheetCleanup.cs(NEW) - 14 tests for clean slate verificationPowerQueryComApiBehaviorTests.cs- Added Scenario 17 proving the Add2 approachTest Coverage (74+ tests)
Scenarios Tested
Backwards Compatibility
✅ Fully backwards compatible - no breaking changes to API
Related Fix
This also enables users to delete existing orphaned connections via
excel_connection deleteaction. The orphan detection logic now correctly identifies:Connection,Connection1, etc.) - always orphanedQuery - {name}connections without corresponding queries - orphaned