[BUG] Upgrading from Microsoft.AspNetCore.DataProtection unable to load existing keys #14761
Description
Query/Question
Based on this breaking change I'm working through upgrading to newer Azure.Extensions.* components to be used for DataProtection.
We were previously using the following pacakges
<PackageReference Include="Microsoft.AspNetCore.DataProtection.AzureStorage" Version="3.1.7" /> <PackageReference Include="Microsoft.AspNetCore.DataProtection.AzureKeyVault" Version="3.1.7" /> And we're now testing with
<PackageReference Include="Azure.Extensions.AspNetCore.DataProtection.Blobs" Version="1.0.1" /> <PackageReference Include="Azure.Extensions.AspNetCore.DataProtection.Keys" Version="1.0.1" /> <PackageReference Include="Azure.Identity" Version="1.2.2" /> I worked through the differences in the APIs and have the data protection registration successfully working with this service registration code
services.AddDataProtection() .SetApplicationName("shared") .PersistKeysToAzureBlobStorage(blobStorageConnectionString, containerName, "data-protection-keys.xml") .ProtectKeysWithAzureKeyVault(dataProtectionKeyUri, credentials); In a new environment where there are no pre-existing keys in a key ring, it's working as expected.
The problem is that when I roll this code out to an environment with an existing key ring, it's unable to read existing keys.
At first, the exception was
[11:50:24 ERR] An exception occurred while processing the key element '<key id="b2f42ac4-a1c4-4f49-adba-ed6a14fcade9" version="1" />'. System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.AspNetCore.DataProtection.AzureKeyVault, Version=3.1.2.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified. File name: 'Microsoft.AspNetCore.DataProtection.AzureKeyVault, Version=3.1.2.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, StackCrawlMarkHandle stackMark, ObjectHandleOnStack assemblyLoadContext, Boolean loadTypeFromPartialName, ObjectHandleOnStack type, ObjectHandleOnStack keepalive) at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext, Boolean loadTypeFromPartialName) at System.RuntimeType.GetType(String typeName, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark) at System.Type.GetType(String typeName, Boolean throwOnError) at Microsoft.AspNetCore.DataProtection.SimpleActivator.CreateInstance(Type expectedBaseType, String implementationTypeName) at Microsoft.AspNetCore.DataProtection.TypeForwardingActivator.CreateInstance(Type expectedBaseType, String originalTypeName, Boolean& forwarded) at Microsoft.AspNetCore.DataProtection.TypeForwardingActivator.CreateInstance(Type expectedBaseType, String originalTypeName) at Microsoft.AspNetCore.DataProtection.ActivatorExtensions.CreateInstance[T](IActivator activator, String implementationTypeName) at Microsoft.AspNetCore.DataProtection.XmlEncryption.XmlEncryptionExtensions.DecryptElement(XElement element, IActivator activator) at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.IInternalXmlKeyManager.DeserializeDescriptorFromKeyElement(XElement keyElement) Which makes sense given that I had removed the reference to the old Microsoft.AspNetCore.DataProtection.* packages. My first thought was to add those references back in again so the internals could read the existing keys and we could run for a period of time until new keys were rolled with the new Azure.Extensions.AspNetCore.DataProtection.* packages. After I made that change, I started getting this exception though.
[11:55:12 ERR] An exception occurred while processing the key element '<key id="b2f42ac4-a1c4-4f49-adba-ed6a14fcade9" version="1" />'. System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.AspNetCore.DataProtection.AzureKeyVault.AzureKeyVaultXmlDecryptor.DecryptAsync(XElement encryptedElement) at Microsoft.AspNetCore.DataProtection.AzureKeyVault.AzureKeyVaultXmlDecryptor.Decrypt(XElement encryptedElement) at Microsoft.AspNetCore.DataProtection.XmlEncryption.XmlEncryptionExtensions.DecryptElement(XElement element, IActivator activator) at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.IInternalXmlKeyManager.DeserializeDescriptorFromKeyElement(XElement keyElement) In the key ring XML, the following XML node is the culprit
<encryptedSecret decryptorType="Microsoft.AspNetCore.DataProtection.AzureKeyVault.AzureKeyVaultXmlDecryptor, Microsoft.AspNetCore.DataProtection.AzureKeyVault, Version=3.1.2.0, Culture=neutral, PublicKeyToken=adb9793829ddae60" xmlns="http://schemas.asp.net/2015/03/dataProtection"> If I manually update the decryptorType attribute on existing keys to the newer decryption class, it's able to read existing keys.
<encryptedSecret decryptorType="Azure.Extensions.AspNetCore.DataProtection.Keys.AzureKeyVaultXmlDecryptor, Azure.Extensions.AspNetCore.DataProtection.Keys, Version=1.0.1.0, Culture=neutral, PublicKeyToken=92742159e12e44c8" xmlns="http://schemas.asp.net/2015/03/dataProtection"> I'm wondering if
- I missed a configuration that should allow for this scenario to be handled
- There is a better path to upgrade that doesn't involve losing all existing keys, or manually modifying every key ring to have the newer decryptorType attribute.
Thank you