Skip to content

Remove support for extension loading in SQLite integrations #735

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 20, 2025
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions docs/diagnostics.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,6 @@ In these cases, refer to the `<remarks>` docs section of the API for more inform

Once a release of .NET Aspire with that API is available, the API in the .NET Aspire Community Toolkit will be marked as obsolete and will be removed in a future release.

## CTASPIRE002

Support for loading extensions into SQLite requires either a NuGet package or folder path to the library to be provided, and as a result there is some custom logic to load the extension based on the path or NuGet package. This logic will require some experimenting to figure out edge cases, so the feature for extension loading will be kept as experimental until it is proven to be stable.

## CTASPIRE003

The API is marked for deprecation and will be removed in a future release.
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,4 @@
<ItemGroup>
<InternalsVisibleTo Include="CommunityToolkit.Aspire.Hosting.Sqlite.Tests"></InternalsVisibleTo>
</ItemGroup>

<ItemGroup>
<Compile Include="$(SharedDir)\Sqlite\SqliteExtensionMetadata.cs" Link="Utils\Sqlite\SqliteExtensionMetadata.cs" />
</ItemGroup>
</Project>
17 changes: 1 addition & 16 deletions src/CommunityToolkit.Aspire.Hosting.Sqlite/SqliteResource.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
using Microsoft.Extensions.Hosting;
using System.Text.Json;

namespace Aspire.Hosting.ApplicationModel;

/// <summary>
Expand All @@ -18,18 +15,6 @@ public class SqliteResource(string name, string databasePath, string databaseFil
internal string DatabaseFilePath => Path.Combine(DatabasePath, DatabaseFileName);

/// <inheritdoc/>
public ReferenceExpression ConnectionStringExpression => ReferenceExpression.Create($"Data Source={DatabaseFilePath};Cache=Shared;Mode=ReadWriteCreate;Extensions={JsonSerializer.Serialize(Extensions)}");

private readonly List<SqliteExtensionMetadata> extensions = [];

/// <summary>
/// Gets the extensions to be loaded into the database.
/// </summary>
/// <remarks>
/// Extensions are not loaded by the hosting integration, the information is provided for the client to load the extensions.
/// </remarks>
public IReadOnlyCollection<SqliteExtensionMetadata> Extensions => extensions;

internal void AddExtension(SqliteExtensionMetadata extension) => extensions.Add(extension);
public ReferenceExpression ConnectionStringExpression => ReferenceExpression.Create($"Data Source={DatabaseFilePath};Cache=Shared;Mode=ReadWriteCreate");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's beyond the scope of this PR, but can we make the Mode and Cache parts settable, with their current as defaults.

Here's a screenshot of what the valid values are:

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that making Mode and Cache configurable would be valuable. However, as you noted, it's beyond the scope of this PR which focuses on removing the extension loading functionality. This would be a great enhancement for a future PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you create a new issue for that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll create a GitHub issue for making Mode and Cache configurable in SQLite integrations. This would indeed be a valuable enhancement that allows users to customize these SQLite connection string parameters with sensible defaults for the current values.

}

Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
using Aspire.Hosting.ApplicationModel;
using System.Diagnostics.CodeAnalysis;

namespace Aspire.Hosting;

Expand Down Expand Up @@ -88,50 +87,5 @@ public static IResourceBuilder<SqliteResource> WithSqliteWeb(this IResourceBuild
return builder;
}

/// <summary>
/// Adds an extension to the Sqlite resource that will be loaded from a NuGet package.
/// </summary>
/// <param name="builder">The resource builder.</param>
/// <param name="extension">The name of the extension file with to add, eg: vec0, without file extension.</param>
/// <param name="packageName">The name of the NuGet package. If this is set to null, the value of <paramref name="extension"/> is used.</param>
/// <returns>The resource builder.</returns>
/// <remarks>
/// Extensions are not loaded by the hosting integration, the information is provided for the client to load the extensions.
///
/// This extension is experimental while the final design of extension loading is decided.
/// </remarks>
[Experimental("CTASPIRE002", UrlFormat = "https://aka.ms/communitytoolkit/aspire/diagnostics#{0}")]
public static IResourceBuilder<SqliteResource> WithNuGetExtension(this IResourceBuilder<SqliteResource> builder, string extension, string? packageName = null)
{
ArgumentNullException.ThrowIfNull(builder, nameof(builder));
ArgumentException.ThrowIfNullOrEmpty(extension, nameof(extension));

builder.Resource.AddExtension(new(extension, packageName ?? extension, IsNuGetPackage: true, ExtensionFolder: null));

return builder;
}

/// <summary>
/// Adds an extension to the Sqlite resource that will be loaded from a local path.
/// </summary>
/// <param name="builder">The resource builder.</param>
/// <param name="extension">The name of the extension file with to add, eg: vec0, without file extension.</param>
/// <param name="extensionPath">The path to the extension file.</param>
/// <returns>The resource builder.</returns>
/// <remarks>
/// Extensions are not loaded by the hosting integration, the information is provided for the client to load the extensions.
///
/// This extension is experimental while the final design of extension loading is decided.
/// </remarks>
[Experimental("CTASPIRE002", UrlFormat = "https://aka.ms/communitytoolkit/aspire/diagnostics#{0}")]
public static IResourceBuilder<SqliteResource> WithLocalExtension(this IResourceBuilder<SqliteResource> builder, string extension, string extensionPath)
{
ArgumentNullException.ThrowIfNull(builder, nameof(builder));
ArgumentException.ThrowIfNullOrEmpty(extension, nameof(extension));
ArgumentException.ThrowIfNullOrEmpty(extensionPath, nameof(extensionPath));

builder.Resource.AddExtension(new(extension, PackageName: null, IsNuGetPackage: false, extensionPath));

return builder;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,6 @@ public static partial class SqliteResourceBuilderExtensions
{
public static ApplicationModel.IResourceBuilder<ApplicationModel.SqliteResource> AddSqlite(this IDistributedApplicationBuilder builder, string name, string? databasePath = null, string? databaseFileName = null) { throw null; }

[System.Diagnostics.CodeAnalysis.Experimental("CTASPIRE002", UrlFormat = "https://aka.ms/communitytoolkit/aspire/diagnostics#{0}")]
public static ApplicationModel.IResourceBuilder<ApplicationModel.SqliteResource> WithLocalExtension(this ApplicationModel.IResourceBuilder<ApplicationModel.SqliteResource> builder, string extension, string extensionPath) { throw null; }

[System.Diagnostics.CodeAnalysis.Experimental("CTASPIRE002", UrlFormat = "https://aka.ms/communitytoolkit/aspire/diagnostics#{0}")]
public static ApplicationModel.IResourceBuilder<ApplicationModel.SqliteResource> WithNuGetExtension(this ApplicationModel.IResourceBuilder<ApplicationModel.SqliteResource> builder, string extension, string? packageName = null) { throw null; }

public static ApplicationModel.IResourceBuilder<ApplicationModel.SqliteResource> WithSqliteWeb(this ApplicationModel.IResourceBuilder<ApplicationModel.SqliteResource> builder, System.Action<ApplicationModel.IResourceBuilder<ApplicationModel.SqliteWebResource>>? configureContainer = null, string? containerName = null) { throw null; }
}
}
Expand All @@ -29,8 +23,6 @@ public partial class SqliteResource : Resource, IResourceWithConnectionString, I
public SqliteResource(string name, string databasePath, string databaseFileName) : base(default!) { }

public ReferenceExpression ConnectionStringExpression { get { throw null; } }

public System.Collections.Generic.IReadOnlyCollection<Microsoft.Extensions.Hosting.SqliteExtensionMetadata> Extensions { get { throw null; } }
}

public partial class SqliteWebResource : ContainerResource, IResourceWithConnectionString, IResource, IManifestExpressionProvider, IValueProvider, IValueWithReferences
Expand All @@ -41,11 +33,4 @@ public SqliteWebResource(string name) : base(default!, default) { }

public EndpointReference PrimaryEndpoint { get { throw null; } }
}
}

namespace Microsoft.Extensions.Hosting
{
public partial record SqliteExtensionMetadata(string Extension, string? PackageName, bool IsNuGetPackage, string? ExtensionFolder)
{
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,8 @@
using Microsoft.Data.Sqlite;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyModel;
using Microsoft.Extensions.Diagnostics.HealthChecks;
using Microsoft.Extensions.Logging;
using System.Data.Common;
using System.Runtime.InteropServices;
using System.Text.Json;
using RuntimeEnvironment = Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment;

namespace Microsoft.Extensions.Hosting;

Expand Down Expand Up @@ -67,15 +62,6 @@ private static void AddSqliteClient(
settings.ConnectionString = connectionString;
}

if (!string.IsNullOrEmpty(settings.ConnectionString))
{
var cbs = new DbConnectionStringBuilder { ConnectionString = settings.ConnectionString };
if (cbs.TryGetValue("Extensions", out var extensions))
{
settings.Extensions = JsonSerializer.Deserialize<IEnumerable<SqliteExtensionMetadata>>((string)extensions) ?? [];
}
}

configureSettings?.Invoke(settings);

builder.RegisterSqliteServices(settings, connectionName, serviceKey);
Expand Down Expand Up @@ -115,181 +101,9 @@ private static void RegisterSqliteServices(

SqliteConnection CreateConnection(IServiceProvider sp, object? key)
{
var logger = sp.GetRequiredService<ILogger<SqliteConnection>>();
ConnectionStringValidation.ValidateConnectionString(settings.ConnectionString, connectionName, DefaultConfigSectionName);
var csb = new DbConnectionStringBuilder { ConnectionString = settings.ConnectionString };
if (csb.ContainsKey("Extensions"))
{
csb.Remove("Extensions");
}
var connection = new SqliteConnection(csb.ConnectionString);

foreach (var extension in settings.Extensions)
{
if (extension.IsNuGetPackage)
{
if (string.IsNullOrEmpty(extension.PackageName))
{
throw new InvalidOperationException("PackageName is required when loading an extension from a NuGet package.");
}

EnsureLoadableFromNuGet(extension.Extension, extension.PackageName, logger);
}
else
{
if (string.IsNullOrEmpty(extension.ExtensionFolder))
{
throw new InvalidOperationException("ExtensionFolder is required when loading an extension from a folder.");
}

EnsureLoadableFromLocalPath(extension.Extension, extension.ExtensionFolder);
}
connection.LoadExtension(extension.Extension);
}

var connection = new SqliteConnection(settings.ConnectionString);
return connection;
}
}

// Adapted from https://github.com/dotnet/docs/blob/dbbeda13bf016a6ff76b0baab1488c927a64ff24/samples/snippets/standard/data/sqlite/ExtensionsSample/Program.cs#L40
internal static void EnsureLoadableFromNuGet(string package, string library, ILogger<SqliteConnection> logger)
{
var runtimeLibrary = DependencyContext.Default?.RuntimeLibraries.FirstOrDefault(l => l.Name == package);
if (runtimeLibrary is null)
{
logger.LogInformation("Could not find the runtime library for package {Package}", package);
return;
}

string sharedLibraryExtension;
string pathVariableName = "PATH";
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
sharedLibraryExtension = ".dll";
}
else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
{
sharedLibraryExtension = ".so";
pathVariableName = "LD_LIBRARY_PATH";
}
else if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
{
sharedLibraryExtension = ".dylib";
pathVariableName = "DYLD_LIBRARY_PATH";
}
else
{
throw new NotSupportedException("Unsupported OS platform");
}

var candidateAssets = new Dictionary<(string? Package, string Asset), int>();
var rid = RuntimeEnvironment.GetRuntimeIdentifier();
var rids = DependencyContext.Default?.RuntimeGraph.First(g => g.Runtime == rid).Fallbacks.ToList() ?? [];
rids.Insert(0, rid);

logger.LogInformation("Looking for {Library} in {Package} runtime assets", library, package);
logger.LogInformation("Possible runtime identifiers: {Rids}", string.Join(", ", rids));

foreach (var group in runtimeLibrary.NativeLibraryGroups)
{
foreach (var file in group.RuntimeFiles)
{
if (string.Equals(
Path.GetFileName(file.Path),
library + sharedLibraryExtension,
StringComparison.OrdinalIgnoreCase))
{
var fallbacks = rids.IndexOf(group.Runtime);
if (fallbacks != -1)
{
logger.LogInformation("Found {Library} in {Package} runtime assets at {Path}", library, package, file.Path);
candidateAssets.Add((runtimeLibrary.Path, file.Path), fallbacks);
}
}
}
}

var assetPath = candidateAssets
.OrderBy(p => p.Value)
.Select(p => p.Key)
.FirstOrDefault();
if (assetPath != default)
{
string? assetDirectory = null;
if (File.Exists(Path.Combine(AppContext.BaseDirectory, assetPath.Asset)))
{
// NB: Framework-dependent deployments copy assets to the application base directory
assetDirectory = Path.Combine(
AppContext.BaseDirectory,
Path.GetDirectoryName(assetPath.Asset.Replace('/', Path.DirectorySeparatorChar))!);

logger.LogInformation("Found {Library} in {Package} runtime assets at {Path}", library, package, assetPath.Asset);
}
else
{
string? assetFullPath = null;
var probingDirectories = ((string?)AppDomain.CurrentDomain.GetData("PROBING_DIRECTORIES"))?
.Split(Path.PathSeparator) ?? [];
foreach (var directory in probingDirectories)
{
var candidateFullPath = Path.Combine(
directory,
assetPath.Package ?? "",
assetPath.Asset);
if (File.Exists(candidateFullPath))
{
assetFullPath = candidateFullPath;
}
}

assetDirectory = Path.GetDirectoryName(assetFullPath);
logger.LogInformation("Found {Library} in {Package} runtime assets at {Path} (using PROBING_DIRECTORIES: {ProbingDirectories})", library, package, assetFullPath, string.Join(",", probingDirectories));
}

var path = new HashSet<string>(Environment.GetEnvironmentVariable(pathVariableName)!.Split(Path.PathSeparator));

if (assetDirectory is not null && path.Add(assetDirectory))
{
logger.LogInformation("Adding {AssetDirectory} to {PathVariableName}", assetDirectory, pathVariableName);
Environment.SetEnvironmentVariable(pathVariableName, string.Join(Path.PathSeparator, path));
logger.LogInformation("Set {PathVariableName} to: {PathVariableValue}", pathVariableName, Environment.GetEnvironmentVariable(pathVariableName));
}
}
else
{
logger.LogInformation("Could not find {Library} in {Package} runtime assets", library, package);
}
}

internal static void EnsureLoadableFromLocalPath(string library, string assetDirectory)
{
string sharedLibraryExtension;
string pathVariableName = "PATH";
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
sharedLibraryExtension = ".dll";
}
else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
{
sharedLibraryExtension = ".so";
pathVariableName = "LD_LIBRARY_PATH";
}
else if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
{
sharedLibraryExtension = ".dylib";
pathVariableName = "DYLD_LIBRARY_PATH";
}
else
{
throw new NotSupportedException("Unsupported OS platform");
}

if (File.Exists(Path.Combine(assetDirectory, library + sharedLibraryExtension)))
{
var path = new HashSet<string>(Environment.GetEnvironmentVariable(pathVariableName)!.Split(Path.PathSeparator));

if (assetDirectory is not null && path.Add(assetDirectory))
Environment.SetEnvironmentVariable(pathVariableName, string.Join(Path.PathSeparator, path));
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@
<ItemGroup>
<PackageReference Include="AspNetCore.HealthChecks.Sqlite" />
<PackageReference Include="Microsoft.Data.Sqlite" />
<PackageReference Include="Microsoft.DotNet.PlatformAbstractions" />
<PackageReference Include="Microsoft.Extensions.DependencyModel" />
<PackageReference Include="Microsoft.Extensions.Diagnostics.HealthChecks" />
<PackageReference Include="Microsoft.Extensions.Hosting.Abstractions" />
<PackageReference Include="Microsoft.Extensions.Configuration.Binder" />
Expand All @@ -18,7 +16,6 @@
<ItemGroup>
<Compile Include="$(SharedDir)\HealthChecksExtensions.cs" Link="Utils\HealthChecksExtensions.cs" />
<Compile Include="$(SharedDir)\ConnectionStringValidation.cs" Link="Utils\ConnectionStringValidation.cs" />
<Compile Include="$(SharedDir)\Sqlite\SqliteExtensionMetadata.cs" Link="Utils\Sqlite\SqliteExtensionMetadata.cs" />
</ItemGroup>

</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,4 @@ public sealed class SqliteConnectionSettings
/// The default value is <see langword="false"/>.
/// </value>
public bool DisableHealthChecks { get; set; }

/// <summary>
/// Extensions to be loaded into the database.
/// </summary>
public IEnumerable<SqliteExtensionMetadata> Extensions { get; set; } = [];
}
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,5 @@ public sealed partial class SqliteConnectionSettings
public string? ConnectionString { get { throw null; } set { } }

public bool DisableHealthChecks { get { throw null; } set { } }

public System.Collections.Generic.IEnumerable<SqliteExtensionMetadata> Extensions { get { throw null; } set { } }
}

public partial record SqliteExtensionMetadata(string Extension, string? PackageName, bool IsNuGetPackage, string? ExtensionFolder)
{
}
}
Loading