feat: Use nixpkgs to resolve with liveness

Since 7683220 we always mark nested `with` expressions alive. This is technically sound because, indeed, there's no general way to determine whether a particular attribute comes from one of the nested `with`s. Sadly, this change made it impossible to detect common problematic patterns in nixpkgs tree like: ``` meta = with lib; { maintainers = with lib.maintainers; [ ... ]; }; ``` This would be very useful in our ongoing attempt to automate cleanup of these patterns from the whole `nixpkgs` tree, see: technowledgy/refactor-tractor#14 This change attempts to address this problem. It is observed that nixd completion controllers already use nixpkgs attrset and heuristics to look for known attributes in `pkgs` and `lib`, assuming these belong to nixpkgs. We can do the same when analysing `with` scopes. 1. This patch reuses the existing nixpkgs client infrastructure for `Sema/VariableLookup.cpp`. Since variable analysis is part of `libnixf` while nixpkgs client and LSP are part of `nixd`, we have to relocate a number of modules plus the `nixd-attrset-eval` tool outside the `nixd` directory under `common`. 2. The `nixd-attrset-eval` tool was truncating the number of returned values by 30, which is useful for interactive use in completion helper context, but is not enough for comprehensive variable scope analysis. For this reason, the json protocol for the tool was extended to support `MaxItems` attribute that can be set to `0` to indicate that no truncation should occur. 3. On performance optimizations: The new nixpkgs client is initialized once per call to `VLA`. It will load and cache discovered attributes as needed for the `with` resolution procedure. (In the future, it may be used for other needs as needed.) Note: It could be beneficial to allow reuse of the same nixpkgs client with its cached results across multiple calls to `VLA`. This is left for a future exercise, if and when consuming tools like `nixd-tidy` gain support for multi-file operation. 4. On build system: Because the project supports "global" meson build from the root directory as well as component specific builds for `libnixf` and `nixd`, we have to setup scaffolding to reuse the `common` library in both. This is achieved with subprojects and a bunch of symlinks. 5. Because `libnixf` / `nixd-tiny` can be used interactively, and each call to `nixd-attrset-eval` is logged in extreme details (including complete json protocol comms between the tool and its driver), we have to suppress this output not to pollute the caller's stdout with these debug messages. Some adjustments were done to allow to disable these log messages when the tool is called from `VLA` nixpkgs client. 6. VLA test timeout is bumped from 30s to 120s because it now has to run nixd-attrset-eval. The Sema test now takes time comparable to nixd regression suite for this reason. 120s is the timeout taken from nixd regression suite.

Author: booxter

nixd/include/nixd/CommandLine/Options.h renamed to common/include/nixd/CommandLine/Options.h

@@ -67,6 +67,10 @@ class AttrSetClient : public lspserver::LSPServer {
 
  void exit() { Exit(nullptr); }
 
+ void setLoggingEnabled(bool Enabled) {
+ LSPServer::setLoggingEnabled(Enabled);
+ }
+
  /// Get executable path for launching the server.
  /// \returns null terminated string.
  static const char *getExe();

nixd/include/nixd/Eval/AttrSetClient.h renamed to common/include/nixd/Eval/AttrSetClient.h

@@ -0,0 +1,15 @@
+#pragma once
+
+#include "nixd/Eval/AttrSetClient.h"
+
+#include <memory>
+#include <string_view>
+
+namespace nixd {
+
+/// Launches an attrset evaluator process whose stderr is redirected to
+/// `WorkerStderr`.
+void spawnAttrSetEval(std::string_view WorkerStderr,
+ std::unique_ptr<AttrSetClientProc> &Worker);
+
+} // namespace nixd

nixd/include/nixd/Eval/AttrSetProvider.h renamed to common/include/nixd/Eval/AttrSetProvider.h

@@ -108,6 +108,7 @@ struct AttrPathCompleteParams {
  Selector Scope;
  /// \brief Search for packages prefixed with this "prefix"
  std::string Prefix;
+ std::optional<int> MaxItems;
 };
 
 llvm::json::Value toJSON(const AttrPathCompleteParams &Params);