The Hassle-Free JavaScript Tool Manager

Volta throws an error Volta error: Global package installs are not supported. on adding a global yarn/npm package. I understand the reasoning for this for projects but a valid use case of global installs are scripts to be run locally outside a project directory, and volta fundamentally breaks this.

I've seen VOLTA_UNSAFE_GLOBAL env posted, however this still doesn't allow requiring the package globally, so it doesn't really serve as a workaround. Is there any other recourse?

This PR is going to implement the first-class support for the pnpm package manager.

Thanks to @mikrostew for the well-written https://github.com/volta-cli/rfcs/pull/46, I've been able to implement this feature initially. Most of the critical functionalities have been implemented, except for the global package management. Due to a lack of further details on how to intercept pnpm to relocate its global package installs to fit Volta's current layout.

Implemented

• pnpm shim
• pnpm fetch / install / pin
• pnpm hooks
• per project pnpm manipulation
• volta run with --pnpm and --no-pnpm

Not Implemented

• global package manipulation/management via pnpm

In fact the argument parser for_pnpm() has been introduced to try to intercept pnpm args, but I found that there is no proper way to make pnpm's global installs whose original location is <path to pnpm home>/global/5/node_modules fit Volta's current global package image layout <path to volta home>/tools/image/packages. Hence I intercepted all --global pnpm commands and marked them as unimplemented.

Trackers

RFC: https://github.com/volta-cli/rfcs/pull/46 Tracking issue: https://github.com/volta-cli/volta/issues/737

This is currently a draft PR, please leave your review or comment if you have any ideas to improve the PR to push forward.

Demo

❯ .\target\debug\volta.exe list
⚡️ Currently active tools:

    Node: v16.16.0 (default)
    npm: v8.15.1 (default)
    pnpm: v7.7.1 (default)
    Yarn: v1.22.19 (default)
    Tool binaries available:
        cleancss (default)
        yarn-deduplicate (default)

See options for more detailed reports by running `volta list --help`.
DESKTOP in volta on  feature/pnpm [?] is 📦 v1.0.8 via 🦀 v1.62.0
❯ .\target\debug\pnpm.exe -h
Version 7.7.1
Usage: pnpm [command] [flags]
       pnpm [ -h | --help | -v | --version ]

Manage your dependencies:
      add                  Installs a package and any packages that it depends on. By default, any new package is installed as a prod dependency
      import               Generates a pnpm-lock.yaml from an npm package-lock.json (or npm-shrinkwrap.json) file
   i, install              Install all dependencies for a project
  it, install-test         Runs a pnpm install followed immediately by a pnpm test
  ln, link                 Connect the local project to another one
      prune                Removes extraneous packages
  rb, rebuild              Rebuild a package
  rm, remove               Removes packages from node_modules and from the project's package.json
      unlink               Unlinks a package. Like yarn unlink but pnpm re-installs the dependency after removing the external link
  up, update               Updates packages to their latest version based on the specified range

Review your dependencies:
      audit                Checks for known security issues with the installed packages
  ls, list                 Print all the versions of packages that are installed, as well as their dependencies, in a tree-structure
      outdated             Check for outdated packages

Run your scripts:
      exec                 Executes a shell command in scope of a project
      run                  Runs a defined package script
      start                Runs an arbitrary command specified in the package's "start" property of its "scripts" object
   t, test                 Runs a package's "test" script, if one was provided

Other:
      pack
      publish              Publishes a package to the registry
      root

Manage your store:
      store add            Adds new packages to the pnpm store directly. Does not modify any projects or files outside the store
      store prune          Removes unreferenced (extraneous, orphan) packages from the store
      store status         Checks for modified packages in the store

Options:
  -r, --recursive          Run the command for each project in the workspace.
DESKTOP in volta on  feature/pnpm [!] is 📦 v1.1.0-alpha.1 via 🦀 v1.62.0
❯ .\target\debug\pnpm.exe dlx create-vite vite-project --template vanilla
.../../../.pnpm-store/v3/tmp/dlx-31720   |   +6 +
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: D:\.pnpm-store\v3
  Virtual store is at:             ../../../../../.pnpm-store/v3/tmp/dlx-31720/node_modules/.pnpm
.../../../.pnpm-store/v3/tmp/dlx-31720   | Progress: resolved 6, reused 6, downloaded 0, added 6, done

Scaffolding project in D:\workspace\repos\chawyehsu\volta\vite-project...

Done. Now run:

  cd vite-project
  pnpm install
  pnpm run dev

DESKTOP in volta on  feature/pnpm [!?] is 📦 v1.1.0-alpha.1 via 🦀 v1.62.0 took 5s
❯ cd vite-project
DESKTOP in volta\vite-project on  feature/pnpm [!?] via ⬢ v16.16.0
❯ ..\target\debug\pnpm.exe i
Packages: +14
++++++++++++++
Packages are copied from the content-addressable store to the virtual store.
  Content-addressable store is at: D:\.pnpm-store\v3
  Virtual store is at:             node_modules/.pnpm
node_modules/.pnpm/[email protected]/node_modules/esbuild: Running postinstall script, done in 318ms
Progress: resolved 34, reused 0, downloaded 14, added 14, done

devDependencies:
+ vite 3.0.4
DESKTOP in volta\vite-project on  feature/pnpm [!?] via ⬢ v16.16.0
❯ ..\target\debug\volta.exe pin node pnpm
success: pinned [email protected] (with [email protected]) in package.json
success: pinned [email protected] in package.json
DESKTOP in volta\vite-project on  feature/pnpm [!?] via ⬢ v16.16.0 took 3s
❯ cat .\package.json
{
  "name": "vite-project",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "vite build",
    "preview": "vite preview"
  },
  "devDependencies": {
    "vite": "^3.0.0"
  },
  "volta": {
    "node": "16.16.0",
    "pnpm": "7.7.1"
  }
}
DESKTOP in volta\vite-project on  feature/pnpm [!?] via ⬢ v16.16.0
❯ ..\target\debug\pnpm.exe --version
7.7.1
DESKTOP in volta\vite-project on  feature/pnpm [!?] via ⬢ v16.16.0
❯

My intention was to have my first volta install node be the default moving forward. I then went ahead and installed another version I wanted to use in another project and that became my default. Is there a way to manually set the global default version?

Closes #102 Closes #52 Closes #20 Closes #329

Info

• There wasn't a Windows installer that actually got the system into a usable state with the changes that have been made to Notion since the installer was initially created.
• There were also some issues causing Notion to not actually work in Windows:
  • The node zip file has a different structure than the *nix tarball, without the intervening lib directory.
  • We need to execute the scripts on Windows using cmd.exe /C <TOOL>, instead of just <TOOL>
  • In PowerShell on Windows, the arg0 path includes .exe, but when we want to execute one of the tools, we don't want to call e.g. npm.exe, since npm is a script, not an executable.

Changes

• Added main.wxs XML file for building the Installer with cargo wix, including creating the necessary directory structure and removing the installed binary files on uninstall.
• Moved node_archive_npm_package_json_path to the OS-specific modules to represent the fact that the Windows archive has a different directory structure than the *nix archive.
  • Also updated the acceptance test fixtures for the Windows .zip files to represent the correct structure.
• Refactored the creation of std::process::Command objects into a module that handles the platform-specific addition of cmd.exe /C.
• Updated logic for getting the tool name from arg0 to remove any .exe suffixes on Windows.
• Removed old dev/windows scripts in favor of cargo wix.
• Added logic for determining if a 3rd party package is a script with a shebang loader and recording that information.
• If a loader is found, when executing the script, call <loader> <tool> <args> instead of <tool> <args>

Tested

• Built the Windows installer using cargo wix.
• Verified that the installer correctly installs and uninstalls Notion.
• After install, verified that notion install node and notion install yarn work correctly.
• Verified that notion, node, npm, npx, and yarn can be executed in the Command Prompt, PowerShell, and Git Bash.

Notes

• Building the Windows Installer requires the WiX Toolset and cargo-wix (cargo install cargo-wix). Once those are installed, you can build the installer using cargo wix in the root of the project.
• The UI for the Installer is very minimal and doesn't include any Notion images at the moment. There are comments in main.wxs about how to add the Welcome Image and the Banner image, as well as the Icon for the program in the "Add / Remove Programs" screen. Related to #8
• An issue was found with Git Bash (MinGW emulated Bash prompt), causing the shims to 3rd party binaries to not execute correctly. A hacky workaround was added (described in the comments here), but if anyone can come up with a better solution, that would be appreciated.

This PR builds on the work @scalvert started in #171 to add full fish support to Notion.

• closes #171
• resolves #152

Testing

1. Check out this branch locally. If you're using hub, hub pr checkout 266. (If you're not using hub… you should do that thing. And also alias it to git to make your life wonderful.)

2. Create a release build: cargo build --release.

3. Do the equivalent of running the install script: ./dev/unix/build.sh; and ./dev/unix/install.sh.

4. Open a new shell, or refresh your current shell with the updated config: source ~/.config/fish/config.fish.

Then you should be able to use Notion!

It seems that install volta when using a fish shell, the executables do not get properly set. Therefore, my system version is either non-existent or it uses a version from a previous workflow.

As a temporary work around, I copied the executables found in ~/.volta/bin/ to /usr/local/bin/. But this breaks down when using volta for multiple projects, with different versions.

In a monorepo I'd like a way to have a single source of truth for what versions are pinned. Can we have a way of extending notion config within a sub-project? E.g.

{
  "toolchain": {
    "extends": "../../package.json"
  }
}

This would still allow for overrides on a per-project basis but simplify the common case within a monorepo.

After installation on ubuntu. I receive this message and notion don't start.

error while loading shared libraries: libssl.so.10: cannot open shared object file: No such file or directory

Any idea why ?

At WWDC 2020, Apple announced upcoming Macs will ship with ARM processors. We'll want to add support for those as soon as makes good sense:

• when Rust itself supports the target architecture
• once someone has a dev kit we can use for testing (this may end up being me, since I have interest for side projects)

This is currently blocked:

• Rust support should come quickly, since Apple's ARM 64 targets are already in the general support matrix—but are currently Tier 2 targets per the Forge, and Rust will need to do some work to bump that up to a Tier 1 support category. (Presumably the existing aarch64-apple-ios is the same architecture, which could make an interesting situation name-wise. 😂)
• Getting a dev kit is a matter of applying for one and being approved. I'll investigate on that front since it's potentially of interest to me anyway.

This pull request extends #1130 by implementing the requested changes and making sure things work on Linux ( arm64 -> aarch64 )

Requested changes that were implemented

• Redundant comments in ./ci/docker/Dockerfile.centos8 were removed
• $arch_str backwards compatibility: the public download URL's haven't changed at all, the ARM versions now append a -aarch64 suffix while the x86 versions do not have this suffix, to stay backwards compatible. This has also been extended to the volta-install.sh file, which only adds the suffix to the download url if the platform is aarch64.

Unsolved questions

(@charlespierce): Given that the arm64 builds take so long, is there a way we can lock them in the same way as the final Publish release job, so that they only run when we're actually publishing a new version? My worry is that currently this build is part of the CI check for PRs and I don't want to run into a spot where every PR is gated on a build that takes ~2 hours. This is possible if we split out the ARM build into a separate CI job instead of using a matrix. I've made that change here, but if this is not preferred ( not DRY, but CI files are never really DRY anyway ), let me know.

Most of the changes were made by @vanstinator, and his commits have been preserved in this PR such that he is credited properly for his work. I made this separate PR to speed up the process to getting these changes released.

Reviews would be massively appreciated 🙏

As mentioned here, you can avoid building parking_lot entirely if you upgrade console, indicatif, and reqwest.

Since reqwest has been swapped out in https://github.com/volta-cli/volta/pull/733 (this PR builds on top of that) and the new stuff doesn't require it, this is now possible. Honestly it's not really that much more to build, but it is completely unnecessary.

Also @charlespierce, it looks like the lockfile is stale, so feel free to cherry-pick e9599ce and see if you get any improvements there too? Here are some cargo trees you can reproduce:

before regeneration (5af6691): https://gist.github.com/joshuarli/9bb2968b764e7a747a78b6697faae39b

after (e9599ce): https://gist.github.com/joshuarli/9289bcbcf5233b93fa7701fbe9faa925

after upgrade (8fb6e2d): https://gist.github.com/joshuarli/7a973b26a518c33de45a040ad3a79bb1

Bumps once_cell from 1.16.0 to 1.17.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps console from 0.15.2 to 0.15.3.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps serde from 1.0.151 to 1.0.152.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps serde_json from 1.0.89 to 1.0.91.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps thiserror from 1.0.37 to 1.0.38.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Solves issue https://github.com/volta-cli/volta/issues/1397 for files.

PS: I'm not a Rust developer and I did my best but I know this code must have some serious issues that I'll be happy to solve,