I started refactoring the project, moving things from depending on 'rustc_…' (rustc internals) to 'ra_ap_…' (rust-analyzer internals)

Progress:

• [x] $ cargo modules tree (now called $ cargo modules generate tree)
• [x] $ cargo modules graph (now called $ cargo modules generate graph)

Since we're moving from performing analysis on a project's AST to analysis on its HIR we should be able to fix a bunch more issues in the long term, which right now see kind of hard if not impossible to fix:

• [x] Edition 2018 Support
• [x] Doesn't understand the crate keyword
• [ ] Doesn't understand cfg-if
• [x] Differentiate between modules with the same name
• [x] Support for cargo workspaces

It should also make these features feasible:

• [x] Feature request: Option to list all public types
• [ ] Identify cycles between imports
• [x] Analyze crates in non-test mode

(Once finished this will fix https://github.com/regexident/cargo-modules/issues/63.)

Edition 2018 changes the way use worksin a few different ways. I would like to add support resolving dependencies in projects that use edition = "2018".

My plan is as below:

• [x] Add a command-line option --enable-edition-2018. Initially this will be ignored. Current code does not take edition into consideration anyway. New option will be documented in --help.
• [x] Implement an alternate version of the dependency graph in a separate module. Keep the existing code intact. The new algorithm will only support edition 2018. There will be tests.
• [x] When --enable-edition-2018 is given and edition is "2018" in Cargo.toml, use the alternate implementation. Otherwise keep everything same.

This issue should also close #29 when implemented fully.

Command:

cargo install cargo-modules

Error:

error[E0063]: missing field `position_encoding` in initializer of `ServerCapabilities`
  --> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/ra_ap_rust-analyzer-0.0.132/src/caps.rs:21:5
   |
21 |     ServerCapabilities {
   |     ^^^^^^^^^^^^^^^^^^ missing `position_encoding`

For more information about this error, try `rustc --explain E0063`.
The following warnings were emitted during compilation:

warning: Could not find `.git/HEAD` from manifest dir!

error: could not compile `ra_ap_rust-analyzer` due to previous error

Cargo & Rust Versions:

cargo 1.65.0-nightly (646e9a0b9 2022-09-02) rustc 1.65.0-nightly (59e7a308e 2022-09-11)

From https://github.com/rust-analyzer/rust-analyzer/pull/4405, you guys can use RA to run this.

I have a visitor trait code (I can extract this to a separate crate) you guys can use.

But from what I understand, you guys do not need it and can do something like this

This code produces output as expected:

pub mod mod1 {
    mod mod2 {
        pub fn mod2_fn() {}
    }

    mod mod3 {
        use crate::mod1::mod2;

        pub fn mod3_fn() {
            mod2::mod2_fn()
        }
    }

    pub use mod3::mod3_fn;
}

pub fn crate_fn() {
    mod1::mod3_fn()
}

# cargo modules tree

temp : crate
 └── mod1 : public
     ├── mod2 : private
     └── mod3 : private

However when I move the same code into a standard directory structure there is almost no output from cargo modules:

# find . -name "*.rs"
./src/lib.rs
./src/mod1/mod.rs
./src/mod1/mod3.rs
./src/mod1/mod2.rs

# cargo modules tree

temp : crate
 └── mod1 : public

cargo modules graph is affected in the same way.

I installed cargo modules using nightly toolchain 2020-10-19.

Besides this functionality it also include some formatting and clippy fixes and a fix to solve #14. That commit also switches from compiler libsyntax to rustc-ap-syntax which should prevent issues with changes in nightlies.

Btw I don't think you need nightly to run cargo modules, only to compile it. So even if you are using stable on a project just running cargo modules should work...

Let me know if you don't like anything in this pull request, I am happy to change stuff

This patch adds dependencies to the graph (when --enable-edition-2018 is present).

• crate is supported (see #29 )
• self & super prefixes are supported.
• Glob (*), module (self) & member imports are supported.
• I have changed graph configuration a little bit for increased readability. (feedback welcome)
• No E2015 support in ng::graph::Graph yet.

Please help review.

I love this tool, but I needed an easy way to parse the output, so created a json printer. I tried to follow the pattern set up by the tree printer to implement the json output.

cargo fmt, cargo clippy, and cargo test all come back clean.

Here is what the output looks like. I didn't pretty print it because one can always use jq to format. Also, I didn't wrap the attributes in #[cfg(...)] like you do in tree. If that's a blocker, I can make that change easy enough. The path is also a full path as that's how it's stored internally. For my consumption, I'm planning to strip that based on the directory in which it was invoked after the fact, but if you'd prefer it to do relative paths, just let me know.

[
{"module":"cargo_modules","path":"/Users/grayson/code/contrib/cargo-modules/src/lib.rs","kind":"crate","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::theme","path":"/Users/grayson/code/contrib/cargo-modules/src/theme.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::theme::tree","path":"/Users/grayson/code/contrib/cargo-modules/src/theme/tree.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::theme::graph","path":"/Users/grayson/code/contrib/cargo-modules/src/theme/graph.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::target","path":"/Users/grayson/code/contrib/cargo-modules/src/target.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::printer","path":"/Users/grayson/code/contrib/cargo-modules/src/printer.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::printer::tree","path":"/Users/grayson/code/contrib/cargo-modules/src/printer/tree.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::printer::json","path":"/Users/grayson/code/contrib/cargo-modules/src/printer/json.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::printer::graph","path":"/Users/grayson/code/contrib/cargo-modules/src/printer/graph.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::orphans","path":"/Users/grayson/code/contrib/cargo-modules/src/orphans.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::graph","path":"/Users/grayson/code/contrib/cargo-modules/src/graph.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::graph::walker","path":"/Users/grayson/code/contrib/cargo-modules/src/graph/walker.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::graph::util","path":"/Users/grayson/code/contrib/cargo-modules/src/graph/util.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::graph::orphans","path":"/Users/grayson/code/contrib/cargo-modules/src/graph/orphans.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::graph::node","path":"/Users/grayson/code/contrib/cargo-modules/src/graph/node.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::graph::node::visibility","path":"/Users/grayson/code/contrib/cargo-modules/src/graph/node/visibility.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::graph::node::attr","path":"/Users/grayson/code/contrib/cargo-modules/src/graph/node/attr.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::graph::edge","path":"/Users/grayson/code/contrib/cargo-modules/src/graph/edge.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::graph::builder","path":"/Users/grayson/code/contrib/cargo-modules/src/graph/builder.rs","kind":"mod","visibility":"pub(crate)","attributes":[]}
,{"module":"cargo_modules::options","path":"/Users/grayson/code/contrib/cargo-modules/src/options.rs","kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::options::general","path":null,"kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::options::project","path":null,"kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::options::generate","path":null,"kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::options::generate::json","path":null,"kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::options::generate::tree","path":null,"kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::options::generate::graph","path":null,"kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::options::graph","path":null,"kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::generate","path":"/Users/grayson/code/contrib/cargo-modules/src/generate.rs","kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::generate::tree","path":"/Users/grayson/code/contrib/cargo-modules/src/generate/tree.rs","kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::generate::json","path":"/Users/grayson/code/contrib/cargo-modules/src/generate/json.rs","kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::generate::graph","path":"/Users/grayson/code/contrib/cargo-modules/src/generate/graph.rs","kind":"mod","visibility":"pub","attributes":[]}
,{"module":"cargo_modules::commands","path":"/Users/grayson/code/contrib/cargo-modules/src/commands.rs","kind":"mod","visibility":"pub","attributes":[]}
]```

As of Rust 1.30.0, the crate keyword indicates that a path should begin from the crate root. It's a lot like a leading :: on a path. But cargo-modules's graph command doesn't understand it, and simply omits imports that use the new keyword.

For example, cargo-modules understands an import like use ::foo::bar;, but not use crate::foo::bar.

https://blog.rust-lang.org/2018/10/25/Rust-1.30.0.html

Running cargo +nightly install cargo-modules gives

$ rustup run nightly cargo install cargo-modules
rustc 1.29.0-nightly (254f8796b 2018-07-13)

$ cargo +nightly install cargo-modules

<snip successful module compilations>

   Compiling rustc-ap-arena v154.0.0                                  
error[E0658]: use of unstable library feature 'raw_vec_internals': implemention detailrustc-ap-rustc_target, rust...
  --> /Users/nick/.cargo/registry/src/github.com-1ecc6299db9ec823/rustc-ap-arena-154.0.0/lib.rs:46:5   
   |                                                                            
46 | use alloc::raw_vec::RawVec;                                     
   |     ^^^^^^^^^^^^^^^^^^^^^^                                                       
   |                                                                                                   
   = help: add #![feature(raw_vec_internals)] to the crate attributes to enable 
                                                                                     
error[E0658]: use of unstable library feature 'raw_vec_internals': implemention detail
  --> /Users/nick/.cargo/registry/src/github.com-1ecc6299db9ec823/rustc-ap-arena-154.0.0/lib.rs:67:14  
   |                                                                            
67 |     storage: RawVec<T>,                                      
   |              ^^^^^^^^^                                                           
   |                                                                                                   
   = help: add #![feature(raw_vec_internals)] to the crate attributes to enable

<snip multiple repetitions of the previous error>

error: Could not compile `rustc-ap-arena`.
warning: build failed, waiting for other jobs to finish...
error: failed to compile `cargo-modules v0.4.1`, intermediate artifacts can be found at `/var/folders/vt/d9_pfw9j10l8jn69g02rjy1h0000gn/T/cargo-installsyJ6gK`

After updating to the latest nightly using rustup update and installing cargo-modules using cargo +nightly install cargo-modules, this build error occured:

error[E0635]: unknown feature `array_value_iter_slice`
  --> /home/kangalioo/.cargo/registry/src/github.com-1ecc6299db9ec823/rustc-ap-rustc_arena-693.0.0/src/lib.rs:14:12
   |
14 | #![feature(array_value_iter_slice)]
   |            ^^^^^^^^^^^^^^^^^^^^^^

Presumably, the latest nightly has stabilized the array_value_iter_slice feature which the rustc-ap-rustc_arena crate is not prepared to deal with. Not sure what the best way to resolve this is on cargo-module's side.

The issue can be worked around by manually navigating into the faulty source file and commenting out the #![feature(array_value_iter_slice)] line.

This PR contains the following updates:

| Package | Type | Update | Change | |---|---|---|---| | insta (source) | dev-dependencies | minor | 1.23.0 -> 1.24.1 | | serde_repr | dependencies | patch | 0.1.8 -> 0.1.10 |

Release Notes

Configuration

📅 Schedule: Branch creation - "before 2am" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

• [ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• [ ] Update all other dependencies (insta, serde_repr)

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• [ ] Update rust-analyzer dependencies to v0.0.146 (ra_ap_base_db, ra_ap_cfg, ra_ap_hir, ra_ap_ide, ra_ap_ide_db, ra_ap_paths, ra_ap_proc_macro_api, ra_ap_project_model, ra_ap_rust-analyzer, ra_ap_syntax, ra_ap_text_edit, ra_ap_vfs)

Detected dependencies

• [ ] Check this box to trigger a request for Renovate to run again on this repository

cargo modules generate graph --with-uses is very useful for understanding a crate. However, I notice that it doesn't generate uses edges when one module uses another's components by their fully namespaced names, rather than through a uses statement. Ideally it would, because whether or not the programmer uses a uses statement is just a syntactical detail. For example, I would expect the following crate to have uses edges for both "a -> b" and "a -> c". But it only has the former.

pub mod a {
    use self::b::X;

    pub mod b  {
        pub struct X{}
    }
    pub mod c {
        pub struct Y{}
    }

    pub struct Z {
        x: X,
        y: c::Y
    }
}

digraph {

    graph [
        label="cargo_modules_test",
        labelloc=t,

        pad=0.4,

        // Consider rendering the graph using a different layout algorithm, such as:
        // [dot, neato, twopi, circo, fdp, sfdp]
        layout=neato,
        overlap=false,
        splines="line",
        rankdir=LR,

        fontname="Helvetica", 
        fontsize="36",
    ];

    node [
        fontname="monospace",
        fontsize="10",
        shape="record",
        style="filled",
    ];

    edge [
        fontname="monospace",
        fontsize="10",
    ];

    "cargo_modules_test" [label="crate|cargo_modules_test", fillcolor="#5397c8"]; // "crate" node
    "cargo_modules_test::a" [label="pub mod|a", fillcolor="#81c169"]; // "mod" node
    "cargo_modules_test::a::b" [label="pub mod|a::b", fillcolor="#81c169"]; // "mod" node
    "cargo_modules_test::a::c" [label="pub mod|a::c", fillcolor="#81c169"]; // "mod" node

    "cargo_modules_test" -> "cargo_modules_test::a" [label="owns", color="#000000", style="solid"]; // "owns" edge
    "cargo_modules_test::a" -> "cargo_modules_test::a::b" [label="owns", color="#000000", style="solid"]; // "owns" edge
    "cargo_modules_test::a" -> "cargo_modules_test::a::c" [label="owns", color="#000000", style="solid"]; // "owns" edge
    "cargo_modules_test::a" -> "cargo_modules_test::a::b" [label="uses", color="#7f7f7f", style="Dashed"]; // "uses" edge

}

I'd love to use cargo modules as a tool to optimize my project's module structure. When rendering items and their interdependencies, the output looks very messy:

If it were possible to disable modules nodes, this graph would look significantly cleaner but still contain the information about item interdependencies.

Based on http://www.pathsensitive.com/2021/03/developer-tools-can-be-magic-instead.html The first part speaks about reflection models. It seems to me that should be something powerful but easy to add to cargo modules:

It is starts as basically an option to point to a file with a mapping between modules (files in the original tool, but I suspect modules work better in rust context) and self defined categories. And instead of drawing the modules as nodes for the usage/dependency graph you use the categories and aggregate what is in de modules grouped by that category..

To implement the full model you could also add stuff to draw the graph into an existing dot graph which could contain the hypothesis graph (that should roughly work if the node ids match). However that might not even be needed to make it useful