Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get nx graph data via stdout #18689

Closed
1 task done
skeate opened this issue Aug 17, 2023 · 6 comments · Fixed by #23446
Closed
1 task done

Get nx graph data via stdout #18689

skeate opened this issue Aug 17, 2023 · 6 comments · Fixed by #23446
Assignees

Comments

@skeate
Copy link

skeate commented Aug 17, 2023

Description

Add a mechanism to get project graph data out of nx in a manner useful for scripts.

Motivation

nx graph by itself spins up a web server to serve an interactive graph, which is fine for humans but unusable for scripting. Sometimes it would be nice to programmatically get (for example) a list of the internal projects that a particular project depends on.

Suggested Implementation

A --stdout option (or something similar) for the nx graph command.

Alternate Implementations

There is a workaround, though it's a bit convoluted; nx graph already can output JSON if you give it a --file=<something>.json option (presumably this looks at the extension, since you can also do --file=<something>.html and get an HTML page). Thus, with some tempfile shenanigans, you can get a one-liner that does the same kind of thing, something like

$TMPDIR=$(mktemp -d) nx graph --file=$TMPDIR/graph.json && \
  jq -r ".graph.dependencies" $TMPDIR/graph.json

but that's ugly, and I'm honestly not sure how portable it is (my original attempts used other options of mktemp which aren't actually options in MacOS's version of the program).

Since there's already the mechanism to output JSON, this should be a fairly simple addition. But maybe there is already a way (aside from my hack) and I just couldn't find it in the docs.

@FrozenPandaz
Copy link
Collaborator

There is actually a way.

You can pass --file stdout but I think we can definitely add a --print flag which would print the graph to the terminal.

Would you be interested in making a PR?

@FrozenPandaz FrozenPandaz added the community This is a good first issue for contributing label Aug 18, 2023
@skeate
Copy link
Author

skeate commented Aug 18, 2023

Hmm. I feel like there's not too much gain with --print over --file stdout; it's just pretty unclear that the latter is an option from the documentation. Maybe this is just an "improve docs" request?

Or do you think it would be worth having a separate --print flag for better discoverability?

Either way, yeah, I can probably make a PR for this.

@FrozenPandaz
Copy link
Collaborator

Exactly, --print is more self-explanatory. It will show up in the docs, --help, etc.

We cannot remove --file stdout because it allows for nx build nx --graph stdout.

--file stdout also needs to be documented properly though. That would be a separate change though.

@AgentEnder AgentEnder added the scope: misc Misc issues label Aug 21, 2023
@rlmestre
Copy link
Contributor

rlmestre commented Sep 6, 2023

I could take this if work hasn't started on it yet. (adding the new flag + docs, docs for --file stdout)

@jhecking
Copy link

jhecking commented Sep 8, 2023

I was looking for a way to use the generated graph programatically as well and was very happy to find out about the stdout option through this ticket. Especially since it works with nx affected --graph <file> as well. Definitely should be documented better!

However, I am still facing some minor issues with this feature:

  1. When trying to pipe the output of nx affected -t build --graph stdout to another command the output seems to be truncated. E.g. nx affected -t build --graph stdout | jq . results in "parse error: Unfinished JSON term at EOF at line 2566, column 4".
❯ nx affected -t build --graph stdout | tail
              "options": {
                "lintFilePatterns": [
                  "apps/borneo-cli/**/*.ts"
                ]
              },
              "configurations": {}
            }
          },
          "tags": [],
          "implicitDepen

On the other hand, when redirecting the stdout output to a file, it seems to be complete – obviously, this defeats the purpose of using --graph stdout in the first place, though.

❯ nx affected -t build --graph stdout > graph.json

❯ tail graph.json
      "smithy:build": [],
      "borneo-workers:bundle-workflows:production": [],
      "borneo-encryption:build": [],
      "borneo-client-internal:build": [
        "smithy:build"
      ],
      "smithy-server-express:build": []
    }
  }
}
  1. Depending on other options passed, nx graph --file stdout includes other content in the output on stdout, which makes it hard to process it, e.g.
❯ nx graph --affected --file stdout | head

 >  NX   Affected criteria defaulted to --base=main --head=HEAD

{
  "graph": {
    "nodes": {
      "borneo-client-internal": {
        "name": "borneo-client-internal",
        "type": "lib",
        "data": {

If this text would be printed to stderr instead it would be easy to filter out, e.g. nx graph --file stdout 2>/dev/null. Or, alternatively, maybe there should be a --quiet or --silent option or some such.

Copy link

github-actions bot commented May 9, 2024

This issue has been automatically marked as stale because it hasn't had any activity for 6 months.
Many things may have changed within this time. The issue may have already been fixed or it may not be relevant anymore.
If at this point, this is still an issue, please respond with updated information.
It will be closed in 21 days if no further activity occurs.
Thanks for being a part of the Nx community! 🙏

@github-actions github-actions bot added the stale label May 9, 2024
@github-actions github-actions bot removed the stale label May 16, 2024
@AgentEnder AgentEnder removed the community This is a good first issue for contributing label May 16, 2024
FrozenPandaz pushed a commit that referenced this issue May 31, 2024
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

## Current Behavior
--graph stdout and --file stdout aren't documented. When printing graph
to stdout, its possible for some output to get missed.

## Expected Behavior
Existing flags are documented, and `--print` is more discoverable. We
await stdout finishing before exiting the program.

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #18689
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants