nx show projects should return packages' pathnames

[dmaklygin]

I was trying to extract affected packages by changes I had made.
Unfortunately, nx graph --affected returns deprecated affectedPackages attribute I should not rely on.
So, I have to run both commands in a sequence:

1. nx show projects --affected
2. nx graph

Unfortunately, nx show projects returns only a simple list of packages names, and it's not enough to determine packages location.
If nx show projects --affected returns extended list of packages, it would be totally enough

Motivation

I was trying to ditribute tests execution between runners, so I had to determine affected packages and gradually distribute tests betweeen runners.

Suggested Implementation

Return a list of objects. Each object should have the following:

1. Package name
2. Package location

[Alonski]

So I did something similar to create a custom gpt (go to project) bash function.
Important for you:

monorepo_root=$(git rev-parse --show-cdup)
project_path=$(nx show project "$1" | jq -r .root)
cd "${monorepo_root}${project_path}"

Full script:

# For usage information see inlined usage conditional
gtp() {
    # Show usage if on errors
    showUsage=0

    # Check that an argument was passed
    # https://unix.stackexchange.com/questions/25945/how-to-check-if-there-are-no-parameters-provided-to-a-command
    if [ $# -eq 0 ]; then
        echo "Argument [projectName] not passed"
        showUsage=1
    fi

    # Check that needed programs exist
    # https://stackoverflow.com/questions/592620/how-can-i-check-if-a-program-exists-from-a-bash-script
    if ! command -v git &> /dev/null; then
        echo "git could not be found, please install it globally"
        showUsage=1
    fi

    if ! command -v nx &> /dev/null; then
        echo "nx could not be found, please install it globally"
        showUsage=1
    fi

    if ! command -v jq &> /dev/null; then
        echo "jq could not be found, please install it globally"
        showUsage=1
    fi

    # Print usage
    if [ $showUsage -eq 1 ]; then
        echo "Usage: $0 [projectName]"
        echo "Example: $0 @crowdstrike/mega-slat-table"
        echo "This function uses NX to CD into a project by the project name"
        echo "Required globally installed commands: git, nx, jq"
        return 1
    fi

    # Get monorepo root using git. Ex: ../../../ (Needs to relative in case of worktree)
    monorepo_root=$(git rev-parse --show-cdup)

    # Get NX project path relative to root. Ex: packages/apps/store
    project_path=$(nx show project "$1" | jq -r .root)

    # Show error if project_path failed
    if [ $? -ne 0 ]; then
        echo "NX failed to show the project, make sure you are in the monorepo and the project name is correct"
        # Run gtp again with no arguments to show usage
        gtp
        return 1
    fi

    # CD into the root+path combo
    cd "${monorepo_root}${project_path}"
}

[dmaklygin]

I was able to make it work via nx show projects and nx graph.
But, having a single command would be more optimal.

[raoulkent]

I was able to make it work via nx show projects and nx graph. But, having a single command would be more optimal.

Is it possible for you to share how? 😄

[aikrez]

For our tooling we relied on the project path name in the workspace.json which is now removed. Currently I am looking for a way to ditch the old workspace.json and get the project path.
It would be nice that nx show projects had an additional argument to return the full detail of the project like nw show project does.

[MaxKless]

Hi!
For automated tooling, I think having a more programatically readable format makes sense. So I think it's a valid feature request.

It needs a little bit more design, though. There is already a --json flag in nx show projects so any changes to that functionality would be a breaking change which I'd like to avoid.

You could read the list of affected projects in JSON format by running npx nx show projects --json --affected and then extract information about the individual projects from .nx/cache/project-graph.json or by using the programmatic API createProjectGraphAsync from @nx/devkit.

I'd love some more concrete thoughts on how you would design this updated API and what use cases you have that aren't solvable with another solution. Let's continue the conversation in a discussion :)

[raoulkent]

I was running into the exact same need. Building github workflows with reusable components that I want to run as a matrix for affected projects can't be an unlikely usecase.

And since some workflows expect to be run in the root of a project (such as SAST tooling) it would be incredibly useful to have the possibility to output the paths via nx show or nx affected somehow.

Has this discussion been converted into a feature request?

[MaxKless]

Have you looked into this>

You could read the list of affected projects in JSON format by running npx nx show projects --json --affected and then extract information about the individual projects from .nx/cache/project-graph.json or by using the programmatic API createProjectGraphAsync from @nx/devkit.

I'd love some more specific ideas on how you would design the commands :) It seems pretty easy to write your own script for atm, no?

[raoulkent]

Do you mean in terms of implementation or in terms of UX?

I have the need to both filter on tags, and to get the paths, which did make it a little messy. And considering that this isn't an isolated issue it seems that there is indeed some need for this?

In my case we needed for some reusable CI components the paths to affected projects with certain tags.
I have looked into it (first option) but I think this has issues:

1. It isn't totally clear to any new user of NX that this is the preferred approach
2. The suggested jq -r .root isn't very performant, as it looks through all the nodes in the json if I am not mistaken

I have a bash script that is likely missing things, as it is not something that is my greatest expertise. Thus I thought it prudent to ask!

#/bin/bash

# Function to get affected projects based on provided tags.
# Usage: get_affected_projects_paths [tag1] [tag2] ...
# - If no tags are provided, all affected projects are returned.
# - If tags are provided, only affected projects with the provided tags are returned.
get_affected_projects_paths() {
  if [ "$1" = "-h" ] || [ "$1" = "--help" ]; then
    echo "Usage: get_affected_projects_paths [tag1] [tag2] ..."
    echo "Description: Retrieves affected projects. Filters by tags if provided."
    return
  fi

  local graphFile=".nx/cache/project-graph.json"
  local jqPath='.nodes[$nodeName].data.root'
  if [ ! -f "$graphFile" ]; then
    echo "Graph file not found. Running 'nx show projects --json --affected' first."
    nx show projects --json --affected
  fi

  local excludeString=""
  local firstTag=true
  for tag in "$@"; do
    if [ "$firstTag" = true ]; then
      firstTag=false
    else
      excludeString+=","
    fi
    excludeString+="*,!tag:$tag"
  done

  local nodeNames=()
  local readCmd="npx nx show projects --affected"
  [ -n "$excludeString" ] && readCmd+=" --exclude=\"$excludeString\""
  IFS=$'\n' read -r -d '' -a nodeNames < <(eval $readCmd && printf '\0')
  echo "Node names: ${nodeNames[@]}"

  for nodeName in "${nodeNames[@]}"; do
    local rootValue=$(jq -r --arg nodeName "$nodeName" "$jqPath" "$graphFile")
    if [ "$rootValue" != "null" ]; then
      echo "Root for $nodeName: $rootValue"
    else
      echo "Node $nodeName not found."
    fi
  done
}

# Example calls
get_affected_projects_paths  # No tags
get_affected_projects_paths tag1  # Filter by tag1

[raoulkent]

And if you meant to ask how one might design the UX of the command:
nx show --projectPath and nx show --path are both decent options if one was to design for just the .data.root node.

The other option would be instead something nx show --data="root" or somesuch if we're talking about getting a generic part of the data object. But that's much more opaque and requires more knowledge about the graph json structure. So good docs are a must if this approach were selected.