-
Notifications
You must be signed in to change notification settings - Fork 557
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DAG loses concurrency with module chaining #7353
Comments
Very cool find! It's not the CLI though, it’s the engine:
Something’s happening in the In |
You're right, something is forcing the sequential evaluation of everything.
func (jobs Jobs) One() *Container { return jobs.Jobs[1].Container }
func (r *MyModule) HackOne(ctx context.Context) *Container {
return r.JobGroup(ctx, false).One()
} |
Changing the Directory field to a string fixes the issue for my usecase. type Job struct {
*Container
OutputDir, Key string
} |
Wrapping the directory in a container doesn't seem to help. ( |
When chaining module functions from the CLI, it appears that the expected DAG concurrency is lost. Here is an example:
Reproduced with Dagger v0.11.3
3 jobs run concurrently with:
dagger call job-group --sync out
dagger call hack
The 3 jobs are run serially with:
dagger call job-group out
The final command,
job-group out
, is functionally identical tohack
, with the difference that the functions are chained by the CLI rather than in codeThe text was updated successfully, but these errors were encountered: