Compare commits

...

16 Commits

Author SHA1 Message Date
JamesHenry 3c4696d8b8 fix(core): tui exit due to cancellation 2025-04-10 10:26:48 +01:00
Jason Jean 2f52c209ce feat(core): enable dte to execute continuous tasks (#29993)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->

Continuous tasks are not supported in DTE and there is no good API for
Agents to utilize

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

There is an API for agents to utilize to run continuous tasks.

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2025-04-09 21:59:58 -04:00
FrozenPandaz dcd99f1362 chore(core): regen native typings 2025-04-09 19:30:50 -04:00
James Henry 621ad8a4d5 feat(core): add the experimental Terminal UI for tasks (#30565) 2025-04-09 19:30:50 -04:00
Colum Ferry 86a2d5e183 docs(module-federation): add sections covering continuous tasks change (#30645)
## Current Behavior
Module Federation Documentation talks about using `--devRemotes`
exclusively for serving remote applications that feature teams are
working on.
With Continuous Tasks this is no longer the case

## Expected Behavior
Add section mentioning Continuous Tasks and the change to the `serve`
flow
2025-04-09 19:30:50 -04:00
Colum Ferry aa10aac08c feat(module-federation): add continuous tasks support to react rspack (#30644)
## Current Behavior
Continuous tasks are not set up for React Rspack Module Federation
Remote projects.
This is important because `--dev-remotes` is no longer supported with
Crystal Module Federation usage.

## Expected Behavior
Add Continuous Tasks support for React Rspack Module Federation Remote
Projects.
This replaces the need for `nx serve shell --dev-remotes=remote1`.

Instead, the command is simply `nx serve remote1` and continuous tasks
means that the `shell:serve` task is executed correctly.
2025-04-09 19:30:50 -04:00
Jason Jean 8ca311988f feat(core): share continuous tasks (#29901)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

<!-- This is the behavior we have today -->

<!-- This is the behavior we should expect with the changes in this PR
-->

<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #

---------

Co-authored-by: Leosvel Pérez Espinosa <leosvel.perez.espinosa@gmail.com>
2025-04-09 19:30:48 -04:00
Jason Jean c64170338a chore(repo): fix unit tests (#30431)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->

Unit tests are broken after rebase

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

Unit tests are passing

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2025-04-09 19:30:05 -04:00
Colum Ferry b65dba2ee0 feat(node): generate new apps with continuous: true (#30148)
## Current Behavior
Node applications rely on the `@nx/js:node` executor to handle serving. 
Unfortunately, it can not be determined if existing usage of the
executor is used for a continuous or finite task.

## Expected Behavior
Generate new node applications with `continuous: true` for their serve
targets
2025-04-09 19:30:05 -04:00
Colum Ferry 1f49c3d7d7 feat(storybook): infer continuous tasks for storybook serve and serve-static (#30069)
## Current Behavior
The `@nx/storybook/plugin` does not set `continuous:true` for serve-like
targets.

## Expected Behavior
The plugin should correctly set `continuous: true` for `serve` and
`serve-static`.
2025-04-09 19:30:05 -04:00
Colum Ferry a5247df4bf feat(vite): infer continuous tasks for dev (#30043)
## Current Behavior
The `dev`, `serve`, `preview` and `serve-static` targets inferred by the
`@nx/vite/plugin` do not infer `continuous:true` indicating to Nx that
these tasks are continuous.


## Expected Behavior
Infer `continuous: true` for the serve-like targets for Vite.
2025-04-09 19:30:05 -04:00
Colum Ferry 9569a82514 feat(webpack): add continuous inference support (#29974)
## Current Behavior
Webpack Inference Plugin does not currently infer `continuous` for
`serve, preview, serve-static` targets.

## Expected Behavior
Webpack Inference Plugin correctly infers `continuous: true`
2025-04-09 19:30:05 -04:00
Colum Ferry bf824771e8 feat(rspack): add continous inference support (#29973)
## Current Behavior
Rspack and Rsbuild Inference Plugins do not infer `continuous` for serve
tasks.

## Expected Behavior
Correctly infer `continuous` true.
2025-04-09 19:30:05 -04:00
Colum Ferry 2285a9a6b8 feat(js): infer continuous for watch-deps task (#29976)
## Current Behavior
The `@nx/js` plugin exposes a helper to generate `build-deps` and
`watch-deps` tasks for inference plugins.
It does not currently infer `continuous` for the `watch-deps` task.

## Expected Behavior
Ensure `watch-deps` is infered with `continuous: true`
2025-04-09 19:30:05 -04:00
Colum Ferry fa461de1ab feat(remix): add continuous task support (#29905)
## Current Behavior
When `serve, dev, start` targets are inferred by the `@nx/remix` plugin,
they are not inferring `continuous`.


## Expected Behavior
When `NX_CONTINUOUS_TASKS_PREVIEW` env var exists, infer the
`continuous` property.
2025-04-09 19:30:04 -04:00
Jason Jean 12ce994632 feat(core): introduce continuous tasks (#29750)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

An RFC about this feature is happening here: #29025. This has the most
information about this feature.

<!-- This is the behavior we have today -->

Nx currently does not explicitly handle tasks which run continuously
until they are terminated.

<!-- This is the behavior we should expect with the changes in this PR
-->

This PR adds the initial support for continuous tasks which run
continuously until they are terminated. This adds the ability to depend
on continuous tasks. There is some more work to be done but this will be
enough as an MVP.

<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2025-04-09 19:30:04 -04:00
122 changed files with 10539 additions and 1438 deletions
Generated
+728 -50
View File
File diff suppressed because it is too large Load Diff
+1
View File
@@ -106,6 +106,7 @@ Print the task graph to the console:
| `--skipRemoteCache`, `--disableRemoteCache` | boolean | Disables the remote cache. (Default: `false`) |
| `--skipSync` | boolean | Skips running the sync generators associated with the tasks. (Default: `false`) |
| `--targets`, `--target`, `--t` | string | Tasks to run for affected projects. |
| `--tuiAutoExit` | string | Whether or not to exit the TUI automatically after all tasks finish, and after how long. If set to `true`, the TUI will exit immediately. If set to `false` the TUI will not automatically exit. If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits. |
| `--uncommitted` | boolean | Uncommitted changes. |
| `--untracked` | boolean | Untracked changes. |
| `--verbose` | boolean | Prints additional information about the commands (e.g., stack traces). |
+1
View File
@@ -110,5 +110,6 @@ Print the task graph to the console:
| `--skipRemoteCache`, `--disableRemoteCache` | boolean | Disables the remote cache. (Default: `false`) |
| `--skipSync` | boolean | Skips running the sync generators associated with the tasks. (Default: `false`) |
| `--targets`, `--target`, `--t` | string | Tasks to run for affected projects. |
| `--tuiAutoExit` | string | Whether or not to exit the TUI automatically after all tasks finish, and after how long. If set to `true`, the TUI will exit immediately. If set to `false` the TUI will not automatically exit. If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits. |
| `--verbose` | boolean | Prints additional information about the commands (e.g., stack traces). |
| `--version` | boolean | Show version number. |
+1
View File
@@ -83,5 +83,6 @@ Run's a target named build:test for the myapp project. Note the quotes around th
| `--skipNxCache`, `--disableNxCache` | boolean | Rerun the tasks even when the results are available in the cache. (Default: `false`) |
| `--skipRemoteCache`, `--disableRemoteCache` | boolean | Disables the remote cache. (Default: `false`) |
| `--skipSync` | boolean | Skips running the sync generators associated with the tasks. (Default: `false`) |
| `--tuiAutoExit` | string | Whether or not to exit the TUI automatically after all tasks finish, and after how long. If set to `true`, the TUI will exit immediately. If set to `false` the TUI will not automatically exit. If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits. |
| `--verbose` | boolean | Prints additional information about the commands (e.g., stack traces). |
| `--version` | boolean | Show version number. |
@@ -42,6 +42,7 @@ Nx.json configuration
- [sync](../../devkit/documents/NxJsonConfiguration#sync): NxSyncConfiguration
- [targetDefaults](../../devkit/documents/NxJsonConfiguration#targetdefaults): TargetDefaults
- [tasksRunnerOptions](../../devkit/documents/NxJsonConfiguration#tasksrunneroptions): Object
- [tui](../../devkit/documents/NxJsonConfiguration#tui): Object
- [useDaemonProcess](../../devkit/documents/NxJsonConfiguration#usedaemonprocess): boolean
- [useInferencePlugins](../../devkit/documents/NxJsonConfiguration#useinferenceplugins): boolean
- [useLegacyCache](../../devkit/documents/NxJsonConfiguration#uselegacycache): boolean
@@ -289,6 +290,21 @@ Available Task Runners for Nx to use
---
### tui
`Optional` **tui**: `Object`
Settings for the Nx Terminal User Interface (TUI)
#### Type declaration
| Name | Type | Description |
| :---------- | :-------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `autoExit?` | `number` \| `boolean` | Whether to exit the TUI automatically after all tasks finish. - If set to `true`, the TUI will exit immediately. - If set to `false` the TUI will not automatically exit. - If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits. |
| `enabled?` | `boolean` | Whether to enable the TUI whenever possible (based on the current environment and terminal). |
---
### useDaemonProcess
`Optional` **useDaemonProcess**: `boolean`
@@ -15,6 +15,7 @@ Target's configuration
- [cache](../../devkit/documents/TargetConfiguration#cache): boolean
- [command](../../devkit/documents/TargetConfiguration#command): string
- [configurations](../../devkit/documents/TargetConfiguration#configurations): Object
- [continuous](../../devkit/documents/TargetConfiguration#continuous): boolean
- [defaultConfiguration](../../devkit/documents/TargetConfiguration#defaultconfiguration): string
- [dependsOn](../../devkit/documents/TargetConfiguration#dependson): (string | TargetDependencyConfig)[]
- [executor](../../devkit/documents/TargetConfiguration#executor): string
@@ -55,6 +56,14 @@ Sets of options
---
### continuous
`Optional` **continuous**: `boolean`
Whether this target runs continuously
---
### defaultConfiguration
`Optional` **defaultConfiguration**: `string`
+9
View File
@@ -7,6 +7,7 @@ A representation of the invocation of an Executor
### Properties
- [cache](../../devkit/documents/Task#cache): boolean
- [continuous](../../devkit/documents/Task#continuous): boolean
- [endTime](../../devkit/documents/Task#endtime): number
- [hash](../../devkit/documents/Task#hash): string
- [hashDetails](../../devkit/documents/Task#hashdetails): Object
@@ -28,6 +29,14 @@ Determines if a given task should be cacheable.
---
### continuous
`Optional` **continuous**: `boolean`
This denotes if the task runs continuously
---
### endTime
`Optional` **endTime**: `number`
+7
View File
@@ -6,12 +6,19 @@ Graph of Tasks to be executed
### Properties
- [continuousDependencies](../../devkit/documents/TaskGraph#continuousdependencies): Record<string, string[]>
- [dependencies](../../devkit/documents/TaskGraph#dependencies): Record<string, string[]>
- [roots](../../devkit/documents/TaskGraph#roots): string[]
- [tasks](../../devkit/documents/TaskGraph#tasks): Record<string, Task>
## Properties
### continuousDependencies
**continuousDependencies**: `Record`\<`string`, `string`[]\>
---
### dependencies
**dependencies**: `Record`\<`string`, `string`[]\>
+20
View File
@@ -41,6 +41,7 @@ use ProjectsConfigurations or NxJsonConfiguration
- [sync](../../devkit/documents/Workspace#sync): NxSyncConfiguration
- [targetDefaults](../../devkit/documents/Workspace#targetdefaults): TargetDefaults
- [tasksRunnerOptions](../../devkit/documents/Workspace#tasksrunneroptions): Object
- [tui](../../devkit/documents/Workspace#tui): Object
- [useDaemonProcess](../../devkit/documents/Workspace#usedaemonprocess): boolean
- [useInferencePlugins](../../devkit/documents/Workspace#useinferenceplugins): boolean
- [useLegacyCache](../../devkit/documents/Workspace#uselegacycache): boolean
@@ -397,6 +398,25 @@ Available Task Runners for Nx to use
---
### tui
`Optional` **tui**: `Object`
Settings for the Nx Terminal User Interface (TUI)
#### Type declaration
| Name | Type | Description |
| :---------- | :-------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `autoExit?` | `number` \| `boolean` | Whether to exit the TUI automatically after all tasks finish. - If set to `true`, the TUI will exit immediately. - If set to `false` the TUI will not automatically exit. - If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits. |
| `enabled?` | `boolean` | Whether to enable the TUI whenever possible (based on the current environment and terminal). |
#### Inherited from
[NxJsonConfiguration](../../devkit/documents/NxJsonConfiguration).[tui](../../devkit/documents/NxJsonConfiguration#tui)
---
### useDaemonProcess
`Optional` **useDaemonProcess**: `boolean`
@@ -106,6 +106,7 @@ Print the task graph to the console:
| `--skipRemoteCache`, `--disableRemoteCache` | boolean | Disables the remote cache. (Default: `false`) |
| `--skipSync` | boolean | Skips running the sync generators associated with the tasks. (Default: `false`) |
| `--targets`, `--target`, `--t` | string | Tasks to run for affected projects. |
| `--tuiAutoExit` | string | Whether or not to exit the TUI automatically after all tasks finish, and after how long. If set to `true`, the TUI will exit immediately. If set to `false` the TUI will not automatically exit. If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits. |
| `--uncommitted` | boolean | Uncommitted changes. |
| `--untracked` | boolean | Untracked changes. |
| `--verbose` | boolean | Prints additional information about the commands (e.g., stack traces). |
@@ -110,5 +110,6 @@ Print the task graph to the console:
| `--skipRemoteCache`, `--disableRemoteCache` | boolean | Disables the remote cache. (Default: `false`) |
| `--skipSync` | boolean | Skips running the sync generators associated with the tasks. (Default: `false`) |
| `--targets`, `--target`, `--t` | string | Tasks to run for affected projects. |
| `--tuiAutoExit` | string | Whether or not to exit the TUI automatically after all tasks finish, and after how long. If set to `true`, the TUI will exit immediately. If set to `false` the TUI will not automatically exit. If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits. |
| `--verbose` | boolean | Prints additional information about the commands (e.g., stack traces). |
| `--version` | boolean | Show version number. |
@@ -83,5 +83,6 @@ Run's a target named build:test for the myapp project. Note the quotes around th
| `--skipNxCache`, `--disableNxCache` | boolean | Rerun the tasks even when the results are available in the cache. (Default: `false`) |
| `--skipRemoteCache`, `--disableRemoteCache` | boolean | Disables the remote cache. (Default: `false`) |
| `--skipSync` | boolean | Skips running the sync generators associated with the tasks. (Default: `false`) |
| `--tuiAutoExit` | string | Whether or not to exit the TUI automatically after all tasks finish, and after how long. If set to `true`, the TUI will exit immediately. If set to `false` the TUI will not automatically exit. If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits. |
| `--verbose` | boolean | Prints additional information about the commands (e.g., stack traces). |
| `--version` | boolean | Show version number. |
@@ -113,7 +113,13 @@ The `host` is the entry point and the `remotes` are modules used by the applicat
To support this, as well as to ensure a great local DX, we built our Module Federation support in such a way that when developing locally you should always run `serve` on your `host` application. This will start up your full Module Federation architecture; serving your `host` with `webpack-dev-server` and each `remote` via a single `http-server`. You can learn more about this on our [Nx Module Federation Technical Overview](/concepts/module-federation/nx-module-federation-technical-overview).
When you're working on a specific `remote` application, you should use the `--devRemotes` option to specify the `remote` you are currently developing; e.g. `nx serve host --devRemotes=remote1`. This ensures that the `remote` is served via `webpack-dev-server` allowing for HMR and live reloading.
With the introduction of Continuous Tasks in Nx 21 when you're working on a specific `remote` application, you now only need to run `nx serve remote` and it will serve the application along with your `host` application.
{% callout type="note" title="Continuous Tasks Support in Module Federation" %}
This is currently only supported for Rspack Module Federation using the `@nx/rspack/plugin` Inference Plugin.
{% /callout %}
If you are using Webpack Module Federation, or are not using [Inferred Tasks](/concepts/inferred-tasks), you should use the `--devRemotes` option to specify the `remote` you are currently developing; e.g. `nx serve host --devRemotes=remote1`. This ensures that the `remote` is served via `webpack-dev-server` allowing for HMR and live reloading.
## Use Cases
@@ -26,13 +26,26 @@ The executor does the following:
5. It will run `http-server` at the common directory such that those files are available on the network from a single port.
6. It will create proxy servers via `express` listening on the ports where each `remote` _should_ be located (as configured in the host's `module-federation.config.ts` or `module-federation.manifest.json` file).
- These proxy servers will proxy requests from the server to the `http-server` to fetch the correct files as requested by Module Federation.
7. If the `--devRemotes` option has been passed, it will serve each `dev remote` via `webpack-dev-server` allowing for HMR and live reloading when working on those remotes.
7. **Only Applicable for Executor Usage**: If the `--devRemotes` option has been passed, it will serve each `dev remote` via `webpack-dev-server` allowing for HMR and live reloading when working on those remotes.
8. It will serve the `host` via `webpack-dev-server`.
If you prefer diagrams, the one below outlines the above steps.
![Nx Module Federation Host Serve Flow](/shared/concepts/module-federation/module-federation-host-serve-light.png)
## Using Module Federation with Continuous Tasks
Continuous Tasks are a new feature in Nx 21. Using Rspack Module Federation, you can now use Continuous Tasks to serve your `remotes` and `host` application.
Thanks to the benefits of Continuous Tasks, you no longer need to run `nx serve host --devRemotes=remote` to serve your `host` application with HMR enabled for your remote applications.
Instead, you can run `nx serve remote` and it will serve the `remote` along with your `host` application with HMR enabled.
This is a great way to develop your application locally and have a great DX. It also makes it easier to explain to your team how to work with Module Federation as there is no longer any special command required to serve their application.
{% callout type="note" title="Using Continuous Tasks with Webpack Module Federation?" %}
Webpack Module Federation does not support Continuous Tasks. If you are using Webpack Module Federation, you should use the `--devRemotes` option to specify the `remote` you are currently developing; e.g. `npx nx serve host --devRemotes=remote`.
{% /callout %}
## The `NxRuntimeLibraryControlPlugin`
Previously, when using shared workspace libraries as part of your Module Federation application, there was a chance that the workspace library would be provided by one of the `static remotes`. This would cause issues where changes to those shared libraries would not be reflected in the locally served application.
-4
View File
@@ -81,7 +81,6 @@ describe('env vars', () => {
`e2e ${myapp}-e2e --config \\'{\\"env\\":{\\"cliArg\\":\\"i am from the cli args\\"}}\\'`
);
expect(run1).toContain('All specs passed!');
await killPort(4200);
// tests should not fail because of a config change
updateFile(
`apps/${myapp}-e2e/cypress.config.ts`,
@@ -114,7 +113,6 @@ export default defineConfig({
`e2e ${myapp}-e2e --config \\'{\\"env\\":{\\"cliArg\\":\\"i am from the cli args\\"}}\\'`
);
expect(run2).toContain('All specs passed!');
await killPort(4200);
// make sure project.json env vars also work
checkFilesExist(`apps/${myapp}-e2e/src/e2e/env.cy.ts`);
@@ -143,8 +141,6 @@ export default defineConfig({
);
const run3 = runCLI(`e2e ${myapp}-e2e`);
expect(run3).toContain('All specs passed!');
expect(await killPort(4200)).toBeTruthy();
}
},
TEN_MINS_MS
+3 -1
View File
@@ -193,7 +193,9 @@ describe('Node Applications + webpack', () => {
return config;
});
runCLI(`serve ${nodeApp1} --watch=false`);
await runCommandUntil(`serve ${nodeApp1} `, (output) =>
output.includes('Hello World')
);
checkFilesExist(`dist/apps/${nodeApp1}/main.js`);
checkFilesExist(`dist/apps/${nodeApp2}/main.js`);
+1
View File
@@ -42,6 +42,7 @@
"@phenomnomnominal/tsquery": "~5.0.1",
"detect-port": "^1.5.1",
"semver": "^7.6.3",
"tree-kill": "1.2.2",
"tslib": "^2.3.0"
},
"peerDependencies": {
+11 -5
View File
@@ -8,6 +8,7 @@ import { dirname, join, relative } from 'path';
import type { InlineConfig } from 'vite';
import vitePreprocessor from '../src/plugins/preprocessor-vite';
import { NX_PLUGIN_OPTIONS } from '../src/utils/constants';
import * as treeKill from 'tree-kill';
// Importing the cypress type here causes the angular and next unit
// tests to fail when transpiling, it seems like the cypress types are
@@ -79,7 +80,7 @@ function startWebServer(webServerCommand: string) {
windowsHide: false,
});
return () => {
return async () => {
if (process.platform === 'win32') {
try {
execSync('taskkill /pid ' + serverProcess.pid + ' /T /F', {
@@ -91,9 +92,14 @@ function startWebServer(webServerCommand: string) {
}
}
} else {
// child.kill() does not work on linux
// process.kill will kill the whole process group on unix
process.kill(-serverProcess.pid, 'SIGKILL');
return new Promise<void>((res, rej) => {
treeKill(serverProcess.pid, (err) => {
if (err) {
rej(err);
}
res();
});
});
}
};
}
@@ -172,7 +178,7 @@ export function nxE2EPreset(
const killWebServer = startWebServer(webServerCommand);
on('after:run', () => {
killWebServer();
return killWebServer();
});
await waitForServer(config.baseUrl, options.webServerConfig);
}
@@ -229,6 +229,7 @@ describe('app', () => {
"buildTarget": "@proj/myapp:build:production",
},
},
"continuous": true,
"defaultConfiguration": "development",
"dependsOn": [
"build",
@@ -397,6 +398,7 @@ describe('app', () => {
"buildTarget": "@proj/myapp:build:production",
},
},
"continuous": true,
"defaultConfiguration": "development",
"dependsOn": [
"build",
@@ -2893,6 +2893,7 @@ describe(`Plugin: ${PLUGIN_NAME}`, () => {
},
"watch-deps": {
"command": "npx nx watch --projects my-lib --includeDependentProjects -- npx nx build-deps my-lib",
"continuous": true,
"dependsOn": [
"build-deps",
],
@@ -3047,6 +3048,7 @@ describe(`Plugin: ${PLUGIN_NAME}`, () => {
},
"watch-deps": {
"command": "npx nx watch --projects my-lib --includeDependentProjects -- npx nx build-deps my-lib",
"continuous": true,
"dependsOn": [
"build-deps",
],
@@ -47,6 +47,7 @@ export function addBuildAndWatchDepsTargets(
dependsOn: ['^build'],
};
targets[options.watchDepsTargetName ?? 'watch-deps'] = {
continuous: true,
dependsOn: [buildDepsTargetName],
command: `${pmc.exec} nx watch --projects ${projectName} --includeDependentProjects -- ${pmc.exec} nx ${buildDepsTargetName} ${projectName}`,
};
@@ -429,6 +429,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
'lib3:build': [],
'lib4:build': [],
},
continuousDependencies: {},
roots: [],
tasks: {
'lib1:build': {
@@ -437,6 +438,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib1', target: 'build' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib2:build': {
id: 'lib2:build',
@@ -444,6 +446,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib2', target: 'build' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib2:build-base': {
id: 'lib2:build-base',
@@ -451,6 +454,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib2', target: 'build-base' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib3:build': {
id: 'lib3:build',
@@ -458,6 +462,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib3', target: 'build' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib4:build': {
id: 'lib4:build',
@@ -465,6 +470,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib4', target: 'build' },
outputs: [],
parallelism: true,
continuous: false,
},
},
};
@@ -604,6 +610,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
'lib4:build': ['lib4:build-base'],
'lib4:build-base': [],
},
continuousDependencies: {},
roots: [],
tasks: {
'lib1:build': {
@@ -612,6 +619,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib1', target: 'build' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib1:build-base': {
id: 'lib1:build-base',
@@ -619,6 +627,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib1', target: 'build-base' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib2:build': {
id: 'lib2:build',
@@ -626,6 +635,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib2', target: 'build' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib2:build-base': {
id: 'lib2:build-base',
@@ -633,6 +643,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib2', target: 'build-base' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib3:build': {
id: 'lib3:build',
@@ -640,6 +651,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib3', target: 'build' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib3:build-base': {
id: 'lib3:build-base',
@@ -647,6 +659,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib3', target: 'build-base' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib4:build': {
id: 'lib4:build',
@@ -654,6 +667,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib4', target: 'build' },
outputs: [],
parallelism: true,
continuous: false,
},
'lib4:build-base': {
id: 'lib4:build-base',
@@ -661,6 +675,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
target: { project: 'lib4', target: 'build-base' },
outputs: [],
parallelism: true,
continuous: false,
},
},
};
@@ -752,6 +767,7 @@ describe('calculateDependenciesFromTaskGraph', () => {
// not relevant for this test case
const taskGraph: TaskGraph = {
dependencies: {},
continuousDependencies: {},
roots: [],
tasks: {},
};
@@ -62,6 +62,7 @@ describe('application generator', () => {
"buildTarget": "my-node-app:build:production",
},
},
"continuous": true,
"defaultConfiguration": "development",
"dependsOn": [
"build",
@@ -242,6 +243,7 @@ describe('application generator', () => {
"buildTarget": "@proj/myapp:build:production",
},
},
"continuous": true,
"defaultConfiguration": "development",
"dependsOn": [
"build",
@@ -407,6 +409,7 @@ describe('application generator', () => {
"buildTarget": "@proj/myapp:build:production",
},
},
"continuous": true,
"defaultConfiguration": "development",
"dependsOn": [
"build",
@@ -64,6 +64,7 @@ exports[`@nx/next/plugin integrated projects should create nodes 1`] = `
},
"watch-deps": {
"command": "npx nx watch --projects my-app --includeDependentProjects -- npx nx build-deps my-app",
"continuous": true,
"dependsOn": [
"build-deps",
],
@@ -140,6 +141,7 @@ exports[`@nx/next/plugin root projects should create nodes 1`] = `
},
"watch-deps": {
"command": "npx nx watch --projects next --includeDependentProjects -- npx nx build-deps next",
"continuous": true,
"dependsOn": [
"build-deps",
],
@@ -51,6 +51,7 @@ describe('app', () => {
"buildTarget": "my-node-app:build:production",
},
},
"continuous": true,
"defaultConfiguration": "development",
"dependsOn": [
"build",
@@ -263,6 +264,7 @@ describe('app', () => {
"buildTarget": "my-node-app:build:production",
},
},
"continuous": true,
"defaultConfiguration": "development",
"dependsOn": [
"build",
@@ -626,6 +628,7 @@ describe('app', () => {
"buildTarget": "@proj/myapp:build:production",
},
},
"continuous": true,
"defaultConfiguration": "development",
"dependsOn": [
"build",
@@ -896,6 +899,7 @@ describe('app', () => {
"buildTarget": "@proj/myapp:build:production",
},
},
"continuous": true,
"defaultConfiguration": "development",
"dependsOn": [
"build",
@@ -146,6 +146,7 @@ function getEsBuildConfig(
function getServeConfig(options: NormalizedSchema): TargetConfiguration {
return {
continuous: true,
executor: '@nx/js:node',
defaultConfiguration: 'development',
// Run build, which includes dependency on "^build" by default, so the first run
@@ -298,10 +299,18 @@ function addAppFiles(tree: Tree, options: NormalizedSchema) {
function addProxy(tree: Tree, options: NormalizedSchema) {
const projectConfig = readProjectConfiguration(tree, options.frontendProject);
if (projectConfig.targets && projectConfig.targets.serve) {
if (
projectConfig.targets &&
['serve', 'dev'].find((t) => !!projectConfig.targets[t])
) {
const targetName = ['serve', 'dev'].find((t) => !!projectConfig.targets[t]);
projectConfig.targets[targetName].dependsOn = [
...(projectConfig.targets[targetName].dependsOn ?? []),
`${options.name}:serve`,
];
const pathToProxyFile = `${projectConfig.root}/proxy.conf.json`;
projectConfig.targets.serve.options = {
...projectConfig.targets.serve.options,
projectConfig.targets[targetName].options = {
...projectConfig.targets[targetName].options,
proxyConfig: pathToProxyFile,
};
@@ -74,6 +74,7 @@ exports[`@nx/nuxt/plugin not root project should create nodes 1`] = `
},
"watch-deps": {
"command": "npx nx watch --projects my-app --includeDependentProjects -- npx nx build-deps my-app",
"continuous": true,
"dependsOn": [
"build-deps",
],
@@ -158,6 +159,7 @@ exports[`@nx/nuxt/plugin root project should create nodes 1`] = `
},
"watch-deps": {
"command": "npx nx watch --projects nuxt --includeDependentProjects -- npx nx build-deps nuxt",
"continuous": true,
"dependsOn": [
"build-deps",
],
+24 -14
View File
@@ -13,50 +13,62 @@ strip = "none"
[dependencies]
anyhow = "1.0.71"
arboard = "3.4.1"
better-panic = "0.3.0"
colored = "2"
color-eyre = "0.6.3"
crossbeam-channel = '0.5'
dashmap = { version = "5.5.3", features = ["rayon"] }
dunce = "1"
flate2 = "1.1.1"
fs_extra = "1.3.0"
futures = "0.3.28"
globset = "0.4.10"
hashbrown = { version = "0.14.5", features = ["rayon", "rkyv"] }
ignore = '0.4'
itertools = "0.10.5"
once_cell = "1.18.0"
parking_lot = { version = "0.12.1", features = ["send_guard"] }
napi = { version = '2.16.0', default-features = false, features = [
'anyhow',
'napi4',
'tokio_rt',
napi = { version = "2.16.0", default-features = false, features = [
"anyhow",
"napi4",
"tokio_rt",
"async",
"chrono_date",
] }
napi-derive = '2.16.0'
nom = '7.1.3'
regex = "1.9.1"
ratatui = { version = "0.29", features = ["scrolling-regions"] }
rayon = "1.7.0"
rkyv = { version = "0.7", features = ["validation"] }
tar = "0.4.44"
thiserror = "1.0.40"
tracing = "0.1.37"
tracing-subscriber = { version = "0.3.17", features = ["env-filter"] }
walkdir = '2.3.3'
xxhash-rust = { version = '0.8.5', features = ['xxh3', 'xxh64'] }
swc_common = "0.31.16"
swc_ecma_parser = { version = "0.137.1", features = ["typescript"] }
swc_ecma_visit = "0.93.0"
swc_ecma_ast = "0.107.0"
sysinfo = "0.33.1"
rand = "0.9.0"
tar = "0.4.44"
thiserror = "1.0.40"
tracing = "0.1.37"
tracing-subscriber = { version = "0.3.17", features = ["env-filter"] }
tokio = { version = "1.32.0", features = ["full"] }
tokio-util = "0.7.9"
tracing-appender = "0.2"
tui-term = "0.2.0"
walkdir = '2.3.3'
xxhash-rust = { version = '0.8.5', features = ['xxh3', 'xxh64'] }
vt100-ctt = { git = "https://github.com/JamesHenry/vt100-rust", rev = "1de895505fe9f697aadac585e4075b8fb45c880d" }
[target.'cfg(windows)'.dependencies]
winapi = { version = "0.3", features = ["fileapi"] }
[target.'cfg(all(not(windows), not(target_family = "wasm")))'.dependencies]
mio = "0.8"
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
crossterm = { version = "0.27.0", features = ["event-stream"] }
portable-pty = { git = "https://github.com/cammisuli/wezterm", rev = "b538ee29e1e89eeb4832fb35ae095564dce34c29" }
crossterm = "0.27.0"
ignore-files = "2.1.0"
fs4 = "0.12.0"
reqwest = "0.12.15"
@@ -78,5 +90,3 @@ assert_fs = "1.0.10"
# This is only used for unit tests
swc_ecma_dep_graph = "0.109.1"
tempfile = "3.13.0"
# We only explicitly use tokio for async tests
tokio = "1.38.0"
+1
View File
@@ -66,6 +66,7 @@
"string-width": "^4.2.3",
"tar-stream": "~2.2.0",
"tmp": "~0.2.1",
"tree-kill": "^1.2.2",
"tsconfig-paths": "^4.1.2",
"tslib": "^2.3.0",
"yaml": "^2.6.0",
+17
View File
@@ -77,6 +77,23 @@
"$ref": "#/definitions/plugins"
}
},
"tui": {
"type": "object",
"description": "Settings for the Nx Terminal User Interface (TUI)",
"properties": {
"enabled": {
"type": "boolean",
"description": "Whether to enable the Terminal UI whenever possible (based on the current environment and terminal).",
"default": true
},
"autoExit": {
"oneOf": [{ "type": "boolean" }, { "type": "number" }],
"description": "Whether to exit the TUI automatically after all tasks finish. If set to `true`, the TUI will exit immediately. If set to `false` the TUI will not automatically exit. If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits.",
"default": 3
}
},
"additionalProperties": false
},
"defaultProject": {
"type": "string",
"description": "Default project. When project isn't provided, the default project will be used."
+1
View File
@@ -83,6 +83,7 @@ export const allowedWorkspaceExtensions = [
'sync',
'useLegacyCache',
'maxCacheSize',
'tui',
] as const;
if (!patched) {
@@ -1,4 +1,5 @@
import { CommandModule } from 'yargs';
import { handleErrors } from '../../utils/handle-errors';
import { linkToNxDevAndExamples } from '../yargs-utils/documentation';
import {
withAffectedOptions,
@@ -8,8 +9,8 @@ import {
withOverrides,
withRunOptions,
withTargetAndConfigurationOption,
withTuiOptions,
} from '../yargs-utils/shared-options';
import { handleErrors } from '../../utils/handle-errors';
export const yargsAffectedCommand: CommandModule = {
command: 'affected',
@@ -17,9 +18,11 @@ export const yargsAffectedCommand: CommandModule = {
builder: (yargs) =>
linkToNxDevAndExamples(
withAffectedOptions(
withRunOptions(
withOutputStyleOption(
withTargetAndConfigurationOption(withBatch(yargs))
withTuiOptions(
withRunOptions(
withOutputStyleOption(
withTargetAndConfigurationOption(withBatch(yargs))
)
)
)
)
@@ -56,7 +59,9 @@ export const yargsAffectedTestCommand: CommandModule = {
builder: (yargs) =>
linkToNxDevAndExamples(
withAffectedOptions(
withRunOptions(withOutputStyleOption(withConfiguration(yargs)))
withTuiOptions(
withRunOptions(withOutputStyleOption(withConfiguration(yargs)))
)
),
'affected'
),
@@ -80,7 +85,9 @@ export const yargsAffectedBuildCommand: CommandModule = {
builder: (yargs) =>
linkToNxDevAndExamples(
withAffectedOptions(
withRunOptions(withOutputStyleOption(withConfiguration(yargs)))
withTuiOptions(
withRunOptions(withOutputStyleOption(withConfiguration(yargs)))
)
),
'affected'
),
@@ -104,7 +111,9 @@ export const yargsAffectedLintCommand: CommandModule = {
builder: (yargs) =>
linkToNxDevAndExamples(
withAffectedOptions(
withRunOptions(withOutputStyleOption(withConfiguration(yargs)))
withTuiOptions(
withRunOptions(withOutputStyleOption(withConfiguration(yargs)))
)
),
'affected'
),
@@ -128,7 +137,9 @@ export const yargsAffectedE2ECommand: CommandModule = {
builder: (yargs) =>
linkToNxDevAndExamples(
withAffectedOptions(
withRunOptions(withOutputStyleOption(withConfiguration(yargs)))
withTuiOptions(
withRunOptions(withOutputStyleOption(withConfiguration(yargs)))
)
),
'affected'
),
@@ -2,12 +2,13 @@ import { CommandModule } from 'yargs';
import {
withOverrides,
withRunManyOptions,
withTuiOptions,
} from '../yargs-utils/shared-options';
export const yargsExecCommand: CommandModule = {
command: 'exec',
describe: 'Executes any command as if it was a target on the project.',
builder: (yargs) => withRunManyOptions(yargs),
builder: (yargs) => withTuiOptions(withRunManyOptions(yargs)),
handler: async (args) => {
try {
await (await import('./exec')).nxExecCommand(withOverrides(args) as any);
@@ -980,6 +980,7 @@ function getAllTaskGraphsForWorkspace(projectGraph: ProjectGraph): {
taskGraphs[taskId] = {
tasks: {},
dependencies: {},
continuousDependencies: {},
roots: [],
};
@@ -1006,6 +1007,7 @@ function getAllTaskGraphsForWorkspace(projectGraph: ProjectGraph): {
taskGraphs[taskId] = {
tasks: {},
dependencies: {},
continuousDependencies: {},
roots: [],
};
@@ -268,7 +268,9 @@ async function runPublishOnProjects(
/**
* Run the relevant nx-release-publish executor on each of the selected projects.
* NOTE: Force TUI to be disabled for now.
*/
process.env.NX_TUI = 'false';
const commandResults = await runCommandForTasks(
projectsWithTarget,
projectGraph,
@@ -1,22 +1,25 @@
import { CommandModule } from 'yargs';
import { handleErrors } from '../../utils/handle-errors';
import { linkToNxDevAndExamples } from '../yargs-utils/documentation';
import {
withRunManyOptions,
withOutputStyleOption,
withTargetAndConfigurationOption,
withOverrides,
withBatch,
withOutputStyleOption,
withOverrides,
withRunManyOptions,
withTargetAndConfigurationOption,
withTuiOptions,
} from '../yargs-utils/shared-options';
import { handleErrors } from '../../utils/handle-errors';
export const yargsRunManyCommand: CommandModule = {
command: 'run-many',
describe: 'Run target for multiple listed projects.',
builder: (yargs) =>
linkToNxDevAndExamples(
withRunManyOptions(
withOutputStyleOption(
withTargetAndConfigurationOption(withBatch(yargs))
withTuiOptions(
withRunManyOptions(
withOutputStyleOption(
withTargetAndConfigurationOption(withBatch(yargs))
)
)
),
'run-many'
@@ -1,10 +1,11 @@
import { CommandModule, showHelp } from 'yargs';
import { handleErrors } from '../../utils/handle-errors';
import {
withBatch,
withOverrides,
withRunOneOptions,
withTuiOptions,
} from '../yargs-utils/shared-options';
import { handleErrors } from '../../utils/handle-errors';
export const yargsRunCommand: CommandModule = {
command: 'run [project][:target][:configuration] [_..]',
@@ -15,7 +16,7 @@ export const yargsRunCommand: CommandModule = {
(e.g., nx serve myapp --configuration=production)
You can skip the use of Nx cache by using the --skip-nx-cache option.`,
builder: (yargs) => withRunOneOptions(withBatch(yargs)),
builder: (yargs) => withTuiOptions(withRunOneOptions(withBatch(yargs))),
handler: async (args) => {
const exitCode = await handleErrors(
(args.verbose as boolean) ?? process.env.NX_VERBOSE_LOGGING === 'true',
+2 -2
View File
@@ -20,7 +20,7 @@ import {
} from '../../utils/async-iterator';
import { getExecutorInformation } from './executor-utils';
import {
getPseudoTerminal,
createPseudoTerminal,
PseudoTerminal,
} from '../../tasks-runner/pseudo-terminal';
import { exec } from 'child_process';
@@ -124,7 +124,7 @@ async function printTargetRunHelpInternal(
...localEnv,
};
if (PseudoTerminal.isSupported()) {
const terminal = getPseudoTerminal();
const terminal = createPseudoTerminal();
await new Promise(() => {
const cp = terminal.runCommand(helpCommand, { jsEnv: env });
cp.onExit((code) => {
@@ -41,6 +41,31 @@ export interface RunOptions {
skipSync: boolean;
}
export interface TuiOptions {
tuiAutoExit: boolean | number;
}
export function withTuiOptions<T>(yargs: Argv<T>): Argv<T & TuiOptions> {
return yargs.options('tuiAutoExit', {
describe:
'Whether or not to exit the TUI automatically after all tasks finish, and after how long. If set to `true`, the TUI will exit immediately. If set to `false` the TUI will not automatically exit. If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits.',
type: 'string',
coerce: (value) => {
if (value === 'true') {
return true;
}
if (value === 'false') {
return false;
}
const num = Number(value);
if (!Number.isNaN(num)) {
return num;
}
throw new Error(`Invalid value for --tui-auto-exit: ${value}`);
},
}) as Argv<T & TuiOptions>;
}
export function withRunOptions<T>(yargs: Argv<T>): Argv<T & RunOptions> {
return withVerbose(withExcludeOption(yargs))
.option('parallel', {
@@ -112,7 +137,6 @@ export function withRunOptions<T>(yargs: Argv<T>): Argv<T & RunOptions> {
type: 'boolean',
hidden: true,
})
.options('dte', {
type: 'boolean',
hidden: true,
@@ -1,5 +1,4 @@
import { ProjectGraph, ProjectGraphProjectNode } from '../config/project-graph';
import { removeIdsFromGraph } from '../tasks-runner/utils';
import { NxArgs } from '../utils/command-line-utils';
import { CommandGraph } from './command-graph';
import { createCommandGraph } from './create-command-graph';
@@ -34,3 +33,35 @@ function getSortedProjects(
return getSortedProjects(newGraph, sortedProjects);
}
function removeIdsFromGraph<T>(
graph: {
roots: string[];
dependencies: Record<string, string[]>;
},
ids: string[],
mapWithIds: Record<string, T>
): {
mapWithIds: Record<string, T>;
roots: string[];
dependencies: Record<string, string[]>;
} {
const filteredMapWithIds = {};
const dependencies = {};
const removedSet = new Set(ids);
for (let id of Object.keys(mapWithIds)) {
if (!removedSet.has(id)) {
filteredMapWithIds[id] = mapWithIds[id];
dependencies[id] = graph.dependencies[id].filter(
(depId) => !removedSet.has(depId)
);
}
}
return {
mapWithIds: filteredMapWithIds,
dependencies: dependencies,
roots: Object.keys(dependencies).filter(
(k) => dependencies[k].length === 0
),
};
}
+18
View File
@@ -654,6 +654,24 @@ export interface NxJsonConfiguration<T = '*' | string[]> {
* Sets the maximum size of the local cache. Accepts a number followed by a unit (e.g. 100MB). Accepted units are B, KB, MB, and GB.
*/
maxCacheSize?: string;
/**
* Settings for the Nx Terminal User Interface (TUI)
*/
tui?: {
/**
* Whether to enable the TUI whenever possible (based on the current environment and terminal).
*/
enabled?: boolean;
/**
* Whether to exit the TUI automatically after all tasks finish.
*
* - If set to `true`, the TUI will exit immediately.
* - If set to `false` the TUI will not automatically exit.
* - If set to a number, an interruptible countdown popup will be shown for that many seconds before the TUI exits.
*/
autoExit?: boolean | number;
};
}
export type PluginConfiguration = string | ExpandedPluginConfiguration;
+7
View File
@@ -81,6 +81,11 @@ export interface Task {
* Determines if a given task should be parallelizable.
*/
parallelism: boolean;
/**
* This denotes if the task runs continuously
*/
continuous?: boolean;
}
/**
@@ -99,4 +104,6 @@ export interface TaskGraph {
* Map of Task IDs to IDs of tasks which the task depends on
*/
dependencies: Record<string, string[]>;
continuousDependencies: Record<string, string[]>;
}
@@ -269,6 +269,11 @@ export interface TargetConfiguration<T = any> {
*/
parallelism?: boolean;
/**
* Whether this target runs continuously
*/
continuous?: boolean;
/**
* List of generators to run before the target to ensure the workspace
* is up to date.
@@ -1,54 +1,41 @@
import { ChildProcess, exec, Serializable } from 'child_process';
import * as path from 'path';
import { Serializable } from 'child_process';
import * as yargsParser from 'yargs-parser';
import { env as appendLocalEnv } from 'npm-run-path';
import { ExecutorContext } from '../../config/misc-interfaces';
import * as chalk from 'chalk';
import { isTuiEnabled } from '../../tasks-runner/is-tui-enabled';
import {
getPseudoTerminal,
createPseudoTerminal,
PseudoTerminal,
PseudoTtyProcess,
} from '../../tasks-runner/pseudo-terminal';
import { signalToCode } from '../../utils/exit-codes';
import {
loadAndExpandDotEnvFile,
unloadDotEnvFile,
} from '../../tasks-runner/task-env';
ParallelRunningTasks,
runSingleCommandWithPseudoTerminal,
SeriallyRunningTasks,
} from './running-tasks';
export const LARGE_BUFFER = 1024 * 1000000;
let pseudoTerminal: PseudoTerminal | null;
const childProcesses = new Set<ChildProcess | PseudoTtyProcess>();
function loadEnvVarsFile(path: string, env: Record<string, string> = {}) {
unloadDotEnvFile(path, env);
const result = loadAndExpandDotEnvFile(path, env);
if (result.error) {
throw result.error;
}
}
export type Json = {
[k: string]: any;
};
export interface RunCommandsCommandOptions {
command: string;
forwardAllArgs?: boolean;
/**
* description was added to allow users to document their commands inline,
* it is not intended to be used as part of the execution of the command.
*/
description?: string;
prefix?: string;
prefixColor?: string;
color?: string;
bgColor?: string;
}
export interface RunCommandsOptions extends Json {
command?: string | string[];
commands?: (
| {
command: string;
forwardAllArgs?: boolean;
/**
* description was added to allow users to document their commands inline,
* it is not intended to be used as part of the execution of the command.
*/
description?: string;
prefix?: string;
prefixColor?: string;
color?: string;
bgColor?: string;
}
| string
)[];
commands?: Array<RunCommandsCommandOptions | string>;
color?: boolean;
parallel?: boolean;
readyWhen?: string | string[];
@@ -84,10 +71,7 @@ const propKeys = [
];
export interface NormalizedRunCommandsOptions extends RunCommandsOptions {
commands: {
command: string;
forwardAllArgs?: boolean;
}[];
commands: Array<RunCommandsCommandOptions>;
unknownOptions?: {
[k: string]: any;
};
@@ -108,7 +92,18 @@ export default async function (
success: boolean;
terminalOutput: string;
}> {
registerProcessListener();
const task = await runCommands(options, context);
const results = await task.getResults();
return {
...results,
success: results.code === 0,
};
}
export async function runCommands(
options: RunCommandsOptions,
context: ExecutorContext
) {
const normalized = normalizeOptions(options);
if (normalized.readyWhenStatus.length && !normalized.parallel) {
@@ -128,11 +123,27 @@ export default async function (
);
}
const isSingleCommand = normalized.commands.length === 1;
const usePseudoTerminal =
(isSingleCommand || !options.parallel) && PseudoTerminal.isSupported();
const isSingleCommandAndCanUsePseudoTerminal =
isSingleCommand &&
usePseudoTerminal &&
process.env.NX_NATIVE_COMMAND_RUNNER !== 'false' &&
!normalized.commands[0].prefix &&
normalized.usePty;
const tuiEnabled = isTuiEnabled();
try {
const result = options.parallel
? await runInParallel(normalized, context)
: await runSerially(normalized, context);
return result;
const runningTask = isSingleCommandAndCanUsePseudoTerminal
? await runSingleCommandWithPseudoTerminal(normalized, context)
: options.parallel
? new ParallelRunningTasks(normalized, context, tuiEnabled)
: new SeriallyRunningTasks(normalized, context, tuiEnabled);
return runningTask;
} catch (e) {
if (process.env.NX_VERBOSE_LOGGING === 'true') {
console.error(e);
@@ -143,77 +154,6 @@ export default async function (
}
}
async function runInParallel(
options: NormalizedRunCommandsOptions,
context: ExecutorContext
): Promise<{ success: boolean; terminalOutput: string }> {
const procs = options.commands.map((c) =>
createProcess(
null,
c,
options.readyWhenStatus,
options.color,
calculateCwd(options.cwd, context),
options.env ?? {},
true,
options.usePty,
options.streamOutput,
options.tty,
options.envFile
).then((result: { success: boolean; terminalOutput: string }) => ({
result,
command: c.command,
}))
);
let terminalOutput = '';
if (options.readyWhenStatus.length) {
const r: {
result: { success: boolean; terminalOutput: string };
command: string;
} = await Promise.race(procs);
terminalOutput += r.result.terminalOutput;
if (!r.result.success) {
const output = `Warning: command "${r.command}" exited with non-zero status code`;
terminalOutput += output;
if (options.streamOutput) {
process.stderr.write(output);
}
return { success: false, terminalOutput };
} else {
return { success: true, terminalOutput };
}
} else {
const r: {
result: { success: boolean; terminalOutput: string };
command: string;
}[] = await Promise.all(procs);
terminalOutput += r.map((f) => f.result.terminalOutput).join('');
const failed = r.filter((v) => !v.result.success);
if (failed.length > 0) {
const output = failed
.map(
(f) =>
`Warning: command "${f.command}" exited with non-zero status code`
)
.join('\r\n');
terminalOutput += output;
if (options.streamOutput) {
process.stderr.write(output);
}
return {
success: false,
terminalOutput,
};
} else {
return {
success: true,
terminalOutput,
};
}
}
}
function normalizeOptions(
options: RunCommandsOptions
): NormalizedRunCommandsOptions {
@@ -279,241 +219,6 @@ function normalizeOptions(
return options as NormalizedRunCommandsOptions;
}
async function runSerially(
options: NormalizedRunCommandsOptions,
context: ExecutorContext
): Promise<{ success: boolean; terminalOutput: string }> {
pseudoTerminal ??= PseudoTerminal.isSupported() ? getPseudoTerminal() : null;
let terminalOutput = '';
for (const c of options.commands) {
const result: { success: boolean; terminalOutput: string } =
await createProcess(
pseudoTerminal,
c,
[],
options.color,
calculateCwd(options.cwd, context),
options.processEnv ?? options.env ?? {},
false,
options.usePty,
options.streamOutput,
options.tty,
options.envFile
);
terminalOutput += result.terminalOutput;
if (!result.success) {
const output = `Warning: command "${c.command}" exited with non-zero status code`;
result.terminalOutput += output;
if (options.streamOutput) {
process.stderr.write(output);
}
return { success: false, terminalOutput };
}
}
return { success: true, terminalOutput };
}
async function createProcess(
pseudoTerminal: PseudoTerminal | null,
commandConfig: {
command: string;
color?: string;
bgColor?: string;
prefix?: string;
prefixColor?: string;
},
readyWhenStatus: { stringToMatch: string; found: boolean }[] = [],
color: boolean,
cwd: string,
env: Record<string, string>,
isParallel: boolean,
usePty: boolean = true,
streamOutput: boolean = true,
tty: boolean,
envFile?: string
): Promise<{ success: boolean; terminalOutput: string }> {
env = processEnv(color, cwd, env, envFile);
// The rust runCommand is always a tty, so it will not look nice in parallel and if we need prefixes
// currently does not work properly in windows
if (
pseudoTerminal &&
process.env.NX_NATIVE_COMMAND_RUNNER !== 'false' &&
!commandConfig.prefix &&
readyWhenStatus.length === 0 &&
!isParallel &&
usePty
) {
let terminalOutput = chalk.dim('> ') + commandConfig.command + '\r\n\r\n';
if (streamOutput) {
process.stdout.write(terminalOutput);
}
const cp = pseudoTerminal.runCommand(commandConfig.command, {
cwd,
jsEnv: env,
quiet: !streamOutput,
tty,
});
childProcesses.add(cp);
return new Promise((res) => {
cp.onOutput((output) => {
terminalOutput += output;
});
cp.onExit((code) => {
if (code >= 128) {
process.exit(code);
} else {
res({ success: code === 0, terminalOutput });
}
});
});
}
return nodeProcess(commandConfig, cwd, env, readyWhenStatus, streamOutput);
}
function nodeProcess(
commandConfig: {
command: string;
color?: string;
bgColor?: string;
prefix?: string;
prefixColor?: string;
},
cwd: string,
env: Record<string, string>,
readyWhenStatus: { stringToMatch: string; found: boolean }[],
streamOutput = true
): Promise<{ success: boolean; terminalOutput: string }> {
let terminalOutput = chalk.dim('> ') + commandConfig.command + '\r\n\r\n';
if (streamOutput) {
process.stdout.write(terminalOutput);
}
return new Promise((res) => {
const childProcess = exec(commandConfig.command, {
maxBuffer: LARGE_BUFFER,
env,
cwd,
windowsHide: false,
});
childProcesses.add(childProcess);
childProcess.stdout.on('data', (data) => {
const output = addColorAndPrefix(data, commandConfig);
terminalOutput += output;
if (streamOutput) {
process.stdout.write(output);
}
if (readyWhenStatus.length && isReady(readyWhenStatus, data.toString())) {
res({ success: true, terminalOutput });
}
});
childProcess.stderr.on('data', (err) => {
const output = addColorAndPrefix(err, commandConfig);
terminalOutput += output;
if (streamOutput) {
process.stderr.write(output);
}
if (readyWhenStatus.length && isReady(readyWhenStatus, err.toString())) {
res({ success: true, terminalOutput });
}
});
childProcess.on('error', (err) => {
const ouptput = addColorAndPrefix(err.toString(), commandConfig);
terminalOutput += ouptput;
if (streamOutput) {
process.stderr.write(ouptput);
}
res({ success: false, terminalOutput });
});
childProcess.on('exit', (code) => {
childProcesses.delete(childProcess);
if (!readyWhenStatus.length || isReady(readyWhenStatus)) {
res({ success: code === 0, terminalOutput });
}
});
});
}
function addColorAndPrefix(
out: string,
config: {
prefix?: string;
prefixColor?: string;
color?: string;
bgColor?: string;
}
) {
if (config.prefix) {
out = out
.split('\n')
.map((l) => {
let prefixText = config.prefix;
if (config.prefixColor && chalk[config.prefixColor]) {
prefixText = chalk[config.prefixColor](prefixText);
}
prefixText = chalk.bold(prefixText);
return l.trim().length > 0 ? `${prefixText} ${l}` : l;
})
.join('\n');
}
if (config.color && chalk[config.color]) {
out = chalk[config.color](out);
}
if (config.bgColor && chalk[config.bgColor]) {
out = chalk[config.bgColor](out);
}
return out;
}
function calculateCwd(
cwd: string | undefined,
context: ExecutorContext
): string {
if (!cwd) return context.root;
if (path.isAbsolute(cwd)) return cwd;
return path.join(context.root, cwd);
}
/**
* Env variables are processed in the following order:
* - env option from executor options
* - env file from envFile option if provided
* - local env variables
*/
function processEnv(
color: boolean,
cwd: string,
envOptionFromExecutor: Record<string, string>,
envFile?: string
) {
let localEnv = appendLocalEnv({ cwd: cwd ?? process.cwd() });
localEnv = {
...process.env,
...localEnv,
};
if (process.env.NX_LOAD_DOT_ENV_FILES !== 'false' && envFile) {
loadEnvVarsFile(envFile, localEnv);
}
let res: Record<string, string> = {
...localEnv,
...envOptionFromExecutor,
};
// need to override PATH to make sure we are using the local node_modules
if (localEnv.PATH) res.PATH = localEnv.PATH; // UNIX-like
if (localEnv.Path) res.Path = localEnv.Path; // Windows
if (color) {
res.FORCE_COLOR = `${color}`;
}
return res;
}
export function interpolateArgsIntoCommand(
command: string,
opts: Pick<
@@ -629,65 +334,6 @@ function filterPropKeysFromUnParsedOptions(
return parsedOptions;
}
let registered = false;
function registerProcessListener() {
if (registered) {
return;
}
registered = true;
// When the nx process gets a message, it will be sent into the task's process
process.on('message', (message: Serializable) => {
// this.publisher.publish(message.toString());
if (pseudoTerminal) {
pseudoTerminal.sendMessageToChildren(message);
}
childProcesses.forEach((p) => {
if ('connected' in p && p.connected) {
p.send(message);
}
});
});
// Terminate any task processes on exit
process.on('exit', () => {
childProcesses.forEach((p) => {
if ('connected' in p ? p.connected : p.isAlive) {
p.kill();
}
});
});
process.on('SIGINT', () => {
childProcesses.forEach((p) => {
if ('connected' in p ? p.connected : p.isAlive) {
p.kill('SIGTERM');
}
});
// we exit here because we don't need to write anything to cache.
process.exit(signalToCode('SIGINT'));
});
process.on('SIGTERM', () => {
childProcesses.forEach((p) => {
if ('connected' in p ? p.connected : p.isAlive) {
p.kill('SIGTERM');
}
});
// no exit here because we expect child processes to terminate which
// will store results to the cache and will terminate this process
});
process.on('SIGHUP', () => {
childProcesses.forEach((p) => {
if ('connected' in p ? p.connected : p.isAlive) {
p.kill('SIGTERM');
}
});
// no exit here because we expect child processes to terminate which
// will store results to the cache and will terminate this process
});
}
function wrapArgIntoQuotesIfNeeded(arg: string): string {
if (arg.includes('=')) {
const [key, value] = arg.split('=');
@@ -705,19 +351,3 @@ function wrapArgIntoQuotesIfNeeded(arg: string): string {
return arg;
}
}
function isReady(
readyWhenStatus: { stringToMatch: string; found: boolean }[] = [],
data?: string
): boolean {
if (data) {
for (const readyWhenElement of readyWhenStatus) {
if (data.toString().indexOf(readyWhenElement.stringToMatch) > -1) {
readyWhenElement.found = true;
break;
}
}
}
return readyWhenStatus.every((readyWhenElement) => readyWhenElement.found);
}
@@ -0,0 +1,568 @@
import * as chalk from 'chalk';
import { ChildProcess, exec, Serializable } from 'child_process';
import { env as appendLocalEnv } from 'npm-run-path';
import { isAbsolute, join } from 'path';
import * as treeKill from 'tree-kill';
import { ExecutorContext } from '../../config/misc-interfaces';
import {
createPseudoTerminal,
PseudoTerminal,
PseudoTtyProcess,
} from '../../tasks-runner/pseudo-terminal';
import { RunningTask } from '../../tasks-runner/running-tasks/running-task';
import {
loadAndExpandDotEnvFile,
unloadDotEnvFile,
} from '../../tasks-runner/task-env';
import { signalToCode } from '../../utils/exit-codes';
import {
LARGE_BUFFER,
NormalizedRunCommandsOptions,
RunCommandsCommandOptions,
} from './run-commands.impl';
export class ParallelRunningTasks implements RunningTask {
private readonly childProcesses: RunningNodeProcess[];
private readyWhenStatus: { stringToMatch: string; found: boolean }[];
private readonly streamOutput: boolean;
private exitCallbacks: Array<(code: number, terminalOutput: string) => void> =
[];
constructor(
options: NormalizedRunCommandsOptions,
context: ExecutorContext,
private readonly tuiEnabled: boolean
) {
this.childProcesses = options.commands.map(
(commandConfig) =>
new RunningNodeProcess(
commandConfig,
options.color,
calculateCwd(options.cwd, context),
options.env ?? {},
options.readyWhenStatus,
options.streamOutput,
options.envFile
)
);
this.readyWhenStatus = options.readyWhenStatus;
this.streamOutput = options.streamOutput;
this.run();
}
async getResults(): Promise<{ code: number; terminalOutput: string }> {
return new Promise((res) => {
this.onExit((code, terminalOutput) => {
res({ code, terminalOutput });
});
});
}
onExit(cb: (code: number, terminalOutput: string) => void): void {
this.exitCallbacks.push(cb);
}
send(message: Serializable): void {
for (const childProcess of this.childProcesses) {
childProcess.send(message);
}
}
async kill(signal?: NodeJS.Signals | number) {
await Promise.all(
this.childProcesses.map(async (p) => {
try {
return p.kill();
} catch (e) {
console.error(`Unable to terminate "${p.command}"\nError:`, e);
}
})
);
}
private async run() {
if (this.readyWhenStatus.length) {
let {
childProcess,
result: { code, terminalOutput },
} = await Promise.race(
this.childProcesses.map(
(childProcess) =>
new Promise<{
childProcess: RunningNodeProcess;
result: { code: number; terminalOutput: string };
}>((res) => {
childProcess.onExit((code, terminalOutput) => {
res({
childProcess,
result: { code, terminalOutput },
});
});
})
)
);
if (code !== 0) {
const output = `Warning: command "${childProcess.command}" exited with non-zero status code`;
terminalOutput += output;
if (this.streamOutput) {
process.stderr.write(output);
}
}
for (const cb of this.exitCallbacks) {
cb(code, terminalOutput);
}
} else {
const results = await Promise.all(
this.childProcesses.map((childProcess) =>
childProcess.getResults().then((result) => ({
childProcess,
result,
}))
)
);
let terminalOutput = results
.map((r) => r.result.terminalOutput)
.join('\r\n');
const failed = results.filter((result) => result.result.code !== 0);
if (failed.length > 0) {
const output = failed
.map(
(failedResult) =>
`Warning: command "${failedResult.childProcess.command}" exited with non-zero status code`
)
.join('\r\n');
terminalOutput += output;
if (this.streamOutput) {
process.stderr.write(output);
}
for (const cb of this.exitCallbacks) {
cb(1, terminalOutput);
}
} else {
for (const cb of this.exitCallbacks) {
cb(0, terminalOutput);
}
}
}
}
}
export class SeriallyRunningTasks implements RunningTask {
private terminalOutput = '';
private currentProcess: RunningTask | PseudoTtyProcess | null = null;
private exitCallbacks: Array<(code: number, terminalOutput: string) => void> =
[];
private code: number | null = 0;
private error: any;
constructor(
options: NormalizedRunCommandsOptions,
context: ExecutorContext,
private readonly tuiEnabled: boolean
) {
this.run(options, context)
.catch((e) => {
this.error = e;
})
.finally(() => {
for (const cb of this.exitCallbacks) {
cb(this.code, this.terminalOutput);
}
});
}
getResults(): Promise<{ code: number; terminalOutput: string }> {
return new Promise((res, rej) => {
this.onExit((code) => {
if (this.error) {
rej(this.error);
} else {
res({ code, terminalOutput: this.terminalOutput });
}
});
});
}
onExit(cb: (code: number, terminalOutput: string) => void): void {
this.exitCallbacks.push(cb);
}
send(message: Serializable): void {
throw new Error('Not implemented');
}
kill(signal?: NodeJS.Signals | number) {
return this.currentProcess.kill(signal);
}
private async run(
options: NormalizedRunCommandsOptions,
context: ExecutorContext
) {
for (const c of options.commands) {
const childProcess = await this.createProcess(
c,
options.color,
calculateCwd(options.cwd, context),
options.processEnv ?? options.env ?? {},
options.usePty,
options.streamOutput,
options.tty,
options.envFile
);
this.currentProcess = childProcess;
let { code, terminalOutput } = await childProcess.getResults();
this.terminalOutput += terminalOutput;
this.code = code;
if (code !== 0) {
const output = `Warning: command "${c.command}" exited with non-zero status code`;
terminalOutput += output;
if (options.streamOutput) {
process.stderr.write(output);
}
this.terminalOutput += terminalOutput;
// Stop running commands
break;
}
}
}
private async createProcess(
commandConfig: RunCommandsCommandOptions,
color: boolean,
cwd: string,
env: Record<string, string>,
usePty: boolean = true,
streamOutput: boolean = true,
tty: boolean,
envFile?: string
): Promise<PseudoTtyProcess | RunningNodeProcess> {
// The rust runCommand is always a tty, so it will not look nice in parallel and if we need prefixes
// currently does not work properly in windows
if (
process.env.NX_NATIVE_COMMAND_RUNNER !== 'false' &&
!commandConfig.prefix &&
usePty &&
PseudoTerminal.isSupported()
) {
const pseudoTerminal = createPseudoTerminal();
registerProcessListener(this, pseudoTerminal);
return createProcessWithPseudoTty(
pseudoTerminal,
commandConfig,
color,
cwd,
env,
streamOutput,
tty,
envFile
);
}
return new RunningNodeProcess(
commandConfig,
color,
cwd,
env,
[],
streamOutput,
envFile
);
}
}
class RunningNodeProcess implements RunningTask {
private terminalOutput = '';
private childProcess: ChildProcess;
private exitCallbacks: Array<(code: number, terminalOutput: string) => void> =
[];
public command: string;
constructor(
commandConfig: RunCommandsCommandOptions,
color: boolean,
cwd: string,
env: Record<string, string>,
private readyWhenStatus: { stringToMatch: string; found: boolean }[],
streamOutput = true,
envFile: string
) {
env = processEnv(color, cwd, env, envFile);
this.command = commandConfig.command;
this.terminalOutput = chalk.dim('> ') + commandConfig.command + '\r\n\r\n';
if (streamOutput) {
process.stdout.write(this.terminalOutput);
}
this.childProcess = exec(commandConfig.command, {
maxBuffer: LARGE_BUFFER,
env,
cwd,
windowsHide: false,
});
this.addListeners(commandConfig, streamOutput);
}
getResults(): Promise<{ code: number; terminalOutput: string }> {
return new Promise((res) => {
this.onExit((code, terminalOutput) => {
res({ code, terminalOutput });
});
});
}
onExit(cb: (code: number, terminalOutput: string) => void): void {
this.exitCallbacks.push(cb);
}
send(message: Serializable): void {
this.childProcess.send(message);
}
kill(signal?: NodeJS.Signals | number): Promise<void> {
return new Promise<void>((res, rej) => {
treeKill(this.childProcess.pid, signal, (err) => {
if (err) {
rej(err);
} else {
res();
}
});
});
}
private addListeners(
commandConfig: RunCommandsCommandOptions,
streamOutput: boolean
) {
this.childProcess.stdout.on('data', (data) => {
const output = addColorAndPrefix(data, commandConfig);
this.terminalOutput += output;
if (streamOutput) {
process.stdout.write(output);
}
if (
this.readyWhenStatus.length &&
isReady(this.readyWhenStatus, data.toString())
) {
for (const cb of this.exitCallbacks) {
cb(0, this.terminalOutput);
}
}
});
this.childProcess.stderr.on('data', (err) => {
const output = addColorAndPrefix(err, commandConfig);
this.terminalOutput += output;
if (streamOutput) {
process.stderr.write(output);
}
if (
this.readyWhenStatus.length &&
isReady(this.readyWhenStatus, err.toString())
) {
for (const cb of this.exitCallbacks) {
cb(1, this.terminalOutput);
}
}
});
this.childProcess.on('error', (err) => {
const output = addColorAndPrefix(err.toString(), commandConfig);
this.terminalOutput += output;
if (streamOutput) {
process.stderr.write(output);
}
for (const cb of this.exitCallbacks) {
cb(1, this.terminalOutput);
}
});
this.childProcess.on('exit', (code) => {
if (!this.readyWhenStatus.length || isReady(this.readyWhenStatus)) {
for (const cb of this.exitCallbacks) {
cb(code, this.terminalOutput);
}
}
});
}
}
export async function runSingleCommandWithPseudoTerminal(
normalized: NormalizedRunCommandsOptions,
context: ExecutorContext
): Promise<PseudoTtyProcess> {
const pseudoTerminal = createPseudoTerminal();
const pseudoTtyProcess = await createProcessWithPseudoTty(
pseudoTerminal,
normalized.commands[0],
normalized.color,
calculateCwd(normalized.cwd, context),
normalized.env,
normalized.streamOutput,
pseudoTerminal ? normalized.isTTY : false,
normalized.envFile
);
registerProcessListener(pseudoTtyProcess, pseudoTerminal);
return pseudoTtyProcess;
}
async function createProcessWithPseudoTty(
pseudoTerminal: PseudoTerminal,
commandConfig: RunCommandsCommandOptions,
color: boolean,
cwd: string,
env: Record<string, string>,
streamOutput: boolean = true,
tty: boolean,
envFile?: string
) {
return pseudoTerminal.runCommand(commandConfig.command, {
cwd,
jsEnv: processEnv(color, cwd, env, envFile),
quiet: !streamOutput,
tty,
});
}
function addColorAndPrefix(out: string, config: RunCommandsCommandOptions) {
if (config.prefix) {
out = out
.split('\n')
.map((l) => {
let prefixText = config.prefix;
if (config.prefixColor && chalk[config.prefixColor]) {
prefixText = chalk[config.prefixColor](prefixText);
}
prefixText = chalk.bold(prefixText);
return l.trim().length > 0 ? `${prefixText} ${l}` : l;
})
.join('\n');
}
if (config.color && chalk[config.color]) {
out = chalk[config.color](out);
}
if (config.bgColor && chalk[config.bgColor]) {
out = chalk[config.bgColor](out);
}
return out;
}
function calculateCwd(
cwd: string | undefined,
context: ExecutorContext
): string {
if (!cwd) return context.root;
if (isAbsolute(cwd)) return cwd;
return join(context.root, cwd);
}
/**
* Env variables are processed in the following order:
* - env option from executor options
* - env file from envFile option if provided
* - local env variables
*/
function processEnv(
color: boolean,
cwd: string,
envOptionFromExecutor: Record<string, string>,
envFile?: string
) {
let localEnv = appendLocalEnv({ cwd: cwd ?? process.cwd() });
localEnv = {
...process.env,
...localEnv,
};
if (process.env.NX_LOAD_DOT_ENV_FILES !== 'false' && envFile) {
loadEnvVarsFile(envFile, localEnv);
}
let res: Record<string, string> = {
...localEnv,
...envOptionFromExecutor,
};
// need to override PATH to make sure we are using the local node_modules
if (localEnv.PATH) res.PATH = localEnv.PATH; // UNIX-like
if (localEnv.Path) res.Path = localEnv.Path; // Windows
if (color) {
res.FORCE_COLOR = `${color}`;
}
return res;
}
function isReady(
readyWhenStatus: { stringToMatch: string; found: boolean }[] = [],
data?: string
): boolean {
if (data) {
for (const readyWhenElement of readyWhenStatus) {
if (data.toString().indexOf(readyWhenElement.stringToMatch) > -1) {
readyWhenElement.found = true;
break;
}
}
}
return readyWhenStatus.every((readyWhenElement) => readyWhenElement.found);
}
function loadEnvVarsFile(path: string, env: Record<string, string> = {}) {
unloadDotEnvFile(path, env);
const result = loadAndExpandDotEnvFile(path, env);
if (result.error) {
throw result.error;
}
}
let registered = false;
function registerProcessListener(
runningTask: PseudoTtyProcess | ParallelRunningTasks | SeriallyRunningTasks,
pseudoTerminal?: PseudoTerminal
) {
if (registered) {
return;
}
registered = true;
// When the nx process gets a message, it will be sent into the task's process
process.on('message', (message: Serializable) => {
// this.publisher.publish(message.toString());
if (pseudoTerminal) {
pseudoTerminal.sendMessageToChildren(message);
}
if ('send' in runningTask) {
runningTask.send(message);
}
});
// Terminate any task processes on exit
process.on('exit', () => {
runningTask.kill();
});
process.on('SIGINT', () => {
runningTask.kill('SIGTERM');
// we exit here because we don't need to write anything to cache.
process.exit(signalToCode('SIGINT'));
});
process.on('SIGTERM', () => {
runningTask.kill('SIGTERM');
// no exit here because we expect child processes to terminate which
// will store results to the cache and will terminate this process
});
process.on('SIGHUP', () => {
runningTask.kill('SIGTERM');
// no exit here because we expect child processes to terminate which
// will store results to the cache and will terminate this process
});
}
@@ -1,11 +1,11 @@
import { execSync } from 'child_process';
import * as path from 'path';
import type { ExecutorContext } from '../../config/misc-interfaces';
import { getPackageManagerCommand } from '../../utils/package-manager';
import { execSync } from 'child_process';
import {
getPseudoTerminal,
createPseudoTerminal,
PseudoTerminal,
} from '../../tasks-runner/pseudo-terminal';
import { getPackageManagerCommand } from '../../utils/package-manager';
export interface RunScriptOptions {
script: string;
@@ -63,7 +63,7 @@ async function ptyProcess(
cwd: string,
env: Record<string, string>
) {
const terminal = getPseudoTerminal();
const terminal = createPseudoTerminal();
return new Promise<void>((res, rej) => {
const cp = terminal.runCommand(command, { cwd, jsEnv: env });
+58 -15
View File
@@ -7,10 +7,25 @@ export declare class ExternalObject<T> {
[K: symbol]: T
}
}
export declare class AppLifeCycle {
constructor(tasks: Array<Task>, pinnedTasks: Array<string>, tuiCliArgs: TuiCliArgs, tuiConfig: TuiConfig, titleText: string)
startCommand(threadCount?: number | undefined | null): void
scheduleTask(task: Task): void
startTasks(tasks: Array<Task>, metadata: object): void
printTaskTerminalOutput(task: Task, status: string, output: string): void
endTasks(taskResults: Array<TaskResult>, metadata: object): void
endCommand(): void
__init(doneCallback: () => any): void
registerRunningTask(taskId: string, parserAndWriter: ExternalObject<[ParserArc, WriterArc]>): void
__setCloudMessage(message: string): Promise<void>
}
export declare class ChildProcess {
getParserAndWriter(): ExternalObject<[ParserArc, WriterArc]>
kill(): void
onExit(callback: (message: string) => void): void
onOutput(callback: (message: string) => void): void
cleanup(): void
}
export declare class FileLock {
@@ -61,6 +76,13 @@ export declare class NxTaskHistory {
getEstimatedTaskTimings(targets: Array<TaskTarget>): Record<string, number>
}
export declare class RunningTasksService {
constructor(db: ExternalObject<NxDbConnection>)
getRunningTasks(ids: Array<string>): Array<string>
addRunningTask(taskId: string): void
removeRunningTask(taskId: string): void
}
export declare class RustPseudoTerminal {
constructor()
runCommand(command: string, commandDir?: string | undefined | null, jsEnv?: Record<string, string> | undefined | null, execArgv?: Array<string> | undefined | null, quiet?: boolean | undefined | null, tty?: boolean | undefined | null): ChildProcess
@@ -122,11 +144,11 @@ export interface CachedResult {
size?: number
}
export declare export declare function closeDbConnection(connection: ExternalObject<NxDbConnection>): void
export declare export function closeDbConnection(connection: ExternalObject<NxDbConnection>): void
export declare export declare function connectToNxDb(cacheDir: string, nxVersion: string, dbName?: string | undefined | null): ExternalObject<NxDbConnection>
export declare export function connectToNxDb(cacheDir: string, nxVersion: string, dbName?: string | undefined | null): ExternalObject<NxDbConnection>
export declare export declare function copy(src: string, dest: string): number
export declare export function copy(src: string, dest: string): number
export interface DepsOutputsInput {
dependentTasksOutputFiles: string
@@ -143,7 +165,7 @@ export declare const enum EventType {
create = 'create'
}
export declare export declare function expandOutputs(directory: string, entries: Array<string>): Array<string>
export declare export function expandOutputs(directory: string, entries: Array<string>): Array<string>
export interface ExternalDependenciesInput {
externalDependencies: Array<string>
@@ -169,21 +191,21 @@ export interface FileSetInput {
fileset: string
}
export declare export declare function findImports(projectFileMap: Record<string, Array<string>>): Array<ImportResult>
export declare export function findImports(projectFileMap: Record<string, Array<string>>): Array<ImportResult>
export declare export declare function getBinaryTarget(): string
export declare export function getBinaryTarget(): string
export declare export declare function getDefaultMaxCacheSize(cachePath: string): number
export declare export function getDefaultMaxCacheSize(cachePath: string): number
/**
* Expands the given outputs into a list of existing files.
* This is used when hashing outputs
*/
export declare export declare function getFilesForOutputs(directory: string, entries: Array<string>): Array<string>
export declare export function getFilesForOutputs(directory: string, entries: Array<string>): Array<string>
export declare export declare function getTransformableOutputs(outputs: Array<string>): Array<string>
export declare export function getTransformableOutputs(outputs: Array<string>): Array<string>
export declare export declare function hashArray(input: Array<string | undefined | null>): string
export declare export function hashArray(input: Array<string | undefined | null>): string
export interface HashDetails {
value: string
@@ -201,7 +223,7 @@ export interface HasherOptions {
selectivelyHashTsConfig: boolean
}
export declare export declare function hashFile(file: string): string | null
export declare export function hashFile(file: string): string | null
export interface InputsInput {
input: string
@@ -241,7 +263,9 @@ export interface ProjectGraph {
externalNodes: Record<string, ExternalNode>
}
export declare export declare function remove(src: string): void
export declare export function remove(src: string): void
export declare export function restoreTerminal(): void
export interface RuntimeInput {
runtime: string
@@ -261,6 +285,9 @@ export interface Task {
target: TaskTarget
outputs: Array<string>
projectRoot?: string
startTime?: number
endTime?: number
continuous?: boolean
}
export interface TaskGraph {
@@ -269,6 +296,13 @@ export interface TaskGraph {
dependencies: Record<string, Array<string>>
}
export interface TaskResult {
task: Task
status: string
code: number
terminalOutput?: string
}
export interface TaskRun {
hash: string
status: string
@@ -283,20 +317,29 @@ export interface TaskTarget {
configuration?: string
}
export declare export declare function testOnlyTransferFileMap(projectFiles: Record<string, Array<FileData>>, nonProjectFiles: Array<FileData>): NxWorkspaceFilesExternals
export declare export function testOnlyTransferFileMap(projectFiles: Record<string, Array<FileData>>, nonProjectFiles: Array<FileData>): NxWorkspaceFilesExternals
/**
* Transfer the project graph from the JS world to the Rust world, so that we can pass the project graph via memory quicker
* This wont be needed once the project graph is created in Rust
*/
export declare export declare function transferProjectGraph(projectGraph: ProjectGraph): ExternalObject<ProjectGraph>
export declare export function transferProjectGraph(projectGraph: ProjectGraph): ExternalObject<ProjectGraph>
export interface TuiCliArgs {
targets?: string[] | undefined
tuiAutoExit?: boolean | number | undefined
}
export interface TuiConfig {
autoExit?: boolean | number | undefined
}
export interface UpdatedWorkspaceFiles {
fileMap: FileMap
externalReferences: NxWorkspaceFilesExternals
}
export declare export declare function validateOutputs(outputs: Array<string>): void
export declare export function validateOutputs(outputs: Array<string>): void
export interface WatchEvent {
path: string
+24 -5
View File
@@ -1,9 +1,11 @@
use colored::Colorize;
use std::io::IsTerminal;
use tracing::{Event, Level, Subscriber};
use tracing_appender::rolling::{RollingFileAppender, Rotation};
use tracing_subscriber::fmt::{format, FmtContext, FormatEvent, FormatFields, FormattedFields};
use tracing_subscriber::prelude::*;
use tracing_subscriber::registry::LookupSpan;
use tracing_subscriber::EnvFilter;
use tracing_subscriber::{EnvFilter, Layer};
struct NxLogFormatter;
impl<S, N> FormatEvent<S, N> for NxLogFormatter
@@ -88,13 +90,30 @@ where
/// - `NX_NATIVE_LOGGING=nx=trace` - enable all logs for the `nx` (this) crate
/// - `NX_NATIVE_LOGGING=nx::native::tasks::hashers::hash_project_files=trace` - enable all logs for the `hash_project_files` module
/// - `NX_NATIVE_LOGGING=[{project_name=project}]` - enable logs that contain the project in its span
/// NX_NATIVE_FILE_LOGGING acts the same but logs to .nx/workspace-data/nx.log instead of stdout
pub(crate) fn enable_logger() {
let env_filter =
EnvFilter::try_from_env("NX_NATIVE_LOGGING").unwrap_or_else(|_| EnvFilter::new("ERROR"));
_ = tracing_subscriber::fmt()
.with_env_filter(env_filter)
let stdout_layer = tracing_subscriber::fmt::layer()
.with_ansi(std::io::stdout().is_terminal())
.with_writer(std::io::stdout)
.event_format(NxLogFormatter)
.with_filter(
EnvFilter::try_from_env("NX_NATIVE_LOGGING")
.unwrap_or_else(|_| EnvFilter::new("ERROR")),
);
let file_appender: RollingFileAppender =
RollingFileAppender::new(Rotation::NEVER, ".nx/workspace-data", "nx.log");
let file_layer = tracing_subscriber::fmt::layer()
.with_writer(file_appender)
.event_format(NxLogFormatter)
.with_ansi(false)
.with_filter(
EnvFilter::try_from_env("NX_NATIVE_FILE_LOGGING")
.unwrap_or_else(|_| EnvFilter::new("ERROR")),
);
tracing_subscriber::registry()
.with(stdout_layer)
.with(file_layer)
.try_init()
.ok();
}
+2
View File
@@ -17,4 +17,6 @@ pub mod db;
#[cfg(not(target_arch = "wasm32"))]
pub mod pseudo_terminal;
#[cfg(not(target_arch = "wasm32"))]
pub mod tui;
#[cfg(not(target_arch = "wasm32"))]
pub mod watch;
@@ -361,6 +361,7 @@ if (!nativeBinding) {
throw new Error(`Failed to load native binding`)
}
module.exports.AppLifeCycle = nativeBinding.AppLifeCycle
module.exports.ChildProcess = nativeBinding.ChildProcess
module.exports.FileLock = nativeBinding.FileLock
module.exports.HashPlanner = nativeBinding.HashPlanner
@@ -368,6 +369,7 @@ module.exports.HttpRemoteCache = nativeBinding.HttpRemoteCache
module.exports.ImportResult = nativeBinding.ImportResult
module.exports.NxCache = nativeBinding.NxCache
module.exports.NxTaskHistory = nativeBinding.NxTaskHistory
module.exports.RunningTasksService = nativeBinding.RunningTasksService
module.exports.RustPseudoTerminal = nativeBinding.RustPseudoTerminal
module.exports.TaskDetails = nativeBinding.TaskDetails
module.exports.TaskHasher = nativeBinding.TaskHasher
@@ -387,6 +389,7 @@ module.exports.hashArray = nativeBinding.hashArray
module.exports.hashFile = nativeBinding.hashFile
module.exports.IS_WASM = nativeBinding.IS_WASM
module.exports.remove = nativeBinding.remove
module.exports.restoreTerminal = nativeBinding.restoreTerminal
module.exports.testOnlyTransferFileMap = nativeBinding.testOnlyTransferFileMap
module.exports.transferProjectGraph = nativeBinding.transferProjectGraph
module.exports.validateOutputs = nativeBinding.validateOutputs
@@ -1,4 +1,7 @@
use crossbeam_channel::Receiver;
use crate::native::pseudo_terminal::pseudo_terminal::{ParserArc, WriterArc};
use crossbeam_channel::Sender;
use crossbeam_channel::{bounded, Receiver};
use napi::bindgen_prelude::External;
use napi::{
threadsafe_function::{
ErrorStrategy::Fatal, ThreadsafeFunction, ThreadsafeFunctionCallMode::NonBlocking,
@@ -6,6 +9,10 @@ use napi::{
Env, JsFunction,
};
use portable_pty::ChildKiller;
use std::io::Write;
use std::sync::{Arc, Mutex, RwLock};
use tracing::warn;
use vt100_ctt::Parser;
pub enum ChildProcessMessage {
Kill,
@@ -13,24 +20,37 @@ pub enum ChildProcessMessage {
#[napi]
pub struct ChildProcess {
parser: Arc<RwLock<Parser>>,
process_killer: Box<dyn ChildKiller + Sync + Send>,
message_receiver: Receiver<String>,
pub(crate) wait_receiver: Receiver<String>,
thread_handles: Vec<Sender<()>>,
writer_arc: Arc<Mutex<Box<dyn Write + Send>>>,
}
#[napi]
impl ChildProcess {
pub fn new(
parser: Arc<RwLock<Parser>>,
writer_arc: Arc<Mutex<Box<dyn Write + Send>>>,
process_killer: Box<dyn ChildKiller + Sync + Send>,
message_receiver: Receiver<String>,
exit_receiver: Receiver<String>,
) -> Self {
Self {
parser,
writer_arc,
process_killer,
message_receiver,
wait_receiver: exit_receiver,
thread_handles: vec![],
}
}
#[napi]
pub fn get_parser_and_writer(&mut self) -> External<(ParserArc, WriterArc)> {
External::new((self.parser.clone(), self.writer_arc.clone()))
}
#[napi]
pub fn kill(&mut self) -> anyhow::Result<()> {
self.process_killer.kill().map_err(anyhow::Error::from)
@@ -68,18 +88,43 @@ impl ChildProcess {
callback_tsfn.unref(&env)?;
std::thread::spawn(move || {
while let Ok(content) = rx.recv() {
// windows will add `ESC[6n` to the beginning of the output,
// we dont want to store this ANSI code in cache, because replays will cause issues
// remove it before sending it to js
#[cfg(windows)]
let content = content.replace("\x1B[6n", "");
let (kill_tx, kill_rx) = bounded::<()>(1);
callback_tsfn.call(content, NonBlocking);
std::thread::spawn(move || {
loop {
if kill_rx.try_recv().is_ok() {
break;
}
if let Ok(content) = rx.try_recv() {
// windows will add `ESC[6n` to the beginning of the output,
// we dont want to store this ANSI code in cache, because replays will cause issues
// remove it before sending it to js
#[cfg(windows)]
let content = content.replace("\x1B[6n", "");
callback_tsfn.call(content, NonBlocking);
}
}
});
self.thread_handles.push(kill_tx);
Ok(())
}
#[napi]
pub fn cleanup(&mut self) {
let handles = std::mem::take(&mut self.thread_handles);
for handle in handles {
if let Err(e) = handle.send(()) {
warn!(error = ?e, "Failed to send kill signal to thread");
}
}
}
}
impl Drop for ChildProcess {
fn drop(&mut self) {
self.cleanup();
}
}
@@ -1,11 +1,12 @@
use mio::{unix::SourceFd, Events};
use std::{
io::{Read, Stdin, Write},
os::fd::AsRawFd,
};
use mio::{unix::SourceFd, Events};
use tracing::trace;
use super::pseudo_terminal::WriterArc;
pub fn handle_path_space(path: String) -> String {
if path.contains(' ') {
format!("'{}'", path)
@@ -14,7 +15,7 @@ pub fn handle_path_space(path: String) -> String {
}
}
pub fn write_to_pty(stdin: &mut Stdin, writer: &mut impl Write) -> anyhow::Result<()> {
pub fn write_to_pty(stdin: &mut Stdin, writer: WriterArc) -> anyhow::Result<()> {
let mut buffer = [0; 1024];
let mut poll = mio::Poll::new()?;
@@ -45,6 +46,8 @@ pub fn write_to_pty(stdin: &mut Stdin, writer: &mut impl Write) -> anyhow::Resul
mio::Token(0) => {
// Read data from stdin
loop {
let mut writer = writer.lock().expect("Failed to lock writer");
match stdin.read(&mut buffer) {
Ok(n) => {
writer.write_all(&buffer[..n])?;
@@ -1,9 +1,10 @@
use std::io::{Stdin, Write};
use std::os::windows::ffi::OsStrExt;
use std::{ffi::OsString, os::windows::ffi::OsStringExt};
use winapi::um::fileapi::GetShortPathNameW;
use super::pseudo_terminal::WriterArc;
pub fn handle_path_space(path: String) -> String {
let wide: Vec<u16> = std::path::PathBuf::from(&path)
.as_os_str()
@@ -24,8 +25,9 @@ pub fn handle_path_space(path: String) -> String {
}
}
pub fn write_to_pty(stdin: &mut Stdin, writer: &mut impl Write) -> anyhow::Result<()> {
std::io::copy(stdin, writer)
pub fn write_to_pty(stdin: &mut Stdin, writer: WriterArc) -> anyhow::Result<()> {
let mut writer = writer.lock().expect("Failed to lock writer");
std::io::copy(stdin, writer.as_mut())
.map_err(|e| anyhow::anyhow!(e))
.map(|_| ())
}
+11 -10
View File
@@ -1,26 +1,30 @@
use std::collections::HashMap;
use tracing::trace;
use super::child_process::ChildProcess;
use super::os;
use super::pseudo_terminal::{create_pseudo_terminal, run_command};
use super::pseudo_terminal::PseudoTerminal;
use crate::native::logger::enable_logger;
#[napi]
pub struct RustPseudoTerminal {}
pub struct RustPseudoTerminal {
pseudo_terminal: PseudoTerminal,
}
#[napi]
impl RustPseudoTerminal {
#[napi(constructor)]
pub fn new() -> napi::Result<Self> {
enable_logger();
Ok(Self {})
let pseudo_terminal = PseudoTerminal::default()?;
Ok(Self { pseudo_terminal })
}
#[napi]
pub fn run_command(
&self,
&mut self,
command: String,
command_dir: Option<String>,
js_env: Option<HashMap<String, String>>,
@@ -28,9 +32,7 @@ impl RustPseudoTerminal {
quiet: Option<bool>,
tty: Option<bool>,
) -> napi::Result<ChildProcess> {
let pseudo_terminal = create_pseudo_terminal()?;
run_command(
&pseudo_terminal,
self.pseudo_terminal.run_command(
command,
command_dir,
js_env,
@@ -43,9 +45,8 @@ impl RustPseudoTerminal {
/// This allows us to run a pseudoterminal with a fake node ipc channel
/// this makes it possible to be backwards compatible with the old implementation
#[napi]
#[allow(clippy::too_many_arguments)]
pub fn fork(
&self,
&mut self,
id: String,
fork_script: String,
pseudo_ipc_path: String,
@@ -3,7 +3,7 @@
mod os;
#[allow(clippy::module_inception)]
mod pseudo_terminal;
pub mod pseudo_terminal;
pub mod child_process;
@@ -4,7 +4,7 @@ use tracing::trace;
use super::child_process::ChildProcess;
use super::os;
use super::pseudo_terminal::{create_pseudo_terminal, run_command, PseudoTerminal};
use super::pseudo_terminal::PseudoTerminal;
use crate::native::logger::enable_logger;
#[napi]
@@ -18,14 +18,14 @@ impl RustPseudoTerminal {
pub fn new() -> napi::Result<Self> {
enable_logger();
let pseudo_terminal = create_pseudo_terminal()?;
let pseudo_terminal = PseudoTerminal::default()?;
Ok(Self { pseudo_terminal })
}
#[napi]
pub fn run_command(
&self,
&mut self,
command: String,
command_dir: Option<String>,
js_env: Option<HashMap<String, String>>,
@@ -33,8 +33,7 @@ impl RustPseudoTerminal {
quiet: Option<bool>,
tty: Option<bool>,
) -> napi::Result<ChildProcess> {
run_command(
&self.pseudo_terminal,
self.pseudo_terminal.run_command(
command,
command_dir,
js_env,
@@ -48,7 +47,7 @@ impl RustPseudoTerminal {
/// this makes it possible to be backwards compatible with the old implementation
#[napi]
pub fn fork(
&self,
&mut self,
id: String,
fork_script: String,
pseudo_ipc_path: String,
@@ -65,6 +64,13 @@ impl RustPseudoTerminal {
);
trace!("nx_fork command: {}", &command);
self.run_command(command, command_dir, js_env, exec_argv, Some(quiet), Some(true))
self.run_command(
command,
command_dir,
js_env,
exec_argv,
Some(quiet),
Some(true),
)
}
}
@@ -1,3 +1,14 @@
use anyhow::anyhow;
use crossbeam_channel::{bounded, unbounded, Receiver};
use crossterm::{
terminal,
terminal::{disable_raw_mode, enable_raw_mode},
tty::IsTty,
};
use napi::bindgen_prelude::*;
use portable_pty::{CommandBuilder, NativePtySystem, PtyPair, PtySize, PtySystem};
use std::io::stdout;
use std::sync::{Mutex, RwLock};
use std::{
collections::HashMap,
io::{Read, Write},
@@ -7,16 +18,9 @@ use std::{
},
time::Instant,
};
use anyhow::anyhow;
use crossbeam_channel::{bounded, unbounded, Receiver};
use crossterm::{
terminal,
terminal::{disable_raw_mode, enable_raw_mode},
tty::IsTty,
};
use portable_pty::{CommandBuilder, NativePtySystem, PtyPair, PtySize, PtySystem};
use tracing::debug;
use tracing::log::trace;
use vt100_ctt::Parser;
use super::os;
use crate::native::pseudo_terminal::child_process::ChildProcess;
@@ -27,192 +31,249 @@ pub struct PseudoTerminal {
pub printing_rx: Receiver<()>,
pub quiet: Arc<AtomicBool>,
pub running: Arc<AtomicBool>,
pub writer: WriterArc,
pub parser: ParserArc,
is_within_nx_tui: bool,
}
pub fn create_pseudo_terminal() -> napi::Result<PseudoTerminal> {
let quiet = Arc::new(AtomicBool::new(true));
let running = Arc::new(AtomicBool::new(false));
pub struct PseudoTerminalOptions {
pub size: (u16, u16),
}
let pty_system = NativePtySystem::default();
let (w, h) = terminal::size().unwrap_or((80, 24));
trace!("Opening Pseudo Terminal");
let pty_pair = pty_system.openpty(PtySize {
rows: h,
cols: w,
pixel_width: 0,
pixel_height: 0,
})?;
let mut writer = pty_pair.master.take_writer()?;
// Stdin -> pty stdin
if std::io::stdout().is_tty() {
trace!("Passing through stdin");
std::thread::spawn(move || {
let mut stdin = std::io::stdin();
if let Err(e) = os::write_to_pty(&mut stdin, &mut writer) {
trace!("Error writing to pty: {:?}", e);
}
});
}
// Why do we do this here when it's already done when running a command?
if std::io::stdout().is_tty() {
trace!("Enabling raw mode");
enable_raw_mode().expect("Failed to enter raw terminal mode");
impl Default for PseudoTerminalOptions {
fn default() -> Self {
let (w, h) = terminal::size().unwrap_or((80, 24));
Self { size: (w, h) }
}
}
let mut reader = pty_pair.master.try_clone_reader()?;
let (message_tx, message_rx) = unbounded();
let (printing_tx, printing_rx) = unbounded();
// Output -> stdout handling
let quiet_clone = quiet.clone();
let running_clone = running.clone();
std::thread::spawn(move || {
let mut stdout = std::io::stdout();
let mut buf = [0; 8 * 1024];
pub type ParserArc = Arc<RwLock<Parser>>;
pub type WriterArc = Arc<Mutex<Box<dyn Write + Send>>>;
'read_loop: loop {
if let Ok(len) = reader.read(&mut buf) {
if len == 0 {
break;
impl PseudoTerminal {
pub fn new(options: PseudoTerminalOptions) -> Result<Self> {
let quiet = Arc::new(AtomicBool::new(true));
let running = Arc::new(AtomicBool::new(false));
let pty_system = NativePtySystem::default();
trace!("Opening Pseudo Terminal");
let (w, h) = options.size;
let pty_pair = pty_system.openpty(PtySize {
rows: h,
cols: w,
pixel_width: 0,
pixel_height: 0,
})?;
let writer = pty_pair.master.take_writer()?;
let writer_arc = Arc::new(Mutex::new(writer));
let writer_clone = writer_arc.clone();
let is_within_nx_tui =
std::env::var("NX_TUI").unwrap_or_else(|_| String::from("false")) == "true";
if !is_within_nx_tui && stdout().is_tty() {
// Stdin -> pty stdin
trace!("Passing through stdin");
std::thread::spawn(move || {
let mut stdin = std::io::stdin();
if let Err(e) = os::write_to_pty(&mut stdin, writer_clone) {
trace!("Error writing to pty: {:?}", e);
}
message_tx
.send(String::from_utf8_lossy(&buf[0..len]).to_string())
.ok();
let quiet = quiet_clone.load(Ordering::Relaxed);
trace!("Quiet: {}", quiet);
if !quiet {
let mut content = String::from_utf8_lossy(&buf[0..len]).to_string();
if content.contains("\x1B[6n") {
trace!("Prevented terminal escape sequence ESC[6n from being printed.");
content = content.replace("\x1B[6n", "");
});
}
let mut reader = pty_pair.master.try_clone_reader()?;
let (message_tx, message_rx) = unbounded();
let (printing_tx, printing_rx) = unbounded();
// Output -> stdout handling
let quiet_clone = quiet.clone();
let running_clone = running.clone();
let parser = Arc::new(RwLock::new(Parser::new(h, w, 10000)));
let parser_clone = parser.clone();
std::thread::spawn(move || {
let mut stdout = std::io::stdout();
let mut buf = [0; 8 * 1024];
let mut first: bool = true;
'read_loop: loop {
if let Ok(len) = reader.read(&mut buf) {
if len == 0 {
break;
}
let mut logged_interrupted_error = false;
while let Err(e) = stdout.write_all(content.as_bytes()) {
match e.kind() {
std::io::ErrorKind::Interrupted => {
if !logged_interrupted_error {
trace!("Interrupted error writing to stdout: {:?}", e);
logged_interrupted_error = true;
message_tx
.send(String::from_utf8_lossy(&buf[0..len]).to_string())
.ok();
let quiet = quiet_clone.load(Ordering::Relaxed);
trace!("Quiet: {}", quiet);
let contains_clear = buf[..len]
.windows(4)
.any(|window| window == [0x1B, 0x5B, 0x32, 0x4A]);
debug!("Contains clear: {}", contains_clear);
debug!("Read {} bytes", len);
if let Ok(mut parser) = parser_clone.write() {
let prev = parser.screen().clone();
parser.process(&buf[..len]);
debug!("{}", parser.get_raw_output().len());
let write_buf = if first {
parser.screen().contents_formatted()
} else {
parser.screen().contents_diff(&prev)
};
first = false;
if !quiet {
let mut logged_interrupted_error = false;
while let Err(e) = stdout.write_all(&write_buf) {
match e.kind() {
std::io::ErrorKind::Interrupted => {
if !logged_interrupted_error {
trace!("Interrupted error writing to stdout: {:?}", e);
logged_interrupted_error = true;
}
continue;
}
_ => {
// We should figure out what to do for more error types as they appear.
trace!("Error writing to stdout: {:?}", e);
trace!("Error kind: {:?}", e.kind());
break 'read_loop;
}
}
continue;
}
_ => {
// We should figure out what to do for more error types as they appear.
trace!("Error writing to stdout: {:?}", e);
trace!("Error kind: {:?}", e.kind());
break 'read_loop;
}
let _ = stdout.flush();
}
} else {
debug!("Failed to lock parser");
}
}
if !running_clone.load(Ordering::SeqCst) {
printing_tx.send(()).ok();
}
}
printing_tx.send(()).ok();
});
Ok(PseudoTerminal {
quiet,
writer: writer_arc,
running,
parser,
pty_pair,
message_rx,
printing_rx,
is_within_nx_tui,
})
}
pub fn default() -> Result<PseudoTerminal> {
Self::new(PseudoTerminalOptions::default())
}
pub fn run_command(
&mut self,
command: String,
command_dir: Option<String>,
js_env: Option<HashMap<String, String>>,
exec_argv: Option<Vec<String>>,
quiet: Option<bool>,
tty: Option<bool>,
) -> napi::Result<ChildProcess> {
let command_dir = get_directory(command_dir)?;
let pair = &self.pty_pair;
let quiet = quiet.unwrap_or(false);
self.quiet.store(quiet, Ordering::Relaxed);
let mut cmd = command_builder();
cmd.arg(command.as_str());
cmd.cwd(command_dir);
if let Some(js_env) = js_env {
for (key, value) in js_env {
cmd.env(key, value);
}
}
if let Some(exec_argv) = exec_argv {
cmd.env("NX_PSEUDO_TERMINAL_EXEC_ARGV", exec_argv.join("|"));
}
let (exit_to_process_tx, exit_to_process_rx) = bounded(1);
trace!("Running {}", command);
// TODO(@FrozenPandaz): This access is too naive, we need to handle the writer lock properly (e.g. multiple invocations of run_command sequentially)
// Prepend the command to the output
self.writer
.lock()
.unwrap()
// Sadly ANSI escape codes don't seem to work properly when writing directly to the writer...
.write_all(format!("> {}\n\n", command).as_bytes())
.unwrap();
let mut child = pair.slave.spawn_command(cmd)?;
self.running.store(true, Ordering::SeqCst);
let is_tty = tty.unwrap_or_else(|| std::io::stdout().is_tty());
// Do not manipulate raw mode if running within the context of the NX_TUI, it handles it itself
let should_control_raw_mode = is_tty && !self.is_within_nx_tui;
if should_control_raw_mode {
trace!("Enabling raw mode");
enable_raw_mode().expect("Failed to enter raw terminal mode");
}
let process_killer = child.clone_killer();
trace!("Getting running clone");
let running_clone = self.running.clone();
trace!("Getting printing_rx clone");
let printing_rx = self.printing_rx.clone();
trace!("spawning thread to wait for command");
std::thread::spawn(move || {
trace!("Waiting for {}", command);
let res = child.wait();
if let Ok(exit) = res {
trace!("{} Exited", command);
// This mitigates the issues with ConPTY on windows and makes it work.
running_clone.store(false, Ordering::SeqCst);
if cfg!(windows) {
trace!("Waiting for printing to finish");
let timeout = 500;
let a = Instant::now();
loop {
if printing_rx.try_recv().is_ok() {
break;
}
if a.elapsed().as_millis() > timeout {
break;
}
}
let _ = stdout.flush();
trace!("Printing finished");
}
}
if !running_clone.load(Ordering::SeqCst) {
printing_tx.send(()).ok();
}
}
printing_tx.send(()).ok();
});
if std::io::stdout().is_tty() {
trace!("Disabling raw mode");
disable_raw_mode().expect("Failed to exit raw terminal mode");
}
Ok(PseudoTerminal {
quiet,
running,
pty_pair,
message_rx,
printing_rx,
})
}
pub fn run_command(
pseudo_terminal: &PseudoTerminal,
command: String,
command_dir: Option<String>,
js_env: Option<HashMap<String, String>>,
exec_argv: Option<Vec<String>>,
quiet: Option<bool>,
tty: Option<bool>,
) -> napi::Result<ChildProcess> {
let command_dir = get_directory(command_dir)?;
let pair = &pseudo_terminal.pty_pair;
let quiet = quiet.unwrap_or(false);
pseudo_terminal.quiet.store(quiet, Ordering::Relaxed);
let mut cmd = command_builder();
cmd.arg(command.as_str());
cmd.cwd(command_dir);
if let Some(js_env) = js_env {
for (key, value) in js_env {
cmd.env(key, value);
}
}
if let Some(exec_argv) = exec_argv {
cmd.env("NX_PSEUDO_TERMINAL_EXEC_ARGV", exec_argv.join("|"));
}
let (exit_to_process_tx, exit_to_process_rx) = bounded(1);
let mut child = pair.slave.spawn_command(cmd)?;
pseudo_terminal.running.store(true, Ordering::SeqCst);
trace!("Running {}", command);
let is_tty = tty.unwrap_or_else(|| std::io::stdout().is_tty());
if is_tty {
trace!("Enabling raw mode");
enable_raw_mode().expect("Failed to enter raw terminal mode");
}
let process_killer = child.clone_killer();
trace!("Getting running clone");
let running_clone = pseudo_terminal.running.clone();
trace!("Getting printing_rx clone");
let printing_rx = pseudo_terminal.printing_rx.clone();
trace!("spawning thread to wait for command");
std::thread::spawn(move || {
trace!("Waiting for {}", command);
let res = child.wait();
if let Ok(exit) = res {
trace!("{} Exited", command);
// This mitigates the issues with ConPTY on windows and makes it work.
running_clone.store(false, Ordering::SeqCst);
if cfg!(windows) {
trace!("Waiting for printing to finish");
let timeout = 500;
let a = Instant::now();
loop {
if printing_rx.try_recv().is_ok() {
break;
}
if a.elapsed().as_millis() > timeout {
break;
}
if should_control_raw_mode {
trace!("Disabling raw mode");
disable_raw_mode().expect("Failed to restore non-raw terminal");
}
trace!("Printing finished");
}
if is_tty {
trace!("Disabling raw mode");
disable_raw_mode().expect("Failed to restore non-raw terminal");
}
exit_to_process_tx.send(exit.to_string()).ok();
} else {
trace!("Error waiting for {}", command);
};
});
exit_to_process_tx.send(exit.to_string()).ok();
} else {
trace!("Error waiting for {}", command);
};
});
trace!("Returning ChildProcess");
Ok(ChildProcess::new(
process_killer,
pseudo_terminal.message_rx.clone(),
exit_to_process_rx,
))
trace!("Returning ChildProcess");
Ok(ChildProcess::new(
self.parser.clone(),
self.writer.clone(),
process_killer,
self.message_rx.clone(),
exit_to_process_rx,
))
}
}
fn get_directory(command_dir: Option<String>) -> anyhow::Result<String> {
@@ -252,11 +313,12 @@ mod tests {
#[test]
fn can_run_commands() {
let mut i = 0;
let pseudo_terminal = create_pseudo_terminal().unwrap();
let mut pseudo_terminal = PseudoTerminal::default().unwrap();
while i < 10 {
println!("Running {}", i);
let cp1 =
run_command(&pseudo_terminal, String::from("whoami"), None, None, None).unwrap();
let cp1 = pseudo_terminal
.run_command(String::from("whoami"), None, None, None)
.unwrap();
cp1.wait_receiver.recv().unwrap();
i += 1;
}
+2
View File
@@ -10,3 +10,5 @@ mod utils;
pub mod details;
#[cfg(not(target_arch = "wasm32"))]
pub mod task_history;
#[cfg(not(target_arch = "wasm32"))]
pub mod running_tasks_service;
@@ -0,0 +1,167 @@
use crate::native::db::connection::NxDbConnection;
use crate::native::utils::Normalize;
use hashbrown::HashSet;
use napi::bindgen_prelude::External;
use std::env::args_os;
use std::ffi::OsString;
use sysinfo::{Pid, ProcessRefreshKind, ProcessesToUpdate, System};
use tracing::debug;
#[napi]
struct RunningTasksService {
db: External<NxDbConnection>,
added_tasks: HashSet<String>,
}
#[napi]
impl RunningTasksService {
#[napi(constructor)]
pub fn new(db: External<NxDbConnection>) -> anyhow::Result<Self> {
let s = Self {
db,
added_tasks: Default::default(),
};
s.setup()?;
Ok(s)
}
#[napi]
pub fn get_running_tasks(&mut self, ids: Vec<String>) -> anyhow::Result<Vec<String>> {
let mut results = Vec::<String>::with_capacity(ids.len());
for id in ids.into_iter() {
if self.is_task_running(&id)? {
results.push(id);
}
}
Ok(results)
}
fn is_task_running(&self, task_id: &String) -> anyhow::Result<bool> {
let mut stmt = self
.db
.prepare("SELECT pid, command, cwd FROM running_tasks WHERE task_id = ?")?;
if let Ok((pid, db_process_command, db_process_cwd)) = stmt.query_row([task_id], |row| {
let pid: u32 = row.get(0)?;
let command: String = row.get(1)?;
let cwd: String = row.get(2)?;
Ok((pid, command, cwd))
}) {
debug!("Checking if {} exists", pid);
let mut sys = System::new();
sys.refresh_processes_specifics(
ProcessesToUpdate::Some(&[Pid::from(pid as usize)]),
true,
ProcessRefreshKind::everything(),
);
match sys.process(sysinfo::Pid::from(pid as usize)) {
Some(process_info) => {
let cmd = process_info.cmd().to_vec();
let cmd_str = cmd
.iter()
.map(|s| s.to_string_lossy().to_string())
.collect::<Vec<_>>()
.join(" ");
if let Some(cwd_path) = process_info.cwd() {
let cwd_str = cwd_path.to_normalized_string();
Ok(cmd_str == db_process_command && cwd_str == db_process_cwd)
} else {
Ok(cmd_str == db_process_command)
}
}
None => Ok(false),
}
} else {
Ok(false)
}
}
#[napi]
pub fn add_running_task(&mut self, task_id: String) -> anyhow::Result<()> {
let pid = std::process::id();
let command = args_os().collect::<Vec<OsString>>();
// Convert command vector to a string representation
let command_str = command
.iter()
.map(|s| s.to_string_lossy().to_string())
.collect::<Vec<_>>()
.join(" ");
let cwd = std::env::current_dir()
.expect("The current working directory does not exist")
.to_normalized_string();
let mut stmt = self.db.prepare(
"INSERT OR REPLACE INTO running_tasks (task_id, pid, command, cwd) VALUES (?, ?, ?, ?)",
)?;
stmt.execute([&task_id, &pid.to_string(), &command_str, &cwd])?;
self.added_tasks.insert(task_id);
Ok(())
}
#[napi]
pub fn remove_running_task(&self, task_id: String) -> anyhow::Result<()> {
let mut stmt = self
.db
.prepare("DELETE FROM running_tasks WHERE task_id = ?")?;
stmt.execute([task_id])?;
Ok(())
}
fn setup(&self) -> anyhow::Result<()> {
self.db.execute_batch(
"
CREATE TABLE IF NOT EXISTS running_tasks (
task_id TEXT PRIMARY KEY NOT NULL,
pid INTEGER NOT NULL,
command TEXT NOT NULL,
cwd TEXT NOT NULL
);
",
)?;
Ok(())
}
}
impl Drop for RunningTasksService {
fn drop(&mut self) {
// Remove tasks added by this service. This might happen if process exits because of SIGKILL
for task_id in self.added_tasks.iter() {
self.remove_running_task(task_id.clone()).ok();
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::env::args_os;
use std::ffi::OsString;
#[test]
fn test_add_task() {
let pid = std::process::id();
let mut sys = System::new();
sys.refresh_processes_specifics(
ProcessesToUpdate::Some(&[Pid::from(pid as usize)]),
true,
ProcessRefreshKind::everything(),
);
if let Some(process_info) = sys.process(sysinfo::Pid::from(pid as usize)) {
// Check if the process name contains "nx" or is related to nx
// TODO: check is the process is actually the same process
dbg!(process_info);
dbg!("Process {} is running", pid);
let cmd = process_info.cmd().to_vec();
let command = args_os().collect::<Vec<OsString>>();
assert_eq!(cmd, command);
} else {
dbg!("Process {} is not running", pid);
}
}
}
+12
View File
@@ -13,6 +13,9 @@ pub struct Task {
pub target: TaskTarget,
pub outputs: Vec<String>,
pub project_root: Option<String>,
pub start_time: Option<f64>,
pub end_time: Option<f64>,
pub continuous: Option<bool>,
}
#[napi(object)]
@@ -23,6 +26,15 @@ pub struct TaskTarget {
pub configuration: Option<String>,
}
#[napi(object)]
#[derive(Default, Clone)]
pub struct TaskResult {
pub task: Task,
pub status: String,
pub code: i32,
pub terminal_output: Option<String>,
}
#[napi(object)]
pub struct TaskGraph {
pub roots: Vec<String>,
@@ -0,0 +1,42 @@
import { RunningTasksService, TaskDetails } from '../index';
import { join } from 'path';
import { TempFs } from '../../internal-testing-utils/temp-fs';
import { rmSync } from 'fs';
import { getDbConnection } from '../../utils/db-connection';
import { randomBytes } from 'crypto';
const dbOutputFolder = 'temp-db-task';
describe('RunningTasksService', () => {
let runningTasksService: RunningTasksService;
let tempFs: TempFs;
beforeEach(() => {
tempFs = new TempFs('running-tasks-service');
const dbConnection = getDbConnection({
directory: join(__dirname, dbOutputFolder),
dbName: `temp-db-${randomBytes(4).toString('hex')}`,
});
runningTasksService = new RunningTasksService(dbConnection);
});
afterAll(() => {
rmSync(join(__dirname, dbOutputFolder), {
recursive: true,
force: true,
});
});
it('should record a task as running', () => {
runningTasksService.addRunningTask('app:build');
expect(runningTasksService.getRunningTasks(['app:build'])).toEqual([
'app:build',
]);
});
it('should remove a task from running tasks', () => {
runningTasksService.addRunningTask('app:build');
runningTasksService.removeRunningTask('app:build');
expect(runningTasksService.getRunningTasks(['app:build'])).toEqual([]);
});
});
+25
View File
@@ -0,0 +1,25 @@
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Action {
Tick,
Render,
Resize(u16, u16),
Quit,
CancelQuit,
Error(String),
Help,
EnterFilterMode,
ClearFilter,
AddFilterChar(char),
RemoveFilterChar,
ScrollUp,
ScrollDown,
NextTask,
PreviousTask,
NextPage,
PreviousPage,
ToggleOutput,
FocusNext,
FocusPrevious,
ScrollPaneUp(usize),
ScrollPaneDown(usize),
}
+687
View File
@@ -0,0 +1,687 @@
use color_eyre::eyre::Result;
use crossterm::event::{KeyCode, KeyEvent, KeyModifiers, MouseEventKind};
use napi::bindgen_prelude::External;
use napi::threadsafe_function::{ErrorStrategy, ThreadsafeFunction};
use ratatui::layout::{Alignment, Rect};
use ratatui::style::Modifier;
use ratatui::style::{Color, Style};
use ratatui::text::{Line, Span};
use ratatui::widgets::Paragraph;
use tokio::sync::mpsc;
use tokio::sync::mpsc::UnboundedSender;
use tracing::debug;
use crate::native::pseudo_terminal::pseudo_terminal::{ParserArc, WriterArc};
use crate::native::tasks::types::{Task, TaskResult};
use crate::native::tui::tui::Tui;
use super::config::TuiConfig;
use super::utils::is_cache_hit;
use super::{
action::Action,
components::{
countdown_popup::CountdownPopup,
help_popup::HelpPopup,
tasks_list::{TaskStatus, TasksList},
Component,
},
tui,
};
pub struct App {
pub components: Vec<Box<dyn Component>>,
pub quit_at: Option<std::time::Instant>,
focus: Focus,
previous_focus: Focus,
done_callback: Option<ThreadsafeFunction<(), ErrorStrategy::Fatal>>,
tui_config: TuiConfig,
// We track whether the user has interacted with the app to determine if we should show perform any auto-exit at all
user_has_interacted: bool,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Focus {
TaskList,
MultipleOutput(usize),
HelpPopup,
CountdownPopup,
}
impl App {
pub fn new(
tasks: Vec<Task>,
pinned_tasks: Vec<String>,
tui_config: TuiConfig,
title_text: String,
) -> Result<Self> {
let tasks_list = TasksList::new(tasks, pinned_tasks, title_text);
let help_popup = HelpPopup::new();
let countdown_popup = CountdownPopup::new();
let focus = tasks_list.get_focus();
let components: Vec<Box<dyn Component>> = vec![
Box::new(tasks_list),
Box::new(help_popup),
Box::new(countdown_popup),
];
Ok(Self {
components,
quit_at: None,
focus,
previous_focus: Focus::TaskList,
done_callback: None,
tui_config,
user_has_interacted: false,
})
}
pub fn start_command(&mut self, thread_count: Option<u32>) {
if let Some(tasks_list) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<TasksList>())
{
tasks_list.set_max_parallel(thread_count);
}
}
pub fn start_tasks(&mut self, tasks: Vec<Task>) {
if let Some(tasks_list) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<TasksList>())
{
tasks_list.start_tasks(tasks);
}
}
pub fn print_task_terminal_output(
&mut self,
task_id: String,
status: TaskStatus,
output: String,
) {
// If the status is a cache hit, we need to create a new parser and writer for the task in order to print the output
if is_cache_hit(status) {
let (parser, parser_and_writer) = TasksList::create_empty_parser_and_noop_writer();
// Add ANSI escape sequence to hide cursor at the end of output, it would be confusing to have it visible when a task is a cache hit
let output_with_hidden_cursor = format!("{}\x1b[?25l", output);
TasksList::write_output_to_parser(parser, output_with_hidden_cursor);
if let Some(tasks_list) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<TasksList>())
{
tasks_list.create_and_register_pty_instance(&task_id, parser_and_writer);
tasks_list.update_task_status(task_id.clone(), status);
let _ = tasks_list.handle_resize(None);
}
}
}
pub fn end_tasks(&mut self, task_results: Vec<TaskResult>) {
if let Some(tasks_list) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<TasksList>())
{
tasks_list.end_tasks(task_results);
}
}
// Show countdown popup for the configured duration (making sure the help popup is not open first)
pub fn end_command(&mut self) {
// If the user has interacted with the app, or auto-exit is disabled, do nothing
if self.user_has_interacted || !self.tui_config.auto_exit.should_exit_automatically() {
return;
}
let countdown_duration = self.tui_config.auto_exit.countdown_seconds();
// If countdown is disabled, exit immediately
if countdown_duration.is_none() {
self.quit_at = Some(std::time::Instant::now());
return;
}
// Otherwise, show the countdown popup for the configured duration
let countdown_duration = countdown_duration.unwrap() as u64;
if let Some(countdown_popup) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<CountdownPopup>())
{
countdown_popup.start_countdown(countdown_duration);
self.previous_focus = self.focus;
self.focus = Focus::CountdownPopup;
self.quit_at = Some(
std::time::Instant::now() + std::time::Duration::from_secs(countdown_duration),
);
}
}
// A pseudo-terminal running task will provide the parser and writer directly
pub fn register_running_task(
&mut self,
task_id: String,
parser_and_writer: External<(ParserArc, WriterArc)>,
task_status: TaskStatus,
) {
if let Some(tasks_list) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<TasksList>())
{
tasks_list.create_and_register_pty_instance(&task_id, parser_and_writer);
tasks_list.update_task_status(task_id.clone(), task_status);
}
}
pub fn handle_event(
&mut self,
event: tui::Event,
action_tx: &mpsc::UnboundedSender<Action>,
) -> Result<bool> {
match event {
tui::Event::Quit => {
action_tx.send(Action::Quit)?;
return Ok(true);
}
tui::Event::Tick => action_tx.send(Action::Tick)?,
tui::Event::Render => action_tx.send(Action::Render)?,
tui::Event::Resize(x, y) => action_tx.send(Action::Resize(x, y))?,
tui::Event::Key(key) => {
debug!("Handling Key Event: {:?}", key);
// Record that the user has interacted with the app
self.user_has_interacted = true;
// Handle Ctrl+C to quit
if key.code == KeyCode::Char('c') && key.modifiers == KeyModifiers::CONTROL {
// Quit immediately
self.quit_at = Some(std::time::Instant::now());
return Ok(true);
}
// Get tasks list component to check interactive mode before handling '?' key
if let Some(tasks_list) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<TasksList>())
{
// Only handle '?' key if we're not in interactive mode and the countdown popup is not open
if matches!(key.code, KeyCode::Char('?'))
&& !tasks_list.is_interactive_mode()
&& !matches!(self.focus, Focus::CountdownPopup)
{
let show_help_popup = !matches!(self.focus, Focus::HelpPopup);
if let Some(help_popup) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<HelpPopup>())
{
help_popup.set_visible(show_help_popup);
}
if show_help_popup {
self.previous_focus = self.focus;
self.focus = Focus::HelpPopup;
} else {
self.focus = self.previous_focus;
}
return Ok(false);
}
}
// If countdown popup is open, handle its keyboard events
if matches!(self.focus, Focus::CountdownPopup) {
// Any key pressed (other than scroll keys if the popup is scrollable) will cancel the countdown
if let Some(countdown_popup) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<CountdownPopup>())
{
if !countdown_popup.is_scrollable() {
countdown_popup.cancel_countdown();
self.quit_at = None;
self.focus = self.previous_focus;
return Ok(false);
}
match key.code {
KeyCode::Up | KeyCode::Char('k') => {
countdown_popup.scroll_up();
return Ok(false);
}
KeyCode::Down | KeyCode::Char('j') => {
countdown_popup.scroll_down();
return Ok(false);
}
_ => {
countdown_popup.cancel_countdown();
self.quit_at = None;
self.focus = self.previous_focus;
}
}
}
return Ok(false);
}
// If shortcuts popup is open, handle its keyboard events
if matches!(self.focus, Focus::HelpPopup) {
match key.code {
KeyCode::Esc => {
if let Some(help_popup) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<HelpPopup>())
{
help_popup.set_visible(false);
}
self.focus = self.previous_focus;
}
KeyCode::Up | KeyCode::Char('k') => {
if let Some(help_popup) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<HelpPopup>())
{
help_popup.scroll_up();
}
return Ok(false);
}
KeyCode::Down | KeyCode::Char('j') => {
if let Some(help_popup) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<HelpPopup>())
{
help_popup.scroll_down();
}
return Ok(false);
}
_ => {}
}
return Ok(false);
}
// Get tasks list component for handling key events
if let Some(tasks_list) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<TasksList>())
{
// Handle Up/Down keys for scrolling first
if matches!(tasks_list.get_focus(), Focus::MultipleOutput(_)) {
match key.code {
KeyCode::Up | KeyCode::Down => {
tasks_list.handle_key_event(key).ok();
return Ok(false);
}
KeyCode::Char('k') | KeyCode::Char('j')
if !tasks_list.is_interactive_mode() =>
{
tasks_list.handle_key_event(key).ok();
return Ok(false);
}
_ => {}
}
}
match tasks_list.get_focus() {
Focus::MultipleOutput(_) => {
if tasks_list.is_interactive_mode() {
// Send all other keys to the task list (and ultimately through the terminal pane to the PTY)
tasks_list.handle_key_event(key).ok();
} else {
// Handle navigation and special actions
match key.code {
KeyCode::Tab => {
tasks_list.focus_next();
self.focus = tasks_list.get_focus();
}
KeyCode::BackTab => {
tasks_list.focus_previous();
self.focus = tasks_list.get_focus();
}
// Add our new shortcuts here
KeyCode::Char('c') => {
tasks_list.handle_key_event(key).ok();
}
KeyCode::Char('u') | KeyCode::Char('d')
if key.modifiers.contains(KeyModifiers::CONTROL) =>
{
tasks_list.handle_key_event(key).ok();
}
KeyCode::Char('b') => {
tasks_list.toggle_task_list();
self.focus = tasks_list.get_focus();
}
_ => {
// Forward other keys for interactivity, scrolling (j/k) etc
tasks_list.handle_key_event(key).ok();
}
}
}
return Ok(false);
}
_ => {
// Handle spacebar toggle regardless of focus
if key.code == KeyCode::Char(' ') {
tasks_list.toggle_output_visibility();
return Ok(false); // Skip other key handling
}
let is_filter_mode = tasks_list.filter_mode;
match self.focus {
Focus::TaskList => match key.code {
KeyCode::Char('j') if !is_filter_mode => {
tasks_list.next();
}
KeyCode::Down => {
tasks_list.next();
}
KeyCode::Char('k') if !is_filter_mode => {
tasks_list.previous();
}
KeyCode::Up => {
tasks_list.previous();
}
KeyCode::Left => {
tasks_list.previous_page();
}
KeyCode::Right => {
tasks_list.next_page();
}
KeyCode::Esc => {
if matches!(self.focus, Focus::HelpPopup) {
if let Some(help_popup) =
self.components.iter_mut().find_map(|c| {
c.as_any_mut().downcast_mut::<HelpPopup>()
})
{
help_popup.set_visible(false);
}
self.focus = self.previous_focus;
} else {
// Only clear filter when help popup is not in focus
tasks_list.clear_filter();
}
}
KeyCode::Char(c) => {
if tasks_list.filter_mode {
tasks_list.add_filter_char(c);
} else {
match c {
'/' => {
if tasks_list.filter_mode {
tasks_list.exit_filter_mode();
} else {
tasks_list.enter_filter_mode();
}
}
c => {
if tasks_list.filter_mode {
tasks_list.add_filter_char(c);
} else {
match c {
'j' => tasks_list.next(),
'k' => tasks_list.previous(),
'1' => tasks_list
.assign_current_task_to_pane(0),
'2' => tasks_list
.assign_current_task_to_pane(1),
'0' => tasks_list.clear_all_panes(),
'h' => tasks_list.previous_page(),
'l' => tasks_list.next_page(),
'b' => {
tasks_list.toggle_task_list();
self.focus = tasks_list.get_focus();
}
_ => {}
}
}
}
}
}
}
KeyCode::Backspace => {
if tasks_list.filter_mode {
tasks_list.remove_filter_char();
}
}
KeyCode::Tab => {
if tasks_list.has_visible_panes() {
tasks_list.focus_next();
self.focus = tasks_list.get_focus();
}
}
KeyCode::BackTab => {
if tasks_list.has_visible_panes() {
tasks_list.focus_previous();
self.focus = tasks_list.get_focus();
}
}
_ => {}
},
Focus::MultipleOutput(_idx) => match key.code {
KeyCode::Tab => {
tasks_list.focus_next();
self.focus = tasks_list.get_focus();
}
KeyCode::BackTab => {
tasks_list.focus_previous();
self.focus = tasks_list.get_focus();
}
_ => {}
},
Focus::HelpPopup => {
// Shortcuts popup has its own key handling above
}
Focus::CountdownPopup => {
// Countdown popup has its own key handling above
}
}
}
}
}
}
tui::Event::Mouse(mouse) => {
// Record that the user has interacted with the app
self.user_has_interacted = true;
if let Some(tasks_list) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<TasksList>())
{
match mouse.kind {
MouseEventKind::ScrollUp => {
if matches!(tasks_list.get_focus(), Focus::MultipleOutput(_)) {
tasks_list
.handle_key_event(KeyEvent::new(
KeyCode::Up,
KeyModifiers::empty(),
))
.ok();
} else if matches!(tasks_list.get_focus(), Focus::TaskList) {
tasks_list.previous();
}
}
MouseEventKind::ScrollDown => {
if matches!(tasks_list.get_focus(), Focus::MultipleOutput(_)) {
tasks_list
.handle_key_event(KeyEvent::new(
KeyCode::Down,
KeyModifiers::empty(),
))
.ok();
} else if matches!(tasks_list.get_focus(), Focus::TaskList) {
tasks_list.next();
}
}
_ => {}
}
}
}
_ => {}
}
for component in self.components.iter_mut() {
if let Some(action) = component.handle_events(Some(event.clone()))? {
action_tx.send(action)?;
}
}
Ok(false)
}
pub fn handle_action(
&mut self,
tui: &mut Tui,
action: Action,
action_tx: &UnboundedSender<Action>,
) {
if action != Action::Tick && action != Action::Render {
debug!("{action:?}");
}
match action {
// Quit immediately
Action::Quit => self.quit_at = Some(std::time::Instant::now()),
// Cancel quitting
Action::CancelQuit => {
self.quit_at = None;
self.focus = self.previous_focus;
}
Action::Resize(w, h) => {
tui.resize(Rect::new(0, 0, w, h)).ok();
// Ensure the help popup is resized correctly
if let Some(help_popup) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<HelpPopup>())
{
help_popup.handle_resize(w, h);
}
// Propagate resize to PTY instances
if let Some(tasks_list) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<TasksList>())
{
tasks_list.handle_resize(Some((w, h))).ok();
}
tui.draw(|f| {
for component in self.components.iter_mut() {
let r = component.draw(f, f.area());
if let Err(e) = r {
action_tx
.send(Action::Error(format!("Failed to draw: {:?}", e)))
.ok();
}
}
})
.ok();
}
Action::Render => {
tui.draw(|f| {
let area = f.area();
// Check for minimum viable viewport size at the app level
if area.height < 10 || area.width < 40 {
let message = Line::from(vec![
Span::raw(" "),
Span::styled(
" NX ",
Style::reset()
.add_modifier(Modifier::BOLD)
.bg(Color::Red)
.fg(Color::Black),
),
Span::raw(" "),
Span::raw("Please make your terminal viewport larger in order to view the terminal UI"),
]);
// Create empty lines for vertical centering
let empty_line = Line::from("");
let mut lines = vec![];
// Add empty lines to center vertically
let vertical_padding = (area.height as usize).saturating_sub(3) / 2;
for _ in 0..vertical_padding {
lines.push(empty_line.clone());
}
// Add the message
lines.push(message);
let paragraph = Paragraph::new(lines)
.alignment(Alignment::Center);
f.render_widget(paragraph, area);
return;
}
// Only render components if viewport is large enough
// Draw main components with dimming if a popup is focused
let current_focus = self.focus();
for component in self.components.iter_mut() {
if let Some(tasks_list) =
component.as_any_mut().downcast_mut::<TasksList>()
{
tasks_list.set_dimmed(matches!(current_focus, Focus::HelpPopup | Focus::CountdownPopup));
tasks_list.set_focus(current_focus);
}
let r = component.draw(f, f.area());
if let Err(e) = r {
action_tx
.send(Action::Error(format!("Failed to draw: {:?}", e)))
.ok();
}
}
}).ok();
}
_ => {}
}
// Update components
for component in self.components.iter_mut() {
if let Ok(Some(new_action)) = component.update(action.clone()) {
action_tx.send(new_action).ok();
}
}
}
pub fn set_done_callback(
&mut self,
done_callback: ThreadsafeFunction<(), ErrorStrategy::Fatal>,
) {
self.done_callback = Some(done_callback);
}
pub fn call_done_callback(&self) {
if let Some(cb) = &self.done_callback {
cb.call(
(),
napi::threadsafe_function::ThreadsafeFunctionCallMode::Blocking,
);
}
}
pub fn focus(&self) -> Focus {
self.focus
}
pub fn set_cloud_message(&mut self, message: Option<String>) {
if let Some(tasks_list) = self
.components
.iter_mut()
.find_map(|c| c.as_any_mut().downcast_mut::<TasksList>())
{
tasks_list.set_cloud_message(message);
}
}
}
+52
View File
@@ -0,0 +1,52 @@
use color_eyre::eyre::Result;
use crossterm::event::{KeyEvent, MouseEvent};
use ratatui::layout::Rect;
use std::any::Any;
use tokio::sync::mpsc::UnboundedSender;
use super::{
action::Action,
tui::{Event, Frame},
};
pub mod countdown_popup;
pub mod help_popup;
pub mod help_text;
pub mod pagination;
pub mod task_selection_manager;
pub mod tasks_list;
pub mod terminal_pane;
pub trait Component: Any + Send {
#[allow(unused_variables)]
fn register_action_handler(&mut self, tx: UnboundedSender<Action>) -> Result<()> {
Ok(())
}
fn init(&mut self) -> Result<()> {
Ok(())
}
fn handle_events(&mut self, event: Option<Event>) -> Result<Option<Action>> {
let r = match event {
Some(Event::Key(key_event)) => self.handle_key_events(key_event)?,
Some(Event::Mouse(mouse_event)) => self.handle_mouse_events(mouse_event)?,
_ => None,
};
Ok(r)
}
#[allow(unused_variables)]
fn handle_key_events(&mut self, key: KeyEvent) -> Result<Option<Action>> {
Ok(None)
}
#[allow(unused_variables)]
fn handle_mouse_events(&mut self, mouse: MouseEvent) -> Result<Option<Action>> {
Ok(None)
}
#[allow(unused_variables)]
fn update(&mut self, action: Action) -> Result<Option<Action>> {
Ok(None)
}
fn draw(&mut self, f: &mut Frame<'_>, rect: Rect) -> Result<()>;
fn as_any(&self) -> &dyn Any;
fn as_any_mut(&mut self) -> &mut dyn Any;
}
@@ -0,0 +1,299 @@
use color_eyre::eyre::Result;
use ratatui::{
layout::{Alignment, Rect},
style::{Color, Modifier, Style},
text::{Line, Span},
widgets::{
Block, BorderType, Borders, Clear, Padding, Paragraph, Scrollbar, ScrollbarOrientation,
ScrollbarState,
},
Frame,
};
use std::any::Any;
use std::time::{Duration, Instant};
use super::Component;
pub struct CountdownPopup {
visible: bool,
start_time: Option<Instant>,
duration: Duration,
scroll_offset: usize,
scrollbar_state: ScrollbarState,
content_height: usize,
viewport_height: usize,
}
impl CountdownPopup {
pub fn new() -> Self {
Self {
visible: false,
start_time: None,
duration: Duration::from_secs(3),
scroll_offset: 0,
scrollbar_state: ScrollbarState::default(),
content_height: 0,
viewport_height: 0,
}
}
pub fn is_scrollable(&self) -> bool {
self.content_height > self.viewport_height
}
pub fn start_countdown(&mut self, duration_secs: u64) {
self.visible = true;
self.start_time = Some(Instant::now());
self.duration = Duration::from_secs(duration_secs);
self.scroll_offset = 0;
self.scrollbar_state = ScrollbarState::default();
}
pub fn cancel_countdown(&mut self) {
self.visible = false;
self.start_time = None;
}
pub fn should_quit(&self) -> bool {
if let Some(start_time) = self.start_time {
return start_time.elapsed() >= self.duration;
}
false
}
pub fn set_visible(&mut self, visible: bool) {
self.visible = visible;
if !visible {
self.start_time = None;
}
}
pub fn is_visible(&self) -> bool {
self.visible
}
pub fn scroll_up(&mut self) {
if self.scroll_offset > 0 {
self.scroll_offset -= 1;
// Update scrollbar state with new position
self.scrollbar_state = self
.scrollbar_state
.content_length(self.content_height)
.viewport_content_length(self.viewport_height)
.position(self.scroll_offset);
}
}
pub fn scroll_down(&mut self) {
let max_scroll = self.content_height.saturating_sub(self.viewport_height);
if self.scroll_offset < max_scroll {
self.scroll_offset += 1;
// Update scrollbar state with new position
self.scrollbar_state = self
.scrollbar_state
.content_length(self.content_height)
.viewport_content_length(self.viewport_height)
.position(self.scroll_offset);
}
}
pub fn render(&mut self, f: &mut Frame<'_>, area: Rect) {
let popup_height = 9;
let popup_width = 70;
// Make sure we don't exceed the available area
let popup_height = popup_height.min(area.height.saturating_sub(4));
let popup_width = popup_width.min(area.width.saturating_sub(4));
// Calculate the top-left position to center the popup
let popup_x = area.x + (area.width.saturating_sub(popup_width)) / 2;
let popup_y = area.y + (area.height.saturating_sub(popup_height)) / 2;
// Create popup area with fixed dimensions
let popup_area = Rect::new(popup_x, popup_y, popup_width, popup_height);
// Calculate seconds remaining
let seconds_remaining = if let Some(start_time) = self.start_time {
let elapsed = start_time.elapsed();
if elapsed >= self.duration {
0
} else {
(self.duration - elapsed).as_secs()
}
} else {
0
};
let time_remaining = seconds_remaining + 1;
let content = vec![
Line::from(vec![
Span::styled("• Press ", Style::default().fg(Color::DarkGray)),
Span::styled("any key", Style::default().fg(Color::Cyan)),
Span::styled(
" to keep the TUI running and interactively explore the results.",
Style::default().fg(Color::DarkGray),
),
]),
Line::from(""),
Line::from(vec![
Span::styled(
"• Learn how to configure auto-exit and more in the docs: ",
Style::default().fg(Color::DarkGray),
),
Span::styled(
// NOTE: I tried OSC 8 sequences here but they broke the layout, see: https://github.com/ratatui/ratatui/issues/1028
"https://nx.dev/terminal-ui",
Style::default().fg(Color::Cyan),
),
]),
];
let block = Block::default()
.title(Line::from(vec![
Span::raw(" "),
Span::styled(
" NX ",
Style::default()
.add_modifier(Modifier::BOLD)
.bg(Color::Cyan)
.fg(Color::Black),
),
Span::styled(" Exiting in ", Style::default().fg(Color::White)),
Span::styled(
format!("{}", time_remaining),
Style::default().fg(Color::Cyan),
),
Span::styled("... ", Style::default().fg(Color::White)),
]))
.title_alignment(Alignment::Left)
.borders(Borders::ALL)
.border_type(BorderType::Plain)
.border_style(Style::default().fg(Color::Cyan))
.padding(Padding::proportional(1));
// Get the inner area
let inner_area = block.inner(popup_area);
self.viewport_height = inner_area.height as usize;
// Calculate content height based on line wrapping
let wrapped_height = content
.iter()
.map(|line| {
let line_width = line.width() as u16;
if line_width == 0 {
1 // Empty lines still take up one row
} else {
(line_width.saturating_sub(1) / inner_area.width).saturating_add(1) as usize
}
})
.sum();
self.content_height = wrapped_height;
// Calculate scrollbar state
let scrollable_rows = self.content_height.saturating_sub(self.viewport_height);
let needs_scrollbar = scrollable_rows > 0;
// Update scrollbar state
self.scrollbar_state = if needs_scrollbar {
self.scrollbar_state
.content_length(scrollable_rows)
.viewport_content_length(self.viewport_height)
.position(self.scroll_offset)
} else {
ScrollbarState::default()
};
// Create scrollable paragraph
let scroll_start = self.scroll_offset;
let scroll_end = (self.scroll_offset + self.viewport_height).min(content.len());
let visible_content = content[scroll_start..scroll_end].to_vec();
let popup = Paragraph::new(visible_content)
.block(block.clone())
.wrap(ratatui::widgets::Wrap { trim: true });
// Render popup
f.render_widget(Clear, popup_area);
f.render_widget(popup, popup_area);
// Render scrollbar if needed
if needs_scrollbar {
// Add padding text at top and bottom of scrollbar
let top_text = Line::from(vec![Span::raw(" ")]);
let bottom_text = Line::from(vec![Span::raw(" ")]);
let text_width = 2; // Width of " "
// Top right padding
let top_right_area = Rect {
x: popup_area.x + popup_area.width - text_width as u16 - 3,
y: popup_area.y,
width: text_width as u16 + 2,
height: 1,
};
// Bottom right padding
let bottom_right_area = Rect {
x: popup_area.x + popup_area.width - text_width as u16 - 3,
y: popup_area.y + popup_area.height - 1,
width: text_width as u16 + 2,
height: 1,
};
// Render padding text
f.render_widget(
Paragraph::new(top_text)
.alignment(Alignment::Right)
.style(Style::default().fg(Color::Cyan)),
top_right_area,
);
f.render_widget(
Paragraph::new(bottom_text)
.alignment(Alignment::Right)
.style(Style::default().fg(Color::Cyan)),
bottom_right_area,
);
let scrollbar = Scrollbar::default()
.orientation(ScrollbarOrientation::VerticalRight)
.begin_symbol(Some(""))
.end_symbol(Some(""))
.style(Style::default().fg(Color::Cyan));
f.render_stateful_widget(scrollbar, popup_area, &mut self.scrollbar_state);
}
}
}
impl Clone for CountdownPopup {
fn clone(&self) -> Self {
Self {
visible: self.visible,
start_time: self.start_time,
duration: self.duration,
scroll_offset: self.scroll_offset,
scrollbar_state: self.scrollbar_state,
content_height: self.content_height,
viewport_height: self.viewport_height,
}
}
}
impl Component for CountdownPopup {
fn draw(&mut self, f: &mut Frame<'_>, rect: Rect) -> Result<()> {
if self.visible {
self.render(f, rect);
}
Ok(())
}
fn as_any(&self) -> &dyn Any {
self
}
fn as_any_mut(&mut self) -> &mut dyn Any {
self
}
}
@@ -0,0 +1,347 @@
use color_eyre::eyre::Result;
use ratatui::{
layout::{Alignment, Constraint, Direction, Layout, Rect},
style::{Color, Modifier, Style},
text::{Line, Span},
widgets::{
Block, BorderType, Borders, Clear, Padding, Paragraph, Scrollbar, ScrollbarOrientation,
ScrollbarState,
},
};
use std::any::Any;
use super::{Component, Frame};
pub struct HelpPopup {
scroll_offset: usize,
scrollbar_state: ScrollbarState,
content_height: usize,
viewport_height: usize,
visible: bool,
}
impl HelpPopup {
pub fn new() -> Self {
Self {
scroll_offset: 0,
scrollbar_state: ScrollbarState::default(),
content_height: 0,
viewport_height: 0,
visible: false,
}
}
pub fn set_visible(&mut self, visible: bool) {
self.visible = visible;
}
// Ensure the scroll state is reset to avoid recalc issues
pub fn handle_resize(&mut self, _width: u16, _height: u16) {
self.scroll_offset = 0;
self.scrollbar_state = ScrollbarState::default();
}
pub fn scroll_up(&mut self) {
if self.scroll_offset > 0 {
self.scroll_offset -= 1;
// Update scrollbar state with new position
self.scrollbar_state = self
.scrollbar_state
.content_length(self.content_height)
.viewport_content_length(self.viewport_height)
.position(self.scroll_offset);
}
}
pub fn scroll_down(&mut self) {
let max_scroll = self.content_height.saturating_sub(self.viewport_height);
if self.scroll_offset < max_scroll {
self.scroll_offset += 1;
// Update scrollbar state with new position
self.scrollbar_state = self
.scrollbar_state
.content_length(self.content_height)
.viewport_content_length(self.viewport_height)
.position(self.scroll_offset);
}
}
pub fn render(&mut self, f: &mut Frame<'_>, area: Rect) {
let percent_y = 85;
let percent_x = 70;
let popup_layout = Layout::default()
.direction(Direction::Vertical)
.constraints([
Constraint::Percentage((100 - percent_y) / 2),
Constraint::Percentage(percent_y),
Constraint::Percentage((100 - percent_y) / 2),
])
.split(area);
let popup_area = Layout::default()
.direction(Direction::Horizontal)
.constraints([
Constraint::Percentage((100 - percent_x) / 2),
Constraint::Percentage(percent_x),
Constraint::Percentage((100 - percent_x) / 2),
])
.split(popup_layout[1])[1];
let keybindings = vec![
// Misc
("?", "Toggle this popup"),
("<ctrl>+c", "Quit the TUI"),
("", ""),
// Navigation
("↑ or k", "Navigate/scroll task output up"),
("↓ or j", "Navigate/scroll task output down"),
("<ctrl>+u", "Scroll task output up"),
("<ctrl>+d", "Scroll task output down"),
("← or h", "Navigate left"),
("→ or l", "Navigate right"),
("", ""),
// Task List Controls
("/", "Filter tasks based on search term"),
("<esc>", "Clear filter"),
("", ""),
// Output Controls
("<space>", "Quick toggle a single output pane"),
("b", "Toggle task list visibility"),
("1", "Pin task to be shown in output pane 1"),
("2", "Pin task to be shown in output pane 2"),
(
"<tab>",
"Move focus between task list and output panes 1 and 2",
),
("c", "Copy focused output to clipboard"),
("", ""),
// Interactive Mode
("i", "Interact with a continuous task when it is in focus"),
("<ctrl>+z", "Stop interacting with a continuous task"),
];
let mut content: Vec<Line> = vec![
// Welcome text
Line::from(vec![
Span::styled(
"Thanks for using Nx! To get the most out of this terminal UI, please check out the docs: ",
Style::default().fg(Color::White),
),
Span::styled(
// NOTE: I tried OSC 8 sequences here but they broke the layout, see: https://github.com/ratatui/ratatui/issues/1028
"https://nx.dev/terminal-ui",
Style::default().fg(Color::Cyan),
),
]),
Line::from(vec![
Span::styled(
"If you are finding Nx useful, please consider giving it a star on GitHub, it means a lot: ",
Style::default().fg(Color::White),
),
Span::styled(
// NOTE: I tried OSC 8 sequences here but they broke the layout, see: https://github.com/ratatui/ratatui/issues/1028
"https://github.com/nrwl/nx",
Style::default().fg(Color::Cyan),
),
]),
Line::from(""), // Empty line for spacing
Line::from(vec![Span::styled(
"Available keyboard shortcuts:",
Style::default().fg(Color::DarkGray),
)]),
Line::from(""), // Empty line for spacing
];
// Add keybindings to content
content.extend(
keybindings
.into_iter()
.map(|(key, desc)| {
if key.is_empty() {
Line::from("")
} else {
// Split the key text on " or " if it exists
let key_parts: Vec<&str> = key.split(" or ").collect();
let mut spans = Vec::new();
// Calculate the total visible length (excluding color codes)
let visible_length = if key_parts.len() > 1 {
key_parts.iter().map(|s| s.len()).sum::<usize>() + 2
// for alignment
} else {
key.len()
};
// Add each key part with the appropriate styling
for (i, part) in key_parts.iter().enumerate() {
if i > 0 {
spans.push(Span::styled(
" or ",
Style::default().fg(Color::DarkGray),
));
}
spans.push(Span::styled(
part.to_string(),
Style::default().fg(Color::Cyan),
));
}
// Add padding to align all descriptions
let padding = " ".repeat(11usize.saturating_sub(visible_length));
spans.push(Span::raw(padding));
// Add the separator and description
spans.push(Span::styled("= ", Style::default().fg(Color::DarkGray)));
spans.push(Span::styled(desc, Style::default().fg(Color::White)));
Line::from(spans)
}
})
.collect::<Vec<Line>>(),
);
// Update content height based on actual content
let block = Block::default()
.title(Line::from(vec![
Span::raw(" "),
Span::styled(
" NX ",
Style::default()
.add_modifier(Modifier::BOLD)
.bg(Color::Cyan)
.fg(Color::Black),
),
Span::styled(" Help ", Style::default().fg(Color::White)),
]))
.title_alignment(Alignment::Left)
.borders(Borders::ALL)
.border_type(BorderType::Plain)
.border_style(Style::default().fg(Color::Cyan))
.padding(Padding::proportional(1));
let inner_area = block.inner(popup_area);
self.viewport_height = inner_area.height as usize;
// Calculate wrapped height by measuring each line
let wrapped_height = content
.iter()
.map(|line| {
// Get total width of all spans in the line
let line_width = line.width() as u16;
// Calculate how many rows this line will take up when wrapped
if line_width == 0 {
1 // Empty lines still take up one row
} else {
(line_width.saturating_sub(1) / inner_area.width).saturating_add(1) as usize
}
})
.sum();
self.content_height = wrapped_height;
// Calculate scrollbar state using the same logic as task list output panes
let scrollable_rows = self.content_height.saturating_sub(self.viewport_height);
let needs_scrollbar = scrollable_rows > 0;
// Reset scrollbar state if no scrolling needed
self.scrollbar_state = if needs_scrollbar {
let position = self.scroll_offset;
self.scrollbar_state
.content_length(scrollable_rows)
.viewport_content_length(self.viewport_height)
.position(position)
} else {
ScrollbarState::default()
};
// Create scrollable paragraph
let scroll_start = self.scroll_offset;
let scroll_end = (self.scroll_offset + self.viewport_height).min(content.len());
let visible_content = content[scroll_start..scroll_end].to_vec();
let popup = Paragraph::new(visible_content)
.block(block)
.alignment(Alignment::Left)
.wrap(ratatui::widgets::Wrap { trim: true });
f.render_widget(Clear, popup_area);
f.render_widget(popup, popup_area);
// Render scrollbar if needed
if needs_scrollbar {
// Add padding text at top and bottom of scrollbar
let top_text = Line::from(vec![Span::raw(" ")]);
let bottom_text = Line::from(vec![Span::raw(" ")]);
let text_width = 2; // Width of " "
// Top right padding
let top_right_area = Rect {
x: popup_area.x + popup_area.width - text_width as u16 - 3,
y: popup_area.y,
width: text_width as u16 + 2,
height: 1,
};
// Bottom right padding
let bottom_right_area = Rect {
x: popup_area.x + popup_area.width - text_width as u16 - 3,
y: popup_area.y + popup_area.height - 1,
width: text_width as u16 + 2,
height: 1,
};
// Render padding text
f.render_widget(
Paragraph::new(top_text)
.alignment(Alignment::Right)
.style(Style::default().fg(Color::Cyan)),
top_right_area,
);
f.render_widget(
Paragraph::new(bottom_text)
.alignment(Alignment::Right)
.style(Style::default().fg(Color::Cyan)),
bottom_right_area,
);
let scrollbar = Scrollbar::default()
.orientation(ScrollbarOrientation::VerticalRight)
.begin_symbol(Some(""))
.end_symbol(Some(""))
.style(Style::default().fg(Color::Cyan));
f.render_stateful_widget(scrollbar, popup_area, &mut self.scrollbar_state);
}
}
}
impl Clone for HelpPopup {
fn clone(&self) -> Self {
Self {
scroll_offset: self.scroll_offset,
scrollbar_state: self.scrollbar_state,
content_height: self.content_height,
viewport_height: self.viewport_height,
visible: self.visible,
}
}
}
impl Component for HelpPopup {
fn draw(&mut self, f: &mut Frame<'_>, rect: Rect) -> Result<()> {
if self.visible {
self.render(f, rect);
}
Ok(())
}
fn as_any(&self) -> &dyn Any {
self
}
fn as_any_mut(&mut self) -> &mut dyn Any {
self
}
}
@@ -0,0 +1,83 @@
use ratatui::{
layout::{Alignment, Rect},
style::{Color, Modifier, Style},
text::{Line, Span},
widgets::Paragraph,
Frame,
};
pub struct HelpText {
collapsed_mode: bool,
is_dimmed: bool,
align_left: bool,
}
impl HelpText {
pub fn new(collapsed_mode: bool, is_dimmed: bool, align_left: bool) -> Self {
Self {
collapsed_mode,
is_dimmed,
align_left,
}
}
pub fn set_collapsed_mode(&mut self, collapsed: bool) {
self.collapsed_mode = collapsed;
}
pub fn render(&self, f: &mut Frame<'_>, area: Rect) {
let base_style = if self.is_dimmed {
Style::default().add_modifier(Modifier::DIM)
} else {
Style::default()
};
if self.collapsed_mode {
// Show minimal hint
let hint = vec![
Span::styled("quit: ", base_style.fg(Color::DarkGray)),
Span::styled("<ctrl>+c", base_style.fg(Color::Cyan)),
Span::styled(" ", base_style.fg(Color::DarkGray)),
Span::styled("help: ", base_style.fg(Color::DarkGray)),
Span::styled("? ", base_style.fg(Color::Cyan)),
];
f.render_widget(
Paragraph::new(Line::from(hint)).alignment(if self.align_left {
Alignment::Left
} else {
Alignment::Right
}),
area,
);
} else {
// Show full shortcuts
let shortcuts = vec![
Span::styled("quit: ", base_style.fg(Color::DarkGray)),
Span::styled("<ctrl>+c", base_style.fg(Color::Cyan)),
Span::styled(" ", base_style.fg(Color::DarkGray)),
Span::styled("help: ", base_style.fg(Color::DarkGray)),
Span::styled("?", base_style.fg(Color::Cyan)),
Span::styled(" ", base_style.fg(Color::DarkGray)),
Span::styled("navigate: ", base_style.fg(Color::DarkGray)),
Span::styled("↑ ↓", base_style.fg(Color::Cyan)),
Span::styled(" ", base_style.fg(Color::DarkGray)),
Span::styled("filter: ", base_style.fg(Color::DarkGray)),
Span::styled("/", base_style.fg(Color::Cyan)),
Span::styled(" ", base_style.fg(Color::DarkGray)),
Span::styled("pin output: ", base_style.fg(Color::DarkGray)),
Span::styled("", base_style.fg(Color::DarkGray)),
Span::styled("1", base_style.fg(Color::Cyan)),
Span::styled(" or ", base_style.fg(Color::DarkGray)),
Span::styled("2", base_style.fg(Color::Cyan)),
Span::styled(" ", base_style.fg(Color::DarkGray)),
Span::styled("focus output: ", base_style.fg(Color::DarkGray)),
Span::styled("<tab>", base_style.fg(Color::Cyan)),
];
f.render_widget(
Paragraph::new(Line::from(shortcuts)).alignment(Alignment::Center),
area,
);
}
}
}
@@ -0,0 +1,64 @@
use ratatui::{
layout::Rect,
style::{Color, Modifier, Style},
text::{Line, Span},
widgets::Paragraph,
Frame,
};
pub struct Pagination {
current_page: usize,
total_pages: usize,
}
impl Pagination {
pub fn new(current_page: usize, total_pages: usize) -> Self {
Self {
current_page,
total_pages,
}
}
pub fn render(&self, f: &mut Frame<'_>, area: Rect, is_dimmed: bool) {
let base_style = if is_dimmed {
Style::default().add_modifier(Modifier::DIM)
} else {
Style::default()
};
let mut spans = vec![];
// Ensure we have at least 1 page
let total_pages = self.total_pages.max(1);
let current_page = self.current_page.min(total_pages - 1);
// Left arrow - dim if we're on the first page
let left_arrow = if current_page == 0 {
Span::styled("", base_style.fg(Color::Cyan).add_modifier(Modifier::DIM))
} else {
Span::styled("", base_style.fg(Color::Cyan))
};
spans.push(left_arrow);
// Page numbers
spans.push(Span::raw(" "));
spans.push(Span::styled(
format!("{}/{}", current_page + 1, total_pages),
base_style.fg(Color::DarkGray),
));
spans.push(Span::raw(" "));
// Right arrow - dim if we're on the last page
let right_arrow = if current_page >= total_pages.saturating_sub(1) {
Span::styled("", base_style.fg(Color::Cyan).add_modifier(Modifier::DIM))
} else {
Span::styled("", base_style.fg(Color::Cyan))
};
spans.push(right_arrow);
let pagination_line = Line::from(spans);
let pagination = Paragraph::new(pagination_line);
f.render_widget(pagination, area);
}
}
@@ -0,0 +1,464 @@
pub struct TaskSelectionManager {
// The list of task names in their current visual order, None represents empty rows
entries: Vec<Option<String>>,
// The currently selected task name
selected_task_name: Option<String>,
// Current page and pagination settings
current_page: usize,
items_per_page: usize,
// Selection mode determines how the selection behaves when entries change
selection_mode: SelectionMode,
}
/// Controls how task selection behaves when entries are updated or reordered
#[derive(Clone, Copy, PartialEq, Debug)]
pub enum SelectionMode {
/// Track a specific task by name regardless of its position in the list
/// Used when a task is pinned or in spacebar mode
TrackByName,
/// Track selection by position/index in the list
/// Used when no tasks are pinned and not in spacebar mode
TrackByPosition,
}
impl TaskSelectionManager {
pub fn new(items_per_page: usize) -> Self {
Self {
entries: Vec::new(),
selected_task_name: None,
current_page: 0,
items_per_page,
selection_mode: SelectionMode::TrackByName,
}
}
/// Sets the selection mode
pub fn set_selection_mode(&mut self, mode: SelectionMode) {
self.selection_mode = mode;
}
/// Gets the current selection mode
pub fn get_selection_mode(&self) -> SelectionMode {
self.selection_mode
}
pub fn update_entries(&mut self, entries: Vec<Option<String>>) {
match self.selection_mode {
SelectionMode::TrackByName => self.update_entries_track_by_name(entries),
SelectionMode::TrackByPosition => self.update_entries_track_by_position(entries),
}
}
/// Updates entries while trying to preserve the selected task by name
fn update_entries_track_by_name(&mut self, entries: Vec<Option<String>>) {
// Keep track of currently selected task name
let selected = self.selected_task_name.clone();
// Update the entries
self.entries = entries;
// Ensure current page is valid before validating selection
self.validate_current_page();
// If we had a selection, try to find it in the new list
if let Some(task_name) = selected {
// First check if the task still exists in the entries
let task_still_exists = self
.entries
.iter()
.any(|entry| entry.as_ref() == Some(&task_name));
if task_still_exists {
// Task is still in the list, keep it selected with the same name
self.selected_task_name = Some(task_name);
// Update the current page to ensure the selected task is visible
if let Some(idx) = self.get_selected_index() {
self.current_page = idx / self.items_per_page;
}
} else {
// If task is no longer in the list, select first available task
self.select_first_available();
}
} else {
// No previous selection, select first available task
self.select_first_available();
}
// Validate selection for current page
self.validate_selection_for_current_page();
}
/// Updates entries while trying to preserve the selected position in the list
fn update_entries_track_by_position(&mut self, entries: Vec<Option<String>>) {
// Get the current selection position within the page
let page_index = self.get_selected_index_in_current_page();
// Update the entries
self.entries = entries;
// Ensure current page is valid
self.validate_current_page();
// If we had a selection and there are entries, try to maintain the position
if let Some(idx) = page_index {
let start = self.current_page * self.items_per_page;
let end = (start + self.items_per_page).min(self.entries.len());
if start < end {
// Convert page index to absolute index
let absolute_idx = start + idx;
// Find the next non-empty entry at or after the position
for i in absolute_idx..end {
if let Some(Some(name)) = self.entries.get(i) {
self.selected_task_name = Some(name.clone());
return;
}
}
// If we can't find one after, try before
for i in (start..absolute_idx).rev() {
if let Some(Some(name)) = self.entries.get(i) {
self.selected_task_name = Some(name.clone());
return;
}
}
}
// If we couldn't find anything on the current page, select first available
self.select_first_available();
} else {
// No previous selection, select first available task
self.select_first_available();
}
}
pub fn select(&mut self, task_name: Option<String>) {
match task_name {
Some(name) if self.entries.iter().any(|e| e.as_ref() == Some(&name)) => {
self.selected_task_name = Some(name);
// Update current page to show selected task
if let Some(idx) = self
.entries
.iter()
.position(|e| e.as_deref() == self.selected_task_name.as_deref())
{
self.current_page = idx / self.items_per_page;
}
}
_ => {
self.selected_task_name = None;
}
}
}
pub fn select_task(&mut self, task_id: String) {
self.selected_task_name = Some(task_id);
}
pub fn next(&mut self) {
if let Some(current_idx) = self.get_selected_index() {
// Find next non-empty entry
for idx in (current_idx + 1)..self.entries.len() {
if self.entries[idx].is_some() {
self.selected_task_name = self.entries[idx].clone();
// Update page if needed
self.current_page = idx / self.items_per_page;
return;
}
}
} else {
self.select_first_available();
}
}
pub fn previous(&mut self) {
if let Some(current_idx) = self.get_selected_index() {
// Find previous non-empty entry
for idx in (0..current_idx).rev() {
if self.entries[idx].is_some() {
self.selected_task_name = self.entries[idx].clone();
// Update page if needed
self.current_page = idx / self.items_per_page;
return;
}
}
} else {
self.select_first_available();
}
}
pub fn next_page(&mut self) {
let total_pages = self.total_pages();
if self.current_page < total_pages - 1 {
self.current_page += 1;
self.validate_selection_for_current_page();
}
}
pub fn previous_page(&mut self) {
if self.current_page > 0 {
self.current_page -= 1;
self.validate_selection_for_current_page();
}
}
pub fn get_current_page_entries(&self) -> Vec<Option<String>> {
let start = self.current_page * self.items_per_page;
let end = (start + self.items_per_page).min(self.entries.len());
self.entries[start..end].to_vec()
}
pub fn is_selected(&self, task_name: &str) -> bool {
self.selected_task_name
.as_ref()
.map_or(false, |selected| selected == task_name)
}
pub fn get_selected_task_name(&self) -> Option<&String> {
self.selected_task_name.as_ref()
}
pub fn total_pages(&self) -> usize {
(self.entries.len() + self.items_per_page - 1) / self.items_per_page
}
pub fn get_current_page(&self) -> usize {
self.current_page
}
fn select_first_available(&mut self) {
self.selected_task_name = self.entries.iter().find_map(|e| e.clone());
// Ensure selected task is on current page
self.validate_selection_for_current_page();
}
fn validate_current_page(&mut self) {
let total_pages = self.total_pages();
if total_pages == 0 {
self.current_page = 0;
} else {
self.current_page = self.current_page.min(total_pages - 1);
}
}
fn validate_selection_for_current_page(&mut self) {
if let Some(task_name) = &self.selected_task_name {
let start = self.current_page * self.items_per_page;
let end = (start + self.items_per_page).min(self.entries.len());
// Check if selected task is on current page
if start < end
&& !self.entries[start..end]
.iter()
.any(|e| e.as_ref() == Some(task_name))
{
// If not, select first available task on current page
self.selected_task_name = self.entries[start..end].iter().find_map(|e| e.clone());
}
}
}
pub fn get_selected_index(&self) -> Option<usize> {
if let Some(task_name) = &self.selected_task_name {
self.entries
.iter()
.position(|entry| entry.as_ref() == Some(task_name))
} else {
None
}
}
pub fn get_selected_index_in_current_page(&self) -> Option<usize> {
if let Some(task_name) = &self.selected_task_name {
let current_page_entries = self.get_current_page_entries();
current_page_entries
.iter()
.position(|entry| entry.as_ref() == Some(task_name))
} else {
None
}
}
pub fn set_items_per_page(&mut self, items_per_page: usize) {
// Ensure we never set items_per_page to 0
self.items_per_page = items_per_page.max(1);
self.validate_current_page();
self.validate_selection_for_current_page();
}
pub fn get_items_per_page(&self) -> usize {
self.items_per_page
}
}
impl Default for TaskSelectionManager {
fn default() -> Self {
Self::new(5) // Default to 5 items per page
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_new_manager() {
let manager = TaskSelectionManager::new(5);
assert_eq!(manager.get_selected_task_name(), None);
assert_eq!(manager.get_current_page(), 0);
assert_eq!(manager.get_selection_mode(), SelectionMode::TrackByName);
}
#[test]
fn test_update_entries_track_by_name() {
let mut manager = TaskSelectionManager::new(2);
manager.set_selection_mode(SelectionMode::TrackByName);
// Initial entries
let entries = vec![Some("Task 1".to_string()), None, Some("Task 2".to_string())];
manager.update_entries(entries);
assert_eq!(
manager.get_selected_task_name(),
Some(&"Task 1".to_string())
);
// Update entries with same tasks but different order
let entries = vec![Some("Task 2".to_string()), None, Some("Task 1".to_string())];
manager.update_entries(entries);
// Selection should still be Task 1 despite order change
assert_eq!(
manager.get_selected_task_name(),
Some(&"Task 1".to_string())
);
}
#[test]
fn test_update_entries_track_by_position() {
let mut manager = TaskSelectionManager::new(2);
manager.set_selection_mode(SelectionMode::TrackByPosition);
// Initial entries
let entries = vec![Some("Task 1".to_string()), None, Some("Task 2".to_string())];
manager.update_entries(entries);
assert_eq!(
manager.get_selected_task_name(),
Some(&"Task 1".to_string())
);
// Update entries with different tasks but same structure
let entries = vec![Some("Task 3".to_string()), None, Some("Task 4".to_string())];
manager.update_entries(entries);
// Selection should be Task 3 (same position as Task 1 was)
assert_eq!(
manager.get_selected_task_name(),
Some(&"Task 3".to_string())
);
}
#[test]
fn test_select() {
let mut manager = TaskSelectionManager::new(2);
let entries = vec![Some("Task 1".to_string()), None, Some("Task 2".to_string())];
manager.update_entries(entries);
manager.select(Some("Task 2".to_string()));
assert_eq!(
manager.get_selected_task_name(),
Some(&"Task 2".to_string())
);
assert_eq!(manager.get_current_page(), 1); // Should move to page containing Task 2
}
#[test]
fn test_navigation() {
let mut manager = TaskSelectionManager::new(2);
let entries = vec![
Some("Task 1".to_string()),
None,
Some("Task 2".to_string()),
Some("Task 3".to_string()),
];
manager.update_entries(entries);
// Test next
assert_eq!(
manager.get_selected_task_name(),
Some(&"Task 1".to_string())
);
manager.next();
assert_eq!(
manager.get_selected_task_name(),
Some(&"Task 2".to_string())
);
// Test previous
manager.previous();
assert_eq!(
manager.get_selected_task_name(),
Some(&"Task 1".to_string())
);
}
#[test]
fn test_pagination() {
let mut manager = TaskSelectionManager::new(2);
let entries = vec![
Some("Task 1".to_string()),
Some("Task 2".to_string()),
Some("Task 3".to_string()),
Some("Task 4".to_string()),
];
manager.update_entries(entries);
assert_eq!(manager.total_pages(), 2);
assert_eq!(manager.get_current_page(), 0);
// Test next page
manager.next_page();
assert_eq!(manager.get_current_page(), 1);
let page_entries = manager.get_current_page_entries();
assert_eq!(page_entries.len(), 2);
assert_eq!(page_entries[0], Some("Task 3".to_string()));
// Test previous page
manager.previous_page();
assert_eq!(manager.get_current_page(), 0);
let page_entries = manager.get_current_page_entries();
assert_eq!(page_entries[0], Some("Task 1".to_string()));
}
#[test]
fn test_is_selected() {
let mut manager = TaskSelectionManager::new(2);
let entries = vec![Some("Task 1".to_string()), Some("Task 2".to_string())];
manager.update_entries(entries);
assert!(manager.is_selected("Task 1"));
assert!(!manager.is_selected("Task 2"));
}
#[test]
fn test_handle_position_tracking_empty_entries() {
let mut manager = TaskSelectionManager::new(2);
manager.set_selection_mode(SelectionMode::TrackByPosition);
// Initial entries
let entries = vec![Some("Task 1".to_string()), Some("Task 2".to_string())];
manager.update_entries(entries);
assert_eq!(
manager.get_selected_task_name(),
Some(&"Task 1".to_string())
);
// Update with empty entries
let entries: Vec<Option<String>> = vec![];
manager.update_entries(entries);
// No entries, so no selection
assert_eq!(manager.get_selected_task_name(), None);
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,493 @@
use arboard::Clipboard;
use crossterm::event::{KeyCode, KeyEvent, KeyModifiers};
use ratatui::{
buffer::Buffer,
layout::{Alignment, Rect},
style::{Color, Modifier, Style},
text::{Line, Span},
widgets::{
Block, BorderType, Borders, Padding, Paragraph, Scrollbar, ScrollbarOrientation,
ScrollbarState, StatefulWidget, Widget,
},
};
use std::{io, sync::Arc};
use tui_term::widget::PseudoTerminal;
use crate::native::tui::pty::PtyInstance;
use super::tasks_list::TaskStatus;
pub struct TerminalPaneData {
pub pty: Option<Arc<PtyInstance>>,
pub is_interactive: bool,
pub is_continuous: bool,
pub is_cache_hit: bool,
}
impl TerminalPaneData {
pub fn new() -> Self {
Self {
pty: None,
is_interactive: false,
is_continuous: false,
is_cache_hit: false,
}
}
pub fn handle_key_event(&mut self, key: KeyEvent) -> io::Result<()> {
if let Some(pty) = &mut self.pty {
let mut pty_mut = pty.as_ref().clone();
match key.code {
// Handle arrow key based scrolling regardless of interactive mode
KeyCode::Up => {
pty_mut.scroll_up();
return Ok(());
}
KeyCode::Down => {
pty_mut.scroll_down();
return Ok(());
}
// Handle j/k for scrolling when not in interactive mode
KeyCode::Char('k') | KeyCode::Char('j') if !self.is_interactive => {
match key.code {
KeyCode::Char('k') => pty_mut.scroll_up(),
KeyCode::Char('j') => pty_mut.scroll_down(),
_ => {}
}
return Ok(());
}
// Handle ctrl+u and ctrl+d for scrolling when not in interactive mode
KeyCode::Char('u')
if key.modifiers.contains(KeyModifiers::CONTROL) && !self.is_interactive =>
{
// Scroll up a somewhat arbitrary "chunk" (12 lines)
for _ in 0..12 {
pty_mut.scroll_up();
}
return Ok(());
}
KeyCode::Char('d')
if key.modifiers.contains(KeyModifiers::CONTROL) && !self.is_interactive =>
{
// Scroll down a somewhat arbitrary "chunk" (12 lines)
for _ in 0..12 {
pty_mut.scroll_down();
}
return Ok(());
}
// Handle 'c' for copying when not in interactive mode
KeyCode::Char('c') if !self.is_interactive => {
if let Some(screen) = pty.get_screen() {
// Unformatted output (no ANSI escape codes)
let output = screen.all_contents();
match Clipboard::new() {
Ok(mut clipboard) => {
clipboard.set_text(output).ok();
}
Err(_) => {
// TODO: Is there a way to handle this error? Maybe a new kind of error popup?
}
}
}
return Ok(());
}
// Handle 'i' to enter interactive mode for non cache hit tasks
KeyCode::Char('i') if !self.is_cache_hit && !self.is_interactive => {
self.set_interactive(true);
return Ok(());
}
// Handle Ctrl+Z to exit interactive mode
KeyCode::Char('z')
if key.modifiers == KeyModifiers::CONTROL
&& !self.is_cache_hit
&& self.is_interactive =>
{
self.set_interactive(false);
return Ok(());
}
// Only send input to PTY if we're in interactive mode
_ if self.is_interactive => match key.code {
KeyCode::Char(c) => {
pty_mut.write_input(c.to_string().as_bytes())?;
}
KeyCode::Enter => {
pty_mut.write_input(b"\r")?;
}
KeyCode::Esc => {
pty_mut.write_input(&[0x1b])?;
}
KeyCode::Backspace => {
pty_mut.write_input(&[0x7f])?;
}
_ => {}
},
_ => {}
}
}
Ok(())
}
pub fn set_interactive(&mut self, interactive: bool) {
self.is_interactive = interactive;
}
pub fn is_interactive(&self) -> bool {
self.is_interactive
}
}
impl Default for TerminalPaneData {
fn default() -> Self {
Self::new()
}
}
pub struct TerminalPaneState {
pub task_name: String,
pub task_status: TaskStatus,
pub is_continuous: bool,
pub is_focused: bool,
pub scroll_offset: usize,
pub scrollbar_state: ScrollbarState,
pub has_pty: bool,
}
impl TerminalPaneState {
pub fn new(
task_name: String,
task_status: TaskStatus,
is_continuous: bool,
is_focused: bool,
has_pty: bool,
) -> Self {
Self {
task_name,
task_status,
is_continuous,
is_focused,
scroll_offset: 0,
scrollbar_state: ScrollbarState::default(),
has_pty,
}
}
}
pub struct TerminalPane<'a> {
pty_data: Option<&'a mut TerminalPaneData>,
is_continuous: bool,
}
impl<'a> TerminalPane<'a> {
pub fn new() -> Self {
Self {
pty_data: None,
is_continuous: false,
}
}
pub fn pty_data(mut self, data: &'a mut TerminalPaneData) -> Self {
self.pty_data = Some(data);
self
}
pub fn continuous(mut self, continuous: bool) -> Self {
self.is_continuous = continuous;
self
}
fn get_status_icon(&self, status: TaskStatus) -> Span {
match status {
TaskStatus::Success => Span::styled(
"",
Style::default()
.fg(Color::Green)
.add_modifier(Modifier::BOLD),
),
TaskStatus::LocalCacheKeptExisting | TaskStatus::LocalCache => Span::styled(
"",
Style::default()
.fg(Color::Green)
.add_modifier(Modifier::BOLD),
),
TaskStatus::RemoteCache => Span::styled(
"",
Style::default()
.fg(Color::Green)
.add_modifier(Modifier::BOLD),
),
TaskStatus::Failure => Span::styled(
"",
Style::default().fg(Color::Red).add_modifier(Modifier::BOLD),
),
TaskStatus::Skipped => Span::styled(
"",
Style::default()
.fg(Color::Yellow)
.add_modifier(Modifier::BOLD),
),
TaskStatus::InProgress => Span::styled(
"",
Style::default()
.fg(Color::LightCyan)
.add_modifier(Modifier::BOLD),
),
TaskStatus::NotStarted => Span::styled(
" · ",
Style::default()
.fg(Color::DarkGray)
.add_modifier(Modifier::BOLD),
),
}
}
fn get_base_style(&self, status: TaskStatus) -> Style {
Style::default().fg(match status {
TaskStatus::Success
| TaskStatus::LocalCacheKeptExisting
| TaskStatus::LocalCache
| TaskStatus::RemoteCache => Color::Green,
TaskStatus::Failure => Color::Red,
TaskStatus::Skipped => Color::Yellow,
TaskStatus::InProgress => Color::LightCyan,
TaskStatus::NotStarted => Color::DarkGray,
})
}
/// Calculates appropriate pty dimensions by applying relevant borders and padding adjustments to the given area
pub fn calculate_pty_dimensions(area: Rect) -> (u16, u16) {
// Account for borders and padding correctly
let pty_height = area
.height
.saturating_sub(2) // borders
.saturating_sub(2); // padding (1 top + 1 bottom)
let pty_width = area
.width
.saturating_sub(2) // borders
.saturating_sub(4) // padding (2 left + 2 right)
.saturating_sub(1); // 1 extra (based on empirical testing) to ensure characters are not cut off
// Ensure minimum sizes
let pty_height = pty_height.max(3);
let pty_width = pty_width.max(20);
(pty_height, pty_width)
}
/// Returns whether currently in interactive mode.
pub fn is_currently_interactive(&self) -> bool {
self.pty_data
.as_ref()
.map(|data| data.is_interactive)
.unwrap_or(false)
}
}
impl<'a> StatefulWidget for TerminalPane<'a> {
type State = TerminalPaneState;
fn render(self, area: Rect, buf: &mut Buffer, state: &mut Self::State) {
let base_style = self.get_base_style(state.task_status);
let border_style = if state.is_focused {
base_style
} else {
base_style.add_modifier(Modifier::DIM)
};
let status_icon = self.get_status_icon(state.task_status);
let block = Block::default()
.title(Line::from(vec![
status_icon.clone(),
Span::raw(format!("{} ", state.task_name))
.style(Style::default().fg(Color::White)),
]))
.title_alignment(Alignment::Left)
.borders(Borders::ALL)
.border_type(BorderType::Plain)
.border_style(border_style)
.padding(Padding::new(2, 2, 1, 1));
// If task hasn't started yet, show pending message
if matches!(state.task_status, TaskStatus::NotStarted) {
let message = vec![Line::from(vec![Span::styled(
"Task is pending...",
Style::default().fg(Color::DarkGray),
)])];
let paragraph = Paragraph::new(message)
.block(block)
.alignment(Alignment::Center)
.style(Style::default());
Widget::render(paragraph, area, buf);
return;
}
// If the task is in progress, we need to check if a pty instance is available, and if not
// it implies that the task is being run outside the pseudo-terminal and all we can do is
// wait for the task results to arrive
if matches!(state.task_status, TaskStatus::InProgress) && !state.has_pty {
let message = vec![Line::from(vec![Span::styled(
"Waiting for task results...",
if state.is_focused {
self.get_base_style(TaskStatus::InProgress)
} else {
self.get_base_style(TaskStatus::InProgress)
.add_modifier(Modifier::DIM)
},
)])];
let paragraph = Paragraph::new(message)
.block(block)
.alignment(Alignment::Center)
.style(Style::default());
Widget::render(paragraph, area, buf);
return;
}
let inner_area = block.inner(area);
if let Some(pty_data) = &self.pty_data {
if let Some(pty) = &pty_data.pty {
if let Some(screen) = pty.get_screen() {
let viewport_height = inner_area.height;
let current_scroll = pty.get_scroll_offset();
let total_content_rows = pty.get_total_content_rows();
let scrollable_rows =
total_content_rows.saturating_sub(viewport_height as usize);
let needs_scrollbar = scrollable_rows > 0;
// Reset scrollbar state if no scrolling needed
state.scrollbar_state = if needs_scrollbar {
let position = scrollable_rows.saturating_sub(current_scroll);
state
.scrollbar_state
.content_length(scrollable_rows)
.viewport_content_length(viewport_height as usize)
.position(position)
} else {
ScrollbarState::default()
};
let pseudo_term = PseudoTerminal::new(&screen).block(block);
Widget::render(pseudo_term, area, buf);
// Only render scrollbar if needed
if needs_scrollbar {
let scrollbar = Scrollbar::default()
.orientation(ScrollbarOrientation::VerticalRight)
.begin_symbol(Some(""))
.end_symbol(Some(""))
.style(border_style);
scrollbar.render(area, buf, &mut state.scrollbar_state);
}
// Show interactive/readonly status for focused, non-cache hit, tasks
if state.is_focused && !pty_data.is_cache_hit {
// Bottom right status
let bottom_text = if self.is_currently_interactive() {
Line::from(vec![
Span::raw(" "),
Span::styled("<ctrl>+z", Style::default().fg(Color::Cyan)),
Span::styled(
" to exit interactive ",
Style::default().fg(Color::White),
),
])
} else {
Line::from(vec![
Span::raw(" "),
Span::styled("i", Style::default().fg(Color::Cyan)),
Span::styled(
" to make interactive ",
Style::default().fg(Color::DarkGray),
),
])
};
let text_width = bottom_text
.spans
.iter()
.map(|span| span.content.len())
.sum::<usize>();
let bottom_right_area = Rect {
x: area.x + area.width - text_width as u16 - 3,
y: area.y + area.height - 1,
width: text_width as u16 + 2,
height: 1,
};
Paragraph::new(bottom_text)
.alignment(Alignment::Right)
.style(border_style)
.render(bottom_right_area, buf);
// Top right status
let top_text = if self.is_currently_interactive() {
Line::from(vec![Span::styled(
" INTERACTIVE ",
Style::default().fg(Color::White),
)])
} else {
Line::from(vec![Span::styled(
" NON-INTERACTIVE ",
Style::default().fg(Color::DarkGray),
)])
};
let mode_width = top_text
.spans
.iter()
.map(|span| span.content.len())
.sum::<usize>();
let top_right_area = Rect {
x: area.x + area.width - mode_width as u16 - 3,
y: area.y,
width: mode_width as u16 + 2,
height: 1,
};
Paragraph::new(top_text)
.alignment(Alignment::Right)
.style(border_style)
.render(top_right_area, buf);
} else if needs_scrollbar {
// Render padding for both top and bottom when scrollbar is present
let padding_text = Line::from(vec![Span::raw(" ")]);
let padding_width = 2;
// Top padding
let top_right_area = Rect {
x: area.x + area.width - padding_width - 3,
y: area.y,
width: padding_width + 2,
height: 1,
};
Paragraph::new(padding_text.clone())
.alignment(Alignment::Right)
.style(border_style)
.render(top_right_area, buf);
// Bottom padding
let bottom_right_area = Rect {
x: area.x + area.width - padding_width - 3,
y: area.y + area.height - 1,
width: padding_width + 2,
height: 1,
};
Paragraph::new(padding_text)
.alignment(Alignment::Right)
.style(border_style)
.render(bottom_right_area, buf);
}
}
}
}
}
}
+62
View File
@@ -0,0 +1,62 @@
#[derive(Clone)]
pub struct TuiCliArgs {
pub targets: Vec<String>,
pub tui_auto_exit: Option<AutoExit>,
}
#[derive(Clone)]
pub enum AutoExit {
Boolean(bool),
Integer(u32),
}
impl AutoExit {
pub const DEFAULT_COUNTDOWN_SECONDS: u32 = 3;
// Return whether the TUI should exit automatically
pub fn should_exit_automatically(&self) -> bool {
match self {
// false means don't auto-exit
AutoExit::Boolean(false) => false,
// true means exit immediately (no countdown)
AutoExit::Boolean(true) => true,
// A number means exit after countdown
AutoExit::Integer(_) => true,
}
}
// Get countdown seconds (if countdown is enabled)
pub fn countdown_seconds(&self) -> Option<u32> {
match self {
// false means no auto-exit, so no countdown
AutoExit::Boolean(false) => None,
// true means exit immediately, so no countdown
AutoExit::Boolean(true) => None,
// A number means show countdown for that many seconds
AutoExit::Integer(seconds) => Some(*seconds),
}
}
}
pub struct TuiConfig {
pub auto_exit: AutoExit,
}
impl TuiConfig {
/// Creates a new TuiConfig from nx.json config properties and CLI args
pub fn new(auto_exit: Option<AutoExit>, cli_args: &TuiCliArgs) -> Self {
// Default to 3-second countdown if nothing is specified
let final_auto_exit = match auto_exit {
Some(config) => config,
None => AutoExit::Integer(AutoExit::DEFAULT_COUNTDOWN_SECONDS),
};
// CLI args take precedence over programmatic config
let final_auto_exit = match &cli_args.tui_auto_exit {
Some(cli_value) => cli_value.clone(),
None => final_auto_exit,
};
Self {
auto_exit: final_auto_exit,
}
}
}
+262
View File
@@ -0,0 +1,262 @@
use napi::bindgen_prelude::*;
use napi::threadsafe_function::{ErrorStrategy, ThreadsafeFunction};
use napi::JsObject;
use std::sync::{Arc, Mutex};
use tracing::debug;
use crate::native::logger::enable_logger;
use crate::native::pseudo_terminal::pseudo_terminal::{ParserArc, WriterArc};
use crate::native::tasks::types::{Task, TaskResult};
use super::app::App;
use super::components::tasks_list::TaskStatus;
use super::config::{AutoExit, TuiCliArgs as RustTuiCliArgs, TuiConfig as RustTuiConfig};
use super::tui::Tui;
#[napi(object)]
#[derive(Clone)]
pub struct TuiCliArgs {
#[napi(ts_type = "string[] | undefined")]
pub targets: Option<Vec<String>>,
#[napi(ts_type = "boolean | number | undefined")]
pub tui_auto_exit: Option<Either<bool, u32>>,
}
impl From<TuiCliArgs> for RustTuiCliArgs {
fn from(js: TuiCliArgs) -> Self {
let js_auto_exit = js.tui_auto_exit.map(|value| match value {
Either::A(bool_value) => AutoExit::Boolean(bool_value),
Either::B(int_value) => AutoExit::Integer(int_value),
});
Self {
targets: js.targets.unwrap_or_default(),
tui_auto_exit: js_auto_exit,
}
}
}
#[napi(object)]
pub struct TuiConfig {
#[napi(ts_type = "boolean | number | undefined")]
pub auto_exit: Option<Either<bool, u32>>,
}
impl From<(TuiConfig, &RustTuiCliArgs)> for RustTuiConfig {
fn from((js_tui_config, rust_tui_cli_args): (TuiConfig, &RustTuiCliArgs)) -> Self {
let js_auto_exit = js_tui_config.auto_exit.map(|value| match value {
Either::A(bool_value) => AutoExit::Boolean(bool_value),
Either::B(int_value) => AutoExit::Integer(int_value),
});
// Pass the converted JSON config value(s) and cli_args to instantiate the config with
RustTuiConfig::new(js_auto_exit, &rust_tui_cli_args)
}
}
#[napi]
#[derive(Clone)]
pub struct AppLifeCycle {
app: Arc<Mutex<App>>,
}
#[napi]
impl AppLifeCycle {
#[napi(constructor)]
pub fn new(
tasks: Vec<Task>,
pinned_tasks: Vec<String>,
tui_cli_args: TuiCliArgs,
tui_config: TuiConfig,
title_text: String,
) -> Self {
// Get the target names from nx_args.targets
let rust_tui_cli_args = tui_cli_args.into();
// Convert JSON TUI configuration to our Rust TuiConfig
let rust_tui_config = RustTuiConfig::from((tui_config, &rust_tui_cli_args));
Self {
app: Arc::new(std::sync::Mutex::new(
App::new(
tasks.into_iter().map(|t| t.into()).collect(),
pinned_tasks,
rust_tui_config,
title_text,
)
.unwrap(),
)),
}
}
#[napi]
pub fn start_command(&mut self, thread_count: Option<u32>) -> napi::Result<()> {
if let Ok(mut app) = self.app.lock() {
app.start_command(thread_count);
}
Ok(())
}
#[napi]
pub fn schedule_task(&mut self, _task: Task) -> napi::Result<()> {
// Always intentional noop
Ok(())
}
#[napi]
pub fn start_tasks(&mut self, tasks: Vec<Task>, _metadata: JsObject) -> napi::Result<()> {
if let Ok(mut app) = self.app.lock() {
app.start_tasks(tasks);
}
Ok(())
}
#[napi]
pub fn print_task_terminal_output(
&mut self,
task: Task,
status: String,
output: String,
) -> napi::Result<()> {
if let Ok(mut app) = self.app.lock() {
app.print_task_terminal_output(task.id, status.parse().unwrap(), output);
}
Ok(())
}
#[napi]
pub fn end_tasks(
&mut self,
task_results: Vec<TaskResult>,
_metadata: JsObject,
) -> napi::Result<()> {
if let Ok(mut app) = self.app.lock() {
app.end_tasks(task_results);
}
Ok(())
}
#[napi]
pub fn end_command(&self) -> napi::Result<()> {
if let Ok(mut app) = self.app.lock() {
app.end_command();
}
Ok(())
}
// Rust-only lifecycle method
#[napi(js_name = "__init")]
pub fn __init(
&self,
done_callback: ThreadsafeFunction<(), ErrorStrategy::Fatal>,
) -> napi::Result<()> {
debug!("Initializing Terminal UI");
enable_logger();
let app_mutex = self.app.clone();
// Initialize our Tui abstraction
let mut tui = Tui::new().map_err(|e| napi::Error::from_reason(e.to_string()))?;
tui.enter()
.map_err(|e| napi::Error::from_reason(e.to_string()))?;
std::panic::set_hook(Box::new(move |panic_info| {
// Restore the terminal to a clean state
if let Ok(mut t) = Tui::new() {
if let Err(r) = t.exit() {
debug!("Unable to exit Terminal: {:?}", r);
}
}
// Capture detailed backtraces in development, more concise in production
better_panic::Settings::auto()
.most_recent_first(false)
.lineno_suffix(true)
.verbosity(better_panic::Verbosity::Full)
.create_panic_handler()(panic_info);
}));
debug!("Initialized Terminal UI");
// Set tick and frame rates
tui.tick_rate(10.0);
tui.frame_rate(60.0);
// Initialize action channel
let (action_tx, mut action_rx) = tokio::sync::mpsc::unbounded_channel();
debug!("Initialized Action Channel");
// Initialize components
if let Ok(mut app) = app_mutex.lock() {
// Store callback for cleanup
app.set_done_callback(done_callback);
for component in app.components.iter_mut() {
component.register_action_handler(action_tx.clone()).ok();
component.init().ok();
}
}
debug!("Initialized Components");
napi::tokio::spawn(async move {
loop {
// Handle events using our Tui abstraction
if let Some(event) = tui.next().await {
if let Ok(mut app) = app_mutex.lock() {
if let Ok(true) = app.handle_event(event, &action_tx) {
tui.exit().ok();
app.call_done_callback();
break;
}
}
}
// Process actions
while let Ok(action) = action_rx.try_recv() {
if let Ok(mut app) = app_mutex.lock() {
app.handle_action(&mut tui, action, &action_tx);
// Check if we should quit based on the timer
if let Some(quit_time) = app.quit_at {
if std::time::Instant::now() >= quit_time {
debug!("Quitting TUI");
tui.stop().ok();
debug!("Exiting TUI");
tui.exit().ok();
debug!("Calling exit callback");
app.call_done_callback();
break;
}
}
}
}
}
});
Ok(())
}
#[napi]
pub fn register_running_task(
&mut self,
task_id: String,
parser_and_writer: External<(ParserArc, WriterArc)>,
) {
let mut app = self.app.lock().unwrap();
app.register_running_task(task_id, parser_and_writer, TaskStatus::InProgress)
}
// Rust-only lifecycle method
#[napi(js_name = "__setCloudMessage")]
pub async fn __set_cloud_message(&self, message: String) -> napi::Result<()> {
if let Ok(mut app) = self.app.lock() {
let _ = app.set_cloud_message(Some(message));
}
Ok(())
}
}
#[napi]
pub fn restore_terminal() -> Result<()> {
// TODO: Maybe need some additional cleanup here in addition to the tui cleanup performed at the end of the render loop?
Ok(())
}
+8
View File
@@ -0,0 +1,8 @@
pub mod action;
pub mod app;
pub mod components;
pub mod config;
pub mod lifecycle;
pub mod pty;
pub mod tui;
pub mod utils;
+120
View File
@@ -0,0 +1,120 @@
use std::{
io::{self, Write},
sync::{Arc, Mutex, RwLock},
};
use vt100_ctt::Parser;
use super::utils::normalize_newlines;
#[derive(Clone)]
pub struct PtyInstance {
pub task_id: String,
pub parser: Arc<RwLock<Parser>>,
pub writer: Arc<Mutex<Box<dyn Write + Send>>>,
rows: u16,
cols: u16,
}
impl PtyInstance {
pub fn new(
task_id: String,
parser: Arc<RwLock<Parser>>,
writer: Arc<Mutex<Box<dyn Write + Send>>>,
) -> io::Result<Self> {
// Read the dimensions from the parser
let (rows, cols) = parser.read().unwrap().screen().size();
Ok(Self {
task_id,
parser,
writer,
rows,
cols,
})
}
pub fn resize(&mut self, rows: u16, cols: u16) -> io::Result<()> {
// Ensure minimum sizes
let rows = rows.max(3);
let cols = cols.max(20);
// Get current dimensions before resize
let old_rows = self.rows;
// Update the stored dimensions
self.rows = rows;
self.cols = cols;
// Create a new parser with the new dimensions while preserving state
if let Ok(mut parser_guard) = self.parser.write() {
let raw_output = parser_guard.get_raw_output().to_vec();
// Create new parser with new dimensions
let mut new_parser = Parser::new(rows, cols, 10000);
new_parser.process(&raw_output);
// If we lost height, scroll up by that amount to maintain relative view position
if rows < old_rows {
// Set to 0 to ensure that the cursor is consistently at the bottom of the visible output on resize
new_parser.screen_mut().set_scrollback(0);
}
*parser_guard = new_parser;
}
Ok(())
}
pub fn write_input(&mut self, input: &[u8]) -> io::Result<()> {
if let Ok(mut writer_guard) = self.writer.lock() {
writer_guard.write_all(input)?;
writer_guard.flush()?;
}
Ok(())
}
pub fn get_screen(&self) -> Option<vt100_ctt::Screen> {
self.parser.read().ok().map(|p| p.screen().clone())
}
pub fn scroll_up(&mut self) {
if let Ok(mut parser) = self.parser.write() {
let current = parser.screen().scrollback();
parser.screen_mut().set_scrollback(current + 1);
}
}
pub fn scroll_down(&mut self) {
if let Ok(mut parser) = self.parser.write() {
let current = parser.screen().scrollback();
if current > 0 {
parser.screen_mut().set_scrollback(current - 1);
}
}
}
pub fn get_scroll_offset(&self) -> usize {
if let Ok(parser) = self.parser.read() {
return parser.screen().scrollback();
}
0
}
pub fn get_total_content_rows(&self) -> usize {
if let Ok(parser) = self.parser.read() {
let screen = parser.screen();
screen.get_total_content_rows()
} else {
0
}
}
/// Process output with an existing parser
pub fn process_output(parser: &RwLock<Parser>, output: &[u8]) -> io::Result<()> {
if let Ok(mut parser_guard) = parser.write() {
let normalized = normalize_newlines(output);
parser_guard.process(&normalized);
}
Ok(())
}
}
+212
View File
@@ -0,0 +1,212 @@
use color_eyre::eyre::Result;
use crossterm::{
cursor,
event::{Event as CrosstermEvent, KeyEvent, KeyEventKind, MouseEvent},
terminal::{EnterAlternateScreen, LeaveAlternateScreen},
};
use futures::{FutureExt, StreamExt};
use ratatui::backend::CrosstermBackend as Backend;
use std::{
ops::{Deref, DerefMut},
time::Duration,
};
use tokio::{
sync::mpsc::{self, UnboundedReceiver, UnboundedSender},
task::JoinHandle,
};
use tokio_util::sync::CancellationToken;
use tracing::debug;
pub type Frame<'a> = ratatui::Frame<'a>;
#[derive(Clone, Debug)]
pub enum Event {
Init,
Quit,
Error,
Closed,
Tick,
Render,
FocusGained,
FocusLost,
Paste(String),
Key(KeyEvent),
Mouse(MouseEvent),
Resize(u16, u16),
}
pub struct Tui {
pub terminal: ratatui::Terminal<Backend<std::io::Stderr>>,
pub task: JoinHandle<()>,
pub cancellation_token: CancellationToken,
pub event_rx: UnboundedReceiver<Event>,
pub event_tx: UnboundedSender<Event>,
pub frame_rate: f64,
pub tick_rate: f64,
}
impl Tui {
pub fn new() -> Result<Self> {
let tick_rate = 4.0;
let frame_rate = 60.0;
let terminal = ratatui::Terminal::new(Backend::new(std::io::stderr()))?;
let (event_tx, event_rx) = mpsc::unbounded_channel();
let cancellation_token = CancellationToken::new();
let task = tokio::spawn(async {});
Ok(Self {
terminal,
task,
cancellation_token,
event_rx,
event_tx,
frame_rate,
tick_rate,
})
}
pub fn tick_rate(&mut self, tick_rate: f64) {
self.tick_rate = tick_rate;
}
pub fn frame_rate(&mut self, frame_rate: f64) {
self.frame_rate = frame_rate;
}
pub fn start(&mut self) {
let tick_delay = std::time::Duration::from_secs_f64(1.0 / self.tick_rate);
let render_delay = std::time::Duration::from_secs_f64(1.0 / self.frame_rate);
self.cancel();
self.cancellation_token = CancellationToken::new();
let _cancellation_token = self.cancellation_token.clone();
let _event_tx = self.event_tx.clone();
self.task = tokio::spawn(async move {
let mut reader = crossterm::event::EventStream::new();
let mut tick_interval = tokio::time::interval(tick_delay);
let mut render_interval = tokio::time::interval(render_delay);
_event_tx.send(Event::Init).unwrap();
debug!("Start Listening for Crossterm Events");
loop {
let crossterm_event = reader.next().fuse();
tokio::select! {
_ = _cancellation_token.cancelled() => {
debug!("Got a cancellation token");
break;
}
_ = tick_interval.tick() => {
_event_tx.send(Event::Tick).expect("cannot send event");
},
_ = render_interval.tick() => {
_event_tx.send(Event::Render).expect("cannot send event");
},
maybe_event = crossterm_event => {
debug!("Maybe Crossterm Event: {:?}", maybe_event);
match maybe_event {
Some(Ok(evt)) => {
debug!("Crossterm Event: {:?}", evt);
match evt {
CrosstermEvent::Key(key) if key.kind == KeyEventKind::Press => {
debug!("Key: {:?}", key);
_event_tx.send(Event::Key(key)).unwrap();
},
CrosstermEvent::Mouse(mouse) => {
_event_tx.send(Event::Mouse(mouse)).unwrap();
},
CrosstermEvent::Resize(x, y) => {
_event_tx.send(Event::Resize(x, y)).unwrap();
},
CrosstermEvent::FocusLost => {
_event_tx.send(Event::FocusLost).unwrap();
},
CrosstermEvent::FocusGained => {
_event_tx.send(Event::FocusGained).unwrap();
},
CrosstermEvent::Paste(s) => {
_event_tx.send(Event::Paste(s)).unwrap();
},
_ => {
debug!("Unhandled Crossterm Event: {:?}", evt);
continue;
}
}
}
Some(Err(e)) => {
debug!("Got an error event: {}", e);
_event_tx.send(Event::Error).unwrap();
}
None => {
debug!("Crossterm Stream Stoped");
break;
},
}
},
}
}
debug!("Crossterm Thread Finished")
});
}
pub fn stop(&self) -> Result<()> {
self.cancel();
let mut counter = 0;
while !self.task.is_finished() {
std::thread::sleep(Duration::from_millis(1));
counter += 1;
if counter > 50 {
self.task.abort();
}
if counter > 100 {
// This log is hit most of the time, but this condition does not seem to be problematic in practice
// TODO: Investigate this moore deeply
// log::error!("Failed to abort task in 100 milliseconds for unknown reason");
break;
}
}
Ok(())
}
pub fn enter(&mut self) -> Result<()> {
debug!("Enabling Raw Mode");
crossterm::terminal::enable_raw_mode()?;
crossterm::execute!(std::io::stderr(), EnterAlternateScreen, cursor::Hide)?;
self.start();
Ok(())
}
pub fn exit(&mut self) -> Result<()> {
self.stop()?;
if crossterm::terminal::is_raw_mode_enabled()? {
self.flush()?;
crossterm::execute!(std::io::stderr(), LeaveAlternateScreen, cursor::Show)?;
crossterm::terminal::disable_raw_mode()?;
}
Ok(())
}
pub fn cancel(&self) {
self.cancellation_token.cancel();
}
pub async fn next(&mut self) -> Option<Event> {
self.event_rx.recv().await
}
}
impl Deref for Tui {
type Target = ratatui::Terminal<Backend<std::io::Stderr>>;
fn deref(&self) -> &Self::Target {
&self.terminal
}
}
impl DerefMut for Tui {
fn deref_mut(&mut self) -> &mut Self::Target {
&mut self.terminal
}
}
impl Drop for Tui {
fn drop(&mut self) {
self.exit().unwrap();
}
}
+392
View File
@@ -0,0 +1,392 @@
use crate::native::tui::components::tasks_list::{TaskItem, TaskStatus};
pub fn format_duration(duration_ms: u128) -> String {
if duration_ms == 0 {
"<1ms".to_string()
} else if duration_ms < 1000 {
format!("{}ms", duration_ms)
} else {
format!("{:.1}s", duration_ms as f64 / 1000.0)
}
}
pub fn format_duration_since(start_ms: u128, end_ms: u128) -> String {
format_duration(end_ms.saturating_sub(start_ms))
}
/// Ensures that all newlines in the output are properly handled by converting
/// lone \n to \r\n sequences. This mimics terminal driver behavior.
pub fn normalize_newlines(input: &[u8]) -> Vec<u8> {
let mut output = Vec::with_capacity(input.len());
let mut i = 0;
while i < input.len() {
if input[i] == b'\n' {
// If this \n isn't preceded by \r, add the \r
if i == 0 || input[i - 1] != b'\r' {
output.push(b'\r');
}
}
output.push(input[i]);
i += 1;
}
output
}
pub fn is_cache_hit(status: TaskStatus) -> bool {
matches!(
status,
TaskStatus::LocalCacheKeptExisting | TaskStatus::LocalCache | TaskStatus::RemoteCache
)
}
/// Sorts a list of TaskItems with a stable, total ordering.
///
/// The sort order is:
/// 1. InProgress tasks first
/// 2. Failure tasks second
/// 3. Other completed tasks third (sorted by end_time if available)
/// 4. NotStarted tasks last
///
/// Within each status category:
/// - For completed tasks: sort by end_time if available, then by name
/// - For other statuses: sort by name
pub fn sort_task_items(tasks: &mut [TaskItem]) {
tasks.sort_by(|a, b| {
// Map status to a numeric category for sorting
let status_to_category = |status: &TaskStatus| -> u8 {
match status {
TaskStatus::InProgress => 0,
TaskStatus::Failure => 1,
TaskStatus::Success
| TaskStatus::LocalCacheKeptExisting
| TaskStatus::LocalCache
| TaskStatus::RemoteCache
| TaskStatus::Skipped => 2,
TaskStatus::NotStarted => 3,
}
};
let a_category = status_to_category(&a.status);
let b_category = status_to_category(&b.status);
// First compare by status category
if a_category != b_category {
return a_category.cmp(&b_category);
}
// For completed tasks, sort by end_time if available
if a_category == 1 || a_category == 2 {
// Failure or Success categories
match (a.end_time, b.end_time) {
(Some(time_a), Some(time_b)) => {
let time_cmp = time_a.cmp(&time_b);
if time_cmp != std::cmp::Ordering::Equal {
return time_cmp;
}
}
(Some(_), None) => return std::cmp::Ordering::Less,
(None, Some(_)) => return std::cmp::Ordering::Greater,
(None, None) => {}
}
}
// For all other cases or as a tiebreaker, sort by name
a.name.cmp(&b.name)
});
}
#[cfg(test)]
mod tests {
use super::*;
// Helper function to create a TaskItem for testing
fn create_task(name: &str, status: TaskStatus, end_time: Option<u128>) -> TaskItem {
let mut task = TaskItem::new(name.to_string(), false);
task.status = status;
task.end_time = end_time;
task
}
#[test]
fn test_sort_by_status_category() {
let mut tasks = vec![
create_task("task1", TaskStatus::NotStarted, None),
create_task("task2", TaskStatus::InProgress, None),
create_task("task3", TaskStatus::Success, Some(100)),
create_task("task4", TaskStatus::Failure, Some(200)),
];
sort_task_items(&mut tasks);
// Expected order: InProgress, Failure, Success, NotStarted
assert_eq!(tasks[0].status, TaskStatus::InProgress);
assert_eq!(tasks[1].status, TaskStatus::Failure);
assert_eq!(tasks[2].status, TaskStatus::Success);
assert_eq!(tasks[3].status, TaskStatus::NotStarted);
}
#[test]
fn test_sort_completed_tasks_by_end_time() {
let mut tasks = vec![
create_task("task1", TaskStatus::Success, Some(300)),
create_task("task2", TaskStatus::Success, Some(100)),
create_task("task3", TaskStatus::Success, Some(200)),
];
sort_task_items(&mut tasks);
// Should be sorted by end_time: 100, 200, 300
assert_eq!(tasks[0].name, "task2");
assert_eq!(tasks[1].name, "task3");
assert_eq!(tasks[2].name, "task1");
}
#[test]
fn test_sort_with_missing_end_times() {
let mut tasks = vec![
create_task("task1", TaskStatus::Success, None),
create_task("task2", TaskStatus::Success, Some(100)),
create_task("task3", TaskStatus::Success, None),
];
sort_task_items(&mut tasks);
// Tasks with end_time come before those without
assert_eq!(tasks[0].name, "task2");
// Then alphabetical for those without end_time
assert_eq!(tasks[1].name, "task1");
assert_eq!(tasks[2].name, "task3");
}
#[test]
fn test_sort_same_status_no_end_time_by_name() {
let mut tasks = vec![
create_task("c", TaskStatus::NotStarted, None),
create_task("a", TaskStatus::NotStarted, None),
create_task("b", TaskStatus::NotStarted, None),
];
sort_task_items(&mut tasks);
// Should be sorted alphabetically: a, b, c
assert_eq!(tasks[0].name, "a");
assert_eq!(tasks[1].name, "b");
assert_eq!(tasks[2].name, "c");
}
#[test]
fn test_sort_mixed_statuses_and_end_times() {
let mut tasks = vec![
create_task("z", TaskStatus::NotStarted, None),
create_task("y", TaskStatus::InProgress, None),
create_task("x", TaskStatus::Success, Some(300)),
create_task("w", TaskStatus::Failure, Some(200)),
create_task("v", TaskStatus::Success, None),
create_task("u", TaskStatus::InProgress, None),
create_task("t", TaskStatus::Failure, None),
create_task("s", TaskStatus::NotStarted, None),
];
sort_task_items(&mut tasks);
// Expected groups by status:
// 1. InProgress: "u", "y" (alphabetical)
// 2. Failure: "w" (with end_time), "t" (without end_time)
// 3. Success: "x" (with end_time), "v" (without end_time)
// 4. NotStarted: "s", "z" (alphabetical)
// Check the order within each status group
let names: Vec<&str> = tasks.iter().map(|t| &t.name[..]).collect();
// First group: InProgress
assert_eq!(tasks[0].status, TaskStatus::InProgress);
assert_eq!(tasks[1].status, TaskStatus::InProgress);
assert!(names[0..2].contains(&"u"));
assert!(names[0..2].contains(&"y"));
assert_eq!(names[0], "u"); // Alphabetical within group
// Second group: Failure
assert_eq!(tasks[2].status, TaskStatus::Failure);
assert_eq!(tasks[3].status, TaskStatus::Failure);
assert_eq!(names[2], "w"); // With end_time comes first
assert_eq!(names[3], "t"); // Without end_time comes second
// Third group: Success
assert_eq!(tasks[4].status, TaskStatus::Success);
assert_eq!(tasks[5].status, TaskStatus::Success);
assert_eq!(names[4], "x"); // With end_time comes first
assert_eq!(names[5], "v"); // Without end_time comes second
// Fourth group: NotStarted
assert_eq!(tasks[6].status, TaskStatus::NotStarted);
assert_eq!(tasks[7].status, TaskStatus::NotStarted);
assert_eq!(names[6], "s"); // Alphabetical within group
assert_eq!(names[7], "z");
}
#[test]
fn test_sort_with_same_end_times() {
let mut tasks = vec![
create_task("c", TaskStatus::Success, Some(100)),
create_task("a", TaskStatus::Success, Some(100)),
create_task("b", TaskStatus::Success, Some(100)),
];
sort_task_items(&mut tasks);
// When end_times are the same, should sort by name
assert_eq!(tasks[0].name, "a");
assert_eq!(tasks[1].name, "b");
assert_eq!(tasks[2].name, "c");
}
#[test]
fn test_sort_empty_list() {
let mut tasks: Vec<TaskItem> = vec![];
// Should not panic on empty list
sort_task_items(&mut tasks);
assert!(tasks.is_empty());
}
#[test]
fn test_sort_single_task() {
let mut tasks = vec![create_task("task", TaskStatus::Success, Some(100))];
// Should not change a single-element list
sort_task_items(&mut tasks);
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].name, "task");
}
#[test]
fn test_sort_stability_for_equal_elements() {
// Create tasks with identical properties
let mut tasks = vec![
create_task("task1", TaskStatus::Success, Some(100)),
create_task("task1", TaskStatus::Success, Some(100)),
];
// Mark the original positions
let original_names = tasks.iter().map(|t| t.name.clone()).collect::<Vec<_>>();
// Sort should maintain original order for equal elements
sort_task_items(&mut tasks);
let sorted_names = tasks.iter().map(|t| t.name.clone()).collect::<Vec<_>>();
assert_eq!(sorted_names, original_names);
}
#[test]
fn test_sort_large_random_dataset() {
use rand::rngs::StdRng;
use rand::{Rng, SeedableRng};
// Use a fixed seed for reproducibility
let mut rng = StdRng::seed_from_u64(42);
// Generate a large dataset with random properties
let statuses = [
TaskStatus::InProgress,
TaskStatus::Failure,
TaskStatus::Success,
TaskStatus::NotStarted,
];
let mut tasks: Vec<TaskItem> = (0..1000)
.map(|i| {
let name = format!("task{}", i);
let status = statuses[rng.random_range(0..statuses.len())];
let end_time = if rng.random_bool(0.7) {
Some(rng.random_range(100..10000))
} else {
None
};
create_task(&name, status, end_time)
})
.collect();
// Sort should not panic with large random dataset
sort_task_items(&mut tasks);
// Verify the sort maintains the expected ordering rules
for i in 1..tasks.len() {
let a = &tasks[i - 1];
let b = &tasks[i];
// Map status to category for comparison
let status_to_category = |status: &TaskStatus| -> u8 {
match status {
TaskStatus::InProgress => 0,
TaskStatus::Failure => 1,
TaskStatus::Success
| TaskStatus::LocalCacheKeptExisting
| TaskStatus::LocalCache
| TaskStatus::RemoteCache
| TaskStatus::Skipped => 2,
TaskStatus::NotStarted => 3,
}
};
let a_category = status_to_category(&a.status);
let b_category = status_to_category(&b.status);
if a_category < b_category {
// If a's category is less than b's, that's correct
continue;
} else if a_category > b_category {
// If a's category is greater than b's, that's an error
panic!(
"Sort order violation: {:?} should come before {:?}",
b.name, a.name
);
}
// Same category, check end_time for completed tasks
if a_category == 1 || a_category == 2 {
match (a.end_time, b.end_time) {
(Some(time_a), Some(time_b)) => {
if time_a > time_b {
panic!("Sort order violation: task with end_time {} should come before task with end_time {}", time_b, time_a);
} else if time_a < time_b {
continue;
}
// If end times are equal, fall through to name check
}
(Some(_), None) => continue, // Correct order
(None, Some(_)) => panic!("Sort order violation: task with end_time should come before task without end_time"),
(None, None) => {} // Fall through to name check
}
}
// If we get here, we're comparing names within the same category
// and with the same end_time status
if a.name > b.name {
panic!(
"Sort order violation: task named {} should come before task named {}",
b.name, a.name
);
}
}
}
#[test]
fn test_sort_edge_cases() {
// Test with extreme end_time values
let mut tasks = vec![
create_task("a", TaskStatus::Success, Some(u128::MAX)),
create_task("b", TaskStatus::Success, Some(0)),
create_task("c", TaskStatus::Success, Some(u128::MAX / 2)),
];
sort_task_items(&mut tasks);
// Should sort by end_time: 0, MAX/2, MAX
assert_eq!(tasks[0].name, "b");
assert_eq!(tasks[1].name, "c");
assert_eq!(tasks[2].name, "a");
}
}
@@ -1,16 +1,16 @@
import { findAncestorNodeModules } from './resolution-helpers';
import {
NxCloudClientUnavailableError,
NxCloudEnterpriseOutdatedError,
verifyOrUpdateNxCloudClient,
} from './update-manager';
import { Task } from '../config/task-graph';
import {
defaultTasksRunner,
DefaultTasksRunnerOptions,
} from '../tasks-runner/default-tasks-runner';
import { TasksRunner } from '../tasks-runner/tasks-runner';
import { output } from '../utils/output';
import { Task } from '../config/task-graph';
import { findAncestorNodeModules } from './resolution-helpers';
import {
NxCloudClientUnavailableError,
NxCloudEnterpriseOutdatedError,
verifyOrUpdateNxCloudClient,
} from './update-manager';
export interface CloudTaskRunnerOptions extends DefaultTasksRunnerOptions {
accessToken?: string;
@@ -56,7 +56,7 @@ export const nxCloudTasksRunnerShell: TasksRunner<
if (e instanceof NxCloudEnterpriseOutdatedError) {
output.warn({
title: e.message,
bodyLines: ['Nx Cloud will not used for this command.', ...body],
bodyLines: ['Nx Cloud will not be used for this command.', ...body],
});
}
const results = await defaultTasksRunner(tasks, options, context);
@@ -48,6 +48,7 @@ describe('createTaskGraph', () => {
executor: 'nx:run-commands',
},
serve: {
continuous: true,
executor: 'nx:run-commands',
},
},
@@ -90,6 +91,7 @@ describe('createTaskGraph', () => {
roots: [],
tasks: {},
dependencies: {},
continuousDependencies: {},
});
});
@@ -117,11 +119,15 @@ describe('createTaskGraph', () => {
overrides: { a: 123 },
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:test': [],
},
continuousDependencies: {
'app1:test': [],
},
});
const twoTasks = createTaskGraph(
@@ -148,6 +154,7 @@ describe('createTaskGraph', () => {
overrides: { a: 123 },
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'lib1:test': {
id: 'lib1:test',
@@ -159,12 +166,17 @@ describe('createTaskGraph', () => {
overrides: { a: 123 },
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:test': [],
'lib1:test': [],
},
continuousDependencies: {
'app1:test': [],
'lib1:test': [],
},
});
});
@@ -299,6 +311,7 @@ describe('createTaskGraph', () => {
overrides: {},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
},
'lib2:compile': {
id: 'lib2:compile',
@@ -312,12 +325,17 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib2-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'lib1:compile:libDefault': ['lib2:compile'],
'lib2:compile': [],
},
continuousDependencies: {
'lib1:compile:libDefault': [],
'lib2:compile': [],
},
});
const compileApp = createTaskGraph(
@@ -343,6 +361,7 @@ describe('createTaskGraph', () => {
overrides: {},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'lib1:compile:libDefault': {
id: 'lib1:compile:libDefault',
@@ -357,6 +376,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
},
'lib2:compile:ci': {
id: 'lib2:compile:ci',
@@ -371,6 +391,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib2-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
@@ -378,6 +399,11 @@ describe('createTaskGraph', () => {
'lib1:compile:libDefault': ['lib2:compile:ci'],
'lib2:compile:ci': [],
},
continuousDependencies: {
'app1:compile:ci': [],
'lib1:compile:libDefault': [],
'lib2:compile:ci': [],
},
});
});
@@ -460,6 +486,10 @@ describe('createTaskGraph', () => {
'app1:compile': ['lib3:compile'],
'lib3:compile': [],
},
continuousDependencies: {
'app1:compile': [],
'lib3:compile': [],
},
roots: ['lib3:compile'],
tasks: {
'app1:compile': {
@@ -472,6 +502,7 @@ describe('createTaskGraph', () => {
target: 'compile',
},
parallelism: true,
continuous: false,
},
'lib3:compile': {
id: 'lib3:compile',
@@ -485,6 +516,7 @@ describe('createTaskGraph', () => {
target: 'compile',
},
parallelism: true,
continuous: false,
},
},
});
@@ -514,11 +546,15 @@ describe('createTaskGraph', () => {
overrides: { a: '--value=app1-root' },
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:test': [],
},
continuousDependencies: {
'app1:test': [],
},
});
});
@@ -546,11 +582,15 @@ describe('createTaskGraph', () => {
overrides: { a: '--base-href=/app1-root${deploymentId}' },
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:test': [],
},
continuousDependencies: {
'app1:test': [],
},
});
});
@@ -661,6 +701,7 @@ describe('createTaskGraph', () => {
overrides: { myFlag: 'flag value' },
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app1:precompile': {
id: 'app1:precompile',
@@ -672,6 +713,7 @@ describe('createTaskGraph', () => {
overrides: { myFlag: 'flag value' },
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'lib1:compile': {
id: 'lib1:compile',
@@ -683,6 +725,7 @@ describe('createTaskGraph', () => {
overrides: { myFlag: 'flag value' },
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
},
'lib2:compile': {
id: 'lib2:compile',
@@ -694,6 +737,7 @@ describe('createTaskGraph', () => {
overrides: { __overrides_unparsed__: [] },
projectRoot: 'lib2-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
@@ -702,6 +746,12 @@ describe('createTaskGraph', () => {
'lib1:compile': ['lib2:compile'],
'lib2:compile': [],
},
continuousDependencies: {
'app1:compile': [],
'app1:precompile': [],
'lib1:compile': [],
'lib2:compile': [],
},
});
});
@@ -732,6 +782,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app1:precompile': {
id: 'app1:precompile',
@@ -745,6 +796,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app1:precompile2': {
id: 'app1:precompile2',
@@ -758,6 +810,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'lib1:compile': {
id: 'lib1:compile',
@@ -771,6 +824,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
@@ -779,6 +833,127 @@ describe('createTaskGraph', () => {
'app1:precompile2': [],
'lib1:compile': [],
},
continuousDependencies: {
'app1:compile': [],
'app1:precompile': [],
'app1:precompile2': [],
'lib1:compile': [],
},
});
});
it('should create graphs with continuous dependencies', () => {
projectGraph.nodes['app1'].data.targets['serve'].dependsOn = [
{
dependencies: true,
target: 'serve',
},
{
target: 'compile',
},
];
projectGraph.nodes['app1'].data.targets['compile'].dependsOn = [
{
dependencies: true,
target: 'compile',
},
];
projectGraph.nodes['lib1'].data.targets['serve'] = {
executor: 'nx:run-command',
continuous: true,
dependsOn: [
{
dependencies: true,
target: 'serve',
},
{
target: 'compile',
},
],
};
const taskGraph = createTaskGraph(
projectGraph,
{},
['app1'],
['serve'],
undefined,
{
__overrides_unparsed__: [],
}
);
// precompile should also be in here
expect(taskGraph).toEqual({
roots: ['lib1:compile'],
tasks: {
'app1:serve': {
id: 'app1:serve',
target: {
project: 'app1',
target: 'serve',
},
outputs: [],
overrides: {
__overrides_unparsed__: [],
},
projectRoot: 'app1-root',
parallelism: true,
continuous: true,
},
'app1:compile': {
id: 'app1:compile',
target: {
project: 'app1',
target: 'compile',
},
outputs: [],
overrides: {
__overrides_unparsed__: [],
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'lib1:serve': {
id: 'lib1:serve',
target: {
project: 'lib1',
target: 'serve',
},
outputs: [],
overrides: {
__overrides_unparsed__: [],
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: true,
},
'lib1:compile': {
id: 'lib1:compile',
target: {
project: 'lib1',
target: 'compile',
},
outputs: [],
overrides: {
__overrides_unparsed__: [],
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:serve': ['app1:compile'],
'app1:compile': ['lib1:compile'],
'lib1:serve': ['lib1:compile'],
'lib1:compile': [],
},
continuousDependencies: {
'app1:serve': ['lib1:serve'],
'app1:compile': [],
'lib1:serve': [],
'lib1:compile': [],
},
});
});
@@ -809,6 +984,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app1:precompile': {
id: 'app1:precompile',
@@ -822,6 +998,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app1:precompile2': {
id: 'app1:precompile2',
@@ -835,6 +1012,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'lib1:compile': {
id: 'lib1:compile',
@@ -848,6 +1026,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
@@ -856,6 +1035,12 @@ describe('createTaskGraph', () => {
'app1:precompile2': [],
'lib1:compile': [],
},
continuousDependencies: {
'app1:compile': [],
'app1:precompile': [],
'app1:precompile2': [],
'lib1:compile': [],
},
});
});
@@ -955,6 +1140,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'lib1:compile': {
id: 'lib1:compile',
@@ -968,6 +1154,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
},
'lib2:compile': {
id: 'lib2:compile',
@@ -981,6 +1168,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib2-root',
parallelism: true,
continuous: false,
},
'lib3:compile': {
id: 'lib3:compile',
@@ -994,6 +1182,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib3-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
@@ -1002,6 +1191,12 @@ describe('createTaskGraph', () => {
'lib2:compile': ['lib3:compile'],
'lib3:compile': [],
},
continuousDependencies: {
'app1:compile': [],
'lib1:compile': [],
'lib2:compile': [],
'lib3:compile': [],
},
});
});
@@ -1118,6 +1313,7 @@ describe('createTaskGraph', () => {
outputs: [],
overrides: { myFlag: 'flag value' },
parallelism: true,
continuous: false,
},
'app2:compile': {
id: 'app2:compile',
@@ -1126,6 +1322,7 @@ describe('createTaskGraph', () => {
outputs: [],
overrides: { __overrides_unparsed__: [] },
parallelism: true,
continuous: false,
},
'coreInfra:apply': {
id: 'coreInfra:apply',
@@ -1134,6 +1331,7 @@ describe('createTaskGraph', () => {
outputs: [],
overrides: { myFlag: 'flag value' },
parallelism: true,
continuous: false,
},
'app1:compile': {
id: 'app1:compile',
@@ -1142,6 +1340,7 @@ describe('createTaskGraph', () => {
outputs: [],
overrides: { __overrides_unparsed__: [] },
parallelism: true,
continuous: false,
},
'infra2:apply': {
id: 'infra2:apply',
@@ -1150,6 +1349,7 @@ describe('createTaskGraph', () => {
outputs: [],
overrides: { myFlag: 'flag value' },
parallelism: true,
continuous: false,
},
},
dependencies: {
@@ -1164,6 +1364,13 @@ describe('createTaskGraph', () => {
'app1:compile': [],
'infra2:apply': ['app2:compile', 'coreInfra:apply'],
},
continuousDependencies: {
'infra1:apply': [],
'app2:compile': [],
'coreInfra:apply': [],
'app1:compile': [],
'infra2:apply': [],
},
});
});
@@ -1217,6 +1424,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app1:test': {
id: 'app1:test',
@@ -1230,12 +1438,17 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:compile': ['app1:test'],
'app1:test': ['app1:compile'],
},
continuousDependencies: {
'app1:compile': [],
'app1:test': [],
},
});
});
@@ -1326,6 +1539,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
}),
'lib2:build': expect.objectContaining({
id: 'lib2:build',
@@ -1339,6 +1553,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib2-root',
parallelism: true,
continuous: false,
}),
'lib3:build': expect.objectContaining({
id: 'lib3:build',
@@ -1352,6 +1567,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib3-root',
parallelism: true,
continuous: false,
}),
'lib4:build': expect.objectContaining({
id: 'lib4:build',
@@ -1365,6 +1581,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib4-root',
parallelism: true,
continuous: false,
}),
},
dependencies: {
@@ -1373,6 +1590,12 @@ describe('createTaskGraph', () => {
'lib3:build': ['lib4:build'],
'lib4:build': ['lib1:build'],
},
continuousDependencies: {
'lib1:build': [],
'lib2:build': [],
'lib3:build': [],
'lib4:build': [],
},
});
});
@@ -1458,6 +1681,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
}),
'lib2:build': expect.objectContaining({
id: 'lib2:build',
@@ -1471,6 +1695,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib2-root',
parallelism: true,
continuous: false,
}),
'lib4:build': expect.objectContaining({
id: 'lib4:build',
@@ -1484,6 +1709,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib4-root',
parallelism: true,
continuous: false,
}),
},
dependencies: {
@@ -1491,6 +1717,11 @@ describe('createTaskGraph', () => {
'lib2:build': ['lib4:build'],
'lib4:build': [],
},
continuousDependencies: {
'lib1:build': [],
'lib2:build': [],
'lib4:build': [],
},
});
});
@@ -1551,11 +1782,15 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
}),
},
dependencies: {
'lib1:build': [],
},
continuousDependencies: {
'lib1:build': [],
},
});
});
@@ -1642,6 +1877,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
}),
'lib2:build': expect.objectContaining({
id: 'lib2:build',
@@ -1655,6 +1891,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib2-root',
parallelism: true,
continuous: false,
}),
'lib4:build': expect.objectContaining({
id: 'lib4:build',
@@ -1668,6 +1905,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib4-root',
parallelism: true,
continuous: false,
}),
},
dependencies: {
@@ -1675,6 +1913,11 @@ describe('createTaskGraph', () => {
'lib2:build': [],
'lib4:build': ['lib1:build'],
},
continuousDependencies: {
'lib1:build': [],
'lib2:build': [],
'lib4:build': [],
},
});
});
@@ -1757,6 +2000,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib1-root',
parallelism: true,
continuous: false,
}),
'lib2:build': expect.objectContaining({
id: 'lib2:build',
@@ -1770,12 +2014,17 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib2-root',
parallelism: true,
continuous: false,
}),
},
dependencies: {
'lib1:build': ['lib2:build'],
'lib2:build': [],
},
continuousDependencies: {
'lib1:build': [],
'lib2:build': [],
},
});
});
@@ -1852,6 +2101,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app3:compile': {
id: 'app3:compile',
@@ -1865,12 +2115,17 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app3-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:compile': [],
'app3:compile': [],
},
continuousDependencies: {
'app1:compile': [],
'app3:compile': [],
},
});
});
@@ -1944,6 +2199,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app3:compile': {
id: 'app3:compile',
@@ -1957,12 +2213,17 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app3-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:compile': [],
'app3:compile': [],
},
continuousDependencies: {
'app1:compile': [],
'app3:compile': [],
},
});
});
@@ -2042,6 +2303,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app1:test': {
id: 'app1:test',
@@ -2055,6 +2317,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'lib2:dep': {
id: 'lib2:dep',
@@ -2068,6 +2331,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib2-root',
parallelism: true,
continuous: false,
},
'lib2:dep2': {
id: 'lib2:dep2',
@@ -2081,6 +2345,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'lib2-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
@@ -2089,6 +2354,12 @@ describe('createTaskGraph', () => {
'lib2:dep': [],
'lib2:dep2': [],
},
continuousDependencies: {
'app1:lint': [],
'app1:test': [],
'lib2:dep': [],
'lib2:dep2': [],
},
});
});
@@ -2160,11 +2431,15 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:compile': [],
},
continuousDependencies: {
'app1:compile': [],
},
});
const taskGraph2 = createTaskGraph(
@@ -2194,6 +2469,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app2:compile': {
id: 'app2:compile',
@@ -2207,12 +2483,17 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app2-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:compile': ['app2:compile'],
'app2:compile': [],
},
continuousDependencies: {
'app1:compile': [],
'app2:compile': [],
},
});
});
@@ -2706,6 +2987,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app1-root',
parallelism: true,
continuous: false,
},
'app4:precompile': {
id: 'app4:precompile',
@@ -2719,12 +3001,17 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app4-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
'app1:compile': ['app4:precompile'],
'app4:precompile': [],
},
continuousDependencies: {
'app1:compile': [],
'app4:precompile': [],
},
});
taskGraph = createTaskGraph(
@@ -2752,6 +3039,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app2-root',
parallelism: true,
continuous: false,
},
'app3:compile': {
id: 'app3:compile',
@@ -2765,6 +3053,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app3-root',
parallelism: true,
continuous: false,
},
'app4:precompile': {
id: 'app4:precompile',
@@ -2778,6 +3067,7 @@ describe('createTaskGraph', () => {
},
projectRoot: 'app4-root',
parallelism: true,
continuous: false,
},
},
dependencies: {
@@ -2785,6 +3075,11 @@ describe('createTaskGraph', () => {
'app3:compile': ['app4:precompile'],
'app4:precompile': [],
},
continuousDependencies: {
'app2:compile': [],
'app3:compile': [],
'app4:precompile': [],
},
});
});
@@ -16,6 +16,7 @@ export class ProcessTasks {
private readonly seen = new Set<string>();
readonly tasks: { [id: string]: Task } = {};
readonly dependencies: { [k: string]: string[] } = {};
readonly continuousDependencies: { [k: string]: string[] } = {};
private readonly allTargetNames: string[];
constructor(
@@ -48,7 +49,7 @@ export class ProcessTasks {
target,
configuration
);
const id = this.getId(projectName, target, resolvedConfiguration);
const id = createTaskId(projectName, target, resolvedConfiguration);
const task = this.createTask(
id,
project,
@@ -58,6 +59,7 @@ export class ProcessTasks {
);
this.tasks[task.id] = task;
this.dependencies[task.id] = [];
this.continuousDependencies[task.id] = [];
}
}
}
@@ -75,6 +77,7 @@ export class ProcessTasks {
if (!initialTasks[t]) {
delete this.tasks[t];
delete this.dependencies[t];
delete this.continuousDependencies[t];
}
}
for (let d of Object.keys(this.dependencies)) {
@@ -82,6 +85,11 @@ export class ProcessTasks {
(dd) => !!initialTasks[dd]
);
}
for (let d of Object.keys(this.continuousDependencies)) {
this.continuousDependencies[d] = this.continuousDependencies[d].filter(
(dd) => !!initialTasks[dd]
);
}
}
filterDummyTasks(this.dependencies);
@@ -96,8 +104,22 @@ export class ProcessTasks {
}
}
return Object.keys(this.dependencies).filter(
(d) => this.dependencies[d].length === 0
filterDummyTasks(this.continuousDependencies);
for (const taskId of Object.keys(this.continuousDependencies)) {
if (this.continuousDependencies[taskId].length > 0) {
this.continuousDependencies[taskId] = [
...new Set(
this.continuousDependencies[taskId].filter((d) => d !== taskId)
).values(),
];
}
}
return Object.keys(this.tasks).filter(
(d) =>
this.dependencies[d].length === 0 &&
this.continuousDependencies[d].length === 0
);
}
@@ -199,14 +221,11 @@ export class ProcessTasks {
dependencyConfig.target,
configuration
);
const selfTaskId = this.getId(
const selfTaskId = createTaskId(
selfProject.name,
dependencyConfig.target,
resolvedConfiguration
);
if (task.id !== selfTaskId) {
this.dependencies[task.id].push(selfTaskId);
}
if (!this.tasks[selfTaskId]) {
const newTask = this.createTask(
selfTaskId,
@@ -217,6 +236,7 @@ export class ProcessTasks {
);
this.tasks[selfTaskId] = newTask;
this.dependencies[selfTaskId] = [];
this.continuousDependencies[selfTaskId] = [];
this.processTask(
newTask,
newTask.target.project,
@@ -224,6 +244,13 @@ export class ProcessTasks {
overrides
);
}
if (task.id !== selfTaskId) {
if (this.tasks[selfTaskId].continuous) {
this.continuousDependencies[task.id].push(selfTaskId);
} else {
this.dependencies[task.id].push(selfTaskId);
}
}
}
}
@@ -259,14 +286,23 @@ export class ProcessTasks {
dependencyConfig.target,
configuration
);
const depTargetId = this.getId(
const depTargetId = createTaskId(
depProject.name,
dependencyConfig.target,
resolvedConfiguration
);
const depTargetConfiguration =
this.projectGraph.nodes[depProject.name].data.targets[
dependencyConfig.target
];
if (task.id !== depTargetId) {
this.dependencies[task.id].push(depTargetId);
if (depTargetConfiguration.continuous) {
this.continuousDependencies[task.id].push(depTargetId);
} else {
this.dependencies[task.id].push(depTargetId);
}
}
if (!this.tasks[depTargetId]) {
const newTask = this.createTask(
@@ -278,6 +314,7 @@ export class ProcessTasks {
);
this.tasks[depTargetId] = newTask;
this.dependencies[depTargetId] = [];
this.continuousDependencies[depTargetId] = [];
this.processTask(
newTask,
@@ -288,7 +325,7 @@ export class ProcessTasks {
}
} else {
// Create a dummy task for task.target.project... which simulates if depProject had dependencyConfig.target
const dummyId = this.getId(
const dummyId = createTaskId(
depProject.name,
task.target.project +
'__' +
@@ -298,6 +335,7 @@ export class ProcessTasks {
);
this.dependencies[task.id].push(dummyId);
this.dependencies[dummyId] ??= [];
this.continuousDependencies[dummyId] ??= [];
const noopTask = this.createDummyTask(dummyId, task);
this.processTask(noopTask, depProject.name, configuration, overrides);
}
@@ -354,6 +392,7 @@ export class ProcessTasks {
),
cache: project.data.targets[target].cache,
parallelism: project.data.targets[target].parallelism ?? true,
continuous: project.data.targets[target].continuous ?? false,
};
}
@@ -369,18 +408,6 @@ export class ProcessTasks {
? configuration
: defaultConfiguration;
}
getId(
project: string,
target: string,
configuration: string | undefined
): string {
let id = `${project}:${target}`;
if (configuration) {
id += `:${configuration}`;
}
return id;
}
}
export function createTaskGraph(
@@ -405,6 +432,7 @@ export function createTaskGraph(
roots,
tasks: p.tasks,
dependencies: p.dependencies,
continuousDependencies: p.continuousDependencies,
};
}
@@ -492,3 +520,15 @@ export function getNonDummyDeps(
return [currentTask];
}
}
export function createTaskId(
project: string,
target: string,
configuration: string | undefined
): string {
let id = `${project}:${target}`;
if (configuration) {
id += `:${configuration}`;
}
return id;
}
@@ -1,5 +1,5 @@
import { TasksRunner, TaskStatus } from './tasks-runner';
import { TaskOrchestrator } from './task-orchestrator';
import { getThreadCount, TaskOrchestrator } from './task-orchestrator';
import { TaskHasher } from '../hasher/task-hasher';
import { LifeCycle } from './life-cycle';
import { ProjectGraph } from '../config/project-graph';
@@ -121,30 +121,16 @@ export const defaultTasksRunner: TasksRunner<
daemon: DaemonClient;
}
): Promise<{ [id: string]: TaskStatus }> => {
if (
(options as any)['parallel'] === 'false' ||
(options as any)['parallel'] === false
) {
(options as any)['parallel'] = 1;
} else if (
(options as any)['parallel'] === 'true' ||
(options as any)['parallel'] === true ||
(options as any)['parallel'] === undefined ||
(options as any)['parallel'] === ''
) {
(options as any)['parallel'] = Number((options as any)['maxParallel'] || 3);
}
await options.lifeCycle.startCommand();
const threadCount = getThreadCount(options, context.taskGraph);
await options.lifeCycle.startCommand(threadCount);
try {
return await runAllTasks(tasks, options, context);
return await runAllTasks(options, context);
} finally {
await options.lifeCycle.endCommand();
}
};
async function runAllTasks(
tasks: Task[],
options: DefaultTasksRunnerOptions,
context: {
initiatingProject?: string;
@@ -1,26 +1,23 @@
import { readFileSync, writeFileSync } from 'fs';
import { ChildProcess, fork, Serializable } from 'child_process';
import * as chalk from 'chalk';
import { writeFileSync } from 'fs';
import { fork, Serializable } from 'child_process';
import { DefaultTasksRunnerOptions } from './default-tasks-runner';
import { output } from '../utils/output';
import { getCliPath, getPrintableCommandArgsForTask } from './utils';
import { Batch } from './tasks-schedule';
import { join } from 'path';
import {
BatchMessage,
BatchMessageType,
BatchResults,
} from './batch/batch-messages';
import { BatchMessageType } from './batch/batch-messages';
import { stripIndents } from '../utils/strip-indents';
import { Task, TaskGraph } from '../config/task-graph';
import { Transform } from 'stream';
import {
PseudoTtyProcess,
getPseudoTerminal,
PseudoTerminal,
} from './pseudo-terminal';
import { PseudoTerminal, PseudoTtyProcess } from './pseudo-terminal';
import { signalToCode } from '../utils/exit-codes';
import { ProjectGraph } from '../config/project-graph';
import {
NodeChildProcessWithDirectOutput,
NodeChildProcessWithNonDirectOutput,
} from './running-tasks/node-child-process';
import { BatchProcess } from './running-tasks/batch-process';
import { RunningTask } from './running-tasks/running-task';
import { RustPseudoTerminal } from '../native';
const forkScript = join(__dirname, './fork.js');
@@ -30,99 +27,60 @@ export class ForkedProcessTaskRunner {
cliPath = getCliPath();
private readonly verbose = process.env.NX_VERBOSE_LOGGING === 'true';
private processes = new Set<ChildProcess | PseudoTtyProcess>();
private processes = new Set<RunningTask | BatchProcess>();
private pseudoTerminals = new Set<PseudoTerminal>();
private pseudoTerminal: PseudoTerminal | null = PseudoTerminal.isSupported()
? getPseudoTerminal()
: null;
constructor(private readonly options: DefaultTasksRunnerOptions) {}
constructor(
private readonly options: DefaultTasksRunnerOptions,
private readonly tuiEnabled: boolean
) {}
async init() {
if (this.pseudoTerminal) {
await this.pseudoTerminal.init();
}
this.setupProcessEventListeners();
}
// TODO: vsavkin delegate terminal output printing
public forkProcessForBatch(
public async forkProcessForBatch(
{ executorName, taskGraph: batchTaskGraph }: Batch,
projectGraph: ProjectGraph,
fullTaskGraph: TaskGraph,
env: NodeJS.ProcessEnv
) {
return new Promise<BatchResults>((res, rej) => {
try {
const count = Object.keys(batchTaskGraph.tasks).length;
if (count > 1) {
output.logSingleLine(
`Running ${output.bold(count)} ${output.bold(
'tasks'
)} with ${output.bold(executorName)}`
);
} else {
const args = getPrintableCommandArgsForTask(
Object.values(batchTaskGraph.tasks)[0]
);
output.logCommand(args.join(' '));
}
): Promise<BatchProcess> {
const count = Object.keys(batchTaskGraph.tasks).length;
if (count > 1) {
output.logSingleLine(
`Running ${output.bold(count)} ${output.bold(
'tasks'
)} with ${output.bold(executorName)}`
);
} else {
const args = getPrintableCommandArgsForTask(
Object.values(batchTaskGraph.tasks)[0]
);
output.logCommand(args.join(' '));
}
const p = fork(workerPath, {
stdio: ['inherit', 'inherit', 'inherit', 'ipc'],
env,
});
this.processes.add(p);
p.once('exit', (code, signal) => {
this.processes.delete(p);
if (code === null) code = signalToCode(signal);
if (code !== 0) {
const results: BatchResults = {};
for (const rootTaskId of batchTaskGraph.roots) {
results[rootTaskId] = {
success: false,
terminalOutput: '',
};
}
rej(
new Error(
`"${executorName}" exited unexpectedly with code: ${code}`
)
);
}
});
p.on('message', (message: BatchMessage) => {
switch (message.type) {
case BatchMessageType.CompleteBatchExecution: {
res(message.results);
break;
}
case BatchMessageType.RunTasks: {
break;
}
default: {
// Re-emit any non-batch messages from the task process
if (process.send) {
process.send(message);
}
}
}
});
// Start the tasks
p.send({
type: BatchMessageType.RunTasks,
executorName,
projectGraph,
batchTaskGraph,
fullTaskGraph,
});
} catch (e) {
rej(e);
}
const p = fork(workerPath, {
stdio: ['inherit', 'inherit', 'inherit', 'ipc'],
env,
});
const cp = new BatchProcess(p, executorName);
this.processes.add(cp);
cp.onExit(() => {
this.processes.delete(cp);
});
// Start the tasks
cp.send({
type: BatchMessageType.RunTasks,
executorName,
projectGraph,
batchTaskGraph,
fullTaskGraph,
});
return cp;
}
public async forkProcessLegacy(
@@ -140,15 +98,15 @@ export class ForkedProcessTaskRunner {
taskGraph: TaskGraph;
env: NodeJS.ProcessEnv;
}
): Promise<{ code: number; terminalOutput: string }> {
): Promise<RunningTask> {
return pipeOutput
? await this.forkProcessPipeOutputCapture(task, {
? this.forkProcessWithPrefixAndNotTTY(task, {
temporaryOutputPath,
streamOutput,
taskGraph,
env,
})
: await this.forkProcessDirectOutputCapture(task, {
: this.forkProcessDirectOutputCapture(task, {
temporaryOutputPath,
streamOutput,
taskGraph,
@@ -172,34 +130,47 @@ export class ForkedProcessTaskRunner {
env: NodeJS.ProcessEnv;
disablePseudoTerminal: boolean;
}
): Promise<{ code: number; terminalOutput: string }> {
): Promise<RunningTask | PseudoTtyProcess> {
const shouldPrefix =
streamOutput && process.env.NX_PREFIX_OUTPUT === 'true';
streamOutput &&
process.env.NX_PREFIX_OUTPUT === 'true' &&
!this.tuiEnabled;
// streamOutput would be false if we are running multiple targets
// there's no point in running the commands in a pty if we are not streaming the output
if (
!this.pseudoTerminal ||
disablePseudoTerminal ||
!streamOutput ||
shouldPrefix
PseudoTerminal.isSupported() &&
!disablePseudoTerminal &&
(this.tuiEnabled || (streamOutput && !shouldPrefix))
) {
return this.forkProcessWithPrefixAndNotTTY(task, {
temporaryOutputPath,
streamOutput,
taskGraph,
env,
});
} else {
return this.forkProcessWithPseudoTerminal(task, {
temporaryOutputPath,
streamOutput,
taskGraph,
env,
});
} else {
return this.forkProcessWithPrefixAndNotTTY(task, {
temporaryOutputPath,
streamOutput,
taskGraph,
env,
});
}
}
private async createPseudoTerminal() {
const terminal = new PseudoTerminal(new RustPseudoTerminal());
await terminal.init();
terminal.onMessageFromChildren((message: Serializable) => {
process.send(message);
});
return terminal;
}
private async forkProcessWithPseudoTerminal(
task: Task,
{
@@ -213,14 +184,11 @@ export class ForkedProcessTaskRunner {
taskGraph: TaskGraph;
env: NodeJS.ProcessEnv;
}
): Promise<{ code: number; terminalOutput: string }> {
const args = getPrintableCommandArgsForTask(task);
if (streamOutput) {
output.logCommand(args.join(' '));
}
): Promise<PseudoTtyProcess> {
const childId = task.id;
const p = await this.pseudoTerminal.fork(childId, forkScript, {
const pseudoTerminal = await this.createPseudoTerminal();
this.pseudoTerminals.add(pseudoTerminal);
const p = await pseudoTerminal.fork(childId, forkScript, {
cwd: process.cwd(),
execArgv: process.execArgv,
jsEnv: env,
@@ -240,41 +208,16 @@ export class ForkedProcessTaskRunner {
terminalOutput += msg;
});
return new Promise((res) => {
p.onExit((code) => {
// If the exit code is greater than 128, it's a special exit code for a signal
if (code >= 128) {
process.exit(code);
}
this.writeTerminalOutput(temporaryOutputPath, terminalOutput);
res({
code,
terminalOutput,
});
});
p.onExit((code) => {
if (code > 128) {
process.exit(code);
}
this.pseudoTerminals.delete(pseudoTerminal);
this.processes.delete(p);
this.writeTerminalOutput(temporaryOutputPath, terminalOutput);
});
}
private forkProcessPipeOutputCapture(
task: Task,
{
streamOutput,
temporaryOutputPath,
taskGraph,
env,
}: {
streamOutput: boolean;
temporaryOutputPath: string;
taskGraph: TaskGraph;
env: NodeJS.ProcessEnv;
}
) {
return this.forkProcessWithPrefixAndNotTTY(task, {
streamOutput,
temporaryOutputPath,
taskGraph,
env,
});
return p;
}
private forkProcessWithPrefixAndNotTTY(
@@ -291,85 +234,49 @@ export class ForkedProcessTaskRunner {
env: NodeJS.ProcessEnv;
}
) {
return new Promise<{ code: number; terminalOutput: string }>((res, rej) => {
try {
const args = getPrintableCommandArgsForTask(task);
if (streamOutput) {
output.logCommand(args.join(' '));
}
const p = fork(this.cliPath, {
stdio: ['inherit', 'pipe', 'pipe', 'ipc'],
env,
});
this.processes.add(p);
// Re-emit any messages from the task process
p.on('message', (message) => {
if (process.send) {
process.send(message);
}
});
// Send message to run the executor
p.send({
targetDescription: task.target,
overrides: task.overrides,
taskGraph,
isVerbose: this.verbose,
});
if (streamOutput) {
if (process.env.NX_PREFIX_OUTPUT === 'true') {
const color = getColor(task.target.project);
const prefixText = `${task.target.project}:`;
p.stdout
.pipe(
logClearLineToPrefixTransformer(color.bold(prefixText) + ' ')
)
.pipe(addPrefixTransformer(color.bold(prefixText)))
.pipe(process.stdout);
p.stderr
.pipe(logClearLineToPrefixTransformer(color(prefixText) + ' '))
.pipe(addPrefixTransformer(color(prefixText)))
.pipe(process.stderr);
} else {
p.stdout.pipe(addPrefixTransformer()).pipe(process.stdout);
p.stderr.pipe(addPrefixTransformer()).pipe(process.stderr);
}
}
let outWithErr = [];
p.stdout.on('data', (chunk) => {
outWithErr.push(chunk.toString());
});
p.stderr.on('data', (chunk) => {
outWithErr.push(chunk.toString());
});
p.on('exit', (code, signal) => {
this.processes.delete(p);
if (code === null) code = signalToCode(signal);
// we didn't print any output as we were running the command
// print all the collected output|
const terminalOutput = outWithErr.join('');
if (!streamOutput) {
this.options.lifeCycle.printTaskTerminalOutput(
task,
code === 0 ? 'success' : 'failure',
terminalOutput
);
}
this.writeTerminalOutput(temporaryOutputPath, terminalOutput);
res({ code, terminalOutput });
});
} catch (e) {
console.error(e);
rej(e);
try {
const args = getPrintableCommandArgsForTask(task);
if (streamOutput) {
output.logCommand(args.join(' '));
}
});
const p = fork(this.cliPath, {
stdio: ['inherit', 'pipe', 'pipe', 'ipc'],
env,
});
// Send message to run the executor
p.send({
targetDescription: task.target,
overrides: task.overrides,
taskGraph,
isVerbose: this.verbose,
});
const cp = new NodeChildProcessWithNonDirectOutput(p, {
streamOutput,
prefix: task.target.project,
});
this.processes.add(cp);
cp.onExit((code, terminalOutput) => {
this.processes.delete(cp);
if (!streamOutput) {
this.options.lifeCycle.printTaskTerminalOutput(
task,
code === 0 ? 'success' : 'failure',
terminalOutput
);
}
this.writeTerminalOutput(temporaryOutputPath, terminalOutput);
});
return cp;
} catch (e) {
console.error(e);
throw e;
}
}
private forkProcessDirectOutputCapture(
@@ -386,70 +293,56 @@ export class ForkedProcessTaskRunner {
env: NodeJS.ProcessEnv;
}
) {
return new Promise<{ code: number; terminalOutput: string }>((res, rej) => {
try {
const args = getPrintableCommandArgsForTask(task);
if (streamOutput) {
output.logCommand(args.join(' '));
}
const p = fork(this.cliPath, {
stdio: ['inherit', 'inherit', 'inherit', 'ipc'],
env,
});
this.processes.add(p);
try {
const args = getPrintableCommandArgsForTask(task);
if (streamOutput) {
output.logCommand(args.join(' '));
}
const p = fork(this.cliPath, {
stdio: ['inherit', 'inherit', 'inherit', 'ipc'],
env,
});
const cp = new NodeChildProcessWithDirectOutput(p, temporaryOutputPath);
// Re-emit any messages from the task process
p.on('message', (message) => {
if (process.send) {
process.send(message);
this.processes.add(cp);
// Send message to run the executor
p.send({
targetDescription: task.target,
overrides: task.overrides,
taskGraph,
isVerbose: this.verbose,
});
cp.onExit((code, signal) => {
this.processes.delete(cp);
// we didn't print any output as we were running the command
// print all the collected output
try {
const terminalOutput = cp.getTerminalOutput();
if (!streamOutput) {
this.options.lifeCycle.printTaskTerminalOutput(
task,
code === 0 ? 'success' : 'failure',
terminalOutput
);
}
});
// Send message to run the executor
p.send({
targetDescription: task.target,
overrides: task.overrides,
taskGraph,
isVerbose: this.verbose,
});
p.on('exit', (code, signal) => {
if (code === null) code = signalToCode(signal);
// we didn't print any output as we were running the command
// print all the collected output
let terminalOutput = '';
try {
terminalOutput = this.readTerminalOutput(temporaryOutputPath);
if (!streamOutput) {
this.options.lifeCycle.printTaskTerminalOutput(
task,
code === 0 ? 'success' : 'failure',
terminalOutput
);
}
} catch (e) {
console.log(stripIndents`
} catch (e) {
console.log(stripIndents`
Unable to print terminal output for Task "${task.id}".
Task failed with Exit Code ${code} and Signal "${signal}".
Received error message:
${e.message}
`);
}
res({
code,
terminalOutput,
});
});
} catch (e) {
console.error(e);
rej(e);
}
});
}
}
});
private readTerminalOutput(outputPath: string) {
return readFileSync(outputPath).toString();
return cp;
} catch (e) {
console.error(e);
throw e;
}
}
private writeTerminalOutput(outputPath: string, content: string) {
@@ -457,21 +350,14 @@ export class ForkedProcessTaskRunner {
}
private setupProcessEventListeners() {
if (this.pseudoTerminal) {
this.pseudoTerminal.onMessageFromChildren((message: Serializable) => {
process.send(message);
});
}
// When the nx process gets a message, it will be sent into the task's process
process.on('message', (message: Serializable) => {
// this.publisher.publish(message.toString());
if (this.pseudoTerminal) {
this.pseudoTerminal.sendMessageToChildren(message);
}
this.pseudoTerminals.forEach((p) => {
p.sendMessageToChildren(message);
});
this.processes.forEach((p) => {
if ('connected' in p && p.connected) {
if ('send' in p) {
p.send(message);
}
});
@@ -480,94 +366,29 @@ export class ForkedProcessTaskRunner {
// Terminate any task processes on exit
process.on('exit', () => {
this.processes.forEach((p) => {
if ('connected' in p ? p.connected : p.isAlive) {
p.kill();
}
p.kill();
});
});
process.on('SIGINT', () => {
this.processes.forEach((p) => {
if ('connected' in p ? p.connected : p.isAlive) {
p.kill('SIGTERM');
}
p.kill('SIGTERM');
});
// we exit here because we don't need to write anything to cache.
process.exit(signalToCode('SIGINT'));
});
process.on('SIGTERM', () => {
this.processes.forEach((p) => {
if ('connected' in p ? p.connected : p.isAlive) {
p.kill('SIGTERM');
}
p.kill('SIGTERM');
});
// no exit here because we expect child processes to terminate which
// will store results to the cache and will terminate this process
});
process.on('SIGHUP', () => {
this.processes.forEach((p) => {
if ('connected' in p ? p.connected : p.isAlive) {
p.kill('SIGTERM');
}
p.kill('SIGTERM');
});
// no exit here because we expect child processes to terminate which
// will store results to the cache and will terminate this process
});
}
}
const colors = [
chalk.green,
chalk.greenBright,
chalk.red,
chalk.redBright,
chalk.cyan,
chalk.cyanBright,
chalk.yellow,
chalk.yellowBright,
chalk.magenta,
chalk.magentaBright,
];
function getColor(projectName: string) {
let code = 0;
for (let i = 0; i < projectName.length; ++i) {
code += projectName.charCodeAt(i);
}
const colorIndex = code % colors.length;
return colors[colorIndex];
}
/**
* Prevents terminal escape sequence from clearing line prefix.
*/
function logClearLineToPrefixTransformer(prefix: string) {
let prevChunk = null;
return new Transform({
transform(chunk, _encoding, callback) {
if (prevChunk && prevChunk.toString() === '\x1b[2K') {
chunk = chunk.toString().replace(/\x1b\[1G/g, (m) => m + prefix);
}
this.push(chunk);
prevChunk = chunk;
callback();
},
});
}
function addPrefixTransformer(prefix?: string) {
const newLineSeparator = process.platform.startsWith('win') ? '\r\n' : '\n';
return new Transform({
transform(chunk, _encoding, callback) {
const list = chunk.toString().split(/\r\n|[\n\v\f\r\x85\u2028\u2029]/g);
list
.filter(Boolean)
.forEach((m) =>
this.push(
prefix ? prefix + ' ' + m + newLineSeparator : m + newLineSeparator
)
);
callback();
},
});
}
@@ -1,14 +1,29 @@
import { readNxJson } from '../config/configuration';
import { readNxJson } from '../config/nx-json';
import { NxArgs } from '../utils/command-line-utils';
import { createProjectGraphAsync } from '../project-graph/project-graph';
import { Task, TaskGraph } from '../config/task-graph';
import { invokeTasksRunner } from './run-command';
import {
constructLifeCycles,
getRunner,
invokeTasksRunner,
} from './run-command';
import { InvokeRunnerTerminalOutputLifeCycle } from './life-cycles/invoke-runner-terminal-output-life-cycle';
import { performance } from 'perf_hooks';
import { getOutputs } from './utils';
import { loadRootEnvFiles } from '../utils/dotenv';
import { TaskResult } from './life-cycle';
import { CompositeLifeCycle, LifeCycle, TaskResult } from './life-cycle';
import { TaskOrchestrator } from './task-orchestrator';
import { createTaskHasher } from '../hasher/create-task-hasher';
import type { ProjectGraph } from '../config/project-graph';
import type { NxJsonConfiguration } from '../config/nx-json';
import { daemonClient } from '../daemon/client/client';
import { RunningTask } from './running-tasks/running-task';
import { TaskResultsLifeCycle } from './life-cycles/task-results-life-cycle';
/**
* This function is deprecated. Do not use this
* @deprecated This function is deprecated. Do not use this
*/
export async function initTasksRunner(nxArgs: NxArgs) {
performance.mark('init-local');
loadRootEnvFiles();
@@ -47,6 +62,10 @@ export async function initTasksRunner(nxArgs: NxArgs) {
acc[task.id] = [];
return acc;
}, {} as any),
continuousDependencies: opts.tasks.reduce((acc, task) => {
acc[task.id] = [];
return acc;
}, {} as any),
};
const taskResults = await invokeTasksRunner({
@@ -73,3 +92,101 @@ export async function initTasksRunner(nxArgs: NxArgs) {
},
};
}
async function createOrchestrator(
tasks: Task[],
projectGraph: ProjectGraph,
taskGraphForHashing: TaskGraph,
nxJson: NxJsonConfiguration,
lifeCycle: LifeCycle
) {
loadRootEnvFiles();
const invokeRunnerTerminalLifecycle = new InvokeRunnerTerminalOutputLifeCycle(
tasks
);
const taskResultsLifecycle = new TaskResultsLifeCycle();
const compositedLifeCycle: LifeCycle = new CompositeLifeCycle([
...constructLifeCycles(invokeRunnerTerminalLifecycle),
taskResultsLifecycle,
lifeCycle,
]);
const { runnerOptions: options } = getRunner({}, nxJson);
let hasher = createTaskHasher(projectGraph, nxJson, options);
const taskGraph: TaskGraph = {
roots: tasks.map((task) => task.id),
tasks: tasks.reduce((acc, task) => {
acc[task.id] = task;
return acc;
}, {} as any),
dependencies: tasks.reduce((acc, task) => {
acc[task.id] = [];
return acc;
}, {} as any),
continuousDependencies: tasks.reduce((acc, task) => {
acc[task.id] = [];
return acc;
}, {} as any),
};
const orchestrator = new TaskOrchestrator(
hasher,
null,
projectGraph,
taskGraph,
nxJson,
{ ...options, parallel: tasks.length, lifeCycle: compositedLifeCycle },
false,
daemonClient,
undefined,
taskGraphForHashing
);
await orchestrator.init();
await Promise.all(tasks.map((task) => orchestrator.processTask(task.id)));
return orchestrator;
}
export async function runDiscreteTasks(
tasks: Task[],
projectGraph: ProjectGraph,
taskGraphForHashing: TaskGraph,
nxJson: NxJsonConfiguration,
lifeCycle: LifeCycle
) {
const orchestrator = await createOrchestrator(
tasks,
projectGraph,
taskGraphForHashing,
nxJson,
lifeCycle
);
return tasks.map((task, index) =>
orchestrator.applyFromCacheOrRunTask(true, task, index)
);
}
export async function runContinuousTasks(
tasks: Task[],
projectGraph: ProjectGraph,
taskGraphForHashing: TaskGraph,
nxJson: NxJsonConfiguration,
lifeCycle: LifeCycle
) {
const orchestrator = await createOrchestrator(
tasks,
projectGraph,
taskGraphForHashing,
nxJson,
lifeCycle
);
return tasks.reduce((current, task, index) => {
current[task.id] = orchestrator.startContinuousTask(task, index);
return current;
}, {} as Record<string, Promise<RunningTask>>);
}
@@ -0,0 +1,66 @@
import type { NxJsonConfiguration } from '../config/nx-json';
import { readNxJsonFromDisk } from '../devkit-internals';
let tuiEnabled = undefined;
export function isTuiEnabled(nxJson?: NxJsonConfiguration) {
if (tuiEnabled !== undefined) {
return tuiEnabled;
}
// If the current terminal/environment is not capable of displaying the TUI, we don't run it
const isWindows = process.platform === 'win32';
const isCapable = process.stderr.isTTY && isUnicodeSupported();
// Windows is not working well right now, temporarily disable it on Windows even if it has been specified as enabled
// TODO(@JamesHenry): Remove this check once Windows issues are fixed.
if (!isCapable || isWindows) {
tuiEnabled = false;
process.env.NX_TUI = 'false';
return tuiEnabled;
}
// The environment variable takes precedence over the nx.json config
if (typeof process.env.NX_TUI === 'string') {
tuiEnabled = process.env.NX_TUI === 'true' ? true : false;
return tuiEnabled;
}
// Only read from disk if nx.json config is not already provided (and we have not been able to determine tuiEnabled based on the above checks)
if (!nxJson) {
nxJson = readNxJsonFromDisk();
}
// Respect user config
if (typeof nxJson.tui?.enabled === 'boolean') {
tuiEnabled = Boolean(nxJson.tui?.enabled);
} else {
// Default to enabling the TUI if the system is capable of displaying it
tuiEnabled = true;
}
// Also set the environment variable for consistency and ease of checking on the rust side, for example
process.env.NX_TUI = tuiEnabled.toString();
return tuiEnabled;
}
// Credit to https://github.com/sindresorhus/is-unicode-supported/blob/e0373335038856c63034c8eef6ac43ee3827a601/index.js
function isUnicodeSupported() {
const { env } = process;
const { TERM, TERM_PROGRAM } = env;
if (process.platform !== 'win32') {
return TERM !== 'linux'; // Linux console (kernel)
}
return (
Boolean(env.WT_SESSION) || // Windows Terminal
Boolean(env.TERMINUS_SUBLIME) || // Terminus (<0.2.27)
env.ConEmuTask === '{cmd::Cmder}' || // ConEmu and cmder
TERM_PROGRAM === 'Terminus-Sublime' ||
TERM_PROGRAM === 'vscode' ||
TERM === 'xterm-256color' ||
TERM === 'alacritty' ||
TERM === 'rxvt-unicode' ||
TERM === 'rxvt-unicode-256color' ||
env.TERMINAL_EMULATOR === 'JetBrains-JediTerm'
);
}
+30 -4
View File
@@ -1,5 +1,7 @@
import { TaskStatus } from './tasks-runner';
import { Task } from '../config/task-graph';
import { ExternalObject } from '../native';
import { RunningTask } from './running-tasks/running-task';
import { TaskStatus } from './tasks-runner';
/**
* The result of a completed {@link Task}
@@ -20,8 +22,16 @@ export interface TaskMetadata {
groupId: number;
}
interface RustRunningTask extends RunningTask {
getResults(): Promise<{ code: number; terminalOutput: string }>;
onExit(cb: (code: number, terminalOutput: string) => void): void;
kill(signal?: NodeJS.Signals | number): Promise<void> | void;
}
export interface LifeCycle {
startCommand?(): void | Promise<void>;
startCommand?(parallel?: number): void | Promise<void>;
endCommand?(): void | Promise<void>;
@@ -53,15 +63,20 @@ export interface LifeCycle {
status: TaskStatus,
output: string
): void;
registerRunningTask?(
taskId: string,
parserAndWriter: ExternalObject<[any, any]>
);
}
export class CompositeLifeCycle implements LifeCycle {
constructor(private readonly lifeCycles: LifeCycle[]) {}
async startCommand(): Promise<void> {
async startCommand(parallel?: number): Promise<void> {
for (let l of this.lifeCycles) {
if (l.startCommand) {
await l.startCommand();
await l.startCommand(parallel);
}
}
}
@@ -132,4 +147,15 @@ export class CompositeLifeCycle implements LifeCycle {
}
}
}
async registerRunningTask(
taskId: string,
parserAndWriter: ExternalObject<any>
): Promise<void> {
for (let l of this.lifeCycles) {
if (l.registerRunningTask) {
await l.registerRunningTask(taskId, parserAndWriter);
}
}
}
}
@@ -59,6 +59,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -78,6 +79,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -99,6 +101,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
{
@@ -109,6 +112,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -131,6 +135,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -150,6 +155,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -171,6 +177,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
{
@@ -181,6 +188,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -200,6 +208,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
{
@@ -210,6 +219,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -229,6 +239,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
{
@@ -239,6 +250,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -262,6 +274,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
{
@@ -272,6 +285,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -295,6 +309,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
{
@@ -305,6 +320,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
{
@@ -315,6 +331,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -338,6 +355,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
{
@@ -348,6 +366,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
{
@@ -358,6 +377,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
{
@@ -368,6 +388,7 @@ describe('formatTargetsAndProjects', () => {
},
overrides: {},
parallelism: false,
continuous: false,
outputs: [],
},
]
@@ -1,11 +1,12 @@
import { serializeTarget } from '../../utils/serialize-target';
import { Task } from '../../config/task-graph';
import { output } from '../../utils/output';
import {
getHistoryForHashes,
TaskRun,
writeTaskRunsToHistory as writeTaskRunsToHistory,
writeTaskRunsToHistory,
} from '../../utils/legacy-task-history';
import { output } from '../../utils/output';
import { serializeTarget } from '../../utils/serialize-target';
import { isTuiEnabled } from '../is-tui-enabled';
import { LifeCycle, TaskResult } from '../life-cycle';
export class LegacyTaskHistoryLifeCycle implements LifeCycle {
@@ -54,6 +55,10 @@ export class LegacyTaskHistoryLifeCycle implements LifeCycle {
);
}
}
// Do not directly print output when using the TUI
if (isTuiEnabled()) {
return;
}
if (flakyTasks.length > 0) {
output.warn({
title: `Nx detected ${
@@ -1,9 +1,10 @@
import { serializeTarget } from '../../utils/serialize-target';
import { Task } from '../../config/task-graph';
import { output } from '../../utils/output';
import { LifeCycle, TaskResult } from '../life-cycle';
import type { TaskRun as NativeTaskRun } from '../../native';
import { output } from '../../utils/output';
import { serializeTarget } from '../../utils/serialize-target';
import { getTaskHistory, TaskHistory } from '../../utils/task-history';
import { isTuiEnabled } from '../is-tui-enabled';
import { LifeCycle, TaskResult } from '../life-cycle';
interface TaskRun extends NativeTaskRun {
target: Task['target'];
@@ -45,6 +46,10 @@ export class TaskHistoryLifeCycle implements LifeCycle {
const flakyTasks = await this.taskHistory.getFlakyTasks(
entries.map(([hash]) => hash)
);
// Do not directly print output when using the TUI
if (isTuiEnabled()) {
return;
}
if (flakyTasks.length > 0) {
output.warn({
title: `Nx detected ${
@@ -0,0 +1,393 @@
import { EOL } from 'node:os';
import { Task } from '../../config/task-graph';
import { output } from '../../utils/output';
import type { LifeCycle } from '../life-cycle';
import type { TaskStatus } from '../tasks-runner';
import { formatFlags, formatTargetsAndProjects } from './formatting-utils';
import { prettyTime } from './pretty-time';
import { viewLogsFooterRows } from './view-logs-utils';
import figures = require('figures');
const LEFT_PAD = ` `;
const SPACER = ` `;
const EXTENDED_LEFT_PAD = ` `;
export function getTuiTerminalSummaryLifeCycle({
projectNames,
tasks,
args,
overrides,
initiatingProject,
resolveRenderIsDonePromise,
}: {
projectNames: string[];
tasks: Task[];
args: { targets?: string[]; configuration?: string; parallel?: number };
overrides: Record<string, unknown>;
initiatingProject: string;
resolveRenderIsDonePromise: (value: void) => void;
}) {
const lifeCycle = {} as Partial<LifeCycle>;
const start = process.hrtime();
const targets = args.targets;
const totalTasks = tasks.length;
let totalCachedTasks = 0;
let totalSuccessfulTasks = 0;
let totalFailedTasks = 0;
let totalCompletedTasks = 0;
let timeTakenText: string;
const failedTasks = new Set<string>();
const inProgressTasks = new Set<string>();
const tasksToTerminalOutputs: Record<
string,
{ terminalOutput: string; taskStatus: TaskStatus }
> = {};
const taskIdsInOrderOfCompletion: string[] = [];
lifeCycle.startTasks = (tasks) => {
for (let t of tasks) {
inProgressTasks.add(t.id);
}
};
lifeCycle.printTaskTerminalOutput = (task, taskStatus, terminalOutput) => {
tasksToTerminalOutputs[task.id] = { terminalOutput, taskStatus };
taskIdsInOrderOfCompletion.push(task.id);
};
lifeCycle.endTasks = (taskResults) => {
for (let t of taskResults) {
totalCompletedTasks++;
inProgressTasks.delete(t.task.id);
switch (t.status) {
case 'remote-cache':
case 'local-cache':
case 'local-cache-kept-existing':
totalCachedTasks++;
totalSuccessfulTasks++;
break;
case 'success':
totalSuccessfulTasks++;
break;
case 'failure':
totalFailedTasks++;
failedTasks.add(t.task.id);
break;
}
}
};
lifeCycle.endCommand = () => {
timeTakenText = prettyTime(process.hrtime(start));
resolveRenderIsDonePromise();
};
const printSummary = () => {
const isRunOne = initiatingProject && targets?.length === 1;
// Handles when the user interrupts the process
timeTakenText ??= prettyTime(process.hrtime(start));
if (totalTasks === 0) {
console.log(`\n${output.applyNxPrefix('gray', 'No tasks were run')}\n`);
return;
}
if (isRunOne) {
printRunOneSummary();
} else {
printRunManySummary();
}
};
const printRunOneSummary = () => {
let lines: string[] = [];
const failure = totalSuccessfulTasks !== totalTasks;
// Prints task outputs in the order they were completed
// above the summary, since run-one should print all task results.
for (const taskId of taskIdsInOrderOfCompletion) {
const { terminalOutput, taskStatus } = tasksToTerminalOutputs[taskId];
output.logCommandOutput(taskId, taskStatus, terminalOutput);
}
lines.push(...output.getVerticalSeparatorLines(failure ? 'red' : 'green'));
if (!failure) {
const text = `Successfully ran ${formatTargetsAndProjects(
[initiatingProject],
targets,
tasks
)}`;
const taskOverridesLines = [];
if (Object.keys(overrides).length > 0) {
taskOverridesLines.push('');
taskOverridesLines.push(
`${EXTENDED_LEFT_PAD}${output.dim.green('With additional flags:')}`
);
Object.entries(overrides)
.map(([flag, value]) =>
output.dim.green(formatFlags(EXTENDED_LEFT_PAD, flag, value))
)
.forEach((arg) => taskOverridesLines.push(arg));
}
lines.push(
output.applyNxPrefix(
'green',
output.colors.green(text) + output.dim(` (${timeTakenText})`)
),
...taskOverridesLines
);
if (totalCachedTasks > 0) {
lines.push(
output.dim(
`${EOL}Nx read the output from the cache instead of running the command for ${totalCachedTasks} out of ${totalTasks} tasks.`
)
);
}
lines = [output.colors.green(lines.join(EOL))];
} else if (totalCompletedTasks === totalTasks) {
let text = `Ran target ${output.bold(
targets[0]
)} for project ${output.bold(initiatingProject)}`;
if (tasks.length > 1) {
text += ` and ${output.bold(tasks.length - 1)} task(s) they depend on`;
}
const taskOverridesLines = [];
if (Object.keys(overrides).length > 0) {
taskOverridesLines.push('');
taskOverridesLines.push(
`${EXTENDED_LEFT_PAD}${output.dim.red('With additional flags:')}`
);
Object.entries(overrides)
.map(([flag, value]) =>
output.dim.red(formatFlags(EXTENDED_LEFT_PAD, flag, value))
)
.forEach((arg) => taskOverridesLines.push(arg));
}
const viewLogs = viewLogsFooterRows(totalFailedTasks);
lines = [
output.colors.red([
output.applyNxPrefix(
'red',
output.colors.red(text) + output.dim(` (${timeTakenText})`)
),
...taskOverridesLines,
'',
`${LEFT_PAD}${output.colors.red(
figures.cross
)}${SPACER}${totalFailedTasks}${`/${totalCompletedTasks}`} failed`,
`${LEFT_PAD}${output.dim(
figures.tick
)}${SPACER}${totalSuccessfulTasks}${`/${totalCompletedTasks}`} succeeded ${output.dim(
`[${totalCachedTasks} read from cache]`
)}`,
...viewLogs,
]),
];
} else {
lines = [
...output.getVerticalSeparatorLines('red'),
output.applyNxPrefix(
'red',
output.colors.red(
`Cancelled running target ${output.bold(
targets[0]
)} for project ${output.bold(initiatingProject)}`
) + output.dim(` (${timeTakenText})`)
),
];
}
// adds some vertical space after the summary to avoid bunching against terminal
lines.push('');
console.log(lines.join(EOL));
};
const printRunManySummary = () => {
console.log('');
const lines: string[] = [];
const failure = totalSuccessfulTasks !== totalTasks;
for (const taskId of taskIdsInOrderOfCompletion) {
const { terminalOutput, taskStatus } = tasksToTerminalOutputs[taskId];
if (taskStatus === 'failure') {
output.logCommandOutput(taskId, taskStatus, terminalOutput);
lines.push(
`${LEFT_PAD}${output.colors.red(
figures.cross
)}${SPACER}${output.colors.gray('nx run ')}${taskId}`
);
} else {
lines.push(
`${LEFT_PAD}${output.colors.green(
figures.tick
)}${SPACER}${output.colors.gray('nx run ')}${taskId}`
);
}
}
lines.push(...output.getVerticalSeparatorLines(failure ? 'red' : 'green'));
if (totalSuccessfulTasks === totalTasks) {
const successSummaryRows = [];
const text = `Successfully ran ${formatTargetsAndProjects(
projectNames,
targets,
tasks
)}`;
const taskOverridesRows = [];
if (Object.keys(overrides).length > 0) {
taskOverridesRows.push('');
taskOverridesRows.push(
`${EXTENDED_LEFT_PAD}${output.dim.green('With additional flags:')}`
);
Object.entries(overrides)
.map(([flag, value]) =>
output.dim.green(formatFlags(EXTENDED_LEFT_PAD, flag, value))
)
.forEach((arg) => taskOverridesRows.push(arg));
}
successSummaryRows.push(
...[
output.applyNxPrefix(
'green',
output.colors.green(text) + output.dim.white(` (${timeTakenText})`)
),
...taskOverridesRows,
]
);
if (totalCachedTasks > 0) {
successSummaryRows.push(
output.dim(
`${EOL}Nx read the output from the cache instead of running the command for ${totalCachedTasks} out of ${totalTasks} tasks.`
)
);
}
lines.push(successSummaryRows.join(EOL));
} else {
const text = `${
inProgressTasks.size ? 'Cancelled while running' : 'Ran'
} ${formatTargetsAndProjects(projectNames, targets, tasks)}`;
const taskOverridesRows = [];
if (Object.keys(overrides).length > 0) {
taskOverridesRows.push('');
taskOverridesRows.push(
`${EXTENDED_LEFT_PAD}${output.dim.red('With additional flags:')}`
);
Object.entries(overrides)
.map(([flag, value]) =>
output.dim.red(formatFlags(EXTENDED_LEFT_PAD, flag, value))
)
.forEach((arg) => taskOverridesRows.push(arg));
}
const numFailedToPrint = 5;
const failedTasksForPrinting = Array.from(failedTasks).slice(
0,
numFailedToPrint
);
const failureSummaryRows = [
output.applyNxPrefix(
'red',
output.colors.red(text) + output.dim.white(` (${timeTakenText})`)
),
...taskOverridesRows,
'',
];
if (totalCompletedTasks > 0) {
if (totalSuccessfulTasks > 0) {
failureSummaryRows.push(
output.dim(
`${LEFT_PAD}${output.dim(
figures.tick
)}${SPACER}${totalSuccessfulTasks}${`/${totalCompletedTasks}`} succeeded ${output.dim(
`[${totalCachedTasks} read from cache]`
)}`
),
''
);
}
if (totalFailedTasks > 0) {
failureSummaryRows.push(
`${LEFT_PAD}${output.colors.red(
figures.cross
)}${SPACER}${totalFailedTasks}${`/${totalCompletedTasks}`} targets failed, including the following:`,
'',
`${failedTasksForPrinting
.map(
(t) =>
`${EXTENDED_LEFT_PAD}${output.colors.red(
'-'
)} ${output.formatCommand(t.toString())}`
)
.join('\n')}`,
''
);
if (failedTasks.size > numFailedToPrint) {
failureSummaryRows.push(
output.dim(
`${EXTENDED_LEFT_PAD}...and ${
failedTasks.size - numFailedToPrint
} more...`
)
);
}
}
if (totalCompletedTasks !== totalTasks) {
const remainingTasks = totalTasks - totalCompletedTasks;
if (inProgressTasks.size) {
failureSummaryRows.push(
`${LEFT_PAD}${output.colors.red(figures.ellipsis)}${SPACER}${
inProgressTasks.size
}${`/${totalTasks}`} targets were in progress, including the following:`,
'',
`${Array.from(inProgressTasks)
.map(
(t) =>
`${EXTENDED_LEFT_PAD}${output.colors.red(
'-'
)} ${output.formatCommand(t.toString())}`
)
.join(EOL)}`,
''
);
}
if (remainingTasks - inProgressTasks.size > 0) {
failureSummaryRows.push(
output.dim(
`${LEFT_PAD}${output.colors.red(figures.ellipsis)}${SPACER}${
remainingTasks - inProgressTasks.size
}${`/${totalTasks}`} targets had not started.`
),
''
);
}
}
failureSummaryRows.push(...viewLogsFooterRows(failedTasks.size));
lines.push(output.colors.red(failureSummaryRows.join(EOL)));
}
}
// adds some vertical space after the summary to avoid bunching against terminal
lines.push('');
console.log(lines.join(EOL));
};
return { lifeCycle, printSummary };
}
@@ -1,9 +1,9 @@
import { getPseudoTerminal, PseudoTerminal } from './pseudo-terminal';
import { createPseudoTerminal, PseudoTerminal } from './pseudo-terminal';
describe('PseudoTerminal', () => {
let terminal: PseudoTerminal;
beforeAll(() => {
terminal = getPseudoTerminal(true);
beforeEach(() => {
terminal = createPseudoTerminal(true);
});
afterAll(() => {
@@ -17,6 +17,7 @@ describe('PseudoTerminal', () => {
done();
});
});
it('should kill a running command', (done) => {
const childProcess = terminal.runCommand(
'sleep 3 && echo "hello world" > file.txt'
@@ -31,15 +32,17 @@ describe('PseudoTerminal', () => {
it('should subscribe to output', (done) => {
const childProcess = terminal.runCommand('echo "hello world"');
let output = '';
childProcess.onOutput((chunk) => {
output += chunk;
});
childProcess.onExit(() => {
expect(output.trim()).toContain('hello world');
done();
try {
expect(output.trim()).toContain('hello world');
} finally {
done();
}
});
});
@@ -54,20 +57,4 @@ describe('PseudoTerminal', () => {
});
});
}
it('should run multiple commands', async () => {
function runCommand() {
return new Promise((res) => {
const cp1 = terminal.runCommand('whoami', {});
cp1.onExit(res);
});
}
let i = 0;
while (i < 10) {
await runCommand();
i++;
}
});
});
+70 -45
View File
@@ -4,19 +4,37 @@ import { getForkedProcessOsSocketPath } from '../daemon/socket-utils';
import { Serializable } from 'child_process';
import * as os from 'os';
let pseudoTerminal: PseudoTerminal;
// Register single event listeners for all pseudo-terminal instances
const pseudoTerminalShutdownCallbacks: Array<() => void> = [];
process.on('SIGINT', () => {
pseudoTerminalShutdownCallbacks.forEach((cb) => cb());
});
process.on('SIGTERM', () => {
pseudoTerminalShutdownCallbacks.forEach((cb) => cb());
});
process.on('SIGHUP', () => {
pseudoTerminalShutdownCallbacks.forEach((cb) => cb());
});
process.on('exit', () => {
pseudoTerminalShutdownCallbacks.forEach((cb) => cb());
});
export function getPseudoTerminal(skipSupportCheck: boolean = false) {
export function createPseudoTerminal(skipSupportCheck: boolean = false) {
if (!skipSupportCheck && !PseudoTerminal.isSupported()) {
throw new Error('Pseudo terminal is not supported on this platform.');
}
pseudoTerminal ??= new PseudoTerminal(new RustPseudoTerminal());
const pseudoTerminal = new PseudoTerminal(new RustPseudoTerminal());
pseudoTerminalShutdownCallbacks.push(
pseudoTerminal.shutdown.bind(pseudoTerminal)
);
return pseudoTerminal;
}
let id = 0;
export class PseudoTerminal {
private pseudoIPCPath = getForkedProcessOsSocketPath(process.pid.toString());
private pseudoIPCPath = getForkedProcessOsSocketPath(
process.pid.toString() + '-' + id++
);
private pseudoIPC = new PseudoIPCServer(this.pseudoIPCPath);
private initialized: boolean = false;
@@ -25,9 +43,7 @@ export class PseudoTerminal {
return process.stdout.isTTY && supportedPtyPlatform();
}
constructor(private rustPseudoTerminal: RustPseudoTerminal) {
this.setupProcessListeners();
}
constructor(private rustPseudoTerminal: RustPseudoTerminal) {}
async init() {
if (this.initialized) {
@@ -37,6 +53,12 @@ export class PseudoTerminal {
this.initialized = true;
}
shutdown() {
if (this.initialized) {
this.pseudoIPC.close();
}
}
runCommand(
command: string,
{
@@ -54,6 +76,7 @@ export class PseudoTerminal {
} = {}
) {
return new PseudoTtyProcess(
this.rustPseudoTerminal,
this.rustPseudoTerminal.runCommand(
command,
cwd,
@@ -84,6 +107,7 @@ export class PseudoTerminal {
throw new Error('Call init() before forking processes');
}
const cp = new PseudoTtyProcessWithSend(
this.rustPseudoTerminal,
this.rustPseudoTerminal.fork(
id,
script,
@@ -109,44 +133,40 @@ export class PseudoTerminal {
onMessageFromChildren(callback: (message: Serializable) => void) {
this.pseudoIPC.onMessageFromChildren(callback);
}
private setupProcessListeners() {
const shutdown = () => {
this.shutdownPseudoIPC();
};
process.on('SIGINT', () => {
this.shutdownPseudoIPC();
});
process.on('SIGTERM', () => {
this.shutdownPseudoIPC();
});
process.on('SIGHUP', () => {
this.shutdownPseudoIPC();
});
process.on('exit', () => {
this.shutdownPseudoIPC();
});
}
private shutdownPseudoIPC() {
if (this.initialized) {
this.pseudoIPC.close();
}
}
}
export class PseudoTtyProcess {
isAlive = true;
exitCallbacks = [];
private exitCallbacks: Array<(code: number) => void> = [];
private outputCallbacks: Array<(output: string) => void> = [];
private terminalOutput = '';
constructor(
public rustPseudoTerminal: RustPseudoTerminal,
private childProcess: ChildProcess
) {
childProcess.onOutput((output) => {
this.terminalOutput += output;
this.outputCallbacks.forEach((cb) => cb(output));
});
constructor(private childProcess: ChildProcess) {
childProcess.onExit((message) => {
this.isAlive = false;
const exitCode = messageToCode(message);
const code = messageToCode(message);
childProcess.cleanup();
this.exitCallbacks.forEach((cb) => cb(exitCode));
this.exitCallbacks.forEach((cb) => cb(code));
});
}
async getResults(): Promise<{ code: number; terminalOutput: string }> {
return new Promise((res) => {
this.onExit((code) => {
res({ code, terminalOutput: this.terminalOutput });
});
});
}
@@ -155,30 +175,35 @@ export class PseudoTtyProcess {
}
onOutput(callback: (message: string) => void): void {
this.childProcess.onOutput(callback);
this.outputCallbacks.push(callback);
}
kill(): void {
try {
this.childProcess.kill();
} catch {
// when the child process completes before we explicitly call kill, this will throw
// do nothing
} finally {
if (this.isAlive == true) {
if (this.isAlive) {
try {
this.childProcess.kill();
} catch {
// when the child process completes before we explicitly call kill, this will throw
// do nothing
} finally {
this.isAlive = false;
}
}
}
getParserAndWriter() {
return this.childProcess.getParserAndWriter();
}
}
export class PseudoTtyProcessWithSend extends PseudoTtyProcess {
constructor(
public rustPseudoTerminal: RustPseudoTerminal,
_childProcess: ChildProcess,
private id: string,
private pseudoIpc: PseudoIPCServer
) {
super(_childProcess);
super(rustPseudoTerminal, _childProcess);
}
send(message: Serializable) {
+221 -14
View File
@@ -1,6 +1,8 @@
import { prompt } from 'enquirer';
import { join } from 'node:path';
import { stripVTControlCharacters } from 'node:util';
import * as ora from 'ora';
import { join } from 'path';
import type { Observable } from 'rxjs';
import {
NxJsonConfiguration,
readNxJson,
@@ -16,13 +18,18 @@ import {
hashTasksThatDoNotDependOnOutputsOfOtherTasks,
} from '../hasher/hash-task';
import { IS_WASM } from '../native';
import {
runPostTasksExecution,
runPreTasksExecution,
} from '../project-graph/plugins/tasks-execution-hooks';
import { createProjectGraphAsync } from '../project-graph/project-graph';
import { NxArgs } from '../utils/command-line-utils';
import { isRelativePath } from '../utils/fileutils';
import { handleErrors } from '../utils/handle-errors';
import { isCI } from '../utils/is-ci';
import { isNxCloudUsed } from '../utils/nx-cloud-utils';
import { printNxKey } from '../utils/nx-key';
import { output } from '../utils/output';
import { handleErrors } from '../utils/handle-errors';
import {
collectEnabledTaskSyncGeneratorsFromTaskGraph,
flushSyncGeneratorChanges,
@@ -33,7 +40,8 @@ import {
processSyncGeneratorResultErrors,
} from '../utils/sync-generators';
import { workspaceRoot } from '../utils/workspace-root';
import { createTaskGraph } from './create-task-graph';
import { createTaskGraph, createTaskId } from './create-task-graph';
import { isTuiEnabled } from './is-tui-enabled';
import {
CompositeLifeCycle,
LifeCycle,
@@ -48,8 +56,9 @@ import { StoreRunInformationLifeCycle } from './life-cycles/store-run-informatio
import { TaskHistoryLifeCycle } from './life-cycles/task-history-life-cycle';
import { LegacyTaskHistoryLifeCycle } from './life-cycles/task-history-life-cycle-old';
import { TaskProfilingLifeCycle } from './life-cycles/task-profiling-life-cycle';
import { TaskTimingsLifeCycle } from './life-cycles/task-timings-life-cycle';
import { TaskResultsLifeCycle } from './life-cycles/task-results-life-cycle';
import { TaskTimingsLifeCycle } from './life-cycles/task-timings-life-cycle';
import { getTuiTerminalSummaryLifeCycle } from './life-cycles/tui-summary-life-cycle';
import {
findCycle,
makeAcyclic,
@@ -58,21 +67,221 @@ import {
import { TasksRunner, TaskStatus } from './tasks-runner';
import { shouldStreamOutput } from './utils';
import chalk = require('chalk');
import type { Observable } from 'rxjs';
import { printNxKey } from '../utils/nx-key';
import {
runPostTasksExecution,
runPreTasksExecution,
} from '../project-graph/plugins/tasks-execution-hooks';
const originalStdoutWrite = process.stdout.write.bind(process.stdout);
const originalStderrWrite = process.stderr.write.bind(process.stderr);
const originalConsoleLog = console.log.bind(console);
const originalConsoleError = console.error.bind(console);
async function getTerminalOutputLifeCycle(
initiatingProject: string,
projectNames: string[],
tasks: Task[],
taskGraph: TaskGraph,
nxArgs: NxArgs,
nxJson: NxJsonConfiguration,
overrides: Record<string, unknown>
): Promise<{ lifeCycle: LifeCycle; renderIsDone: Promise<void> }> {
const overridesWithoutHidden = { ...overrides };
delete overridesWithoutHidden['__overrides_unparsed__'];
if (isTuiEnabled(nxJson)) {
const interceptedNxCloudLogs: (string | Uint8Array<ArrayBufferLike>)[] = [];
const createPatchedConsoleMethod = (
originalMethod: typeof console.log | typeof console.error
): typeof console.log | typeof console.error => {
return (...args: any[]) => {
// Check if the log came from the Nx Cloud client, otherwise invoke the original write method
const stackTrace = new Error().stack;
const isNxCloudLog = stackTrace.includes(
join(workspaceRoot, '.nx', 'cache', 'cloud')
);
if (!isNxCloudLog) {
return originalMethod(...args);
}
// No-op the Nx Cloud client logs
};
};
// The cloud client calls console.log when NX_VERBOSE_LOGGING is set to true
console.log = createPatchedConsoleMethod(originalConsoleLog);
console.error = createPatchedConsoleMethod(originalConsoleError);
const patchedWrite = (_chunk, _encoding, callback) => {
// Preserve original behavior around callback and return value, just in case
if (callback) {
callback();
}
return true;
};
process.stdout.write = patchedWrite as any;
process.stderr.write = patchedWrite as any;
const { AppLifeCycle, restoreTerminal } = await import('../native');
let appLifeCycle;
const isRunOne = initiatingProject != null;
const pinnedTasks: string[] = [];
const taskText = tasks.length === 1 ? 'task' : 'tasks';
const projectText = projectNames.length === 1 ? 'project' : 'projects';
let titleText = '';
if (isRunOne) {
const mainTaskId = createTaskId(
initiatingProject,
nxArgs.targets[0],
nxArgs.configuration
);
pinnedTasks.push(mainTaskId);
const mainContinuousDependencies =
taskGraph.continuousDependencies[mainTaskId];
if (mainContinuousDependencies.length > 0) {
pinnedTasks.push(mainContinuousDependencies[0]);
}
const [project, target] = mainTaskId.split(':');
titleText = `${target} ${project}`;
if (tasks.length > 1) {
titleText += `, and ${tasks.length - 1} requisite ${taskText}`;
}
} else {
titleText =
nxArgs.targets.join(', ') +
` for ${projectNames.length} ${projectText}`;
if (tasks.length > projectNames.length) {
titleText += `, and ${
tasks.length - projectNames.length
} requisite ${taskText}`;
}
}
let resolveRenderIsDonePromise: (value: void) => void;
// Default renderIsDone that will be overridden if the TUI is used
let renderIsDone = new Promise<void>(
(resolve) => (resolveRenderIsDonePromise = resolve)
);
const { lifeCycle: tsLifeCycle, printSummary } =
getTuiTerminalSummaryLifeCycle({
projectNames,
tasks,
args: nxArgs,
overrides: overridesWithoutHidden,
initiatingProject,
resolveRenderIsDonePromise,
});
if (tasks.length === 0) {
renderIsDone = renderIsDone.then(() => {
// Revert the patched methods
process.stdout.write = originalStdoutWrite;
process.stderr.write = originalStderrWrite;
console.log = originalConsoleLog;
console.error = originalConsoleError;
printSummary();
});
}
const lifeCycles: LifeCycle[] = [tsLifeCycle];
// Only run the TUI if there are tasks to run
if (tasks.length > 0) {
appLifeCycle = new AppLifeCycle(
tasks,
pinnedTasks,
nxArgs ?? {},
nxJson.tui ?? {},
titleText
);
lifeCycles.unshift(appLifeCycle);
/**
* Patch stdout.write and stderr.write methods to pass Nx Cloud client logs to the TUI via the lifecycle
*/
const createPatchedLogWrite = (
originalWrite: typeof process.stdout.write | typeof process.stderr.write
): typeof process.stdout.write | typeof process.stderr.write => {
// @ts-ignore
return (chunk, encoding, callback) => {
// Check if the log came from the Nx Cloud client, otherwise invoke the original write method
const stackTrace = new Error().stack;
const isNxCloudLog = stackTrace.includes(
join(workspaceRoot, '.nx', 'cache', 'cloud')
);
if (isNxCloudLog) {
interceptedNxCloudLogs.push(chunk);
// Do not bother to store logs with only whitespace characters, they aren't relevant for the TUI
const trimmedChunk = chunk.toString().trim();
if (trimmedChunk.length) {
// Remove ANSI escape codes, the TUI will control the formatting
appLifeCycle?.__setCloudMessage(
stripVTControlCharacters(trimmedChunk)
);
}
}
// Preserve original behavior around callback and return value, just in case
if (callback) {
callback();
}
return true;
};
};
const createPatchedConsoleMethod = (
originalMethod: typeof console.log | typeof console.error
): typeof console.log | typeof console.error => {
return (...args: any[]) => {
// Check if the log came from the Nx Cloud client, otherwise invoke the original write method
const stackTrace = new Error().stack;
const isNxCloudLog = stackTrace.includes(
join(workspaceRoot, '.nx', 'cache', 'cloud')
);
if (!isNxCloudLog) {
return originalMethod(...args);
}
// No-op the Nx Cloud client logs
};
};
process.stdout.write = createPatchedLogWrite(originalStdoutWrite);
process.stderr.write = createPatchedLogWrite(originalStderrWrite);
// The cloud client calls console.log when NX_VERBOSE_LOGGING is set to true
console.log = createPatchedConsoleMethod(originalConsoleLog);
console.error = createPatchedConsoleMethod(originalConsoleError);
renderIsDone = new Promise<void>((resolve) => {
appLifeCycle.__init(() => {
resolve();
});
})
.then(() => {
restoreTerminal();
})
.finally(() => {
// Revert the patched methods
process.stdout.write = originalStdoutWrite;
process.stderr.write = originalStderrWrite;
console.log = originalConsoleLog;
console.error = originalConsoleError;
printSummary();
// Print the intercepted Nx Cloud logs
for (const log of interceptedNxCloudLogs) {
const logString = log.toString().trimStart();
process.stdout.write(logString);
if (logString) {
process.stdout.write('\n');
}
}
});
}
return {
lifeCycle: new CompositeLifeCycle(lifeCycles),
renderIsDone,
};
}
const { runnerOptions } = getRunner(nxArgs, nxJson);
const isRunOne = initiatingProject != null;
const useDynamicOutput = shouldUseDynamicLifeCycle(
@@ -81,9 +290,6 @@ async function getTerminalOutputLifeCycle(
nxArgs.outputStyle
);
const overridesWithoutHidden = { ...overrides };
delete overridesWithoutHidden['__overrides_unparsed__'];
if (isRunOne) {
if (useDynamicOutput) {
return await createRunOneDynamicOutputRenderer({
@@ -255,6 +461,7 @@ export async function runCommandForTasks(
initiatingProject,
projectNames,
tasks,
taskGraph,
nxArgs,
nxJson,
overrides
@@ -726,7 +933,7 @@ export async function invokeTasksRunner({
return taskResultsLifecycle.getTaskResults();
}
function constructLifeCycles(lifeCycle: LifeCycle): LifeCycle[] {
export function constructLifeCycles(lifeCycle: LifeCycle): LifeCycle[] {
const lifeCycles = [] as LifeCycle[];
lifeCycles.push(new StoreRunInformationLifeCycle());
lifeCycles.push(lifeCycle);
@@ -0,0 +1,84 @@
import {
BatchMessage,
BatchMessageType,
BatchResults,
} from '../batch/batch-messages';
import { ChildProcess, Serializable } from 'child_process';
import { signalToCode } from '../../utils/exit-codes';
export class BatchProcess {
private exitCallbacks: Array<(code: number) => void> = [];
private resultsCallbacks: Array<(results: BatchResults) => void> = [];
constructor(
private childProcess: ChildProcess,
private executorName: string
) {
this.childProcess.on('message', (message: BatchMessage) => {
switch (message.type) {
case BatchMessageType.CompleteBatchExecution: {
for (const cb of this.resultsCallbacks) {
cb(message.results);
}
break;
}
case BatchMessageType.RunTasks: {
break;
}
default: {
// Re-emit any non-batch messages from the task process
if (process.send) {
process.send(message);
}
}
}
});
this.childProcess.once('exit', (code, signal) => {
if (code === null) code = signalToCode(signal);
for (const cb of this.exitCallbacks) {
cb(code);
}
});
}
onExit(cb: (code: number) => void) {
this.exitCallbacks.push(cb);
}
onResults(cb: (results: BatchResults) => void) {
this.resultsCallbacks.push(cb);
}
async getResults(): Promise<BatchResults> {
return Promise.race<BatchResults>([
new Promise((_, rej) => {
this.onExit((code) => {
if (code !== 0) {
rej(
new Error(
`"${this.executorName}" exited unexpectedly with code: ${code}`
)
);
}
});
}),
new Promise((res) => {
this.onResults(res);
}),
]);
}
send(message: Serializable): void {
if (this.childProcess.connected) {
this.childProcess.send(message);
}
}
kill(signal?: NodeJS.Signals | number): void {
if (this.childProcess.connected) {
this.childProcess.kill(signal);
}
}
}
@@ -0,0 +1,215 @@
import { ChildProcess, Serializable } from 'child_process';
import { signalToCode } from '../../utils/exit-codes';
import { RunningTask } from './running-task';
import { Transform } from 'stream';
import * as chalk from 'chalk';
import { readFileSync } from 'fs';
export class NodeChildProcessWithNonDirectOutput implements RunningTask {
private terminalOutput: string = '';
private exitCallbacks: Array<(code: number, terminalOutput: string) => void> =
[];
constructor(
private childProcess: ChildProcess,
{ streamOutput, prefix }: { streamOutput: boolean; prefix: string }
) {
if (streamOutput) {
if (process.env.NX_PREFIX_OUTPUT === 'true') {
const color = getColor(prefix);
const prefixText = `${prefix}:`;
this.childProcess.stdout
.pipe(logClearLineToPrefixTransformer(color.bold(prefixText) + ' '))
.pipe(addPrefixTransformer(color.bold(prefixText)))
.pipe(process.stdout);
this.childProcess.stderr
.pipe(logClearLineToPrefixTransformer(color(prefixText) + ' '))
.pipe(addPrefixTransformer(color(prefixText)))
.pipe(process.stderr);
} else {
this.childProcess.stdout
.pipe(addPrefixTransformer())
.pipe(process.stdout);
this.childProcess.stderr
.pipe(addPrefixTransformer())
.pipe(process.stderr);
}
}
this.childProcess.on('exit', (code, signal) => {
if (code === null) code = signalToCode(signal);
for (const cb of this.exitCallbacks) {
cb(code, this.terminalOutput);
}
});
// Re-emit any messages from the task process
this.childProcess.on('message', (message) => {
if (process.send) {
process.send(message);
}
});
this.childProcess.stdout.on('data', (chunk) => {
this.terminalOutput += chunk.toString();
});
this.childProcess.stderr.on('data', (chunk) => {
this.terminalOutput += chunk.toString();
});
}
onExit(cb: (code: number, terminalOutput: string) => void) {
this.exitCallbacks.push(cb);
}
async getResults(): Promise<{ code: number; terminalOutput: string }> {
return new Promise((res) => {
this.onExit((code, terminalOutput) => {
res({ code, terminalOutput });
});
});
}
send(message: Serializable): void {
if (this.childProcess.connected) {
this.childProcess.send(message);
}
}
public kill(signal?: NodeJS.Signals | number) {
if (this.childProcess.connected) {
this.childProcess.kill(signal);
}
}
}
function addPrefixTransformer(prefix?: string) {
const newLineSeparator = process.platform.startsWith('win') ? '\r\n' : '\n';
return new Transform({
transform(chunk, _encoding, callback) {
const list = chunk.toString().split(/\r\n|[\n\v\f\r\x85\u2028\u2029]/g);
list
.filter(Boolean)
.forEach((m) =>
this.push(
prefix ? prefix + ' ' + m + newLineSeparator : m + newLineSeparator
)
);
callback();
},
});
}
const colors = [
chalk.green,
chalk.greenBright,
chalk.red,
chalk.redBright,
chalk.cyan,
chalk.cyanBright,
chalk.yellow,
chalk.yellowBright,
chalk.magenta,
chalk.magentaBright,
];
function getColor(projectName: string) {
let code = 0;
for (let i = 0; i < projectName.length; ++i) {
code += projectName.charCodeAt(i);
}
const colorIndex = code % colors.length;
return colors[colorIndex];
}
/**
* Prevents terminal escape sequence from clearing line prefix.
*/
function logClearLineToPrefixTransformer(prefix: string) {
let prevChunk = null;
return new Transform({
transform(chunk, _encoding, callback) {
if (prevChunk && prevChunk.toString() === '\x1b[2K') {
chunk = chunk.toString().replace(/\x1b\[1G/g, (m) => m + prefix);
}
this.push(chunk);
prevChunk = chunk;
callback();
},
});
}
export class NodeChildProcessWithDirectOutput implements RunningTask {
private terminalOutput: string | undefined;
private exitCallbacks: Array<(code: number, signal: string) => void> = [];
private exited = false;
private exitCode: number;
constructor(
private childProcess: ChildProcess,
private temporaryOutputPath: string
) {
// Re-emit any messages from the task process
this.childProcess.on('message', (message) => {
if (process.send) {
process.send(message);
}
});
this.childProcess.on('exit', (code, signal) => {
if (code === null) code = signalToCode(signal);
this.exited = true;
this.exitCode = code;
for (const cb of this.exitCallbacks) {
cb(code, signal);
}
});
}
send(message: Serializable): void {
if (this.childProcess.connected) {
this.childProcess.send(message);
}
}
onExit(cb: (code: number, signal: NodeJS.Signals) => void) {
this.exitCallbacks.push(cb);
}
async getResults(): Promise<{ code: number; terminalOutput: string }> {
const terminalOutput = this.getTerminalOutput();
if (this.exited) {
return Promise.resolve({
code: this.exitCode,
terminalOutput,
});
}
await this.waitForExit();
return Promise.resolve({
code: this.exitCode,
terminalOutput,
});
}
waitForExit() {
return new Promise<void>((res) => {
this.onExit(() => res());
});
}
getTerminalOutput() {
this.terminalOutput ??= readFileSync(this.temporaryOutputPath).toString();
return this.terminalOutput;
}
kill(signal?: NodeJS.Signals | number): void {
if (this.childProcess.connected) {
this.childProcess.kill(signal);
}
}
}
@@ -0,0 +1,20 @@
import { Serializable } from 'child_process';
import { RunningTask } from './running-task';
export class NoopChildProcess implements RunningTask {
constructor(private results: { code: number; terminalOutput: string }) {}
send(): void {}
async getResults(): Promise<{ code: number; terminalOutput: string }> {
return this.results;
}
kill(): void {
return;
}
onExit(cb: (code: number) => void): void {
cb(this.results.code);
}
}
@@ -0,0 +1,7 @@
export abstract class RunningTask {
abstract getResults(): Promise<{ code: number; terminalOutput: string }>;
abstract onExit(cb: (code: number) => void): void;
abstract kill(signal?: NodeJS.Signals | number): Promise<void> | void;
}
+4 -2
View File
@@ -37,7 +37,7 @@ export function getEnvVariablesForTask(
captureStderr: boolean,
outputPath: string,
streamOutput: boolean
) {
): NodeJS.ProcessEnv {
const res = {
// Start With Dotenv Variables
...taskSpecificEnv,
@@ -95,7 +95,7 @@ function getNxEnvVariablesForTask(
captureStderr: boolean,
outputPath: string,
streamOutput: boolean
) {
): NodeJS.ProcessEnv {
const env: NodeJS.ProcessEnv = {
NX_TASK_TARGET_PROJECT: task.target.project,
NX_TASK_TARGET_TARGET: task.target.target,
@@ -119,6 +119,8 @@ function getNxEnvVariablesForTask(
streamOutput
),
...env,
// Ensure the TUI does not get spawned within the TUI if ever tasks invoke Nx again
NX_TUI: 'false',
};
}
+339 -140
View File
@@ -1,13 +1,35 @@
import { defaultMaxListeners } from 'events';
import { performance } from 'perf_hooks';
import { relative } from 'path';
import { writeFileSync } from 'fs';
import { relative } from 'path';
import { performance } from 'perf_hooks';
import { NxJsonConfiguration } from '../config/nx-json';
import { ProjectGraph } from '../config/project-graph';
import { Task, TaskGraph } from '../config/task-graph';
import { DaemonClient } from '../daemon/client/client';
import { runCommands } from '../executors/run-commands/run-commands.impl';
import { getTaskDetails, hashTask } from '../hasher/hash-task';
import { TaskHasher } from '../hasher/task-hasher';
import runCommandsImpl from '../executors/run-commands/run-commands.impl';
import { ForkedProcessTaskRunner } from './forked-process-task-runner';
import { RunningTasksService, TaskDetails } from '../native';
import { NxArgs } from '../utils/command-line-utils';
import { getDbConnection } from '../utils/db-connection';
import { output } from '../utils/output';
import { combineOptionsForExecutor } from '../utils/params';
import { workspaceRoot } from '../utils/workspace-root';
import { Cache, DbCache, dbCacheEnabled, getCache } from './cache';
import { DefaultTasksRunnerOptions } from './default-tasks-runner';
import { ForkedProcessTaskRunner } from './forked-process-task-runner';
import { isTuiEnabled } from './is-tui-enabled';
import { TaskMetadata } from './life-cycle';
import { PseudoTtyProcess } from './pseudo-terminal';
import { NoopChildProcess } from './running-tasks/noop-child-process';
import { RunningTask } from './running-tasks/running-task';
import {
getEnvVariablesForBatchProcess,
getEnvVariablesForTask,
getTaskSpecificEnv,
} from './task-env';
import { TaskStatus } from './tasks-runner';
import { Batch, TasksSchedule } from './tasks-schedule';
import {
calculateReverseDeps,
getExecutorForTask,
@@ -17,28 +39,17 @@ import {
removeTasksFromTaskGraph,
shouldStreamOutput,
} from './utils';
import { Batch, TasksSchedule } from './tasks-schedule';
import { TaskMetadata } from './life-cycle';
import { ProjectGraph } from '../config/project-graph';
import { Task, TaskGraph } from '../config/task-graph';
import { DaemonClient } from '../daemon/client/client';
import { getTaskDetails, hashTask } from '../hasher/hash-task';
import {
getEnvVariablesForBatchProcess,
getEnvVariablesForTask,
getTaskSpecificEnv,
} from './task-env';
import { workspaceRoot } from '../utils/workspace-root';
import { output } from '../utils/output';
import { combineOptionsForExecutor } from '../utils/params';
import { NxJsonConfiguration } from '../config/nx-json';
import type { TaskDetails } from '../native';
export class TaskOrchestrator {
private taskDetails: TaskDetails | null = getTaskDetails();
private cache: DbCache | Cache = getCache(this.options);
private forkedProcessTaskRunner = new ForkedProcessTaskRunner(this.options);
private readonly tuiEnabled = isTuiEnabled(this.nxJson);
private forkedProcessTaskRunner = new ForkedProcessTaskRunner(
this.options,
this.tuiEnabled
);
private runningTasksService = new RunningTasksService(getDbConnection());
private tasksSchedule = new TasksSchedule(
this.projectGraph,
this.taskGraph,
@@ -64,6 +75,10 @@ export class TaskOrchestrator {
private bailed = false;
private runningContinuousTasks = new Map<string, RunningTask>();
private cleaningUp = false;
// endregion internal state
constructor(
@@ -72,32 +87,39 @@ export class TaskOrchestrator {
private readonly projectGraph: ProjectGraph,
private readonly taskGraph: TaskGraph,
private readonly nxJson: NxJsonConfiguration,
private readonly options: DefaultTasksRunnerOptions,
private readonly options: NxArgs & DefaultTasksRunnerOptions,
private readonly bail: boolean,
private readonly daemon: DaemonClient,
private readonly outputStyle: string
private readonly outputStyle: string,
private readonly taskGraphForHashing: TaskGraph = taskGraph
) {}
async run() {
async init() {
// Init the ForkedProcessTaskRunner, TasksSchedule, and Cache
await Promise.all([
this.forkedProcessTaskRunner.init(),
this.tasksSchedule.init(),
'init' in this.cache ? this.cache.init() : null,
]);
}
async run() {
await this.init();
// initial scheduling
await this.tasksSchedule.scheduleNextTasks();
performance.mark('task-execution:start');
const threadCount = getThreadCount(this.options, this.taskGraph);
const threads = [];
process.stdout.setMaxListeners(this.options.parallel + defaultMaxListeners);
process.stderr.setMaxListeners(this.options.parallel + defaultMaxListeners);
process.stdout.setMaxListeners(threadCount + defaultMaxListeners);
process.stderr.setMaxListeners(threadCount + defaultMaxListeners);
// initial seeding of the queue
for (let i = 0; i < this.options.parallel; ++i) {
for (let i = 0; i < threadCount; ++i) {
threads.push(this.executeNextBatchOfTasksUsingTaskSchedule());
}
await Promise.all(threads);
@@ -110,6 +132,8 @@ export class TaskOrchestrator {
);
this.cache.removeOldCacheRecords();
await this.cleanup();
return this.completedTasks;
}
@@ -139,7 +163,11 @@ export class TaskOrchestrator {
if (task) {
const groupId = this.closeGroup();
await this.applyFromCacheOrRunTask(doNotSkipCache, task, groupId);
if (task.continuous) {
await this.startContinuousTask(task, groupId);
} else {
await this.applyFromCacheOrRunTask(doNotSkipCache, task, groupId);
}
this.openGroup(groupId);
@@ -153,9 +181,7 @@ export class TaskOrchestrator {
}
// region Processing Scheduled Tasks
private async processScheduledTask(
taskId: string
): Promise<NodeJS.ProcessEnv> {
async processTask(taskId: string): Promise<NodeJS.ProcessEnv> {
const task = this.taskGraph.tasks[taskId];
const taskSpecificEnv = getTaskSpecificEnv(task);
@@ -163,7 +189,7 @@ export class TaskOrchestrator {
await hashTask(
this.hasher,
this.projectGraph,
this.taskGraph,
this.taskGraphForHashing,
task,
taskSpecificEnv,
this.taskDetails
@@ -182,7 +208,7 @@ export class TaskOrchestrator {
await hashTask(
this.hasher,
this.projectGraph,
this.taskGraph,
this.taskGraphForHashing,
task,
this.batchEnv,
this.taskDetails
@@ -203,7 +229,7 @@ export class TaskOrchestrator {
for (const taskId of scheduledTasks) {
// Task is already handled or being handled
if (!this.processedTasks.has(taskId)) {
this.processedTasks.set(taskId, this.processScheduledTask(taskId));
this.processedTasks.set(taskId, this.processTask(taskId));
}
}
}
@@ -214,6 +240,7 @@ export class TaskOrchestrator {
private async applyCachedResults(tasks: Task[]): Promise<
{
task: Task;
code: number;
status: 'local-cache' | 'local-cache-kept-existing' | 'remote-cache';
}[]
> {
@@ -228,6 +255,7 @@ export class TaskOrchestrator {
private async applyCachedResult(task: Task): Promise<{
task: Task;
code: number;
status: 'local-cache' | 'local-cache-kept-existing' | 'remote-cache';
}> {
const cachedResult = await this.cache.get(task);
@@ -255,6 +283,7 @@ export class TaskOrchestrator {
cachedResult.terminalOutput
);
return {
code: cachedResult.code,
task,
status,
};
@@ -324,12 +353,14 @@ export class TaskOrchestrator {
private async runBatch(batch: Batch, env: NodeJS.ProcessEnv) {
try {
const results = await this.forkedProcessTaskRunner.forkProcessForBatch(
batch,
this.projectGraph,
this.taskGraph,
env
);
const batchProcess =
await this.forkedProcessTaskRunner.forkProcessForBatch(
batch,
this.projectGraph,
this.taskGraph,
env
);
const results = await batchProcess.getResults();
const batchResultEntries = Object.entries(results);
return batchResultEntries.map(([taskId, result]) => ({
...result,
@@ -352,7 +383,7 @@ export class TaskOrchestrator {
// endregion Batch
// region Single Task
private async applyFromCacheOrRunTask(
async applyFromCacheOrRunTask(
doNotSkipCache: boolean,
task: Task,
groupId: number
@@ -394,110 +425,162 @@ export class TaskOrchestrator {
let results: {
task: Task;
code: number;
status: TaskStatus;
terminalOutput?: string;
}[] = doNotSkipCache ? await this.applyCachedResults([task]) : [];
// the task wasn't cached
if (results.length === 0) {
const shouldPrefix =
streamOutput && process.env.NX_PREFIX_OUTPUT === 'true';
const targetConfiguration = getTargetConfigurationForTask(
const childProcess = await this.runTask(
task,
this.projectGraph
streamOutput,
env,
temporaryOutputPath,
pipeOutput
);
if (
process.env.NX_RUN_COMMANDS_DIRECTLY !== 'false' &&
targetConfiguration.executor === 'nx:run-commands' &&
!shouldPrefix
) {
try {
const { schema } = getExecutorForTask(task, this.projectGraph);
const isRunOne = this.initiatingProject != null;
const combinedOptions = combineOptionsForExecutor(
task.overrides,
task.target.configuration ??
targetConfiguration.defaultConfiguration,
targetConfiguration,
schema,
task.target.project,
relative(task.projectRoot ?? workspaceRoot, process.cwd()),
process.env.NX_VERBOSE_LOGGING === 'true'
);
if (combinedOptions.env) {
env = {
...env,
...combinedOptions.env,
};
}
if (streamOutput) {
const args = getPrintableCommandArgsForTask(task);
output.logCommand(args.join(' '));
}
const { success, terminalOutput } = await runCommandsImpl(
{
...combinedOptions,
env,
usePty: isRunOne && !this.tasksSchedule.hasTasks(),
streamOutput,
},
{
root: workspaceRoot, // only root is needed in runCommandsImpl
} as any
);
const status = success ? 'success' : 'failure';
if (!streamOutput) {
this.options.lifeCycle.printTaskTerminalOutput(
task,
status,
terminalOutput
);
}
writeFileSync(temporaryOutputPath, terminalOutput);
results.push({
task,
status,
terminalOutput,
});
} catch (e) {
if (process.env.NX_VERBOSE_LOGGING === 'true') {
console.error(e);
} else {
console.error(e.message);
}
const terminalOutput = e.stack ?? e.message ?? '';
writeFileSync(temporaryOutputPath, terminalOutput);
results.push({
task,
status: 'failure',
terminalOutput,
});
}
} else if (targetConfiguration.executor === 'nx:noop') {
writeFileSync(temporaryOutputPath, '');
results.push({
task,
status: 'success',
terminalOutput: '',
});
} else {
// cache prep
const { code, terminalOutput } = await this.runTaskInForkedProcess(
task,
env,
pipeOutput,
temporaryOutputPath,
streamOutput
const { code, terminalOutput } = await childProcess.getResults();
results.push({
task,
code,
status: code === 0 ? 'success' : 'failure',
terminalOutput,
});
}
await this.postRunSteps([task], results, doNotSkipCache, { groupId });
return results[0];
}
private async runTask(
task: Task,
streamOutput: boolean,
env: { [p: string]: string | undefined; TZ?: string },
temporaryOutputPath: string,
pipeOutput: boolean
): Promise<RunningTask> {
const shouldPrefix =
streamOutput && process.env.NX_PREFIX_OUTPUT === 'true';
const targetConfiguration = getTargetConfigurationForTask(
task,
this.projectGraph
);
if (
process.env.NX_RUN_COMMANDS_DIRECTLY !== 'false' &&
targetConfiguration.executor === 'nx:run-commands' &&
!shouldPrefix
) {
try {
const { schema } = getExecutorForTask(task, this.projectGraph);
const combinedOptions = combineOptionsForExecutor(
task.overrides,
task.target.configuration ?? targetConfiguration.defaultConfiguration,
targetConfiguration,
schema,
task.target.project,
relative(task.projectRoot ?? workspaceRoot, process.cwd()),
process.env.NX_VERBOSE_LOGGING === 'true'
);
results.push({
task,
status: code === 0 ? 'success' : 'failure',
if (combinedOptions.env) {
env = {
...env,
...combinedOptions.env,
};
}
if (streamOutput) {
const args = getPrintableCommandArgsForTask(task);
output.logCommand(args.join(' '));
}
const runCommandsOptions = {
...combinedOptions,
env,
usePty:
this.tuiEnabled ||
(!this.tasksSchedule.hasTasks() &&
this.runningContinuousTasks.size === 0),
streamOutput,
};
const runningTask = await runCommands(runCommandsOptions, {
root: workspaceRoot, // only root is needed in runCommands
} as any);
if (this.tuiEnabled && runningTask instanceof PseudoTtyProcess) {
// This is an external of a the pseudo terminal where a task is running and can be passed to the TUI
this.options.lifeCycle.registerRunningTask(
task.id,
runningTask.getParserAndWriter()
);
}
if (!streamOutput) {
if (runningTask instanceof PseudoTtyProcess) {
// TODO: shouldn't this be checking if the task is continuous before writing anything to disk or calling printTaskTerminalOutput?
let terminalOutput = '';
runningTask.onOutput((data) => {
terminalOutput += data;
});
runningTask.onExit((code) => {
this.options.lifeCycle.printTaskTerminalOutput(
task,
code === 0 ? 'success' : 'failure',
terminalOutput
);
writeFileSync(temporaryOutputPath, terminalOutput);
});
} else {
runningTask.onExit((code, terminalOutput) => {
this.options.lifeCycle.printTaskTerminalOutput(
task,
code === 0 ? 'success' : 'failure',
terminalOutput
);
writeFileSync(temporaryOutputPath, terminalOutput);
});
}
}
return runningTask;
} catch (e) {
if (process.env.NX_VERBOSE_LOGGING === 'true') {
console.error(e);
} else {
console.error(e.message);
}
const terminalOutput = e.stack ?? e.message ?? '';
writeFileSync(temporaryOutputPath, terminalOutput);
return new NoopChildProcess({
code: 1,
terminalOutput,
});
}
} else if (targetConfiguration.executor === 'nx:noop') {
writeFileSync(temporaryOutputPath, '');
return new NoopChildProcess({
code: 0,
terminalOutput: '',
});
} else {
// cache prep
const runningTask = await this.runTaskInForkedProcess(
task,
env,
pipeOutput,
temporaryOutputPath,
streamOutput
);
if (this.tuiEnabled && runningTask instanceof PseudoTtyProcess) {
// This is an external of a the pseudo terminal where a task is running and can be passed to the TUI
this.options.lifeCycle.registerRunningTask(
task.id,
runningTask.getParserAndWriter()
);
}
return runningTask;
}
await this.postRunSteps([task], results, doNotSkipCache, { groupId });
}
private async runTaskInForkedProcess(
@@ -510,10 +593,11 @@ export class TaskOrchestrator {
try {
const usePtyFork = process.env.NX_NATIVE_COMMAND_RUNNER !== 'false';
// Disable the pseudo terminal if this is a run-many
const disablePseudoTerminal = !this.initiatingProject;
// Disable the pseudo terminal if this is a run-many or when running a continuous task as part of a run-one
const disablePseudoTerminal =
!this.tuiEnabled && (!this.initiatingProject || task.continuous);
// execution
const { code, terminalOutput } = usePtyFork
const childProcess = usePtyFork
? await this.forkedProcessTaskRunner.forkProcess(task, {
temporaryOutputPath,
streamOutput,
@@ -530,20 +614,95 @@ export class TaskOrchestrator {
env,
});
return {
code,
terminalOutput,
};
return childProcess;
} catch (e) {
if (process.env.NX_VERBOSE_LOGGING === 'true') {
console.error(e);
}
return {
return new NoopChildProcess({
code: 1,
};
terminalOutput: undefined,
});
}
}
async startContinuousTask(task: Task, groupId: number) {
if (this.runningTasksService.getRunningTasks([task.id]).length) {
// task is already running, we need to poll and wait for the running task to finish
do {
console.log(`Waiting for ${task.id} in another nx process`);
await new Promise((resolve) => setTimeout(resolve, 100));
} while (this.runningTasksService.getRunningTasks([task.id]).length);
return;
}
const taskSpecificEnv = await this.processedTasks.get(task.id);
await this.preRunSteps([task], { groupId });
const pipeOutput = await this.pipeOutputCapture(task);
// obtain metadata
const temporaryOutputPath = this.cache.temporaryOutputPath(task);
const streamOutput =
this.outputStyle === 'static'
? false
: shouldStreamOutput(task, this.initiatingProject);
let env = pipeOutput
? getEnvVariablesForTask(
task,
taskSpecificEnv,
process.env.FORCE_COLOR === undefined
? 'true'
: process.env.FORCE_COLOR,
this.options.skipNxCache,
this.options.captureStderr,
null,
null
)
: getEnvVariablesForTask(
task,
taskSpecificEnv,
undefined,
this.options.skipNxCache,
this.options.captureStderr,
temporaryOutputPath,
streamOutput
);
const childProcess = await this.runTask(
task,
streamOutput,
env,
temporaryOutputPath,
pipeOutput
);
this.runningTasksService.addRunningTask(task.id);
this.runningContinuousTasks.set(task.id, childProcess);
childProcess.onExit((code) => {
this.runningTasksService.removeRunningTask(task.id);
if (!this.cleaningUp) {
console.error(
`Task "${task.id}" is continuous but exited with code ${code}`
);
this.cleanup();
}
});
if (
this.initiatingProject === task.target.project &&
this.options.targets.length === 1 &&
this.options.targets[0] === task.target.target
) {
await childProcess.getResults();
} else {
await this.tasksSchedule.scheduleNextTasks();
// release blocked threads
this.waitingForTasks.forEach((f) => f(null));
this.waitingForTasks.length = 0;
}
return childProcess;
}
// endregion Single Task
// region Lifecycle
@@ -730,4 +889,44 @@ export class TaskOrchestrator {
}
// endregion utils
private async cleanup() {
this.cleaningUp = true;
await Promise.all(
Array.from(this.runningContinuousTasks).map(async ([taskId, t]) => {
try {
return t.kill();
} catch (e) {
console.error(`Unable to terminate ${taskId}\nError:`, e);
} finally {
this.runningTasksService.removeRunningTask(taskId);
}
})
);
}
}
export function getThreadCount(
options: NxArgs & DefaultTasksRunnerOptions,
taskGraph: TaskGraph
) {
if (
(options as any)['parallel'] === 'false' ||
(options as any)['parallel'] === false
) {
(options as any)['parallel'] = 1;
} else if (
(options as any)['parallel'] === 'true' ||
(options as any)['parallel'] === true ||
(options as any)['parallel'] === undefined ||
(options as any)['parallel'] === ''
) {
(options as any)['parallel'] = Number((options as any)['maxParallel'] || 3);
}
const maxParallel =
options['parallel'] +
Object.values(taskGraph.tasks).filter((t) => t.continuous).length;
const totalTasks = Object.keys(taskGraph.tasks).length;
return Math.min(maxParallel, totalTasks);
}
@@ -18,6 +18,7 @@ function createMockTask(id: string, parallelism: boolean = true): Task {
outputs: [],
overrides: {},
parallelism,
continuous: false,
};
}
@@ -65,6 +66,11 @@ describe('TasksSchedule', () => {
'app2:build': [],
'lib1:build': [],
},
continuousDependencies: {
'app1:build': [],
'app2:build': [],
'lib1:build': [],
},
roots: ['lib1:build', 'app2:build'],
};
jest.spyOn(nxJsonUtils, 'readNxJson').mockReturnValue({});
@@ -274,6 +280,13 @@ describe('TasksSchedule', () => {
'app4:test': [],
'lib1:test': [],
},
continuousDependencies: {
'app1:test': [],
'app2:test': [],
'app3:test': [],
'app4:test': [],
'lib1:test': [],
},
roots: [
'app1:test',
'app2:test',
@@ -552,6 +565,11 @@ describe('TasksSchedule', () => {
'app2:build': [],
'lib1:build': [],
},
continuousDependencies: {
'app1:build': [],
'app2:build': [],
'lib1:build': [],
},
roots: ['lib1:build', 'app2:build'],
};
jest.spyOn(nxJsonUtils, 'readNxJson').mockReturnValue({});
@@ -719,6 +737,11 @@ describe('TasksSchedule', () => {
'app2:test': [],
'lib1:test': [],
},
continuousDependencies: {
'app1:test': [],
'app2:test': [],
'lib1:test': [],
},
roots: ['app1:test', 'app2:test', 'lib1:test'],
};
jest.spyOn(nxJsonUtils, 'readNxJson').mockReturnValue({});
@@ -212,12 +212,15 @@ export class TasksSchedule {
({
tasks: {},
dependencies: {},
continuousDependencies: {},
roots: [],
} as TaskGraph));
batch.tasks[task.id] = task;
batch.dependencies[task.id] =
this.notScheduledTaskGraph.dependencies[task.id];
batch.continuousDependencies[task.id] =
this.notScheduledTaskGraph.continuousDependencies[task.id];
if (isRoot) {
batch.roots.push(task.id);
}
@@ -251,9 +254,13 @@ export class TasksSchedule {
const hasDependenciesCompleted = this.taskGraph.dependencies[taskId].every(
(id) => this.completedTasks.has(id)
);
const hasContinuousDependenciesStarted =
this.taskGraph.continuousDependencies[taskId].every((id) =>
this.runningTasks.has(id)
);
// if dependencies have not completed, cannot schedule
if (!hasDependenciesCompleted) {
if (!hasDependenciesCompleted || !hasContinuousDependenciesStarted) {
return false;
}

Some files were not shown because too many files have changed in this diff Show More