Compare commits

..
Author SHA1 Message Date
Henry Mercer 977e6ceaea Merge pull request #4124 from github/henrymercer/toolcache-bundle-cleanup
Delete unused CodeQL bundles from the toolcache before downloading
2026-09-08 17:04:25 +00:00
Henry MercerandCopilot App 40a6b38247 Address toolcache cleanup review feedback
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-08 15:55:41 +01:00
Henry Mercer deece8f852 Apply suggestion from @henrymercer 2026-09-07 14:08:36 +01:00
copilot-swe-agent[bot] 034db721dd Merge remote-tracking branch 'origin/main' into henrymercer/toolcache-bundle-cleanup
Co-authored-by: henrymercer <14129055+henrymercer@users.noreply.github.com>
2026-09-07 12:57:51 +00:00
Henry Mercer 313a0b9922 Merge pull request #4122 from github/henrymercer/friendly-potato
Don't record an overlay status when the job was cancelled
2026-09-04 16:55:52 +00:00
Henry MercerandCopilot App 31da345c07 Address review comments
Delete each version directory individually so that a symlinked one is skipped rather than removed, take an `ActionState` so the environment is read through `ReadOnlyEnv` rather than the deprecated `getOptionalEnvVar`, let `deleteToolcacheBundles` report its own failure to locate the toolcache instead of having the caller catch it, quote paths in log messages, and rename `HAS_OBTAINED_CODEQL_TOOLS` to `HAS_SET_UP_CODEQL`, which is also set when we find the tools in the toolcache rather than downloading them.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 17:43:40 +01:00
Henry Mercer 46dfb14111 Merge pull request #4123 from github/redsun82-address-pr-4072-comments
Analyze all supported languages in the Linux Arm64 check
2026-09-04 16:39:50 +00:00
Henry Mercer e06b60f75f Merge pull request #4125 from github/henrymercer/merge-queue-sarif-upload-flake
Don't upload the ESLint SARIF from merge queue refs
2026-09-04 16:37:26 +00:00
Henry MercerandCopilot App a48f2d3077 Record an overlay status only for conclusive job statuses
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 17:33:02 +01:00
Henry MercerandCopilot App 657964c39f Read the job status from a ReadOnlyEnv in recordOverlayStatus
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 17:31:24 +01:00
Henry MercerandCopilot App d2fe508a6d Don't upload the ESLint SARIF from merge queue refs
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 17:22:52 +01:00
Paolo TranquilliandCopilot App 16da74769e Log found databases in Linux Arm64 check
Log a success line for each language database found in the Linux Arm64 PR check.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 12:27:23 +02:00
Henry MercerandCopilot App e13c3dc834 Only clean up a toolcache on the workspace filesystem
Some runner images keep the toolcache on a different volume to the workspace, so deleting the tools there frees up disk space that the analysis cannot use, and costs a later step that wanted them in the toolcache a download. Windows runners are laid out this way, with the toolcache on `C:` and the workspace on `D:`.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 11:00:56 +01:00
Henry MercerandCopilot App 1331773b9a Address Copilot review feedback
Run the cleanup even when the download will not be cached in the toolcache, since the toolcache shares a filesystem with the directory we extract to, so freeing it helps either way, and report an error other than the toolcache being absent as a failure rather than as an empty toolcache.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:26:09 +01:00
Henry MercerandCopilot App 2681b03bd6 Add a changelog note
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:18:43 +01:00
Henry MercerandCopilot App a7a90f3ffb Enable toolcache cleanup in the multi-language PR check
The check installs the tools from a URL, so most versions in its matrix are downloaded rather than found in the toolcache, which is when the cleanup runs, and it then builds and analyses seven languages, so a bundle we damaged on the way in would show up.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:12:11 +01:00
Henry MercerandCopilot App 4a0b22ec25 Delete unused CodeQL bundles from the toolcache before downloading
When we download a bundle the toolcache often already holds a different one that the job will not use, and on GitHub-hosted runners it shares a filesystem with the workspace, so it takes space away from the analysis. Empty the toolcache before downloading, which also frees space for the archive during extraction, and which is safe because getting as far as a download means the tools were not resolved from the toolcache. Skip this once a step has obtained the tools, since a later step may run a path it was given, and gate it on the runner being GitHub-hosted and on a feature flag that is off by default.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:12:10 +01:00
Henry MercerandCopilot App 762a5ed7f7 Add a helper to delete the CodeQL tools from the toolcache
`deleteToolcacheBundles` removes `$RUNNER_TOOL_CACHE/CodeQL` and reports which versions were there. It refuses to follow a symlinked CodeQL directory so that it can only ever delete paths that are really inside the toolcache, and reports failures rather than throwing. Not called yet.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:12:10 +01:00
Henry MercerandCopilot App ceb85f25b5 Distinguish GitHub-hosted runners from ones that look hosted
`isHostedRunner` infers hostedness from the runner name and the toolcache path, so it also matches self-hosted runners that are configured to resemble hosted ones. Rename it to `looksLikeHostedRunner` so callers can see they are getting a heuristic, and add `isGitHubHostedRunner`, which reads the `RUNNER_ENVIRONMENT` value the Actions service reports. The existing callers keep the heuristic, so there is no behaviour change.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 10:12:09 +01:00
Henry Mercer 9fddc16f0d Merge pull request #4088 from github/dependabot/npm_and_yarn/octokit/plugin-rest-endpoint-methods-18.0.0
Bump @octokit/plugin-rest-endpoint-methods from 17.0.0 to 18.0.0
2026-09-04 07:23:47 +00:00
Paolo TranquilliandCopilot App 36cbf13b5e Address review comments
- Analyze all languages CodeQL supports on Linux Arm64 (except Swift,
  which is macOS only) in the Linux Arm64 check, building the compiled
  ones via the shared multi-language-repo build.sh
- De-duplicate the check's language list via a shared LANGUAGES env var
  used by both the init input and the assert loop
- Fix CHANGELOG tense: "download" -> "downloads"

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-04 08:30:10 +02:00
Henry MercerandCopilot App 38dd4a088a Don't record an overlay status when the job was cancelled
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-03 18:21:46 +01:00
Henry MercerandCopilot App 2f3c1c964a Add an internal job-status input to the init Action
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-03 18:21:14 +01:00
Michael B. Gale 5914b031b1 Merge pull request #4111 from github/dependabot/github_actions/dot-github/workflows/actions/setup-java-6.0.0
Bump actions/setup-java from 5.7.0 to 6.0.0 in /.github/workflows
2026-09-03 16:50:30 +00:00
Michael B. Gale 102499482f Merge pull request #4119 from github/dependabot/npm_and_yarn/npm-minor-21ac56865f
Bump the npm-minor group across 1 directory with 2 updates
2026-09-03 16:22:14 +00:00
Michael B. Gale 9401a335a3 Merge pull request #4118 from github/dependabot/npm_and_yarn/browserslist-4.28.8
Bump browserslist from 4.24.2 to 4.28.8
2026-09-03 16:21:07 +00:00
Michael B. Gale 3ee8a9398b Merge branch 'main' into dependabot/npm_and_yarn/browserslist-4.28.8 2026-09-03 17:03:16 +01:00
Michael B. Gale 25da1b1495 Merge branch 'main' into dependabot/npm_and_yarn/npm-minor-21ac56865f 2026-09-03 17:00:00 +01:00
Michael B. Gale 9509fd0822 Merge pull request #4120 from github/dependabot/npm_and_yarn/eslint-plugin-jsdoc-64.2.1
Bump eslint-plugin-jsdoc from 62.9.0 to 64.2.1
2026-09-03 15:58:16 +00:00
Michael B. Gale 20ec60c2d9 Merge branch 'main' into dependabot/npm_and_yarn/eslint-plugin-jsdoc-64.2.1 2026-09-03 15:12:29 +01:00
Paolo Tranquilli 4cccb3aa86 Merge pull request #4072 from github/redsun82-linux-arm64-support
Add support for Linux Arm64 runners
2026-09-03 13:58:15 +00:00
github-actions[bot] b724a86493 Rebuild 2026-09-02 17:56:48 +00:00
dependabot[bot] 2a03009e3f Bump eslint-plugin-jsdoc from 62.9.0 to 64.2.1
Bumps [eslint-plugin-jsdoc](https://github.com/gajus/eslint-plugin-jsdoc) from 62.9.0 to 64.2.1.
- [Release notes](https://github.com/gajus/eslint-plugin-jsdoc/releases)
- [Commits](https://github.com/gajus/eslint-plugin-jsdoc/compare/v62.9.0...v64.2.1)

---
updated-dependencies:
- dependency-name: eslint-plugin-jsdoc
  dependency-version: 64.2.1
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-02 17:55:01 +00:00
dependabot[bot] da21771e2e Bump the npm-minor group across 1 directory with 2 updates
Bumps the npm-minor group with 2 updates in the / directory: [js-yaml](https://github.com/nodeca/js-yaml) and [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint).


Updates `js-yaml` from 5.3.0 to 5.4.0
- [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/5.3.0...5.4.0)

Updates `typescript-eslint` from 8.67.0 to 8.68.0
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.68.0/packages/typescript-eslint)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 5.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm-minor
- dependency-name: typescript-eslint
  dependency-version: 8.68.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: npm-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-02 17:54:50 +00:00
dependabot[bot] bb36f8049c Bump browserslist from 4.24.2 to 4.28.8
Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.24.2 to 4.28.8.
- [Release notes](https://github.com/browserslist/browserslist/releases)
- [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/browserslist/browserslist/compare/4.24.2...4.28.8)

---
updated-dependencies:
- dependency-name: browserslist
  dependency-version: 4.28.8
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-02 13:55:34 +00:00
Michael B. Gale 276e2ce25d Merge pull request #4117 from github/mbg/deps/1sep
Update dependencies
2026-09-02 13:39:31 +00:00
Michael B. Gale ccd5275aa4 Merge branch 'main' into dependabot/github_actions/dot-github/workflows/actions/setup-java-6.0.0 2026-09-02 14:36:24 +01:00
Mario Campos a0c73122a6 Merge pull request #4096 from github/mario-campos/use-json-module
Use `json` module for JSON validation in `output-cache`
2026-09-01 15:17:48 +00:00
Michael B. Gale 50e244824e Update dependencies 2026-09-01 12:16:07 +01:00
Paolo TranquilliandCopilot App 025009006c Address review comments
- Stub process.platform/arch in the supported-platform test and loop
  over all supported pairs (including linux/arm64) so it no longer
  depends on the host
- Run the default queries in the Linux Arm64 PR check so the databases
  are finalized end-to-end

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fb7e2d12-6620-4a67-9c1c-4e82f4a5e8d9
2026-08-28 17:28:29 +02:00
Mario Campos 6f530319d8 Merge pull request #4113 from github/mario-campos/add-changenote-script
Create `changetool` script for validating change-notes
2026-08-27 18:09:46 +00:00
Henry MercerandCopilot App 1bcdd0c019 Derive the API client type from @actions/github
`ApiClient` was hand-composed from `Octokit`, `Api` and `PaginateInterface`, imported directly from separately versioned Octokit packages. That asserted a shape matching what `@actions/github` actually returns, which held only while the versions happened to agree.

`@octokit/plugin-rest-endpoint-methods` v18 adds twelve Actions cache-limit methods. `@actions/github` still depends on `^17.0.0`, so the client it constructs no longer satisfies the v18-derived alias and the build fails.

Deriving the type from the constructor removes the assumption, so the alias tracks whatever `@actions/github` returns.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-27 18:56:47 +01:00
Mario CamposandHenry Mercer f049ecb9ce Delete unused Node.js flag --experimental-test-module-mocks
Co-authored-by: Henry Mercer <henrymercer@github.com>
2026-08-27 11:22:56 -05:00
dependabot[bot] 25694c08ea Bump @octokit/plugin-rest-endpoint-methods from 17.0.0 to 18.0.0
Bumps [@octokit/plugin-rest-endpoint-methods](https://github.com/octokit/plugin-rest-endpoint-methods.js) from 17.0.0 to 18.0.0.
- [Release notes](https://github.com/octokit/plugin-rest-endpoint-methods.js/releases)
- [Commits](https://github.com/octokit/plugin-rest-endpoint-methods.js/compare/v17.0.0...v18.0.0)

---
updated-dependencies:
- dependency-name: "@octokit/plugin-rest-endpoint-methods"
  dependency-version: 18.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-27 11:01:07 +00:00
Henry Mercer 0963041ab4 Merge pull request #4090 from github/dependabot/npm_and_yarn/octokit/plugin-paginate-rest-15.0.0
Bump @octokit/plugin-paginate-rest from 14.0.0 to 15.0.0
2026-08-27 10:45:55 +00:00
github-actions[bot] 1a8ddd7325 Rebuild 2026-08-27 10:32:49 +00:00
Henry Mercer abc579a511 Merge pull request #4112 from github/dependabot/npm_and_yarn/npm-minor-62a3d68ea9
Bump the npm-minor group across 1 directory with 2 updates
2026-08-27 10:28:51 +00:00
dependabot[bot] 4d1d53ec73 Bump @octokit/plugin-paginate-rest from 14.0.0 to 15.0.0
Bumps [@octokit/plugin-paginate-rest](https://github.com/octokit/plugin-paginate-rest.js) from 14.0.0 to 15.0.0.
- [Release notes](https://github.com/octokit/plugin-paginate-rest.js/releases)
- [Commits](https://github.com/octokit/plugin-paginate-rest.js/compare/v14.0.0...v15.0.0)

---
updated-dependencies:
- dependency-name: "@octokit/plugin-paginate-rest"
  dependency-version: 15.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-27 10:08:06 +00:00
Henry Mercer 6d4d5dfcd6 Merge pull request #4089 from github/dependabot/npm_and_yarn/octokit/types-17.0.0
Bump @octokit/types from 16.0.0 to 17.0.0
2026-08-27 09:37:27 +00:00
Paolo Tranquilli 7acc225a21 Merge pull request #4110 from github/redsun82-release-merge-instructions
Document merge-commit requirement for release PRs
2026-08-27 08:26:42 +00:00
Paolo Tranquilli 76ba03be44 Merge branch 'main' into redsun82-release-merge-instructions 2026-08-27 10:10:14 +02:00
Paolo Tranquilli 6441b26a49 Merge remote-tracking branch 'origin/main' into redsun82-linux-arm64-support 2026-08-27 10:09:22 +02:00
Mario Campos cacb7b37ed Delete unused script/changetool NPM scripts 2026-08-26 23:39:11 -05:00
Mario Campos e2cd31e32e Refactor tsconfig.json to extend base configuration and simplify options 2026-08-26 23:32:24 -05:00
Mario Campos be555cf0a3 Extend scripts/changetool as a NPM workspace 2026-08-26 23:25:06 -05:00
Mario Campos 0447ab23b5 Enhance changenote validation to support nested bullet lists 2026-08-26 23:06:47 -05:00
Mario Campos 50ec5a5c18 Use Object.hasOwn instead of in operator
Apparently, the `in` operator will check inherited properties too.
2026-08-26 22:23:00 -05:00
Mario Campos 5aea817801 Replace Array.prototype.every() with for loop
`.every()` short-circuits on the first `false`
2026-08-26 22:20:25 -05:00
Mario Campos 580da893f6 Handle file errors as false 2026-08-26 22:15:34 -05:00
Mario Campos 426c10420b Create changetool script for validating change-notes 2026-08-26 17:13:40 -05:00
github-actions[bot] dc5bc53050 Rebuild 2026-08-26 18:07:30 +00:00
github-actions[bot] d75af13705 Rebuild 2026-08-26 18:03:17 +00:00
dependabot[bot] 0006c77502 Bump the npm-minor group across 1 directory with 2 updates
Bumps the npm-minor group with 2 updates in the / directory: [js-yaml](https://github.com/nodeca/js-yaml) and [uuid](https://github.com/uuidjs/uuid).


Updates `js-yaml` from 5.2.3 to 5.3.0
- [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/5.2.3...5.3.0)

Updates `uuid` from 14.0.1 to 14.0.2
- [Release notes](https://github.com/uuidjs/uuid/releases)
- [Changelog](https://github.com/uuidjs/uuid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/uuidjs/uuid/compare/v14.0.1...v14.0.2)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 5.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm-minor
- dependency-name: uuid
  dependency-version: 14.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: npm-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-26 18:01:20 +00:00
dependabot[bot] ec1bd1999f Bump actions/setup-java from 5.7.0 to 6.0.0 in /.github/workflows
Bumps [actions/setup-java](https://github.com/actions/setup-java) from 5.7.0 to 6.0.0.
- [Release notes](https://github.com/actions/setup-java/releases)
- [Commits](https://github.com/actions/setup-java/compare/b6effb05e454b25005698d916606bdc6ffcbf961...dd06d9cba3e5552c54d9f8ea23572deb30010f7c)

---
updated-dependencies:
- dependency-name: actions/setup-java
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-26 17:59:42 +00:00
Henry Mercer 4d40c93072 Merge pull request #4108 from github/mergeback/v4.37.9-to-main-cdf488f5
Mergeback v4.37.9 refs/heads/releases/v4 into main
2026-08-26 17:19:52 +00:00
Henry Mercer 3e93618b99 Merge pull request #4105 from github/henrymercer/tools-download-telemetry
Improve CodeQL tools download time telemetry
2026-08-26 17:01:23 +00:00
Paolo TranquilliandCopilot App 42c2ea9ef7 Document merge-commit requirement for release PRs
Release, mergeback, and backport PRs must be merged with a merge commit
so the branch linkage the release automation depends on is preserved
(the mergeback tags the release using the merge commit of the
"Merge main into releases/vN" PR, so squashing or rebasing breaks
tagging).

Add a path-scoped Copilot instructions file scoped to the files that
reliably change in these PRs (CHANGELOG.md, src/defaults.json,
lib/defaults.json, src/api-compatibility.json), and add an explicit note
to the Releasing runbook in CONTRIBUTING.md so the rule is enforced
regardless of who merges.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0dae6e42-7fc6-4fb1-b1ea-15dc17e402a2
2026-08-26 17:13:12 +02:00
github-actions[bot] f37565646f Rebuild 2026-08-26 14:41:38 +00:00
github-actions[bot] 0f2e2bde5c Update changelog and version after v4.37.9 2026-08-26 14:41:24 +00:00
Paolo Tranquilli cdf488f595 Merge pull request #4107 from github/update-v4.37.9-920ba7cd1
Merge main into releases/v4
2026-08-26 16:39:15 +02:00
github-actions[bot] 7243f38558 Update changelog for v4.37.9 2026-08-26 12:56:36 +00:00
Paolo Tranquilli 920ba7cd15 Merge pull request #4106 from github/update-bundle/codeql-bundle-v2.26.4
Update default bundle to 2.26.4
2026-08-26 08:49:41 +00:00
github-actions[bot] ecfa6e1681 Add changelog note 2026-08-26 08:23:26 +00:00
github-actions[bot] adcdf4a70d Update default bundle to codeql-bundle-v2.26.4 2026-08-26 08:23:20 +00:00
Paolo Tranquilli b5383aad77 Merge remote-tracking branch 'origin/redsun82-linux-arm64-support' into redsun82-linux-arm64-support 2026-08-25 16:08:34 +02:00
Henry MercerandCopilot App 9d89e2d1d6 Rebuild
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 628ce334-991a-4578-9c1b-93d2e96bbddb
2026-08-24 18:32:11 +01:00
Henry MercerandCopilot App bee82de8ba Add tools download durations to the init status report
Surface `tools_extraction_duration_ms` and `tools_total_duration_ms` from
both the `init` and `setup-codeql` actions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 628ce334-991a-4578-9c1b-93d2e96bbddb
2026-08-24 18:32:08 +01:00
Henry MercerandCopilot App c05e445696 Report tools download durations on both download paths
The streaming path reported no timings at all, so we have no data for the
path that most runs take. It now reports a total duration, which is also
populated on the download-then-extract path.

That path additionally reports the extraction duration, which was
previously computed but only logged.

`downloadDurationMs` keeps its existing meaning of time spent downloading
alone, so existing telemetry stays comparable.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 628ce334-991a-4578-9c1b-93d2e96bbddb
2026-08-24 18:32:04 +01:00
Paolo Tranquilli a952c0706c Merge branch 'main' into redsun82-linux-arm64-support 2026-08-24 10:55:44 +02:00
Paolo Tranquilli 97b7459e09 Merge remote-tracking branch 'origin/main' into redsun82-linux-arm64-support 2026-08-21 17:43:09 +02:00
Henry Mercer 486fec2a3e Merge pull request #4099 from github/update-supported-enterprise-server-versions
Update supported GitHub Enterprise Server versions
2026-08-21 12:54:32 +00:00
Henry Mercer 134624c67b Merge pull request #4101 from github/dependabot/npm_and_yarn/npm-minor-457d827a35
Bump the npm-minor group across 1 directory with 4 updates
2026-08-21 12:53:13 +00:00
Henry Mercer ff43db8f98 Merge pull request #4103 from github/mergeback/v4.37.8-to-main-db488dde
Mergeback v4.37.8 refs/heads/releases/v4 into main
2026-08-21 12:51:15 +00:00
github-actions[bot] 4605e03a74 Rebuild 2026-08-21 12:26:00 +00:00
github-actions[bot] 099c869cad Update changelog and version after v4.37.8 2026-08-21 12:25:47 +00:00
Henry Mercer db488ddef3 Merge pull request #4102 from github/update-v4.37.8-9ee088e13
Merge main into releases/v4
2026-08-21 13:24:15 +01:00
github-actions[bot] 1845f5ba8b Update changelog for v4.37.8 2026-08-21 12:09:34 +00:00
dependabot[bot] 79a73408b4 Bump the npm-minor group across 1 directory with 4 updates
Bumps the npm-minor group with 4 updates in the / directory: [esbuild](https://github.com/evanw/esbuild), [globals](https://github.com/sindresorhus/globals), [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint) and [tsx](https://github.com/privatenumber/tsx).


Updates `esbuild` from 0.28.1 to 0.28.2
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.28.1...v0.28.2)

Updates `globals` from 17.9.0 to 17.11.0
- [Release notes](https://github.com/sindresorhus/globals/releases)
- [Commits](https://github.com/sindresorhus/globals/compare/v17.9.0...v17.11.0)

Updates `typescript-eslint` from 8.66.0 to 8.67.0
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.67.0/packages/typescript-eslint)

Updates `tsx` from 4.23.8 to 4.23.12
- [Release notes](https://github.com/privatenumber/tsx/releases)
- [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs)
- [Commits](https://github.com/privatenumber/tsx/compare/v4.23.8...v4.23.12)

---
updated-dependencies:
- dependency-name: esbuild
  dependency-version: 0.28.2
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm-minor
- dependency-name: globals
  dependency-version: 17.11.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: npm-minor
- dependency-name: typescript-eslint
  dependency-version: 8.67.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: npm-minor
- dependency-name: tsx
  dependency-version: 4.23.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-19 17:55:06 +00:00
github-actions[bot] f9d9f07d37 Update supported GitHub Enterprise Server versions 2026-08-19 00:11:43 +00:00
Henry Mercer 9ee088e136 Merge pull request #4080 from github/henrymercer/studious-giggle
Determine the overlay minimum disk space requirement from feature flags
2026-08-18 12:25:05 +00:00
Henry MercerandCopilot App 1aef003397 Address review feedback on overlay disk flags
Document each minimum disk feature flag individually and replace the tuple list with an explicit feature-to-threshold mapping.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-18 12:09:33 +01:00
Henry MercerandCopilot App 508b83bc41 Merge main into overlay minimum disk feature branch
Resolve the overlap with the separately shipped promotion of the overlay resource checks while preserving the feature-flagged minimum disk thresholds.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-18 12:09:09 +01:00
Mario Campos fcd8d74cda Refactor output-caching functions to accept file path dependency
This makes it easier to test. Credit to @mbg.
2026-08-14 14:41:04 -05:00
Mario Campos b5d34388b7 Refactor tests in output-cache.test.ts to use named imports from output-cache 2026-08-14 14:28:17 -05:00
Mario Campos 1f46830a1f Fix VersionInfo derived type to pass linter 2026-08-14 14:24:37 -05:00
Mario Campos b60777a215 Delete unnecessary statement from JSDoc comments 2026-08-14 14:05:42 -05:00
Mario Campos f3deecb42d Refactor OutputCache with json module 2026-08-14 11:38:38 -05:00
Mario Campos c8ba2d2a05 Refactor isVersionInfo with json module 2026-08-14 11:35:14 -05:00
Mario Campos d7d901a925 Refactor tests to use getCommandCacheFilePath
As well as fixup a few other problems.
2026-08-14 10:56:07 -05:00
Mario Campos cb9d39fcc1 Export getCommandCacheFilePath function for use in tests 2026-08-14 10:55:33 -05:00
Michael B. Gale d97b3428e8 Merge pull request #4098 from github/mbg/permission-error-as-configuration-error
Make `EACCES` when installing CodeQL CLI a `ConfigurationError`
2026-08-14 11:56:43 +00:00
Michael B. Gale 47fa622223 Make EACCES a ConfigurationError 2026-08-14 11:56:26 +01:00
Michael B. Gale 45693cc688 Refactor ENOSPC check into isDiskConfigurationError function 2026-08-14 11:53:32 +01:00
Mario Campos 93d793c5fe Use json module for JSON validation in output-cache 2026-08-13 23:19:37 -05:00
Mario Campos c2fd8f54d1 Merge pull request #4081 from github/mario-campos/version-cache-to-disk
Persist CodeQL version output to file rather than environment
2026-08-13 14:27:41 +00:00
Mario Campos c56f48e9bd Log unexpected conditions during caching CLI output 2026-08-13 09:14:00 -05:00
Mario Campos aa0eadc572 Merge branch 'main' into mario-campos/version-cache-to-disk
# Conflicts:
#	lib/entry-points.js
2026-08-13 09:02:15 -05:00
Mario Campos 43250d671a Change cache key to string type to include CLI args 2026-08-13 08:59:09 -05:00
Michael B. Gale 5008effa71 Merge pull request #4094 from github/mergeback/v4.37.7-to-main-ff2f1c62
Mergeback v4.37.7 refs/heads/releases/v4 into main
2026-08-13 13:48:05 +00:00
github-actions[bot] 053d41e61e Rebuild 2026-08-13 13:30:41 +00:00
github-actions[bot] 1158e1c92a Update changelog and version after v4.37.7 2026-08-13 13:30:28 +00:00
Mario Campos 2d49edbac6 Re-order env to be first argument for consistency 2026-08-13 08:13:49 -05:00
dependabot[bot] 9875827ada Bump @octokit/types from 16.0.0 to 17.0.0
Bumps [@octokit/types](https://github.com/octokit/types.ts) from 16.0.0 to 17.0.0.
- [Release notes](https://github.com/octokit/types.ts/releases)
- [Commits](https://github.com/octokit/types.ts/compare/v16.0.0...v17.0.0)

---
updated-dependencies:
- dependency-name: "@octokit/types"
  dependency-version: 17.0.0
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-13 10:32:23 +00:00
Mario Campos 6dc633238e Bolster output-cache unit tests with more test cases 2026-08-12 13:45:22 -05:00
Mario Campos 6c0d9018d4 Change OutputCache to use object for entries
This will ensure it works nicely with `JSON.stringify`. And, then we can validate the type before marshaling.
2026-08-12 12:01:20 -05:00
Mario Campos bf96b0df93 Expand test to ensure it does not throw an exception 2026-08-12 11:39:32 -05:00
Mario Campos 337136ab8a Rename CommandCacheRecord -> OutputCache 2026-08-12 11:29:11 -05:00
Mario Campos a9baab8dee Export CLI cache types 2026-08-12 11:26:52 -05:00
Mario Campos 33d70867d5 Pass environment explicitly to CLI caching functions 2026-08-12 11:24:37 -05:00
Mario Campos 40f80a8df0 Rename type to better match generic intention 2026-08-11 18:09:09 -05:00
Mario Campos b222c3aaea Generalize file cache data structure 2026-08-11 18:09:09 -05:00
Mario Campos 11569df0a1 Update JSDoc of getCachedCodeQlVersion 2026-08-11 15:55:16 -05:00
Mario Campos 0a99875ae5 Move VersionInfo-related types to cli/output-cache.ts
This brings them out of the crowded all-purpose `util.ts` and into `cli/output-cache.ts` where they are exclusively used.
2026-08-11 15:55:15 -05:00
Mario Campos 246018e041 Move VersionInfo to dedicated module 2026-08-11 15:55:15 -05:00
Mario Campos 1332611f51 Move cache-related util functions into dedicated module 2026-08-11 15:55:15 -05:00
Mario Campos 4dc327a942 Introduce basic cli/output-cache.ts module 2026-08-11 15:36:17 -05:00
Mario Campos bb19330c5e Add test of getCachedCodeQlVersion with no file 2026-08-11 15:36:17 -05:00
Mario Campos 0e85c0e99c Refactor unit test to extract testing values 2026-08-11 15:36:17 -05:00
Mario Campos bfcd769ba1 Fix JSDoc of env param 2026-08-11 15:36:17 -05:00
Mario CamposandMichael B. Gale 208a88adc7 Simplify JSDoc of getCachedCodeQlVersion
Co-authored-by: Michael B. Gale <mbg@github.com>
2026-08-11 11:06:17 -05:00
Mario Campos 9183a7b6e1 Handle file-read errors as cache misses
This is particularly important for the first time that `getCachedCodeQlVersion` is invoked, as this cache file will not yet exist.
2026-08-10 10:44:48 -05:00
Paolo Tranquilli b4c390c362 Merge remote-tracking branch 'origin/main' into redsun82-linux-arm64-support 2026-08-10 09:51:23 +02:00
Mario Campos 5f8c44ba62 Persist CodeQL version output to file rather than environment 2026-08-07 15:11:20 -05:00
Henry MercerandCopilot App 794f5bc385 Fix the memory check debug message at equality
The comparison accepts exactly the minimum, so say "at or above", to
match the wording of the disk space check.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-06 16:48:44 +01:00
Henry MercerandCopilot App 54109818e0 Address review feedback on the disk space check
Say "at or above" in the debug message logged when the check passes,
since the comparison accepts exactly the minimum.

Check each feature flag against the threshold its name declares, rather
than only exercising a few of them, so that a mistake in one of the
mappings cannot go unnoticed. Both sides of the boundary are needed to
pin a threshold down: a mapping to a lower value would still pass the
case at the limit, and one to a higher value would still fail the case
below it.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-06 16:41:48 +01:00
Henry MercerandCopilot App 99caaa8b90 Remove the overlay_analysis_resource_checks_v2 feature flag
The flag no longer has any effect now that its 14 GB threshold is the
unconditional default, so remove it. Setting
CODEQL_ACTION_OVERLAY_ANALYSIS_RESOURCE_CHECKS_V2 no longer does
anything.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-06 16:38:34 +01:00
Henry MercerandCopilot App 6117bb503a Derive overlay minimum disk space from feature flags
Overlay analysis required 20 GB of available disk space, lowered to
14 GB when overlay_analysis_resource_checks_v2 was enabled. That gave
us a single step to roll out, and any further reduction needed another
flag and another release.

Determine the threshold from the new overlay_analysis_min_disk_N_gb
flags instead, taking the lowest one that is enabled so that a lower
limit can be rolled out to a subset of repositories without first
disabling the flag above it. When none are enabled, the 14 GB limit
now applies unconditionally, replacing the 20 GB default.

Thresholds remain in decimal MB, matching the bytes-per-MB convention
the disk check already used, so the effective byte values are unchanged
from the previous 14 GB path. Also log the available and required space
at debug level when the check passes, so that run logs show which
threshold took effect.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-06 16:38:33 +01:00
Henry MercerandCopilot App af767ec1f6 Add overlay_analysis_min_disk_N_gb feature flags
Add six feature flags, overlay_analysis_min_disk_8_gb through
overlay_analysis_min_disk_13_gb, which will be used to control the
amount of available disk space that overlay analysis requires.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-06 16:38:33 +01:00
Paolo TranquilliandCopilot App 43ae5e70b0 Address review comments
- Add darwin/x64 case to linked bundle tests and disambiguate the
  parametrized test title by platform/arch
- Reword CHANGELOG entry to clarify the CLI supports Linux Arm64
- Add a dedicated Linux Arm64 end-to-end PR check on ubuntu-24.04-arm

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 308c78bb-53a3-4877-a373-70fab87543f1
2026-08-05 15:51:01 +02:00
Paolo TranquilliandCopilot App 4c96123771 Add support for linux/arm64
Recognize the linux/arm64 platform so the CodeQL CLI is accepted on
Arm64 Linux runners and the native per-platform bundle
codeql-bundle-linux-arm64.tar.zst is resolved, rather than falling
through to the combined all-platforms gzip fallback (which deliberately
omits arm64).

Add ["linux", "arm64"] to the supported platforms and make the linux
bundle name arch-aware. Keep the download mock faithful and cover the
new arch branch with tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-04 15:32:12 +02:00
63 changed files with 6067 additions and 2539 deletions

No files matched your search

+1 -1
View File
@@ -16,5 +16,5 @@ inputs:
Comma separated list of query ids that should NOT be included in this SARIF file.
runs:
using: node20
using: node24
main: index.js
@@ -1,6 +1,6 @@
name: Verify that the best-effort debug artifact scan completed
description: Verifies that the best-effort debug artifact scan completed successfully during tests
runs:
using: node20
using: node24
main: index.js
post: post.js
@@ -0,0 +1,16 @@
---
applyTo: "CHANGELOG.md,src/defaults.json,lib/defaults.json,src/api-compatibility.json"
---
# Merging release, mergeback, and backport PRs
The release process creates a cascade of PRs (`main` → `releases/vN`, then
`releases/vN` → `main` mergeback, then `releases/vN` → `releases/v(N-1)`
backport). These PRs reliably touch `CHANGELOG.md`, `src/defaults.json` /
`lib/defaults.json` (bundle/CLI version bump), and `src/api-compatibility.json`.
Such PRs **must be merged with a merge commit**. Never squash or rebase, as
that breaks the branch linkage the release automation relies on.
When arming auto-merge on these PRs, use `--merge` (e.g. `gh pr merge --merge`),
not `--squash` or `--rebase`.
@@ -63,7 +63,7 @@ jobs:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Java
uses: actions/setup-java@b6effb05e454b25005698d916606bdc6ffcbf961 # v5.7.0
uses: actions/setup-java@dd06d9cba3e5552c54d9f8ea23572deb30010f7c # v6.0.0
with:
java-version: ${{ inputs.java-version || '17' }}
distribution: temurin
+1 -1
View File
@@ -63,7 +63,7 @@ jobs:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Java
uses: actions/setup-java@b6effb05e454b25005698d916606bdc6ffcbf961 # v5.7.0
uses: actions/setup-java@dd06d9cba3e5552c54d9f8ea23572deb30010f7c # v6.0.0
with:
java-version: ${{ inputs.java-version || '17' }}
distribution: temurin
+106
View File
@@ -0,0 +1,106 @@
# Warning: This file is generated automatically, and should not be modified.
# Instead, please modify the template in the pr-checks directory and run:
# pr-checks/sync.sh
# to regenerate this file.
name: PR Check - Linux Arm64
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GO111MODULE: auto
on:
push:
branches:
- main
- releases/v*
pull_request: {}
merge_group:
types:
- checks_requested
schedule:
- cron: '0 5 * * *'
workflow_dispatch:
inputs:
dotnet-version:
type: string
description: The version of .NET to install
required: false
default: 9.x
go-version:
type: string
description: The version of Go to install
required: false
default: '>=1.21.0'
workflow_call:
inputs:
dotnet-version:
type: string
description: The version of .NET to install
required: false
default: 9.x
go-version:
type: string
description: The version of Go to install
required: false
default: '>=1.21.0'
defaults:
run:
shell: bash
concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' || false }}
group: linux-arm64-${{github.ref}}-${{inputs.dotnet-version}}-${{inputs.go-version}}
jobs:
linux-arm64:
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04-arm
version: nightly-latest
name: Linux Arm64
if: github.triggering_actor != 'dependabot[bot]'
permissions:
contents: read
security-events: read
timeout-minutes: 45
runs-on: ${{ matrix.os }}
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install .NET
uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0
with:
dotnet-version: ${{ inputs.dotnet-version || '9.x' }}
- name: Install Go
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version: ${{ inputs.go-version || '>=1.21.0' }}
cache: false
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
with:
version: ${{ matrix.version }}
use-all-platform-bundle: 'false'
setup-kotlin: 'true'
- uses: ./../action/init
with:
languages: ${{ env.LANGUAGES }}
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Build code
run: ./build.sh
- uses: ./../action/analyze
with:
upload-database: false
- name: Assert databases exist
run: |
cd "$RUNNER_TEMP/codeql_databases"
for lang in ${LANGUAGES//,/ }; do
if [[ ! -d "$lang" ]]; then
echo "Did not find a database for $lang"
exit 1
fi
echo "Found database for $lang"
done
env:
LANGUAGES: cpp,csharp,go,java,javascript,python,ruby
CODEQL_ACTION_TEST_MODE: true
+1
View File
@@ -191,5 +191,6 @@ jobs:
exit 1
fi
env:
CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES: true
CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI: true
CODEQL_ACTION_TEST_MODE: true
+33 -1
View File
@@ -67,11 +67,43 @@ jobs:
- name: Upload sarif
uses: ./upload-sarif
if: matrix.os == 'ubuntu-latest' && matrix.node-version == 24
# The merge queue deletes its `gh-readonly-queue` ref as soon as the queue entry resolves,
# so uploading against it races with that deletion. Both the `merge_group` run and the
# paired `push` run that the queue branch creates use that ref, so gate on the ref itself
# rather than the event. The same results are uploaded by the `pull_request` run and again
# by the `push` run on `main`.
if: matrix.os == 'ubuntu-latest' && matrix.node-version == 24 && !startsWith(github.ref, 'refs/heads/gh-readonly-queue/')
with:
sarif_file: eslint.sarif
category: eslint
changetool-tests:
name: changetool unit tests
permissions:
contents: read
runs-on: ubuntu-slim
timeout-minutes: 10
concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' || false }}
group: pr-checks-changetool-tests-${{ github.ref }}-${{ github.event_name }}
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 24
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run changetool unit tests
run: npm --workspace changetool test
# These checks do not need to be run as part of the same matrix that we use for the `unit-tests`
# job.
other-checks:
+57 -44
View File
@@ -2,74 +2,87 @@
See the [releases page](https://github.com/github/codeql-action/releases) for the relevant changes to the CodeQL CLI and language packs.
## 3.37.7 - 13 Aug 2026
## [UNRELEASED]
- On GitHub-hosted runners, the CodeQL Action now deletes unused CodeQL bundles from the toolcache before downloading a different bundle, which frees up disk space for the analysis. We expect to roll this change out to everyone in September. [#4124](https://github.com/github/codeql-action/pull/4124)
- The CodeQL Action now supports CodeQL releases that are compatible with Linux Arm64 and downloads the native `linux-arm64` CodeQL bundle when available. [#4072](https://github.com/github/codeql-action/pull/4072)
## 4.37.9 - 26 Aug 2026
- Update default CodeQL bundle version to [2.26.4](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.26.4). [#4106](https://github.com/github/codeql-action/pull/4106)
## 4.37.8 - 21 Aug 2026
No user facing changes.
## 4.37.7 - 13 Aug 2026
- Update default CodeQL bundle version to [2.26.3](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.26.3). [#4085](https://github.com/github/codeql-action/pull/4085)
## 3.37.6 - 04 Aug 2026
## 4.37.6 - 04 Aug 2026
- Changed the default filepath for the new remote file address format that was introduced in CodeQL Action 4.37.0 / 3.37.0 to `.github/codeql-config.yml` to align it with the suggested path that is used elsewhere. [#4070](https://github.com/github/codeql-action/pull/4070)
## 3.37.5 - 03 Aug 2026
## 4.37.5 - 03 Aug 2026
- Fixed a bug where a network error while streaming the download of the CodeQL bundle could terminate the `init` Action instead of falling back to downloading the bundle before extracting it. [#4061](https://github.com/github/codeql-action/pull/4061)
## 3.37.4 - 29 Jul 2026
## 4.37.4 - 29 Jul 2026
- This version of the CodeQL Action adds support for the `tools` input for the `codeql-action/init` step to be specified using a `github-codeql-tools` [repository property](https://docs.github.com/en/organizations/managing-organization-settings/managing-custom-properties-for-repositories-in-your-organization). This feature will gradually be rolled out following the release of this version. Once rolled out, this allows for the CodeQL CLI version that is used in GitHub-managed workflows, such as Default Setup, to be set to a custom value. For example, customers who run into issues with rate limits when a new CodeQL CLI version is released can set the value to `toolcache` to always use the CodeQL CLI version that is available in the runner toolcache. For Advanced Setup workflows, the value provided for `tools` in the workflow definition always takes precedence unless the value of the repository property starts with `!`. [#4037](https://github.com/github/codeql-action/pull/4037)
- Update default CodeQL bundle version to [2.26.2](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.26.2). [#4051](https://github.com/github/codeql-action/pull/4051)
## 3.37.3 - 22 Jul 2026
## 4.37.3 - 22 Jul 2026
No user facing changes.
## 3.37.2 - 21 Jul 2026
## 4.37.2 - 21 Jul 2026
- The new address format for the `config-file` input that was introduced in CodeQL Action 4.37.0 is now enabled by default. In addition to the format described there, the `remote=` prefix can now be used to explicitly indicate that the input refers to a remote file. All previous input formats continue to be accepted as well. [#4023](https://github.com/github/codeql-action/pull/4023)
- The CodeQL Action can now make use of [configured private registries](https://docs.github.com/en/code-security/how-tos/secure-at-scale/configure-organization-security/manage-usage-and-access/giving-org-access-private-registries) in Default Setup to retrieve CodeQL configuration files from remote repositories that require authentication. This will allow customers to store their CodeQL configuration in a single repository that can then be referenced by Default Setup workflows in other repositories. We expect to roll this and other, related changes out to everyone in July. [#4007](https://github.com/github/codeql-action/pull/4007)
## 3.37.1 - 16 Jul 2026
## 4.37.1 - 16 Jul 2026
- _Upcoming breaking change_: Add a deprecation warning for customers using CodeQL version 2.20.6 and earlier. These versions of CodeQL were discontinued on 1 July 2026 alongside GitHub Enterprise Server 3.16, and will be unsupported by the next minor release of the CodeQL Action. [#3956](https://github.com/github/codeql-action/pull/3956)
- Update default CodeQL bundle version to [2.26.1](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.26.1). [#4019](https://github.com/github/codeql-action/pull/4019)
## 3.37.0 - 08 Jul 2026
## 4.37.0 - 08 Jul 2026
- Update default CodeQL bundle version to [2.26.0](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.26.0). [#3995](https://github.com/github/codeql-action/pull/3995)
- In addition to the existing input format, the `config-file` input for the `codeql-action/init` step will soon support a new `[owner/]repo[@ref][:path]` format. All components except the repository name are optional. If omitted, `owner` defaults to the same owner as the repository the analysis is running for, `ref` to `main`, and `path` to `.github/codeql-action.yaml`. Support for this format ships in this version of the CodeQL Action, but will only be enabled over the coming weeks. [#3973](https://github.com/github/codeql-action/pull/3973)
## 3.36.3 - 01 Jul 2026
## 4.36.3 - 01 Jul 2026
No user facing changes.
## 3.36.2 - 04 Jun 2026
## 4.36.2 - 04 Jun 2026
- Cache CodeQL CLI version information across Actions steps. [#3943](https://github.com/github/codeql-action/pull/3943)
- Reduce requests while waiting for analysis processing by using exponential backoff when polling SARIF processing status. [#3937](https://github.com/github/codeql-action/pull/3937)
- Update default CodeQL bundle version to [2.25.6](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.6). [#3948](https://github.com/github/codeql-action/pull/3948)
## 3.36.1 - 02 Jun 2026
## 4.36.1 - 02 Jun 2026
No user facing changes.
## 3.36.0 - 22 May 2026
## 4.36.0 - 22 May 2026
- _Breaking change_: Bump the minimum required CodeQL bundle version to 2.19.4. [#3894](https://github.com/github/codeql-action/pull/3894)
- Add support for SHA-256 Git object IDs. [#3893](https://github.com/github/codeql-action/pull/3893)
- Update default CodeQL bundle version to [2.25.5](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.5). [#3926](https://github.com/github/codeql-action/pull/3926)
## 3.35.5 - 15 May 2026
## 4.35.5 - 15 May 2026
- We have improved how the JavaScript bundles for the CodeQL Action are generated to avoid duplication across bundles and reduce the size of the repository by around 70%. This should have no effect on the runtime behaviour of the CodeQL Action. [#3899](https://github.com/github/codeql-action/pull/3899)
- For performance and accuracy reasons, [improved incremental analysis](https://github.com/github/roadmap/issues/1158) will now only be enabled on a pull request when diff-informed analysis is also enabled for that run. If diff-informed analysis is unavailable (for example, because the PR diff ranges could not be computed), the action will fall back to a full analysis. [#3791](https://github.com/github/codeql-action/pull/3791)
- If multiple inputs are provided for the GitHub-internal `analysis-kinds` input, only `code-scanning` will be enabled. The `analysis-kinds` input is experimental, for GitHub-internal use only, and may change without notice at any time. [#3892](https://github.com/github/codeql-action/pull/3892)
- Added an experimental change which, when running a Code Scanning analysis for a PR with [improved incremental analysis](https://github.com/github/roadmap/issues/1158) enabled, prefers CodeQL CLI versions that have a cached overlay-base database for the configured languages. This speeds up analysis for a repository when there is not yet a cached overlay-base database for the latest CLI version. We expect to roll this change out to everyone in May. [#3880](https://github.com/github/codeql-action/pull/3880)
## 3.35.4 - 07 May 2026
## 4.35.4 - 07 May 2026
- Update default CodeQL bundle version to [2.25.4](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.4). [#3881](https://github.com/github/codeql-action/pull/3881)
## 3.35.3 - 01 May 2026
## 4.35.3 - 01 May 2026
- _Upcoming breaking change_: Add a deprecation warning for customers using CodeQL version 2.19.3 and earlier. These versions of CodeQL were discontinued on 9 April 2026 alongside GitHub Enterprise Server 3.15, and will be unsupported by the next minor release of the CodeQL Action. [#3837](https://github.com/github/codeql-action/pull/3837)
- Configurations for private registries that use Cloudsmith or GCP OIDC are now accepted. [#3850](https://github.com/github/codeql-action/pull/3850)
@@ -77,7 +90,7 @@ No user facing changes.
- Fixed a bug where two diagnostics produced within the same millisecond could overwrite each other on disk, causing one of them to be lost. [#3852](https://github.com/github/codeql-action/pull/3852)
- Update default CodeQL bundle version to [2.25.3](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.3). [#3865](https://github.com/github/codeql-action/pull/3865)
## 3.35.2 - 15 Apr 2026
## 4.35.2 - 15 Apr 2026
- The undocumented TRAP cache cleanup feature that could be enabled using the `CODEQL_ACTION_CLEANUP_TRAP_CACHES` environment variable is deprecated and will be removed in May 2026. If you are affected by this, we recommend disabling TRAP caching by passing the `trap-caching: false` input to the `init` Action. [#3795](https://github.com/github/codeql-action/pull/3795)
- The Git version 2.36.0 requirement for improved incremental analysis now only applies to repositories that contain submodules. [#3789](https://github.com/github/codeql-action/pull/3789)
@@ -85,26 +98,26 @@ No user facing changes.
- Fixed a bug in the validation of OIDC configurations for private registries that was added in CodeQL Action 4.33.0 / 3.33.0. [#3807](https://github.com/github/codeql-action/pull/3807)
- Update default CodeQL bundle version to [2.25.2](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.2). [#3823](https://github.com/github/codeql-action/pull/3823)
## 3.35.1 - 27 Mar 2026
## 4.35.1 - 27 Mar 2026
- Fix incorrect minimum required Git version for [improved incremental analysis](https://github.com/github/roadmap/issues/1158): it should have been 2.36.0, not 2.11.0. [#3781](https://github.com/github/codeql-action/pull/3781)
## 3.35.0 - 27 Mar 2026
## 4.35.0 - 27 Mar 2026
- Reduced the minimum Git version required for [improved incremental analysis](https://github.com/github/roadmap/issues/1158) from 2.38.0 to 2.11.0. [#3767](https://github.com/github/codeql-action/pull/3767)
- Update default CodeQL bundle version to [2.25.1](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.1). [#3773](https://github.com/github/codeql-action/pull/3773)
## 3.34.1 - 20 Mar 2026
## 4.34.1 - 20 Mar 2026
- Downgrade default CodeQL bundle version to [2.24.3](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.24.3) due to issues with a small percentage of Actions and JavaScript analyses. [#3762](https://github.com/github/codeql-action/pull/3762)
## 3.34.0 - 20 Mar 2026
## 4.34.0 - 20 Mar 2026
- Added an experimental change which disables TRAP caching when [improved incremental analysis](https://github.com/github/roadmap/issues/1158) is enabled, since improved incremental analysis supersedes TRAP caching. This will improve performance and reduce Actions cache usage. We expect to roll this change out to everyone in March. [#3569](https://github.com/github/codeql-action/pull/3569)
- We are rolling out improved incremental analysis to C/C++ analyses that use build mode `none`. We expect this rollout to be complete by the end of April 2026. [#3584](https://github.com/github/codeql-action/pull/3584)
- Update default CodeQL bundle version to [2.25.0](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.0). [#3585](https://github.com/github/codeql-action/pull/3585)
## 3.33.0 - 16 Mar 2026
## 4.33.0 - 16 Mar 2026
- Upcoming change: Starting April 2026, the CodeQL Action will skip collecting file coverage information on pull requests to improve analysis performance. File coverage information will still be computed on non-PR analyses. Pull request analyses will log a warning about this upcoming change. [#3562](https://github.com/github/codeql-action/pull/3562)
@@ -118,11 +131,11 @@ No user facing changes.
- Fixed the retry mechanism for database uploads. Previously this would fail with the error "Response body object should not be disturbed or locked". [#3564](https://github.com/github/codeql-action/pull/3564)
- A warning is now emitted if the CodeQL Action detects a repository property whose name suggests that it relates to the CodeQL Action, but which is not one of the properties recognised by the current version of the CodeQL Action. [#3570](https://github.com/github/codeql-action/pull/3570)
## 3.32.6 - 05 Mar 2026
## 4.32.6 - 05 Mar 2026
- Update default CodeQL bundle version to [2.24.3](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.24.3). [#3548](https://github.com/github/codeql-action/pull/3548)
## 3.32.5 - 02 Mar 2026
## 4.32.5 - 02 Mar 2026
- Repositories owned by an organization can now set up the `github-codeql-disable-overlay` custom repository property to disable [improved incremental analysis for CodeQL](https://github.com/github/roadmap/issues/1158). First, create a custom repository property with the name `github-codeql-disable-overlay` and the type "True/false" in the organization's settings. Then in the repository's settings, set this property to `true` to disable improved incremental analysis. For more information, see [Managing custom properties for repositories in your organization](https://docs.github.com/en/organizations/managing-organization-settings/managing-custom-properties-for-repositories-in-your-organization). This feature is not yet available on GitHub Enterprise Server. [#3507](https://github.com/github/codeql-action/pull/3507)
- Added an experimental change so that when [improved incremental analysis](https://github.com/github/roadmap/issues/1158) fails on a runner — potentially due to insufficient disk space — the failure is recorded in the Actions cache so that subsequent runs will automatically skip improved incremental analysis until something changes (e.g. a larger runner is provisioned or a new CodeQL version is released). We expect to roll this change out to everyone in March. [#3487](https://github.com/github/codeql-action/pull/3487)
@@ -132,7 +145,7 @@ No user facing changes.
- Added an experimental change which allows the `start-proxy` action to resolve the CodeQL CLI version from feature flags instead of using the linked CLI bundle version. We expect to roll this change out to everyone in March. [#3512](https://github.com/github/codeql-action/pull/3512)
- The previously experimental changes from versions 4.32.3, 4.32.4, 3.32.3 and 3.32.4 are now enabled by default. [#3503](https://github.com/github/codeql-action/pull/3503), [#3504](https://github.com/github/codeql-action/pull/3504)
## 3.32.4 - 20 Feb 2026
## 4.32.4 - 20 Feb 2026
- Update default CodeQL bundle version to [2.24.2](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.24.2). [#3493](https://github.com/github/codeql-action/pull/3493)
- Added an experimental change which improves how certificates are generated for the authentication proxy that is used by the CodeQL Action in Default Setup when [private package registries are configured](https://docs.github.com/en/code-security/how-tos/secure-at-scale/configure-organization-security/manage-usage-and-access/giving-org-access-private-registries). This is expected to generate more widely compatible certificates and should have no impact on analyses which are working correctly already. We expect to roll this change out to everyone in February. [#3473](https://github.com/github/codeql-action/pull/3473)
@@ -140,89 +153,89 @@ No user facing changes.
- Added a setting which allows the CodeQL Action to enable network debugging for Java programs. This will help GitHub staff support customers with troubleshooting issues in GitHub-managed CodeQL workflows, such as Default Setup. This setting can only be enabled by GitHub staff. [#3485](https://github.com/github/codeql-action/pull/3485)
- Added a setting which enables GitHub-managed workflows, such as Default Setup, to use a [nightly CodeQL CLI release](https://github.com/dsp-testing/codeql-cli-nightlies) instead of the latest, stable release that is used by default. This will help GitHub staff support customers whose analyses for a given repository or organization require early access to a change in an upcoming CodeQL CLI release. This setting can only be enabled by GitHub staff. [#3484](https://github.com/github/codeql-action/pull/3484)
## 3.32.3 - 13 Feb 2026
## 4.32.3 - 13 Feb 2026
- Added experimental support for testing connections to [private package registries](https://docs.github.com/en/code-security/how-tos/secure-at-scale/configure-organization-security/manage-usage-and-access/giving-org-access-private-registries). This feature is not currently enabled for any analysis. In the future, it may be enabled by default for Default Setup. [#3466](https://github.com/github/codeql-action/pull/3466)
## 3.32.2 - 05 Feb 2026
## 4.32.2 - 05 Feb 2026
- Update default CodeQL bundle version to [2.24.1](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.24.1). [#3460](https://github.com/github/codeql-action/pull/3460)
## 3.32.1 - 02 Feb 2026
## 4.32.1 - 02 Feb 2026
- A warning is now shown in Default Setup workflow logs if a [private package registry is configured](https://docs.github.com/en/code-security/how-tos/secure-at-scale/configure-organization-security/manage-usage-and-access/giving-org-access-private-registries) using a GitHub Personal Access Token (PAT), but no username is configured. [#3422](https://github.com/github/codeql-action/pull/3422)
- Fixed a bug which caused the CodeQL Action to fail when repository properties cannot successfully be retrieved. [#3421](https://github.com/github/codeql-action/pull/3421)
## 3.32.0 - 26 Jan 2026
## 4.32.0 - 26 Jan 2026
- Update default CodeQL bundle version to [2.24.0](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.24.0). [#3425](https://github.com/github/codeql-action/pull/3425)
## 3.31.11 - 23 Jan 2026
## 4.31.11 - 23 Jan 2026
- When running a Default Setup workflow with [Actions debugging enabled](https://docs.github.com/en/actions/how-tos/monitor-workflows/enable-debug-logging), the CodeQL Action will now use more unique names when uploading logs from the Dependabot authentication proxy as workflow artifacts. This ensures that the artifact names do not clash between multiple jobs in a build matrix. [#3409](https://github.com/github/codeql-action/pull/3409)
- Improved error handling throughout the CodeQL Action. [#3415](https://github.com/github/codeql-action/pull/3415)
- Added experimental support for automatically excluding [generated files](https://docs.github.com/en/repositories/working-with-files/managing-files/customizing-how-changed-files-appear-on-github) from the analysis. This feature is not currently enabled for any analysis. In the future, it may be enabled by default for some GitHub-managed analyses. [#3318](https://github.com/github/codeql-action/pull/3318)
- The changelog extracts that are included with releases of the CodeQL Action are now shorter to avoid duplicated information from appearing in Dependabot PRs. [#3403](https://github.com/github/codeql-action/pull/3403)
## 3.31.10 - 12 Jan 2026
## 4.31.10 - 12 Jan 2026
- Update default CodeQL bundle version to 2.23.9. [#3393](https://github.com/github/codeql-action/pull/3393)
## 3.31.9 - 16 Dec 2025
## 4.31.9 - 16 Dec 2025
No user facing changes.
## 3.31.8 - 11 Dec 2025
## 4.31.8 - 11 Dec 2025
- Update default CodeQL bundle version to 2.23.8. [#3354](https://github.com/github/codeql-action/pull/3354)
## 3.31.7 - 05 Dec 2025
## 4.31.7 - 05 Dec 2025
- Update default CodeQL bundle version to 2.23.7. [#3343](https://github.com/github/codeql-action/pull/3343)
## 3.31.6 - 01 Dec 2025
## 4.31.6 - 01 Dec 2025
No user facing changes.
## 3.31.5 - 24 Nov 2025
## 4.31.5 - 24 Nov 2025
- Update default CodeQL bundle version to 2.23.6. [#3321](https://github.com/github/codeql-action/pull/3321)
## 3.31.4 - 18 Nov 2025
## 4.31.4 - 18 Nov 2025
No user facing changes.
## 3.31.3 - 13 Nov 2025
## 4.31.3 - 13 Nov 2025
- CodeQL Action v3 will be deprecated in December 2026. The Action now logs a warning for customers who are running v3 but could be running v4. For more information, see [Upcoming deprecation of CodeQL Action v3](https://github.blog/changelog/2025-10-28-upcoming-deprecation-of-codeql-action-v3/).
- Update default CodeQL bundle version to 2.23.5. [#3288](https://github.com/github/codeql-action/pull/3288)
## 3.31.2 - 30 Oct 2025
## 4.31.2 - 30 Oct 2025
No user facing changes.
## 3.31.1 - 30 Oct 2025
## 4.31.1 - 30 Oct 2025
- The `add-snippets` input has been removed from the `analyze` action. This input has been deprecated since CodeQL Action 3.26.4 in August 2024 when this removal was announced.
## 3.31.0 - 24 Oct 2025
## 4.31.0 - 24 Oct 2025
- Bump minimum CodeQL bundle version to 2.17.6. [#3223](https://github.com/github/codeql-action/pull/3223)
- When SARIF files are uploaded by the `analyze` or `upload-sarif` actions, the CodeQL Action automatically performs post-processing steps to prepare the data for the upload. Previously, these post-processing steps were only performed before an upload took place. We are now changing this so that the post-processing steps will always be performed, even when the SARIF files are not uploaded. This does not change anything for the `upload-sarif` action. For `analyze`, this may affect Advanced Setup for CodeQL users who specify a value other than `always` for the `upload` input. [#3222](https://github.com/github/codeql-action/pull/3222)
## 3.30.9 - 17 Oct 2025
## 4.30.9 - 17 Oct 2025
- Update default CodeQL bundle version to 2.23.3. [#3205](https://github.com/github/codeql-action/pull/3205)
- Experimental: A new `setup-codeql` action has been added which is similar to `init`, except it only installs the CodeQL CLI and does not initialize a database. Do not use this in production as it is part of an internal experiment and subject to change at any time. [#3204](https://github.com/github/codeql-action/pull/3204)
## 3.30.8 - 10 Oct 2025
## 4.30.8 - 10 Oct 2025
No user facing changes.
## 3.30.7 - 06 Oct 2025
## 4.30.7 - 06 Oct 2025
- [v4+ only] The CodeQL Action now runs on Node.js v24. [#3169](https://github.com/github/codeql-action/pull/3169)
No user facing changes.
## 3.30.6 - 02 Oct 2025
- Update default CodeQL bundle version to 2.23.2. [#3168](https://github.com/github/codeql-action/pull/3168)
+5 -2
View File
@@ -60,10 +60,13 @@ Here are a few things you can do that will increase the likelihood of your pull
This workflow goes through the pull requests that have been merged to `main` since the last release, creates a changelog, then opens a pull request to merge the changes since the last release into the `releases/v3` release branch.
You can start a release by triggering this workflow via [workflow dispatch](https://github.com/github/codeql-action/actions/workflows/update-release-branch.yml).
1. The workflow run will open a pull request titled "Merge main into releases/v3". Follow the steps on the checklist in the pull request. Once you've checked off all but the last two of these, approve the PR and automerge it.
1. The workflow run will open a pull request titled "Merge main into releases/v3". Follow the steps on the checklist in the pull request. Once you've checked off all but the last two of these, approve the PR and automerge it **with a merge commit** (`gh pr merge --merge`).
1. When the "Merge main into releases/v3" pull request is merged into the `releases/v3` branch, a mergeback pull request to `main` will be automatically created. This mergeback pull request incorporates the changelog updates into `main`, tags the release using the merge commit of the "Merge main into releases/v3" pull request, and bumps the patch version of the CodeQL Action.
1. If a backport to an older major version is required, a pull request targeting that version's branch will also be automatically created.
1. Approve the mergeback and backport pull request (if applicable) and automerge them.
1. Approve the mergeback and backport pull request (if applicable) and automerge them **with a merge commit** (`gh pr merge --merge`).
> [!NOTE]
> The release, mergeback, and backport pull requests must always be merged with a merge commit — **never squash or rebase**. The mergeback tags the release using the merge commit of the "Merge main into releases/v3" pull request, so squashing or rebasing breaks tagging and the branch linkage the release automation relies on.
Once the mergeback and backport pull request have been merged, the release is complete.
+1 -1
View File
@@ -94,6 +94,6 @@ outputs:
sarif-id:
description: The ID of the uploaded SARIF file.
runs:
using: node20
using: node24
main: "../lib/analyze-entry.js"
post: "../lib/analyze-post-entry.js"
+1 -1
View File
@@ -15,5 +15,5 @@ inputs:
$GITHUB_WORKSPACE as its working directory.
required: false
runs:
using: node20
using: node24
main: '../lib/autobuild-entry.js'
+14
View File
@@ -209,4 +209,18 @@ export default [
],
},
},
{
files: ["scripts/changetool/**/*.ts"],
languageOptions: {
parserOptions: {
project: "./scripts/changetool/tsconfig.json",
},
},
rules: {
"no-console": "off",
"import/extensions": "off",
},
},
];
+8 -1
View File
@@ -164,12 +164,19 @@ inputs:
[Internal] The ID of the check run, as provided by the Actions runtime environment. Do not set this value manually.
default: ${{ job.check_run_id }}
required: false
job-status:
description: >-
[Internal] The status of the job, as provided by the Actions runtime environment. This is how the
post step learns whether the job as a whole succeeded, failed, or was cancelled. Do not set this
value manually.
default: ${{ job.status }}
required: false
outputs:
codeql-path:
description: The path of the CodeQL binary used for analysis
codeql-version:
description: The version of the CodeQL binary used for analysis
runs:
using: node20
using: node24
main: '../lib/init-entry.js'
post: '../lib/init-post-entry.js'
+4 -4
View File
@@ -1,6 +1,6 @@
{
"bundleVersion": "codeql-bundle-v2.26.3",
"cliVersion": "2.26.3",
"priorBundleVersion": "codeql-bundle-v2.26.2",
"priorCliVersion": "2.26.2"
"bundleVersion": "codeql-bundle-v2.26.4",
"cliVersion": "2.26.4",
"priorBundleVersion": "codeql-bundle-v2.26.3",
"priorCliVersion": "2.26.3"
}
+2239 -1818
View File
File diff suppressed because it is too large. Load diff
+1345 -404
View File
File diff suppressed because it is too large. Load diff
+12 -11
View File
@@ -1,6 +1,6 @@
{
"name": "codeql",
"version": "3.37.7",
"version": "4.37.10",
"private": true,
"description": "CodeQL action",
"scripts": {
@@ -17,7 +17,8 @@
},
"license": "MIT",
"workspaces": [
"pr-checks"
"pr-checks",
"scripts/changetool"
],
"dependencies": {
"@actions/artifact": "^5.0.3",
@@ -31,27 +32,27 @@
"@actions/io": "^2.0.0",
"@actions/tool-cache": "^3.0.1",
"@octokit/core": "^7.0.7",
"@octokit/plugin-paginate-rest": "^14.0.0",
"@octokit/plugin-rest-endpoint-methods": "^17.0.0",
"@octokit/plugin-paginate-rest": "^15.0.0",
"@octokit/plugin-rest-endpoint-methods": "^18.0.0",
"@octokit/plugin-retry": "^8.1.1",
"archiver": "^8.0.0",
"fast-deep-equal": "^3.1.3",
"follow-redirects": "^1.16.0",
"get-folder-size": "^5.0.0",
"https-proxy-agent": "^7.0.6",
"js-yaml": "^5.2.3",
"js-yaml": "^5.4.0",
"jsonschema": "1.5.0",
"long": "^5.3.2",
"node-forge": "^1.4.0",
"semver": "^7.8.5",
"uuid": "^14.0.1",
"uuid": "^14.0.2",
"undici": "^6.28.0"
},
"devDependencies": {
"@ava/typescript": "6.0.0",
"@eslint/compat": "^2.1.0",
"@microsoft/eslint-formatter-sarif": "^3.1.0",
"@octokit/types": "^16.0.0",
"@octokit/types": "^17.0.0",
"@types/archiver": "^8.0.0",
"@types/follow-redirects": "^1.14.4",
"@types/js-yaml": "^4.0.9",
@@ -61,19 +62,19 @@
"@types/semver": "^7.8.0",
"@types/sinon": "^22.0.0",
"ava": "^6.4.1",
"esbuild": "^0.28.1",
"esbuild": "^0.28.2",
"eslint": "^9.39.5",
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-github": "^6.1.2",
"eslint-plugin-import-x": "^4.17.1",
"eslint-plugin-jsdoc": "^62.9.0",
"eslint-plugin-jsdoc": "^64.2.1",
"eslint-plugin-no-async-foreach": "^0.1.1",
"glob": "^13.0.6",
"globals": "^17.9.0",
"globals": "^17.11.0",
"nock": "^14.0.17",
"sinon": "^22.1.0",
"typescript": "^6.0.3",
"typescript-eslint": "^8.66.0"
"typescript-eslint": "^8.68.0"
},
"overrides": {
"@actions/tool-cache": {
+1 -4
View File
@@ -1,10 +1,7 @@
import * as githubUtils from "@actions/github/lib/utils";
import { type Octokit } from "@octokit/core";
import { type PaginateInterface } from "@octokit/plugin-paginate-rest";
import { type Api } from "@octokit/plugin-rest-endpoint-methods";
/** The type of the Octokit client. */
export type ApiClient = Octokit & Api & { paginate: PaginateInterface };
export type ApiClient = InstanceType<typeof githubUtils.GitHub>;
/** Constructs an `ApiClient` using `token` for authentication. */
export function getApiClient(token: string): ApiClient {
+35
View File
@@ -0,0 +1,35 @@
name: "Linux Arm64"
description: "An end-to-end integration test running on a Linux Arm64 runner, checking that the native linux-arm64 CodeQL bundle is downloaded and can analyze interpreted and compiled code"
operatingSystems:
- os: ubuntu
runner-image: ubuntu-24.04-arm
# The native linux-arm64 CodeQL bundle is only available in recent CLI releases, so we restrict this
# check to `nightly-latest`, which is guaranteed to ship it. Older stable versions do not have an
# arm64 asset, and `prepare-test` would resolve an x64 bundle URL for them on this runner.
versions:
- nightly-latest
installGo: true
installDotNet: true
# The set of languages CodeQL supports on this platform, excluding Swift (macOS only).
env:
LANGUAGES: cpp,csharp,go,java,javascript,python,ruby
steps:
- uses: ./../action/init
with:
languages: ${{ env.LANGUAGES }}
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Build code
run: ./build.sh
- uses: ./../action/analyze
with:
upload-database: false
- name: Assert databases exist
run: |
cd "$RUNNER_TEMP/codeql_databases"
for lang in ${LANGUAGES//,/ }; do
if [[ ! -d "$lang" ]]; then
echo "Did not find a database for $lang"
exit 1
fi
echo "Found database for $lang"
done
@@ -15,6 +15,7 @@ operatingSystems:
- stable-v2.21.4
- stable-v2.22.4
env:
CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES: true
CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI: true
installGo: true
installDotNet: true
+3 -3
View File
@@ -5,13 +5,13 @@
"@actions/core": "^2.0.3",
"@actions/github": "^8.0.1",
"@octokit/core": "^7.0.7",
"@octokit/plugin-paginate-rest": ">=9.2.2",
"@octokit/plugin-rest-endpoint-methods": "^17.0.0",
"@octokit/plugin-paginate-rest": ">=15.0.0",
"@octokit/plugin-rest-endpoint-methods": "^18.0.0",
"semver": "^7.8.5",
"yaml": "^2.9.0"
},
"devDependencies": {
"@types/node": "^20.19.43",
"tsx": "^4.23.8"
"tsx": "^4.23.12"
}
}
+2 -2
View File
@@ -253,8 +253,8 @@ const languageSetups: LanguageSetups = {
name: "Install Java",
uses: pinnedUses(
"actions/setup-java",
"b6effb05e454b25005698d916606bdc6ffcbf961",
"v5.7.0",
"dd06d9cba3e5552c54d9f8ea23572deb30010f7c",
"v6.0.0",
),
with: {
"java-version": `\${{ inputs.java-version || '${defaultLanguageVersions.java}' }}`,
+1 -1
View File
@@ -21,5 +21,5 @@ outputs:
environment:
description: The inferred build environment configuration.
runs:
using: node20
using: node24
main: '../lib/resolve-environment-entry.js'
+200
View File
@@ -0,0 +1,200 @@
import assert from "node:assert/strict";
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { describe, it } from "node:test";
import {
isValidChangenoteContent,
isValidChangenoteFile,
isValidChangenoteFilename,
hasValidChangenoteCategory,
VALID_CHANGE_NOTE_CATEGORIES,
} from "./validate.ts";
async function withTmpFile<T>(
baseFileName: string,
contents: string,
body: (filePath: string) => Promise<T>,
): Promise<T> {
const tmpDir = fs.mkdtempSync(
path.join(os.tmpdir(), "changetool-validate-test-"),
);
try {
const filePath = path.join(tmpDir, baseFileName);
fs.writeFileSync(filePath, contents);
return await body(filePath);
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
}
await describe("isValidChangenoteContent", async () => {
await it("recognizes an unordered Markdown list", async () => {
const inputs = [
"- One changenote entry",
"- First item\n- Second item",
"\n\n\n\n- Fixed a bug\n- Added a feature",
];
for (const input of inputs) {
assert.equal(isValidChangenoteContent(input), true);
}
});
await it("does not recognize non-Markdown text", async () => {
const inputs = [
"This is not a list.",
'["this", "is", "JSON"]',
"---",
"***",
"___",
"paragraph",
];
for (const input of inputs) {
assert.equal(isValidChangenoteContent(input), false);
}
});
await it("does not recognize ordered Markdown lists", async () => {
const inputs = [
"1. First item\n2. Second item",
"\n\n\n1. First item\n1. Second item",
];
for (const input of inputs) {
assert.equal(isValidChangenoteContent(input), false);
}
});
await it("requires all list items to use a hyphen bullet", async () => {
const inputs = [
"* Fixed a bug\n* Added feature",
"+ Fixed a bug\n+ Added feature",
"- Fixed a bug\n* Added feature",
"- Fixed a bug\n+ Added feature",
"- Fixed a bug\n * Added feature\n + Updated docs",
"\n\n\n* Fixed a bug",
"\n\n\n+ Fixed a bug",
"---\n* Fixed a bug\n* Added feature",
] as const;
for (const input of inputs) {
assert.equal(isValidChangenoteContent(input), false);
}
});
await it("does not contain other Markdown elements", async () => {
const inputs = [
"- Fixed a bug\n\nParagraph of text",
"- Fixed a bug\n\n* Added a feature",
"# Header\n- Fixed a bug",
"- Fixed a bug\n## Subheader",
];
for (const input of inputs) {
assert.equal(isValidChangenoteContent(input), false);
}
});
});
await describe("isValidChangenoteFilename", async () => {
await it("accepts valid filenames", async () => {
const inputs = [
"2023-01-01-fix-bug.md",
"2023-12-31-add-feature.md",
"2023-06-15-update-docs.md",
];
for (const input of inputs) {
assert.equal(isValidChangenoteFilename(input), true);
}
});
await it("rejects invalid filenames", async () => {
const inputs = [
"missing-date-from-filename.md",
"2021-01-01.md",
"2026-12-19-wrong-file-name-extension.txt",
];
for (const input of inputs) {
assert.equal(isValidChangenoteFilename(input), false);
}
});
});
await describe("hasValidChangenoteCategory", async () => {
await it("accepts valid categories", async () => {
for (const category of Object.keys(VALID_CHANGE_NOTE_CATEGORIES)) {
const frontmatter = { category };
assert.equal(hasValidChangenoteCategory(frontmatter), true);
}
});
await it("rejects invalid categories", async () => {
const inputs = [
"",
"invalid-category",
"bug-fix",
"new-feature",
"security-patch",
"miscellaneous",
"documentation",
];
for (const category of inputs) {
const frontmatter = { category };
assert.equal(hasValidChangenoteCategory(frontmatter), false);
}
});
await it("reject missing category", async () => {
assert.equal(hasValidChangenoteCategory({}), false);
assert.equal(hasValidChangenoteCategory({ category: null }), false);
assert.equal(hasValidChangenoteCategory({ category: undefined }), false);
});
});
await describe("isValidChangenoteFile", async () => {
await it("accepts a valid change-note file", async () => {
await withTmpFile(
"2026-01-01-fix-bug.md",
"---\ncategory: fix\n---\n- Fixed a bug\n",
async (filePath) => {
assert.equal(isValidChangenoteFile(filePath), true);
},
);
});
await it("rejects invalid filename", async () => {
await withTmpFile(
"fix-bug.md",
"---\ncategory: fix\n---\n- Fixed a bug\n",
async (filePath) => {
assert.equal(isValidChangenoteFile(filePath), false);
},
);
});
await it("rejects missing frontmatter", async () => {
await withTmpFile(
"2026-01-01-fix-bug.md",
"- Fixed a bug\n",
async (filePath) => {
assert.equal(isValidChangenoteFile(filePath), false);
},
);
});
await it("rejects invalid Markdown", async () => {
await withTmpFile(
"2026-01-01-fix-bug.md",
"---\ncategory: fix\n---\n* Fixed a bug\n",
async (filePath) => {
assert.equal(isValidChangenoteFile(filePath), false);
},
);
});
});
+121
View File
@@ -0,0 +1,121 @@
import * as fs from "node:fs";
import * as path from "node:path";
import { matter } from "lite-matter";
import type { List, ListItem } from "mdast";
import { fromMarkdown } from "mdast-util-from-markdown";
// Regex for filename: YYYY-MM-DD-id.md
const VALID_CHANGE_NOTE_FILENAME_PATTERN =
/^(\d{4})-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])-([a-z0-9]+(?:-[a-z0-9]+)*)\.md$/;
export const VALID_CHANGE_NOTE_CATEGORIES = {
breaking: "Breaking Changes",
feature: "New Features",
improvement: "Improvements",
securityFix: "Security Fixes",
fix: "Bug Fixes",
unship: "Removed Features",
deprecation: "Deprecations",
knownIssue: "Known Issues",
misc: "Miscellaneous",
};
/**
* Validates that the given Markdown string meets the criteria for a change-note, which is:
* - A single unordered list
* - Each list item must start with a hyphen (-)
* - No other Markdown elements are allowed
* @param content The Markdown string to validate
* @returns True if the string is a valid change-note, false otherwise
*/
export function isValidChangenoteContent(content: string): boolean {
const ast = fromMarkdown(content);
const lines = content.split("\n");
function listHasHyphenBullets(node: List | ListItem): boolean {
if (node.type === "list") {
return node.children.every(listHasHyphenBullets);
}
const line = lines[node.position!.start.line - 1].trim();
return (
line.startsWith("-") &&
node.children.every(
(child) => child.type !== "list" || listHasHyphenBullets(child),
)
);
}
return (
ast.children.length === 1 &&
ast.children[0].type === "list" &&
ast.children[0].ordered === false &&
listHasHyphenBullets(ast.children[0])
);
}
/**
* Validates that the given filename meets the criteria for a change-note filename.
* @param filename The name of the change-note file to validate.
* @returns True if the filename is valid, false otherwise.
*/
export function isValidChangenoteFilename(filename: string): boolean {
return filename.match(VALID_CHANGE_NOTE_FILENAME_PATTERN) !== null;
}
/**
* Validates that the given frontmatter has a valid change-note category.
* @param frontmatter The frontmatter object to validate.
* @returns True if the frontmatter has a valid category, false otherwise.
*/
export function hasValidChangenoteCategory(
frontmatter: Record<string, unknown>,
): boolean {
const category = frontmatter["category"];
return (
typeof category === "string" &&
Object.hasOwn(VALID_CHANGE_NOTE_CATEGORIES, category)
);
}
/**
* Validates that the given change-note file meets all of the criteria for a change-note.
* @param filename The name of the change-note file to validate.
* @returns True if the file is a valid change-note, false otherwise.
*/
export function isValidChangenoteFile(filename: string): boolean {
let isValid: boolean = true;
let fileData: string | undefined;
try {
fileData = fs.readFileSync(filename, "utf8");
} catch (error) {
console.error(`${filename}: failed to read file`, error);
return false;
}
const { data: frontmatter, content } = matter(fileData);
if (!isValidChangenoteFilename(path.basename(filename))) {
isValid = false;
console.error(
`${filename}: invalid filename; must match pattern YYYY-MM-DD-id.md`,
);
}
if (!hasValidChangenoteCategory(frontmatter)) {
isValid = false;
const categories = Object.keys(VALID_CHANGE_NOTE_CATEGORIES).join(", ");
console.error(
`${filename}: invalid category; must be one of: ${categories}`,
);
}
if (!isValidChangenoteContent(content)) {
isValid = false;
console.error(
`${filename}: invalid Markdown; content must be a single unordered list with hyphen bullets and no other Markdown elements`,
);
}
return isValid;
}
+51
View File
@@ -0,0 +1,51 @@
import { pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { isValidChangenoteFile } from "./cli/validate.ts";
const entryPoint = process.argv[1];
if (entryPoint && import.meta.url === pathToFileURL(entryPoint).href) {
try {
process.exit(main());
} catch (error) {
console.error(error);
process.exit(1);
}
}
function main(): number {
const { positionals } = parseArgs({
allowPositionals: true,
strict: true,
});
const [command, ...paths] = positionals;
switch (command) {
case undefined:
case "help":
return usage();
case "validate":
return validate(paths);
default:
console.error(`Unknown command: ${command}`);
return 1;
}
}
function usage(): number {
console.log("Usage: changetool validate <path> [<path> ...]");
return 0;
}
function validate(paths: string[]): number {
let valid = true;
if (paths.length === 0) {
console.error("error: no paths provided (see 'help' command for usage)");
return 1;
}
for (const path of paths) {
if (!isValidChangenoteFile(path)) {
valid = false;
}
}
return valid ? 0 : 1;
}
+21
View File
@@ -0,0 +1,21 @@
{
"name": "changetool",
"version": "1.0.0",
"private": true,
"description": "Validates change-notes and merges them into CHANGELOG.md",
"license": "MIT",
"type": "module",
"scripts": {
"start": "tsx index.ts",
"test": "node --test --experimental-strip-types cli/*.test.ts"
},
"devDependencies": {
"@types/node": "^26.2.0",
"tsx": "^4.23.12",
"typescript": "^7.0.2"
},
"dependencies": {
"lite-matter": "^0.1.2",
"mdast-util-from-markdown": "^2.0.3"
}
}
+11
View File
@@ -0,0 +1,11 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"module": "preserve",
"allowImportingTsExtensions": true,
"rootDir": ".",
"sourceMap": false
},
"include": ["./**/*.ts"],
"exclude": ["node_modules"]
}
+1 -1
View File
@@ -54,5 +54,5 @@ outputs:
codeql-version:
description: The version of the CodeQL binary that was installed.
runs:
using: node20
using: node24
main: '../lib/setup-codeql-entry.js'
+13
View File
@@ -283,6 +283,19 @@ export function isSelfHostedRunner(env: Env = getEnv()) {
return env.getOptional(ActionsEnvVars.RUNNER_ENVIRONMENT) === "self-hosted";
}
/**
* Whether the job is running on a runner that GitHub hosts, and whose toolcache is therefore thrown
* away once the job has finished.
*
* Unlike `looksLikeHostedRunner`, this is based on what the service reports for the job rather than
* on how the runner's filesystem happens to be laid out, so it does not match self-hosted runners
* that are configured to resemble hosted ones, such as those that mount a persistent volume at
* `/opt/hostedtoolcache`.
*/
export function isGitHubHostedRunner(env: Env = getEnv()) {
return env.getOptional(ActionsEnvVars.RUNNER_ENVIRONMENT) === "github-hosted";
}
/** Determines whether the workflow trigger is `dynamic`. */
export function isDynamicWorkflow(env: Env = getEnv()): boolean {
return getWorkflowEventName(env) === "dynamic";
+1 -4
View File
@@ -1,8 +1,5 @@
import * as core from "@actions/core";
import * as githubUtils from "@actions/github/lib/utils";
import { type Octokit } from "@octokit/core";
import { type PaginateInterface } from "@octokit/plugin-paginate-rest";
import { type Api } from "@octokit/plugin-rest-endpoint-methods";
import * as retry from "@octokit/plugin-retry";
import { RequestRequestOptions } from "@octokit/types";
import {
@@ -128,7 +125,7 @@ export function makeProxyRequestOptions(
}
/** The type of GitHub API client we use. */
export type ApiClient = Octokit & Api & { paginate: PaginateInterface };
export type ApiClient = InstanceType<typeof githubUtils.GitHub>;
/** Options for `createApiClientWithDetails`. */
interface CreateApiClientOptions {
+1 -1
View File
@@ -1 +1 @@
{"maximumVersion": "3.22", "minimumVersion": "3.17"}
{"maximumVersion":"3.23","minimumVersion":"3.17"}
+2 -2
View File
@@ -5,7 +5,7 @@ import * as core from "@actions/core";
import { getOptionalInput, isDefaultSetup } from "./actions-util";
import { EnvVar } from "./environment";
import { Logger } from "./logging";
import { isHostedRunner, tryGetFolderBytes } from "./util";
import { looksLikeHostedRunner, tryGetFolderBytes } from "./util";
/**
* Returns the total size of all the specified paths.
@@ -109,7 +109,7 @@ export function getDependencyCachingEnabled(): CachingKind {
if (dependencyCaching !== undefined) return getCachingKind(dependencyCaching);
// On self-hosted runners which may have dependencies installed centrally, disable caching by default
if (!isHostedRunner()) return CachingKind.None;
if (!looksLikeHostedRunner()) return CachingKind.None;
// Disable in advanced workflows by default.
if (!isDefaultSetup()) return CachingKind.None;
+27 -14
View File
@@ -128,7 +128,6 @@ test("CliError constructor with empty stderr", (t) => {
for (const [platform, arch] of [
["weird_plat", "x64"],
["linux", "arm64"],
["win32", "arm64"],
]) {
test.serial(
@@ -157,20 +156,34 @@ for (const [platform, arch] of [
);
}
test("wrapCliConfigurationError - supported platform", (t) => {
const commandError = new CommandInvocationError(
"codeql",
["version"],
1,
"Some error",
for (const [platform, arch] of [
["linux", "x64"],
["linux", "arm64"],
["win32", "x64"],
["darwin", "x64"],
["darwin", "arm64"],
]) {
test.serial(
`wrapCliConfigurationError - ${platform}/${arch} supported`,
(t) => {
sinon.stub(process, "platform").value(platform);
sinon.stub(process, "arch").value(arch);
const commandError = new CommandInvocationError(
"codeql",
["version"],
1,
"Some error",
);
const cliError = new CliError(commandError);
const wrappedError = wrapCliConfigurationError(cliError);
// Should return the original error since the platform is supported, rather
// than replacing it with the unsupported-platform ConfigurationError.
t.is(wrappedError, cliError);
},
);
const cliError = new CliError(commandError);
const wrappedError = wrapCliConfigurationError(cliError);
// Should return the original error since platform is supported
t.is(wrappedError, cliError);
});
}
test("wrapCliConfigurationError - autobuild error", (t) => {
const commandError = new CommandInvocationError(
+1
View File
@@ -8,6 +8,7 @@ import { ConfigurationError } from "./util";
const SUPPORTED_PLATFORMS = [
["linux", "x64"],
["linux", "arm64"],
["win32", "x64"],
["darwin", "x64"],
["darwin", "arm64"],
+123
View File
@@ -0,0 +1,123 @@
import * as fs from "fs";
import path from "path";
import test from "ava";
import { getRunnerLogger } from "../logging";
import { setupTests } from "../testing-utils";
import * as util from "../util";
import { getCachedCodeQlVersion } from "./output-cache";
setupTests(test);
const logger = getRunnerLogger(true);
test.serial(
"getCachedCodeQlVersion reuses a version persisted by an earlier step",
async (t) => {
await util.withTmpDir(async (tmpDir: string) => {
const cacheFilePath = path.join(tmpDir, "cache.json");
fs.writeFileSync(
cacheFilePath,
JSON.stringify({
cmd: "/path/to/codeql",
entries: { version: { version: "2.20.0" } },
}),
"utf8",
);
t.deepEqual(
getCachedCodeQlVersion(logger, cacheFilePath, "/path/to/codeql"),
{
version: "2.20.0",
},
);
});
},
);
test.serial(
"getCachedCodeQlVersion ignores a persisted version from a different CLI",
async (t) => {
await util.withTmpDir(async (tmpDir: string) => {
const cacheFilePath = path.join(tmpDir, "cache.json");
fs.writeFileSync(
cacheFilePath,
JSON.stringify({
cmd: "/path/to/other-codeql",
entries: { version: { version: "2.20.0" } },
}),
"utf8",
);
t.is(
getCachedCodeQlVersion(logger, cacheFilePath, "/path/to/codeql"),
undefined,
);
});
},
);
test.serial(
"getCachedCodeQlVersion ignores a malformed persisted value",
async (t) => {
await util.withTmpDir(async (tmpDir: string) => {
const cacheFilePath = path.join(tmpDir, "cache.json");
fs.writeFileSync(cacheFilePath, "not valid json", "utf8");
t.is(
getCachedCodeQlVersion(logger, cacheFilePath, "/path/to/codeql"),
undefined,
);
});
},
);
test.serial(
"getCachedCodeQlVersion ignores a persisted value with the wrong structure",
async (t) => {
await util.withTmpDir(async (tmpDir: string) => {
const cacheFilePath = path.join(tmpDir, "cache.json");
const testValues = [
{ cmd: "/path/to/codeql" },
{ entries: { version: { version: "2.20.0" } } },
{ cmd: "/path/to/codeql", entries: {} },
{ cmd: "/path/to/codeql", entries: null },
{ cmd: "/path/to/codeql", entries: { version: {} } },
{ cmd: "/path/to/codeql", entries: { version: null } },
{ cmd: "/path/to/codeql", entries: { version: "2.20.0" } },
{ cmd: "/path/to/codeql", entries: { version: { version: null } } },
{ cmd: "/path/to/codeql", entries: { version: { version: 2.2 } } },
{ cmd: "/path/to/codeql", entries: { version: { version: 2 } } },
{
cmd: "/path/to/codeql",
entries: { version: { version: "2.20.0", overlayVersion: "1" } },
},
{
cmd: "/path/to/codeql",
entries: { version: { version: "2.20.0", features: "nope" } },
},
].map((v) => JSON.stringify(v));
for (const value of testValues) {
fs.writeFileSync(cacheFilePath, value, "utf8");
t.is(
getCachedCodeQlVersion(logger, cacheFilePath, "/path/to/codeql"),
undefined,
value,
);
}
});
},
);
test.serial("getCachedCodeQlVersion ignores non-existent file", async (t) => {
await util.withTmpDir(async (tmpDir: string) => {
const cacheFilePath = path.join(tmpDir, "cache.json");
t.notThrows(() => {
t.is(
getCachedCodeQlVersion(logger, cacheFilePath, "/path/to/codeql"),
undefined,
);
});
});
});
+146
View File
@@ -0,0 +1,146 @@
import * as fs from "fs";
import path from "path";
import { getTemporaryDirectory } from "../actions-util";
import { Env } from "../environment";
import * as json from "../json";
import { Logger } from "../logging";
import { VersionInfo, versionInfoBaseSchema } from "./types";
/**
* The keys of the command cache. Each key corresponds to a command whose output we cache.
*/
export type CommandCacheKey = string;
/**
* The JSON schema of the command cache that is persisted to disk.
*/
const outputCacheSchema = {
cmd: json.string,
entries: json.object({}),
} as const satisfies json.Schema;
/**
* The type that describes the command cache that is persisted to disk.
*/
export type OutputCache = json.FromSchema<typeof outputCacheSchema> & {
entries: { version: VersionInfo };
};
/**
* The name of the temporary file that backs the on-disk cache of
* CLI responses between workflow steps.
*/
const COMMAND_CACHE_FILENAME = "codeql-action-command-cache.json";
/**
* The module-global variable that caches the CodeQL CLI version in-memory.
*/
let cachedCodeQlVersion: undefined | VersionInfo = undefined;
/**
* Resets the in-process cache of the CodeQL CLI version. Only for use in tests,
* which exercise multiple "steps" within a single process.
*/
export function resetCachedCodeQlVersion(): void {
cachedCodeQlVersion = undefined;
}
/**
* Returns the path to the temporary file that backs the
* on-disk cache of CLI responses between workflow steps.
*/
export function getCommandCacheFilePath(env: Env): string {
return path.join(getTemporaryDirectory(env), COMMAND_CACHE_FILENAME);
}
/**
* Caches the CodeQL CLI version both in-memory and on disk.
* @param cacheFilePath The path to the cache file.
* @param cmd The path to the CodeQL CLI.
* @param version The version information to cache.
*/
export function cacheCodeQlVersion(
cacheFilePath: string,
cmd: string,
version: VersionInfo,
): void {
if (cachedCodeQlVersion !== undefined) {
throw new Error("cacheCodeQlVersion() should be called only once");
}
cachedCodeQlVersion = version;
const outputCache = {
cmd,
entries: { version },
} satisfies OutputCache;
// Persist the version so that subsequent Actions steps, which run in separate
// processes, can reuse it rather than invoking `codeql version` again. We
// record the CLI path so that a different step using a different CodeQL bundle
// doesn't pick up a stale version.
fs.writeFileSync(cacheFilePath, JSON.stringify(outputCache), "utf8");
}
/**
* Returns the cached CodeQL CLI version, if any.
* @param logger The logger to use for logging messages.
* @param cacheFilePath The path to the cache file.
* @param cmd The path to the CodeQL CLI.
*/
export function getCachedCodeQlVersion(
logger: Logger,
cacheFilePath: string,
cmd?: string,
): undefined | VersionInfo {
if (cachedCodeQlVersion !== undefined) {
return cachedCodeQlVersion;
}
// Fall back to the value persisted by an earlier Actions step, if any. This is
// best-effort: any malformed or mismatched value is ignored so that the caller
// invokes `codeql version` instead.
let serialized: string;
try {
serialized = fs.readFileSync(cacheFilePath, "utf8");
} catch (e) {
logger.debug(`Cannot read CLI-cache file ${cacheFilePath}: ${e}`);
return undefined;
}
let persisted: unknown;
try {
persisted = JSON.parse(serialized);
} catch (e) {
logger.debug(`Cannot parse CLI-cache data as JSON: ${e}`);
return undefined;
}
if (
!isOutputCache(persisted) ||
(cmd !== undefined && persisted.cmd !== cmd)
) {
return undefined;
}
// Memoize the parsed value so that subsequent calls in this process don't
// re-parse the environment variable.
cachedCodeQlVersion = persisted.entries.version as VersionInfo;
return cachedCodeQlVersion;
}
/**
* Determines whether a value is a `VersionInfo` object.
* @param x The value to test
*/
function isVersionInfo(x: unknown): x is VersionInfo {
return json.isObject(x) && json.validateSchema(versionInfoBaseSchema, x);
}
/**
* Determines whether a value is a `OutputCache` object.
* @param x The value to test
*/
function isOutputCache(x: unknown): x is OutputCache {
return (
json.isObject(x) &&
json.validateSchema(outputCacheSchema, x) &&
json.isObject<{ version: unknown }>(x.entries) &&
isVersionInfo(x.entries.version)
);
}
+30
View File
@@ -0,0 +1,30 @@
import * as json from "../json";
/**
* The JSON schema of the expected output of the `codeql version` command.
*/
export const versionInfoBaseSchema = {
version: json.string,
features: json.optional(json.object({})),
/**
* The overlay version helps deal with backward incompatible changes for
* overlay analysis. When a precompiled query pack reports the same overlay
* version as the CodeQL CLI, we can use the CodeQL CLI to perform overlay
* analysis with that pack. Otherwise, if the overlay versions are different,
* or if either the pack or the CLI does not report an overlay version,
* we need to revert to non-overlay analysis.
*/
overlayVersion: json.optional(json.number),
} as const satisfies json.Schema;
/**
* The base type that describes the expected output of the `codeql version` command.
*/
export type VersionInfoBase = json.FromSchema<typeof versionInfoBaseSchema>;
/**
* The full type that describes the expected output of the `codeql version` command.
*/
export type VersionInfo = Omit<VersionInfoBase, "features"> & {
features?: { [name: string]: boolean };
};
+25
View File
@@ -51,6 +51,31 @@ test.beforeEach(() => {
});
});
test("isDiskConfigurationError - true for expected errors", async (t) => {
t.true(
codeql.isDiskConfigurationError(new Error("ENOSPC: Out of disk space")),
);
t.true(
codeql.isDiskConfigurationError(
new Error(
"EACCES: permission denied, mkdir /opt/hostedtoolcache/CodeQL/",
),
),
);
});
test("isDiskConfigurationError - false for other errors", async (t) => {
t.false(codeql.isDiskConfigurationError("Not an Error instance"));
const otherMessages = [
"Does not contain an error code we test for",
"ENOSP: Not quite the full error code",
];
for (const otherMessage of otherMessages) {
t.false(codeql.isDiskConfigurationError(new Error(otherMessage)));
}
});
async function installIntoToolcache({
apiDetails = SAMPLE_DOTCOM_API_DETAILS,
cliVersion,
+31 -19
View File
@@ -12,10 +12,12 @@ import {
runTool,
} from "./actions-util";
import * as api from "./api-client";
import * as outputCache from "./cli/output-cache";
import type { VersionInfo } from "./cli/types";
import { CliError, wrapCliConfigurationError } from "./cli-errors";
import { appendExtraQueryExclusions, type Config } from "./config-utils";
import { DocUrl } from "./doc-url";
import { EnvVar } from "./environment";
import { EnvVar, getEnv } from "./environment";
import {
CodeQLDefaultVersionInfo,
Feature,
@@ -214,20 +216,6 @@ export interface CodeQL {
): Promise<void>;
}
export interface VersionInfo {
version: string;
features?: { [name: string]: boolean };
/**
* The overlay version helps deal with backward incompatible changes for
* overlay analysis. When a precompiled query pack reports the same overlay
* version as the CodeQL CLI, we can use the CodeQL CLI to perform overlay
* analysis with that pack. Otherwise, if the overlay versions are different,
* or if either the pack or the CLI does not report an overlay version,
* we need to revert to non-overlay analysis.
*/
overlayVersion?: number;
}
export interface ResolveDatabaseOutput {
overlayBaseSpecifier?: string;
}
@@ -285,6 +273,26 @@ const GHES_MOST_RECENT_DEPRECATION_DATE = "2026-07-01";
/** The CLI verbosity level to use for extraction in debug mode. */
const EXTRACTION_DEBUG_MODE_VERBOSITY = "progress++";
/**
* Decides whether `e` is a disk-related error outside of our control
* that should be classified as a `ConfigurationError`.
*
* @param e The error to check.
* @returns True if the error should be treated as a `ConfigurationError` or false if not.
*/
export function isDiskConfigurationError(e: unknown): boolean {
if (!(e instanceof Error)) {
return false;
}
return (
// out of disk space
e.message.includes("ENOSPC") ||
// access denied
e.message.includes("EACCES")
);
}
/**
* Set up CodeQL CLI access.
*
@@ -355,8 +363,7 @@ export async function setupCodeQL(
} catch (rawError) {
const e = api.wrapApiConfigurationError(rawError);
const ErrorClass =
e instanceof util.ConfigurationError ||
(e instanceof Error && e.message.includes("ENOSPC")) // out of disk space
e instanceof util.ConfigurationError || isDiskConfigurationError(e)
? util.ConfigurationError
: Error;
@@ -503,7 +510,12 @@ async function getCodeQLForCmd(
return cmd;
},
async getVersion() {
let result = util.getCachedCodeQlVersion(cmd);
const cacheFilePath = outputCache.getCommandCacheFilePath(getEnv());
let result = outputCache.getCachedCodeQlVersion(
logger,
cacheFilePath,
cmd,
);
if (result === undefined) {
result = await runCliJson<VersionInfo>(
cmd,
@@ -512,7 +524,7 @@ async function getCodeQLForCmd(
noStreamStdout: true,
},
);
util.cacheCodeQlVersion(cmd, result);
outputCache.cacheCodeQlVersion(cacheFilePath, cmd, result);
}
return result;
},
+95
View File
@@ -1333,6 +1333,101 @@ checkOverlayEnablementMacro.serial(
},
);
// Check that each feature flag lowers the limit to the threshold that its name
// declares. Both sides of the boundary are needed to pin the threshold down: a
// mapping to a lower value would still pass the case at the limit, and one to a
// higher value would still fail the case below it.
for (const [feature, thresholdGb] of [
[Feature.OverlayAnalysisMinDisk8Gb, 8],
[Feature.OverlayAnalysisMinDisk9Gb, 9],
[Feature.OverlayAnalysisMinDisk10Gb, 10],
[Feature.OverlayAnalysisMinDisk11Gb, 11],
[Feature.OverlayAnalysisMinDisk12Gb, 12],
[Feature.OverlayAnalysisMinDisk13Gb, 13],
] as Array<[Feature, number]>) {
const features = [
Feature.OverlayAnalysis,
Feature.OverlayAnalysisCodeScanningJavascript,
feature,
];
checkOverlayEnablementMacro.serial(
`Overlay-base database on default branch if ${feature} is enabled and runner disk space is at its limit`,
{
languages: [BuiltInLanguage.javascript],
features,
isDefaultBranch: true,
diskUsage: {
numAvailableBytes: thresholdGb * 1_000_000_000,
numTotalBytes: 100_000_000_000,
},
},
{
overlayDatabaseMode: OverlayDatabaseMode.OverlayBase,
useOverlayDatabaseCaching: true,
},
);
checkOverlayEnablementMacro.serial(
`No overlay-base database on default branch if ${feature} is enabled and runner disk space is below its limit`,
{
languages: [BuiltInLanguage.javascript],
features,
isDefaultBranch: true,
diskUsage: {
numAvailableBytes: thresholdGb * 1_000_000_000 - 1_000_000,
numTotalBytes: 100_000_000_000,
},
},
{
disabledReason: OverlayDisabledReason.InsufficientDiskSpace,
},
);
}
checkOverlayEnablementMacro.serial(
"Overlay-base database on default branch if runner disk space is exactly at the lowest limit enabled by a feature flag",
{
languages: [BuiltInLanguage.javascript],
features: [
Feature.OverlayAnalysis,
Feature.OverlayAnalysisCodeScanningJavascript,
Feature.OverlayAnalysisMinDisk9Gb,
Feature.OverlayAnalysisMinDisk12Gb,
],
isDefaultBranch: true,
diskUsage: {
numAvailableBytes: 9_000_000_000,
numTotalBytes: 100_000_000_000,
},
},
{
overlayDatabaseMode: OverlayDatabaseMode.OverlayBase,
useOverlayDatabaseCaching: true,
},
);
checkOverlayEnablementMacro.serial(
"No overlay-base database on default branch if runner disk space is below the lowest limit enabled by a feature flag",
{
languages: [BuiltInLanguage.javascript],
features: [
Feature.OverlayAnalysis,
Feature.OverlayAnalysisCodeScanningJavascript,
Feature.OverlayAnalysisMinDisk9Gb,
Feature.OverlayAnalysisMinDisk12Gb,
],
isDefaultBranch: true,
diskUsage: {
numAvailableBytes: 8_500_000_000,
numTotalBytes: 100_000_000_000,
},
},
{
disabledReason: OverlayDisabledReason.InsufficientDiskSpace,
},
);
checkOverlayEnablementMacro.serial(
"No overlay-base database on default branch if memory flag is too low",
{
+56 -13
View File
@@ -48,7 +48,7 @@ import {
import { prepareDiffInformedAnalysis } from "./diff-informed-analysis-utils";
import { EnvVar } from "./environment";
import * as errorMessages from "./error-messages";
import { Feature, FeatureEnablement } from "./feature-flags";
import { Feature, FeatureEnablement, FeatureWithoutCLI } from "./feature-flags";
import {
RepositoryProperties,
RepositoryPropertyName,
@@ -90,9 +90,8 @@ import {
Result,
Success,
Failure,
isHostedRunner,
looksLikeHostedRunner,
} from "./util";
export { type Config } from "./config/action-config";
/**
@@ -101,10 +100,23 @@ export { type Config } from "./config/action-config";
* whether to perform overlay analysis, then the action will not perform overlay
* analysis unless overlay analysis has been explicitly enabled via environment
* variable.
*
* This threshold can be lowered by the feature flags in
* `OVERLAY_MINIMUM_DISK_SPACE_MB_BY_FEATURE`.
*/
const OVERLAY_MINIMUM_AVAILABLE_DISK_SPACE_MB = 14000;
const OVERLAY_MINIMUM_AVAILABLE_DISK_SPACE_BYTES =
OVERLAY_MINIMUM_AVAILABLE_DISK_SPACE_MB * 1_000_000;
/**
* Minimum available disk space (in MB) enabled by each overlay feature flag.
*/
const OVERLAY_MINIMUM_DISK_SPACE_MB_BY_FEATURE = {
[Feature.OverlayAnalysisMinDisk8Gb]: 8000,
[Feature.OverlayAnalysisMinDisk9Gb]: 9000,
[Feature.OverlayAnalysisMinDisk10Gb]: 10000,
[Feature.OverlayAnalysisMinDisk11Gb]: 11000,
[Feature.OverlayAnalysisMinDisk12Gb]: 12000,
[Feature.OverlayAnalysisMinDisk13Gb]: 13000,
} satisfies Partial<Record<FeatureWithoutCLI, number>>;
/**
* The minimum memory (in MB) that must be available for CodeQL to perform overlay analysis. If
@@ -579,21 +591,44 @@ async function checkOverlayAnalysisFeatureEnabled(
return new Success(undefined);
}
/**
* Returns the minimum available disk space (in MB) required to perform overlay
* analysis, which is the lowest threshold enabled by a feature flag, or the
* default threshold if no such feature flag is enabled.
*/
async function getMinimumDiskSpaceMb(
features: FeatureEnablement,
): Promise<number> {
let minimumMb = OVERLAY_MINIMUM_AVAILABLE_DISK_SPACE_MB;
for (const [feature, thresholdMb] of Object.entries(
OVERLAY_MINIMUM_DISK_SPACE_MB_BY_FEATURE,
)) {
if (await features.getValue(feature as FeatureWithoutCLI)) {
minimumMb = Math.min(minimumMb, thresholdMb);
}
}
return minimumMb;
}
/** Checks if the runner has enough disk space for overlay analysis. */
function runnerHasSufficientDiskSpace(
diskUsage: DiskUsage,
logger: Logger,
minimumDiskSpaceMb: number,
): boolean {
const minimumDiskSpaceBytes = OVERLAY_MINIMUM_AVAILABLE_DISK_SPACE_BYTES;
if (diskUsage.numAvailableBytes < minimumDiskSpaceBytes) {
const diskSpaceMb = Math.round(diskUsage.numAvailableBytes / 1_000_000);
const minimumDiskSpaceMb = Math.round(minimumDiskSpaceBytes / 1_000_000);
const diskSpaceMb = Math.round(diskUsage.numAvailableBytes / 1_000_000);
if (diskUsage.numAvailableBytes < minimumDiskSpaceMb * 1_000_000) {
logger.info(
`Setting overlay database mode to ${OverlayDatabaseMode.None} ` +
`due to insufficient disk space (${diskSpaceMb} MB, needed ${minimumDiskSpaceMb} MB).`,
);
return false;
}
logger.debug(
`Disk space available for CodeQL analysis is ${diskSpaceMb} MB, which is at or above the ` +
`minimum of ${minimumDiskSpaceMb} MB.`,
);
return true;
}
@@ -625,7 +660,7 @@ async function runnerHasSufficientMemory(
}
logger.debug(
`Memory available for CodeQL analysis is ${memoryFlagValue} MB, which is above the minimum of ${OVERLAY_MINIMUM_MEMORY_MB} MB.`,
`Memory available for CodeQL analysis is ${memoryFlagValue} MB, which is at or above the minimum of ${OVERLAY_MINIMUM_MEMORY_MB} MB.`,
);
return true;
}
@@ -636,11 +671,13 @@ async function runnerHasSufficientMemory(
*/
async function checkRunnerResources(
codeql: CodeQL,
features: FeatureEnablement,
diskUsage: DiskUsage,
ramInput: string | undefined,
logger: Logger,
): Promise<Result<void, OverlayDisabledReason>> {
if (!runnerHasSufficientDiskSpace(diskUsage, logger)) {
const minimumDiskSpaceMb = await getMinimumDiskSpaceMb(features);
if (!runnerHasSufficientDiskSpace(diskUsage, logger, minimumDiskSpaceMb)) {
return new Failure(OverlayDisabledReason.InsufficientDiskSpace);
}
if (!(await runnerHasSufficientMemory(codeql, ramInput, logger))) {
@@ -752,7 +789,13 @@ export async function checkOverlayEnablement(
}
const resourceResult =
performResourceChecks && diskUsage !== undefined
? await checkRunnerResources(codeql, diskUsage, ramInput, logger)
? await checkRunnerResources(
codeql,
features,
diskUsage,
ramInput,
logger,
)
: new Success<void>(undefined);
if (resourceResult.isFailure()) {
return resourceResult;
@@ -894,7 +937,7 @@ export async function isTrapCachingEnabled(
if (trapCaching !== undefined) return trapCaching === "true";
// On self-hosted runners which may have slow network access, disable TRAP caching by default.
if (!isHostedRunner()) return false;
if (!looksLikeHostedRunner()) return false;
// If overlay analysis is enabled, then disable TRAP caching since overlay analysis supersedes it.
// This change is gated behind a feature flag.
+4 -4
View File
@@ -1,6 +1,6 @@
{
"bundleVersion": "codeql-bundle-v2.26.3",
"cliVersion": "2.26.3",
"priorBundleVersion": "codeql-bundle-v2.26.2",
"priorCliVersion": "2.26.2"
"bundleVersion": "codeql-bundle-v2.26.4",
"cliVersion": "2.26.4",
"priorBundleVersion": "codeql-bundle-v2.26.3",
"priorCliVersion": "2.26.3"
}
+6 -6
View File
@@ -39,12 +39,6 @@ export enum EnvVar {
*/
CODE_SCANNING_REF = "CODE_SCANNING_REF",
/**
* `PersistedVersionInfo` for the CodeQL CLI, so later Actions steps can reuse it instead of
* invoking `codeql version` again.
*/
CODEQL_VERSION_INFO = "CODEQL_ACTION_CLI_VERSION_INFO",
/** Whether the CodeQL Action has invoked the Go autobuilder. */
DID_AUTOBUILD_GOLANG = "CODEQL_ACTION_DID_AUTOBUILD_GOLANG",
@@ -69,6 +63,12 @@ export enum EnvVar {
/** Whether the CodeQL Action has already warned the user about low disk space. */
HAS_WARNED_ABOUT_DISK_SPACE = "CODEQL_ACTION_HAS_WARNED_ABOUT_DISK_SPACE",
/**
* Whether a step in this job has already set up CodeQL. Steps that run afterwards may be holding
* a path into the toolcache, so we must not delete anything from it.
*/
HAS_SET_UP_CODEQL = "CODEQL_ACTION_HAS_SET_UP_CODEQL",
/** Whether the `setup-codeql` action has been run. */
SETUP_CODEQL_ACTION_HAS_RUN = "CODEQL_ACTION_SETUP_CODEQL_HAS_RUN",
+70
View File
@@ -74,6 +74,11 @@ export enum Feature {
AllowMergeConfigFiles = "allow_merge_config_files",
/** Controls whether we allow multiple values for the `analysis-kinds` input. */
AllowMultipleAnalysisKinds = "allow_multiple_analysis_kinds",
/**
* Controls whether we delete CodeQL bundles that we are not going to use from the toolcache
* before downloading a different bundle, in order to reclaim disk space.
*/
CleanupToolcacheBundles = "cleanup_toolcache_bundles",
CleanupTrapCaches = "cleanup_trap_caches",
/** Whether to allow the `config-file` input to be specified via a repository property. */
ConfigFileRepositoryProperty = "config_file_repository_property",
@@ -121,6 +126,36 @@ export enum Feature {
* `OverlayAnalysisMatchCodeqlVersion` overrides this flag.
*/
OverlayAnalysisMatchCodeqlVersionDryRun = "overlay_analysis_match_codeql_version_dry_run",
/**
* Lowers the overlay minimum available disk space to 8 GB. The lowest enabled limit wins; if
* none are enabled, the default applies.
*/
OverlayAnalysisMinDisk8Gb = "overlay_analysis_min_disk_8_gb",
/**
* Lowers the overlay minimum available disk space to 9 GB. The lowest enabled limit wins; if
* none are enabled, the default applies.
*/
OverlayAnalysisMinDisk9Gb = "overlay_analysis_min_disk_9_gb",
/**
* Lowers the overlay minimum available disk space to 10 GB. The lowest enabled limit wins; if
* none are enabled, the default applies.
*/
OverlayAnalysisMinDisk10Gb = "overlay_analysis_min_disk_10_gb",
/**
* Lowers the overlay minimum available disk space to 11 GB. The lowest enabled limit wins; if
* none are enabled, the default applies.
*/
OverlayAnalysisMinDisk11Gb = "overlay_analysis_min_disk_11_gb",
/**
* Lowers the overlay minimum available disk space to 12 GB. The lowest enabled limit wins; if
* none are enabled, the default applies.
*/
OverlayAnalysisMinDisk12Gb = "overlay_analysis_min_disk_12_gb",
/**
* Lowers the overlay minimum available disk space to 13 GB. The lowest enabled limit wins; if
* none are enabled, the default applies.
*/
OverlayAnalysisMinDisk13Gb = "overlay_analysis_min_disk_13_gb",
OverlayAnalysisPython = "overlay_analysis_python",
OverlayAnalysisRuby = "overlay_analysis_ruby",
/** Controls whether hardware checks are skipped for overlay analysis. */
@@ -181,6 +216,11 @@ export const featureConfig = {
envVar: "CODEQL_ACTION_ALLOW_MULTIPLE_ANALYSIS_KINDS",
minimumVersion: undefined,
},
[Feature.CleanupToolcacheBundles]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES",
minimumVersion: undefined,
},
[Feature.CleanupTrapCaches]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CLEANUP_TRAP_CACHES",
@@ -349,6 +389,36 @@ export const featureConfig = {
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MATCH_CODEQL_VERSION_DRY_RUN",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisMinDisk8Gb]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MIN_DISK_8_GB",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisMinDisk9Gb]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MIN_DISK_9_GB",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisMinDisk10Gb]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MIN_DISK_10_GB",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisMinDisk11Gb]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MIN_DISK_11_GB",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisMinDisk12Gb]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MIN_DISK_12_GB",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisMinDisk13Gb]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MIN_DISK_13_GB",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisStatusCheck]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_STATUS_CHECK",
+151 -1
View File
@@ -15,10 +15,12 @@ import { getRunnerLogger } from "./logging";
import { OverlayDatabaseMode } from "./overlay/overlay-database-mode";
import * as overlayStatus from "./overlay/status";
import { parseRepositoryNwo } from "./repository";
import { JobStatus } from "./status-report";
import {
createFeatures,
createTestConfig,
DEFAULT_ACTIONS_VARS,
getTestEnv,
makeMacro,
makeVersionInfo,
RecordingLogger,
@@ -58,6 +60,8 @@ test.serial("init-post action with debug mode off", async (t) => {
createTestConfig({ debugMode: false }),
parseRepositoryNwo("github/codeql-action"),
createFeatures([]),
"success",
getTestEnv(),
getRunnerLogger(true),
);
@@ -80,6 +84,8 @@ test.serial("init-post action with debug mode on", async (t) => {
createTestConfig({ debugMode: true }),
parseRepositoryNwo("github/codeql-action"),
createFeatures([]),
"success",
getTestEnv(),
getRunnerLogger(true),
);
@@ -375,6 +381,8 @@ test.serial(
}),
parseRepositoryNwo("github/codeql-action"),
createFeatures([Feature.OverlayAnalysisStatusSave]),
"success",
getTestEnv(),
getRunnerLogger(true),
);
@@ -443,6 +451,8 @@ test.serial(
}),
parseRepositoryNwo("github/codeql-action"),
createFeatures([]),
"success",
getTestEnv(),
getRunnerLogger(true),
);
@@ -457,8 +467,13 @@ test.serial(
test.serial("does not save overlay status when build successful", async (t) => {
return await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
// Mark analyze as having completed successfully.
// Mark analyze as having completed successfully. `tryUploadSarifIfRunFailed` reads this from
// the process environment, while `recordOverlayStatus` reads it from the environment it is
// given.
process.env[EnvVar.ANALYZE_DID_COMPLETE_SUCCESSFULLY] = "true";
const env = getTestEnv({
[EnvVar.ANALYZE_DID_COMPLETE_SUCCESSFULLY]: "true",
});
sinon.stub(util, "checkDiskUsage").resolves({
numAvailableBytes: 100 * NUM_BYTES_PER_GIB,
@@ -480,6 +495,8 @@ test.serial("does not save overlay status when build successful", async (t) => {
}),
parseRepositoryNwo("github/codeql-action"),
createFeatures([Feature.OverlayAnalysisStatusSave]),
"success",
env,
getRunnerLogger(true),
);
@@ -517,6 +534,8 @@ test.serial(
}),
parseRepositoryNwo("github/codeql-action"),
createFeatures([]),
"success",
getTestEnv(),
getRunnerLogger(true),
);
@@ -528,6 +547,137 @@ test.serial(
},
);
/**
* Runs `uploadFailureInfo` for an overlay-base job that did not complete successfully, with the
* given job status from the Actions runtime environment.
*/
async function runOverlayPostStep({
jobStatus,
codeQlReportedError = false,
}: {
jobStatus: string | undefined;
codeQlReportedError?: boolean;
}) {
return await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
delete process.env[EnvVar.ANALYZE_DID_COMPLETE_SUCCESSFULLY];
const env = getTestEnv(
codeQlReportedError
? { [EnvVar.JOB_STATUS]: JobStatus.FailureStatus }
: {},
);
sinon.stub(util, "checkDiskUsage").resolves({
numAvailableBytes: 100 * NUM_BYTES_PER_GIB,
numTotalBytes: 200 * NUM_BYTES_PER_GIB,
});
const saveOverlayStatusStub = sinon
.stub(overlayStatus, "saveOverlayStatus")
.resolves(true);
await initActionPostHelper.uploadFailureInfo(
sinon.spy(),
sinon.spy(),
codeql.createStubCodeQL({}),
createTestConfig({
debugMode: false,
languages: ["javascript"],
overlayDatabaseMode: OverlayDatabaseMode.OverlayBase,
}),
parseRepositoryNwo("github/codeql-action"),
createFeatures([Feature.OverlayAnalysisStatusSave]),
jobStatus,
env,
getRunnerLogger(true),
);
return { saveOverlayStatusStub };
});
}
test.serial(
"does not save overlay status when the job was cancelled",
async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: "cancelled",
});
t.true(
saveOverlayStatusStub.notCalled,
"a cancellation tells us nothing about whether the analysis would have succeeded",
);
},
);
test.serial(
"does not save overlay status when the job status is not recognised",
async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: "some-new-status",
});
t.true(
saveOverlayStatusStub.notCalled,
"a status we do not recognise tells us nothing about whether the analysis would have succeeded",
);
},
);
test.serial(
"does not save overlay status when the job status is unavailable",
async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: undefined,
});
t.true(
saveOverlayStatusStub.notCalled,
"without a job status we cannot tell whether the analysis would have succeeded",
);
},
);
test.serial(
"saves overlay status when the job failed rather than being cancelled",
async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: "failure",
});
t.true(
saveOverlayStatusStub.calledOnce,
"a failed job indicates that the analysis itself failed",
);
},
);
test.serial("saves overlay status when the job succeeded", async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: "success",
});
t.true(
saveOverlayStatusStub.calledOnce,
"the analysis did not complete successfully even though the job as a whole succeeded",
);
});
test.serial(
"saves overlay status when a CodeQL Action reported an error before the run was cancelled",
async (t) => {
const { saveOverlayStatusStub } = await runOverlayPostStep({
jobStatus: "cancelled",
codeQlReportedError: true,
});
t.true(
saveOverlayStatusStub.calledOnce,
"the analysis genuinely failed, even though the run was later cancelled",
);
},
);
function createTestWorkflow(
steps: workflow.WorkflowJobStep[],
): workflow.Workflow {
+52 -3
View File
@@ -18,7 +18,7 @@ import {
sanitizeArtifactName,
} from "./debug-artifacts";
import * as dependencyCaching from "./dependency-caching";
import { EnvVar } from "./environment";
import { EnvVar, ReadOnlyEnv } from "./environment";
import { Feature, FeatureEnablement } from "./feature-flags";
import { Logger } from "./logging";
import { OverlayDatabaseMode } from "./overlay/overlay-database-mode";
@@ -316,6 +316,8 @@ export async function tryUploadSarifIfRunFailed(
* @param config The CodeQL Action configuration.
* @param repositoryNwo The name and owner of the repository.
* @param features Information about enabled features.
* @param jobStatus The status of the job, as reported by the Actions runtime environment.
* @param env The environment to read variables from.
* @param logger The logger to use.
* @returns The results of uploading the SARIF file for the failure.
*/
@@ -331,9 +333,11 @@ export async function uploadFailureInfo(
config: Config,
repositoryNwo: RepositoryNwo,
features: FeatureEnablement,
jobStatus: string | undefined,
env: ReadOnlyEnv,
logger: Logger,
): Promise<UploadFailedSarifResult> {
await recordOverlayStatus(codeql, config, features, logger);
await recordOverlayStatus(codeql, config, features, jobStatus, env, logger);
const uploadFailedSarifResult = await tryUploadSarifIfRunFailed(
config,
@@ -412,6 +416,37 @@ export async function uploadFailureInfo(
return uploadFailedSarifResult;
}
/**
* Whether one of the CodeQL Actions reported an error for this job, which means the analysis
* genuinely failed.
*
* Note that the converse does not hold: an Action that is terminated abruptly, or that fails before
* it can gather telemetry, does not get to report anything.
*/
function didCodeQlReportError(env: ReadOnlyEnv): boolean {
const jobStatus = env.getOptional(EnvVar.JOB_STATUS);
return (
jobStatus === JobStatus.FailureStatus ||
jobStatus === JobStatus.ConfigErrorStatus
);
}
/**
* Whether the job status tells us anything about whether the analysis itself would have succeeded.
*
* We check for the statuses we know to be meaningful rather than excluding the ones that are not,
* so that a status we do not recognise is treated as inconclusive.
*/
function isConclusiveJobStatus(jobStatus: string | undefined): boolean {
switch (jobStatus?.trim().toLowerCase()) {
case "failure":
case "success":
return true;
default:
return false;
}
}
/**
* If overlay base database creation was attempted but the analysis did not complete
* successfully, save the failure status to the Actions cache so that subsequent runs
@@ -421,16 +456,30 @@ async function recordOverlayStatus(
codeql: CodeQL,
config: Config,
features: FeatureEnablement,
jobStatus: string | undefined,
env: ReadOnlyEnv,
logger: Logger,
) {
if (
config.overlayDatabaseMode !== OverlayDatabaseMode.OverlayBase ||
process.env[EnvVar.ANALYZE_DID_COMPLETE_SUCCESSFULLY] === "true" ||
env.getOptional(EnvVar.ANALYZE_DID_COMPLETE_SUCCESSFULLY) === "true" ||
!(await features.getValue(Feature.OverlayAnalysisStatusSave))
) {
return;
}
// Only record a failure when the job outcome tells us something about the analysis. A cancelled
// job, or a status we do not recognise, says nothing about whether the analysis would have
// succeeded, so recording a failure would disable overlay analysis needlessly. We still record
// one if a CodeQL Action reported an error before the job ended.
if (!isConclusiveJobStatus(jobStatus) && !didCodeQlReportError(env)) {
logger.info(
"Not recording an improved incremental analysis failure for this job because the job " +
`status (${jobStatus ?? "unset"}) does not tell us whether the analysis itself failed.`,
);
return;
}
const checkRunIdInput = actionsUtil.getOptionalInput("check-run-id");
const checkRunId =
checkRunIdInput !== undefined ? parseInt(checkRunIdInput, 10) : undefined;
+9 -1
View File
@@ -8,6 +8,7 @@ import * as core from "@actions/core";
import {
restoreInputs,
getOptionalInput,
getTemporaryDirectory,
printDebugLogs,
} from "./actions-util";
@@ -20,7 +21,7 @@ import {
DependencyCachingUsageReport,
getDependencyCacheUsage,
} from "./dependency-caching";
import { EnvVar } from "./environment";
import { EnvVar, getEnv } from "./environment";
import { initFeatures } from "./feature-flags";
import * as gitUtils from "./git-utils";
import * as initActionPostHelper from "./init-action-post-helper";
@@ -55,6 +56,11 @@ async function run(startedAt: Date) {
| undefined;
let dependencyCachingUsage: DependencyCachingUsageReport | undefined;
try {
// Read the job status before restoring inputs, since it is provided by the Actions runtime
// environment for this step and would otherwise be overwritten by the value that the `init`
// Action saw, which is always a success.
const jobStatus = getOptionalInput("job-status");
// Restore inputs from `init` Action.
restoreInputs();
@@ -84,6 +90,8 @@ async function run(startedAt: Date) {
config,
repositoryNwo,
features,
jobStatus,
getEnv(),
logger,
);
+8
View File
@@ -174,6 +174,14 @@ async function sendCompletedStatusReport(
initToolsDownloadFields.tools_download_duration_ms =
toolsDownloadStatusReport.downloadDurationMs;
}
if (toolsDownloadStatusReport?.extractionDurationMs !== undefined) {
initToolsDownloadFields.tools_extraction_duration_ms =
toolsDownloadStatusReport.extractionDurationMs;
}
if (toolsDownloadStatusReport?.totalDurationMs !== undefined) {
initToolsDownloadFields.tools_total_duration_ms =
toolsDownloadStatusReport.totalDurationMs;
}
if (toolsFeatureFlagsValid !== undefined) {
initToolsDownloadFields.tools_feature_flags_valid = toolsFeatureFlagsValid;
}
+8
View File
@@ -85,6 +85,14 @@ async function sendCompletedStatusReport(
initToolsDownloadFields.tools_download_duration_ms =
toolsDownloadStatusReport.downloadDurationMs;
}
if (toolsDownloadStatusReport?.extractionDurationMs !== undefined) {
initToolsDownloadFields.tools_extraction_duration_ms =
toolsDownloadStatusReport.extractionDurationMs;
}
if (toolsDownloadStatusReport?.totalDurationMs !== undefined) {
initToolsDownloadFields.tools_total_duration_ms =
toolsDownloadStatusReport.totalDurationMs;
}
if (toolsFeatureFlagsValid !== undefined) {
initToolsDownloadFields.tools_feature_flags_valid = toolsFeatureFlagsValid;
}
+641 -2
View File
@@ -1,3 +1,5 @@
import * as fs from "fs";
import * as os from "os";
import * as path from "path";
import * as github from "@actions/github";
@@ -7,7 +9,8 @@ import * as sinon from "sinon";
import * as actionsUtil from "./actions-util";
import * as api from "./api-client";
import { EnvVar } from "./environment";
import * as diagnostics from "./diagnostics";
import { ActionsEnvVars, EnvVar, ReadOnlyEnv } from "./environment";
import { Feature } from "./feature-flags";
import { getRunnerLogger } from "./logging";
import { getCacheRestoreKeyPrefix } from "./overlay/caching";
@@ -27,6 +30,7 @@ import {
setupActionsVars,
setupTests,
} from "./testing-utils";
import * as toolsDownload from "./tools-download";
import {
getErrorMessage,
GitHubVariant,
@@ -120,24 +124,42 @@ test.serial(
const LINKED_BUNDLE_TEST_CASES = [
{
platform: "linux",
arch: "x64",
tarSupportsZstd: true,
expectedBundleName: "codeql-bundle-linux64.tar.zst",
expectedCompressionMethod: "zstd",
},
{
platform: "linux",
arch: "arm64",
tarSupportsZstd: true,
expectedBundleName: "codeql-bundle-linux-arm64.tar.zst",
expectedCompressionMethod: "zstd",
},
{
platform: "darwin",
arch: "arm64",
tarSupportsZstd: true,
expectedBundleName: "codeql-bundle-osx64.tar.zst",
expectedCompressionMethod: "zstd",
},
{
platform: "darwin",
arch: "x64",
tarSupportsZstd: true,
expectedBundleName: "codeql-bundle-osx64.tar.zst",
expectedCompressionMethod: "zstd",
},
{
platform: "win32",
arch: "x64",
tarSupportsZstd: true,
expectedBundleName: "codeql-bundle-win64.tar.gz",
expectedCompressionMethod: "gzip",
},
{
platform: "linux",
arch: "x64",
tarSupportsZstd: false,
expectedBundleName: "codeql-bundle-linux64.tar.gz",
expectedCompressionMethod: "gzip",
@@ -146,15 +168,17 @@ const LINKED_BUNDLE_TEST_CASES = [
for (const {
platform,
arch,
tarSupportsZstd,
expectedBundleName,
expectedCompressionMethod,
} of LINKED_BUNDLE_TEST_CASES) {
test.serial(
`getCodeQLSource selects ${expectedBundleName} for linked tools`,
`getCodeQLSource selects ${expectedBundleName} for linked tools on ${platform}/${arch}`,
async (t) => {
const features = createFeatures([]);
sinon.stub(process, "platform").value(platform);
sinon.stub(process, "arch").value(arch);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -234,6 +258,7 @@ test.serial(
codeqlFolder: "codeql",
statusReport: {
downloadDurationMs: 200,
totalDurationMs: 300,
},
toolsVersion: LINKED_CLI_VERSION.cliVersion,
});
@@ -286,6 +311,7 @@ test.serial(
codeqlFolder: "codeql",
statusReport: {
downloadDurationMs: 200,
totalDurationMs: 300,
},
toolsVersion: expectedVersion,
});
@@ -917,3 +943,616 @@ test.serial(
]);
},
);
/** The CLI version that the toolcache cleanup tests download. */
const CLEANUP_CLI_VERSION = "2.21.0";
/** The bundle version that the toolcache cleanup tests download. */
const CLEANUP_BUNDLE_VERSION = "20240101";
/** A version of the CodeQL tools that is already in the toolcache but that we are not going to use. */
const CLEANUP_STALE_VERSION = "2.20.0";
/** Creates a directory in the toolcache that looks like a tool that `tool-cache` has cached. */
function createToolcacheEntry(
toolcacheRoot: string,
tool: string,
version: string,
): string {
const versionDirectory = path.join(toolcacheRoot, tool, version);
const archDirectory = path.join(versionDirectory, os.arch());
fs.mkdirSync(archDirectory, { recursive: true });
fs.writeFileSync(path.join(archDirectory, "contents"), "x".repeat(1024));
fs.writeFileSync(`${archDirectory}.complete`, "");
return versionDirectory;
}
/**
* Stubs out the download and the diagnostic sink, then downloads the CodeQL tools.
*
* @returns the extraction directory and the toolcache cleanup diagnostic, if emitted.
*/
async function runDownloadCodeQL(
toolcacheRoot: string,
features: Feature[],
bundleVersion: string | undefined,
): Promise<{
codeqlFolder: string;
cleanupDiagnostic: toolsDownload.ToolcacheCleanupResult | undefined;
}> {
sinon
.stub(toolsDownload, "downloadAndExtract")
.callsFake(async (_url, _compressionMethod, dest) => {
// The real implementation creates the destination directory, which matters here because the
// cleanup deletes it first and `writeToolcacheMarkerFile` writes into its parent afterwards.
fs.mkdirSync(dest, { recursive: true });
return { totalDurationMs: 1 };
});
const addDiagnostic = sinon.stub(diagnostics, "addNoLanguageDiagnostic");
const { codeqlFolder } = await setupCodeql.downloadCodeQL(
"https://example.com/codeql-bundle.tar.gz",
"gzip",
bundleVersion,
CLEANUP_CLI_VERSION,
SAMPLE_DOTCOM_API_DETAILS,
undefined, // tarVersion
toolcacheRoot, // tempDir
createFeatures(features),
getRunnerLogger(true),
);
const diagnostic = addDiagnostic
.getCalls()
.map((call) => call.args[1])
.find((d) => d.source?.id === "codeql-action/toolcache-bundle-cleanup");
return {
codeqlFolder,
cleanupDiagnostic: diagnostic?.attributes as
| toolsDownload.ToolcacheCleanupResult
| undefined,
};
}
/**
* Sets up a toolcache containing the version of the CodeQL tools that we are about to download, a
* different version of the CodeQL tools, and an unrelated tool, then downloads the CodeQL tools.
*/
async function testToolcacheCleanup(
t: ExecutionContext<unknown>,
{
features,
runnerEnvironment,
setUp,
}: {
features: Feature[];
runnerEnvironment: string | undefined;
setUp?: () => void;
},
check: (context: {
cleanupDiagnostic: toolsDownload.ToolcacheCleanupResult | undefined;
destinationDirectory: string;
staleDirectory: string;
}) => void,
) {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
if (runnerEnvironment === undefined) {
delete process.env[ActionsEnvVars.RUNNER_ENVIRONMENT];
} else {
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = runnerEnvironment;
}
setUp?.();
// The extraction of the bundle would normally create this directory.
const destinationDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_CLI_VERSION,
);
const staleDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const otherToolDirectory = createToolcacheEntry(tmpDir, "Node", "20.0.0");
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
features,
CLEANUP_BUNDLE_VERSION,
);
t.true(
fs.existsSync(otherToolDirectory),
"Should never delete other tools from the toolcache.",
);
check({ cleanupDiagnostic, destinationDirectory, staleDirectory });
});
}
test.serial(
"downloadCodeQL does not clean up the toolcache when the feature flag is disabled",
async (t) => {
await testToolcacheCleanup(
t,
{ features: [], runnerEnvironment: "github-hosted" },
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"downloadCodeQL does not clean up the toolcache when the runner is not GitHub-hosted",
async (t) => {
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "self-hosted",
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"downloadCodeQL does not clean up the toolcache when the runner environment is unknown",
async (t) => {
// A runner that doesn't report its environment must be treated as not GitHub-hosted, since its
// toolcache may well outlive the job.
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: undefined,
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"downloadCodeQL deletes other CodeQL bundles from the toolcache when enabled on a GitHub-hosted runner",
async (t) => {
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "github-hosted",
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.false(
fs.existsSync(staleDirectory),
"Should delete the version directory, including the `tool-cache` marker file it contains.",
);
t.false(
fs.existsSync(path.join(destinationDirectory, os.arch(), "contents")),
"Should also delete a partial entry for the version we are about to download, rather " +
"than extracting over it.",
);
t.deepEqual(cleanupDiagnostic, {
deletedVersions: [CLEANUP_STALE_VERSION, CLEANUP_CLI_VERSION].sort(),
failed: false,
});
},
);
},
);
test.serial(
"downloadCodeQL reports no deleted versions when the toolcache has no CodeQL bundles",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
// A toolcache with other tools in it, but no CodeQL.
const otherToolDirectory = createToolcacheEntry(tmpDir, "Node", "20.0.0");
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
t.true(fs.existsSync(otherToolDirectory));
t.deepEqual(cleanupDiagnostic, { deletedVersions: [], failed: false });
});
},
);
test.serial(
"downloadCodeQL continues when deleting a CodeQL bundle from the toolcache fails",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
createToolcacheEntry(tmpDir, "CodeQL", CLEANUP_CLI_VERSION);
const staleDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const rmStub = sinon
.stub(fs.promises, "rm")
.rejects(new Error("EACCES: permission denied"));
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
// Restore before `withTmpDir` cleans up after itself.
rmStub.restore();
t.true(fs.existsSync(staleDirectory));
t.deepEqual(
cleanupDiagnostic,
{ deletedVersions: [], failed: true },
"Should not report versions that we failed to delete.",
);
});
},
);
test.serial(
"deleteToolcacheBundles reports a failure when the toolcache location is unknown",
async (t) => {
const messages: LoggedMessage[] = [];
const result = await toolsDownload.deleteToolcacheBundles({
env: new ReadOnlyEnv({}),
logger: getRecordingLogger(messages),
});
t.deepEqual(
result,
{ deletedVersions: [], failed: true },
"Should report a failure rather than throwing, so the download can continue.",
);
checkExpectedLogMessages(t, messages, [
"Unable to determine toolcache directory: RUNNER_TOOL_CACHE environment variable must be set",
]);
},
);
test.serial(
"deleteToolcacheBundles uses the supplied environment rather than process.env",
async (t) => {
await withTmpDir(async (tmpDir) => {
const ambientRoot = path.join(tmpDir, "ambient");
const injectedRoot = path.join(tmpDir, "injected");
setupActionsVars(tmpDir, ambientRoot);
const ambientVersion = createToolcacheEntry(
ambientRoot,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const injectedVersion = createToolcacheEntry(
injectedRoot,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const fileEntry = path.join(injectedRoot, "CodeQL", "not-a-directory");
fs.writeFileSync(fileEntry, "keep");
const result = await toolsDownload.deleteToolcacheBundles({
env: new ReadOnlyEnv({
[ActionsEnvVars.RUNNER_TOOL_CACHE]: injectedRoot,
}),
logger: getRunnerLogger(true),
});
t.deepEqual(result, {
deletedVersions: [CLEANUP_STALE_VERSION],
failed: false,
});
t.false(fs.existsSync(injectedVersion));
t.true(fs.existsSync(ambientVersion));
t.is(fs.readFileSync(fileEntry, "utf8"), "keep");
});
},
);
test.serial(
"deleteToolcacheBundles reports a failure when the toolcache directory cannot be read",
async (t) => {
await withTmpDir(async (tmpDir) => {
const versionDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const messages: LoggedMessage[] = [];
const readdir = sinon
.stub(fs.promises, "readdir")
.rejects(new Error("permission denied"));
try {
const result = await toolsDownload.deleteToolcacheBundles({
env: new ReadOnlyEnv({
[ActionsEnvVars.RUNNER_TOOL_CACHE]: tmpDir,
}),
logger: getRecordingLogger(messages),
});
t.deepEqual(result, { deletedVersions: [], failed: true });
t.true(fs.existsSync(versionDirectory));
checkExpectedLogMessages(t, messages, [
`Failed to clean up the CodeQL toolcache at '${path.join(tmpDir, "CodeQL")}': permission denied`,
]);
} finally {
readdir.restore();
}
});
},
);
test.serial(
"downloadCodeQL does not follow a symlinked CodeQL toolcache directory",
async (t) => {
await withTmpDir(async (tmpDir) => {
const toolcacheRoot = path.join(tmpDir, "toolcache");
setupActionsVars(tmpDir, toolcacheRoot);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
// Somewhere the toolcache cleanup must never reach.
const outsideDirectory = path.join(tmpDir, "outside");
createToolcacheEntry(outsideDirectory, "CodeQL", CLEANUP_STALE_VERSION);
createToolcacheEntry(outsideDirectory, "CodeQL", CLEANUP_CLI_VERSION);
fs.mkdirSync(toolcacheRoot, { recursive: true });
fs.symlinkSync(
path.join(outsideDirectory, "CodeQL"),
path.join(toolcacheRoot, "CodeQL"),
);
const { cleanupDiagnostic } = await runDownloadCodeQL(
toolcacheRoot,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
t.true(
fs.existsSync(
path.join(outsideDirectory, "CodeQL", CLEANUP_STALE_VERSION),
),
"Should not delete anything through a symlinked CodeQL directory.",
);
t.deepEqual(cleanupDiagnostic, { deletedVersions: [], failed: true });
});
},
);
test.serial(
"downloadCodeQL does not clean up the toolcache once a step has already set up CodeQL",
async (t) => {
// `.github/workflows/codeql.yml` sets up CodeQL twice and then runs both returned paths. If the
// second setup downloads, it must not delete the bundle the first one handed out.
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "github-hosted",
setUp: () => {
process.env[EnvVar.HAS_SET_UP_CODEQL] = "true";
},
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"setupCodeQLBundle records that this job has set up CodeQL",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
delete process.env[EnvVar.HAS_SET_UP_CODEQL];
sinon.stub(setupCodeql, "downloadCodeQL").resolves({
codeqlFolder: "codeql",
statusReport: { totalDurationMs: 1 },
toolsVersion: LINKED_CLI_VERSION.cliVersion,
});
await setupCodeql.setupCodeQLBundle(
"linked",
SAMPLE_DOTCOM_API_DETAILS,
tmpDir,
GitHubVariant.DOTCOM,
SAMPLE_DEFAULT_CLI_VERSION,
undefined, // rawLanguages
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
getRunnerLogger(true),
);
t.is(
process.env[EnvVar.HAS_SET_UP_CODEQL],
"true",
"A later step must be able to tell that the toolcache is in use.",
);
});
},
);
test.serial(
"downloadCodeQL cleans up the toolcache even when the download will not be cached",
async (t) => {
// A `tools` URL we can't derive a bundle version from is extracted to a temporary directory
// rather than the toolcache, but the toolcache is on the same filesystem, so emptying it still
// frees up space for the analysis.
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
const staleDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_STALE_VERSION,
);
const { codeqlFolder, cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
undefined, // bundleVersion
);
t.is(path.dirname(codeqlFolder), tmpDir);
t.not(codeqlFolder, path.join(tmpDir, "CodeQL"));
t.true(fs.existsSync(codeqlFolder));
t.false(fs.existsSync(`${codeqlFolder}.complete`));
t.false(fs.existsSync(staleDirectory));
t.deepEqual(cleanupDiagnostic, {
deletedVersions: [CLEANUP_STALE_VERSION],
failed: false,
});
});
},
);
test.serial(
"downloadCodeQL reports a failure when the toolcache cannot be inspected",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
createToolcacheEntry(tmpDir, "CodeQL", CLEANUP_STALE_VERSION);
const lstatStub = sinon.stub(fs.promises, "lstat").rejects(
Object.assign(new Error("permission denied"), {
code: "EACCES",
}),
);
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
lstatStub.restore();
t.deepEqual(
cleanupDiagnostic,
{ deletedVersions: [], failed: true },
"An error other than the toolcache being absent must not be reported as success.",
);
});
},
);
test.serial(
"downloadCodeQL does not clean up a toolcache on a different filesystem to the workspace",
async (t) => {
// Some runner images keep the toolcache on a different volume to the workspace, in which case
// deleting the tools frees up disk space that the analysis cannot use.
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
const staleDirectory = createToolcacheEntry(
tmpDir,
"CodeQL",
CLEANUP_STALE_VERSION,
);
sinon
.stub(toolsDownload, "isToolcacheOnWorkspaceFilesystem")
.returns(false);
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
t.true(fs.existsSync(staleDirectory));
t.is(cleanupDiagnostic, undefined);
});
},
);
test.serial(
"isToolcacheOnWorkspaceFilesystem assumes a different filesystem when it cannot tell",
async (t) => {
await withTmpDir(async (tmpDir) => {
const logger = getRunnerLogger(true);
setupActionsVars(tmpDir, tmpDir);
t.true(toolsDownload.isToolcacheOnWorkspaceFilesystem(logger));
// If we can't tell, we assume the toolcache is not somewhere we can reclaim space from.
process.env[ActionsEnvVars.RUNNER_TOOL_CACHE] = path.join(
tmpDir,
"does-not-exist",
);
t.false(toolsDownload.isToolcacheOnWorkspaceFilesystem(logger));
});
},
);
test.serial(
"downloadCodeQL does not delete through a symlinked version directory",
async (t) => {
await withTmpDir(async (tmpDir) => {
const toolcacheRoot = path.join(tmpDir, "toolcache");
setupActionsVars(tmpDir, toolcacheRoot);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
createToolcacheEntry(toolcacheRoot, "CodeQL", CLEANUP_STALE_VERSION);
// Somewhere outside the toolcache that a version directory points at.
const outsideDirectory = path.join(tmpDir, "outside");
fs.mkdirSync(outsideDirectory, { recursive: true });
fs.writeFileSync(path.join(outsideDirectory, "contents"), "x");
fs.symlinkSync(
outsideDirectory,
path.join(toolcacheRoot, "CodeQL", "9.9.9"),
);
const { cleanupDiagnostic } = await runDownloadCodeQL(
toolcacheRoot,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
t.true(
fs.existsSync(path.join(outsideDirectory, "contents")),
"Should not delete anything through a symlinked version directory.",
);
t.true(
fs
.lstatSync(path.join(toolcacheRoot, "CodeQL", "9.9.9"))
.isSymbolicLink(),
);
t.deepEqual(cleanupDiagnostic, {
deletedVersions: [CLEANUP_STALE_VERSION],
failed: false,
});
});
},
);
+58 -1
View File
@@ -2,14 +2,17 @@ import * as fs from "fs";
import { OutgoingHttpHeaders } from "http";
import * as path from "path";
import * as core from "@actions/core";
import * as toolcache from "@actions/tool-cache";
import { default as deepEqual } from "fast-deep-equal";
import * as semver from "semver";
import { v4 as uuidV4 } from "uuid";
import { ActionState } from "./action-common";
import {
isAnalyzingPullRequest,
isDynamicWorkflow,
isGitHubHostedRunner,
isRunningLocalAction,
} from "./actions-util";
import * as api from "./api-client";
@@ -19,6 +22,7 @@ import {
makeDiagnostic,
makeTelemetryDiagnostic,
} from "./diagnostics";
import { EnvVar, getEnv } from "./environment";
import {
CODEQL_VERSION_ZSTD_BUNDLE,
CodeQLDefaultVersionInfo,
@@ -30,8 +34,10 @@ import { Logger } from "./logging";
import { getCodeQlVersionsForOverlayBaseDatabases } from "./overlay/caching";
import * as tar from "./tar";
import {
deleteToolcacheBundles,
downloadAndExtract,
getToolcacheDirectory,
isToolcacheOnWorkspaceFilesystem,
ToolsDownloadStatusReport,
writeToolcacheMarkerFile,
} from "./tools-download";
@@ -75,7 +81,7 @@ export function getCodeQLBundleName(
if (process.platform === "win32") {
platform = "win64";
} else if (process.platform === "linux") {
platform = "linux64";
platform = process.arch === "arm64" ? "linux-arm64" : "linux64";
} else if (process.platform === "darwin") {
platform = "osx64";
} else {
@@ -784,6 +790,7 @@ export const downloadCodeQL = async function (
apiDetails: api.GitHubApiDetails,
tarVersion: tar.TarVersion | undefined,
tempDir: string,
features: FeatureEnablement,
logger: Logger,
): Promise<{
codeqlFolder: string;
@@ -817,6 +824,8 @@ export const downloadCodeQL = async function (
const extractedBundlePath =
toolcacheInfo?.path ?? getTempExtractionDir(tempDir);
await tryDeleteToolcacheBundles({ env: getEnv(), features, logger });
const statusReport = await downloadAndExtract(
codeqlURL,
compressionMethod,
@@ -869,6 +878,48 @@ function getToolcacheDestinationInfo(
return undefined;
}
/**
* Reclaims disk space by deleting the CodeQL tools from the toolcache, if enabled.
*
* On GitHub-hosted runners the toolcache shares a filesystem with the workspace, so tools left in
* the toolcache take up space that the analysis could use instead. This holds wherever we extract
* the tools we are obtaining, since the toolcache is on that filesystem either way.
*/
async function tryDeleteToolcacheBundles({
env,
features,
logger,
}: ActionState<["Logger", "ReadOnlyEnv", "FeatureFlags"]>): Promise<void> {
// A step that has already set up CodeQL may hand out a path into the toolcache that a later step
// runs, so only the first step to set it up can know that nothing else relies on the toolcache.
if (env.getOptional(EnvVar.HAS_SET_UP_CODEQL) !== undefined) {
logger.debug(
"Not deleting the CodeQL tools from the toolcache since a previous step in this job has " +
"already set up CodeQL.",
);
return;
}
if (
!isGitHubHostedRunner() ||
!isToolcacheOnWorkspaceFilesystem(logger) ||
!(await features.getValue(Feature.CleanupToolcacheBundles))
) {
return;
}
const result = await deleteToolcacheBundles({ env, logger });
addNoLanguageDiagnostic(
undefined,
makeTelemetryDiagnostic(
"codeql-action/toolcache-bundle-cleanup",
"Toolcache CodeQL bundle cleanup",
{ ...result },
),
);
}
export function getCodeQLURLVersion(url: string): string {
const match = url.match(/\/codeql-bundle-(.*)\//);
if (match === null || match.length < 2) {
@@ -978,6 +1029,7 @@ export async function setupCodeQLBundle(
apiDetails,
zstdAvailability.version,
tempDir,
features,
logger,
);
toolsVersion = result.toolsVersion;
@@ -989,6 +1041,11 @@ export async function setupCodeQLBundle(
default:
util.assertNever(source);
}
// Record that this job now has a copy of the CodeQL tools, so that a later step doesn't delete
// the toolcache out from under the path we are about to return.
core.exportVariable(EnvVar.HAS_SET_UP_CODEQL, "true");
return {
codeqlFolder,
toolsDownloadStatusReport,
+22 -3
View File
@@ -14,6 +14,10 @@ import {
isSelfHostedRunner,
} from "./actions-util";
import { getAnalysisKey, getApiClient } from "./api-client";
import {
getCachedCodeQlVersion,
getCommandCacheFilePath,
} from "./cli/output-cache";
import type { Config } from "./config/action-config";
import type { ComputedInput, InputName } from "./config/inputs";
import { parseRegistriesWithoutCredentials } from "./config/pack-registries";
@@ -30,7 +34,6 @@ import { registryBaseSchema } from "./start-proxy/types";
import {
ConfigurationError,
getRequiredEnvParam,
getCachedCodeQlVersion,
isInTestMode,
GITHUB_DOTCOM_URL,
DiskUsage,
@@ -376,7 +379,10 @@ export async function createStatusReportBase(
core.exportVariable(EnvVar.WORKFLOW_STARTED_AT, workflowStartedAt);
}
const runnerOs = getRequiredEnvParam("RUNNER_OS");
const codeQlCliVersion = getCachedCodeQlVersion();
const codeQlCliVersion = getCachedCodeQlVersion(
logger,
getCommandCacheFilePath(getEnv()),
);
const actionRef = process.env["GITHUB_ACTION_REF"] || "";
const testingEnvironment = getTestingEnvironment();
// re-export the testing environment variable so that it is available to subsequent steps,
@@ -620,8 +626,21 @@ export interface InitWithConfigStatusReport extends InitStatusReport {
/** Fields of the init status report populated when the tools source is `download`. */
export interface InitToolsDownloadFields {
/** Time taken to download the bundle, in milliseconds. */
/**
* Time taken to download the bundle, in milliseconds. Not populated when the bundle is downloaded
* and extracted concurrently.
*/
tools_download_duration_ms?: number;
/**
* Time taken to extract the bundle, in milliseconds. Not populated when the bundle is downloaded
* and extracted concurrently.
*/
tools_extraction_duration_ms?: number;
/**
* Total time taken to make the bundle available on disk, in milliseconds. This includes any time
* spent on a streaming attempt that failed and fell back to downloading before extracting.
*/
tools_total_duration_ms?: number;
/**
* Whether the relevant tools dotcom feature flags have been misconfigured.
* Only populated if we attempt to determine the default version based on the dotcom feature flags. */
+6 -3
View File
@@ -18,6 +18,8 @@ import { AnalysisKind } from "./analyses";
import * as apiClient from "./api-client";
import { GitHubApiDetails } from "./api-client";
import { CachingKind } from "./caching-utils";
import { resetCachedCodeQlVersion } from "./cli/output-cache";
import type { VersionInfo } from "./cli/types";
import * as codeql from "./codeql";
import { Config } from "./config-utils";
import * as defaults from "./defaults.json";
@@ -39,7 +41,6 @@ import {
GitHubVariant,
GitHubVersion,
HTTPError,
resetCachedCodeQlVersion,
Result,
Success,
} from "./util";
@@ -872,7 +873,7 @@ export const makeVersionInfo = (
version: string,
features?: { [name: string]: boolean },
overlayVersion?: number,
): codeql.VersionInfo => ({
): VersionInfo => ({
version,
features,
overlayVersion,
@@ -936,7 +937,9 @@ export function mockBundleDownloadApi({
process.platform === "win32"
? "win64"
: process.platform === "linux"
? "linux64"
? process.arch === "arm64"
? "linux-arm64"
: "linux64"
: "osx64";
const baseUrl = apiDetails?.url ?? "https://example.com";
+8 -3
View File
@@ -15,7 +15,7 @@ import { withTmpDir } from "./util";
setupTests(test);
test.serial(
"downloadAndExtract reports the duration when downloading before extracting",
"downloadAndExtract reports the durations when downloading before extracting",
async (t) => {
await withTmpDir(async (tmpDir) => {
const archivePath = path.join(tmpDir, "codeql-bundle.tar.gz");
@@ -34,6 +34,8 @@ test.serial(
);
t.assert(Number.isInteger(statusReport.downloadDurationMs));
t.assert(Number.isInteger(statusReport.extractionDurationMs));
t.assert(Number.isInteger(statusReport.totalDurationMs));
});
},
);
@@ -67,6 +69,7 @@ test.serial(
);
t.assert(Number.isInteger(statusReport.downloadDurationMs));
t.assert(Number.isInteger(statusReport.totalDurationMs));
t.true(request.isDone());
t.false(extractTarZst.called);
t.true(downloadTool.calledOnce);
@@ -76,7 +79,7 @@ test.serial(
);
test.serial(
"downloadAndExtract omits the download duration when streaming extraction",
"downloadAndExtract reports only the total duration when streaming extraction",
async (t) => {
await withTmpDir(async (tmpDir) => {
sinon.stub(process, "platform").value("linux");
@@ -106,7 +109,9 @@ test.serial(
getRunnerLogger(true),
);
t.deepEqual(statusReport, {});
t.assert(Number.isInteger(statusReport.totalDurationMs));
t.is(statusReport.downloadDurationMs, undefined);
t.is(statusReport.extractionDurationMs, undefined);
t.false(downloadTool.called);
t.true(extractTarZst.calledOnce);
t.true(request.isDone());
+168 -11
View File
@@ -10,6 +10,8 @@ import * as toolcache from "@actions/tool-cache";
import { https } from "follow-redirects";
import * as semver from "semver";
import { ActionState } from "./action-common";
import { ActionsEnvVars, getEnv, ReadOnlyEnv } from "./environment";
import { formatDuration, Logger } from "./logging";
import * as tar from "./tar";
import { cleanUpPath, getErrorMessage, getRequiredEnvParam } from "./util";
@@ -31,7 +33,21 @@ const STREAMING_STALL_TIMEOUT_MS = 5 * 60 * 1000; // 5 minutes
const TOOLCACHE_TOOL_NAME = "CodeQL";
export type ToolsDownloadStatusReport = {
/**
* Time spent downloading the bundle, in milliseconds. Not populated when the bundle is downloaded
* and extracted concurrently, since the two cannot be told apart.
*/
downloadDurationMs?: number;
/**
* Time spent extracting the bundle, in milliseconds. Not populated when the bundle is downloaded
* and extracted concurrently, since the two cannot be told apart.
*/
extractionDurationMs?: number;
/**
* Total time taken to make the bundle available on disk, in milliseconds. This includes any time
* spent on a streaming attempt that failed and fell back to downloading before extracting.
*/
totalDurationMs: number;
};
export async function downloadAndExtract(
@@ -47,11 +63,12 @@ export async function downloadAndExtract(
`Downloading CodeQL tools from ${codeqlURL} . This may take a while.`,
);
const startTime = performance.now();
try {
if (compressionMethod === "zstd" && process.platform === "linux") {
logger.info(`Streaming the extraction of the CodeQL bundle.`);
const toolsInstallStart = performance.now();
await downloadAndExtractZstdWithStreaming(
codeqlURL,
dest,
@@ -61,16 +78,14 @@ export async function downloadAndExtract(
logger,
);
const combinedDurationMs = Math.round(
performance.now() - toolsInstallStart,
);
const totalDurationMs = Math.round(performance.now() - startTime);
logger.info(
`Finished downloading and extracting CodeQL bundle to ${dest} (${formatDuration(
combinedDurationMs,
totalDurationMs,
)}).`,
);
return {};
return { totalDurationMs };
}
} catch (e) {
core.warning(
@@ -98,7 +113,7 @@ export async function downloadAndExtract(
)}).`,
);
let extractionDurationMs: number;
let extractionDurationMs: number | undefined;
try {
logger.info("Extracting CodeQL bundle.");
@@ -120,7 +135,11 @@ export async function downloadAndExtract(
await cleanUpPath(archivedBundlePath, "CodeQL bundle archive", logger);
}
return { downloadDurationMs };
return {
downloadDurationMs,
extractionDurationMs,
totalDurationMs: Math.round(performance.now() - startTime),
};
}
async function downloadAndExtractZstdWithStreaming(
@@ -180,16 +199,154 @@ async function downloadAndExtractZstdWithStreaming(
await tar.extractTarZst(response, dest, tarVersion, logger);
}
/** Gets the path to the toolcache directory that holds all versions of the CodeQL tools. */
function getToolcacheToolDirectory(env: ReadOnlyEnv): string {
return path.join(
env.getRequired(ActionsEnvVars.RUNNER_TOOL_CACHE),
TOOLCACHE_TOOL_NAME,
);
}
/** Gets the name of the toolcache directory that holds the given version of the CodeQL tools. */
function getToolcacheVersionDirectoryName(version: string): string {
return semver.clean(version) || version;
}
/** Gets the path to the toolcache directory for the specified version of the CodeQL tools. */
export function getToolcacheDirectory(version: string): string {
return path.join(
getRequiredEnvParam("RUNNER_TOOL_CACHE"),
TOOLCACHE_TOOL_NAME,
semver.clean(version) || version,
getToolcacheToolDirectory(getEnv()),
getToolcacheVersionDirectoryName(version),
os.arch() || "",
);
}
/**
* Whether the toolcache is on the same filesystem as the workspace, and so whether deleting the
* tools frees up disk space that the analysis can use.
*
* These are separate volumes on some runner images. Windows runners, for example, keep the
* toolcache on `C:` while the workspace is on `D:`.
*/
export function isToolcacheOnWorkspaceFilesystem(logger: Logger): boolean {
try {
return (
fs.statSync(getRequiredEnvParam("RUNNER_TOOL_CACHE")).dev ===
fs.statSync(getRequiredEnvParam("GITHUB_WORKSPACE")).dev
);
} catch (e) {
logger.debug(
`Could not determine whether the toolcache is on the same filesystem as the workspace: ${getErrorMessage(e)}`,
);
return false;
}
}
/** The outcome of trying to reclaim disk space by deleting the CodeQL tools from the toolcache. */
export interface ToolcacheCleanupResult {
/** The versions of the CodeQL tools that were deleted. */
deletedVersions: string[];
/**
* Whether we hit an error while trying to delete the tools. Distinguishes a toolcache that had
* nothing to reclaim from one we failed to clean up.
*/
failed: boolean;
}
/**
* Deletes every version of the CodeQL tools from the toolcache.
*
* Only safe to call when we are about to download the tools, since that means we did not resolve
* them from the toolcache and so nothing in there is in use by this job.
*
* This only ever touches the CodeQL directory of the toolcache. Cleanup errors are logged and
* returned as `failed: true` rather than thrown.
*
* @returns the versions that were deleted, and whether we hit an error while trying.
*/
export async function deleteToolcacheBundles({
env,
logger,
}: ActionState<["Logger", "ReadOnlyEnv"]>): Promise<ToolcacheCleanupResult> {
let toolDirectory: string;
try {
toolDirectory = getToolcacheToolDirectory(env);
} catch (e) {
logger.info(
`Unable to determine toolcache directory: ${getErrorMessage(e)}`,
);
return { deletedVersions: [], failed: true };
}
try {
// Refuse to follow a symlinked CodeQL directory, so that we can only ever delete paths that are
// really inside the toolcache.
if ((await fs.promises.lstat(toolDirectory)).isSymbolicLink()) {
logger.info(
`Not deleting the CodeQL tools from the toolcache since '${toolDirectory}' is a symlink.`,
);
return { deletedVersions: [], failed: true };
}
} catch (e: any) {
if (e?.code === "ENOENT") {
logger.debug(
`There are no CodeQL tools at '${toolDirectory}' to delete from the toolcache.`,
);
return { deletedVersions: [], failed: false };
}
logger.info(
`Failed to inspect the CodeQL tools at '${toolDirectory}': ${getErrorMessage(e)}`,
);
return { deletedVersions: [], failed: true };
}
try {
const entries = await fs.promises.readdir(toolDirectory, {
withFileTypes: true,
});
const deletedVersions: string[] = [];
let failed = false;
for (const entry of entries) {
// `isDirectory` is false for a symlink, so we never delete a version directory that is
// really somewhere else.
if (!entry.isDirectory()) {
logger.debug(
`Not deleting '${entry.name}' from the CodeQL toolcache since it is not a directory.`,
);
continue;
}
const versionDirectory = path.join(toolDirectory, entry.name);
try {
await fs.promises.rm(versionDirectory, {
force: true,
recursive: true,
});
deletedVersions.push(entry.name);
logger.info(
`Deleted the CodeQL tools at '${versionDirectory}' from the toolcache to free up disk space.`,
);
} catch (e) {
failed = true;
logger.info(
`Failed to delete the CodeQL tools at '${versionDirectory}' from the toolcache: ${getErrorMessage(e)}`,
);
}
}
return { deletedVersions: deletedVersions.sort(), failed };
} catch (e) {
logger.info(
`Failed to clean up the CodeQL toolcache at '${toolDirectory}': ${getErrorMessage(e)}`,
);
return { deletedVersions: [], failed: true };
}
}
export function writeToolcacheMarkerFile(
extractedPath: string,
logger: Logger,
+1 -1
View File
@@ -1,6 +1,6 @@
import * as semver from "semver";
import type { VersionInfo } from "./codeql";
import type { VersionInfo } from "./cli/types";
export enum ToolsFeature {
BuiltinExtractorsSpecifyDefaultQueries = "builtinExtractorsSpecifyDefaultQueries",
-55
View File
@@ -532,58 +532,3 @@ test("Failure.orElse returns the default value for a failure result", (t) => {
const result = new util.Failure(new Error("test error"));
t.is(result.orElse("default value"), "default value");
});
test.serial(
"getCachedCodeQlVersion reuses a version persisted by an earlier step",
(t) => {
process.env[EnvVar.CODEQL_VERSION_INFO] = JSON.stringify({
cmd: "/path/to/codeql",
version: { version: "2.20.0" },
});
t.deepEqual(util.getCachedCodeQlVersion("/path/to/codeql"), {
version: "2.20.0",
});
},
);
test.serial(
"getCachedCodeQlVersion ignores a persisted version from a different CLI",
(t) => {
process.env[EnvVar.CODEQL_VERSION_INFO] = JSON.stringify({
cmd: "/path/to/other-codeql",
version: { version: "2.20.0" },
});
t.is(util.getCachedCodeQlVersion("/path/to/codeql"), undefined);
},
);
test.serial(
"getCachedCodeQlVersion ignores a malformed persisted value",
(t) => {
process.env[EnvVar.CODEQL_VERSION_INFO] = "not valid json";
t.is(util.getCachedCodeQlVersion("/path/to/codeql"), undefined);
},
);
test.serial(
"getCachedCodeQlVersion ignores a persisted value with the wrong structure",
(t) => {
for (const value of [
JSON.stringify({ cmd: "/path/to/codeql" }),
JSON.stringify({ cmd: "/path/to/codeql", version: {} }),
JSON.stringify({ cmd: "/path/to/codeql", version: { version: 2 } }),
JSON.stringify({ version: { version: "2.20.0" } }),
JSON.stringify({
cmd: "/path/to/codeql",
version: { version: "2.20.0", overlayVersion: "1" },
}),
JSON.stringify({
cmd: "/path/to/codeql",
version: { version: "2.20.0", features: "nope" },
}),
]) {
process.env[EnvVar.CODEQL_VERSION_INFO] = value;
t.is(util.getCachedCodeQlVersion("/path/to/codeql"), undefined, value);
}
},
);
+7 -87
View File
@@ -10,7 +10,7 @@ import * as yaml from "js-yaml";
import * as semver from "semver";
import * as apiCompatibility from "./api-compatibility.json";
import type { CodeQL, VersionInfo } from "./codeql";
import type { CodeQL } from "./codeql";
import type { Pack } from "./config/db-config";
import type { Config } from "./config-utils";
import { EnvVar, getRequiredEnvParam } from "./environment";
@@ -598,90 +598,6 @@ export function asHTTPError(arg: any): HTTPError | undefined {
return undefined;
}
let cachedCodeQlVersion: undefined | VersionInfo = undefined;
/**
* Resets the in-process cache of the CodeQL CLI version. Only for use in tests,
* which exercise multiple "steps" within a single process.
*/
export function resetCachedCodeQlVersion(): void {
cachedCodeQlVersion = undefined;
}
/** The persisted version together with the CLI path it was obtained from. */
interface PersistedVersionInfo {
cmd: string;
version: VersionInfo;
}
function isVersionInfo(x: unknown): x is VersionInfo {
const candidate = x as Partial<VersionInfo> | null;
return (
typeof candidate === "object" &&
candidate !== null &&
typeof candidate.version === "string" &&
(candidate.features === undefined ||
(typeof candidate.features === "object" &&
candidate.features !== null)) &&
(candidate.overlayVersion === undefined ||
typeof candidate.overlayVersion === "number")
);
}
function isPersistedVersionInfo(x: unknown): x is PersistedVersionInfo {
const candidate = x as Partial<PersistedVersionInfo> | null;
return (
typeof candidate === "object" &&
candidate !== null &&
typeof candidate.cmd === "string" &&
isVersionInfo(candidate.version)
);
}
export function cacheCodeQlVersion(cmd: string, version: VersionInfo): void {
if (cachedCodeQlVersion !== undefined) {
throw new Error("cacheCodeQlVersion() should be called only once");
}
cachedCodeQlVersion = version;
// Persist the version so that subsequent Actions steps, which run in separate
// processes, can reuse it rather than invoking `codeql version` again. We
// record the CLI path so that a different step using a different CodeQL bundle
// doesn't pick up a stale version.
core.exportVariable(
EnvVar.CODEQL_VERSION_INFO,
JSON.stringify({ cmd, version }),
);
}
export function getCachedCodeQlVersion(cmd?: string): undefined | VersionInfo {
if (cachedCodeQlVersion !== undefined) {
return cachedCodeQlVersion;
}
// Fall back to the value persisted by an earlier Actions step, if any. This is
// best-effort: any malformed or mismatched value is ignored so that the caller
// invokes `codeql version` instead.
const serialized = process.env[EnvVar.CODEQL_VERSION_INFO];
if (!serialized) {
return undefined;
}
let persisted: unknown;
try {
persisted = JSON.parse(serialized);
} catch {
return undefined;
}
if (
!isPersistedVersionInfo(persisted) ||
(cmd !== undefined && persisted.cmd !== cmd)
) {
return undefined;
}
// Memoize the parsed value so that subsequent calls in this process don't
// re-parse the environment variable.
cachedCodeQlVersion = persisted.version;
return cachedCodeQlVersion;
}
export async function codeQlVersionAtLeast(
codeql: CodeQL,
requiredVersion: string,
@@ -926,9 +842,13 @@ export async function checkForTimeout() {
* directory with the name hostedtoolcache which is present on
* GitHub-hosted runners.
*
* @returns true iff the runner is hosted by GitHub
* Since this is a heuristic over how the runner happens to be named and laid out, it also matches
* self-hosted runners that are configured to resemble hosted ones. Prefer
* `isGitHubHostedRunner` when you need the answer the Actions service reports.
*
* @returns true iff the runner looks like it is hosted by GitHub
*/
export function isHostedRunner() {
export function looksLikeHostedRunner() {
return (
// Name of the runner on hosted Windows runners
process.env["RUNNER_NAME"]?.includes("Hosted Agent") ||
+1 -1
View File
@@ -29,6 +29,6 @@ outputs:
proxy_urls:
description: A stringified JSON array of objects containing the types and URLs of the configured registries.
runs:
using: node20
using: node24
main: "../lib/start-proxy-entry.js"
post: "../lib/start-proxy-post-entry.js"
+1 -1
View File
@@ -37,5 +37,5 @@
"@octokit/core/dist-types/types": ["./node_modules/@octokit/core/dist-types/types.d.ts"]
},
},
"exclude": ["node_modules", "pr-checks"]
"exclude": ["node_modules", "pr-checks", "scripts/changetool"]
}
+1 -1
View File
@@ -41,6 +41,6 @@ outputs:
{ "code-scanning": "some-id", "code-quality": "some-other-id" }
runs:
using: node20
using: node24
main: '../lib/upload-sarif-entry.js'
post: '../lib/upload-sarif-post-entry.js'