Compare commits

..
Author SHA1 Message Date
Michael B. Gale ba454b8ab4 Merge pull request #3284 from github/mbg/ci/fix-enterprise-workflow
Fix `update-supported-enterprise-server-versions.yml` workflow
2025-11-12 15:35:56 +00:00
Michael B. Gale 7a7cd8565c Don't push for PR event 2025-11-12 15:09:25 +00:00
Michael B. Gale fd830db27b Trigger on PR for relevant changes 2025-11-12 15:05:11 +00:00
Michael B. Gale a7e52b690b Perform sparse checkout 2025-11-12 15:04:21 +00:00
Michael B. Gale 71c3720f43 Run npm ci in update-supported-enterprise-server-versions.yml 2025-11-12 14:57:05 +00:00
Michael B. Gale 534824ea1b Merge pull request #3117 from github/mbg/csharp/new-cache-key-calculation
Support non-lock files for C# cache key computation
2025-11-12 11:03:09 +00:00
Henry Mercer 1d9f357d01 Merge pull request #3281 from github/dependabot/npm_and_yarn/npm-minor-9dd9c1a8e4
Bump the npm-minor group with 4 updates
2025-11-11 10:21:24 +00:00
github-actions[bot] 3d7be7bf78 Rebuild 2025-11-10 17:49:58 +00:00
dependabot[bot] 63bb415fff Bump the npm-minor group with 4 updates
Bumps the npm-minor group with 4 updates: [@eslint/js](https://github.com/eslint/eslint/tree/HEAD/packages/js), [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin), [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) and [esbuild](https://github.com/evanw/esbuild).


Updates `@eslint/js` from 9.39.0 to 9.39.1
- [Release notes](https://github.com/eslint/eslint/releases)
- [Commits](https://github.com/eslint/eslint/commits/v9.39.1/packages/js)

Updates `@typescript-eslint/eslint-plugin` from 8.46.3 to 8.46.4
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.46.4/packages/eslint-plugin)

Updates `@typescript-eslint/parser` from 8.46.3 to 8.46.4
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.46.4/packages/parser)

Updates `esbuild` from 0.25.12 to 0.27.0
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.25.12...v0.27.0)

---
updated-dependencies:
- dependency-name: "@eslint/js"
  dependency-version: 9.39.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm-minor
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-version: 8.46.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm-minor
- dependency-name: "@typescript-eslint/parser"
  dependency-version: 8.46.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm-minor
- dependency-name: esbuild
  dependency-version: 0.27.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: npm-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-10 17:48:25 +00:00
Michael B. Gale 04bd5c6aab Merge pull request #3279 from github/mbg/lint/jsdoc-param-names
Linter: check JSDoc parameter names exist
2025-11-10 11:36:02 +00:00
Michael B. Gale 48a56f6b93 Add some tests for downloadDependencyCaches related to feature prefixes 2025-11-09 12:03:18 +00:00
Michael B. Gale 4885eb2ad9 Insert new featurePrefix after general cache key prefix 2025-11-09 11:17:02 +00:00
Michael B. Gale a47d5507cf Restore earlier log messages for checkHashPatterns 2025-11-09 11:11:10 +00:00
Michael B. Gale b0e9dfce55 Restore missing status.push resulting from a bad merge 2025-11-09 11:10:27 +00:00
Michael B. Gale 35c91ef0af Add tests for getCsharpHashPatterns
- Make the function more easily testable by allowing `makePatternCheck` to be stubbed.
- Use `makePatternCheck` for base patterns as well.
2025-11-09 11:03:15 +00:00
Michael B. Gale 71abac76d2 Fix comment in getCsharpHashPatterns 2025-11-09 10:32:10 +00:00
Michael B. Gale 5b58b8f9c5 Linter: check JSDoc parameter names exist 2025-11-09 10:26:36 +00:00
Michael B. Gale 46e03b48bc Fix JSDoc param name 2025-11-09 10:05:18 +00:00
Henry Mercer 71d0a56d44 Merge pull request #3278 from github/henrymercer/type-fun
Use generic types for durations in status report
2025-11-06 10:47:44 +00:00
Henry Mercer 04285cbe85 Use generic types for durations in status report
This means we don't need to update this interface for every new language.
2025-11-05 18:54:44 +00:00
Michael B. Gale 26804552e4 Use undefined instead of NoMatchingFilesError
Add tests for `makePatternCheck` and `checkHashPatterns`
2025-11-05 17:23:22 +00:00
Michael B. Gale 03b2dc2a3f Add and use getFeaturePrefix for dependency caching 2025-11-05 16:33:21 +00:00
Michael B. Gale 0cbd930deb Move createCacheKeyHash to caching-utils 2025-11-05 16:15:26 +00:00
Michael B. Gale 0324490286 Use additional files for C# key hashes if Feature.CsharpNewCacheKey is enabled 2025-11-05 16:03:41 +00:00
Michael B. Gale 6b48207907 Move check whether there are files for hashing into getHashPatterns 2025-11-05 16:03:39 +00:00
Mario Campos 320a6b661b Merge pull request #3272 from github/mario-campos/v4-warning
Update deprecation warnings for CodeQL Action to v4
2025-11-05 10:01:33 -06:00
Michael B. Gale ab1c84236a Change hash to be a function that can use Features 2025-11-05 15:57:57 +00:00
Michael B. Gale 2a7680fca6 Change getDefaultCacheConfig to be a const by turning paths into a function
Changing `paths` to be a function is necessary to allow `getTemporaryDirectory` to be called
2025-11-05 15:57:55 +00:00
Michael B. Gale 2aa1f55f3d Propagate features into cachePrefix function 2025-11-05 15:54:28 +00:00
Michael B. Gale 1ca20ab026 Add CsharpNewCacheKey FF 2025-11-05 15:48:04 +00:00
Mario Campos b5e5a258e6 Merge branch 'main' into mario-campos/v4-warning 2025-11-05 09:39:54 -06:00
Mario Campos 74f662193b Reformat with eslint 2025-11-05 09:37:42 -06:00
Mario Campos ecee3ea8f5 Update CHANGELOG.md. 2025-11-05 09:18:30 -06:00
Mario Campos 6a63bc6af3 Change warning message to just v3 (exclude v1, v2). 2025-11-05 09:15:53 -06:00
Henry Mercer e2ef519c75 Merge pull request #3224 from github/henrymercer/clean-up-resolve-languages-ff
Clean up `resolve_supported_languages_using_cli` FF
2025-11-05 11:57:35 +00:00
Michael B. Gale 423d14e583 Merge pull request #3277 from github/mbg/ci/update-bundle-python
Install Python in `update-bundle` workflow
2025-11-05 11:53:17 +00:00
Michael B. GaleandCopilot 6dd11f73d3 Update .github/workflows/script/bundle_changelog.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-05 11:34:26 +00:00
Michael B. Gale c9f82f2adf Move python script to file 2025-11-05 11:22:38 +00:00
Michael B. Gale 33684ef869 Add setup-python step to update-bundle workflow 2025-11-05 11:20:06 +00:00
Michael B. Gale 5aa2d63d5b Merge branch 'main' into mario-campos/v4-warning 2025-11-05 11:03:25 +00:00
Michael B. Gale 862f5666b3 Merge pull request #3275 from github/mbg/checks/filter-ccr
Filter CCR jobs in `update-required-checks.sh`
2025-11-05 10:15:57 +00:00
Michael B. Gale b00addd1d3 Merge pull request #3274 from github/mbg/macos-13 2025-11-05 09:01:53 +00:00
Michael B. Gale d03fd76232 Filter CCR jobs in update-required-checks.sh 2025-11-04 22:23:12 +00:00
Michael B. Gale 9d5565fba2 Remove macos-13 from codeql workflow 2025-11-04 21:29:25 +00:00
Mario Campos a570795dfc Clarify the CHANGELOG.md entry to reflect the whole action, and not just init. 2025-11-04 12:02:15 -06:00
Mario CamposandCopilot 9366f80399 Reference GHES 3.20 in the comment, not 3.19.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-04 12:00:11 -06:00
Mario Campos c443dff433 Simplify warning language to not enumerate deprecated versions. 2025-11-04 11:57:44 -06:00
Mario Campos b32a1e0627 Update test cases for GitHub Enterprise Server versions 3.11 and 3.12 to reflect correct deprecation status 2025-11-04 11:51:23 -06:00
Mario Campos 08dc635f27 Restore use of sinon.match(). 2025-11-04 11:11:08 -06:00
Mario Campos f1ca6a4f47 Update tests to reflect deprecation status of CodeQL Action v3 2025-11-04 11:07:15 -06:00
Mario Campos ba82f9bd34 Fix deprecation warning to reflect that v3 is not actually deprecated yet. 2025-11-04 10:13:07 -06:00
Mario Campos 5a9b49de7e Update CHANGELOG to reflect warning for v3 users migrating to v4 of CodeQL Action 2025-11-04 10:09:47 -06:00
Mario Campos 1aade295bc Update deprecation warnings for CodeQL Action to v4 2025-11-04 09:59:16 -06:00
Henry Mercer 95b1867cf7 Merge pull request #3269 from github/dependabot/npm_and_yarn/actions/io-2.0.0
Bump @actions/io from 1.1.3 to 2.0.0
2025-11-04 14:23:27 +00:00
github-actions[bot] 238f5f2946 Rebuild 2025-11-04 14:00:06 +00:00
dependabot[bot] a53b4967d7 Bump @actions/io from 1.1.3 to 2.0.0
Bumps [@actions/io](https://github.com/actions/toolkit/tree/HEAD/packages/io) from 1.1.3 to 2.0.0.
- [Changelog](https://github.com/actions/toolkit/blob/main/packages/io/RELEASES.md)
- [Commits](https://github.com/actions/toolkit/commits/HEAD/packages/io)

---
updated-dependencies:
- dependency-name: "@actions/io"
  dependency-version: 2.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-04 13:58:17 +00:00
Henry Mercer 493ffd8e5c Merge pull request #3268 from github/dependabot/npm_and_yarn/types/archiver-7.0.0
Bump @types/archiver from 6.0.4 to 7.0.0
2025-11-04 13:56:51 +00:00
github-actions[bot] f23547cd26 Rebuild 2025-11-04 12:25:05 +00:00
dependabot[bot] 58f5e3dab8 Bump @types/archiver from 6.0.4 to 7.0.0
Bumps [@types/archiver](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/archiver) from 6.0.4 to 7.0.0.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/archiver)

---
updated-dependencies:
- dependency-name: "@types/archiver"
  dependency-version: 7.0.0
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-04 12:19:24 +00:00
Henry Mercer 98e0ffef96 Merge pull request #3266 from github/dependabot/npm_and_yarn/npm-minor-29e8df1594
Bump the npm-minor group with 7 updates
2025-11-04 11:09:56 +00:00
github-actions[bot] 85eb524170 Rebuild 2025-11-04 10:34:56 +00:00
dependabot[bot] a3ea4ef532 Bump the npm-minor group with 7 updates
Bumps the npm-minor group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [@octokit/request-error](https://github.com/octokit/request-error.js) | `7.0.1` | `7.0.2` |
| [octokit](https://github.com/octokit/octokit.js) | `5.0.4` | `5.0.5` |
| [@eslint/compat](https://github.com/eslint/rewrite/tree/HEAD/packages/compat) | `1.4.0` | `1.4.1` |
| [@eslint/js](https://github.com/eslint/eslint/tree/HEAD/packages/js) | `9.38.0` | `9.39.0` |
| [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) | `8.46.2` | `8.46.3` |
| [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) | `8.46.2` | `8.46.3` |
| [esbuild](https://github.com/evanw/esbuild) | `0.25.11` | `0.25.12` |


Updates `@octokit/request-error` from 7.0.1 to 7.0.2
- [Release notes](https://github.com/octokit/request-error.js/releases)
- [Commits](https://github.com/octokit/request-error.js/compare/v7.0.1...v7.0.2)

Updates `octokit` from 5.0.4 to 5.0.5
- [Release notes](https://github.com/octokit/octokit.js/releases)
- [Commits](https://github.com/octokit/octokit.js/compare/v5.0.4...v5.0.5)

Updates `@eslint/compat` from 1.4.0 to 1.4.1
- [Release notes](https://github.com/eslint/rewrite/releases)
- [Changelog](https://github.com/eslint/rewrite/blob/main/packages/compat/CHANGELOG.md)
- [Commits](https://github.com/eslint/rewrite/commits/compat-v1.4.1/packages/compat)

Updates `@eslint/js` from 9.38.0 to 9.39.0
- [Release notes](https://github.com/eslint/eslint/releases)
- [Commits](https://github.com/eslint/eslint/commits/v9.39.0/packages/js)

Updates `@typescript-eslint/eslint-plugin` from 8.46.2 to 8.46.3
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.46.3/packages/eslint-plugin)

Updates `@typescript-eslint/parser` from 8.46.2 to 8.46.3
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.46.3/packages/parser)

Updates `esbuild` from 0.25.11 to 0.25.12
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.25.11...v0.25.12)

---
updated-dependencies:
- dependency-name: "@octokit/request-error"
  dependency-version: 7.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: npm-minor
- dependency-name: octokit
  dependency-version: 5.0.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: npm-minor
- dependency-name: "@eslint/compat"
  dependency-version: 1.4.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm-minor
- dependency-name: "@eslint/js"
  dependency-version: 9.39.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: npm-minor
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-version: 8.46.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm-minor
- dependency-name: "@typescript-eslint/parser"
  dependency-version: 8.46.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm-minor
- dependency-name: esbuild
  dependency-version: 0.25.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-03 18:47:33 +00:00
Henry Mercer 97580d7533 Merge pull request #3267 from github/dependabot/npm_and_yarn/octokit/types-16.0.0
Bump @octokit/types from 15.0.1 to 16.0.0
2025-11-03 18:12:51 +00:00
Henry Mercer 7ad64f0258 Merge pull request #3270 from github/dependabot/npm_and_yarn/actions/http-client-3.0.0
Bump @actions/http-client from 2.2.3 to 3.0.0
2025-11-03 18:11:30 +00:00
github-actions[bot] 61bcb70dce Rebuild 2025-11-03 17:22:15 +00:00
github-actions[bot] 5313cd14a8 Rebuild 2025-11-03 17:21:50 +00:00
dependabot[bot] 8fa298d060 Bump @actions/http-client from 2.2.3 to 3.0.0
Bumps [@actions/http-client](https://github.com/actions/toolkit/tree/HEAD/packages/http-client) from 2.2.3 to 3.0.0.
- [Changelog](https://github.com/actions/toolkit/blob/main/packages/http-client/RELEASES.md)
- [Commits](https://github.com/actions/toolkit/commits/HEAD/packages/http-client)

---
updated-dependencies:
- dependency-name: "@actions/http-client"
  dependency-version: 3.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-03 17:20:38 +00:00
dependabot[bot] 7d1bed2dd0 Bump @octokit/types from 15.0.1 to 16.0.0
Bumps [@octokit/types](https://github.com/octokit/types.ts) from 15.0.1 to 16.0.0.
- [Release notes](https://github.com/octokit/types.ts/releases)
- [Commits](https://github.com/octokit/types.ts/compare/v15.0.1...v16.0.0)

---
updated-dependencies:
- dependency-name: "@octokit/types"
  dependency-version: 16.0.0
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-03 17:20:12 +00:00
Michael B. Gale 338146ca93 Merge pull request #3264 from github/mbg/ci/publish-on-tag
Create immutable action version on tag push
2025-10-31 16:48:43 +00:00
Michael B. Gale 64db1da706 Create immutable action version on tag push 2025-10-31 16:24:23 +00:00
Henry Mercer 9bd8638576 Merge pull request #3262 from github/mergeback/v4.31.2-to-main-0499de31
Mergeback v4.31.2 refs/heads/releases/v4 into main
2025-10-30 15:07:33 +00:00
github-actions[bot] 65da12b256 Rebuild 2025-10-30 14:36:57 +00:00
github-actions[bot] c228fecc25 Update changelog and version after v4.31.2 2025-10-30 14:33:10 +00:00
Henry Mercer a691497d40 Clean up resolve_supported_languages_using_cli FF 2025-10-22 17:04:42 +01:00
46 changed files with 59518 additions and 1546 deletions

No files matched your search

+1 -1
View File
@@ -16,5 +16,5 @@ inputs:
Comma separated list of query ids that should NOT be included in this SARIF file.
runs:
using: node20
using: node24
main: index.js
+1 -1
View File
@@ -81,7 +81,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-22.04,ubuntu-24.04,windows-2022,windows-2025,macos-13,macos-14,macos-15]
os: [ubuntu-22.04,ubuntu-24.04,windows-2022,windows-2025,macos-14,macos-15]
tools: ${{ fromJson(needs.check-codeql-versions.outputs.versions) }}
runs-on: ${{ matrix.os }}
+7 -19
View File
@@ -1,8 +1,10 @@
name: 'Publish Immutable Action Version'
on:
release:
types: [published]
push:
tags:
# Match version tags, but not the major version tags.
- 'v[0-9]+.**'
defaults:
run:
@@ -17,23 +19,9 @@ jobs:
packages: write
steps:
- name: Check release name
id: check
env:
RELEASE_NAME: ${{ github.event.release.name }}
run: |
echo "Release name: ${{ github.event.release.name }}"
if [[ $RELEASE_NAME == v* ]]; then
echo "This is a CodeQL Action release. Create an Immutable Action"
echo "is-action-release=true" >> $GITHUB_OUTPUT
else
echo "This is a CodeQL Bundle release. Do not create an Immutable Action"
echo "is-action-release=false" >> $GITHUB_OUTPUT
fi
- name: Checking out
if: steps.check.outputs.is-action-release == 'true'
- name: Checkout repository
uses: actions/checkout@v5
- name: Publish
if: steps.check.outputs.is-action-release == 'true'
- name: Publish immutable release
id: publish
uses: actions/publish-immutable-action@v0.0.4
@@ -0,0 +1,18 @@
import os
import re
# Get the PR number from the PR URL.
pr_number = os.environ['PR_URL'].split('/')[-1]
changelog_note = f"- Update default CodeQL bundle version to {os.environ['CLI_VERSION']}. [#{pr_number}]({os.environ['PR_URL']})"
# If the "[UNRELEASED]" section starts with "no user facing changes", remove that line.
with open('CHANGELOG.md', 'r') as f:
changelog = f.read()
changelog = changelog.replace('## [UNRELEASED]\n\nNo user facing changes.', '## [UNRELEASED]\n')
# Add the changelog note to the bottom of the "[UNRELEASED]" section.
changelog = re.sub(r'\n## (\d+\.\d+\.\d+)', f'{changelog_note}\n\n## \\1', changelog, count=1)
with open('CHANGELOG.md', 'w') as f:
f.write(changelog)
@@ -29,7 +29,7 @@ fi
echo "Getting checks for $GITHUB_SHA"
# Ignore any checks with "https://", CodeQL, LGTM, Update, and ESLint checks.
CHECKS="$(gh api repos/github/codeql-action/commits/"${GITHUB_SHA}"/check-runs --paginate | jq --slurp --compact-output --raw-output '[.[].check_runs.[] | select(.conclusion != "skipped") | .name | select(contains("https://") or . == "CodeQL" or . == "Dependabot" or . == "check-expected-release-files" or contains("Update") or contains("ESLint") or contains("update") or contains("test-setup-python-scripts") | not)] | unique | sort')"
CHECKS="$(gh api repos/github/codeql-action/commits/"${GITHUB_SHA}"/check-runs --paginate | jq --slurp --compact-output --raw-output '[.[].check_runs.[] | select(.conclusion != "skipped") | .name | select(contains("https://") or . == "CodeQL" or . == "Dependabot" or . == "check-expected-release-files" or contains("Update") or contains("ESLint") or contains("update") or contains("test-setup-python-scripts") or . == "Agent" or . == "Cleanup artifacts" or . == "Prepare" or . == "Upload results" | not)] | unique | sort')"
echo "$CHECKS" | jq
+6 -21
View File
@@ -40,6 +40,11 @@ jobs:
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config --global user.name "github-actions[bot]"
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.12'
- name: Set up Node.js
uses: actions/setup-node@v6
with:
@@ -78,28 +83,8 @@ jobs:
echo "PR_URL=$pr_url" | tee -a "$GITHUB_ENV"
- name: Create changelog note
shell: python
run: |
import os
import re
# Get the PR number from the PR URL.
pr_number = os.environ['PR_URL'].split('/')[-1]
changelog_note = f"- Update default CodeQL bundle version to {os.environ['CLI_VERSION']}. [#{pr_number}]({os.environ['PR_URL']})"
# If the "[UNRELEASED]" section starts with "no user facing changes", remove that line.
# Use perl to avoid having to escape the newline character.
with open('CHANGELOG.md', 'r') as f:
changelog = f.read()
changelog = changelog.replace('## [UNRELEASED]\n\nNo user facing changes.', '## [UNRELEASED]\n')
# Add the changelog note to the bottom of the "[UNRELEASED]" section.
changelog = re.sub(r'\n## (\d+\.\d+\.\d+)', f'{changelog_note}\n\n## \\1', changelog, count=1)
with open('CHANGELOG.md', 'w') as f:
f.write(changelog)
python .github/workflows/script/bundle_changelog.py
- name: Push changelog note
run: |
@@ -4,6 +4,12 @@ on:
schedule:
- cron: "0 0 * * *"
workflow_dispatch:
pull_request:
branches:
- main
paths:
- .github/workflows/update-supported-enterprise-server-versions.yml
- .github/workflows/update-supported-enterprise-server-versions/update.py
jobs:
update-supported-enterprise-server-versions:
@@ -28,6 +34,7 @@ jobs:
repository: github/enterprise-releases
token: ${{ secrets.ENTERPRISE_RELEASE_TOKEN }}
path: ${{ github.workspace }}/enterprise-releases/
sparse-checkout: releases.json
- name: Update Supported Enterprise Server Versions
run: |
cd ./.github/workflows/update-supported-enterprise-server-versions/
@@ -35,6 +42,7 @@ jobs:
pipenv install
pipenv run ./update.py
rm --recursive "$ENTERPRISE_RELEASES_PATH"
npm ci
npm run build
env:
ENTERPRISE_RELEASES_PATH: ${{ github.workspace }}/enterprise-releases/
@@ -44,25 +52,33 @@ jobs:
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config --global user.name "github-actions[bot]"
- name: Commit changes and open PR
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Commit changes
id: prepare-commit
run: |
if [[ -z $(git status --porcelain) ]]; then
echo "No changes to commit"
echo "committed=false" >> $GITHUB_OUTPUT
else
git checkout -b update-supported-enterprise-server-versions
git add .
git commit --message "Update supported GitHub Enterprise Server versions"
git push origin update-supported-enterprise-server-versions
body="This PR updates the list of supported GitHub Enterprise Server versions, either because a new "
body+="version is about to be feature frozen, or because an old release has been deprecated."
body+=$'\n\n'
body+="If an old release has been deprecated, please follow the instructions in CONTRIBUTING.md to "
body+="deprecate the corresponding version of CodeQL."
gh pr create --draft \
--title "Update supported GitHub Enterprise Server versions" \
--body "$body"
echo "committed=true" >> $GITHUB_OUTPUT
fi
- name: Open PR
if: github.event_name != 'pull_request' && steps.prepare-commit.outputs.committed == 'true'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
git push origin update-supported-enterprise-server-versions
body="This PR updates the list of supported GitHub Enterprise Server versions, either because a new "
body+="version is about to be feature frozen, or because an old release has been deprecated."
body+=$'\n\n'
body+="If an old release has been deprecated, please follow the instructions in CONTRIBUTING.md to "
body+="deprecate the corresponding version of CodeQL."
gh pr create --draft \
--title "Update supported GitHub Enterprise Server versions" \
--body "$body"
+18 -7
View File
@@ -2,31 +2,35 @@
See the [releases page](https://github.com/github/codeql-action/releases) for the relevant changes to the CodeQL CLI and language packs.
## 3.31.2 - 30 Oct 2025
## [UNRELEASED]
- CodeQL Action v3 will be deprecated in December 2026. The Action now logs a warning for customers who are running v3 but could be running v4. For more information, see [Upcoming deprecation of CodeQL Action v3](https://github.blog/changelog/2025-10-28-upcoming-deprecation-of-codeql-action-v3/).
## 4.31.2 - 30 Oct 2025
No user facing changes.
## 3.31.1 - 30 Oct 2025
## 4.31.1 - 30 Oct 2025
- The `add-snippets` input has been removed from the `analyze` action. This input has been deprecated since CodeQL Action 3.26.4 in August 2024 when this removal was announced.
## 3.31.0 - 24 Oct 2025
## 4.31.0 - 24 Oct 2025
- Bump minimum CodeQL bundle version to 2.17.6. [#3223](https://github.com/github/codeql-action/pull/3223)
- When SARIF files are uploaded by the `analyze` or `upload-sarif` actions, the CodeQL Action automatically performs post-processing steps to prepare the data for the upload. Previously, these post-processing steps were only performed before an upload took place. We are now changing this so that the post-processing steps will always be performed, even when the SARIF files are not uploaded. This does not change anything for the `upload-sarif` action. For `analyze`, this may affect Advanced Setup for CodeQL users who specify a value other than `always` for the `upload` input. [#3222](https://github.com/github/codeql-action/pull/3222)
## 3.30.9 - 17 Oct 2025
## 4.30.9 - 17 Oct 2025
- Update default CodeQL bundle version to 2.23.3. [#3205](https://github.com/github/codeql-action/pull/3205)
- Experimental: A new `setup-codeql` action has been added which is similar to `init`, except it only installs the CodeQL CLI and does not initialize a database. Do not use this in production as it is part of an internal experiment and subject to change at any time. [#3204](https://github.com/github/codeql-action/pull/3204)
## 3.30.8 - 10 Oct 2025
## 4.30.8 - 10 Oct 2025
No user facing changes.
## 3.30.7 - 06 Oct 2025
## 4.30.7 - 06 Oct 2025
No user facing changes.
- [v4+ only] The CodeQL Action now runs on Node.js v24. [#3169](https://github.com/github/codeql-action/pull/3169)
## 3.30.6 - 02 Oct 2025
@@ -262,13 +266,17 @@ No user facing changes.
## 3.26.12 - 07 Oct 2024
- _Upcoming breaking change_: Add a deprecation warning for customers using CodeQL version 2.14.5 and earlier. These versions of CodeQL were discontinued on 24 September 2024 alongside GitHub Enterprise Server 3.10, and will be unsupported by CodeQL Action versions 3.27.0 and later and versions 2.27.0 and later. [#2520](https://github.com/github/codeql-action/pull/2520)
- If you are using one of these versions, please update to CodeQL CLI version 2.14.6 or later. For instance, if you have specified a custom version of the CLI using the 'tools' input to the 'init' Action, you can remove this input to use the default version.
- Alternatively, if you want to continue using a version of the CodeQL CLI between 2.13.5 and 2.14.5, you can replace `github/codeql-action/*@v3` by `github/codeql-action/*@v3.26.11` and `github/codeql-action/*@v2` by `github/codeql-action/*@v2.26.11` in your code scanning workflow to ensure you continue using this version of the CodeQL Action.
## 3.26.11 - 03 Oct 2024
- _Upcoming breaking change_: Add support for using `actions/download-artifact@v4` to programmatically consume CodeQL Action debug artifacts.
Starting November 30, 2024, GitHub.com customers will [no longer be able to use `actions/download-artifact@v3`](https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/). Therefore, to avoid breakage, customers who programmatically download the CodeQL Action debug artifacts should set the `CODEQL_ACTION_ARTIFACT_V4_UPGRADE` environment variable to `true` and bump `actions/download-artifact@v3` to `actions/download-artifact@v4` in their workflows. The CodeQL Action will enable this behavior by default in early November and workflows that have not yet bumped `actions/download-artifact@v3` to `actions/download-artifact@v4` will begin failing then.
This change is currently unavailable for GitHub Enterprise Server customers, as `actions/upload-artifact@v4` and `actions/download-artifact@v4` are not yet compatible with GHES.
- Update default CodeQL bundle version to 2.19.1. [#2519](https://github.com/github/codeql-action/pull/2519)
@@ -391,9 +399,12 @@ No user facing changes.
## 3.25.0 - 15 Apr 2024
- The deprecated feature for extracting dependencies for a Python analysis has been removed. [#2224](https://github.com/github/codeql-action/pull/2224)
As a result, the following inputs and environment variables are now ignored:
- The `setup-python-dependencies` input to the `init` Action
- The `CODEQL_ACTION_DISABLE_PYTHON_DEPENDENCY_INSTALLATION` environment variable
We recommend removing any references to these from your workflows. For more information, see the release notes for CodeQL Action v3.23.0 and v2.23.0.
- Automatically overwrite an existing database if found on the filesystem. [#2229](https://github.com/github/codeql-action/pull/2229)
- Bump the minimum CodeQL bundle version to 2.12.6. [#2232](https://github.com/github/codeql-action/pull/2232)
+1 -1
View File
@@ -94,6 +94,6 @@ outputs:
sarif-id:
description: The ID of the uploaded SARIF file.
runs:
using: node20
using: node24
main: "../lib/analyze-action.js"
post: "../lib/analyze-action-post.js"
+1 -1
View File
@@ -15,5 +15,5 @@ inputs:
$GITHUB_WORKSPACE as its working directory.
required: false
runs:
using: node20
using: node24
main: '../lib/autobuild-action.js'
+12
View File
@@ -12,6 +12,7 @@ import filenames from "eslint-plugin-filenames";
import github from "eslint-plugin-github";
import _import from "eslint-plugin-import";
import noAsyncForeach from "eslint-plugin-no-async-foreach";
import jsdoc from "eslint-plugin-jsdoc";
import globals from "globals";
const __filename = fileURLToPath(import.meta.url);
@@ -52,6 +53,7 @@ export default [
github: fixupPluginRules(github),
import: fixupPluginRules(_import),
"no-async-foreach": noAsyncForeach,
"jsdoc": jsdoc,
},
languageOptions: {
@@ -133,6 +135,16 @@ export default [
"@typescript-eslint/no-shadow": "error",
"@typescript-eslint/prefer-optional-chain": "error",
"one-var": ["error", "never"],
// Check param names to ensure that we don't have outdated JSDocs.
"jsdoc/check-param-names": [
"error",
{
// We don't currently require full JSDoc coverage, so this rule
// should not error on missing @param annotations.
disableMissingParamChecks: true,
}
],
},
},
{
+1 -1
View File
@@ -165,6 +165,6 @@ outputs:
codeql-version:
description: The version of the CodeQL binary used for analysis
runs:
using: node20
using: node24
main: '../lib/init-action.js'
post: '../lib/init-action-post.js'
+5936 -56
View File
File diff suppressed because it is too large. Load diff
+4454 -117
View File
File diff suppressed because it is too large. Load diff
+4319 -49
View File
File diff suppressed because it is too large. Load diff
+5938 -58
View File
File diff suppressed because it is too large. Load diff
+4457 -126
View File
File diff suppressed because it is too large. Load diff
+4319 -49
View File
File diff suppressed because it is too large. Load diff
+4335 -65
View File
File diff suppressed because it is too large. Load diff
+5936 -56
View File
File diff suppressed because it is too large. Load diff
+3553 -46
View File
File diff suppressed because it is too large. Load diff
+4315 -45
View File
File diff suppressed because it is too large. Load diff
+5936 -56
View File
File diff suppressed because it is too large. Load diff
+4336 -66
View File
File diff suppressed because it is too large. Load diff
+758 -379
View File
File diff suppressed because it is too large. Load diff
+12 -11
View File
@@ -1,6 +1,6 @@
{
"name": "codeql",
"version": "3.31.2",
"version": "4.31.3",
"private": true,
"description": "CodeQL action",
"scripts": {
@@ -31,11 +31,11 @@
"@actions/exec": "^1.1.1",
"@actions/github": "^6.0.0",
"@actions/glob": "^0.5.0",
"@actions/http-client": "^2.2.3",
"@actions/io": "^1.1.3",
"@actions/http-client": "^3.0.0",
"@actions/io": "^2.0.0",
"@actions/tool-cache": "^2.0.2",
"@octokit/plugin-retry": "^6.0.0",
"@octokit/request-error": "^7.0.1",
"@octokit/request-error": "^7.0.2",
"@schemastore/package": "0.0.10",
"archiver": "^7.0.1",
"fast-deep-equal": "^3.1.3",
@@ -45,33 +45,34 @@
"jsonschema": "1.4.1",
"long": "^5.3.2",
"node-forge": "^1.3.1",
"octokit": "^5.0.4",
"octokit": "^5.0.5",
"semver": "^7.7.3",
"uuid": "^13.0.0"
},
"devDependencies": {
"@ava/typescript": "6.0.0",
"@eslint/compat": "^1.4.0",
"@eslint/compat": "^1.4.1",
"@eslint/eslintrc": "^3.3.1",
"@eslint/js": "^9.38.0",
"@eslint/js": "^9.39.1",
"@microsoft/eslint-formatter-sarif": "^3.1.0",
"@octokit/types": "^15.0.1",
"@types/archiver": "^6.0.4",
"@octokit/types": "^16.0.0",
"@types/archiver": "^7.0.0",
"@types/follow-redirects": "^1.14.4",
"@types/js-yaml": "^4.0.9",
"@types/node": "20.19.9",
"@types/node-forge": "^1.3.14",
"@types/semver": "^7.7.1",
"@types/sinon": "^17.0.4",
"@typescript-eslint/eslint-plugin": "^8.46.2",
"@typescript-eslint/eslint-plugin": "^8.46.4",
"@typescript-eslint/parser": "^8.41.0",
"ava": "^6.4.1",
"esbuild": "^0.25.11",
"esbuild": "^0.27.0",
"eslint": "^8.57.1",
"eslint-import-resolver-typescript": "^3.8.7",
"eslint-plugin-filenames": "^1.3.2",
"eslint-plugin-github": "^5.1.8",
"eslint-plugin-import": "2.29.1",
"eslint-plugin-jsdoc": "^61.1.12",
"eslint-plugin-no-async-foreach": "^0.1.1",
"glob": "^11.0.3",
"nock": "^14.0.10",
+1 -1
View File
@@ -21,5 +21,5 @@ outputs:
environment:
description: The inferred build environment configuration.
runs:
using: node20
using: node24
main: '../lib/resolve-environment-action.js'
+1 -1
View File
@@ -35,5 +35,5 @@ outputs:
codeql-version:
description: The version of the CodeQL binary that was installed.
runs:
using: node20
using: node24
main: '../lib/setup-codeql-action.js'
+2 -5
View File
@@ -438,14 +438,11 @@ async function run() {
// Store dependency cache(s) if dependency caching is enabled.
if (shouldStoreCache(config.dependencyCachingEnabled)) {
const minimizeJavaJars = await features.getValue(
Feature.JavaMinimizeDependencyJars,
codeql,
);
dependencyCacheResults = await uploadDependencyCaches(
codeql,
features,
config,
logger,
minimizeJavaJars,
);
}
+18 -81
View File
@@ -38,89 +38,26 @@ export class CodeQLAnalysisError extends Error {
}
}
export interface QueriesStatusReport {
/**
* Time taken in ms to run queries for actions (or undefined if this language was not analyzed).
*
* The "builtin" designation is now outdated with the move to CLI config parsing: this is the time
* taken to run _all_ the queries.
*/
analyze_builtin_queries_actions_duration_ms?: number;
/**
* Time taken in ms to run queries for cpp (or undefined if this language was not analyzed).
*
* The "builtin" designation is now outdated with the move to CLI config parsing: this is the time
* taken to run _all_ the queries.
*/
analyze_builtin_queries_cpp_duration_ms?: number;
/**
* Time taken in ms to run queries for csharp (or undefined if this language was not analyzed).
*
* The "builtin" designation is now outdated with the move to CLI config parsing: this is the time
* taken to run _all_ the queries.
*/
analyze_builtin_queries_csharp_duration_ms?: number;
/**
* Time taken in ms to run queries for go (or undefined if this language was not analyzed).
*
* The "builtin" designation is now outdated with the move to CLI config parsing: this is the time
* taken to run _all_ the queries.
*/
analyze_builtin_queries_go_duration_ms?: number;
/**
* Time taken in ms to run queries for java (or undefined if this language was not analyzed).
*
* The "builtin" designation is now outdated with the move to CLI config parsing: this is the time
* taken to run _all_ the queries.
*/
analyze_builtin_queries_java_duration_ms?: number;
/**
* Time taken in ms to run queries for javascript (or undefined if this language was not analyzed).
*
* The "builtin" designation is now outdated with the move to CLI config parsing: this is the time
* taken to run _all_ the queries.
*/
analyze_builtin_queries_javascript_duration_ms?: number;
/**
* Time taken in ms to run queries for python (or undefined if this language was not analyzed).
*
* The "builtin" designation is now outdated with the move to CLI config parsing: this is the time
* taken to run _all_ the queries.
*/
analyze_builtin_queries_python_duration_ms?: number;
/**
* Time taken in ms to run queries for ruby (or undefined if this language was not analyzed).
*
* The "builtin" designation is now outdated with the move to CLI config parsing: this is the time
* taken to run _all_ the queries.
*/
analyze_builtin_queries_ruby_duration_ms?: number;
/** Time taken in ms to run queries for swift (or undefined if this language was not analyzed).
*
* The "builtin" designation is now outdated with the move to CLI config parsing: this is the time
* taken to run _all_ the queries.
*/
analyze_builtin_queries_swift_duration_ms?: number;
type KnownLanguageKey = keyof typeof KnownLanguage;
/** Time taken in ms to interpret results for actions (or undefined if this language was not analyzed). */
interpret_results_actions_duration_ms?: number;
/** Time taken in ms to interpret results for cpp (or undefined if this language was not analyzed). */
interpret_results_cpp_duration_ms?: number;
/** Time taken in ms to interpret results for csharp (or undefined if this language was not analyzed). */
interpret_results_csharp_duration_ms?: number;
/** Time taken in ms to interpret results for go (or undefined if this language was not analyzed). */
interpret_results_go_duration_ms?: number;
/** Time taken in ms to interpret results for java (or undefined if this language was not analyzed). */
interpret_results_java_duration_ms?: number;
/** Time taken in ms to interpret results for javascript (or undefined if this language was not analyzed). */
interpret_results_javascript_duration_ms?: number;
/** Time taken in ms to interpret results for python (or undefined if this language was not analyzed). */
interpret_results_python_duration_ms?: number;
/** Time taken in ms to interpret results for ruby (or undefined if this language was not analyzed). */
interpret_results_ruby_duration_ms?: number;
/** Time taken in ms to interpret results for swift (or undefined if this language was not analyzed). */
interpret_results_swift_duration_ms?: number;
type RunQueriesDurationStatusReport = {
/**
* Time taken in ms to run queries for the language (or undefined if this language was not analyzed).
*
* The "builtin" designation is now outdated with the move to CLI config parsing: this is the time
* taken to run _all_ the queries.
*/
[L in KnownLanguageKey as `analyze_builtin_queries_${L}_duration_ms`]?: number;
};
type InterpretResultsDurationStatusReport = {
/** Time taken in ms to interpret results for the language (or undefined if this language was not analyzed). */
[L in KnownLanguageKey as `interpret_results_${L}_duration_ms`]?: number;
};
export interface QueriesStatusReport
extends RunQueriesDurationStatusReport,
InterpretResultsDurationStatusReport {
/**
* Whether the analysis is diff-informed (in the sense that the action generates a diff-range data
* extension for the analysis, regardless of whether the data extension is actually used by queries).
+29
View File
@@ -1,3 +1,5 @@
import * as crypto from "crypto";
import * as core from "@actions/core";
import { getOptionalInput, isDefaultSetup } from "./actions-util";
@@ -71,6 +73,33 @@ export function getCachingKind(input: string | undefined): CachingKind {
}
}
// The length to which `createCacheKeyHash` truncates hash strings.
export const cacheKeyHashLength = 16;
/**
* Creates a SHA-256 hash of the cache key components to ensure uniqueness
* while keeping the cache key length manageable.
*
* @param components Object containing all components that should influence cache key uniqueness
* @returns A short SHA-256 hash (first 16 characters) of the components
*/
export function createCacheKeyHash(components: Record<string, any>): string {
// From https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify
//
// "Properties are visited using the same algorithm as Object.keys(), which
// has a well-defined order and is stable across implementations. For example,
// JSON.stringify on the same object will always produce the same string, and
// JSON.parse(JSON.stringify(obj)) would produce an object with the same key
// ordering as the original (assuming the object is completely
// JSON-serializable)."
const componentsJson = JSON.stringify(components);
return crypto
.createHash("sha256")
.update(componentsJson)
.digest("hex")
.substring(0, cacheKeyHashLength);
}
/** Determines whether dependency caching is enabled. */
export function getDependencyCachingEnabled(): CachingKind {
// If the workflow specified something always respect that
+8 -4
View File
@@ -1071,8 +1071,11 @@ export async function getCodeQLForCmd(
/**
* Gets the options for `path` of `options` as an array of extra option strings.
*
* @param ignoringOptions Options that should be ignored, for example because they have already
* been passed and it is an error to pass them more than once.
* @param paths The CLI command components to get extra options for.
* @param args Additional arguments for this function.
* @param args.ignoringOptions
* Options that should be ignored, for example because they have already
* been passed and it is an error to pass them more than once.
*/
function getExtraOptionsFromEnv(
paths: string[],
@@ -1154,8 +1157,9 @@ async function runCli(
/**
* Writes the code scanning configuration that is to be used by the CLI.
*
* @param codeql The CodeQL object to use.
* @param config The CodeQL Action state to use.
* @param config The CodeQL Action state to write.
* @param logger The logger to use.
*
* @returns The path to the generated user configuration file.
*/
async function writeCodeScanningConfigFile(
+55 -64
View File
@@ -873,71 +873,62 @@ const mockRepositoryNwo = parseRepositoryNwo("owner/repo");
expectedLanguages: ["javascript"],
},
].forEach((args) => {
for (const resolveSupportedLanguagesUsingCli of [true, false]) {
test(`getLanguages${resolveSupportedLanguagesUsingCli ? " (supported languages via CLI)" : ""}: ${args.name}`, async (t) => {
const features = createFeatures(
resolveSupportedLanguagesUsingCli
? [Feature.ResolveSupportedLanguagesUsingCli]
: [],
);
const mockRequest = mockLanguagesInRepo(args.languagesInRepository);
const stubExtractorEntry = {
extractor_root: "",
};
const codeQL = createStubCodeQL({
betterResolveLanguages: (options) =>
Promise.resolve({
aliases: {
"c#": KnownLanguage.csharp,
c: KnownLanguage.cpp,
kotlin: KnownLanguage.java,
typescript: KnownLanguage.javascript,
},
extractors: {
cpp: [stubExtractorEntry],
csharp: [stubExtractorEntry],
java: [stubExtractorEntry],
javascript: [stubExtractorEntry],
python: [stubExtractorEntry],
...(options?.filterToLanguagesWithQueries
? {}
: {
html: [stubExtractorEntry],
}),
},
}),
});
if (args.expectedLanguages) {
// happy path
const actualLanguages = await configUtils.getLanguages(
codeQL,
args.languagesInput,
mockRepositoryNwo,
".",
features,
mockLogger,
);
t.deepEqual(actualLanguages.sort(), args.expectedLanguages.sort());
} else {
// there is an error
await t.throwsAsync(
async () =>
await configUtils.getLanguages(
codeQL,
args.languagesInput,
mockRepositoryNwo,
".",
features,
mockLogger,
),
{ message: args.expectedError },
);
}
t.deepEqual(mockRequest.called, args.expectedApiCall);
test(`getLanguages: ${args.name}`, async (t) => {
const mockRequest = mockLanguagesInRepo(args.languagesInRepository);
const stubExtractorEntry = {
extractor_root: "",
};
const codeQL = createStubCodeQL({
betterResolveLanguages: (options) =>
Promise.resolve({
aliases: {
"c#": KnownLanguage.csharp,
c: KnownLanguage.cpp,
kotlin: KnownLanguage.java,
typescript: KnownLanguage.javascript,
},
extractors: {
cpp: [stubExtractorEntry],
csharp: [stubExtractorEntry],
java: [stubExtractorEntry],
javascript: [stubExtractorEntry],
python: [stubExtractorEntry],
...(options?.filterToLanguagesWithQueries
? {}
: {
html: [stubExtractorEntry],
}),
},
}),
});
}
if (args.expectedLanguages) {
// happy path
const actualLanguages = await configUtils.getLanguages(
codeQL,
args.languagesInput,
mockRepositoryNwo,
".",
mockLogger,
);
t.deepEqual(actualLanguages.sort(), args.expectedLanguages.sort());
} else {
// there is an error
await t.throwsAsync(
async () =>
await configUtils.getLanguages(
codeQL,
args.languagesInput,
mockRepositoryNwo,
".",
mockLogger,
),
{ message: args.expectedError },
);
}
t.deepEqual(mockRequest.called, args.expectedApiCall);
});
});
for (const { displayName, language, feature } of [
+4 -8
View File
@@ -34,6 +34,7 @@ import {
OverlayDatabaseMode,
} from "./overlay-database-utils";
import { RepositoryNwo } from "./repository";
import { ToolsFeature } from "./tools-features";
import { downloadTrapCaches } from "./trap-caching";
import {
GitHubVersion,
@@ -177,12 +178,10 @@ export interface Config {
export async function getSupportedLanguageMap(
codeql: CodeQL,
features: FeatureEnablement,
logger: Logger,
): Promise<Record<string, string>> {
const resolveSupportedLanguagesUsingCli = await features.getValue(
Feature.ResolveSupportedLanguagesUsingCli,
codeql,
const resolveSupportedLanguagesUsingCli = await codeql.supportsFeature(
ToolsFeature.BuiltinExtractorsSpecifyDefaultQueries,
);
const resolveResult = await codeql.betterResolveLanguages({
filterToLanguagesWithQueries: resolveSupportedLanguagesUsingCli,
@@ -283,7 +282,6 @@ export async function getLanguages(
languagesInput: string | undefined,
repository: RepositoryNwo,
sourceRoot: string,
features: FeatureEnablement,
logger: Logger,
): Promise<Language[]> {
// Obtain languages without filtering them.
@@ -294,7 +292,7 @@ export async function getLanguages(
logger,
);
const languageMap = await getSupportedLanguageMap(codeql, features, logger);
const languageMap = await getSupportedLanguageMap(codeql, logger);
const languagesSet = new Set<Language>();
const unknownLanguages: string[] = [];
@@ -431,7 +429,6 @@ export async function initActionState(
languagesInput,
repository,
sourceRoot,
features,
logger,
);
@@ -1036,7 +1033,6 @@ export async function getConfig(
* pack.
*
* @param registriesInput The value of the `registries` input.
* @param codeQL a codeQL object, used only for checking the version of CodeQL.
* @param tempDir a temporary directory to store the generated qlconfig.yml file.
* @param logger a logger object.
* @returns The path to the generated `qlconfig.yml` file and the auth tokens to
-1
View File
@@ -160,7 +160,6 @@ const PACK_IDENTIFIER_PATTERN = (function () {
* Version and path are optional.
*
* @param packStr the package specification to verify.
* @param configFile Config file to use for error reporting
*/
export function parsePacksSpecification(packStr: string): Pack {
if (typeof packStr !== "string") {
+389
View File
@@ -0,0 +1,389 @@
import * as fs from "fs";
import path from "path";
import * as actionsCache from "@actions/cache";
import * as glob from "@actions/glob";
import test from "ava";
import * as sinon from "sinon";
import { cacheKeyHashLength } from "./caching-utils";
import { createStubCodeQL } from "./codeql";
import {
CacheConfig,
checkHashPatterns,
getCsharpHashPatterns,
getFeaturePrefix,
makePatternCheck,
internal,
CSHARP_BASE_PATTERNS,
CSHARP_EXTRA_PATTERNS,
downloadDependencyCaches,
CacheHitKind,
cacheKey,
} from "./dependency-caching";
import { Feature } from "./feature-flags";
import { KnownLanguage } from "./languages";
import {
setupTests,
createFeatures,
getRecordingLogger,
checkExpectedLogMessages,
LoggedMessage,
} from "./testing-utils";
import { withTmpDir } from "./util";
setupTests(test);
function makeAbsolutePatterns(tmpDir: string, patterns: string[]): string[] {
return patterns.map((pattern) => path.join(tmpDir, pattern));
}
test("makePatternCheck - returns undefined if no patterns match", async (t) => {
await withTmpDir(async (tmpDir) => {
fs.writeFileSync(path.join(tmpDir, "test.java"), "");
const result = await makePatternCheck(
makeAbsolutePatterns(tmpDir, ["**/*.cs"]),
);
t.is(result, undefined);
});
});
test("makePatternCheck - returns all patterns if any pattern matches", async (t) => {
await withTmpDir(async (tmpDir) => {
fs.writeFileSync(path.join(tmpDir, "test.java"), "");
const patterns = makeAbsolutePatterns(tmpDir, ["**/*.cs", "**/*.java"]);
const result = await makePatternCheck(patterns);
t.deepEqual(result, patterns);
});
});
test("getCsharpHashPatterns - returns base patterns if any pattern matches", async (t) => {
const codeql = createStubCodeQL({});
const features = createFeatures([]);
const makePatternCheckStub = sinon.stub(internal, "makePatternCheck");
makePatternCheckStub
.withArgs(CSHARP_BASE_PATTERNS)
.resolves(CSHARP_BASE_PATTERNS);
makePatternCheckStub.withArgs(CSHARP_EXTRA_PATTERNS).rejects();
await t.notThrowsAsync(async () => {
const result = await getCsharpHashPatterns(codeql, features);
t.deepEqual(result, CSHARP_BASE_PATTERNS);
});
});
test("getCsharpHashPatterns - returns base patterns if any base pattern matches and CsharpNewCacheKey is enabled", async (t) => {
const codeql = createStubCodeQL({});
const features = createFeatures([Feature.CsharpNewCacheKey]);
const makePatternCheckStub = sinon.stub(internal, "makePatternCheck");
makePatternCheckStub
.withArgs(CSHARP_BASE_PATTERNS)
.resolves(CSHARP_BASE_PATTERNS);
makePatternCheckStub
.withArgs(CSHARP_EXTRA_PATTERNS)
.resolves(CSHARP_EXTRA_PATTERNS);
await t.notThrowsAsync(async () => {
const result = await getCsharpHashPatterns(codeql, features);
t.deepEqual(result, CSHARP_BASE_PATTERNS);
});
});
test("getCsharpHashPatterns - returns extra patterns if any extra pattern matches and CsharpNewCacheKey is enabled", async (t) => {
const codeql = createStubCodeQL({});
const features = createFeatures([Feature.CsharpNewCacheKey]);
const makePatternCheckStub = sinon.stub(internal, "makePatternCheck");
makePatternCheckStub.withArgs(CSHARP_BASE_PATTERNS).resolves(undefined);
makePatternCheckStub
.withArgs(CSHARP_EXTRA_PATTERNS)
.resolves(CSHARP_EXTRA_PATTERNS);
await t.notThrowsAsync(async () => {
const result = await getCsharpHashPatterns(codeql, features);
t.deepEqual(result, CSHARP_EXTRA_PATTERNS);
});
});
test("getCsharpHashPatterns - returns undefined if neither base nor extra patterns match", async (t) => {
const codeql = createStubCodeQL({});
const features = createFeatures([Feature.CsharpNewCacheKey]);
const makePatternCheckStub = sinon.stub(internal, "makePatternCheck");
makePatternCheckStub.withArgs(CSHARP_BASE_PATTERNS).resolves(undefined);
makePatternCheckStub.withArgs(CSHARP_EXTRA_PATTERNS).resolves(undefined);
await t.notThrowsAsync(async () => {
const result = await getCsharpHashPatterns(codeql, features);
t.deepEqual(result, undefined);
});
});
test("checkHashPatterns - logs when no patterns match", async (t) => {
const codeql = createStubCodeQL({});
const features = createFeatures([]);
const messages: LoggedMessage[] = [];
const config: CacheConfig = {
getDependencyPaths: () => [],
getHashPatterns: async () => undefined,
};
const result = await checkHashPatterns(
codeql,
features,
KnownLanguage.csharp,
config,
"download",
getRecordingLogger(messages),
);
t.is(result, undefined);
checkExpectedLogMessages(t, messages, [
"Skipping download of dependency cache",
]);
});
test("checkHashPatterns - returns patterns when patterns match", async (t) => {
await withTmpDir(async (tmpDir) => {
const codeql = createStubCodeQL({});
const features = createFeatures([]);
const messages: LoggedMessage[] = [];
const patterns = makeAbsolutePatterns(tmpDir, ["**/*.cs", "**/*.java"]);
fs.writeFileSync(path.join(tmpDir, "test.java"), "");
const config: CacheConfig = {
getDependencyPaths: () => [],
getHashPatterns: async () => makePatternCheck(patterns),
};
const result = await checkHashPatterns(
codeql,
features,
KnownLanguage.csharp,
config,
"upload",
getRecordingLogger(messages),
);
t.deepEqual(result, patterns);
t.deepEqual(messages, []);
});
});
type RestoreCacheFunc = (
paths: string[],
primaryKey: string,
restoreKeys: string[] | undefined,
) => Promise<string | undefined>;
/**
* Constructs a function that `actionsCache.restoreCache` can be stubbed with.
*
* @param mockCacheKeys The keys of caches that we want to exist in the Actions cache.
*
* @returns Returns a function that `actionsCache.restoreCache` can be stubbed with.
*/
function makeMockCacheCheck(mockCacheKeys: string[]): RestoreCacheFunc {
return async (
_paths: string[],
primaryKey: string,
restoreKeys: string[] | undefined,
) => {
// The behaviour here mirrors what the real `restoreCache` would do:
// - Starting with the primary restore key, check all caches for a match:
// even for the primary restore key, this only has to be a prefix match.
// - If the primary restore key doesn't prefix-match any cache, then proceed
// in the same way for each restore key in turn.
for (const restoreKey of [primaryKey, ...(restoreKeys || [])]) {
for (const mockCacheKey of mockCacheKeys) {
if (mockCacheKey.startsWith(restoreKey)) {
return mockCacheKey;
}
}
}
// Only if no restore key matches any cache key prefix, there is no matching
// cache and we return `undefined`.
return undefined;
};
}
test("downloadDependencyCaches - does not restore caches with feature keys if no features are enabled", async (t) => {
process.env["RUNNER_OS"] = "Linux";
const codeql = createStubCodeQL({});
const messages: LoggedMessage[] = [];
const logger = getRecordingLogger(messages);
sinon.stub(glob, "hashFiles").resolves("abcdef");
const keyWithFeature = await cacheKey(
codeql,
createFeatures([Feature.CsharpNewCacheKey]),
KnownLanguage.csharp,
// Patterns don't matter here because we have stubbed `hashFiles` to always return a specific hash above.
[],
);
const restoreCacheStub = sinon
.stub(actionsCache, "restoreCache")
.callsFake(makeMockCacheCheck([keyWithFeature]));
const makePatternCheckStub = sinon.stub(internal, "makePatternCheck");
makePatternCheckStub
.withArgs(CSHARP_BASE_PATTERNS)
.resolves(CSHARP_BASE_PATTERNS);
makePatternCheckStub.withArgs(CSHARP_EXTRA_PATTERNS).resolves(undefined);
const results = await downloadDependencyCaches(
codeql,
createFeatures([]),
[KnownLanguage.csharp],
logger,
);
t.is(results.length, 1);
t.is(results[0].language, KnownLanguage.csharp);
t.is(results[0].hit_kind, CacheHitKind.Miss);
t.assert(restoreCacheStub.calledOnce);
});
test("downloadDependencyCaches - restores caches with feature keys if features are enabled", async (t) => {
process.env["RUNNER_OS"] = "Linux";
const codeql = createStubCodeQL({});
const messages: LoggedMessage[] = [];
const logger = getRecordingLogger(messages);
const features = createFeatures([Feature.CsharpNewCacheKey]);
sinon.stub(glob, "hashFiles").resolves("abcdef");
const keyWithFeature = await cacheKey(
codeql,
features,
KnownLanguage.csharp,
// Patterns don't matter here because we have stubbed `hashFiles` to always return a specific hash above.
[],
);
const restoreCacheStub = sinon
.stub(actionsCache, "restoreCache")
.callsFake(makeMockCacheCheck([keyWithFeature]));
const makePatternCheckStub = sinon.stub(internal, "makePatternCheck");
makePatternCheckStub
.withArgs(CSHARP_BASE_PATTERNS)
.resolves(CSHARP_BASE_PATTERNS);
makePatternCheckStub.withArgs(CSHARP_EXTRA_PATTERNS).resolves(undefined);
const results = await downloadDependencyCaches(
codeql,
features,
[KnownLanguage.csharp],
logger,
);
t.is(results.length, 1);
t.is(results[0].language, KnownLanguage.csharp);
t.is(results[0].hit_kind, CacheHitKind.Exact);
t.assert(restoreCacheStub.calledOnce);
});
test("downloadDependencyCaches - restores caches with feature keys if features are enabled for partial matches", async (t) => {
process.env["RUNNER_OS"] = "Linux";
const codeql = createStubCodeQL({});
const messages: LoggedMessage[] = [];
const logger = getRecordingLogger(messages);
const features = createFeatures([Feature.CsharpNewCacheKey]);
const hashFilesStub = sinon.stub(glob, "hashFiles");
hashFilesStub.onFirstCall().resolves("abcdef");
hashFilesStub.onSecondCall().resolves("123456");
const keyWithFeature = await cacheKey(
codeql,
features,
KnownLanguage.csharp,
// Patterns don't matter here because we have stubbed `hashFiles` to always return a specific hash above.
[],
);
const restoreCacheStub = sinon
.stub(actionsCache, "restoreCache")
.callsFake(makeMockCacheCheck([keyWithFeature]));
const makePatternCheckStub = sinon.stub(internal, "makePatternCheck");
makePatternCheckStub
.withArgs(CSHARP_BASE_PATTERNS)
.resolves(CSHARP_BASE_PATTERNS);
makePatternCheckStub.withArgs(CSHARP_EXTRA_PATTERNS).resolves(undefined);
const results = await downloadDependencyCaches(
codeql,
features,
[KnownLanguage.csharp],
logger,
);
t.is(results.length, 1);
t.is(results[0].language, KnownLanguage.csharp);
t.is(results[0].hit_kind, CacheHitKind.Partial);
t.assert(restoreCacheStub.calledOnce);
});
test("getFeaturePrefix - returns empty string if no features are enabled", async (t) => {
const codeql = createStubCodeQL({});
const features = createFeatures([]);
for (const knownLanguage of Object.values(KnownLanguage)) {
const result = await getFeaturePrefix(codeql, features, knownLanguage);
t.deepEqual(result, "", `Expected no feature prefix for ${knownLanguage}`);
}
});
test("getFeaturePrefix - Java - returns 'minify-' if JavaMinimizeDependencyJars is enabled", async (t) => {
const codeql = createStubCodeQL({});
const features = createFeatures([Feature.JavaMinimizeDependencyJars]);
const result = await getFeaturePrefix(codeql, features, KnownLanguage.java);
t.deepEqual(result, "minify-");
});
test("getFeaturePrefix - non-Java - returns '' if JavaMinimizeDependencyJars is enabled", async (t) => {
const codeql = createStubCodeQL({});
const features = createFeatures([Feature.JavaMinimizeDependencyJars]);
for (const knownLanguage of Object.values(KnownLanguage)) {
// Skip Java since we expect a result for it, which is tested in the previous test.
if (knownLanguage === KnownLanguage.java) {
continue;
}
const result = await getFeaturePrefix(codeql, features, knownLanguage);
t.deepEqual(result, "", `Expected no feature prefix for ${knownLanguage}`);
}
});
test("getFeaturePrefix - C# - returns prefix if CsharpNewCacheKey is enabled", async (t) => {
const codeql = createStubCodeQL({});
const features = createFeatures([Feature.CsharpNewCacheKey]);
const result = await getFeaturePrefix(codeql, features, KnownLanguage.csharp);
t.notDeepEqual(result, "");
t.assert(result.endsWith("-"));
// Check the length of the prefix, which should correspond to `cacheKeyHashLength` + 1 for the trailing `-`.
t.is(result.length, cacheKeyHashLength + 1);
});
test("getFeaturePrefix - non-C# - returns '' if CsharpNewCacheKey is enabled", async (t) => {
const codeql = createStubCodeQL({});
const features = createFeatures([Feature.CsharpNewCacheKey]);
for (const knownLanguage of Object.values(KnownLanguage)) {
// Skip C# since we expect a result for it, which is tested in the previous test.
if (knownLanguage === KnownLanguage.csharp) {
continue;
}
const result = await getFeaturePrefix(codeql, features, knownLanguage);
t.deepEqual(result, "", `Expected no feature prefix for ${knownLanguage}`);
}
});
+272 -75
View File
@@ -6,9 +6,11 @@ import * as glob from "@actions/glob";
import { getTemporaryDirectory } from "./actions-util";
import { listActionsCaches } from "./api-client";
import { getTotalCacheSize } from "./caching-utils";
import { createCacheKeyHash, getTotalCacheSize } from "./caching-utils";
import { CodeQL } from "./codeql";
import { Config } from "./config-utils";
import { EnvVar } from "./environment";
import { Feature, FeatureEnablement } from "./feature-flags";
import { KnownLanguage, Language } from "./languages";
import { Logger } from "./logging";
import { getErrorMessage, getRequiredEnvParam } from "./util";
@@ -16,15 +18,21 @@ import { getErrorMessage, getRequiredEnvParam } from "./util";
/**
* Caching configuration for a particular language.
*/
interface CacheConfig {
/** The paths of directories on the runner that should be included in the cache. */
paths: string[];
export interface CacheConfig {
/** Gets the paths of directories on the runner that should be included in the cache. */
getDependencyPaths: () => string[];
/**
* Patterns for the paths of files whose contents affect which dependencies are used
* by a project. We find all files which match these patterns, calculate a hash for
* their contents, and use that hash as part of the cache key.
* Gets an array of glob patterns for the paths of files whose contents affect which dependencies are used
* by a project. This function also checks whether there are any matching files and returns
* `undefined` if no files match.
*
* The glob patterns are intended to be used for cache keys, where we find all files which match these
* patterns, calculate a hash for their contents, and use that hash as part of the cache key.
*/
hash: string[];
getHashPatterns: (
codeql: CodeQL,
features: FeatureEnablement,
) => Promise<string[] | undefined>;
}
const CODEQL_DEPENDENCY_CACHE_PREFIX = "codeql-dependencies";
@@ -39,21 +47,105 @@ export function getJavaTempDependencyDir(): string {
return join(getTemporaryDirectory(), "codeql_java", "repository");
}
/**
* Returns an array of paths of directories on the runner that should be included in a dependency cache
* for a Java analysis. It is important that this is a function, because we call `getTemporaryDirectory`
* which would otherwise fail in tests if we haven't had a chance to initialise `RUNNER_TEMP`.
*
* @returns The paths of directories on the runner that should be included in a dependency cache
* for a Java analysis.
*/
export function getJavaDependencyDirs(): string[] {
return [
// Maven
join(os.homedir(), ".m2", "repository"),
// Gradle
join(os.homedir(), ".gradle", "caches"),
// CodeQL Java build-mode: none
getJavaTempDependencyDir(),
];
}
/**
* Checks that there are files which match `patterns`. If there are matching files for any of the patterns,
* this function returns all `patterns`. Otherwise, `undefined` is returned.
*
* @param patterns The glob patterns to find matching files for.
* @returns The array of glob patterns if there are matching files, or `undefined` otherwise.
*/
export async function makePatternCheck(
patterns: string[],
): Promise<string[] | undefined> {
const globber = await makeGlobber(patterns);
if ((await globber.glob()).length === 0) {
return undefined;
}
return patterns;
}
/** These files contain accurate information about dependencies, including the exact versions
* that the relevant package manager has determined for the project. Using these gives us
* stable hashes unless the dependencies change.
*/
export const CSHARP_BASE_PATTERNS = [
// NuGet
"**/packages.lock.json",
// Paket
"**/paket.lock",
];
/** These are less accurate for use in cache key calculations, because they:
*
* - Don't contain the exact versions used. They may only contain version ranges or none at all.
* - They contain information unrelated to dependencies, which we don't care about.
*
* As a result, the hash we compute from these files may change, even if
* the dependencies haven't changed.
*/
export const CSHARP_EXTRA_PATTERNS = [
"**/*.csproj",
"**/packages.config",
"**/nuget.config",
];
/**
* Returns the list of glob patterns that should be used to calculate the cache key hash
* for a C# dependency cache. This will try to use `CSHARP_BASE_PATTERNS` whenever possible.
* As a fallback, it will also use `CSHARP_EXTRA_PATTERNS` if the corresponding FF is enabled.
*
* @param codeql The CodeQL instance to use.
* @param features Information about which FFs are enabled.
* @returns A list of glob patterns to use for hashing.
*/
export async function getCsharpHashPatterns(
codeql: CodeQL,
features: FeatureEnablement,
): Promise<string[] | undefined> {
const basePatterns = await internal.makePatternCheck(CSHARP_BASE_PATTERNS);
if (basePatterns !== undefined) {
return basePatterns;
}
if (await features.getValue(Feature.CsharpNewCacheKey, codeql)) {
return internal.makePatternCheck(CSHARP_EXTRA_PATTERNS);
}
// If we get to this point, we didn't find any files with `CSHARP_BASE_PATTERNS`,
// and `Feature.CsharpNewCacheKey` is not enabled.
return undefined;
}
/**
* Default caching configurations per language.
*/
function getDefaultCacheConfig(): { [language: string]: CacheConfig } {
return {
java: {
paths: [
// Maven
join(os.homedir(), ".m2", "repository"),
// Gradle
join(os.homedir(), ".gradle", "caches"),
// CodeQL Java build-mode: none
getJavaTempDependencyDir(),
],
hash: [
const defaultCacheConfigs: { [language: string]: CacheConfig } = {
java: {
getDependencyPaths: getJavaDependencyDirs,
getHashPatterns: async () =>
internal.makePatternCheck([
// Maven
"**/pom.xml",
// Gradle
@@ -63,23 +155,17 @@ function getDefaultCacheConfig(): { [language: string]: CacheConfig } {
"buildSrc/**/Dependencies.kt",
"gradle/*.versions.toml",
"**/versions.properties",
],
},
csharp: {
paths: [join(os.homedir(), ".nuget", "packages")],
hash: [
// NuGet
"**/packages.lock.json",
// Paket
"**/paket.lock",
],
},
go: {
paths: [join(os.homedir(), "go", "pkg", "mod")],
hash: ["**/go.sum"],
},
};
}
]),
},
csharp: {
getDependencyPaths: () => [join(os.homedir(), ".nuget", "packages")],
getHashPatterns: getCsharpHashPatterns,
},
go: {
getDependencyPaths: () => [join(os.homedir(), "go", "pkg", "mod")],
getHashPatterns: async () => internal.makePatternCheck(["**/go.sum"]),
},
};
async function makeGlobber(patterns: string[]): Promise<glob.Globber> {
return glob.create(patterns.join("\n"));
@@ -107,23 +193,57 @@ export interface DependencyCacheRestoreStatus {
/** An array of `DependencyCacheRestoreStatus` objects for each analysed language with a caching configuration. */
export type DependencyCacheRestoreStatusReport = DependencyCacheRestoreStatus[];
/**
* A wrapper around `cacheConfig.getHashPatterns` which logs when there are no files to calculate
* a hash for the cache key from.
*
* @param codeql The CodeQL instance to use.
* @param features Information about which FFs are enabled.
* @param language The language the `CacheConfig` is for. For use in the log message.
* @param cacheConfig The caching configuration to call `getHashPatterns` on.
* @param checkType Whether we are checking the patterns for a download or upload.
* @param logger The logger to write the log message to if there is an error.
* @returns An array of glob patterns to use for hashing files, or `undefined` if there are no matching files.
*/
export async function checkHashPatterns(
codeql: CodeQL,
features: FeatureEnablement,
language: Language,
cacheConfig: CacheConfig,
checkType: "download" | "upload",
logger: Logger,
): Promise<string[] | undefined> {
const patterns = await cacheConfig.getHashPatterns(codeql, features);
if (patterns === undefined) {
logger.info(
`Skipping ${checkType} of dependency cache for ${language} as we cannot calculate a hash for the cache key.`,
);
}
return patterns;
}
/**
* Attempts to restore dependency caches for the languages being analyzed.
*
* @param codeql The CodeQL instance to use.
* @param features Information about which FFs are enabled.
* @param languages The languages being analyzed.
* @param logger A logger to record some informational messages to.
* @param minimizeJavaJars Whether the Java extractor should rewrite downloaded JARs to minimize their size.
*
* @returns An array of `DependencyCacheRestoreStatus` objects for each analysed language with a caching configuration.
*/
export async function downloadDependencyCaches(
codeql: CodeQL,
features: FeatureEnablement,
languages: Language[],
logger: Logger,
minimizeJavaJars: boolean,
): Promise<DependencyCacheRestoreStatusReport> {
const status: DependencyCacheRestoreStatusReport = [];
for (const language of languages) {
const cacheConfig = getDefaultCacheConfig()[language];
const cacheConfig = defaultCacheConfigs[language];
if (cacheConfig === undefined) {
logger.info(
@@ -134,19 +254,22 @@ export async function downloadDependencyCaches(
// Check that we can find files to calculate the hash for the cache key from, so we don't end up
// with an empty string.
const globber = await makeGlobber(cacheConfig.hash);
if ((await globber.glob()).length === 0) {
const patterns = await checkHashPatterns(
codeql,
features,
language,
cacheConfig,
"download",
logger,
);
if (patterns === undefined) {
status.push({ language, hit_kind: CacheHitKind.NoHash });
logger.info(
`Skipping download of dependency cache for ${language} as we cannot calculate a hash for the cache key.`,
);
continue;
}
const primaryKey = await cacheKey(language, cacheConfig, minimizeJavaJars);
const primaryKey = await cacheKey(codeql, features, language, patterns);
const restoreKeys: string[] = [
await cachePrefix(language, minimizeJavaJars),
await cachePrefix(codeql, features, language),
];
logger.info(
@@ -157,7 +280,7 @@ export async function downloadDependencyCaches(
const start = performance.now();
const hitKey = await actionsCache.restoreCache(
cacheConfig.paths,
cacheConfig.getDependencyPaths(),
primaryKey,
restoreKeys,
);
@@ -203,20 +326,22 @@ export type DependencyCacheUploadStatusReport = DependencyCacheUploadStatus[];
/**
* Attempts to store caches for the languages that were analyzed.
*
* @param codeql The CodeQL instance to use.
* @param features Information about which FFs are enabled.
* @param config The configuration for this workflow.
* @param logger A logger to record some informational messages to.
* @param minimizeJavaJars Whether the Java extractor should rewrite downloaded JARs to minimize their size.
*
* @returns An array of `DependencyCacheUploadStatus` objects for each analysed language with a caching configuration.
*/
export async function uploadDependencyCaches(
codeql: CodeQL,
features: FeatureEnablement,
config: Config,
logger: Logger,
minimizeJavaJars: boolean,
): Promise<DependencyCacheUploadStatusReport> {
const status: DependencyCacheUploadStatusReport = [];
for (const language of config.languages) {
const cacheConfig = getDefaultCacheConfig()[language];
const cacheConfig = defaultCacheConfigs[language];
if (cacheConfig === undefined) {
logger.info(
@@ -227,13 +352,16 @@ export async function uploadDependencyCaches(
// Check that we can find files to calculate the hash for the cache key from, so we don't end up
// with an empty string.
const globber = await makeGlobber(cacheConfig.hash);
if ((await globber.glob()).length === 0) {
const patterns = await checkHashPatterns(
codeql,
features,
language,
cacheConfig,
"upload",
logger,
);
if (patterns === undefined) {
status.push({ language, result: CacheStoreResult.NoHash });
logger.info(
`Skipping upload of dependency cache for ${language} as we cannot calculate a hash for the cache key.`,
);
continue;
}
@@ -247,7 +375,11 @@ export async function uploadDependencyCaches(
// use the cache quota that we compete with. In that case, we do not wish to use up all of the quota
// with the dependency caches. For this, we could use the Cache API to check whether other workflows
// are using the quota and how full it is.
const size = await getTotalCacheSize(cacheConfig.paths, logger, true);
const size = await getTotalCacheSize(
cacheConfig.getDependencyPaths(),
logger,
true,
);
// Skip uploading an empty cache.
if (size === 0) {
@@ -258,7 +390,7 @@ export async function uploadDependencyCaches(
continue;
}
const key = await cacheKey(language, cacheConfig, minimizeJavaJars);
const key = await cacheKey(codeql, features, language, patterns);
logger.info(
`Uploading cache of size ${size} for ${language} with key ${key}...`,
@@ -266,7 +398,7 @@ export async function uploadDependencyCaches(
try {
const start = performance.now();
await actionsCache.saveCache(cacheConfig.paths, key);
await actionsCache.saveCache(cacheConfig.getDependencyPaths(), key);
const upload_duration_ms = Math.round(performance.now() - start);
status.push({
@@ -299,31 +431,86 @@ export async function uploadDependencyCaches(
/**
* Computes a cache key for the specified language.
*
* @param codeql The CodeQL instance to use.
* @param features Information about which FFs are enabled.
* @param language The language being analyzed.
* @param cacheConfig The cache configuration for the language.
* @param minimizeJavaJars Whether the Java extractor should rewrite downloaded JARs to minimize their size.
* @param patterns The file patterns to hash.
*
* @returns A cache key capturing information about the project(s) being analyzed in the specified language.
*/
async function cacheKey(
export async function cacheKey(
codeql: CodeQL,
features: FeatureEnablement,
language: Language,
cacheConfig: CacheConfig,
minimizeJavaJars: boolean = false,
patterns: string[],
): Promise<string> {
const hash = await glob.hashFiles(cacheConfig.hash.join("\n"));
return `${await cachePrefix(language, minimizeJavaJars)}${hash}`;
const hash = await glob.hashFiles(patterns.join("\n"));
return `${await cachePrefix(codeql, features, language)}${hash}`;
}
/**
* If experimental features which the cache contents depend on are enabled for the current language,
* this function returns a prefix that uniquely identifies the set of enabled features. The purpose of
* this is to avoid restoring caches whose contents depended on experimental features, if those
* experimental features are later disabled.
*
* @param codeql The CodeQL instance.
* @param features Information about enabled features.
* @param language The language we are creating the key for.
*
* @returns A cache key prefix identifying the enabled, experimental features that the cache depends on.
*/
export async function getFeaturePrefix(
codeql: CodeQL,
features: FeatureEnablement,
language: Language,
): Promise<string> {
const enabledFeatures: Feature[] = [];
const addFeatureIfEnabled = async (feature: Feature) => {
if (await features.getValue(feature, codeql)) {
enabledFeatures.push(feature);
}
};
if (language === KnownLanguage.java) {
// To ensure a safe rollout of JAR minimization, we change the key when the feature is enabled.
const minimizeJavaJars = await features.getValue(
Feature.JavaMinimizeDependencyJars,
codeql,
);
// To maintain backwards compatibility with this, we return "minify-" instead of a hash.
if (minimizeJavaJars) {
return "minify-";
}
} else if (language === KnownLanguage.csharp) {
await addFeatureIfEnabled(Feature.CsharpNewCacheKey);
}
// If any features that affect the cache are enabled, return a feature prefix by
// computing a hash of the feature array.
if (enabledFeatures.length > 0) {
return `${createCacheKeyHash(enabledFeatures)}-`;
}
// No feature prefix.
return "";
}
/**
* Constructs a prefix for the cache key, comprised of a CodeQL-specific prefix, a version number that
* can be changed to invalidate old caches, the runner's operating system, and the specified language name.
*
* @param codeql The CodeQL instance to use.
* @param features Information about which FFs are enabled.
* @param language The language being analyzed.
* @param minimizeJavaJars Whether the Java extractor should rewrite downloaded JARs to minimize their size.
* @returns The prefix that identifies what a cache is for.
*/
async function cachePrefix(
codeql: CodeQL,
features: FeatureEnablement,
language: Language,
minimizeJavaJars: boolean,
): Promise<string> {
const runnerOs = getRequiredEnvParam("RUNNER_OS");
const customPrefix = process.env[EnvVar.DEPENDENCY_CACHING_PREFIX];
@@ -333,12 +520,18 @@ async function cachePrefix(
prefix = `${prefix}-${customPrefix}`;
}
// To ensure a safe rollout of JAR minimization, we change the key when the feature is enabled.
if (language === KnownLanguage.java && minimizeJavaJars) {
prefix = `minify-${prefix}`;
}
// Calculate the feature prefix for the cache, if any. This is a hash that identifies
// experimental features that affect the cache contents.
const featurePrefix = await getFeaturePrefix(codeql, features, language);
return `${prefix}-${CODEQL_DEPENDENCY_CACHE_VERSION}-${runnerOs}-${language}-`;
// Assemble the cache key. For backwards compatibility with the JAR minification experiment's existing
// feature prefix usage, we add that feature prefix at the start. Other feature prefixes are inserted
// after the general CodeQL dependency cache prefix.
if (featurePrefix === "minify-") {
return `${featurePrefix}${prefix}-${CODEQL_DEPENDENCY_CACHE_VERSION}-${runnerOs}-${language}-`;
} else {
return `${prefix}-${featurePrefix}${CODEQL_DEPENDENCY_CACHE_VERSION}-${runnerOs}-${language}-`;
}
}
/** Represents information about our overall cache usage for CodeQL dependency caches. */
@@ -371,3 +564,7 @@ export async function getDependencyCacheUsage(
return undefined;
}
export const internal = {
makePatternCheck,
};
+6 -7
View File
@@ -47,6 +47,7 @@ export enum Feature {
AnalyzeUseNewUpload = "analyze_use_new_upload",
CleanupTrapCaches = "cleanup_trap_caches",
CppDependencyInstallation = "cpp_dependency_installation_enabled",
CsharpNewCacheKey = "csharp_new_cache_key",
DiffInformedQueries = "diff_informed_queries",
DisableCsharpBuildless = "disable_csharp_buildless",
DisableJavaBuildlessEnabled = "disable_java_buildless_enabled",
@@ -76,7 +77,6 @@ export enum Feature {
OverlayAnalysisSwift = "overlay_analysis_swift",
PythonDefaultIsToNotExtractStdlib = "python_default_is_to_not_extract_stdlib",
QaTelemetryEnabled = "qa_telemetry_enabled",
ResolveSupportedLanguagesUsingCli = "resolve_supported_languages_using_cli",
UseRepositoryProperties = "use_repository_properties",
ValidateDbConfig = "validate_db_config",
}
@@ -133,6 +133,11 @@ export const featureConfig: Record<
legacyApi: true,
minimumVersion: "2.15.0",
},
[Feature.CsharpNewCacheKey]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CSHARP_NEW_CACHE_KEY",
minimumVersion: undefined,
},
[Feature.DiffInformedQueries]: {
defaultValue: true,
envVar: "CODEQL_ACTION_DIFF_INFORMED_QUERIES",
@@ -161,12 +166,6 @@ export const featureConfig: Record<
legacyApi: true,
minimumVersion: undefined,
},
[Feature.ResolveSupportedLanguagesUsingCli]: {
defaultValue: false,
envVar: "CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI",
minimumVersion: undefined,
toolsFeature: ToolsFeature.BuiltinExtractorsSpecifyDefaultQueries,
},
[Feature.OverlayAnalysis]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS",
+3 -6
View File
@@ -578,15 +578,12 @@ async function run() {
}
// Restore dependency cache(s), if they exist.
const minimizeJavaJars = await features.getValue(
Feature.JavaMinimizeDependencyJars,
codeql,
);
if (shouldRestoreCache(config.dependencyCachingEnabled)) {
dependencyCachingResults = await downloadDependencyCaches(
codeql,
features,
config.languages,
logger,
minimizeJavaJars,
);
}
@@ -648,7 +645,7 @@ async function run() {
`${EnvVar.JAVA_EXTRACTOR_MINIMIZE_DEPENDENCY_JARS} is already set to '${process.env[EnvVar.JAVA_EXTRACTOR_MINIMIZE_DEPENDENCY_JARS]}', so the Action will not override it.`,
);
} else if (
minimizeJavaJars &&
(await features.getValue(Feature.JavaMinimizeDependencyJars, codeql)) &&
config.dependencyCachingEnabled &&
config.buildMode === BuildMode.None &&
config.languages.includes(KnownLanguage.java)
+1 -25
View File
@@ -1,4 +1,3 @@
import * as crypto from "crypto";
import * as fs from "fs";
import * as path from "path";
@@ -11,6 +10,7 @@ import {
getWorkflowRunID,
} from "./actions-util";
import { getAutomationID } from "./api-client";
import { createCacheKeyHash } from "./caching-utils";
import { type CodeQL } from "./codeql";
import { type Config } from "./config-utils";
import { getCommitOid, getFileOidsUnderPath } from "./git-utils";
@@ -514,27 +514,3 @@ export async function getCacheRestoreKeyPrefix(
// easier to debug and understand the cache key structure.
return `${CACHE_PREFIX}-${CACHE_VERSION}-${componentsHash}-${languages}-${codeQlVersion}-`;
}
/**
* Creates a SHA-256 hash of the cache key components to ensure uniqueness
* while keeping the cache key length manageable.
*
* @param components Object containing all components that should influence cache key uniqueness
* @returns A short SHA-256 hash (first 16 characters) of the components
*/
function createCacheKeyHash(components: Record<string, any>): string {
// From https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify
//
// "Properties are visited using the same algorithm as Object.keys(), which
// has a well-defined order and is stable across implementations. For example,
// JSON.stringify on the same object will always produce the same string, and
// JSON.parse(JSON.stringify(obj)) would produce an object with the same key
// ordering as the original (assuming the object is completely
// JSON-serializable)."
const componentsJson = JSON.stringify(components);
return crypto
.createHash("sha256")
.update(componentsJson)
.digest("hex")
.substring(0, 16);
}
+1 -1
View File
@@ -252,7 +252,7 @@ export interface EventReport {
*
* @param actionName The name of the action, e.g. 'init', 'finish', 'upload-sarif'
* @param status The status. Must be 'success', 'failure', or 'starting'
* @param startedAt The time this action started executing.
* @param actionStartedAt The time this action started executing.
* @param cause Cause of failure (only supply if status is 'failure')
* @param exception Exception (only supply if status is 'failure')
* @returns undefined if an exception was thrown.
+1 -1
View File
@@ -738,7 +738,7 @@ export async function postProcessSarifFiles(
* @param logger The logger to use.
* @param pathInput The input provided for `post-processed-sarif-path`.
* @param uploadTarget The upload target.
* @param processingResults The results of post-processing SARIF files.
* @param postProcessingResults The results of post-processing SARIF files.
*/
export async function writePostProcessedFiles(
logger: Logger,
+13 -7
View File
@@ -447,13 +447,21 @@ const CHECK_ACTION_VERSION_TESTS: Array<[string, util.GitHubVersion, boolean]> =
["2.2.1", { type: util.GitHubVariant.DOTCOM }, true],
["2.2.1", { type: util.GitHubVariant.GHE_DOTCOM }, true],
["2.2.1", { type: util.GitHubVariant.GHES, version: "3.10" }, false],
["2.2.1", { type: util.GitHubVariant.GHES, version: "3.11" }, true],
["2.2.1", { type: util.GitHubVariant.GHES, version: "3.12" }, true],
["3.2.1", { type: util.GitHubVariant.DOTCOM }, false],
["3.2.1", { type: util.GitHubVariant.GHE_DOTCOM }, false],
["2.2.1", { type: util.GitHubVariant.GHES, version: "3.11" }, false],
["2.2.1", { type: util.GitHubVariant.GHES, version: "3.12" }, false],
["3.2.1", { type: util.GitHubVariant.DOTCOM }, true],
["3.2.1", { type: util.GitHubVariant.GHE_DOTCOM }, true],
["3.2.1", { type: util.GitHubVariant.GHES, version: "3.10" }, false],
["3.2.1", { type: util.GitHubVariant.GHES, version: "3.11" }, false],
["3.2.1", { type: util.GitHubVariant.GHES, version: "3.12" }, false],
["3.2.1", { type: util.GitHubVariant.GHES, version: "3.19" }, false],
["3.2.1", { type: util.GitHubVariant.GHES, version: "3.20" }, true],
["3.2.1", { type: util.GitHubVariant.GHES, version: "3.21" }, true],
["4.2.1", { type: util.GitHubVariant.DOTCOM }, false],
["4.2.1", { type: util.GitHubVariant.GHE_DOTCOM }, false],
["4.2.1", { type: util.GitHubVariant.GHES, version: "3.19" }, false],
["4.2.1", { type: util.GitHubVariant.GHES, version: "3.20" }, false],
["4.2.1", { type: util.GitHubVariant.GHES, version: "3.21" }, false],
];
for (const [
@@ -480,9 +488,7 @@ for (const [
if (shouldReportError) {
t.true(
warningSpy.calledOnceWithExactly(
sinon.match(
"CodeQL Action major versions v1 and v2 have been deprecated.",
),
sinon.match("CodeQL Action v3 will be deprecated in December 2026."),
),
);
} else {
+12 -12
View File
@@ -1114,38 +1114,38 @@ export async function checkDiskUsage(
}
/**
* Prompt the customer to upgrade to CodeQL Action v3, if appropriate.
* Prompt the customer to upgrade to CodeQL Action v4, if appropriate.
*
* Check whether a customer is running v1 or v2. If they are, and we can determine that the GitHub
* instance supports v3, then log an error prompting the customer to upgrade to v3.
* Check whether a customer is running v3. If they are, and we can determine that the GitHub
* instance supports v4, then log an error prompting the customer to upgrade to v4.
*/
export function checkActionVersion(
version: string,
githubVersion: GitHubVersion,
) {
if (
!semver.satisfies(version, ">=3") && // do not log error if the customer is already running v3
!semver.satisfies(version, ">=4") && // do not log error if the customer is already running v4
!process.env[EnvVar.LOG_VERSION_DEPRECATION] // do not log error if we have already
) {
// Only error for versions of GHES that are compatible with CodeQL Action version 3.
// Only error for versions of GHES that are compatible with CodeQL Action version 4.
//
// GHES 3.11 shipped without the v3 tag, but it also shipped without this warning message code.
// Therefore users who are seeing this warning message code have pulled in a new version of the
// Action, and with it the v3 tag.
// GHES 3.20 is the first version to ship with the v4 tag and this warning message code.
// Therefore, users who are seeing this warning message code are running on GHES 3.20 or newer,
// and should update to CodeQL Action v4.
if (
githubVersion.type === GitHubVariant.DOTCOM ||
githubVersion.type === GitHubVariant.GHE_DOTCOM ||
(githubVersion.type === GitHubVariant.GHES &&
semver.satisfies(
semver.coerce(githubVersion.version) ?? "0.0.0",
">=3.11",
">=3.20",
))
) {
core.error(
"CodeQL Action major versions v1 and v2 have been deprecated. " +
"Please update all occurrences of the CodeQL Action in your workflow files to v3. " +
"CodeQL Action v3 will be deprecated in December 2026. " +
"Please update all occurrences of the CodeQL Action in your workflow files to v4. " +
"For more information, see " +
"https://github.blog/changelog/2025-01-10-code-scanning-codeql-action-v2-is-now-deprecated/",
"https://github.blog/changelog/2025-10-28-upcoming-deprecation-of-codeql-action-v3/",
);
// set LOG_VERSION_DEPRECATION env var to prevent the warning from being logged multiple times
core.exportVariable(EnvVar.LOG_VERSION_DEPRECATION, "true");
+1 -1
View File
@@ -29,6 +29,6 @@ outputs:
proxy_urls:
description: A stringified JSON array of objects containing the types and URLs of the configured registries.
runs:
using: node20
using: node24
main: "../lib/start-proxy-action.js"
post: "../lib/start-proxy-action-post.js"
+1 -1
View File
@@ -41,6 +41,6 @@ outputs:
{ "code-scanning": "some-id", "code-quality": "some-other-id" }
runs:
using: node20
using: node24
main: '../lib/upload-sarif-action.js'
post: '../lib/upload-sarif-action-post.js'