Compare commits

..
114 changed files with 9010 additions and 17626 deletions

No files matched your search

+4 -7
View File
@@ -6,11 +6,11 @@ inputs:
required: true
use-all-platform-bundle:
description: "If true, we output a tools URL with codeql-bundle.tar.gz file rather than platform-specific URL"
default: "false"
default: 'false'
required: false
setup-kotlin:
description: "If true, we setup kotlin"
default: "true"
default: 'true'
required: true
outputs:
tools-url:
@@ -73,10 +73,7 @@ runs:
exit 1
fi
- uses: fwilhe2/setup-kotlin@b325539ca8f6e1d9ee04efed06f49bdbf5426dbb # v2.1
- uses: fwilhe2/setup-kotlin@9c245a6425255f5e98ba1ce6c15d31fce7eca9da
if: ${{ inputs.setup-kotlin == 'true' }}
with:
# Kotlin is installed on the runners by default, but the version that is pre-installed is too new
# to be supported by CodeQL. This Kotlin version is selected to be compatible with both the default Java
# version on the `ubuntu-latest` runners and CodeQL.
version: 2.1.20
version: 1.8.21
@@ -16,7 +16,7 @@ runs:
shell: bash
- name: Set up Node
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 24
cache: 'npm'
+2 -15
View File
@@ -11,12 +11,9 @@ The CodeQL Action is used in GitHub Actions workflows to run CodeQL scans using
## Review instructions
- Prioritise identifying critical issues, such as logic bugs, over comments on documentation or typos.
- When wording review comments, be helpful and friendly. Assume that the PR author has written the code with the best of intentions. Never word suggestions as commands or using the imperative. Examples of what you should never write: "Preserve that non-fatal behaviour here" or "Please preserve that non-fatal behaviour here".
- When writing feedback, assume that the PR author is competent and is able to understand whether the point you are making is valid or not, why you are raising it, and how to best address it. While you can suggest how to address a point you are making, don't phrase it as a requirement or assume that it is the only solution.
- Always word your comments constructively as suggestions for improvements, for example: "It would make sense to ..." or "Could you add ...?" or "Would it make sense to ...?"
- When wording review comments, be helpful and friendly. Assume that the PR author has written the code with the best of intentions. Word your comments constructively as suggestions for improvements. Do not word suggestions as commands.
- If you want to comment on a change that you believe will fail a CI check, do not present the CI failure you expect as a fact. Instead, write that you think a change "may" lead to a failure in CI. Suggest that, if such a failure manifests, the changes you are commenting on may be the place responsible for the failure and are worth looking at.
- If a suggestion you make is suitable for a follow-up, such as a refactoring that doesn't change the behaviour or fixing a typo in a comment, mention that it can be addressed in a later PR rather than blocking this one or prefix your comment with "Minor: ".
- If a suggestion you make is suitable for a follow-up, such as a refactoring that doesn't change the behaviour or fixing a typo in a comment, mention that it can be addressed in a later PR rather than blocking this one.
- If a change is a net improvement, for example because it improves on an existing limitation of existing code, do not complain about pre-existing problems that remain. You may comment on them, but you should make it clear that the thing you are commenting on is not new by writing e.g. "Not new in this PR, but [..]" followed by your description of the issue and a suggestion that it could be improved at the same time with e.g. "Consider whether this is worth addressing as part of this PR as well."
## Generated code
@@ -25,16 +22,6 @@ The main codebase of the CodeQL Action is written in TypeScript and compiled to
GitHub Actions workflows in the `.github/workflows` directory whose filenames start with two underscores (e.g. `__all-platform-bundle.yml`) are automatically generated using the `pr-checks/sync.sh` script from template files in the `pr-checks/checks` directory. Therefore, you do not need to review files in the `.github/workflows` directory that start with two underscores. However, you should review changes to the `pr-checks` directory as well as workflows in the `.github/workflows` directory that do not start with underscores.
## Feature flags
Some functionality in the code may be gated behind feature flags, which are controlled server-side. If the functionality that is gated behind them is user-facing, then this should be documented in a change note at the time that the gated functionality is added. See `CHANGELOG.md` for existing notes that mention "experimental" for examples. These typically include information about when users may expect to see the feature rolled out.
When a feature flag that gates some functionality is removed, this does not need to be documented in a change note since the feature is already rolled out and the notes for the release which introduced the feature already describe it. The only exception is functionality which will start having an effect on GHES as a result of the feature flag removal:
- Most functionality gated behind feature flags does not affect GHES.
- We can make functionality available on GHES by setting `defaultValue` to `true` first and waiting for this to have had an effect on GHES. No change note is needed if a feature flag is removed where `defaultValue` was `true`.
- When removing a feature flag where `defaultValue` was `false` and the functionality starts to affect GHES, a change note may be appropriate.
## Release process
Changes that are merged to the `main` branch are not used in production straight away. Instead, changes from `main` must first be released using the release process, which has to be manually triggered by a staff member. Therefore, it is possible for further changes to be made to the code after some changes have been merged to `main` but before they are released. Once the release process automation is started, a series of PRs are automatically generated:
+1 -9
View File
@@ -16,14 +16,6 @@ updates:
# This might be fixed when we move to eslint v9.
- dependency-name: "eslint-plugin-import"
versions: [">=2.30.0"]
# Some of our dependencies are lagging behind with support for new versions of
# `undici` which is why we have to override it in `package.json`. Dependabot
# workflows fail when they try to update a dependency that is explicitly
# overriden. Since this happens somewhat routinely at the moment, we are
# excluding `undici` here and have to manually update it when needed.
- dependency-name: "undici"
versions: [">=7.30.1"]
groups:
npm-minor:
patterns:
@@ -34,7 +26,7 @@ updates:
- package-ecosystem: github-actions
directories:
- "/.github/workflows"
- "/.github/actions/*/"
- "/.github/actions"
schedule:
interval: weekly
cooldown:
@@ -63,7 +63,7 @@ jobs:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Java
uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6.1.0
uses: actions/setup-java@de7274f081f381c8f8158605e0321c36c376e2e6 # v6.0.1
with:
java-version: ${{ inputs.java-version || '17' }}
distribution: temurin
+1 -1
View File
@@ -63,7 +63,7 @@ jobs:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Java
uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6.1.0
uses: actions/setup-java@de7274f081f381c8f8158605e0321c36c376e2e6 # v6.0.1
with:
java-version: ${{ inputs.java-version || '17' }}
distribution: temurin
+19 -17
View File
@@ -46,15 +46,6 @@ jobs:
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
@@ -62,20 +53,31 @@ jobs:
version: ${{ matrix.version }}
use-all-platform-bundle: 'false'
setup-kotlin: 'true'
- name: Install @actions/tool-cache
run: npm install @actions/tool-cache@3
- name: Check toolcache contains CodeQL
continue-on-error: true
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts installed
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const toolcache = require('@actions/tool-cache');
const allCodeqlVersions = toolcache.findAllVersions('CodeQL');
if (allCodeqlVersions.length === 0) {
throw new Error(`CodeQL could not be found in the toolcache`);
}
- id: setup-codeql
uses: ./../action/setup-codeql
with:
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Check CodeQL is installed within the toolcache
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts installed
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const toolcache = require('@actions/tool-cache');
const allCodeqlVersions = toolcache.findAllVersions('CodeQL');
console.log(`Found CodeQL versions: ${allCodeqlVersions}`);
if (allCodeqlVersions.length === 0) {
throw new Error('CodeQL not found in toolcache');
}
env:
CODEQL_ACTION_TEST_MODE: true
+30 -22
View File
@@ -50,15 +50,6 @@ jobs:
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
@@ -67,30 +58,47 @@ jobs:
use-all-platform-bundle: 'false'
setup-kotlin: 'true'
- name: Remove CodeQL from toolcache
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts clear
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const fs = require('fs');
const path = require('path');
const codeqlPath = path.join(process.env['RUNNER_TOOL_CACHE'], 'CodeQL');
fs.rmdirSync(codeqlPath, { recursive: true });
- name: Install @actions/tool-cache
run: npm install @actions/tool-cache@3
- name: Check toolcache does not contain CodeQL
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts absent
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const toolcache = require('@actions/tool-cache');
const allCodeqlVersions = toolcache.findAllVersions('CodeQL');
if (allCodeqlVersions.length !== 0) {
throw new Error(`CodeQL should not be found in the toolcache, but found ${allCodeqlVersions}`);
}
console.log('No versions of CodeQL found in the toolcache');
- id: init
uses: ./../action/init
with:
# Request multiple languages so this check uses the combined bundle.
languages: javascript,python
tools: ${{ steps.prepare-test.outputs.tools-url }}
- uses: ./../action/analyze
with:
output: ${{ runner.temp }}/results
upload-database: false
- name: Check CodeQL is installed within the toolcache
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts installed -n 1
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const toolcache = require('@actions/tool-cache');
const allCodeqlVersions = toolcache.findAllVersions('CodeQL');
console.log(`Found CodeQL versions: ${allCodeqlVersions}`);
if (allCodeqlVersions.length === 0) {
throw new Error('CodeQL not found in toolcache');
}
if (allCodeqlVersions.length > 1) {
throw new Error('Multiple CodeQL versions found in toolcache');
}
env:
CODEQL_ACTION_TEST_MODE: true
+47 -13
View File
@@ -48,15 +48,6 @@ jobs:
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
@@ -90,13 +81,56 @@ jobs:
name: diagnostics-export-${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: '${{ runner.temp }}/results/javascript.sarif'
retention-days: 7
- name: Check diagnostics appear in SARIF
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
env:
SARIF_PATH: '${{ runner.temp }}/results/javascript.sarif'
run: |
cd ../action
npx tsx ./pr-checks/check-diagnostics.ts
with:
script: |
const fs = require('fs');
function checkStatusPageNotification(n) {
const expectedMessage = 'Plaintext message';
if (n.message.text !== expectedMessage) {
core.setFailed(`Expected the status page diagnostic to have the message '${expectedMessage}', but found '${n.message.text}'.`);
}
if (n.locations.length !== 1) {
core.setFailed(`Expected the status page diagnostic to have exactly 1 location, but found ${n.locations.length}.`);
}
}
const sarif = JSON.parse(fs.readFileSync(process.env['SARIF_PATH'], 'utf8'));
const run = sarif.runs[0];
const toolExecutionNotifications = run.invocations[0].toolExecutionNotifications;
const statusPageNotifications = toolExecutionNotifications.filter(n =>
n.descriptor.id === 'lang/diagnostics/example' && n.properties?.visibility?.statusPage
);
if (statusPageNotifications.length !== 1) {
core.setFailed(
'Expected exactly one status page reporting descriptor for this diagnostic in the ' +
`'runs[].invocations[].toolExecutionNotifications[]' SARIF property, but found ` +
`${statusPageNotifications.length}. All notification reporting descriptors: ` +
`${JSON.stringify(toolExecutionNotifications)}.`
);
}
checkStatusPageNotification(statusPageNotifications[0]);
const notifications = run.tool.driver.notifications;
const diagnosticNotification = notifications.filter(n =>
n.id === 'lang/diagnostics/example' && n.name === 'lang/diagnostics/example' &&
n.fullDescription.text === 'Diagnostic name'
);
if (diagnosticNotification.length !== 1) {
core.setFailed(
'Expected exactly one notification for this diagnostic in the ' +
`'runs[].tool.driver.notifications[]' SARIF property, but found ` +
`${diagnosticNotification.length}. All notifications: ` +
`${JSON.stringify(notifications)}.`
);
}
core.info('Finished diagnostic export test');
env:
CODEQL_ACTION_EXPORT_DIAGNOSTICS: true
CODEQL_ACTION_TEST_MODE: true
-143
View File
@@ -1,143 +0,0 @@
# Warning: This file is generated automatically, and should not be modified.
# Instead, please modify the template in the pr-checks directory and run:
# pr-checks/sync.sh
# to regenerate this file.
name: PR Check - Diagnostic export without database
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GO111MODULE: auto
on:
push:
branches:
- main
- releases/v*
pull_request: {}
merge_group:
types:
- checks_requested
schedule:
- cron: '0 5 * * *'
workflow_dispatch:
inputs: {}
workflow_call:
inputs: {}
defaults:
run:
shell: bash
concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' || false }}
group: diagnostics-without-db-${{github.ref}}
jobs:
diagnostics-without-db:
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
version: linked
name: Diagnostic export without database
if: github.triggering_actor != 'dependabot[bot]'
permissions:
contents: read
security-events: read
timeout-minutes: 45
runs-on: ${{ matrix.os }}
steps:
# We deliberately need to make this step fail before the database is initialised, but after
# some diagnostic has been produced. The simplest way to do that right now is to:
#
# - Produce a diagnostic before the db is initialised by forcing the use of a nightly release
# with the `CODEQL_ACTION_FORCE_NIGHTLY` feature flag.
# - Cause `init` to abort because we are attempting to analyse Swift on a non-macOS runner.
#
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
with:
version: ${{ matrix.version }}
use-all-platform-bundle: 'false'
setup-kotlin: 'true'
- name: Initialise failing analysis
id: init
uses: ./../action/init
continue-on-error: true
env:
CODEQL_ACTION_FORCE_NIGHTLY: true
with:
languages: swift
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Analyze
uses: ./analyze
# In a real workflow, this step wouldn't run. Since we used `continue-on-error`
# above, we manually disable it with an `if` condition. Having this step is required
# for the init post-action because it checks the workflow definition for e.g. the
# `category` input. Without this step, that part of the init post-action would fail.
if: false
with:
category: '/test-codeql-version:${{ matrix.version }}'
- name: Check diagnostics appear on disk
env:
DIAGNOSTICS_DIR: '${{ runner.temp }}/codeql-action-temp-diagnostics/'
run: |
cd ../action
npx tsx ./pr-checks/diagnostics-without-db.ts
env:
CODEQL_ACTION_SKIP_GENERATE_FAILED_SARIF: false
CODEQL_ACTION_SKIP_SARIF_UPLOAD: true
CODEQL_ACTION_TESTING_ENVIRONMENT: codeql-action-pr-checks
CODEQL_ACTION_TEST_MODE: true
diagnostics-without-db-validation:
if: github.triggering_actor != 'dependabot[bot]'
needs:
- diagnostics-without-db
permissions:
contents: read
security-events: read
timeout-minutes: 5
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Download workflow artifact
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
pattern: test-mode-sarif-*
path: sarif
- name: List artifact contents
run: ls -laR ./sarif
- name: Check diagnostics appear in SARIF
env:
SARIF_PATH: "./sarif/codeql-failed-run.sarif"
DIAGNOSTIC_ID: "codeql-action/forced-nightly-cli"
DIAGNOSTIC_NAME: "A nightly release of CodeQL was used"
EXPECTED_LOCATIONS: "0"
EXPECTED_MESSAGE: |
GitHub configured this analysis to use a nightly release of CodeQL to allow you to preview changes from an upcoming release.
run: npx tsx ./pr-checks/check-diagnostics.ts
env:
CODEQL_ACTION_TEST_MODE: true
+1 -5
View File
@@ -91,11 +91,6 @@ jobs:
with:
languages: javascript
tools: ${{ steps.prepare-test.outputs.tools-url }}
# To balance speed and coverage, analyze only JavaScript but use the combined bundle so
# baseline information is reported for every language in the multi-language repository. A
# custom configuration prevents per-language bundles from being used.
config: |
paths-ignore: []
- name: Build code
run: ./build.sh
- uses: ./../action/analyze
@@ -129,4 +124,5 @@ jobs:
env:
CODEQL_ACTION_SKIP_FILE_COVERAGE_ON_PRS: false
CODEQL_ACTION_SUBLANGUAGE_FILE_COVERAGE: true
CODEQL_ACTION_PER_LANGUAGE_BUNDLES: false
CODEQL_ACTION_TEST_MODE: true
+1 -1
View File
@@ -44,7 +44,7 @@ jobs:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
- os: ubuntu-latest
version: stable-v2.20.7
- os: ubuntu-latest
version: stable-v2.21.4
+1 -1
View File
@@ -44,7 +44,7 @@ jobs:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
- os: ubuntu-latest
version: stable-v2.20.7
- os: ubuntu-latest
version: stable-v2.21.4
+1 -1
View File
@@ -44,7 +44,7 @@ jobs:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
- os: ubuntu-latest
version: stable-v2.20.7
- os: ubuntu-latest
version: stable-v2.21.4
+2 -1
View File
@@ -54,7 +54,7 @@ jobs:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
- os: ubuntu-latest
version: stable-v2.20.7
- os: macos-15-xlarge
version: stable-v2.20.7
@@ -192,5 +192,6 @@ jobs:
exit 1
fi
env:
CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES: true
CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI: true
CODEQL_ACTION_TEST_MODE: true
+14 -25
View File
@@ -36,40 +36,40 @@ jobs:
include:
- language: actions
os: ubuntu-latest
version: linked
version: nightly-latest
expected-extractors: actions javascript
- language: cpp
os: ubuntu-latest
version: linked
version: nightly-latest
build-mode: manual
build-command: gcc -o main main.c
- language: csharp
os: ubuntu-latest
version: linked
version: nightly-latest
build-mode: none
- language: go
os: ubuntu-latest
version: linked
version: nightly-latest
build-mode: autobuild
- language: java
os: ubuntu-latest
version: linked
version: nightly-latest
build-mode: none
- language: javascript
os: ubuntu-latest
version: linked
version: nightly-latest
- language: python
os: ubuntu-latest
version: linked
version: nightly-latest
- language: ruby
os: ubuntu-latest
version: linked
version: nightly-latest
- language: rust
os: ubuntu-latest
version: linked
version: nightly-latest
- language: swift
os: macos-latest-xlarge
version: linked
version: nightly-latest
build-mode: autobuild
name: Per-language bundles
if: github.triggering_actor != 'dependabot[bot]'
@@ -81,15 +81,6 @@ jobs:
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
@@ -97,10 +88,6 @@ jobs:
version: ${{ matrix.version }}
use-all-platform-bundle: 'false'
setup-kotlin: 'true'
- name: Remove CodeQL from toolcache
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts clear
- uses: ./../action/init
id: init
with:
@@ -141,8 +128,9 @@ jobs:
CODEQL_PATH: ${{ steps.init.outputs.codeql-path }}
run: |
# A bundle that is missing most of its extractors must never be left in the toolcache,
# where a later job analyzing a different language could pick it up. Per-language bundles
# must instead be extracted under the runner's temporary directory.
# where a later job analyzing a different language could pick it up. The runner image
# ships with its own CodeQL in the toolcache, so check where this bundle was extracted to
# rather than whether the toolcache contains CodeQL at all.
echo "CodeQL is at $CODEQL_PATH"
if [[ "$CODEQL_PATH" == "$RUNNER_TOOL_CACHE"/* ]]; then
echo "::error::The per-language bundle was added to the toolcache at $CODEQL_PATH."
@@ -172,4 +160,5 @@ jobs:
fi
echo "Created a ${LANGUAGE} database at ${database}."
env:
CODEQL_ACTION_PER_LANGUAGE_BUNDLES: true
CODEQL_ACTION_TEST_MODE: true
+74
View File
@@ -0,0 +1,74 @@
# Warning: This file is generated automatically, and should not be modified.
# Instead, please modify the template in the pr-checks directory and run:
# pr-checks/sync.sh
# to regenerate this file.
name: PR Check - RuboCop multi-language
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GO111MODULE: auto
on:
push:
branches:
- main
- releases/v*
pull_request: {}
merge_group:
types:
- checks_requested
schedule:
- cron: '0 5 * * *'
workflow_dispatch:
inputs: {}
workflow_call:
inputs: {}
defaults:
run:
shell: bash
concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' || false }}
group: rubocop-multi-language-${{github.ref}}
jobs:
rubocop-multi-language:
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
version: default
name: RuboCop multi-language
if: github.triggering_actor != 'dependabot[bot]'
permissions:
contents: read
security-events: read
timeout-minutes: 45
runs-on: ${{ matrix.os }}
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
with:
version: ${{ matrix.version }}
use-all-platform-bundle: 'false'
setup-kotlin: 'true'
- name: Set up Ruby
uses: ruby/setup-ruby@14594264cd68ce8a2345dd349bc3d138a4ef85c8 # v1.327.0
with:
ruby-version: 2.6
- name: Install Code Scanning integration
run: bundle add code-scanning-rubocop --version 0.3.0 --skip-install
- name: Install dependencies
run: bundle install
- name: RuboCop run
run: |
bash -c "
bundle exec rubocop --require code_scanning --format CodeScanning::SarifFormatter -o rubocop.sarif
[[ $? -ne 2 ]]
"
- uses: ./../action/upload-sarif
with:
sarif_file: rubocop.sarif
env:
CODEQL_ACTION_TEST_MODE: true
+1 -1
View File
@@ -34,7 +34,7 @@ jobs:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
- os: ubuntu-latest
version: stable-v2.20.7
- os: ubuntu-latest
version: stable-v2.22.1
+1 -3
View File
@@ -71,9 +71,7 @@ jobs:
run: exit 1
- uses: ./analyze
# In a real workflow, this step wouldn't run. Since we used `continue-on-error`
# above, we manually disable it with an `if` condition. Having this step is required
# for the init post-action because it checks the workflow definition for e.g. the
# `category` input. Without this step, that part of the init post-action would fail.
# above, we manually disable it with an `if` condition.
if: false
with:
category: '/test-codeql-version:${{ matrix.version }}'
@@ -6,13 +6,13 @@ env:
on:
push:
branches:
- main
- releases/v*
- main
- releases/v*
pull_request:
merge_group:
types: [checks_requested]
schedule:
- cron: "0 5 * * *"
- cron: '0 5 * * *'
workflow_dispatch:
concurrency:
@@ -30,10 +30,10 @@ jobs:
fail-fast: false
matrix:
version:
- stable-v2.20.7
- default
- linked
- nightly-latest
- stable-v2.20.7
- default
- linked
- nightly-latest
name: Upload debug artifacts after failure in analyze
continue-on-error: true
env:
@@ -43,12 +43,7 @@ jobs:
# We currently need `security-events: read` to access feature flags.
security-events: read
timeout-minutes: 45
# stable-v2.20.7 requires an older version of Java than the default on `ubuntu-latest` (`ubuntu-26.04`).
# We could install that older version on the newer runners, or use the older runner.
# We do the latter here. This can be removed once we stop supporting `v2.20.7`.
runs-on: ${{ case(matrix.version == 'stable-v2.20.7', 'ubuntu-24.04', 'ubuntu-latest') }}
runs-on: ubuntu-latest
steps:
- name: Dump GitHub event
run: cat "${GITHUB_EVENT_PATH}"
@@ -65,7 +60,7 @@ jobs:
- name: Install .NET
uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0
with:
dotnet-version: "9.x"
dotnet-version: '9.x'
- name: Assert best-effort artifact scan completed
uses: ./../action/.github/actions/verify-debug-artifact-scan-completed
- uses: ./../action/init
@@ -97,7 +92,6 @@ jobs:
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
- name: Check expected artifacts exist
run: |
VERSIONS="stable-v2.20.7 default linked nightly-latest"
LANGUAGES="cpp csharp go java javascript python"
for version in $VERSIONS; do
echo "Artifacts from version $version:"
+9 -14
View File
@@ -5,13 +5,13 @@ env:
on:
push:
branches:
- main
- releases/v*
- main
- releases/v*
pull_request:
merge_group:
types: [checks_requested]
schedule:
- cron: "0 5 * * *"
- cron: '0 5 * * *'
workflow_dispatch:
concurrency:
@@ -29,10 +29,10 @@ jobs:
fail-fast: false
matrix:
version:
- stable-v2.20.7
- default
- linked
- nightly-latest
- stable-v2.20.7
- default
- linked
- nightly-latest
name: Upload debug artifacts
env:
CODEQL_ACTION_TEST_MODE: true
@@ -41,12 +41,7 @@ jobs:
contents: read
# We currently need `security-events: read` to access feature flags.
security-events: read
# stable-v2.20.7 requires an older version of Java than the default on `ubuntu-latest` (`ubuntu-26.04`).
# We could install that older version on the newer runners, or use the older runner.
# We do the latter here. This can be removed once we stop supporting `v2.20.7`.
runs-on: ${{ case(matrix.version == 'stable-v2.20.7', 'ubuntu-24.04', 'ubuntu-latest') }}
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -61,7 +56,7 @@ jobs:
- name: Install .NET
uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0
with:
dotnet-version: "9.x"
dotnet-version: '9.x'
- name: Assert best-effort artifact scan completed
uses: ./../action/.github/actions/verify-debug-artifact-scan-completed
- uses: ./../action/init
+1
View File
@@ -60,6 +60,7 @@ on:
- "PR Check - Proxy test"
- "PR Check - Remote config file"
- "PR Check - Resolve environment"
- "PR Check - RuboCop multi-language"
- "PR Check - Ruby analysis"
- "PR Check - Rust analysis"
- "PR Check - Split workflow"
+1 -1
View File
@@ -134,7 +134,7 @@ jobs:
echo "::endgroup::"
- name: Generate token
uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
id: app-token
with:
app-id: ${{ vars.AUTOMATION_APP_ID }}
+5 -7
View File
@@ -2,9 +2,6 @@ name: PR Checks
on:
push:
branches:
- "main"
- "releases/v*"
pull_request:
merge_group:
types: [checks_requested]
@@ -148,11 +145,12 @@ jobs:
- name: Fetch base commit
id: fetch-base
# Skip this (and subsequent steps) if the PR originates from a fork, since this workflow
# won't have the necessary permissions to post comments on the PR.
# Forks and Dependabot PRs don't have permission to write comments, so skip the repo size
# check in those cases.
if: >-
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.repository
github.event.pull_request.head.repo.full_name == github.repository &&
github.event.pull_request.user.login != 'dependabot[bot]'
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
@@ -211,7 +209,7 @@ jobs:
if: >-
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.repository &&
github.actor != 'dependabot[bot]' &&
github.event.pull_request.user.login != 'dependabot[bot]' &&
needs.other-checks.result == 'success'
permissions:
contents: read
-43
View File
@@ -1,43 +0,0 @@
name: PR dequeued
on:
pull_request:
types:
- dequeued
jobs:
notify:
name: Notify
runs-on: ubuntu-slim
if: github.event.pull_request.merged != true
permissions:
pull-requests: write
steps:
- name: Notify that the PR was removed from the merge queue
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REASON: ${{ github.event.reason }}
run: |
# Mention whoever added the PR to the merge queue rather than the PR author, who may be a
# bot such as Dependabot. The `dequeued` event payload doesn't identify the enqueuer, so
# look them up in the most recent `AddedToMergeQueueEvent` on the PR's timeline.
enqueuer=$(gh api graphql \
-F number="$PR_NUMBER" \
-f query='
query($number: Int!) {
repository(owner: "github", name: "codeql-action") {
pullRequest(number: $number) {
timelineItems(itemTypes: [ADDED_TO_MERGE_QUEUE_EVENT], last: 1) {
nodes { ... on AddedToMergeQueueEvent { enqueuer { login } } }
}
}
}
}' \
--jq '.data.repository.pullRequest.timelineItems.nodes[0].enqueuer.login')
echo "@$enqueuer The pull request was removed from the merge queue (\`$REASON\`)." \
"Please resolve any issues if required and enqueue the PR again." |
gh pr comment --body-file - --repo github/codeql-action "$PR_NUMBER"
+1 -1
View File
@@ -139,7 +139,7 @@ jobs:
- name: Generate token
if: github.event_name == 'workflow_dispatch'
uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
id: app-token
with:
app-id: ${{ vars.AUTOMATION_APP_ID }}
+1 -1
View File
@@ -95,7 +95,7 @@ jobs:
pull-requests: write # needed to create pull request
steps:
- name: Generate token
uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
id: app-token
with:
app-id: ${{ vars.AUTOMATION_APP_ID }}
-4
View File
@@ -6,10 +6,6 @@ See the [releases page](https://github.com/github/codeql-action/releases) for th
No user facing changes.
## 4.39.1 - 21 Oct 2026
- Update default CodeQL bundle version to [2.27.3](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.27.3). [#4256](https://github.com/github/codeql-action/pull/4256)
## 4.39.0 - 15 Oct 2026
- _Breaking change_: Bump the minimum required CodeQL bundle version to 2.20.7. [#4205](https://github.com/github/codeql-action/pull/4205)
+1
View File
@@ -76,6 +76,7 @@ We typically release new minor versions of the CodeQL Action and Bundle when a n
| `v4.33.0` | `2.24.3` | Enterprise Server 3.21 | |
| `v4.31.10` | `2.23.9` | Enterprise Server 3.20 | |
| `v3.29.11` | `2.22.4` | Enterprise Server 3.19 | |
| `v3.28.21` | `2.21.3` | Enterprise Server 3.18 | |
See the full list of GHES release and deprecation dates at [GitHub Enterprise Server releases](https://docs.github.com/en/enterprise-server/admin/all-releases#releases-of-github-enterprise-server).
+1 -4
View File
@@ -153,10 +153,7 @@ inputs:
required: false
trap-caching:
description: >-
Explicitly enable or disable TRAP caching instead of using the automatically determined default.
TRAP caching is disabled by default when improved incremental analysis is enabled,
because improved incremental analysis of a pull request only extracts changed files,
which rarely benefit from TRAP caching.
Explicitly enable or disable TRAP caching rather than respecting the feature flag for it.
required: false
dependency-caching:
description: >-
+4 -4
View File
@@ -1,6 +1,6 @@
{
"bundleVersion": "codeql-bundle-v2.27.3",
"cliVersion": "2.27.3",
"priorBundleVersion": "codeql-bundle-v2.27.2",
"priorCliVersion": "2.27.2"
"bundleVersion": "codeql-bundle-v2.27.2",
"cliVersion": "2.27.2",
"priorBundleVersion": "codeql-bundle-v2.27.1",
"priorCliVersion": "2.27.1"
}
+7015 -14781
View File
File diff suppressed because it is too large. Load diff
+132 -215
View File
@@ -1,12 +1,12 @@
{
"name": "codeql",
"version": "4.39.2",
"version": "4.39.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "codeql",
"version": "4.39.2",
"version": "4.39.1",
"license": "MIT",
"workspaces": [
"pr-checks"
@@ -30,13 +30,13 @@
"follow-redirects": "^1.16.1",
"get-folder-size": "^5.0.0",
"https-proxy-agent": "^7.0.6",
"js-yaml": "^5.5.0",
"js-yaml": "^5.4.3",
"jsonschema": "1.5.0",
"long": "^5.3.2",
"node-forge": "^1.4.0",
"semver": "^7.8.5",
"undici": "^7.30.1",
"uuid": "^14.0.3"
"undici": "^6.28.0",
"uuid": "^14.0.2"
},
"devDependencies": {
"@ava/typescript": "6.0.0",
@@ -50,21 +50,29 @@
"@types/node-forge": "^1.3.14",
"@types/sarif": "^2.1.7",
"@types/semver": "^7.8.0",
"@types/sinon": "^22.0.1",
"@types/sinon": "^22.0.0",
"ava": "^6.4.1",
"esbuild": "^0.28.2",
"eslint": "^9.39.5",
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-github": "^6.1.2",
"eslint-plugin-import-x": "^4.17.1",
"eslint-plugin-jsdoc": "^65.4.0",
"eslint-plugin-jsdoc": "^65.2.0",
"eslint-plugin-no-async-foreach": "^0.1.1",
"glob": "^13.0.6",
"globals": "^17.13.0",
"nock": "^14.0.17",
"sinon": "^22.1.0",
"typescript": "^6.0.3",
"typescript-eslint": "^8.72.0"
"typescript-eslint": "^8.71.1"
}
},
"node_modules/@aashutoshrathi/word-wrap": {
"version": "1.2.6",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/@actions/artifact": {
@@ -950,89 +958,18 @@
"tslib": "^2.4.0"
}
},
"node_modules/@es-joy/escodegen": {
"version": "4.2.0",
"resolved": "https://registry.npmjs.org/@es-joy/escodegen/-/escodegen-4.2.0.tgz",
"integrity": "sha512-jWqqS3StW6vGwJeT8yu6alS4nipOBi3kD/pR63+9hG2NfGafCJ4viDME15mVf+fyupyptcUJDXfv63gFF5Gzfg==",
"dev": true,
"license": "BSD-2-Clause",
"dependencies": {
"@es-joy/estraverse": "^7.1.1",
"@es-joy/jsdoccomment": "^0.62.0",
"esprima": "^4.0.1",
"esutils": "^2.0.3",
"optionator": "^0.9.4"
},
"bin": {
"escodegen": "bin/escodegen.js",
"esgenerate": "bin/esgenerate.js"
},
"engines": {
"node": ">=6.0"
},
"optionalDependencies": {
"source-map": "~0.7.6"
}
},
"node_modules/@es-joy/escodegen/node_modules/@es-joy/jsdoccomment": {
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@es-joy/jsdoccomment/-/jsdoccomment-0.62.0.tgz",
"integrity": "sha512-yWi6sm7INEwnfS7IJvE0dU+RTrwzLPFcY7e7eGpu/l5Q9lWfQ2ROwZ0qVnc242jw2TUPsfHX3XMIISkGBv57RQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/estree": "^1.0.8",
"@typescript-eslint/types": "^8.44.1",
"comment-parser": "1.4.1",
"esquery": "^1.6.0",
"jsdoc-type-pratt-parser": "~5.9.0"
},
"engines": {
"node": ">=20.11.0"
}
},
"node_modules/@es-joy/escodegen/node_modules/comment-parser": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/comment-parser/-/comment-parser-1.4.1.tgz",
"integrity": "sha512-buhp5kePrmda3vhc5B9t7pUQXAb2Tnd0qgpkIhPhkHXxJpiPJ11H0ZEU0oBpJ2QztSbzG/ZxMj/CHsYJqRHmyg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 12.0.0"
}
},
"node_modules/@es-joy/escodegen/node_modules/jsdoc-type-pratt-parser": {
"version": "5.9.2",
"resolved": "https://registry.npmjs.org/jsdoc-type-pratt-parser/-/jsdoc-type-pratt-parser-5.9.2.tgz",
"integrity": "sha512-TYzkACp/wPvDJLRY7qpHXtrhgwoAaojIOnLaaVNi+AbPU2u1kkjfKd9hXXTq0qSAGsyYXvwUXt99h9I5iCmjjw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=12.0.0"
}
},
"node_modules/@es-joy/estraverse": {
"version": "7.1.1",
"resolved": "https://registry.npmjs.org/@es-joy/estraverse/-/estraverse-7.1.1.tgz",
"integrity": "sha512-h2d737iS0LFAqypZOmXizu54qAbHofI/RpD30Zom03O6BCXKHuFevWKPcGarZbGq82h7qSBb5G6kLdhTMO+caw==",
"dev": true,
"license": "BSD-2-Clause",
"engines": {
"node": ">=4.0"
}
},
"node_modules/@es-joy/jsdoccomment": {
"version": "0.100.0",
"resolved": "https://registry.npmjs.org/@es-joy/jsdoccomment/-/jsdoccomment-0.100.0.tgz",
"integrity": "sha512-QuIT5kMg3XWN22Wa1rcXmR/GkfSg6Ygs/ikMiHihdSpu+o3O59LygkFV9srKa5xHplBFwNd5ZCo3bywL4k7qMQ==",
"version": "0.98.0",
"resolved": "https://registry.npmjs.org/@es-joy/jsdoccomment/-/jsdoccomment-0.98.0.tgz",
"integrity": "sha512-QS+b7I2aaSM+gZfDz1b9E4B367JJ2qTjFXRKF289iKkh9sYdZ21Q5MiF/Duzouy3J0C4VhM3I/8f2F4+zT8gYQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/estree": "^1.0.9",
"@typescript-eslint/types": "^8.72.0",
"@typescript-eslint/types": "^8.70.0",
"comment-parser": "1.4.9",
"esquery": "^1.7.0",
"jsdoc-type-pratt-parser": "~9.3.0"
"jsdoc-type-pratt-parser": "~9.2.2"
},
"engines": {
"node": "^22.22.2 || >=24.15.0"
@@ -2744,15 +2681,20 @@
"license": "MIT"
},
"node_modules/@types/sinon": {
"version": "22.0.1",
"resolved": "https://registry.npmjs.org/@types/sinon/-/sinon-22.0.1.tgz",
"integrity": "sha512-MThluJXPmpBY+Z5S3565F2z+hjkCXnBrr36j2mitVQ7JUtxA+A+YQhs+0Q6J8jxb0SVJvydO+6NWfj9EBzpiBQ==",
"version": "22.0.0",
"resolved": "https://registry.npmjs.org/@types/sinon/-/sinon-22.0.0.tgz",
"integrity": "sha512-TDbVpbccc2HfiqHR09Argj3mHV1KMW7sCCKj52fsl8lbRLkEn7fB1966EWhOKWUBcqfBueZuPoA7/OK1CKiy3g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@sinonjs/fake-timers": "^15.4.0"
"@types/sinonjs__fake-timers": "*"
}
},
"node_modules/@types/sinonjs__fake-timers": {
"version": "8.1.2",
"dev": true,
"license": "MIT"
},
"node_modules/@types/unist": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/@types/unist/-/unist-3.0.3.tgz",
@@ -2760,17 +2702,17 @@
"license": "MIT"
},
"node_modules/@typescript-eslint/eslint-plugin": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.72.0.tgz",
"integrity": "sha512-aysKgysPX9XOv/dk95UkvUDe++Q3n3dJPG9CEWT3RUccxfkPJ+Qn6nhKPDhZXU6Ofo9kgwGWwKyQCnZIXTgngg==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.71.1.tgz",
"integrity": "sha512-nNlBf6HuotqMgZvSkWD5hNdemiBIxQ2NmrlgfcUIeZRs+BbakjscCQRq/Hb74EFVnFiSWWdL12JruqwBy3ERDw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@eslint-community/regexpp": "^4.12.2",
"@typescript-eslint/scope-manager": "8.72.0",
"@typescript-eslint/type-utils": "8.72.0",
"@typescript-eslint/utils": "8.72.0",
"@typescript-eslint/visitor-keys": "8.72.0",
"@typescript-eslint/scope-manager": "8.71.1",
"@typescript-eslint/type-utils": "8.71.1",
"@typescript-eslint/utils": "8.71.1",
"@typescript-eslint/visitor-keys": "8.71.1",
"ignore": "^7.0.5",
"natural-compare": "^1.4.0",
"ts-api-utils": "^2.5.0"
@@ -2783,7 +2725,7 @@
"url": "https://opencollective.com/typescript-eslint"
},
"peerDependencies": {
"@typescript-eslint/parser": "^8.72.0",
"@typescript-eslint/parser": "^8.71.1",
"eslint": "^8.57.0 || ^9.0.0 || ^10.0.0",
"typescript": ">=4.8.4 <6.1.0"
}
@@ -2799,16 +2741,16 @@
}
},
"node_modules/@typescript-eslint/parser": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.72.0.tgz",
"integrity": "sha512-0O+x3ynciD6zuXb1CKSu165haQV6ey53DKhHHUnujP7U+XlltJG+/keT0Xhd1HXCze+IHf7Tvt39ovyepX/LNw==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.71.1.tgz",
"integrity": "sha512-oM/5sAqz/l1v/zYK1qTOPrn2+JIrBGC1V6uNYucRO5gh6eT9vWJ/RhPbGE59byvZEUQ4VP6XkLBaS5Jti5jpvw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/scope-manager": "8.72.0",
"@typescript-eslint/types": "8.72.0",
"@typescript-eslint/typescript-estree": "8.72.0",
"@typescript-eslint/visitor-keys": "8.72.0",
"@typescript-eslint/scope-manager": "8.71.1",
"@typescript-eslint/types": "8.71.1",
"@typescript-eslint/typescript-estree": "8.71.1",
"@typescript-eslint/visitor-keys": "8.71.1",
"debug": "^4.4.3"
},
"engines": {
@@ -2842,14 +2784,14 @@
}
},
"node_modules/@typescript-eslint/project-service": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.72.0.tgz",
"integrity": "sha512-bzHngBbmIgzj6TsTjVUCntGx6FFLgE7mPjpO0PzHycOaZHXqS43XnrkFIHRS16LLr/xO6FHSXG3GJFo/F4bSBQ==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.71.1.tgz",
"integrity": "sha512-Oijc9RUsohHjncg6iGi7gCtjnEjYt9FWv9u6ygK8uKcHK5eLjLAkqERQ9BaytbpceOFdk6kifPCS/zAZE+A99g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/tsconfig-utils": "^8.72.0",
"@typescript-eslint/types": "^8.72.0",
"@typescript-eslint/tsconfig-utils": "^8.71.1",
"@typescript-eslint/types": "^8.71.1",
"debug": "^4.4.3"
},
"engines": {
@@ -2882,14 +2824,14 @@
}
},
"node_modules/@typescript-eslint/scope-manager": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.72.0.tgz",
"integrity": "sha512-JewPyJOYcXBSAS7PIgufscHOwZqhTGyBRiQJhhuDln66tMchxvvt7nJ90JKKIoOelmdHA3kAQbm9cEUHc4JSEA==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.71.1.tgz",
"integrity": "sha512-1YUDZXdTnXLlob03SFHBipdyMBsDVm7zoSz0ytTJ39dk5J1TgsTK03VlR/dLXG/RLLUrPcYbD4rp23BmHxHP0w==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/types": "8.72.0",
"@typescript-eslint/visitor-keys": "8.72.0"
"@typescript-eslint/types": "8.71.1",
"@typescript-eslint/visitor-keys": "8.71.1"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -2900,9 +2842,9 @@
}
},
"node_modules/@typescript-eslint/tsconfig-utils": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.72.0.tgz",
"integrity": "sha512-0vbxCf7xLVPSjcOdm6eQbgViH8EDJ8xnl1laLT9yoqSz2YmcSfSg5Vdl0JUvZf6rcuJ0ZivJIuMD4edTg8obaA==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.71.1.tgz",
"integrity": "sha512-oTvkml5SxXhgg+WWKhod5qWQICDXtOB+/Hi9VdPKFAANJ2b7YOgMffgvzXuqog7R6A/JtrzRf128dsxummyWhg==",
"dev": true,
"license": "MIT",
"engines": {
@@ -2917,15 +2859,15 @@
}
},
"node_modules/@typescript-eslint/type-utils": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-8.72.0.tgz",
"integrity": "sha512-gEXxmmbSTSOVTC37IYYvremOM31v+UN2W772bJxmVYPhAOPSUxtacyb7wkm1EhkYKU6KzFJsdU44DCV4to0LWw==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-8.71.1.tgz",
"integrity": "sha512-kH3t3jZCYv2PZSBEWPKTariLrCaoF1CLoPitYFBJOhAGEwWnSRAtGEifrfrpBa4K9JlwWNZotgA5w3/wLSk/FQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/types": "8.72.0",
"@typescript-eslint/typescript-estree": "8.72.0",
"@typescript-eslint/utils": "8.72.0",
"@typescript-eslint/types": "8.71.1",
"@typescript-eslint/typescript-estree": "8.71.1",
"@typescript-eslint/utils": "8.71.1",
"debug": "^4.4.3",
"ts-api-utils": "^2.5.0"
},
@@ -2960,9 +2902,9 @@
}
},
"node_modules/@typescript-eslint/types": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.72.0.tgz",
"integrity": "sha512-Adfoy59kxNEjT5E5fnyrp7GpHYuWs8ONbdohu4ez6IU46qDq8LltG90tLmkucZB2uhYL8mqBjtXqGd8yhfjXqQ==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.71.1.tgz",
"integrity": "sha512-ybYnPTUwg3VQQOFEiBPLs8Jxl1ry2vXdn1kVKguomBbvxpetdu3MKTNDqFNUrlG8ylP9MpZosmXMFYlGQHk1oQ==",
"dev": true,
"license": "MIT",
"engines": {
@@ -2974,16 +2916,16 @@
}
},
"node_modules/@typescript-eslint/typescript-estree": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.72.0.tgz",
"integrity": "sha512-gwLfFgWcw15tN2KXDCJvxEOQjPn+P3j/n3eSqkZ5sX4VpW9Cosodr+tWwBZRvJT3TcKx6HnPRGZMCNAiP1Xhug==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.71.1.tgz",
"integrity": "sha512-pSKSEK1JJpHs6KiKVtABCE4iJK5d7FLFzDXQNBQ2oRY7LEd3UwOt3TZemTgu9syTIf56dJWlTn/pdmrO9oKUXQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/project-service": "8.72.0",
"@typescript-eslint/tsconfig-utils": "8.72.0",
"@typescript-eslint/types": "8.72.0",
"@typescript-eslint/visitor-keys": "8.72.0",
"@typescript-eslint/project-service": "8.71.1",
"@typescript-eslint/tsconfig-utils": "8.71.1",
"@typescript-eslint/types": "8.71.1",
"@typescript-eslint/visitor-keys": "8.71.1",
"debug": "^4.4.3",
"minimatch": "^10.2.2",
"semver": "^7.7.3",
@@ -3059,16 +3001,16 @@
}
},
"node_modules/@typescript-eslint/utils": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.72.0.tgz",
"integrity": "sha512-0up3oArJZBYnzOaEXKo8DmI43QvMZyPoFEt9LWzRJcyl/RcJjfKKPebknN1VbTgXBf3M/KggUh5uUMdURLnseg==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.71.1.tgz",
"integrity": "sha512-CBjT6gfAz3DW2j9Q4moFnqt208Aq+506sVGXfpRcyAeqn0EpayL0mjdWoCO2t3RC7GewNrFxdeehuEaPg6DMzQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@eslint-community/eslint-utils": "^4.9.1",
"@typescript-eslint/scope-manager": "8.72.0",
"@typescript-eslint/types": "8.72.0",
"@typescript-eslint/typescript-estree": "8.72.0"
"@typescript-eslint/scope-manager": "8.71.1",
"@typescript-eslint/types": "8.71.1",
"@typescript-eslint/typescript-estree": "8.71.1"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -3083,13 +3025,13 @@
}
},
"node_modules/@typescript-eslint/visitor-keys": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.72.0.tgz",
"integrity": "sha512-E2mroviasaWecDzTGiGMtuEguuLVOeRggnrIn93CJuxYJzQBHbyymlT08aVQydbBal5538o+JkBKXh2sRuCyIA==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.71.1.tgz",
"integrity": "sha512-0GxiUGqMU0qXJt58cBsiG3DP8lUmBaFo2vChtUX+CT7PnR3v/VGLLrzVP4K9r1lNiZXjHYefMBbPunplr09v6w==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/types": "8.72.0",
"@typescript-eslint/types": "8.71.1",
"eslint-visitor-keys": "^5.0.0"
},
"engines": {
@@ -3634,7 +3576,6 @@
},
"node_modules/argparse": {
"version": "2.0.1",
"dev": true,
"license": "Python-2.0"
},
"node_modules/aria-query": {
@@ -5424,16 +5365,15 @@
}
},
"node_modules/eslint-plugin-jsdoc": {
"version": "65.4.0",
"resolved": "https://registry.npmjs.org/eslint-plugin-jsdoc/-/eslint-plugin-jsdoc-65.4.0.tgz",
"integrity": "sha512-bRiU6LfCFmLL7GuePKvA6ZIcXLonhZGCg2TgQLmkcExeVdAirQp2Hsf3xeSUTmlKwAAsREVEOLbZYD8F7oA69g==",
"version": "65.2.0",
"resolved": "https://registry.npmjs.org/eslint-plugin-jsdoc/-/eslint-plugin-jsdoc-65.2.0.tgz",
"integrity": "sha512-IMZaNwdDa5otUQYGWN1pzywZLsGKswitvFFNhcO8IOsBfaWpWIDXYuZh2sYhe9MNyKN9hz0YryOFaKdpbTP5Bw==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
"@es-joy/escodegen": "^4.2.0",
"@es-joy/jsdoccomment": "~0.100.0",
"@es-joy/jsdoccomment": "~0.98.0",
"@es-joy/resolve.exports": "1.2.0",
"@typescript-eslint/utils": "^8.72.0",
"@typescript-eslint/utils": "^8.70.1",
"are-docs-informative": "^0.1.1",
"comment-parser": "1.4.9",
"debug": "^4.4.3",
@@ -7181,9 +7121,9 @@
}
},
"node_modules/js-yaml": {
"version": "5.5.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-5.5.0.tgz",
"integrity": "sha512-paNr4LFIFPjeeSmwNYdrb8FafLLG/Cr2Af3GjbsHJ5CxIjH4r0c2emlmRdAtpPAieP1gRVzk6buBbVP5gBMGAg==",
"version": "5.4.3",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-5.4.3.tgz",
"integrity": "sha512-yzzNtgczwgoVHNigCyKTXcQJU4Y8t/ikU4UxXW0BvrLfHTGWZhsNcOT2s/a7vn81/4KqVnNmHjaUEPlvWnNwVw==",
"funding": [
{
"type": "github",
@@ -7196,28 +7136,12 @@
],
"license": "MIT",
"dependencies": {
"argparse": "^3.0.3"
"argparse": "^2.0.1"
},
"bin": {
"js-yaml": "bin/js-yaml.mjs"
}
},
"node_modules/js-yaml/node_modules/argparse": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/argparse/-/argparse-3.0.3.tgz",
"integrity": "sha512-Sn98ftV3KfsO8xfLPHG6oT8y+b0yvbrLgZjgPDXsharNY15tTAjqIsIju4S32nl6/iEUNNE4v+yL2Zhm8ooP3w==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/puzrin"
},
{
"type": "github",
"url": "https://github.com/sponsors/nodeca"
}
],
"license": "PSF-2.0"
},
"node_modules/jschardet": {
"version": "3.1.4",
"resolved": "https://registry.npmjs.org/jschardet/-/jschardet-3.1.4.tgz",
@@ -7229,18 +7153,36 @@
}
},
"node_modules/jsdoc-type-pratt-parser": {
"version": "9.3.0",
"resolved": "https://registry.npmjs.org/jsdoc-type-pratt-parser/-/jsdoc-type-pratt-parser-9.3.0.tgz",
"integrity": "sha512-JZ8lKMCZXDlM1jAjAK5tTmtFJtgv6iFQQfjE3Fza6jBRACUueaEXLY17n1u05JO1lcPkNxgP+ddEDtiPsk5ktg==",
"version": "9.2.2",
"resolved": "https://registry.npmjs.org/jsdoc-type-pratt-parser/-/jsdoc-type-pratt-parser-9.2.2.tgz",
"integrity": "sha512-eFY7Rlscvv/BpY6UsVsjKwMhu1tJO/F4U/vC9g3yeRy/JqszuO1UP6Vdq+85G+3FP6fEw/sWR8a5ftPFY86csw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/estree": "^1.0.9"
"@types/estree": "^1.0.9",
"@types/node": "^26.4.0"
},
"engines": {
"node": "^22.22.2 || >=24.15.0"
}
},
"node_modules/jsdoc-type-pratt-parser/node_modules/@types/node": {
"version": "26.6.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-26.6.2.tgz",
"integrity": "sha512-X1P21scMv4zGKLYqjdGjaKa7COa0RKVYYZZN/NfvLQ1JegxFhdhpZG/Lyn8AXx6CDUavKAd11v6BvfpkDByK8g==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~8.9.0"
}
},
"node_modules/jsdoc-type-pratt-parser/node_modules/undici-types": {
"version": "8.9.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.9.0.tgz",
"integrity": "sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg==",
"dev": true,
"license": "MIT"
},
"node_modules/json-buffer": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/json-buffer/-/json-buffer-3.0.1.tgz",
@@ -8401,18 +8343,16 @@
}
},
"node_modules/optionator": {
"version": "0.9.4",
"resolved": "https://registry.npmjs.org/optionator/-/optionator-0.9.4.tgz",
"integrity": "sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g==",
"version": "0.9.3",
"dev": true,
"license": "MIT",
"dependencies": {
"@aashutoshrathi/word-wrap": "^1.2.3",
"deep-is": "^0.1.3",
"fast-levenshtein": "^2.0.6",
"levn": "^0.4.1",
"prelude-ls": "^1.2.1",
"type-check": "^0.4.0",
"word-wrap": "^1.2.5"
"type-check": "^0.4.0"
},
"engines": {
"node": ">= 0.8.0"
@@ -9336,17 +9276,6 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/source-map": {
"version": "0.7.6",
"resolved": "https://registry.npmjs.org/source-map/-/source-map-0.7.6.tgz",
"integrity": "sha512-i5uvt8C3ikiWeNZSVZNWcfZPItFQOsYTUAOkcUPGd8DqDy1uOUikjt5dG+uRlwyvR108Fb9DOd4GvXfT0N2/uQ==",
"dev": true,
"license": "BSD-3-Clause",
"optional": true,
"engines": {
"node": ">= 12"
}
},
"node_modules/spdx-exceptions": {
"version": "2.5.0",
"resolved": "https://registry.npmjs.org/spdx-exceptions/-/spdx-exceptions-2.5.0.tgz",
@@ -10052,16 +9981,16 @@
}
},
"node_modules/typescript-eslint": {
"version": "8.72.0",
"resolved": "https://registry.npmjs.org/typescript-eslint/-/typescript-eslint-8.72.0.tgz",
"integrity": "sha512-r26Fkc8As+mZvqp7SLsMmNFyYUup/6CjK7/eHBHmhOw6GH6n6YFxHVFrG0jN1c5CMpDzelLPxvgeCOzx7BjN+A==",
"version": "8.71.1",
"resolved": "https://registry.npmjs.org/typescript-eslint/-/typescript-eslint-8.71.1.tgz",
"integrity": "sha512-oiNVPC3/NbV1FpaHU07l8O6ynqG9v4PAiW2WvTrAmZdR0f2HI/gE668ElMLCK7Xbsu5k2fK5C4fXNiq05j7/nQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/eslint-plugin": "8.72.0",
"@typescript-eslint/parser": "8.72.0",
"@typescript-eslint/typescript-estree": "8.72.0",
"@typescript-eslint/utils": "8.72.0"
"@typescript-eslint/eslint-plugin": "8.71.1",
"@typescript-eslint/parser": "8.71.1",
"@typescript-eslint/typescript-estree": "8.71.1",
"@typescript-eslint/utils": "8.71.1"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -10095,12 +10024,12 @@
}
},
"node_modules/undici": {
"version": "7.30.1",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.30.1.tgz",
"integrity": "sha512-03KrMbjIFDWQR5RNPrSw3bd1r12JpyFQl9/yxjUbJtqkjmllqC5lFjZh6ujeuJCKr8EOQvRW872Oy/az+nLYxg==",
"version": "6.28.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-6.28.0.tgz",
"integrity": "sha512-LIY910g9TI13YS95lrMFrs8Rm/u/irgHeTWoKCoteeJ04CUJ92eEfj0rVn+7VKMPBpUPiUoBKfhNyLI23EE/KA==",
"license": "MIT",
"engines": {
"node": ">=20.18.1"
"node": ">=18.17"
}
},
"node_modules/undici-types": {
@@ -10238,9 +10167,9 @@
"license": "MIT"
},
"node_modules/uuid": {
"version": "14.0.3",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-14.0.3.tgz",
"integrity": "sha512-Sc5u1VVln93y1Xnz4R+CunOZc7R9LUcTmhj7Rc7Eg3HVcf7rVn2FOoa812fe0W8p3jD7xeoGSYbfqCGGH2B6xw==",
"version": "14.0.2",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-14.0.2.tgz",
"integrity": "sha512-xZe/16rV4aa+HGSOCiY2YeLT1OybRLrrkL/Rqaq7p7GMVXjFh+6wN4oMYgjFmnSnhY8t6Xpdl2l9qmnHYuMHwQ==",
"funding": [
"https://github.com/sponsors/broofa",
"https://github.com/sponsors/ctavan"
@@ -10380,16 +10309,6 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/word-wrap": {
"version": "1.2.5",
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.5.tgz",
"integrity": "sha512-BN22B5eaMMI9UMtjrGd5g5eCYPpCPDUy0FJXbYsaT5zYxjFOckS53SQDE3pWkVoWpHXVb3BrYcEN4Twa55B5cA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/wrap-ansi": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz",
@@ -10565,11 +10484,9 @@
"dependencies": {
"@actions/core": "^2.0.3",
"@actions/github": "^8.0.1",
"@actions/tool-cache": "^3.0.1",
"@octokit/core": "^7.0.8",
"@octokit/plugin-paginate-rest": ">=16.0.0",
"@octokit/plugin-rest-endpoint-methods": "^18.0.0",
"glob": "^13.0.6",
"lite-matter": "^0.1.2",
"mdast-util-from-markdown": "^2.1.0",
"semver": "^7.8.5",
+8 -8
View File
@@ -1,6 +1,6 @@
{
"name": "codeql",
"version": "4.39.2",
"version": "4.39.1",
"private": true,
"description": "CodeQL action",
"scripts": {
@@ -38,13 +38,13 @@
"follow-redirects": "^1.16.1",
"get-folder-size": "^5.0.0",
"https-proxy-agent": "^7.0.6",
"js-yaml": "^5.5.0",
"js-yaml": "^5.4.3",
"jsonschema": "1.5.0",
"long": "^5.3.2",
"node-forge": "^1.4.0",
"semver": "^7.8.5",
"undici": "^7.30.1",
"uuid": "^14.0.3"
"undici": "^6.28.0",
"uuid": "^14.0.2"
},
"devDependencies": {
"@ava/typescript": "6.0.0",
@@ -58,21 +58,21 @@
"@types/node-forge": "^1.3.14",
"@types/sarif": "^2.1.7",
"@types/semver": "^7.8.0",
"@types/sinon": "^22.0.1",
"@types/sinon": "^22.0.0",
"ava": "^6.4.1",
"esbuild": "^0.28.2",
"eslint": "^9.39.5",
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-github": "^6.1.2",
"eslint-plugin-import-x": "^4.17.1",
"eslint-plugin-jsdoc": "^65.4.0",
"eslint-plugin-jsdoc": "^65.2.0",
"eslint-plugin-no-async-foreach": "^0.1.1",
"glob": "^13.0.6",
"globals": "^17.13.0",
"nock": "^14.0.17",
"sinon": "^22.1.0",
"typescript": "^6.0.3",
"typescript-eslint": "^8.72.0"
"typescript-eslint": "^8.71.1"
},
"overrides": {
"@actions/tool-cache": {
@@ -94,6 +94,6 @@
"semver": ">=6.3.1"
},
"glob": "^13.0.6",
"undici": "^7.30.1"
"undici": "^6.28.0"
}
}
-118
View File
@@ -1,118 +0,0 @@
import fs from "fs";
import path from "path";
import * as core from "@actions/core";
import { globSync } from "glob";
import * as sarif from "../src/sarif";
const DIAGNOSTIC_ID =
process.env["DIAGNOSTIC_ID"] ?? "lang/diagnostics/example";
const DIAGNOSTIC_NAME = process.env["DIAGNOSTIC_NAME"] ?? "Diagnostic name";
const EXPECTED_MESSAGE = process.env["EXPECTED_MESSAGE"] ?? "Plaintext message";
const EXPECTED_LOCATIONS = Number.parseInt(
process.env["EXPECTED_LOCATIONS"] ?? "1",
);
function checkStatusPageNotification(n: sarif.Notification) {
const expectedMessage = EXPECTED_MESSAGE.trim();
if (!n.message.text?.includes(expectedMessage)) {
if (n.message.text !== undefined) {
core.setFailed(
`Expected the status page diagnostic to contain the message '${expectedMessage}', but found '${n.message.text.replaceAll("\n", "\\n")}'.`,
);
} else {
core.setFailed(
`Expected the status page diagnostic to contain the message '${expectedMessage}', but found nothing.`,
);
}
}
const locationCount = n.locations?.length ?? 0;
if (locationCount !== EXPECTED_LOCATIONS) {
core.setFailed(
`Expected the status page diagnostic to have exactly ${EXPECTED_LOCATIONS} location(s), but found ${locationCount}.`,
);
}
}
function checkSarif(sarifFile: string) {
core.info(`Checking SARIF file at '${sarifFile}'`);
const log = JSON.parse(fs.readFileSync(sarifFile, "utf8")) as sarif.Log;
const run = log.runs[0];
if (run.invocations === undefined) {
throw new Error("run.invocations is undefined");
}
const toolExecutionNotifications =
run.invocations[0].toolExecutionNotifications;
if (toolExecutionNotifications === undefined) {
throw new Error("toolExecutionNotifications is undefined");
}
const statusPageNotifications = toolExecutionNotifications.filter(
(n) =>
n.descriptor?.id === DIAGNOSTIC_ID &&
n.properties?.visibility?.statusPage,
);
if (statusPageNotifications.length !== 1) {
core.setFailed(
"Expected exactly one status page reporting descriptor for this diagnostic in the " +
`'runs[].invocations[].toolExecutionNotifications[]' SARIF property, but found ` +
`${statusPageNotifications.length}. All notification reporting descriptors: ` +
`${JSON.stringify(toolExecutionNotifications)}.`,
);
}
checkStatusPageNotification(statusPageNotifications[0]);
const notifications = run.tool.driver.notifications;
if (notifications === undefined) {
throw new Error("notifications is undefined");
}
const diagnosticNotification = notifications.filter(
(n) =>
n.id === DIAGNOSTIC_ID &&
n.name === DIAGNOSTIC_ID &&
n.fullDescription?.text === DIAGNOSTIC_NAME,
);
if (diagnosticNotification.length !== 1) {
core.setFailed(
"Expected exactly one notification for this diagnostic in the " +
`'runs[].tool.driver.notifications[]' SARIF property, but found ` +
`${diagnosticNotification.length}. All notifications: ` +
`${JSON.stringify(notifications)}.`,
);
}
}
const sarifPath = process.env["SARIF_PATH"];
if (sarifPath === undefined) {
throw new Error(
"The path to a SARIF file or directory must be set in 'SARIF_PATH'.",
);
}
if (fs.statSync(sarifPath).isDirectory()) {
const sarifFiles = globSync("**/*.sarif", {
cwd: path.resolve(sarifPath),
absolute: true,
});
if (sarifFiles.length < 1) {
throw new Error(`Unable to find any .sarif files in '${sarifPath}'.`);
}
for (const sarifFile of sarifFiles) {
checkSarif(sarifFile);
}
} else {
checkSarif(sarifPath);
}
core.info("Finished diagnostic export test");
+19 -9
View File
@@ -2,20 +2,30 @@ name: "Bundle: From toolcache"
description: "The CodeQL bundle should be cached within the toolcache"
versions:
- toolcache
installNode: true
steps:
- name: Install @actions/tool-cache
run: npm install @actions/tool-cache@3
- name: Check toolcache contains CodeQL
continue-on-error: true
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts installed
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const toolcache = require('@actions/tool-cache');
const allCodeqlVersions = toolcache.findAllVersions('CodeQL');
if (allCodeqlVersions.length === 0) {
throw new Error(`CodeQL could not be found in the toolcache`);
}
- id: setup-codeql
uses: ./../action/setup-codeql
with:
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Check CodeQL is installed within the toolcache
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts installed
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const toolcache = require('@actions/tool-cache');
const allCodeqlVersions = toolcache.findAllVersions('CodeQL');
console.log(`Found CodeQL versions: ${allCodeqlVersions}`);
if (allCodeqlVersions.length === 0) {
throw new Error('CodeQL not found in toolcache');
}
+30 -14
View File
@@ -6,31 +6,47 @@ operatingSystems:
- ubuntu
- macos
- windows
installNode: true
steps:
- name: Remove CodeQL from toolcache
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts clear
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const fs = require('fs');
const path = require('path');
const codeqlPath = path.join(process.env['RUNNER_TOOL_CACHE'], 'CodeQL');
fs.rmdirSync(codeqlPath, { recursive: true });
- name: Install @actions/tool-cache
run: npm install @actions/tool-cache@3
- name: Check toolcache does not contain CodeQL
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts absent
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const toolcache = require('@actions/tool-cache');
const allCodeqlVersions = toolcache.findAllVersions('CodeQL');
if (allCodeqlVersions.length !== 0) {
throw new Error(`CodeQL should not be found in the toolcache, but found ${allCodeqlVersions}`);
}
console.log('No versions of CodeQL found in the toolcache');
- id: init
uses: ./../action/init
with:
# Request multiple languages so this check uses the combined bundle.
languages: javascript,python
tools: ${{ steps.prepare-test.outputs.tools-url }}
- uses: ./../action/analyze
with:
output: ${{ runner.temp }}/results
upload-database: false
- name: Check CodeQL is installed within the toolcache
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts installed -n 1
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const toolcache = require('@actions/tool-cache');
const allCodeqlVersions = toolcache.findAllVersions('CodeQL');
console.log(`Found CodeQL versions: ${allCodeqlVersions}`);
if (allCodeqlVersions.length === 0) {
throw new Error('CodeQL not found in toolcache');
}
if (allCodeqlVersions.length > 1) {
throw new Error('Multiple CodeQL versions found in toolcache');
}
+47 -5
View File
@@ -5,7 +5,6 @@ versions:
- nightly-latest
env:
CODEQL_ACTION_EXPORT_DIAGNOSTICS: true
installNode: true
steps:
- uses: ./../action/init
id: init
@@ -33,10 +32,53 @@ steps:
name: diagnostics-export-${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: "${{ runner.temp }}/results/javascript.sarif"
retention-days: 7
- name: Check diagnostics appear in SARIF
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
env:
SARIF_PATH: "${{ runner.temp }}/results/javascript.sarif"
run: |
cd ../action
npx tsx ./pr-checks/check-diagnostics.ts
with:
script: |
const fs = require('fs');
function checkStatusPageNotification(n) {
const expectedMessage = 'Plaintext message';
if (n.message.text !== expectedMessage) {
core.setFailed(`Expected the status page diagnostic to have the message '${expectedMessage}', but found '${n.message.text}'.`);
}
if (n.locations.length !== 1) {
core.setFailed(`Expected the status page diagnostic to have exactly 1 location, but found ${n.locations.length}.`);
}
}
const sarif = JSON.parse(fs.readFileSync(process.env['SARIF_PATH'], 'utf8'));
const run = sarif.runs[0];
const toolExecutionNotifications = run.invocations[0].toolExecutionNotifications;
const statusPageNotifications = toolExecutionNotifications.filter(n =>
n.descriptor.id === 'lang/diagnostics/example' && n.properties?.visibility?.statusPage
);
if (statusPageNotifications.length !== 1) {
core.setFailed(
'Expected exactly one status page reporting descriptor for this diagnostic in the ' +
`'runs[].invocations[].toolExecutionNotifications[]' SARIF property, but found ` +
`${statusPageNotifications.length}. All notification reporting descriptors: ` +
`${JSON.stringify(toolExecutionNotifications)}.`
);
}
checkStatusPageNotification(statusPageNotifications[0]);
const notifications = run.tool.driver.notifications;
const diagnosticNotification = notifications.filter(n =>
n.id === 'lang/diagnostics/example' && n.name === 'lang/diagnostics/example' &&
n.fullDescription.text === 'Diagnostic name'
);
if (diagnosticNotification.length !== 1) {
core.setFailed(
'Expected exactly one notification for this diagnostic in the ' +
`'runs[].tool.driver.notifications[]' SARIF property, but found ` +
`${diagnosticNotification.length}. All notifications: ` +
`${JSON.stringify(notifications)}.`
);
}
core.info('Finished diagnostic export test');
@@ -1,72 +0,0 @@
name: "Diagnostic export without database"
description: "Tests that temporary diagnostics are correctly included in the failed SARIF."
versions:
# Overridden by the FF in the `init` step below.
- linked
installNode: true
env:
# Allow the failed SARIF file to be generated.
CODEQL_ACTION_SKIP_GENERATE_FAILED_SARIF: false
# But don't actually upload it.
CODEQL_ACTION_SKIP_SARIF_UPLOAD: true
# Mark telemetry for this workflow so it can be treated separately.
CODEQL_ACTION_TESTING_ENVIRONMENT: codeql-action-pr-checks
steps:
# We deliberately need to make this step fail before the database is initialised, but after
# some diagnostic has been produced. The simplest way to do that right now is to:
#
# - Produce a diagnostic before the db is initialised by forcing the use of a nightly release
# with the `CODEQL_ACTION_FORCE_NIGHTLY` feature flag.
# - Cause `init` to abort because we are attempting to analyse Swift on a non-macOS runner.
#
- name: Initialise failing analysis
id: init
uses: ./../action/init
continue-on-error: true
env:
CODEQL_ACTION_FORCE_NIGHTLY: true
with:
languages: swift
tools: ${{ steps.prepare-test.outputs.tools-url }}
- name: Analyze
uses: ./analyze
# In a real workflow, this step wouldn't run. Since we used `continue-on-error`
# above, we manually disable it with an `if` condition. Having this step is required
# for the init post-action because it checks the workflow definition for e.g. the
# `category` input. Without this step, that part of the init post-action would fail.
if: false
with:
category: "/test-codeql-version:${{ matrix.version }}"
- name: Check diagnostics appear on disk
env:
DIAGNOSTICS_DIR: "${{ runner.temp }}/codeql-action-temp-diagnostics/"
run: |
cd ../action
npx tsx ./pr-checks/diagnostics-without-db.ts
validationJobs:
diagnostics-without-db-validation:
checkout: true
installNode: true
steps:
- name: Download workflow artifact
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
pattern: test-mode-sarif-*
path: sarif
- name: List artifact contents
run: ls -laR ./sarif
- name: Check diagnostics appear in SARIF
env:
SARIF_PATH: "./sarif/codeql-failed-run.sarif"
DIAGNOSTIC_ID: "codeql-action/forced-nightly-cli"
DIAGNOSTIC_NAME: "A nightly release of CodeQL was used"
EXPECTED_LOCATIONS: "0"
EXPECTED_MESSAGE: |
GitHub configured this analysis to use a nightly release of CodeQL to allow you to preview changes from an upcoming release.
run: npx tsx ./pr-checks/check-diagnostics.ts
@@ -11,17 +11,16 @@ installDotNet: true
env:
CODEQL_ACTION_SKIP_FILE_COVERAGE_ON_PRS: false
CODEQL_ACTION_SUBLANGUAGE_FILE_COVERAGE: true
# To balance speed and coverage, we analyze only a single language (JavaScript), but use the
# combined bundle so we can test that baseline information is reported for each language in the
# multi-language source directory.
CODEQL_ACTION_PER_LANGUAGE_BUNDLES: false
steps:
- uses: ./../action/init
id: init
with:
languages: javascript
tools: ${{ steps.prepare-test.outputs.tools-url }}
# To balance speed and coverage, analyze only JavaScript but use the combined bundle so
# baseline information is reported for every language in the multi-language repository. A
# custom configuration prevents per-language bundles from being used.
config: |
paths-ignore: []
- name: Build code
run: ./build.sh
- uses: ./../action/analyze
@@ -14,6 +14,7 @@ operatingSystems:
- stable-v2.21.4
- stable-v2.22.4
env:
CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES: true
CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI: true
installGo: true
installDotNet: true
@@ -1,51 +1,49 @@
name: Per-language bundles
description: Validates extraction and analysis using each per-language CodeQL bundle.
# TODO: Use a released bundle once releases include per-language bundles.
matrix:
include:
- language: actions
os: ubuntu-latest
version: linked
version: nightly-latest
# Actions also needs the JavaScript extractor.
expected-extractors: actions javascript
- language: cpp
os: ubuntu-latest
version: linked
version: nightly-latest
build-mode: manual
build-command: gcc -o main main.c
- language: csharp
os: ubuntu-latest
version: linked
version: nightly-latest
build-mode: none
- language: go
os: ubuntu-latest
version: linked
version: nightly-latest
build-mode: autobuild
- language: java
os: ubuntu-latest
version: linked
version: nightly-latest
build-mode: none
- language: javascript
os: ubuntu-latest
version: linked
version: nightly-latest
- language: python
os: ubuntu-latest
version: linked
version: nightly-latest
- language: ruby
os: ubuntu-latest
version: linked
version: nightly-latest
- language: rust
os: ubuntu-latest
version: linked
version: nightly-latest
- language: swift
os: macos-latest-xlarge
version: linked
version: nightly-latest
build-mode: autobuild
installNode: true
env:
CODEQL_ACTION_PER_LANGUAGE_BUNDLES: true
steps:
- name: Remove CodeQL from toolcache
run: |
cd ./../action
npx tsx ./pr-checks/toolcache.ts clear
- uses: ./../action/init
id: init
with:
@@ -86,8 +84,9 @@ steps:
CODEQL_PATH: ${{ steps.init.outputs.codeql-path }}
run: |
# A bundle that is missing most of its extractors must never be left in the toolcache,
# where a later job analyzing a different language could pick it up. Per-language bundles
# must instead be extracted under the runner's temporary directory.
# where a later job analyzing a different language could pick it up. The runner image
# ships with its own CodeQL in the toolcache, so check where this bundle was extracted to
# rather than whether the toolcache contains CodeQL at all.
echo "CodeQL is at $CODEQL_PATH"
if [[ "$CODEQL_PATH" == "$RUNNER_TOOL_CACHE"/* ]]; then
echo "::error::The per-language bundle was added to the toolcache at $CODEQL_PATH."
@@ -0,0 +1,23 @@
name: "RuboCop multi-language"
description: "Tests using RuboCop to analyze a multi-language repository and then using the CodeQL Action to upload the resulting SARIF"
# This check doesn't use CodeQL, so the `version` matrix variable is unused.
versions:
- default
steps:
- name: Set up Ruby
uses: ruby/setup-ruby@14594264cd68ce8a2345dd349bc3d138a4ef85c8 # v1.327.0
with:
ruby-version: 2.6
- name: Install Code Scanning integration
run: bundle add code-scanning-rubocop --version 0.3.0 --skip-install
- name: Install dependencies
run: bundle install
- name: RuboCop run
run: |
bash -c "
bundle exec rubocop --require code_scanning --format CodeScanning::SarifFormatter -o rubocop.sarif
[[ $? -ne 2 ]]
"
- uses: ./../action/upload-sarif
with:
sarif_file: rubocop.sarif
+1 -3
View File
@@ -35,9 +35,7 @@ steps:
run: exit 1
- uses: ./analyze
# In a real workflow, this step wouldn't run. Since we used `continue-on-error`
# above, we manually disable it with an `if` condition. Having this step is required
# for the init post-action because it checks the workflow definition for e.g. the
# `category` input. Without this step, that part of the init post-action would fail.
# above, we manually disable it with an `if` condition.
if: false
with:
category: "/test-codeql-version:${{ matrix.version }}"
-48
View File
@@ -1,48 +0,0 @@
import * as fs from "fs";
import * as path from "path";
import * as core from "@actions/core";
import type * as diagnostics from "../src/diagnostics";
const DIAGNOSTIC_ID = "codeql-action/forced-nightly-cli";
const diagnosticDirectory = process.env["DIAGNOSTICS_DIR"];
if (diagnosticDirectory === undefined) {
throw new Error("DIAGNOSTICS_DIR is undefined.");
}
const paths = fs
.readdirSync(diagnosticDirectory, {
recursive: true,
encoding: "utf-8",
})
.filter((p) => p.endsWith(".json"));
core.info(`Found ${paths.length} paths in '${diagnosticDirectory}'.`);
core.info(`Searching for a diagnostic with id '${DIAGNOSTIC_ID}'...`);
let found = false;
for (const diagnosticPath of paths) {
if (path.extname(diagnosticPath) !== ".json") {
core.info(`Skipping ${diagnosticPath}.`);
continue;
}
core.info(`Found ${diagnosticPath}.`);
const contents = JSON.parse(
fs.readFileSync(path.resolve(diagnosticDirectory, diagnosticPath), "utf-8"),
) as diagnostics.DiagnosticMessage;
if (contents.source.id === DIAGNOSTIC_ID) {
core.info(
`Found '${DIAGNOSTIC_ID}' in '${diagnosticPath}'. Skipping remaining files.`,
);
found = true;
break;
}
}
if (!found) {
throw new Error(`Didn't find '${DIAGNOSTIC_ID}' diagnostic.`);
}
-4
View File
@@ -20,7 +20,3 @@ is:
- "Release info"
- "Upload results"
- "Update release branch"
# These checks are known not to run on every PR (for example, some release
# PRs), so requiring them would block the PRs they don't run on.
- "github-advanced-security"
- "License compliance check"
-2
View File
@@ -5,11 +5,9 @@
"dependencies": {
"@actions/core": "^2.0.3",
"@actions/github": "^8.0.1",
"@actions/tool-cache": "^3.0.1",
"@octokit/core": "^7.0.8",
"@octokit/plugin-paginate-rest": ">=16.0.0",
"@octokit/plugin-rest-endpoint-methods": "^18.0.0",
"glob": "^13.0.6",
"lite-matter": "^0.1.2",
"mdast-util-from-markdown": "^2.1.0",
"semver": "^7.8.5",
+23 -63
View File
@@ -90,7 +90,7 @@ interface Specification extends JobSpecification {
services?: any;
/** Additional jobs to run after the main PR check job. */
validationJobs?: Record<string, ValidationJobSpecification>;
validationJobs?: Record<string, JobSpecification>;
/** If set, this check is part of a named collection that gets its own caller workflow. */
collection?: string;
@@ -122,10 +122,6 @@ interface JobSpecification {
installYq?: boolean;
}
type ValidationJobSpecification = JobSpecification & {
checkout?: boolean;
};
/** Describes language/framework-specific steps and inputs. */
interface LanguageSetup {
specProperty: keyof JobSpecification;
@@ -137,15 +133,7 @@ interface LanguageSetup {
/** Describes partial mappings from built-in languages to their specific setup information. */
type LanguageSetups = Partial<Record<BuiltInLanguage, LanguageSetup>>;
/** A type representing version identifiers. */
type Version =
| "default"
| "linked"
| "nightly-latest"
// This is an approximation, but sufficient validation for our needs here.
| `stable-v${number}.${number}.${number}`;
/** The default set of CodeQL Bundle versions to use for the PR checks. */
// The default set of CodeQL Bundle versions to use for the PR checks.
const defaultTestVersions = [
// The oldest supported CodeQL version. If bumping, update `CODEQL_MINIMUM_VERSION` in `codeql.ts`
"stable-v2.20.7",
@@ -167,10 +155,7 @@ const defaultTestVersions = [
"linked",
// A nightly build directly from the our private repo, built in the last 24 hours.
"nightly-latest",
] as const satisfies Version[];
/** A type representing the default test versions. */
type DefaultTestVersion = (typeof defaultTestVersions)[number];
];
/** The default versions we use for languages / frameworks, if not specified as a workflow input. */
const defaultLanguageVersions = {
@@ -278,8 +263,8 @@ const languageSetups: LanguageSetups = {
name: "Install Java",
uses: pinnedUses(
"actions/setup-java",
"b843ebf9b913146b38da2eaa7aeddf76074ae1ad",
"v6.1.0",
"de7274f081f381c8f8158605e0321c36c376e2e6",
"v6.0.1",
),
with: {
"java-version": `\${{ inputs.java-version || '${defaultLanguageVersions.java}' }}`,
@@ -324,11 +309,9 @@ const languageSetups: LanguageSetups = {
},
};
/**
* This is essentially an arbitrary version of `yq`, which happened to be the one that
* `choco` fetched when we moved away from using that here.
* See https://github.com/github/codeql-action/pull/3423
*/
// This is essentially an arbitrary version of `yq`, which happened to be the one that
// `choco` fetched when we moved away from using that here.
// See https://github.com/github/codeql-action/pull/3423
const YQ_VERSION = "v4.50.1";
const CHECKS_DIR = path.join(PR_CHECKS_DIR, "checks");
@@ -455,23 +438,10 @@ function generateJobMatrix(
: [operatingSystemConfig["runner-image"]];
for (const runnerImage of runnerImagesForOs) {
// stable-v2.20.7 requires an older version of Java than the default on `ubuntu-latest`.
// We could install that older version on the newer runners, or use the older runner.
// We do the latter here. This can be removed once we stop supporting `v2.20.7`.
// Since this is typed as `DefaultTestVersion`, it will become an error once `stable-v2.20.7`
// is removed from `defaultTestVersions`.
const requiresOlderRunnerImage: DefaultTestVersion = "stable-v2.20.7";
if (
runnerImage === "ubuntu-latest" &&
version === requiresOlderRunnerImage
) {
matrix.push({ os: "ubuntu-24.04", version });
} else {
matrix.push({
os: runnerImage,
version,
});
}
matrix.push({
os: runnerImage,
version,
});
}
}
}
@@ -492,20 +462,6 @@ function generateJobMatrix(
return matrix;
}
/**
* Gets the `actions/checkout` step.
*/
function getCheckoutStep(): Step {
return {
name: "Check out repository",
uses: pinnedUses(
"actions/checkout",
"3d3c42e5aac5ba805825da76410c181273ba90b1",
"v7.0.1",
),
};
}
/**
* Retrieves setup steps and additional input definitions based on specific languages or frameworks
* that are requested by the `checkSpecification`.
@@ -575,7 +531,14 @@ function generateJob(
// Construct the workflow steps needed for this check.
const steps: Step[] = [
getCheckoutStep(),
{
name: "Check out repository",
uses: pinnedUses(
"actions/checkout",
"3d3c42e5aac5ba805825da76410c181273ba90b1",
"v7.0.1",
),
},
...setupInfo.steps,
{
name: "Prepare test",
@@ -646,16 +609,13 @@ function generateJob(
/** Generates a validation job. */
function generateValidationJob(
specDocument: yaml.Document,
jobSpecification: ValidationJobSpecification,
jobSpecification: JobSpecification,
checkName: string,
name: string,
) {
// Determine which languages or frameworks have to be installed.
const { inputs, steps } = getSetupSteps(jobSpecification);
// Construct a checkout step if requested.
const checkoutStep = jobSpecification.checkout ? [getCheckoutStep()] : [];
// Extract the sequence of steps from the YAML document to persist as much formatting as possible.
const specSteps = specDocument.getIn([
"validationJobs",
@@ -664,7 +624,7 @@ function generateValidationJob(
]) as yaml.YAMLSeq;
// Add the generated steps in front of the ones from the specification.
specSteps.items.unshift(...checkoutStep, ...steps);
specSteps.items.unshift(...steps);
const validationJob: Record<string, any> = {
name: jobSpecification.name,
@@ -675,7 +635,7 @@ function generateValidationJob(
"security-events": "read",
},
"timeout-minutes": 5,
"runs-on": "ubuntu-latest",
"runs-on": "ubuntu-slim",
steps: specSteps,
};
-94
View File
@@ -1,94 +0,0 @@
#!/usr/bin/env npx tsx
/**
* Provides utility commands for working with the Actions toolcache in PR checks.
*/
import fs from "fs";
import { parseArgs } from "node:util";
import path from "path";
import * as core from "@actions/core";
import * as toolcache from "@actions/tool-cache";
function removeCodeQLFromToolcache() {
const toolcachePath = process.env["RUNNER_TOOL_CACHE"];
if (toolcachePath === undefined) {
throw new Error("Expected RUNNER_TOOL_CACHE to be defined");
}
const codeqlPath = path.join(toolcachePath, "CodeQL");
fs.rmSync(codeqlPath, { recursive: true });
core.info(`Removed '${codeqlPath}' from the toolcache.`);
}
function findCodeQL() {
return toolcache.findAllVersions("CodeQL");
}
function checkNoCodeQLInToolcache() {
const allCodeqlVersions = findCodeQL();
if (allCodeqlVersions.length !== 0) {
throw new Error(
`CodeQL should not be found in the toolcache, but found ${allCodeqlVersions}`,
);
}
core.info("No versions of CodeQL found in the toolcache");
}
function hasCodeQLInToolcache(expectedNumber?: string) {
const expected =
expectedNumber !== undefined
? Number.parseInt(expectedNumber, 10)
: undefined;
const allCodeqlVersions = findCodeQL();
if (allCodeqlVersions.length === 0) {
throw new Error("CodeQL not found in toolcache");
}
core.info(`Found CodeQL versions: ${allCodeqlVersions.join(", ")}`);
if (expected !== undefined && allCodeqlVersions.length !== expected) {
throw new Error(
`Expected ${expected} versions of CodeQL in the toolcache, but found ${allCodeqlVersions.length}`,
);
}
}
function main() {
const { positionals, values } = parseArgs({
options: {
// The number of expected versions in the toolcache.
"expected-version-count": { type: "string", short: "n" },
},
allowPositionals: true,
});
if (positionals.length < 1) {
throw new Error("Command not specified.");
}
const cmd = positionals[0];
switch (cmd.trim().toLowerCase()) {
case "clear":
removeCodeQLFromToolcache();
return;
case "absent":
checkNoCodeQLInToolcache();
return;
case "installed":
hasCodeQLInToolcache(values["expected-version-count"]);
return;
default:
throw new Error(`Unknown command '${cmd}'.`);
}
}
try {
main();
} catch (err) {
core.setFailed(err instanceof Error ? err.message : String(err));
}
+2 -1
View File
@@ -11,6 +11,7 @@ import {
} from "./actions-util";
import { computeAutomationID } from "./api-client";
import { EnvVar } from "./environment";
import { getRunnerLogger } from "./logging";
import { RecordingLogger, setupTests } from "./testing-utils";
import { initializeEnvironment, withTmpDir } from "./util";
@@ -237,7 +238,7 @@ test.serial("fixCodeQualityCategory", (t) => {
GITHUB_EVENT_NAME: "dynamic",
},
() => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
// Categories that should get adjusted.
t.is(fixCodeQualityCategory(logger, "/language:c#"), "/language:csharp");
+2 -4
View File
@@ -72,7 +72,7 @@ export const getOptionalInput = function (name: string): string | undefined {
* directory that has been set in `CODEQL_ACTION_TEMP` by e.g. a previous step, or the
* value of `RUNNER_TEMP` otherwise.
*/
export function getTemporaryDirectory(env: ReadOnlyEnv = getEnv()): string {
export function getTemporaryDirectory(env: Env = getEnv()): string {
return (
env.getOptional(EnvVar.TEMP) ?? env.getRequired(ActionsEnvVars.RUNNER_TEMP)
);
@@ -361,10 +361,9 @@ const MAX_STDERR_BUFFER_SIZE = 20000;
* Runs a CLI tool.
*
* @returns Standard output produced by the tool.
* @throws A {@link CommandInvocationError} if the tool exits with a non-zero status code.
* @throws A `CommandInvocationError` if the tool exits with a non-zero status code.
*/
export async function runTool(
env: Env,
cmd: string,
args: string[] = [],
opts: { stdin?: string; noStreamStdout?: boolean } = {},
@@ -375,7 +374,6 @@ export async function runTool(
process.stdout.write(`[command]${cmd} ${args.join(" ")}\n`);
}
const exitCode = await new toolrunner.ToolRunner(cmd, args, {
env: env.getDefined(),
ignoreReturnCode: true,
listeners: {
stdout: (data: Buffer) => {
+6 -5
View File
@@ -15,6 +15,7 @@ import {
supportedAnalysisKinds,
} from "./analyses";
import { EnvVar } from "./environment";
import { getRunnerLogger } from "./logging";
import {
createFeatures,
RecordingLogger,
@@ -64,7 +65,7 @@ test.serial(
.withArgs("analysis-kinds")
.returns("code-scanning,code-quality");
const result = await getAnalysisKinds(
new RecordingLogger(),
getRunnerLogger(true),
features,
true,
);
@@ -138,7 +139,7 @@ test.serial(
const optionalInputStub = sinon.stub(actionsUtil, "getOptionalInput");
optionalInputStub.withArgs("quality-queries").returns("code-quality");
const result = await getAnalysisKinds(
new RecordingLogger(),
getRunnerLogger(true),
features,
true,
);
@@ -154,7 +155,7 @@ test.serial(
const requiredInputStub = sinon.stub(actionsUtil, "getRequiredInput");
requiredInputStub.withArgs("analysis-kinds").returns("no-such-thing");
await t.throwsAsync(
getAnalysisKinds(new RecordingLogger(), features, true),
getAnalysisKinds(getRunnerLogger(true), features, true),
);
},
);
@@ -180,7 +181,7 @@ for (let i = 0; i < analysisKinds.length; i++) {
.withArgs("analysis-kinds")
.returns([analysisKind, otherAnalysis].join(","));
const result = await getAnalysisKinds(
new RecordingLogger(),
getRunnerLogger(true),
features,
true,
);
@@ -198,7 +199,7 @@ for (let i = 0; i < analysisKinds.length; i++) {
.withArgs("analysis-kinds")
.returns([analysisKind, otherAnalysis].join(","));
await t.throwsAsync(
getAnalysisKinds(new RecordingLogger(), features, true),
getAnalysisKinds(getRunnerLogger(true), features, true),
{
instanceOf: ConfigurationError,
message: `${analysisKind} and ${otherAnalysis} cannot be enabled at the same time`,
+19 -8
View File
@@ -50,8 +50,9 @@ import {
StatusReportBase,
} from "./status-report";
import {
cleanupTrapCaches,
TrapCacheCleanupStatusReport,
uploadTrapCaches,
warnIfTrapCacheCleanupEnvVarIsSet,
} from "./trap-caching";
import * as uploadLib from "./upload-lib";
import { UploadResult } from "./upload-lib";
@@ -85,6 +86,7 @@ async function sendStatusReport(
trapCacheUploadTime: number | undefined,
dbCreationTimings: DatabaseCreationTimings | undefined,
didUploadTrapCaches: boolean,
trapCacheCleanup: TrapCacheCleanupStatusReport | undefined,
dependencyCacheResults: DependencyCacheUploadStatusReport | undefined,
databaseUploadResults: DatabaseUploadResult[],
logger: Logger,
@@ -105,6 +107,7 @@ async function sendStatusReport(
...statusReportBase,
...(stats || {}),
...(dbCreationTimings || {}),
...(trapCacheCleanup || {}),
dependency_caching_upload_results: dependencyCacheResults,
database_upload_results: databaseUploadResults,
};
@@ -226,6 +229,8 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
| undefined = undefined;
let runStats: QueriesStatusReport | undefined = undefined;
let config: Config | undefined = undefined;
let trapCacheCleanupTelemetry: TrapCacheCleanupStatusReport | undefined =
undefined;
let trapCacheUploadTime: number | undefined = undefined;
let dbCreationTimings: DatabaseCreationTimings | undefined = undefined;
let didUploadTrapCaches = false;
@@ -273,8 +278,6 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
);
}
warnIfTrapCacheCleanupEnvVarIsSet(action);
const apiDetails = getApiDetails();
const outputDir = actionsUtil.getRequiredInput("output");
action.env.export(EnvVar.SARIF_RESULTS_OUTPUT_DIR, outputDir);
@@ -358,7 +361,6 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
uploadResults = await postProcessAndUploadSarif(
logger,
features,
config,
uploadKind,
checkoutPath,
outputDir,
@@ -413,6 +415,13 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
didUploadTrapCaches = await uploadTrapCaches(codeql, config, logger);
trapCacheUploadTime = performance.now() - trapCacheUploadStartTime;
// Clean up TRAP caches
trapCacheCleanupTelemetry = await cleanupTrapCaches(
config,
features,
logger,
);
// Store dependency cache(s) if dependency caching is enabled.
if (shouldStoreCache(config.dependencyCachingEnabled)) {
dependencyCacheResults = await uploadDependencyCaches(
@@ -424,10 +433,8 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
}
// We don't upload results in test mode, so don't wait for processing
if (util.shouldSkipSarifUpload()) {
logger.debug(
"SARIF upload is disabled and nothing was uploaded, so we do not need to wait for processing to finish.",
);
if (util.isInTestMode()) {
logger.debug("In test mode. Waiting for processing is disabled.");
} else if (
uploadResults?.[analyses.AnalysisKind.CodeScanning] !== undefined &&
actionsUtil.getRequiredInput("wait-for-processing") === "true"
@@ -464,6 +471,7 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
trapCacheUploadTime,
dbCreationTimings,
didUploadTrapCaches,
trapCacheCleanupTelemetry,
dependencyCacheResults,
databaseUploadResults,
logger,
@@ -486,6 +494,7 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
trapCacheUploadTime,
dbCreationTimings,
didUploadTrapCaches,
trapCacheCleanupTelemetry,
dependencyCacheResults,
databaseUploadResults,
logger,
@@ -499,6 +508,7 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
trapCacheUploadTime,
dbCreationTimings,
didUploadTrapCaches,
trapCacheCleanupTelemetry,
dependencyCacheResults,
databaseUploadResults,
logger,
@@ -512,6 +522,7 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
trapCacheUploadTime,
dbCreationTimings,
didUploadTrapCaches,
trapCacheCleanupTelemetry,
dependencyCacheResults,
databaseUploadResults,
logger,
+2 -2
View File
@@ -17,12 +17,12 @@ import { defaultSuites } from "./config/db-config";
import { Feature } from "./feature-flags";
import * as gitUtils from "./git-utils";
import { BuiltInLanguage } from "./languages";
import { getRunnerLogger } from "./logging";
import {
setupTests,
setupActionsVars,
createFeatures,
createTestConfig,
RecordingLogger,
callee,
} from "./testing-utils";
import * as uploadLib from "./upload-lib";
@@ -195,7 +195,7 @@ test.serial("status report fields", async (t) => {
undefined,
codeql,
config,
new RecordingLogger(),
getRunnerLogger(true),
createFeatures([Feature.QaTelemetryEnabled]),
);
t.deepEqual(Object.keys(statusReport).sort(), [
+31 -56
View File
@@ -308,70 +308,45 @@ test.serial(
test("getAuthorizationHeaderFor - returns undefined if there's no token", (t) => {
const logger = new RecordingLogger();
t.is(
api.getAuthorizationHeaderFor(
logger,
{ ...SAMPLE_DOTCOM_API_DETAILS, auth: undefined },
`${GITHUB_DOTCOM_URL}/github`,
),
undefined,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
{ ...SAMPLE_DOTCOM_API_DETAILS, auth: "" },
`${GITHUB_DOTCOM_URL}/github`,
),
undefined,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
{ ...SAMPLE_DOTCOM_API_DETAILS, auth: " " },
`${GITHUB_DOTCOM_URL}/github`,
),
undefined,
);
for (const authValue of [undefined, "", " "]) {
const client = new api.InstanceApiClient({
...SAMPLE_DOTCOM_API_DETAILS,
auth: authValue,
});
t.is(
client.getAuthorizationHeaderFor(logger, `${GITHUB_DOTCOM_URL}/github`),
undefined,
);
}
});
test("getAuthorizationHeaderFor - returns header value with token if url is allowed", (t) => {
const logger = new RecordingLogger();
const client = new api.InstanceApiClient({
...SAMPLE_DOTCOM_API_DETAILS,
});
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`${GITHUB_DOTCOM_URL}/github`,
),
`token ${SAMPLE_DOTCOM_API_DETAILS.auth}`,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`${GITHUB_DOTCOM_API_URL}/github`,
),
`token ${SAMPLE_DOTCOM_API_DETAILS.auth}`,
);
for (const url of [
`${GITHUB_DOTCOM_URL}/github`,
`${GITHUB_DOTCOM_API_URL}/github`,
]) {
t.is(
client.getAuthorizationHeaderFor(logger, url),
`token ${SAMPLE_DOTCOM_API_DETAILS.auth}`,
);
}
});
test("getAuthorizationHeaderFor - returns undefined if url is not allowed", (t) => {
const logger = new RecordingLogger();
const client = new api.InstanceApiClient({
...SAMPLE_DOTCOM_API_DETAILS,
});
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`https://localhost/github`,
),
undefined,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`https://api.localhost/github`,
),
undefined,
);
for (const url of [
`https://localhost/github`,
`https://api.localhost/github`,
]) {
t.is(client.getAuthorizationHeaderFor(logger, url), undefined);
}
});
+101 -38
View File
@@ -18,9 +18,11 @@ import {
RegistryProxyVars,
getEnv,
exportEnvVar,
ActionsEnvVars,
} from "./environment";
import {
getCurrentInstance,
getDotcomInstance,
GITHUB_DOTCOM_URL,
parseGitHubUrl,
type GitHubApiCombinedDetails,
@@ -120,6 +122,66 @@ export function makeProxyRequestOptions(
/** The type of GitHub API client we use. */
export type ApiClient = InstanceType<typeof githubUtils.GitHub>;
/**
* Encapsulates a {@link GitHubApiCombinedDetails} along with operations which ensure that any credentials
* are only used with the GitHub instance that is represented by the {@link GitHubApiCombinedDetails} value.
*/
export class InstanceApiClient {
private readonly client: ApiClient;
private readonly apiDetails: GitHubApiCombinedDetails;
private readonly authorizationHeader: string | undefined;
constructor(
apiDetails: GitHubApiCombinedDetails,
clientOptions: CreateApiClientOptions = {},
) {
this.apiDetails = apiDetails;
this.client = createApiClientWithDetails(apiDetails, clientOptions);
// Compute the Authorization header value if we have a token.
const token = this.apiDetails.auth?.trim();
if (token !== undefined && token !== "") {
this.authorizationHeader = `token ${token}`;
}
}
/**
* In most cases, we use an {@link ApiClient} which automatically handles auth for requests
* made with it and this method is not needed and should not be used in that setting.
*
* In some cases, we need to make requests that can't go through an {@link ApiClient}, but still
* require auth. For example, this is the case when using the `toolcache` module to download
* files. For these cases, this method gets a value for the `Authorization` header for a request
* to `url`; or `undefined` if the `Authorization` header should not be set for `url`.
*
* @param logger The logger to use for debugging messages.
* @param url The URL for which we want to add an `Authorization` header.
*/
public getAuthorizationHeaderFor(
logger: Logger,
url: string,
): string | undefined {
// We don't have to check the `url` if we don't have a token to begin with.
if (this.authorizationHeader === undefined) {
return undefined;
}
// We only want to provide an authorization header if we are downloading
// from the same GitHub instance the Action is running on.
// This avoids leaking Enterprise tokens to dotcom.
if (
url.startsWith(`${this.apiDetails.url}/`) ||
(this.apiDetails.apiURL && url.startsWith(`${this.apiDetails.apiURL}/`))
) {
logger.debug(`Providing an authorization token.`);
return this.authorizationHeader;
}
logger.debug(`Not using an authorization token.`);
return undefined;
}
}
/** Options for `createApiClientWithDetails`. */
interface CreateApiClientOptions {
allowExternal?: boolean;
@@ -172,8 +234,8 @@ export function getApiDetails(env: ReadOnlyEnv = getEnv()): GitHubApiDetails {
/**
* Constructs an {@link ApiClient} using {@link apiDetails}.
*
* If {@link apiDetails} is not specified, then the GitHub instance URLs and auth token
* are taken from {@link env}.
* If {@link apiDetails} is not specified, then the GitHub instance URLs are taken from {@link env}
* and the auth token is taken from the `token` input.
*
* If {@link env} is not specified, then the process environment is used.
*
@@ -187,6 +249,28 @@ export function getApiClient(
return createApiClientWithDetails(apiDetails ?? getApiDetails(env));
}
/*
* Gets a `GitHubApiDetails` object describing the GitHub Dotcom instance.
*
* @param env The environment variables to read.
*/
export function getDotcomApiDetails(env: ReadOnlyEnv): GitHubApiDetails {
// If we are currently running on dotcom, then just call `getApiDetails` which will
// construct the appropriate `GitHubApiDetails` from the environment and include the
// value of the `token` input. The `token` input relates to the GitHub instance that
// we are running on only. Therefore, if we are not currently running on dotcom,
// then we construct `GitHubApiDetails` without the `token` value and using the
// hard-coded instance and API URLs.
if (env.getRequired(ActionsEnvVars.GITHUB_SERVER_URL) === GITHUB_DOTCOM_URL) {
return getApiDetails(env);
} else {
return {
auth: undefined,
...getDotcomInstance(),
};
}
}
export function getApiClientWithExternalAuth(
apiDetails: GitHubApiCombinedDetails,
proxy?: ProxyAgent,
@@ -194,42 +278,6 @@ export function getApiClientWithExternalAuth(
return createApiClientWithDetails(apiDetails, { allowExternal: true, proxy });
}
/**
* Gets a value for the `Authorization` header for a request to `url`; or `undefined` if the
* `Authorization` header should not be set for `url`.
*
* @param logger The logger to use for debugging messages.
* @param apiDetails Details of the GitHub API we are using.
* @param url The URL for which we want to add an `Authorization` header.
*
* @returns The value for the `Authorization` header or `undefined` if it shouldn't be populated.
*/
export function getAuthorizationHeaderFor(
logger: Logger,
apiDetails: GitHubApiDetails,
url: string,
): string | undefined {
// We don't have to check the `url` if we don't have a token to begin with.
const token = apiDetails.auth?.trim();
if (token === undefined || token === "") {
return undefined;
}
// We only want to provide an authorization header if we are downloading
// from the same GitHub instance the Action is running on.
// This avoids leaking Enterprise tokens to dotcom.
if (
url.startsWith(`${apiDetails.url}/`) ||
(apiDetails.apiURL && url.startsWith(`${apiDetails.apiURL}/`))
) {
logger.debug(`Providing an authorization token.`);
return `token ${token}`;
}
logger.debug(`Not using an authorization token.`);
return undefined;
}
let cachedGitHubVersion: GitHubVersion | undefined = undefined;
export async function getGitHubVersionFromApi(
@@ -401,6 +449,21 @@ export async function listActionsCaches(
);
}
/**
* Delete an Actions cache item by its ID.
*
* See https://docs.github.com/en/rest/actions/cache#delete-a-github-actions-cache-for-a-repository-using-a-cache-id.
*/
export async function deleteActionsCache(id: number) {
const repositoryNwo = getRepositoryNwo();
await getApiClient().rest.actions.deleteActionsCacheById({
owner: repositoryNwo.owner,
repo: repositoryNwo.repo,
cache_id: id,
});
}
/**
* Retrieve all custom repository properties.
*
+1 -1
View File
@@ -1 +1 @@
{"maximumVersion":"3.23","minimumVersion":"3.19"}
{"maximumVersion":"3.23","minimumVersion":"3.18"}
+2 -2
View File
@@ -10,12 +10,12 @@ import {
scanArtifactsForTokens,
TokenType,
} from "./artifact-scanner";
import { getRunnerLogger } from "./logging";
import {
checkExpectedLogMessages,
getRecordingLogger,
LoggedMessage,
makeTestToken,
RecordingLogger,
} from "./testing-utils";
test("makeTestToken", (t) => {
@@ -121,7 +121,7 @@ for (const { type, value, checkPattern } of testTokens) {
}
test("scanArtifactsForTokens handles files without tokens", async (t) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "scanner-test-"));
try {
+3 -2
View File
@@ -3,14 +3,15 @@ import path from "path";
import test from "ava";
import { RecordingLogger, setupTests } from "../testing-utils";
import { getRunnerLogger } from "../logging";
import { setupTests } from "../testing-utils";
import * as util from "../util";
import { getCachedCodeQlVersion } from "./output-cache";
setupTests(test);
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
test.serial(
"getCachedCodeQlVersion reuses a version persisted by an earlier step",
+2 -2
View File
@@ -2,7 +2,7 @@ import * as fs from "fs";
import path from "path";
import { getTemporaryDirectory } from "../actions-util";
import { ReadOnlyEnv } from "../environment";
import { Env } from "../environment";
import * as json from "../json";
import { Logger } from "../logging";
@@ -51,7 +51,7 @@ export function resetCachedCodeQlVersion(): void {
* Returns the path to the temporary file that backs the
* on-disk cache of CLI responses between workflow steps.
*/
export function getCommandCacheFilePath(env: ReadOnlyEnv): string {
export function getCommandCacheFilePath(env: Env): string {
return path.join(getTemporaryDirectory(env), COMMAND_CACHE_FILENAME);
}
+3
View File
@@ -1,3 +1,4 @@
import type { GitHubApiDetails } from "./github";
import { BuiltInLanguage, parseBuiltInLanguage } from "./languages";
import { BundlePlatform } from "./platform";
import type { CompressionMethod } from "./tar";
@@ -17,6 +18,8 @@ export type CodeQLBundle =
export interface CodeQLDownloadSource {
/** Distinguishes downloads from local archives and cached installations. */
sourceType: "download";
/** Information about the GitHub instance that contains the asset. */
apiDetails: GitHubApiDetails;
/** The bundle to download. */
bundle: CodeQLBundle;
/** The compression format of the bundle archive. */
+19 -29
View File
@@ -22,6 +22,7 @@ import type { Config } from "./config-utils";
import * as defaults from "./defaults.json";
import { DocUrl } from "./doc-url";
import { BuiltInLanguage } from "./languages";
import { getRunnerLogger } from "./logging";
import { ToolsSource } from "./setup-codeql";
import {
setupTests,
@@ -33,8 +34,7 @@ import {
makeVersionInfo,
createTestConfig,
makeMacro,
getCodeQLForTesting,
RecordingLogger,
mockApiDetails,
} from "./testing-utils";
import { ToolsDownloadStatusReport } from "./tools-download";
import * as util from "./util";
@@ -103,7 +103,7 @@ async function installIntoToolcache({
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
new RecordingLogger(),
getRunnerLogger(true),
false,
);
}
@@ -127,21 +127,8 @@ function mockReleaseApi({
});
}
function mockApiDetails(apiDetails: Required<GitHubApiDetails>) {
// This is a workaround to mock `api.getApiDetails()` since it doesn't seem to be possible to
// mock this directly. The difficulty is that `getApiDetails()` is called locally in
// `api-client.ts`, but `sinon.stub(api, "getApiDetails")` only affects calls to
// `getApiDetails()` via an imported `api` module.
sinon
.stub(actionsUtil, "getRequiredInput")
.withArgs("token")
.returns(apiDetails.auth);
process.env["GITHUB_SERVER_URL"] = apiDetails.url;
process.env["GITHUB_API_URL"] = apiDetails.apiURL || "";
}
async function stubCodeql(): Promise<codeql.CodeQL> {
const codeqlObject = await getCodeQLForTesting();
const codeqlObject = await codeql.getCodeQLForTesting();
sinon.stub(codeqlObject, "getVersion").resolves(makeVersionInfo("2.17.6"));
sinon
.stub(codeqlObject, "isTracedLanguage")
@@ -177,7 +164,7 @@ test.serial(
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
features,
new RecordingLogger(),
getRunnerLogger(true),
false,
);
@@ -213,7 +200,7 @@ test.serial(
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
features,
new RecordingLogger(),
getRunnerLogger(true),
false,
);
@@ -253,7 +240,7 @@ test.serial(
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
features,
new RecordingLogger(),
getRunnerLogger(true),
false,
);
t.assert(toolcache.find("CodeQL", "0.0.0-20200610"));
@@ -291,6 +278,7 @@ for (const { cliVersion, tagName, expectedToolcacheVersion } of [
async (t) => {
await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const result = await installIntoToolcache({
cliVersion,
@@ -363,7 +351,7 @@ for (const {
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
features,
new RecordingLogger(),
getRunnerLogger(true),
false,
);
t.assert(toolcache.find("CodeQL", expectedToolcacheVersion));
@@ -406,7 +394,7 @@ for (const toolcacheVersion of [
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
features,
new RecordingLogger(),
getRunnerLogger(true),
false,
);
t.is(
@@ -451,7 +439,7 @@ test.serial(
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
features,
new RecordingLogger(),
getRunnerLogger(true),
false,
);
t.deepEqual(result.toolsVersion, "0.0.0-20200601");
@@ -471,6 +459,7 @@ test.serial(
await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
await installIntoToolcache({
tagName: "codeql-bundle-20200601",
@@ -498,7 +487,7 @@ test.serial(
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
features,
new RecordingLogger(),
getRunnerLogger(true),
false,
);
t.deepEqual(result.toolsVersion, defaults.cliVersion);
@@ -518,6 +507,7 @@ test.serial(
await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
await installIntoToolcache({
tagName: "codeql-bundle-20200601",
@@ -538,7 +528,7 @@ test.serial(
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
features,
new RecordingLogger(),
getRunnerLogger(true),
false,
);
t.deepEqual(result.toolsVersion, defaults.cliVersion);
@@ -580,7 +570,7 @@ test.serial(
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
features,
new RecordingLogger(),
getRunnerLogger(true),
false,
);
@@ -665,7 +655,7 @@ const injectedConfigMacro = makeMacro({
tempDir,
};
thisStubConfig.computedConfig = generateCodeScanningConfig(
new RecordingLogger(),
getRunnerLogger(true),
thisStubConfig.originalUserInput,
augmentationProperties,
);
@@ -1040,7 +1030,7 @@ test.serial("runTool summarizes autobuilder errors", async (t) => {
[2019-09-18 12:00:01] [autobuild] A non-error message
`;
stubToolRunnerConstructor(1, stderr);
const codeqlObject = await getCodeQLForTesting();
const codeqlObject = await codeql.getCodeQLForTesting();
sinon.stub(codeqlObject, "getVersion").resolves(makeVersionInfo("2.17.6"));
sinon.stub(codeqlObject, "resolveExtractor").resolves("/path/to/extractor");
// io throws because of the test CodeQL object.
@@ -1096,7 +1086,7 @@ test.serial("runTool recognizes fatal internal errors", async (t) => {
com.semmle.util.exception.CatastrophicError: An error occurred while evaluating ControlFlowGraph::ControlFlow::Root.isRootOf/1#dispred#f610e6ed/2@86282cc8
Severe disk cache trouble (corruption or out of space) at /home/runner/work/_temp/codeql_databases/go/db-go/default/cache/pages/28/33.pack: Failed to write item to disk`;
stubToolRunnerConstructor(1, stderr);
const codeqlObject = await getCodeQLForTesting();
const codeqlObject = await codeql.getCodeQLForTesting();
sinon.stub(codeqlObject, "getVersion").resolves(makeVersionInfo("2.17.6"));
sinon.stub(codeqlObject, "resolveExtractor").resolves("/path/to/extractor");
// io throws because of the test CodeQL object.
+55 -96
View File
@@ -18,7 +18,7 @@ import { CliError, wrapCliConfigurationError } from "./cli-errors";
import type { ActionConfig } from "./config/action-config";
import { appendExtraQueryExclusions, type Config } from "./config-utils";
import { DocUrl } from "./doc-url";
import { EnvVar, getEnv, Env, ReadOnlyEnv } from "./environment";
import { EnvVar, getEnv, exportEnvVar } from "./environment";
import {
CodeQLDefaultVersionInfo,
Feature,
@@ -26,7 +26,7 @@ import {
} from "./feature-flags";
import { isAnalyzingDefaultBranch } from "./git-utils";
import { Language } from "./languages";
import type { Logger } from "./logging";
import { getRunnerLogger, Logger } from "./logging";
import { writeBaseDatabaseOidsFile, writeOverlayChangesFile } from "./overlay";
import { OverlayDatabaseMode } from "./overlay/overlay-database-mode";
import * as setupCodeql from "./setup-codeql";
@@ -199,7 +199,6 @@ export interface CodeQL {
sarifFile: string,
automationDetailsId: string | undefined,
config: ActionConfig<"tempDir">,
diagnosticDir: string | undefined,
): Promise<void>;
/** Get the location of an extractor for the specified language. */
resolveExtractor(language: Language): Promise<string>;
@@ -260,17 +259,17 @@ const CODEQL_MINIMUM_VERSION = "2.20.7";
/**
* This version will shortly become the oldest version of CodeQL that the Action will run with.
*/
const CODEQL_NEXT_MINIMUM_VERSION = "2.22.4";
const CODEQL_NEXT_MINIMUM_VERSION = "2.21.3";
/**
* This is the version of GHES that was most recently deprecated.
*/
const GHES_VERSION_MOST_RECENTLY_DEPRECATED = "3.18";
const GHES_VERSION_MOST_RECENTLY_DEPRECATED = "3.17";
/**
* This is the deprecation date for the version of GHES that was most recently deprecated.
*/
const GHES_MOST_RECENT_DEPRECATION_DATE = "2026-10-16";
const GHES_MOST_RECENT_DEPRECATION_DATE = "2026-09-24";
/** The CLI verbosity level to use for extraction in debug mode. */
const EXTRACTION_DEBUG_MODE_VERBOSITY = "progress++";
@@ -487,6 +486,18 @@ export function createStubCodeQL(partialCodeql: Partial<CodeQL>): CodeQL {
};
}
/**
* Get a real, newly created CodeQL instance for testing. The instance refers to
* a non-existent placeholder codeql command, so tests that use this function
* should also stub the toolrunner.ToolRunner constructor.
*/
export async function getCodeQLForTesting(
cmd = "codeql-for-testing",
logger: Logger = getRunnerLogger(true),
): Promise<CodeQL> {
return getCodeQLForCmd(logger, cmd, false);
}
/**
* Return a CodeQL object for CodeQL CLI access.
*
@@ -495,18 +506,17 @@ export function createStubCodeQL(partialCodeql: Partial<CodeQL>): CodeQL {
* version requirement. Must be set to true outside tests.
* @returns A new CodeQL object
*/
export async function getCodeQLForCmd(
async function getCodeQLForCmd(
logger: Logger,
cmd: string,
checkVersion: boolean,
env: Env = getEnv(),
): Promise<CodeQL> {
const codeql: CodeQL = {
getPath() {
return cmd;
},
async getVersion() {
const cacheFilePath = outputCache.getCommandCacheFilePath(env);
const cacheFilePath = outputCache.getCommandCacheFilePath(getEnv());
let result = outputCache.getCachedCodeQlVersion(
logger,
cacheFilePath,
@@ -514,7 +524,6 @@ export async function getCodeQLForCmd(
);
if (result === undefined) {
result = await runCliJson<VersionInfo>(
env,
cmd,
["version", "--format=json"],
{
@@ -598,7 +607,6 @@ export async function getCodeQLForCmd(
: ["--no-calculate-baseline"];
await runCli(
env,
cmd,
[
"database",
@@ -634,7 +642,7 @@ export async function getCodeQLForCmd(
}
},
async runAutobuild(config: Config, language: Language) {
applyAutobuildAzurePipelinesTimeoutFix(env);
applyAutobuildAzurePipelinesTimeoutFix();
const autobuildCmd = path.join(
await this.resolveExtractor(language),
@@ -644,11 +652,8 @@ export async function getCodeQLForCmd(
// Bump the verbosity of the autobuild command if we're in debug mode
if (config.debugMode) {
env.set(
EnvVar.CLI_VERBOSITY,
env.getOptional(EnvVar.CLI_VERBOSITY) ??
EXTRACTION_DEBUG_MODE_VERBOSITY,
);
process.env[EnvVar.CLI_VERBOSITY] =
process.env[EnvVar.CLI_VERBOSITY] || EXTRACTION_DEBUG_MODE_VERBOSITY;
}
// On macOS, System Integrity Protection (SIP) typically interferes with
@@ -665,10 +670,10 @@ export async function getCodeQLForCmd(
// When `DYLD_INSERT_LIBRARIES` is set in the environment for a step,
// the Actions runtime introduces its own workaround for SIP
// (https://github.com/actions/runner/pull/416).
await runCli(env, autobuildCmd);
await runCli(autobuildCmd);
},
async extractScannedLanguage(config: Config, language: Language) {
await runCli(env, cmd, [
await runCli(cmd, [
"database",
"trace-command",
"--index-traceless-dbs",
@@ -680,10 +685,10 @@ export async function getCodeQLForCmd(
},
async extractUsingBuildMode(config: Config, language: Language) {
if (config.buildMode === BuildMode.Autobuild) {
applyAutobuildAzurePipelinesTimeoutFix(env);
applyAutobuildAzurePipelinesTimeoutFix();
}
try {
await runCli(env, cmd, [
await runCli(cmd, [
"database",
"trace-command",
"--use-build-mode",
@@ -722,7 +727,7 @@ export async function getCodeQLForCmd(
...getExtraOptionsFromEnv(["database", "finalize"]),
databasePath,
];
await runCli(env, cmd, args);
await runCli(cmd, args);
},
async resolveLanguages(
{
@@ -731,7 +736,7 @@ export async function getCodeQLForCmd(
filterToLanguagesWithQueries: boolean;
} = { filterToLanguagesWithQueries: false },
) {
return runCliJson<ResolveLanguagesOutput>(env, cmd, [
return runCliJson<ResolveLanguagesOutput>(cmd, [
"resolve",
"languages",
"--format=betterjson",
@@ -760,11 +765,7 @@ export async function getCodeQLForCmd(
if (workingDir !== undefined) {
codeqlArgs.push("--working-dir", workingDir);
}
return await runCliJson<ResolveBuildEnvironmentOutput>(
env,
cmd,
codeqlArgs,
);
return await runCliJson<ResolveBuildEnvironmentOutput>(cmd, codeqlArgs);
},
async databaseRunQueries(
databasePath: string,
@@ -783,7 +784,7 @@ export async function getCodeQLForCmd(
ignoringOptions: ["--expect-discarded-cache"],
}),
];
await runCli(env, cmd, codeqlArgs);
await runCli(cmd, codeqlArgs);
},
async databaseInterpretResults(
databasePath: string,
@@ -816,7 +817,7 @@ export async function getCodeQLForCmd(
"--sarif-group-rules-by-pack",
"--sarif-include-query-help=always",
"--sublanguage-file-coverage",
...(await getJobRunUuidSarifOptions(env)),
...(await getJobRunUuidSarifOptions()),
...getExtraOptionsFromEnv(["database", "interpret-results"]),
];
if (sarifRunPropertyFlag !== undefined) {
@@ -836,7 +837,7 @@ export async function getCodeQLForCmd(
}
// Capture the stdout, which contains the analysis summary. Don't stream it to the Actions
// logs to avoid printing it twice.
return await runCli(env, cmd, codeqlArgs, {
return await runCli(cmd, codeqlArgs, {
noStreamStdout: true,
});
},
@@ -853,7 +854,7 @@ export async function getCodeQLForCmd(
`--cache-cleanup=${cleanupLevel}`,
...getExtraOptionsFromEnv(["database", "cleanup"]),
];
await runCli(env, cmd, codeqlArgs);
await runCli(cmd, codeqlArgs);
}
},
async databaseBundle(
@@ -887,9 +888,7 @@ export async function getCodeQLForCmd(
]),
);
}
await new toolrunner.ToolRunner(cmd, args, {
env: env.getDefined(),
}).exec();
await new toolrunner.ToolRunner(cmd, args).exec();
},
async databaseExportDiagnostics(
databasePath: string,
@@ -910,59 +909,27 @@ export async function getCodeQLForCmd(
if (automationDetailsId !== undefined) {
args.push("--sarif-category", automationDetailsId);
}
await new toolrunner.ToolRunner(cmd, args, {
env: env.getDefined(),
}).exec();
await new toolrunner.ToolRunner(cmd, args).exec();
},
async diagnosticsExport(
sarifFile: string,
automationDetailsId: string | undefined,
config: ActionConfig<"tempDir">,
diagnosticDir: string | undefined,
): Promise<void> {
// We may be exporting diagnostics for a failed SARIF, in which case the configuration
// file may not have been written to disk and there may not be a database. The CLI
// will produce a SARIF for diagnostics from `diagnosticDir` without it. Therefore,
// we check whether `configPath` exists and only provide it to the CLI if it does.
// Although not needed, the CLI will produce an error if it
// is provided with an argument to `--sarif-codescanning-config` that does not exist.
let configPath: string | undefined =
getGeneratedCodeScanningConfigPath(config);
if (!fs.existsSync(configPath)) {
// Warn if there is no configuration file, even though `config` claims that `init`
// was successful. This shouldn't be happening.
if (config.successfullyInitialised) {
logger.warning(
[
`Expected a configuration file at '${configPath}', which does not exist.`,
"Continuing with diagnostics export anyway, without a configuration file.",
].join(" "),
);
}
configPath = undefined;
}
const args = [
"diagnostics",
"export",
"--format=sarif-latest",
`--output=${sarifFile}`,
...(configPath !== undefined
? [`--sarif-codescanning-config=${configPath}`]
: []),
...(diagnosticDir !== undefined
? [`--diagnostic-dir=${diagnosticDir}`]
: []),
`--sarif-codescanning-config=${getGeneratedCodeScanningConfigPath(
config,
)}`,
...getExtraOptionsFromEnv(["diagnostics", "export"]),
];
if (automationDetailsId !== undefined) {
args.push("--sarif-category", automationDetailsId);
}
await new toolrunner.ToolRunner(cmd, args, {
env: env.getDefined(),
}).exec();
await new toolrunner.ToolRunner(cmd, args).exec();
},
async resolveExtractor(language: Language): Promise<string> {
// Request it using `format=json` so we don't need to strip the trailing new line generated by
@@ -980,7 +947,6 @@ export async function getCodeQLForCmd(
...getExtraOptionsFromEnv(["resolve", "extractor"]),
],
{
env: env.getDefined(),
silent: true,
listeners: {
stdout: (data) => {
@@ -1008,7 +974,7 @@ export async function getCodeQLForCmd(
...getExtraOptionsFromEnv(["resolve", "queries"]),
...queries,
];
return await runCliJson<string[]>(env, cmd, codeqlArgs, {
return await runCliJson<string[]>(cmd, codeqlArgs, {
noStreamStdout: true,
});
},
@@ -1022,7 +988,7 @@ export async function getCodeQLForCmd(
"--format=json",
...getExtraOptionsFromEnv(["resolve", "database"]),
];
return await runCliJson<ResolveDatabaseOutput>(env, cmd, codeqlArgs, {
return await runCliJson<ResolveDatabaseOutput>(cmd, codeqlArgs, {
noStreamStdout: true,
});
},
@@ -1049,7 +1015,7 @@ export async function getCodeQLForCmd(
args.push("--sarif-merge-runs-from-equal-category");
}
await runCli(env, cmd, args);
await runCli(cmd, args);
},
};
// To ensure that status reports include the CodeQL CLI version wherever
@@ -1071,7 +1037,7 @@ export async function getCodeQLForCmd(
);
} else if (
checkVersion &&
env.getOptional(EnvVar.SUPPRESS_DEPRECATED_SOON_WARNING) !== "true" &&
process.env[EnvVar.SUPPRESS_DEPRECATED_SOON_WARNING] !== "true" &&
!(await util.codeQlVersionAtLeast(codeql, CODEQL_NEXT_MINIMUM_VERSION))
) {
const result = await codeql.getVersion();
@@ -1089,7 +1055,7 @@ export async function getCodeQLForCmd(
}' by 'github/codeql-action/*@v${getActionVersion()}' in your code scanning workflow to ` +
"continue using this version of the CodeQL Action.",
);
env.export(EnvVar.SUPPRESS_DEPRECATED_SOON_WARNING, "true");
exportEnvVar(EnvVar.SUPPRESS_DEPRECATED_SOON_WARNING, "true");
}
return codeql;
}
@@ -1166,13 +1132,12 @@ export function getExtraOptions(
}
async function runCli(
env: Env,
cmd: string,
args: string[] = [],
opts: { stdin?: string; noStreamStdout?: boolean } = {},
): Promise<string> {
try {
return await runTool(env, cmd, args, opts);
return await runTool(cmd, args, opts);
} catch (e) {
if (e instanceof CommandInvocationError) {
throw wrapCliConfigurationError(new CliError(e));
@@ -1183,8 +1148,6 @@ async function runCli(
/**
* Wraps the command executor {@link runCli} and tries to parse the output as JSON.
*
* @param env The environment variables.
* @param cmd The command to run.
* @param args The arguments to pass to the command.
* @param opts The options for running the command.
@@ -1193,12 +1156,11 @@ async function runCli(
* @returns The parsed JSON output from the command.
*/
async function runCliJson<T>(
env: Env,
cmd: string,
args: string[] = [],
opts: { stdin?: string; noStreamStdout?: boolean } = {},
): Promise<T> {
const output = await runCli(env, cmd, args, opts);
const output = await runCli(cmd, args, opts);
try {
return JSON.parse(output) as T;
} catch (e) {
@@ -1297,20 +1259,17 @@ function getExtractionVerbosityArguments(
* Without the fix, long build processes will timeout when pulling down Java packages
* https://developercommunity.visualstudio.com/content/problem/292284/maven-hosted-agent-connection-timeout.html
*/
function applyAutobuildAzurePipelinesTimeoutFix(env: Env) {
const javaToolOptions = env.getOptional("JAVA_TOOL_OPTIONS") ?? "";
env.set(
"JAVA_TOOL_OPTIONS",
[
...javaToolOptions.split(/\s+/),
"-Dhttp.keepAlive=false",
"-Dmaven.wagon.http.pool=false",
].join(" "),
);
function applyAutobuildAzurePipelinesTimeoutFix() {
const javaToolOptions = process.env["JAVA_TOOL_OPTIONS"] || "";
process.env["JAVA_TOOL_OPTIONS"] = [
...javaToolOptions.split(/\s+/),
"-Dhttp.keepAlive=false",
"-Dmaven.wagon.http.pool=false",
].join(" ");
}
async function getJobRunUuidSarifOptions(env: ReadOnlyEnv) {
const jobRunUuid = env.getOptional(EnvVar.JOB_RUN_UUID);
async function getJobRunUuidSarifOptions() {
const jobRunUuid = process.env[EnvVar.JOB_RUN_UUID];
return jobRunUuid ? [`--sarif-run-property=jobRunUuid=${jobRunUuid}`] : [];
}
+83 -61
View File
@@ -21,6 +21,7 @@ import { RepositoryProperties } from "./feature-flags/properties";
import * as gitUtils from "./git-utils";
import { GitVersionInfo } from "./git-utils";
import { BuiltInLanguage, Language } from "./languages";
import { getRunnerLogger } from "./logging";
import { CODEQL_OVERLAY_MINIMUM_VERSION } from "./overlay";
import * as overlayDiagnostics from "./overlay/diagnostics";
import { OverlayDisabledReason } from "./overlay/diagnostics";
@@ -41,7 +42,6 @@ import {
callee,
SAMPLE_DOTCOM_API_DETAILS,
AssertableTarget,
RecordingLogger,
} from "./testing-utils";
import {
GitHubVariant,
@@ -102,7 +102,7 @@ function createTestInitConfigInputs(
features: createFeatures([]),
repositoryProperties: {},
enableFileCoverageInformation: true,
logger: new RecordingLogger(),
logger: getRunnerLogger(true),
} satisfies configUtils.InitConfigInputs,
overrides,
);
@@ -150,7 +150,7 @@ function mockListLanguages(languages: string[]) {
test.serial("load empty config", async (t) => {
return await withTmpDir(async (tempDir) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const languages = "javascript,python";
setupActionsVars(tempDir, tempDir);
@@ -196,7 +196,7 @@ test.serial("load empty config", async (t) => {
test.serial("load code quality config", async (t) => {
return await withTmpDir(async (tempDir) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const languages = "actions";
setupActionsVars(tempDir, tempDir);
@@ -252,7 +252,7 @@ test.serial(
"initActionState doesn't throw if there are queries configured in the repository properties",
async (t) => {
return await withTmpDir(async (tempDir) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const languages = "javascript";
setupActionsVars(tempDir, tempDir);
@@ -315,7 +315,7 @@ test.serial(
test.serial("loading a saved config produces the same config", async (t) => {
return await withTmpDir(async (tempDir) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const codeql = createStubCodeQL({
async resolveLanguages() {
@@ -363,7 +363,7 @@ test.serial("loading a saved config produces the same config", async (t) => {
test.serial("loading config with version mismatch throws", async (t) => {
return await withTmpDir(async (tempDir) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const codeql = createStubCodeQL({
async resolveLanguages() {
@@ -788,12 +788,12 @@ test.serial("Unknown languages", async (t) => {
});
});
const mockLogger = new RecordingLogger();
const mockLogger = getRunnerLogger(true);
test.serial("no generateRegistries when registries is undefined", async (t) => {
return await withTmpDir(async (tmpDir) => {
const registriesInput = undefined;
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const { registriesAuthTokens, qlconfigFile } =
await configUtils.generateRegistries(registriesInput, tmpDir, logger);
@@ -814,7 +814,7 @@ test.serial(
token: "not-a-token",
},
]);
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const { registriesAuthTokens, qlconfigFile } =
await configUtils.generateRegistries(registriesInput, tmpDir, logger);
@@ -2126,7 +2126,7 @@ test.serial("getPrimaryAnalysisConfig - Code Scanning + Code Quality", (t) => {
});
test.serial(
"isTrapCachingEnabled: explicit true input overrides self-hosted default",
"isTrapCachingEnabled: explicit input true is respected",
async (t) => {
return await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -2134,40 +2134,12 @@ test.serial(
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns("true");
t.true(configUtils.isTrapCachingEnabled(OverlayDatabaseMode.None));
});
},
);
test.serial(
"isTrapCachingEnabled: explicit true input overrides overlay defaults",
async (t) => {
return await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, path.join(tmpDir, "hostedtoolcache"));
sinon
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns("true");
for (const overlayDatabaseMode of [
OverlayDatabaseMode.Overlay,
OverlayDatabaseMode.OverlayBase,
]) {
t.true(configUtils.isTrapCachingEnabled(overlayDatabaseMode));
}
});
},
);
test.serial(
"isTrapCachingEnabled: explicit false input overrides hosted default",
async (t) => {
return await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, path.join(tmpDir, "hostedtoolcache"));
sinon
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns("false");
t.false(configUtils.isTrapCachingEnabled(OverlayDatabaseMode.None));
t.true(
await configUtils.isTrapCachingEnabled(
createFeatures([]),
OverlayDatabaseMode.None,
),
);
});
},
);
@@ -2181,13 +2153,18 @@ test.serial(
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns(undefined);
t.false(configUtils.isTrapCachingEnabled(OverlayDatabaseMode.None));
t.false(
await configUtils.isTrapCachingEnabled(
createFeatures([]),
OverlayDatabaseMode.None,
),
);
});
},
);
test.serial(
"isTrapCachingEnabled: enabled on hosted runner without overlay analysis",
"isTrapCachingEnabled: enabled on hosted runner by default",
async (t) => {
return await withTmpDir(async (tmpDir) => {
const hostedToolCache = path.join(tmpDir, "hostedtoolcache");
@@ -2196,13 +2173,18 @@ test.serial(
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns(undefined);
t.true(configUtils.isTrapCachingEnabled(OverlayDatabaseMode.None));
t.true(
await configUtils.isTrapCachingEnabled(
createFeatures([]),
OverlayDatabaseMode.None,
),
);
});
},
);
test.serial(
"isTrapCachingEnabled: disabled on hosted runner with overlay analysis",
"isTrapCachingEnabled: enabled on hosted runner when overlay enabled but feature flag off",
async (t) => {
return await withTmpDir(async (tmpDir) => {
const hostedToolCache = path.join(tmpDir, "hostedtoolcache");
@@ -2211,12 +2193,52 @@ test.serial(
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns(undefined);
for (const overlayDatabaseMode of [
OverlayDatabaseMode.Overlay,
OverlayDatabaseMode.OverlayBase,
]) {
t.false(configUtils.isTrapCachingEnabled(overlayDatabaseMode));
}
t.true(
await configUtils.isTrapCachingEnabled(
createFeatures([]),
OverlayDatabaseMode.Overlay,
),
);
});
},
);
test.serial(
"isTrapCachingEnabled: disabled on hosted runner when overlay enabled and feature flag on",
async (t) => {
return await withTmpDir(async (tmpDir) => {
const hostedToolCache = path.join(tmpDir, "hostedtoolcache");
setupActionsVars(tmpDir, hostedToolCache);
sinon
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns(undefined);
t.false(
await configUtils.isTrapCachingEnabled(
createFeatures([Feature.OverlayAnalysisDisableTrapCaching]),
OverlayDatabaseMode.Overlay,
),
);
});
},
);
test.serial(
"isTrapCachingEnabled: enabled on hosted runner when overlay is None even with feature flag on",
async (t) => {
return await withTmpDir(async (tmpDir) => {
const hostedToolCache = path.join(tmpDir, "hostedtoolcache");
setupActionsVars(tmpDir, hostedToolCache);
sinon
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns(undefined);
t.true(
await configUtils.isTrapCachingEnabled(
createFeatures([Feature.OverlayAnalysisDisableTrapCaching]),
OverlayDatabaseMode.None,
),
);
});
},
);
@@ -2225,7 +2247,7 @@ test("applyIncrementalAnalysisSettings: no-op when mode is not Overlay and diff
const config = createTestConfig({});
config.overlayDatabaseMode = OverlayDatabaseMode.None;
const codeql = createStubCodeQL({});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
await configUtils.applyIncrementalAnalysisSettings(
config,
@@ -2243,7 +2265,7 @@ test("applyIncrementalAnalysisSettings: keeps overlay mode and adds exclusions w
overlayDatabaseMode: OverlayDatabaseMode.Overlay,
});
const codeql = createStubCodeQL({});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
await configUtils.applyIncrementalAnalysisSettings(
config,
@@ -2266,7 +2288,7 @@ test.serial(
});
config.useOverlayDatabaseCaching = true;
const codeql = createStubCodeQL({});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const addDiagnosticsStub = sinon
.stub(overlayDiagnostics, "addOverlayDisablementDiagnostics")
.resolves();
@@ -2298,7 +2320,7 @@ test.serial(
config.useOverlayDatabaseCaching = false;
config.overlayModeSetExplicitly = true;
const codeql = createStubCodeQL({});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const addDiagnosticsStub = sinon
.stub(overlayDiagnostics, "addOverlayDisablementDiagnostics")
.resolves();
@@ -2321,7 +2343,7 @@ test("applyIncrementalAnalysisSettings: adds exclusions for diff-informed-only r
const config = createTestConfig({});
config.overlayDatabaseMode = OverlayDatabaseMode.None;
const codeql = createStubCodeQL({});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
await configUtils.applyIncrementalAnalysisSettings(
config,
@@ -2535,7 +2557,7 @@ test("determineUserConfig - merges configs if FF is enabled in Default Setup", a
// also check whether loading the configuration from disk that was written
// by `determineUserConfig` matches our expectations.
const loadedFromDisk = configUtils.getLocalConfig(
new RecordingLogger(),
getRunnerLogger(true),
expectedConfigPath,
false,
);
+10 -12
View File
@@ -465,8 +465,6 @@ export async function initActionState(
overlayModeSetExplicitly: false,
repositoryProperties,
enableFileCoverageInformation,
// We don't set this to `true` until `init` is successful.
successfullyInitialised: false,
};
}
@@ -946,14 +944,10 @@ async function validateOverlayDatabaseMode(
});
}
/**
* Determines whether TRAP caching is enabled for the current run.
*
* @returns Whether TRAP caches should be downloaded and uploaded.
*/
export function isTrapCachingEnabled(
export async function isTrapCachingEnabled(
features: FeatureEnablement,
overlayDatabaseMode: OverlayDatabaseMode,
): boolean {
): Promise<boolean> {
// If the workflow specified something, always respect that.
const trapCaching = getOptionalInput("trap-caching");
if (trapCaching !== undefined) return trapCaching === "true";
@@ -961,8 +955,12 @@ export function isTrapCachingEnabled(
// On self-hosted runners which may have slow network access, disable TRAP caching by default.
if (!looksLikeHostedRunner()) return false;
// Overlay analysis supersedes TRAP caching, so disable TRAP caching by default when it is active.
if (overlayDatabaseMode !== OverlayDatabaseMode.None) {
// If overlay analysis is enabled, then disable TRAP caching since overlay analysis supersedes it.
// This change is gated behind a feature flag.
if (
overlayDatabaseMode !== OverlayDatabaseMode.None &&
(await features.getValue(Feature.OverlayAnalysisDisableTrapCaching))
) {
return false;
}
@@ -1304,7 +1302,7 @@ export async function initConfig(
logger,
);
if (isTrapCachingEnabled(config.overlayDatabaseMode)) {
if (await isTrapCachingEnabled(features, config.overlayDatabaseMode)) {
const { trapCaches, trapCacheDownloadTime } = await downloadCacheWithTime(
inputs.codeql,
config.languages,
-3
View File
@@ -130,9 +130,6 @@ export interface Config {
* Whether to enable file coverage information.
*/
enableFileCoverageInformation: boolean;
/** Whether the analysis was successfully initialised using the configuration. */
successfullyInitialised: boolean;
}
/** Properties available in `Config`. */
+59 -74
View File
@@ -2,15 +2,15 @@ import test, { ExecutionContext } from "ava";
import { RepositoryProperties } from "../feature-flags/properties";
import { BuiltInLanguage, Language } from "../languages";
import { getRunnerLogger } from "../logging";
import {
checkExpectedLogMessages,
getRecordingLogger,
LoggedMessage,
makeMacro,
RecordingLogger,
setupActionsVars,
} from "../testing-utils";
import { ConfigurationError, prettyPrintPack, withTmpDir } from "../util";
import { ConfigurationError, prettyPrintPack } from "../util";
import * as dbConfig from "./db-config";
@@ -384,7 +384,7 @@ calculateAugmentationErrorMacro(
test("parseUserConfig - successfully parses valid YAML", (t) => {
const result = dbConfig.parseUserConfig(
new RecordingLogger(),
getRunnerLogger(true),
"test",
`
paths-ignore:
@@ -410,7 +410,7 @@ test("parseUserConfig - throws a ConfigurationError if the file is not valid YAM
t.throws(
() =>
dbConfig.parseUserConfig(
new RecordingLogger(),
getRunnerLogger(true),
"test",
`
paths-ignore:
@@ -562,89 +562,74 @@ test("mergeDefaultSetupAndUserConfigs - keeps other properties from user-supplie
t.deepEqual(result, configFile);
});
test.serial(
"mergeDefaultSetupAndUserConfigs - ignores, but warns about, unknown keys from Default Setup",
async (t) => {
const logger = new RecordingLogger();
const configFile: dbConfig.UserConfig = {
"query-filters": [{ exclude: { a: "b" } }],
"paths-ignore": ["path"],
};
test("mergeDefaultSetupAndUserConfigs - ignores, but warns about, unknown keys from Default Setup", async (t) => {
const logger = new RecordingLogger();
const configFile: dbConfig.UserConfig = {
"query-filters": [{ exclude: { a: "b" } }],
"paths-ignore": ["path"],
};
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const result = dbConfig.mergeDefaultSetupAndUserConfigs(
logger,
{
"default-setup": {
borg: [],
org: {
unknown: "foo",
"model-packs": [],
},
} as unknown as dbConfig.DefaultSetupConfig,
"paths-ignore": ["other-path"],
const result = dbConfig.mergeDefaultSetupAndUserConfigs(
logger,
{
"default-setup": {
borg: [],
org: {
unknown: "foo",
"model-packs": [],
},
configFile,
);
} as unknown as dbConfig.DefaultSetupConfig,
"paths-ignore": ["other-path"],
},
configFile,
);
t.deepEqual(result, {
...configFile,
"default-setup": { org: { "model-packs": [] } },
});
t.deepEqual(result, {
...configFile,
"default-setup": { org: { "model-packs": [] } },
});
const expectedUnrecognisedKeys = [
".default-setup.org.unknown",
".default-setup.borg",
".paths-ignore",
].join(", ");
checkExpectedLogMessages(t, logger.messages, [
`Unrecognised keys in Default Setup configuration: ${expectedUnrecognisedKeys}`,
]);
});
},
);
const expectedUnrecognisedKeys = [
".default-setup.org.unknown",
".default-setup.borg",
".paths-ignore",
].join(", ");
checkExpectedLogMessages(t, logger.messages, [
`Unrecognised keys in Default Setup configuration: ${expectedUnrecognisedKeys}`,
]);
});
test.serial(
"mergeDefaultSetupAndUserConfigs - warns about invalid keys from Default Setup",
async (t) => {
const logger = new RecordingLogger();
const configFile: dbConfig.UserConfig = {};
test("mergeDefaultSetupAndUserConfigs - warns about invalid keys from Default Setup", async (t) => {
const logger = new RecordingLogger();
const configFile: dbConfig.UserConfig = {};
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const result = dbConfig.mergeDefaultSetupAndUserConfigs(
logger,
{
"default-setup": {
org: {
"model-packs": [123],
},
} as unknown as dbConfig.DefaultSetupConfig,
const result = dbConfig.mergeDefaultSetupAndUserConfigs(
logger,
{
"default-setup": {
org: {
"model-packs": [123],
},
configFile,
);
} as unknown as dbConfig.DefaultSetupConfig,
},
configFile,
);
t.deepEqual(result, {
...configFile,
"default-setup": { org: { "model-packs": [123] } },
});
t.deepEqual(result, {
...configFile,
"default-setup": { org: { "model-packs": [123] } },
});
const expectedInvalidKeys = [".default-setup.org.model-packs[0]"].join(
", ",
);
checkExpectedLogMessages(t, logger.messages, [
`Invalid keys in Default Setup configuration: ${expectedInvalidKeys}`,
]);
});
},
);
const expectedInvalidKeys = [".default-setup.org.model-packs[0]"].join(", ");
checkExpectedLogMessages(t, logger.messages, [
`Invalid keys in Default Setup configuration: ${expectedInvalidKeys}`,
]);
});
/** Parses `contents` as a configuration without validating it. */
function parseUnvalidatedConfig(contents: string): dbConfig.UserConfig {
return dbConfig.parseUserConfig(
new RecordingLogger(),
getRunnerLogger(true),
"test",
contents,
false,
+4
View File
@@ -599,6 +599,7 @@ test.serial(
const nwo: RepositoryNwo = { owner: "test", repo: "foo" };
const language: BuiltInLanguage = BuiltInLanguage.java;
const env = getTestEnv();
await t.notThrowsAsync(async () => {
await uploadBundledDatabase(
nwo,
@@ -607,11 +608,14 @@ test.serial(
dbPath,
content.length,
SAMPLE_DOTCOM_API_DETAILS,
env,
);
});
// Check that our stubs were called and that they received the expected arguments.
t.is(getApiClient.callCount, 1);
t.deepEqual(getApiClient.args[0], [env, SAMPLE_DOTCOM_API_DETAILS]);
t.is(fetchStub.callCount, 1);
const args = fetchStub.args[0];
+6 -4
View File
@@ -10,6 +10,7 @@ import {
} from "./api-client";
import { type CodeQL } from "./codeql";
import { Config } from "./config-utils";
import type { ReadOnlyEnv } from "./environment";
import { Feature } from "./feature-flags";
import * as gitUtils from "./git-utils";
import { parseGitHubUrl } from "./github";
@@ -266,11 +267,12 @@ export async function uploadBundledDatabase(
bundledDb: string,
bundledDbSize: number,
apiDetails: GitHubApiDetails,
env: ReadOnlyEnv = util.getEnv(),
): Promise<number> {
// Ensure that the `apiDetails` that `getApiClient` uses match the ones we were given here,
// because we use `apiDetails.auth` below which must be used with the same instance that
// `apiDetails` refers to.
const client = getApiClient(util.getEnv(), apiDetails);
// It is important that the `apiDetails` that `getApiClient` uses match the ones we were given,
// because we use `apiDetails.url` below to construct the URL to upload the database to. That
// URL will be specific to the GitHub instance given by `apiDetails` and so the client must match it.
const client = getApiClient(env, apiDetails);
const uploadsUrl = new URL(parseGitHubUrl(apiDetails.url));
uploadsUrl.hostname = `uploads.${uploadsUrl.hostname}`;
+4 -4
View File
@@ -1,6 +1,6 @@
{
"bundleVersion": "codeql-bundle-v2.27.3",
"cliVersion": "2.27.3",
"priorBundleVersion": "codeql-bundle-v2.27.2",
"priorCliVersion": "2.27.2"
"bundleVersion": "codeql-bundle-v2.27.2",
"cliVersion": "2.27.2",
"priorBundleVersion": "codeql-bundle-v2.27.1",
"priorCliVersion": "2.27.1"
}
-165
View File
@@ -1,165 +0,0 @@
import * as fs from "fs";
import path from "path";
import test, { type ExecutionContext } from "ava";
import * as diagnostics from "./diagnostics";
import { BuiltInLanguage } from "./languages";
import {
checkExpectedLogMessages,
createTestConfig,
initAllState,
RecordingLogger,
setupTests,
} from "./testing-utils";
import { withTmpDir } from "./util";
setupTests(test);
const id = "codeql-action/test-diagnostic";
const name = "Test title";
test("makeDiagnostic - adds expected properties", (t) => {
const diagnostic = diagnostics.makeDiagnostic(id, name, {});
t.truthy(diagnostic.timestamp);
t.is(diagnostic.source.id, id);
t.is(diagnostic.source.name, name);
});
test("getLanguageDiagnosticsPath gets language-specific path", async (t) =>
withTmpDir(async (tmpDir) => {
for (const language of Object.values(BuiltInLanguage)) {
t.is(
diagnostics.getLanguageDiagnosticsPath(tmpDir, language),
path.resolve(
tmpDir,
language,
...diagnostics.CODEQL_ACTION_DIAGNOSTIC_SUBDIR_PARTS,
),
);
}
}));
test("getTempLanguageDiagnosticsPath returns path without language", async (t) =>
withTmpDir(async (tmpDir) => {
t.is(
diagnostics.getTempLanguageDiagnosticsPath(tmpDir),
path.resolve(
tmpDir,
diagnostics.TEMP_DIAGNOSTIC_DIR_NAME,
...diagnostics.CODEQL_ACTION_DIAGNOSTIC_SUBDIR_PARTS,
),
);
}));
test("getTempLanguageDiagnosticsPath returns language-specific path if a language is provided", async (t) =>
withTmpDir(async (tmpDir) => {
t.is(
diagnostics.getTempLanguageDiagnosticsPath(tmpDir, BuiltInLanguage.swift),
path.resolve(
tmpDir,
diagnostics.TEMP_DIAGNOSTIC_DIR_NAME,
BuiltInLanguage.swift,
...diagnostics.CODEQL_ACTION_DIAGNOSTIC_SUBDIR_PARTS,
),
);
}));
test("getDatabaseDiagnosticsPath returns language-specific path", async (t) =>
withTmpDir(async (tmpDir) => {
t.is(
diagnostics.getDatabaseDiagnosticsPath(
createTestConfig({ dbLocation: tmpDir }),
BuiltInLanguage.swift,
),
path.resolve(
tmpDir,
BuiltInLanguage.swift,
...diagnostics.CODEQL_ACTION_DIAGNOSTIC_SUBDIR_PARTS,
),
);
}));
/**
* Diagnostic files that are written to disk don't have deterministic names.
* This function recursively finds all `.json` files at {@link rootPath},
* asserts that there is exactly one, and that it matches {@link expected}.
*
* @param rootPath The path to start searching from.
*/
function assertSingleDiagnosticIn(
t: ExecutionContext<unknown>,
rootPath: string,
expected: diagnostics.DiagnosticMessage,
) {
// Find all `.json` files in `rootPath` and check that there is exactly one.
const filePaths = fs
.readdirSync(rootPath, {
recursive: true,
encoding: "utf-8",
})
.filter((p) => p.endsWith(".json"));
t.is(filePaths.length, 1);
// Check that the written diagnostic matches the one we added.
const contents = fs.readFileSync(
path.resolve(rootPath, filePaths[0]),
"utf-8",
);
const readDiagnostic = JSON.parse(contents) as diagnostics.DiagnosticMessage;
t.deepEqual(readDiagnostic, expected);
}
test("addDiagnostic writes temporary diagnostics and flushDiagnostics moves them to the database", async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const tempDiagDir = path.resolve(tmpDir, "temp");
const testDbDir = path.resolve(tmpDir, "test", "database");
const diagnostic = diagnostics.makeDiagnostic(id, name, {});
// Write the diagnostic. The path provided for `dbLocation` doesn't exist,
// so the diagnostic will be written to a temporary location.
t.notThrows(() => {
diagnostics.addDiagnostic(
createTestConfig({ tempDir: tempDiagDir, dbLocation: testDbDir }),
BuiltInLanguage.actions,
diagnostic,
logger,
);
});
// Check that `tempDiagDir` was created.
t.true(fs.existsSync(tempDiagDir));
// Check that the diagnostic exists under `tempDiagDir`.
assertSingleDiagnosticIn(t, tempDiagDir, diagnostic);
// Flush the diagnostics. This will cause the diagnostic we wrote previously to
// be moved to a location under `dbLocation`.
t.notThrows(() => {
diagnostics.flushDiagnostics(
initAllState({ logger }),
createTestConfig({
tempDir: tempDiagDir,
dbLocation: testDbDir,
}),
);
});
// Check that `testDbDir` was created.
t.true(fs.existsSync(testDbDir));
// Check that the diagnostic exists under `testDbDir`.
assertSingleDiagnosticIn(t, testDbDir, diagnostic);
// Check that we logged that the database directory doesn't exist yet
// and that we moved one diagnostic.
checkExpectedLogMessages(t, logger.messages, [
`but the database at ${path.join(testDbDir, "actions")} does not exist yet`,
"Moving 1 diagnostic(s) to their respective databases.",
]);
}));
+87 -225
View File
@@ -1,24 +1,10 @@
import * as fs from "fs";
import { existsSync, mkdirSync, writeFileSync } from "fs";
import path from "path";
import type { ActionState } from "./action-common";
import { getTemporaryDirectory } from "./actions-util";
import type { ActionConfig } from "./config/action-config";
import type { Config } from "./config-utils";
import { Language } from "./languages";
import { getActionsLogger, type Logger } from "./logging";
import {
getCodeQLDatabasePath,
getErrorMessage,
isErrnoException,
} from "./util";
/** The name of the directory we use to temporarily store diagnostics. */
export const TEMP_DIAGNOSTIC_DIR_NAME = "codeql-action-temp-diagnostics";
export const CODEQL_ACTION_DIAGNOSTIC_SUBDIR_PARTS = [
"diagnostic",
"codeql-action",
];
import { getActionsLogger } from "./logging";
import { getCodeQLDatabasePath } from "./util";
/**
* Known tags for diagnostics. There is currently only "internal-error",
@@ -98,16 +84,22 @@ export type DiagnosticMessage = DiagnosticMessageOptions & {
source: DiagnosticSource;
};
/** Represents a diagnostic message that has not yet been saved to the database. */
interface TemporaryDiagnostic {
/** The path to which the diagnostic has temporarily been written to. */
path: string;
/** The language the diagnostic is for, if any. */
language?: Language;
/** Represents a diagnostic message that has not yet been written to the database. */
interface UnwrittenDiagnostic {
/** The diagnostic message that has not yet been written. */
diagnostic: DiagnosticMessage;
/** The language the diagnostic is for. */
language: Language;
}
/** A list of diagnostics which have not yet been saved to the database. */
let temporaryDiagnostics: TemporaryDiagnostic[] = [];
/** A list of diagnostics which have not yet been written to disk. */
let unwrittenDiagnostics: UnwrittenDiagnostic[] = [];
/**
* A list of diagnostics which have not yet been written to disk,
* and where the language does not matter.
*/
let unwrittenDefaultLanguageDiagnostics: DiagnosticMessage[] = [];
/**
* Counter used to generate a unique suffix for each diagnostic filename, so that
@@ -136,54 +128,6 @@ export function makeDiagnostic(
};
}
/**
* Gets a path relative to {@link basePath} where diagnostics for {@link language} should be stored at.
*/
export function getLanguageDiagnosticsPath(
basePath: string,
language: Language,
) {
return path.resolve(
basePath,
language,
...CODEQL_ACTION_DIAGNOSTIC_SUBDIR_PARTS,
);
}
/**
* Gets a path relative to {@link tmpDir} where diagnostics should temporarily be stored
* until they can be moved to the database.
*/
export function getTempDiagnosticsRoot(tmpDir: string): string {
return path.resolve(tmpDir, TEMP_DIAGNOSTIC_DIR_NAME);
}
/**
* Gets a path relative to {@link tmpDir} where diagnostics for {@link language}
* can temporarily be stored at before the database is initialised. If {@link language}
* is `undefined`, then a non-language-specific path is returned.
*/
export function getTempLanguageDiagnosticsPath(
tmpDir: string,
language?: Language,
) {
const root = getTempDiagnosticsRoot(tmpDir);
if (language !== undefined) {
return getLanguageDiagnosticsPath(root, language);
}
return path.resolve(root, ...CODEQL_ACTION_DIAGNOSTIC_SUBDIR_PARTS);
}
/** Gets the path where diagnostics for {@link language} should be stored in the database. */
export function getDatabaseDiagnosticsPath(
config: ActionConfig<"dbLocation">,
language: Language,
) {
return getLanguageDiagnosticsPath(config.dbLocation, language);
}
/**
* Adds the given diagnostic to the database. If the database does not yet exist,
* the diagnostic will be written to it once it has been created.
@@ -193,34 +137,32 @@ export function getDatabaseDiagnosticsPath(
* @param diagnostic The diagnostic message to add to the database.
*/
export function addDiagnostic(
config: ActionConfig<"dbLocation" | "tempDir">,
config: Config,
language: Language,
diagnostic: DiagnosticMessage,
logger: Logger = getActionsLogger(),
) {
const logger = getActionsLogger();
const databasePath = language
? getCodeQLDatabasePath(config, language)
: config.dbLocation;
// Check that the database exists before writing to it. If the database does not yet exist,
// store the diagnostic in a temporary location and move it to the database later.
if (fs.existsSync(databasePath)) {
writeDiagnostic({ logger }, config, language, diagnostic);
// store the diagnostic in memory and write it later.
if (existsSync(databasePath)) {
writeDiagnostic(config, language, diagnostic);
} else {
logger.debug(
`Writing a diagnostic for ${language}, but the database at ${databasePath} does not exist yet.`,
);
// Write the diagnostic to a temporary location.
writeTemporaryDiagnosticFile(logger, config.tempDir, language, diagnostic);
unwrittenDiagnostics.push({ diagnostic, language });
}
}
/** Adds a diagnostic that is not specific to any language. */
export function addNoLanguageDiagnostic(
config: ActionConfig<"dbLocation" | "languages" | "tempDir"> | undefined,
config: Config | undefined,
diagnostic: DiagnosticMessage,
logger: Logger = getActionsLogger(),
) {
if (config !== undefined) {
addDiagnostic(
@@ -231,74 +173,10 @@ export function addNoLanguageDiagnostic(
diagnostic,
);
} else {
logger.debug(
`Writing a diagnostic not associated with any language, but we have no configuration to tell us where the database is yet.`,
);
// Write the diagnostic to a temporary location.
writeTemporaryDiagnosticFile(
logger,
getTemporaryDirectory(),
undefined,
diagnostic,
);
unwrittenDefaultLanguageDiagnostics.push(diagnostic);
}
}
/**
* Writes {@link diagnostic} for an optional {@link language} to a subdirectory of {@link tempDir}
* and tracks it as a diagnostic that was written to a temporary location.
*/
function writeTemporaryDiagnosticFile(
logger: Logger,
tempDir: string,
language: Language | undefined,
diagnostic: DiagnosticMessage,
) {
const tempDiagnosticsPath = getTempLanguageDiagnosticsPath(tempDir, language);
try {
// Write the diagnostic to a temporary location.
const diagnosticPath = writeDiagnosticFile(tempDiagnosticsPath, diagnostic);
// Track the temporary file.
temporaryDiagnostics.push({ path: diagnosticPath, language });
} catch (err) {
logger.warning(
`Unable to write temporary diagnostic to '${tempDiagnosticsPath}': ${getErrorMessage(err)}\n\n${JSON.stringify(diagnostic)}`,
);
}
}
/**
* Writes {@link diagnostic} to a file in {@link diagnosticsPath}.
*/
function writeDiagnosticFile(
diagnosticsPath: string,
diagnostic: DiagnosticMessage,
): string {
// Create the directory if it doesn't exist yet.
fs.mkdirSync(diagnosticsPath, { recursive: true });
// Include a monotonically increasing suffix to avoid filename collisions
// between diagnostics produced within the same millisecond.
const uniqueSuffix = (diagnosticCounter++).toString();
// We should only need to remove colons, but to be defensive, only allow a restricted set of
// characters.
const sanitizedTimestamp = diagnostic.timestamp.replace(
/[^a-zA-Z0-9.-]/g,
"",
);
const jsonPath = path.resolve(
diagnosticsPath,
`codeql-action-${sanitizedTimestamp}-${uniqueSuffix}.json`,
);
fs.writeFileSync(jsonPath, JSON.stringify(diagnostic));
return jsonPath;
}
/**
* Writes the given diagnostic to the database.
*
@@ -307,94 +185,78 @@ function writeDiagnosticFile(
* @param diagnostic The diagnostic message to add to the database.
*/
function writeDiagnostic(
action: ActionState<["Logger"]>,
config: ActionConfig<"dbLocation">,
language: Language,
config: Config,
language: Language | undefined,
diagnostic: DiagnosticMessage,
) {
const diagnosticsPath = getDatabaseDiagnosticsPath(config, language);
try {
writeDiagnosticFile(diagnosticsPath, diagnostic);
} catch (err) {
action.logger.warning(
`Unable to write diagnostic message to database: ${err}`,
);
action.logger.debug(JSON.stringify(diagnostic));
}
}
/** Report if there are temporary diagnostics and write them to the log. */
export function logTemporaryDiagnostics(action: ActionState<["Logger"]>) {
const num = temporaryDiagnostics.length;
if (num > 0 && action.logger.isDebug()) {
action.logger.startGroup("Diagnostics in temporary location");
action.logger.debug(
`${num} diagnostic(s) were not moved to the database and remain in the temporary directory.`,
);
for (const temporary of temporaryDiagnostics) {
try {
action.logger.debug(fs.readFileSync(temporary.path, "utf-8"));
} catch (err) {
action.logger.debug(
`Failed to read diagnostics from temporary location at '${temporary.path}': ${getErrorMessage(err)}`,
);
}
}
action.logger.endGroup();
}
}
/** Relocates all temporary diagnostics to the respective databases. */
export function flushDiagnostics(
action: ActionState<["Logger"]>,
config: ActionConfig<"dbLocation" | "languages">,
) {
const diagnosticsCount = temporaryDiagnostics.length;
action.logger.debug(
`Moving ${diagnosticsCount} diagnostic(s) to their respective databases.`,
const logger = getActionsLogger();
const databasePath = language
? getCodeQLDatabasePath(config, language)
: config.dbLocation;
const diagnosticsPath = path.resolve(
databasePath,
"diagnostic",
"codeql-action",
);
const failedFlushDiagnostics: TemporaryDiagnostic[] = [];
for (const temporary of temporaryDiagnostics) {
// If `temporary.language` is `undefined`, then we didn't have a `config` at the time that
// `addNoLanguageDiagnostic` was called. In that case, we arbitrarily choose the first
// configured language here like we would have done in `addNoLanguageDiagnostic`.
const directory = getDatabaseDiagnosticsPath(
config,
temporary.language ?? config.languages[0],
try {
// Create the directory if it doesn't exist yet.
mkdirSync(diagnosticsPath, { recursive: true });
// Include a monotonically increasing suffix to avoid filename collisions
// between diagnostics produced within the same millisecond.
const uniqueSuffix = (diagnosticCounter++).toString();
// We should only need to remove colons, but to be defensive, only allow a restricted set of
// characters.
const sanitizedTimestamp = diagnostic.timestamp.replace(
/[^a-zA-Z0-9.-]/g,
"",
);
const jsonPath = path.resolve(
diagnosticsPath,
`codeql-action-${sanitizedTimestamp}-${uniqueSuffix}.json`,
);
const filename = path.basename(temporary.path);
const destination = path.join(directory, filename);
try {
fs.mkdirSync(directory, { recursive: true });
try {
fs.renameSync(temporary.path, destination);
} catch (renameErr: unknown) {
// Rethrow the error if it isn't an EXDEV one.
if (!isErrnoException(renameErr) || renameErr.code !== "EXDEV") {
throw renameErr;
}
writeFileSync(jsonPath, JSON.stringify(diagnostic));
} catch (err) {
logger.warning(`Unable to write diagnostic message to database: ${err}`);
logger.debug(JSON.stringify(diagnostic));
}
}
// The error was an EXDEV one: try to copy the file and delete the temporary one.
fs.copyFileSync(temporary.path, destination);
fs.unlinkSync(temporary.path);
}
} catch (err) {
action.logger.warning(
`Failed to relocate '${temporary.path}' to '${destination}': ${getErrorMessage(err)}`,
);
/** Report if there are unwritten diagnostics and write them to the log. */
export function logUnwrittenDiagnostics() {
const logger = getActionsLogger();
const num = unwrittenDiagnostics.length;
if (num > 0) {
logger.warning(
`${num} diagnostic(s) could not be written to the database and will not appear on the Tool Status Page.`,
);
failedFlushDiagnostics.push(temporary);
for (const unwritten of unwrittenDiagnostics) {
logger.debug(JSON.stringify(unwritten.diagnostic));
}
}
}
// Reset the temporary diagnostics arrays.
temporaryDiagnostics = failedFlushDiagnostics;
/** Writes all unwritten diagnostics to disk. */
export function flushDiagnostics(config: Config) {
const logger = getActionsLogger();
const diagnosticsCount =
unwrittenDiagnostics.length + unwrittenDefaultLanguageDiagnostics.length;
logger.debug(`Writing ${diagnosticsCount} diagnostic(s) to database.`);
for (const unwritten of unwrittenDiagnostics) {
writeDiagnostic(config, unwritten.language, unwritten.diagnostic);
}
for (const unwritten of unwrittenDefaultLanguageDiagnostics) {
addNoLanguageDiagnostic(config, unwritten);
}
// Reset the unwritten diagnostics arrays.
unwrittenDiagnostics = [];
unwrittenDefaultLanguageDiagnostics = [];
}
/**
+7 -7
View File
@@ -10,6 +10,7 @@ import {
exportedForTesting,
} from "./diff-informed-analysis-utils";
import { Feature, FeatureEnablement, initFeatures } from "./feature-flags";
import { getRunnerLogger } from "./logging";
import { parseRepositoryNwo } from "./repository";
import {
setupTests,
@@ -18,7 +19,6 @@ import {
mockFeatureFlagApiEndpoint,
setupActionsVars,
makeMacro,
RecordingLogger,
} from "./testing-utils";
import { GitHubVariant, withTmpDir } from "./util";
import type { GitHubVersion } from "./util";
@@ -55,7 +55,7 @@ const testShouldPerformDiffInformedAnalysis = makeMacro({
setupActionsVars(tmpDir, tmpDir);
const testCase = { ...defaultTestCase, ...partialTestCase };
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const codeql = mockCodeQLVersion(testCase.codeQLVersion);
if (testCase.diffInformedQueriesEnvVar !== undefined) {
@@ -182,7 +182,7 @@ test.serial(
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const codeql = mockCodeQLVersion("2.21.0");
const features = createFeatures([Feature.DiffInformedQueries]);
@@ -207,7 +207,7 @@ test.serial(
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const codeql = mockCodeQLVersion("2.21.0");
// A features implementation whose getValue rejects, simulating an
// unexpected failure when determining whether diff-informed analysis
@@ -237,7 +237,7 @@ test.serial(
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const codeql = mockCodeQLVersion("2.21.0");
const features = createFeatures([Feature.DiffInformedQueries]);
@@ -274,7 +274,7 @@ test.serial(
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const codeql = mockCodeQLVersion("2.21.0");
const features = createFeatures([Feature.DiffInformedQueries]);
@@ -313,7 +313,7 @@ function runGetDiffRanges(changes: number, patch: string[] | undefined): any {
changes,
patch: patch?.join("\n"),
},
new RecordingLogger(),
getRunnerLogger(true),
);
}
+1
View File
@@ -3,6 +3,7 @@
*/
export enum DocUrl {
ASSIGNING_PERMISSIONS_TO_JOBS = "https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs",
AUTOMATIC_BUILD_FAILED = "https://docs.github.com/en/code-security/code-scanning/troubleshooting-code-scanning/automatic-build-failed",
CODEQL_BUILD_MODES = "https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#codeql-build-modes",
DEFINE_ENV_VARIABLES = "https://docs.github.com/en/actions/learn-github-actions/variables#defining-environment-variables-for-a-single-workflow",
+10 -34
View File
@@ -165,21 +165,9 @@ export enum EnvVar {
*/
SARIF_DUMP_DIR = "CODEQL_ACTION_SARIF_DUMP_DIR",
/**
* Whether to skip generating a failed SARIF file. Intended for testing purposes.
* This setting is more specific than `CODEQL_ACTION_TEST_MODE` and
* `CODEQL_ACTION_SKIP_SARIF_UPLOAD`, which imply this option.
*
* Setting this to `false` takes precedence over `CODEQL_ACTION_TEST_MODE: true` or
* `CODEQL_ACTION_SKIP_SARIF_UPLOAD: true`.
*/
SKIP_GENERATE_FAILED_SARIF = "CODEQL_ACTION_SKIP_GENERATE_FAILED_SARIF",
/**
* Whether to skip uploading SARIF results to GitHub. Intended for testing purposes.
* This setting is more specific than `CODEQL_ACTION_TEST_MODE`, which implies this option.
*
* Setting this to `false` takes precedence over `CODEQL_ACTION_TEST_MODE: true`.
*/
SKIP_SARIF_UPLOAD = "CODEQL_ACTION_SKIP_SARIF_UPLOAD",
@@ -293,8 +281,8 @@ export function getOptionalEnvVar(paramName: string): string | undefined {
* in tests, while clearly signalling in regular code that the consumer of the `ReadOnlyEnv` instance
* will only read from it.
*/
export class ReadOnlyEnv {
constructor(protected readonly vars: Record<string, string | undefined>) {}
export class ReadOnlyEnv<T extends string | undefined = string | undefined> {
constructor(protected readonly vars: Record<string, T>) {}
/** Clones the object while detaching the underlying environment from the original. */
public clone(): this {
@@ -302,24 +290,10 @@ export class ReadOnlyEnv {
}
/** Gets a copy of the underlying environment. */
public get(): Record<string, string | undefined> {
public get(): Record<string, T> {
return { ...this.vars };
}
/** Gets a copy of the underlying environment, where the values are not `undefined`. */
public getDefined(): Record<string, string> {
const result: Record<string, string> = {};
for (const [key, val] of Object.entries(this.vars)) {
if (val === undefined) {
continue;
}
result[key] = val;
}
return result;
}
/** Tries to get the value for `name` and throws if there isn't one. */
public getRequired(name: string): string {
return getRequiredEnvVar(this.vars, name);
@@ -331,7 +305,7 @@ export class ReadOnlyEnv {
}
/** Gets the entries of the underlying `ProcessEnv`. */
public entries(): Array<[string, string | undefined]> {
public entries(): Array<[string, T]> {
return Object.entries(this.vars);
}
@@ -361,17 +335,19 @@ export class ReadOnlyEnv {
* Use `ReadOnlyEnv` instead if you only plan to read from the environment.
* This type allows writing to the environment.
*/
export class Env extends ReadOnlyEnv {
export class Env<
T extends string | undefined = string | undefined,
> extends ReadOnlyEnv<T> {
private changed: boolean = false;
/** Sets an environment variable. */
public set(name: string, value: string | undefined): void {
public set(name: string, value: T): void {
this.vars[name] = value;
this.changed = true;
}
/** Sets all environment variables given by `vars`. */
public setAll(vars: Record<string, string | undefined>): void {
public setAll(vars: Record<string, T>): void {
for (const [key, val] of Object.entries(vars)) {
this.set(key, val);
}
@@ -385,7 +361,7 @@ export class Env extends ReadOnlyEnv {
* @param name The name of the environment variable to set and export.
* @param val The value to set and export for the environment variable.
*/
public export(name: string, val: string): void {
public export(name: string, val: T): void {
// Setting the environment variable for this instance is always OK, including
// in tests, since we use fresh `Env` instances whenever needed. This allows
// tests to pass that rely on that part of the `core.exportVariable` behaviour.
+56
View File
@@ -74,6 +74,12 @@ export enum Feature {
AllowMergeConfigFiles = "allow_merge_config_files",
/** Controls whether we allow multiple values for the `analysis-kinds` input. */
AllowMultipleAnalysisKinds = "allow_multiple_analysis_kinds",
/**
* Controls whether we delete CodeQL bundles that we are not going to use from the toolcache
* before downloading a different bundle, in order to reclaim disk space.
*/
CleanupToolcacheBundles = "cleanup_toolcache_bundles",
CleanupTrapCaches = "cleanup_trap_caches",
/** Whether to allow the `config-file` input to be specified via a repository property. */
ConfigFileRepositoryProperty = "config_file_repository_property",
CppDependencyInstallation = "cpp_dependency_installation_enabled",
@@ -102,9 +108,24 @@ export enum Feature {
OverlayAnalysisCodeScanningRuby = "overlay_analysis_code_scanning_ruby",
OverlayAnalysisCpp = "overlay_analysis_cpp",
OverlayAnalysisCsharp = "overlay_analysis_csharp",
/** Disable TRAP caching when overlay analysis is enabled. */
OverlayAnalysisDisableTrapCaching = "overlay_analysis_disable_trap_caching",
OverlayAnalysisGo = "overlay_analysis_go",
OverlayAnalysisJava = "overlay_analysis_java",
OverlayAnalysisJavascript = "overlay_analysis_javascript",
/**
* When set, chooses the default CodeQL CLI version as the highest version that is both enabled by
* feature flags and present as an overlay-base database in the Actions cache for the configured
* languages. Falls back to the highest feature flagged version if no intersecting overlay-base
* database exists in the cache.
*/
OverlayAnalysisMatchCodeqlVersion = "overlay_analysis_match_codeql_version",
/**
* Like `OverlayAnalysisMatchCodeqlVersion`, but only logs a diagnostic with the version that
* would have been chosen instead of actually changing the default CodeQL CLI version.
* `OverlayAnalysisMatchCodeqlVersion` overrides this flag.
*/
OverlayAnalysisMatchCodeqlVersionDryRun = "overlay_analysis_match_codeql_version_dry_run",
/**
* Lowers the overlay minimum available disk space to 8 GB. The lowest enabled limit wins; if
* none are enabled, the default applies.
@@ -143,6 +164,11 @@ export enum Feature {
OverlayAnalysisStatusCheck = "overlay_analysis_status_check",
/** Controls whether overlay build failures on the default branch are stored in the Actions cache. */
OverlayAnalysisStatusSave = "overlay_analysis_status_save",
/**
* Controls whether we may download a bundle containing only the single language being analysed,
* rather than the combined bundle that contains every language.
*/
PerLanguageBundles = "per_language_bundles_v2",
QaTelemetryEnabled = "qa_telemetry_enabled",
/** Routes (some) API requests through the registry proxy. */
ProxyApiRequests = "proxy_api_requests",
@@ -195,6 +221,16 @@ export const featureConfig = {
envVar: "CODEQL_ACTION_ALLOW_MULTIPLE_ANALYSIS_KINDS",
minimumVersion: undefined,
},
[Feature.CleanupToolcacheBundles]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES",
minimumVersion: undefined,
},
[Feature.CleanupTrapCaches]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CLEANUP_TRAP_CACHES",
minimumVersion: undefined,
},
[Feature.ConfigFileRepositoryProperty]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CONFIG_FILE_REPOSITORY_PROPERTY",
@@ -343,6 +379,21 @@ export const featureConfig = {
minimumVersion: CODEQL_OVERLAY_MINIMUM_VERSION_RUBY,
},
// Other overlay-related feature flags
[Feature.OverlayAnalysisDisableTrapCaching]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_DISABLE_TRAP_CACHING",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisMatchCodeqlVersion]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MATCH_CODEQL_VERSION",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisMatchCodeqlVersionDryRun]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MATCH_CODEQL_VERSION_DRY_RUN",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisMinDisk8Gb]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MIN_DISK_8_GB",
@@ -388,6 +439,11 @@ export const featureConfig = {
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_SKIP_RESOURCE_CHECKS",
minimumVersion: undefined,
},
[Feature.PerLanguageBundles]: {
defaultValue: false,
envVar: "CODEQL_ACTION_PER_LANGUAGE_BUNDLES",
minimumVersion: undefined,
},
[Feature.QaTelemetryEnabled]: {
defaultValue: false,
envVar: "CODEQL_ACTION_QA_TELEMETRY",
+9 -8
View File
@@ -2,6 +2,7 @@ import test from "ava";
import * as sinon from "sinon";
import * as api from "../api-client";
import { getRunnerLogger } from "../logging";
import { parseRepositoryNwo } from "../repository";
import { RecordingLogger, setupTests } from "../testing-utils";
@@ -18,7 +19,7 @@ test.serial(
url: "",
data: {},
});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const mockRepositoryNwo = parseRepositoryNwo("owner/repo");
await t.throwsAsync(
properties.loadPropertiesFromApi(logger, mockRepositoryNwo),
@@ -38,7 +39,7 @@ test.serial(
url: "",
data: [{}],
});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const mockRepositoryNwo = parseRepositoryNwo("owner/repo");
await t.throwsAsync(
properties.loadPropertiesFromApi(logger, mockRepositoryNwo),
@@ -62,7 +63,7 @@ test.serial(
{ property_name: "also-not-used-by-us", value: ["A", "B", "C"] },
],
});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const mockRepositoryNwo = parseRepositoryNwo("owner/repo");
await t.notThrowsAsync(
properties.loadPropertiesFromApi(logger, mockRepositoryNwo),
@@ -85,7 +86,7 @@ test.serial("loadPropertiesFromApi loads known properties", async (t) => {
{ property_name: "unknown-property", value: "something" },
] satisfies properties.GitHubPropertiesResponse,
});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const mockRepositoryNwo = parseRepositoryNwo("owner/repo");
const response = await properties.loadPropertiesFromApi(
logger,
@@ -112,7 +113,7 @@ test.serial("loadPropertiesFromApi parses true boolean property", async (t) => {
{ property_name: "github-codeql-extra-queries", value: "+queries" },
] satisfies properties.GitHubPropertiesResponse,
});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const warningSpy = sinon.spy(logger, "warning");
const mockRepositoryNwo = parseRepositoryNwo("owner/repo");
const response = await properties.loadPropertiesFromApi(
@@ -140,7 +141,7 @@ test.serial(
},
] satisfies properties.GitHubPropertiesResponse,
});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const warningSpy = sinon.spy(logger, "warning");
const mockRepositoryNwo = parseRepositoryNwo("owner/repo");
const response = await properties.loadPropertiesFromApi(
@@ -163,7 +164,7 @@ test.serial(
url: "",
data: [{ property_name: "github-codeql-extra-queries", value: 123 }],
});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const mockRepositoryNwo = parseRepositoryNwo("owner/repo");
await t.throwsAsync(
properties.loadPropertiesFromApi(logger, mockRepositoryNwo),
@@ -189,7 +190,7 @@ test.serial(
},
] satisfies properties.GitHubPropertiesResponse,
});
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const warningSpy = sinon.spy(logger, "warning");
const mockRepositoryNwo = parseRepositoryNwo("owner/repo");
const response = await properties.loadPropertiesFromApi(
+2 -3
View File
@@ -8,12 +8,11 @@ import {
FeatureWithoutCLI,
initFeatures,
} from "../feature-flags";
import type { Logger } from "../logging";
import { getRunnerLogger } from "../logging";
import { parseRepositoryNwo } from "../repository";
import {
LoggedMessage,
mockCodeQLVersion,
RecordingLogger,
setupActionsVars,
} from "../testing-utils";
import { ToolsFeature } from "../tools-features";
@@ -52,7 +51,7 @@ export function assertAllFeaturesUndefinedInApi(
export function setUpFeatureFlagTests(
tmpDir: string,
logger: Logger = new RecordingLogger(),
logger = getRunnerLogger(true),
gitHubVersion = { type: GitHubVariant.DOTCOM } as util.GitHubVersion,
): FeatureEnablement {
setupActionsVars(tmpDir, tmpDir);
+5 -4
View File
@@ -5,8 +5,9 @@ import * as ava from "ava";
import test from "ava";
import * as fingerprints from "./fingerprints";
import { getRunnerLogger } from "./logging";
import * as sarif from "./sarif";
import { RecordingLogger, setupTests } from "./testing-utils";
import { setupTests } from "./testing-utils";
import * as util from "./util";
setupTests(test);
@@ -130,7 +131,7 @@ function testResolveUriToFile(uri: any, index: any, artifactsURIs: any[]) {
location,
artifacts,
process.cwd(),
new RecordingLogger(),
getRunnerLogger(true),
);
}
@@ -217,7 +218,7 @@ test("addFingerprints", async (t) => {
await fingerprints.addFingerprints(
input,
sourceRoot,
new RecordingLogger(),
getRunnerLogger(true),
),
expected,
);
@@ -245,7 +246,7 @@ test("missingRegions", async (t) => {
await fingerprints.addFingerprints(
input,
sourceRoot,
new RecordingLogger(),
getRunnerLogger(true),
),
expected,
);
+11 -57
View File
@@ -1,6 +1,3 @@
import * as fs from "fs";
import * as path from "path";
import * as core from "@actions/core";
import test, { ExecutionContext } from "ava";
import * as sinon from "sinon";
@@ -11,10 +8,10 @@ import * as apiClient from "./api-client";
import * as codeql from "./codeql";
import * as configUtils from "./config-utils";
import * as debugArtifacts from "./debug-artifacts";
import { getTempDiagnosticsRoot } from "./diagnostics";
import { EnvVar } from "./environment";
import { Feature } from "./feature-flags";
import * as initActionPostHelper from "./init-action-post-helper";
import { getRunnerLogger } from "./logging";
import { OverlayDatabaseMode } from "./overlay/overlay-database-mode";
import * as overlayStatus from "./overlay/status";
import { parseRepositoryNwo } from "./repository";
@@ -23,7 +20,6 @@ import {
createFeatures,
createTestConfig,
DEFAULT_ACTIONS_VARS,
getCodeQLForTesting,
getTestEnv,
makeMacro,
makeVersionInfo,
@@ -39,47 +35,6 @@ const NUM_BYTES_PER_GIB = 1024 * 1024 * 1024;
setupTests(test);
test.serial("generateFailedSarif picks up unwritten diagnostics", async (t) =>
util.withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const tempDiagDir = getTempDiagnosticsRoot(tmpDir);
fs.mkdirSync(tempDiagDir, { recursive: true });
const codeqlObject = await getCodeQLForTesting();
const diagnosticsExport = sinon
.stub(codeqlObject, "diagnosticsExport")
.resolves();
sinon.stub(codeql, "getCodeQL").resolves(codeqlObject);
const sarifFile = path.join(tmpDir, "failed.sarif");
const result = await initActionPostHelper.generateFailedSarif(
logger,
createFeatures([Feature.ExportDiagnosticsEnabled]),
createTestConfig({
tempDir: tmpDir,
// Explicitly undefined to ensure we get the "no database" path.
dbLocation: undefined,
}),
"category",
"",
sarifFile,
);
t.truthy(result);
t.is(diagnosticsExport.callCount, 1);
t.true(
diagnosticsExport.calledOnceWith(
sarifFile,
"category",
sinon.match.any,
tempDiagDir,
),
);
}),
);
test.serial("init-post action with debug mode off", async (t) => {
return await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -107,7 +62,7 @@ test.serial("init-post action with debug mode off", async (t) => {
createFeatures([]),
"success",
getTestEnv(),
new RecordingLogger(),
getRunnerLogger(true),
);
t.assert(uploadAllAvailableDebugArtifactsSpy.notCalled);
@@ -131,7 +86,7 @@ test.serial("init-post action with debug mode on", async (t) => {
createFeatures([]),
"success",
getTestEnv(),
new RecordingLogger(),
getRunnerLogger(true),
);
t.assert(uploadAllAvailableDebugArtifactsSpy.called);
@@ -428,7 +383,7 @@ test.serial(
createFeatures([Feature.OverlayAnalysisStatusSave]),
"success",
getTestEnv(),
new RecordingLogger(),
getRunnerLogger(true),
);
t.true(
@@ -498,7 +453,7 @@ test.serial(
createFeatures([]),
"success",
getTestEnv(),
new RecordingLogger(),
getRunnerLogger(true),
);
t.true(
@@ -542,7 +497,7 @@ test.serial("does not save overlay status when build successful", async (t) => {
createFeatures([Feature.OverlayAnalysisStatusSave]),
"success",
env,
new RecordingLogger(),
getRunnerLogger(true),
);
t.true(
@@ -581,7 +536,7 @@ test.serial(
createFeatures([]),
"success",
getTestEnv(),
new RecordingLogger(),
getRunnerLogger(true),
);
t.true(
@@ -634,7 +589,7 @@ async function runOverlayPostStep({
createFeatures([Feature.OverlayAnalysisStatusSave]),
jobStatus,
env,
new RecordingLogger(),
getRunnerLogger(true),
);
return { saveOverlayStatusStub };
@@ -782,7 +737,7 @@ async function testFailedSarifUpload(
.withArgs("matrix")
.returns(JSON.stringify(matrix));
const codeqlObject = await getCodeQLForTesting();
const codeqlObject = await codeql.getCodeQLForTesting();
sinon.stub(codeql, "getCodeQL").resolves(codeqlObject);
sinon.stub(codeqlObject, "getVersion").resolves(makeVersionInfo("2.17.6"));
const databaseExportDiagnosticsStub = sinon.stub(
@@ -809,7 +764,7 @@ async function testFailedSarifUpload(
config,
parseRepositoryNwo("github/codeql-action"),
createFeatures(features),
new RecordingLogger(),
getRunnerLogger(true),
);
if (expectUpload) {
t.deepEqual(result, {
@@ -838,7 +793,6 @@ async function testFailedSarifUpload(
}
t.true(
uploadFiles.calledOnceWith(
config,
sinon.match.string,
sinon.match.string,
category,
@@ -878,7 +832,7 @@ async function mockRiskAssessmentEnv(matrix: string) {
.stub(apiClient, "getGitHubVersion")
.resolves({ type: util.GitHubVariant.GHES, version: "3.0.0" });
const codeqlObject = await getCodeQLForTesting();
const codeqlObject = await codeql.getCodeQLForTesting();
const databaseExportDiagnostics = sinon
.stub(codeqlObject, "databaseExportDiagnostics")
.resolves();
+20 -66
View File
@@ -24,7 +24,6 @@ import {
type UploadDebugArtifactsConfig,
} from "./debug-artifacts";
import * as dependencyCaching from "./dependency-caching";
import { getTempDiagnosticsRoot } from "./diagnostics";
import { EnvVar, ReadOnlyEnv } from "./environment";
import { Feature, FeatureEnablement } from "./feature-flags";
import { Logger } from "./logging";
@@ -45,7 +44,6 @@ import {
getRequiredEnvParam,
parseMatrixInput,
Result,
shouldSkipGenerateFailedSarif,
shouldSkipSarifUpload,
Success,
wrapError,
@@ -117,7 +115,7 @@ async function prepareFailedSarif(
const jobName = getRequiredEnvParam("GITHUB_JOB");
const matrix = parseMatrixInput(actionsUtil.getRequiredInput("matrix"));
if (shouldSkipGenerateFailedSarif()) {
if (shouldSkipSarifUpload()) {
return new Failure({
upload_failed_run_skipped_because: "SARIF upload is disabled",
});
@@ -178,16 +176,7 @@ async function prepareFailedSarif(
}
}
/**
* Generates a SARIF file for a failed run, i.e. an analysis where no SARIF was produced as a result of the analysis.
*
* Not intended to be called directly, other than by `prepareFailedSarif`.
*
* @param category The analysis category to include in the SARIF.
* @param checkoutPath The path where the repository was checked out at.
* @param sarifFile The SARIF file to produce.
*/
export async function generateFailedSarif(
async function generateFailedSarif(
logger: Logger,
features: FeatureEnablement,
config: ActionConfig<"codeQLCmd" | "tempDir">,
@@ -204,26 +193,11 @@ export async function generateFailedSarif(
}
// If there is no database or the feature flag is off, we run 'export diagnostics'
const exportDiagnosticsEnabled = await features.getValue(
Feature.ExportDiagnosticsEnabled,
codeql,
);
if (databasePath === undefined || !exportDiagnosticsEnabled) {
// If we don't have a database path, then we may have diagnostics that were written to a
// temporary location. Get the path that they would have been written to and check whether
// it exists. If so, pass it on to the CLI. The CLI will not complain if we give it a
// path that doesn't exist, so we are just being extra defensive here.
// We also don't pass the path to the CLI if `exportDiagnosticsEnabled` is not enabled, to preserve
// its intended behaviour of controlling whether diagnostics are included in the SARIF or not.
let diagnosticDir: string | undefined = getTempDiagnosticsRoot(
config.tempDir,
);
if (!exportDiagnosticsEnabled || !fs.existsSync(diagnosticDir)) {
diagnosticDir = undefined;
}
await codeql.diagnosticsExport(sarifFile, category, config, diagnosticDir);
if (
databasePath === undefined ||
!(await features.getValue(Feature.ExportDiagnosticsEnabled, codeql))
) {
await codeql.diagnosticsExport(sarifFile, category, config);
} else {
// We call 'database export-diagnostics' to find any per-database diagnostics.
await codeql.databaseExportDiagnostics(databasePath, sarifFile, category);
@@ -252,7 +226,6 @@ async function maybeUploadFailedSarif(
logger.info(`Uploading failed SARIF file ${failedSarif.sarifFile}`);
const uploadResult = await uploadLib.uploadFiles(
config,
failedSarif.sarifFile,
failedSarif.checkoutPath,
failedSarif.category,
@@ -266,14 +239,6 @@ async function maybeUploadFailedSarif(
logger,
{ isUnsuccessfulExecution: true },
);
// If SARIF upload is disabled for testing, `uploadFiles` will not have uploaded the failed
// SARIF file and instead will have written it to disk. Upload it as a workflow artifact so that
// we can inspect it in validation jobs.
if (uploadResult.sarifID && shouldSkipSarifUpload()) {
await uploadSarifArtifact(logger, "test-mode-sarif", failedSarif);
}
return uploadResult
? { ...uploadResult.statusReport, sarifID: uploadResult.sarifID }
: {};
@@ -292,34 +257,23 @@ async function maybeUploadFailedSarifArtifact(
}
const failedSarif = failedSarifResult.value;
return uploadSarifArtifact(logger, "sarif-artifact", failedSarif);
}
/** Uploads a failed SARIF file as workflow artifact. */
async function uploadSarifArtifact(
logger: Logger,
prefix: string,
failedSarif: FailedSarifInfo,
): Promise<UploadFailedSarifResult> {
logger.startGroup(`Uploading failed SARIF file as artifact`);
logger.info(`Uploading '${failedSarif.sarifFile}' as artifact`);
logger.info(
`Uploading failed SARIF file ${failedSarif.sarifFile} as artifact`,
);
try {
const gitHubVersion = await getGitHubVersion();
const client = await getArtifactUploaderClient(logger, gitHubVersion.type);
const gitHubVersion = await getGitHubVersion();
const client = await getArtifactUploaderClient(logger, gitHubVersion.type);
const suffix = getArtifactSuffix(actionsUtil.getOptionalInput("matrix"));
const name = sanitizeArtifactName(`${prefix}-${suffix}`);
await client.uploadArtifact(
name,
[path.normalize(failedSarif.sarifFile)],
path.normalize(path.dirname(failedSarif.sarifFile)),
);
const suffix = getArtifactSuffix(actionsUtil.getOptionalInput("matrix"));
const name = sanitizeArtifactName(`sarif-artifact-${suffix}`);
await client.uploadArtifact(
name,
[path.normalize(failedSarif.sarifFile)],
path.normalize(".."),
);
return { sarifID: name };
} finally {
logger.endGroup();
}
return { sarifID: name };
}
/**
-3
View File
@@ -76,9 +76,6 @@ async function run(action: ActionState<["Base", "Logger", "Env"]>) {
// Try to load the "failed" configuration file instead which may have been written by the `init` step
// with a minimal configuration for this post-action.
if (config === undefined) {
logger.debug(
"Will try to load the configuration file from an aborted 'init' step, since there is no regular one.",
);
config = getAbortedInitConfig(action, tmpDir);
}
+13 -9
View File
@@ -40,7 +40,7 @@ import {
addDiagnostic,
addNoLanguageDiagnostic,
flushDiagnostics,
logTemporaryDiagnostics,
logUnwrittenDiagnostics,
makeDiagnostic,
makeTelemetryDiagnostic,
} from "./diagnostics";
@@ -62,7 +62,6 @@ import {
runDatabaseInitCluster,
} from "./init";
import { JavaEnvVars, BuiltInLanguage } from "./languages";
import { checkSwiftCompatibility } from "./languages/swift";
import { Logger, withGroupAsync } from "./logging";
import {
downloadOverlayBaseDatabaseFromCache,
@@ -419,7 +418,14 @@ async function run(
logger,
});
checkSwiftCompatibility(actionState, config);
if (
config.languages.includes(BuiltInLanguage.swift) &&
process.platform !== "darwin"
) {
throw new ConfigurationError(
`Swift analysis is only supported on macOS runner images. Please migrate to a macOS runner.`,
);
}
if (repositoryPropertiesResult.isFailure()) {
addNoLanguageDiagnostic(
@@ -478,7 +484,6 @@ async function run(
if (statusReportBase !== undefined) {
await sendStatusReport(statusReportBase);
}
logTemporaryDiagnostics(actionState);
return;
}
@@ -761,15 +766,14 @@ async function run(
);
}
// Move diagnostics to the database that we previously kept in a temporary location
// because the database did not exist until now
flushDiagnostics(actionState, config);
// Write diagnostics to the database that we previously stored in memory because the database
// did not exist until now.
flushDiagnostics(config);
// We save the config here instead of at the end of `initConfig` because we
// may have updated the config returned from `initConfig`, e.g. to revert to
// `OverlayDatabaseMode.None` if we failed to download an overlay-base
// database.
config.successfullyInitialised = true;
await configUtils.saveConfig(config, logger);
core.setOutput("codeql-path", config.codeQLCmd);
@@ -793,7 +797,7 @@ async function run(
);
return;
} finally {
logTemporaryDiagnostics(actionState);
logUnwrittenDiagnostics();
}
await sendCompletedStatusReport(
startedAt,
-70
View File
@@ -1,70 +0,0 @@
import test from "ava";
import sinon from "sinon";
import { ActionsEnvVars } from "../environment";
import {
createTestConfig,
getCodeQLForTesting,
getTestEnv,
initAllState,
makeVersionInfo,
RecordingLogger,
} from "../testing-utils";
import { ConfigurationError, withTmpDir } from "../util";
import { checkSwiftCompatibility } from "./swift";
import { BuiltInLanguage } from ".";
test("checkSwiftCompatibility doesn't throw for non-Swift languages", (t) => {
for (const language of Object.values(BuiltInLanguage)) {
if (language === BuiltInLanguage.swift) {
continue;
}
t.notThrows(() =>
checkSwiftCompatibility(
initAllState(),
createTestConfig({ languages: [language] }),
),
);
}
});
test("checkSwiftCompatibility doesn't throw for Swift on darwin", async (t) =>
withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const env = getTestEnv();
env.set(ActionsEnvVars.RUNNER_TEMP, tmpDir);
const codeql = await getCodeQLForTesting("codeql-for-testing", logger, env);
sinon.stub(codeql, "getVersion").resolves(makeVersionInfo("2.27.0"));
t.notThrows(() =>
checkSwiftCompatibility(
initAllState({ platform: "darwin", env, logger }),
createTestConfig({
tempDir: tmpDir,
languages: [BuiltInLanguage.swift],
}),
),
);
}));
const nonDarwinPlatforms: NodeJS.Platform[] = ["linux", "win32"];
for (const nonDarwinPlatform of nonDarwinPlatforms) {
test(`checkSwiftCompatibility throws for Swift on ${nonDarwinPlatform}`, (t) => {
t.throws(
() => {
checkSwiftCompatibility(
initAllState({ platform: nonDarwinPlatform }),
createTestConfig({ languages: [BuiltInLanguage.swift] }),
);
},
{
instanceOf: ConfigurationError,
message: /^Swift analysis is only supported on macOS runner images/,
},
);
});
}
-27
View File
@@ -1,27 +0,0 @@
import { ActionState } from "../action-common";
import { Config } from "../config-utils";
import { ConfigurationError } from "../util";
import { BuiltInLanguage } from ".";
/**
* Determines whether we can run a Swift analysis on the current runner.
*
* @param action The Action state.
* @param config The Action configuration.
*
* @throws {ConfigurationError} If Swift analysis is not possible on the current runner.
*/
export function checkSwiftCompatibility(
action: ActionState<["Base"]>,
config: Config,
) {
if (
config.languages.includes(BuiltInLanguage.swift) &&
action.platform !== "darwin"
) {
throw new ConfigurationError(
`Swift analysis is only supported on macOS runner images. Please migrate to a macOS runner.`,
);
}
}
-17
View File
@@ -1,17 +0,0 @@
import test from "ava";
import { joinMessageStrings } from "./logging";
import { setupTests } from "./testing-utils";
setupTests(test);
test("joinMessageStrings", async (t) => {
// For strings and errors, it is the identity function.
t.deepEqual(joinMessageStrings("Hello"), "Hello");
const error = new Error("Some error");
t.deepEqual(joinMessageStrings(error), error);
// For arrays of strings, we get a join-ed string.
t.deepEqual(joinMessageStrings(["foo", "bar"]), "foo bar");
});
+25 -72
View File
@@ -1,92 +1,45 @@
import * as core from "@actions/core";
/** Types that all of our logging functions accept. */
export type Loggable = string | string[];
/** Some logging functions also accept errors directly. */
export type LoggableError = Loggable | Error;
export interface Logger {
/**
* Logs {@link message} at debug-level.
* In Actions, this will only be output if workflow debugging is enabled.
*
* @param message
* A string or an array of strings. If an array of strings is provided, they are joined using
* space as the delimiter and printed as a single line.
*/
debug: (message: Loggable) => void;
/**
* Logs {@link message} at info-level.
* In Actions, this will always be output.
*
* @param message
* A string or an array of strings. If an array of strings is provided, they are joined using
* space as the delimiter and printed as a single line.
*/
info: (message: Loggable) => void;
/**
* Logs {@link message} at warning-level.
* In Actions, this will always be output and highlighted as a warning. It will additionally
* result in a warning-level workflow annotation.
*
* @param message
* A string or an array of strings. If an array of strings is provided, they are joined using
* space as the delimiter and printed as a single line.
*/
warning: (message: LoggableError) => void;
/**
* Logs {@link message} at error-level.
* In Actions, this will always be output and highlighted as an error. It will additionally
* result in an error-level workflow annotation.
*
* @param message
* A string or an array of strings. If an array of strings is provided, they are joined using
* space as the delimiter and printed as a single line.
*/
error: (message: LoggableError) => void;
/**
* Gets a value indicating whether debugging is enabled.
*/
debug: (message: string) => void;
info: (message: string) => void;
warning: (message: string | Error) => void;
error: (message: string | Error) => void;
isDebug: () => boolean;
startGroup: (name: string) => void;
endGroup: () => void;
}
/** If `message` is an array of strings, the array is `join`-ed into one string separated by spaces. */
export const joinMessageStrings = <T>(
message: string[] | string | T,
): string | T => {
if (Array.isArray(message)) {
return message.join(" ");
} else {
return message;
}
};
/**
* Wraps a logging function so that `joinMessageStrings` is applied to the input before
* calling the wrapped logging function.
*/
const autoJoinMessageStrings = <T>(fn: (msg: string | T) => void) => {
return (message: string[] | string | T) => {
return fn(joinMessageStrings(message));
};
};
export function getActionsLogger(): Logger {
return {
debug: autoJoinMessageStrings(core.debug),
info: autoJoinMessageStrings(core.info),
warning: autoJoinMessageStrings(core.warning),
error: autoJoinMessageStrings(core.error),
debug: core.debug,
info: core.info,
warning: core.warning,
error: core.error,
isDebug: core.isDebug,
startGroup: core.startGroup,
endGroup: core.endGroup,
};
}
export function getRunnerLogger(debugMode: boolean): Logger {
return {
// eslint-disable-next-line no-console
debug: debugMode ? console.debug : () => undefined,
// eslint-disable-next-line no-console
info: console.info,
// eslint-disable-next-line no-console
warning: console.warn,
// eslint-disable-next-line no-console
error: console.error,
isDebug: () => debugMode,
startGroup: () => undefined,
endGroup: () => undefined,
};
}
export function withGroup<T>(groupName: string, f: () => T): T {
core.startGroup(groupName);
try {
+10 -10
View File
@@ -11,11 +11,11 @@ import type { ResolveDatabaseOutput } from "../codeql";
import * as environment from "../environment";
import * as gitUtils from "../git-utils";
import { BuiltInLanguage } from "../languages";
import { getRunnerLogger } from "../logging";
import {
createTestConfig,
makeMacro,
mockCodeQLVersion,
RecordingLogger,
setupTests,
} from "../testing-utils";
import * as utils from "../util";
@@ -63,7 +63,7 @@ const testDownloadOverlayBaseDatabaseFromCache = makeMacro({
const dbLocation = path.join(tmpDir, "db");
await fs.promises.mkdir(dbLocation, { recursive: true });
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const testCase = { ...defaultDownloadTestCase, ...partialTestCase };
const config = createTestConfig({
dbLocation,
@@ -214,7 +214,7 @@ testDownloadOverlayBaseDatabaseFromCache.serial(
);
test.serial("overlay-base database cache keys remain stable", async (t) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const config = createTestConfig({ languages: ["python", "javascript"] });
const codeQlVersion = "2.23.0";
const commitOid = "abc123def456";
@@ -261,7 +261,7 @@ test.serial("overlay-base database cache keys remain stable", async (t) => {
test.serial(
"getCodeQlVersionsForOverlayBaseDatabases returns unique versions sorted latest first",
async (t) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
sinon.stub(apiClient, "getAutomationID").resolves("test-automation-id/");
sinon.stub(apiClient, "listActionsCaches").resolves([
@@ -287,7 +287,7 @@ test.serial(
test.serial(
"getCodeQlVersionsForOverlayBaseDatabases returns empty list when no caches exist",
async (t) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
sinon.stub(apiClient, "getAutomationID").resolves("test-automation-id/");
sinon.stub(apiClient, "listActionsCaches").resolves([]);
@@ -303,7 +303,7 @@ test.serial(
test.serial(
"getCodeQlVersionsForOverlayBaseDatabases returns empty list when cache keys are unparseable",
async (t) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
sinon.stub(apiClient, "getAutomationID").resolves("test-automation-id/");
sinon.stub(apiClient, "listActionsCaches").resolves([
@@ -324,7 +324,7 @@ test.serial(
test.serial(
"getCodeQlVersionsForOverlayBaseDatabases returns the single version when only one cache exists",
async (t) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
sinon.stub(apiClient, "getAutomationID").resolves("test-automation-id/");
sinon.stub(apiClient, "listActionsCaches").resolves([
@@ -344,7 +344,7 @@ test.serial(
test.serial(
"getCodeQlVersionsForOverlayBaseDatabases resolves language aliases",
async (t) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
// The alias `c++` should be resolved to "cpp" and match cache entries keyed with "cpp"
sinon.stub(apiClient, "getAutomationID").resolves("test-automation-id/");
@@ -365,7 +365,7 @@ test.serial(
test.serial(
"getCodeQlVersionsForOverlayBaseDatabases de-duplicates resolved language aliases",
async (t) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
sinon.stub(apiClient, "getAutomationID").resolves("test-automation-id/");
const listActionsCachesStub = sinon
@@ -391,7 +391,7 @@ test.serial(
test.serial(
"getCodeQlVersionsForOverlayBaseDatabases ignores nightly versions with build metadata",
async (t) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
sinon.stub(apiClient, "getAutomationID").resolves("test-automation-id/");
sinon.stub(apiClient, "listActionsCaches").resolves([
+6 -9
View File
@@ -6,11 +6,8 @@ import * as sinon from "sinon";
import * as actionsUtil from "../actions-util";
import * as gitUtils from "../git-utils";
import {
createTestConfig,
RecordingLogger,
setupTests,
} from "../testing-utils";
import { getRunnerLogger } from "../logging";
import { createTestConfig, setupTests } from "../testing-utils";
import { withTmpDir } from "../util";
import { writeBaseDatabaseOidsFile, writeOverlayChangesFile } from ".";
@@ -30,7 +27,7 @@ test.serial(
),
);
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const config = createTestConfig({ dbLocation });
// Mock the getFileOidsUnderPath function to return base OIDs
@@ -94,7 +91,7 @@ test.serial(
),
);
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const config = createTestConfig({ dbLocation });
// Mock the getFileOidsUnderPath function to return base OIDs
@@ -172,7 +169,7 @@ test.serial(
),
);
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const config = createTestConfig({ dbLocation });
// Mock the getFileOidsUnderPath function to return base OIDs
@@ -235,7 +232,7 @@ test.serial(
),
);
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
const config = createTestConfig({ dbLocation });
// Base OIDs (sourceRoot-relative paths)
+28
View File
@@ -1,6 +1,7 @@
import test from "ava";
import { ActionsEnvVars } from "./environment";
import { Feature } from "./feature-flags";
import { BuiltInLanguage } from "./languages";
import {
getOtherLanguagePacksReason,
@@ -11,6 +12,7 @@ import {
} from "./per-language-bundles";
import { BundlePlatform } from "./platform";
import {
createFeatures,
getRecordingLogger,
getTestEnv,
initAllState,
@@ -38,6 +40,7 @@ async function checkEligibility(
env: getTestEnv({
[ActionsEnvVars.RUNNER_ENVIRONMENT]: "github-hosted",
}),
features: createFeatures([Feature.PerLanguageBundles]),
...stateOverrides,
}),
{ ...ELIGIBLE_OPTIONS, ...overrides },
@@ -142,6 +145,27 @@ test("getPerLanguageBundleLanguage requires a supported release version", async
t.is(await checkEligibility({ cliVersion: "2.27.1" }), BuiltInLanguage.java);
});
test("getPerLanguageBundleLanguage requires the feature flag", async (t) => {
t.is(await checkEligibility({}, { features: createFeatures([]) }), undefined);
});
test("getPerLanguageBundleLanguage explains a disabled feature before checking eligibility", async (t) => {
const messages: LoggedMessage[] = [];
const language = await getPerLanguageBundleLanguage(
initAllState({
features: createFeatures([]),
logger: getRecordingLogger(messages, { logToConsole: false }),
}),
{ ...ELIGIBLE_OPTIONS, rawLanguages: undefined, cliVersion: undefined },
);
t.is(language, undefined);
t.deepEqual(
messages.map((message) => message.message),
["Not using a per-language CodeQL bundle since the feature is disabled."],
);
});
test("getPerLanguageBundleLanguage skips only the release version check for the latest nightly", async (t) => {
const nightly = { isLatestNightly: true, cliVersion: undefined };
t.is(await checkEligibility(nightly), BuiltInLanguage.java);
@@ -157,6 +181,10 @@ test("getPerLanguageBundleLanguage skips only the release version check for the
]) {
t.is(await checkEligibility({ ...nightly, ...overrides }), undefined);
}
t.is(
await checkEligibility(nightly, { features: createFeatures([]) }),
undefined,
);
t.is(
await checkEligibility(nightly, {
env: getTestEnv({ [ActionsEnvVars.RUNNER_ENVIRONMENT]: "self-hosted" }),
+14 -7
View File
@@ -9,6 +9,7 @@ import {
QuerySpec,
UserConfig,
} from "./config/db-config";
import { Feature } from "./feature-flags";
import { RepositoryPropertyName } from "./feature-flags/properties";
import { BuiltInLanguage, parseBuiltInLanguage } from "./languages";
import { BundlePlatform } from "./platform";
@@ -138,10 +139,14 @@ export interface PerLanguageBundleOptions {
}
/** Returns the eligible bundle language, or undefined for the combined bundle. */
export function getPerLanguageBundleLanguage(
action: ActionState<["Logger", "ReadOnlyEnv"]>,
export async function getPerLanguageBundleLanguage(
{
env,
features,
logger,
}: ActionState<["Logger", "ReadOnlyEnv", "FeatureFlags"]>,
options: PerLanguageBundleOptions,
): BuiltInLanguage | undefined {
): Promise<BuiltInLanguage | undefined> {
const {
rawLanguages,
otherLanguagePacksReason,
@@ -153,12 +158,14 @@ export function getPerLanguageBundleLanguage(
} = options;
const explain = (reason: string) => {
action.logger.debug(
`Not using a per-language CodeQL bundle since ${reason}.`,
);
logger.debug(`Not using a per-language CodeQL bundle since ${reason}.`);
return undefined;
};
if (!(await features.getValue(Feature.PerLanguageBundles))) {
return explain("the feature is disabled");
}
// A defined reason means the CodeQL CLI may need packs for other languages, for example because
// of the configured queries. That applies whichever languages were requested, so check it first
// to avoid suggesting that requesting a single language would be enough.
@@ -190,7 +197,7 @@ export function getPerLanguageBundleLanguage(
return explain(`we are running against ${variant}`);
}
if (!isGitHubHostedRunner(action.env)) {
if (!isGitHubHostedRunner(env)) {
// Per-language installs stay out of the toolcache; self-hosted runners should retain
// the reusable combined bundle instead.
return explain("the job is not running on a GitHub-hosted runner");
+266 -214
View File
@@ -14,6 +14,7 @@ import * as diagnostics from "./diagnostics";
import { ActionsEnvVars, Env, EnvVar, ReadOnlyEnv } from "./environment";
import { Feature } from "./feature-flags";
import { BuiltInLanguage } from "./languages";
import { getRunnerLogger } from "./logging";
import { getCacheRestoreKeyPrefix } from "./overlay/caching";
import { MIN_PER_LANGUAGE_BUNDLE_CLI_VERSION } from "./per-language-bundles";
import * as setupCodeql from "./setup-codeql";
@@ -21,7 +22,6 @@ import * as tar from "./tar";
import {
LINKED_CLI_VERSION,
LoggedMessage,
RecordingLogger,
SAMPLE_DEFAULT_CLI_VERSION,
SAMPLE_DOTCOM_API_DETAILS,
checkExpectedLogMessages,
@@ -32,6 +32,7 @@ import {
getTestEnv,
initAllState,
makeMacro,
mockApiDetails,
mockBundleDownloadApi,
setupActionsVars,
setupTests,
@@ -117,7 +118,7 @@ test.serial("convert to semver", (t) => {
try {
const parsedVersion = setupCodeql.convertToSemVer(
version,
new RecordingLogger(),
getRunnerLogger(true),
);
t.deepEqual(parsedVersion, expectedVersion);
} catch (e) {
@@ -145,7 +146,7 @@ test.serial(
GitHubVariant.DOTCOM,
false,
features,
new RecordingLogger(),
getRunnerLogger(true),
);
t.deepEqual(source, {
@@ -155,6 +156,7 @@ test.serial(
compressionMethod: "gzip",
sourceType: "download",
toolsVersion: "1.2.3",
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
} satisfies setupCodeql.CodeQLDownloadSource);
});
},
@@ -221,6 +223,7 @@ for (const {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const source = await setupCodeql.getCodeQLSource(
"linked",
SAMPLE_DEFAULT_CLI_VERSION,
@@ -231,7 +234,7 @@ for (const {
GitHubVariant.DOTCOM,
tarSupportsZstd,
features,
new RecordingLogger(),
getRunnerLogger(true),
);
t.is(source.toolsVersion, LINKED_CLI_VERSION.cliVersion);
@@ -255,6 +258,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const source = await setupCodeql.getCodeQLSource(
"latest",
SAMPLE_DEFAULT_CLI_VERSION,
@@ -298,6 +302,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const result = await setupCodeql.setupCodeQLBundle(
"linked",
SAMPLE_DOTCOM_API_DETAILS,
@@ -431,6 +436,7 @@ test.serial(
compressionMethod: "zstd",
sourceType: "download",
toolsVersion: expectedVersion,
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
} satisfies setupCodeql.CodeQLToolsSource);
const result = await setupCodeql.setupCodeQLBundle(
@@ -439,7 +445,7 @@ test.serial(
tmpDir,
GitHubVariant.DOTCOM,
SAMPLE_DEFAULT_CLI_VERSION,
undefined, // rawLanguages
["javascript"],
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
features,
@@ -479,6 +485,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir, { GITHUB_EVENT_NAME: "dynamic" });
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const source = await setupCodeql.getCodeQLSource(
undefined,
@@ -503,6 +510,7 @@ test.serial(
compressionMethod: "zstd",
sourceType: "download",
toolsVersion: expectedVersion,
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
} satisfies setupCodeql.CodeQLToolsSource);
// Afterwards, ensure that we see the expected messages in the log.
@@ -596,8 +604,8 @@ test.serial(
undefined, // rawLanguages
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
new RecordingLogger(),
createFeatures([Feature.CleanupToolcacheBundles]),
getRunnerLogger(true),
);
t.is(result.toolsVersion, "local");
@@ -624,6 +632,7 @@ for (const toolsInput of ["nightly", "nightly-latest"]) {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const source = await setupCodeql.getCodeQLSource(
toolsInput,
SAMPLE_DEFAULT_CLI_VERSION,
@@ -633,8 +642,8 @@ for (const toolsInput of ["nightly", "nightly-latest"]) {
SAMPLE_DOTCOM_API_DETAILS,
GitHubVariant.DOTCOM,
true, // tarSupportsZstd
createFeatures([]),
new RecordingLogger(),
createFeatures([Feature.PerLanguageBundles]),
getRunnerLogger(true),
);
t.deepEqual(source, {
@@ -649,6 +658,7 @@ for (const toolsInput of ["nightly", "nightly-latest"]) {
cliVersion: undefined,
compressionMethod: "zstd",
toolsVersion: "0.0.0-30260213",
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
} satisfies setupCodeql.CodeQLDownloadSource);
t.true(latestNightlyRequest.calledOnce);
});
@@ -664,16 +674,24 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
for (const { languages, otherLanguagePacksReason } of [
for (const { languages, otherLanguagePacksReason, features } of [
// The per-language feature is disabled.
{
languages: ["java"],
otherLanguagePacksReason: undefined,
features: createFeatures([]),
},
// More than one language requires a combined bundle.
{
languages: ["java", "python"],
otherLanguagePacksReason: undefined,
features: createFeatures([Feature.PerLanguageBundles]),
},
// The configured queries may need library packs for other languages.
{
languages: ["java"],
otherLanguagePacksReason: "an example reason",
features: createFeatures([Feature.PerLanguageBundles]),
},
]) {
const source = await setupCodeql.getCodeQLSource(
@@ -685,8 +703,8 @@ test.serial(
SAMPLE_DOTCOM_API_DETAILS,
GitHubVariant.DOTCOM,
true, // tarSupportsZstd
createFeatures([]),
new RecordingLogger(),
features,
getRunnerLogger(true),
);
t.is(source.sourceType, "download");
@@ -701,48 +719,58 @@ test.serial(
},
);
test.serial(
"getCodeQLSource uses the latest per-language bundle for a forced nightly",
async (t) => {
const expectedTag = "codeql-bundle-30260213";
const baseURL = `https://github.com/dsp-testing/codeql-cli-nightlies/releases/download/${expectedTag}`;
const latestNightlyRequest = stubHostedNightly(expectedTag);
for (const perLanguageBundles of [false, true]) {
test.serial(
`getCodeQLSource uses the latest ${perLanguageBundles ? "per-language" : "combined"} bundle for a forced nightly`,
async (t) => {
const expectedTag = "codeql-bundle-30260213";
const baseURL = `https://github.com/dsp-testing/codeql-cli-nightlies/releases/download/${expectedTag}`;
const latestNightlyRequest = stubHostedNightly(expectedTag);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir, { GITHUB_EVENT_NAME: "dynamic" });
const source = await setupCodeql.getCodeQLSource(
undefined, // toolsInput: the nightly is selected by ForceNightly
SAMPLE_DEFAULT_CLI_VERSION,
["java"],
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
SAMPLE_DOTCOM_API_DETAILS,
GitHubVariant.DOTCOM,
true, // tarSupportsZstd
createFeatures([Feature.ForceNightly]),
new RecordingLogger(),
);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir, { GITHUB_EVENT_NAME: "dynamic" });
const source = await setupCodeql.getCodeQLSource(
undefined, // toolsInput: the nightly is selected by ForceNightly
SAMPLE_DEFAULT_CLI_VERSION,
["java"],
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
SAMPLE_DOTCOM_API_DETAILS,
GitHubVariant.DOTCOM,
true, // tarSupportsZstd
createFeatures(
perLanguageBundles
? [Feature.ForceNightly, Feature.PerLanguageBundles]
: [Feature.ForceNightly],
),
getRunnerLogger(true),
);
t.is(source.sourceType, "download");
t.true(latestNightlyRequest.calledOnce);
if (source.sourceType === "download") {
t.deepEqual(source.bundle, {
kind: "per-language",
language: BuiltInLanguage.java,
url: `${baseURL}/codeql-bundle-java-linux64.tar.zst`,
combinedBundleURL: `${baseURL}/codeql-bundle-linux64.tar.zst`,
});
}
});
},
);
t.is(source.sourceType, "download");
t.true(latestNightlyRequest.calledOnce);
if (source.sourceType === "download") {
const combinedURL = `${baseURL}/codeql-bundle-linux64.tar.zst`;
t.deepEqual(
source.bundle,
perLanguageBundles
? {
kind: "per-language",
language: BuiltInLanguage.java,
url: `${baseURL}/codeql-bundle-java-linux64.tar.zst`,
combinedBundleURL: combinedURL,
}
: { kind: "combined", url: combinedURL },
);
}
});
},
);
}
for (const date of ["20200101", "30260213"]) {
for (const bundle of ["combined", "per-language"] as const) {
test.serial(
`getCodeQLSource preserves an explicit ${bundle} nightly URL for ${date}${
bundle === "per-language" ? " on GHES" : ""
}`,
`getCodeQLSource preserves an explicit ${bundle} nightly URL for ${date}`,
async (t) => {
const latestNightlyRequest = stubHostedNightly(
"codeql-bundle-30260213",
@@ -752,9 +780,7 @@ for (const date of ["20200101", "30260213"]) {
? "codeql-bundle-linux64.tar.zst"
: "codeql-bundle-java-linux64.tar.zst";
const url = `https://github.com/dsp-testing/codeql-cli-nightlies/releases/download/codeql-bundle-${date}/${asset}`;
const features = createFeatures([]);
const variant =
bundle === "per-language" ? GitHubVariant.GHES : GitHubVariant.DOTCOM;
const features = createFeatures([Feature.PerLanguageBundles]);
const logger = getRecordingLogger([], { logToConsole: false });
const error = new HTTPError("Not Found", 404);
const extractStub = sinon
@@ -770,7 +796,7 @@ for (const date of ["20200101", "30260213"]) {
undefined, // otherLanguagePacksReason
false,
SAMPLE_DOTCOM_API_DETAILS,
variant,
GitHubVariant.DOTCOM,
true,
features,
logger,
@@ -785,6 +811,7 @@ for (const date of ["20200101", "30260213"]) {
cliVersion: undefined,
compressionMethod: "zstd",
toolsVersion: `0.0.0-${date}`,
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
} satisfies setupCodeql.CodeQLDownloadSource);
await t.throwsAsync(
@@ -792,7 +819,7 @@ for (const date of ["20200101", "30260213"]) {
url,
SAMPLE_DOTCOM_API_DETAILS,
tmpDir,
variant,
GitHubVariant.DOTCOM,
SAMPLE_DEFAULT_CLI_VERSION,
["java"],
undefined, // otherLanguagePacksReason
@@ -832,7 +859,7 @@ test.serial(
GitHubVariant.DOTCOM,
true,
createFeatures([]),
new RecordingLogger(),
getRunnerLogger(true),
),
{
message:
@@ -929,6 +956,7 @@ const toolcacheInputFallbackMacro = makeMacro({
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
for (const [k, v] of Object.entries(environment)) {
process.env[k] = v;
@@ -1004,7 +1032,7 @@ test.serial(
t.is(
setupCodeql.tryGetTagNameFromUrl(
"https://github.com/org/codeql-bundle-testing/releases/download/codeql-bundle-v2.19.0/codeql-bundle-linux64.tar.zst",
new RecordingLogger(),
getRunnerLogger(true),
),
"codeql-bundle-v2.19.0",
);
@@ -1016,7 +1044,7 @@ test.serial(
(t) => {
sinon.stub(toolcache, "findAllVersions").returns([]);
t.is(
setupCodeql.getLatestToolcacheVersion(new RecordingLogger()),
setupCodeql.getLatestToolcacheVersion(getRunnerLogger(true)),
undefined,
);
},
@@ -1028,7 +1056,7 @@ test.serial(
const testVersions = ["2.3.1", "3.2.1", "1.2.3"];
sinon.stub(toolcache, "findAllVersions").returns(testVersions);
t.is(setupCodeql.getLatestToolcacheVersion(new RecordingLogger()), "3.2.1");
t.is(setupCodeql.getLatestToolcacheVersion(getRunnerLogger(true)), "3.2.1");
},
);
@@ -1054,7 +1082,7 @@ async function fakeOverlayBaseCacheKey(
}
test.serial(
"getCodeQLSource uses overlay-aware default version for a Code Scanning PR",
"getCodeQLSource uses overlay-aware default version when requested for a PR",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -1081,8 +1109,8 @@ test.serial(
SAMPLE_DOTCOM_API_DETAILS,
GitHubVariant.DOTCOM,
false,
createFeatures([]),
new RecordingLogger(),
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersion]),
getRunnerLogger(true),
);
t.assert(listStub.calledOnce);
@@ -1093,95 +1121,7 @@ test.serial(
);
test.serial(
"getCodeQLSource skips the Actions cache when only one default version is enabled",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
process.env[EnvVar.CODE_SCANNING_REF] = "refs/heads/feature-branch";
process.env[EnvVar.CODE_SCANNING_BASE_BRANCH] = "main";
sinon.stub(api, "getAutomationID").resolves("test/");
const listStub = sinon.stub(api, "listActionsCaches").resolves([
{
key: await fakeOverlayBaseCacheKey(
"javascript",
SAMPLE_DEFAULT_CLI_VERSION.enabledVersions[0].cliVersion,
"abc-1-1",
),
},
]);
sinon
.stub(toolcache, "find")
.withArgs(
"CodeQL",
SAMPLE_DEFAULT_CLI_VERSION.enabledVersions[0].cliVersion,
)
.returns("/path/to/codeql");
const source = await setupCodeql.getCodeQLSource(
undefined,
SAMPLE_DEFAULT_CLI_VERSION,
["javascript"],
undefined, // otherLanguagePacksReason
true,
SAMPLE_DOTCOM_API_DETAILS,
GitHubVariant.DOTCOM,
false,
createFeatures([]),
new RecordingLogger(),
);
t.assert(listStub.notCalled);
t.is(source.sourceType, "toolcache");
t.is(
source.toolsVersion,
SAMPLE_DEFAULT_CLI_VERSION.enabledVersions[0].cliVersion,
);
});
},
);
test.serial(
"getCodeQLSource skips overlay-aware default version outside pull request analysis",
async (t) => {
sinon.stub(actionsUtil, "isAnalyzingPullRequest").returns(false);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
sinon.stub(api, "getAutomationID").resolves("test/");
const listStub = sinon.stub(api, "listActionsCaches").resolves([
{
key: await fakeOverlayBaseCacheKey("javascript", "2.20.1", "abc-1-1"),
},
]);
sinon
.stub(toolcache, "find")
.withArgs("CodeQL", "2.20.2")
.returns("/path/to/codeql-2.20.2");
const source = await setupCodeql.getCodeQLSource(
undefined,
overlayMatchEnabledVersions,
["javascript"],
undefined, // otherLanguagePacksReason
true,
SAMPLE_DOTCOM_API_DETAILS,
GitHubVariant.DOTCOM,
false,
createFeatures([]),
new RecordingLogger(),
);
t.assert(listStub.notCalled);
t.is(source.sourceType, "toolcache");
t.is(source.toolsVersion, "2.20.2");
});
},
);
test.serial(
"getCodeQLSource skips overlay-aware default version when not requested for Code Scanning",
"getCodeQLSource skips overlay-aware default version when not requested",
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -1208,8 +1148,8 @@ test.serial(
SAMPLE_DOTCOM_API_DETAILS,
GitHubVariant.DOTCOM,
false,
createFeatures([]),
new RecordingLogger(),
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersion]),
getRunnerLogger(true),
);
t.assert(listStub.notCalled);
@@ -1237,6 +1177,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const source = await setupCodeql.getCodeQLSource(
undefined,
PER_LANGUAGE_CLI_VERSION,
@@ -1246,8 +1187,8 @@ test.serial(
SAMPLE_DOTCOM_API_DETAILS,
GitHubVariant.DOTCOM,
true, // tarSupportsZstd
createFeatures([]),
new RecordingLogger(),
createFeatures([Feature.PerLanguageBundles]),
getRunnerLogger(true),
);
t.is(source.sourceType, "download");
@@ -1279,16 +1220,14 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
for (const { otherLanguagePacksReason, variant } of [
// Automatic per-language selection is limited to GitHub.com.
{
otherLanguagePacksReason: undefined,
variant: GitHubVariant.GHES,
},
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
for (const { otherLanguagePacksReason, features } of [
// The per-language feature is disabled.
{ otherLanguagePacksReason: undefined, features: createFeatures([]) },
// The configured queries may need library packs for other languages.
{
otherLanguagePacksReason: "an example reason",
variant: GitHubVariant.DOTCOM,
features: createFeatures([Feature.PerLanguageBundles]),
},
]) {
const source = await setupCodeql.getCodeQLSource(
@@ -1298,10 +1237,10 @@ test.serial(
otherLanguagePacksReason,
false, // useOverlayAwareDefaultCliVersion
SAMPLE_DOTCOM_API_DETAILS,
variant,
GitHubVariant.DOTCOM,
true, // tarSupportsZstd
createFeatures([]),
new RecordingLogger(),
features,
getRunnerLogger(true),
);
t.is(source.sourceType, "download");
@@ -1351,7 +1290,14 @@ for (const fallback of [false, true]) {
request: { fetch: fetchRelease },
});
sinon.stub(api, "getApiClient").value(() => client);
const authorizationSpy = sinon.spy(api, "getAuthorizationHeaderFor");
const spyClient = new api.InstanceApiClient(SAMPLE_DOTCOM_API_DETAILS);
const authorizationSpy = sinon.spy(
spyClient,
"getAuthorizationHeaderFor",
);
sinon.stub(api, "InstanceApiClient").returns(spyClient);
const extractStub = stubDownloadAndExtract();
if (fallback) {
extractStub.onFirstCall().rejects(new HTTPError("Not Found", 404));
@@ -1361,6 +1307,7 @@ for (const fallback of [false, true]) {
setupActionsVars(tmpDir, tmpDir, {
GITHUB_ACTION_REPOSITORY: "codeql-testing/action-fork",
});
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const result = await setupCodeql.setupCodeQLBundle(
undefined,
SAMPLE_DOTCOM_API_DETAILS,
@@ -1370,8 +1317,8 @@ for (const fallback of [false, true]) {
["java"],
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
new RecordingLogger(),
createFeatures([Feature.PerLanguageBundles]),
getRunnerLogger(true),
);
t.true(fetchRelease.calledTwice);
@@ -1383,9 +1330,9 @@ for (const fallback of [false, true]) {
t.is(extractStub.firstCall.args[0], assetURL);
t.is(extractStub.lastCall.args[0], fallback ? combinedURL : assetURL);
t.is(authorizationSpy.callCount, extractStub.callCount);
t.is(authorizationSpy.firstCall.args[2], assetURL);
t.is(authorizationSpy.firstCall.args[1], assetURL);
t.is(
authorizationSpy.lastCall.args[2],
authorizationSpy.lastCall.args[1],
fallback ? combinedURL : assetURL,
);
t.is(extractStub.lastCall.args[3], "token token");
@@ -1450,7 +1397,10 @@ for (const bundle of ["per-language", "combined", "fallback"] as const) {
};
});
const addDiagnostic = sinon.stub(diagnostics, "addNoLanguageDiagnostic");
const features = createFeatures([]);
const features = createFeatures([
Feature.PerLanguageBundles,
Feature.CleanupToolcacheBundles,
]);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -1625,6 +1575,7 @@ for (const error of [
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
await t.throwsAsync(
setupCodeql.setupCodeQLBundle(
undefined,
@@ -1635,8 +1586,8 @@ for (const error of [
["java"],
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
new RecordingLogger(),
createFeatures([Feature.PerLanguageBundles]),
getRunnerLogger(true),
),
{ is: error },
);
@@ -1672,7 +1623,7 @@ test.serial(
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
new RecordingLogger(),
getRunnerLogger(true),
),
{ is: error },
);
@@ -1683,7 +1634,7 @@ test.serial(
);
test.serial(
"getEnabledVersionsWithOverlayBaseDatabases returns enabled versions present in cache, sorted desc",
"getEnabledVersionsWithOverlayBaseDatabases returns flag-enabled versions present in cache, sorted desc",
async (t) => {
sinon.stub(api, "getAutomationID").resolves("test/");
sinon.stub(api, "listActionsCaches").resolves([
@@ -1701,24 +1652,21 @@ test.serial(
},
]);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const result =
await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
new RecordingLogger(),
);
t.deepEqual(result, [
{ cliVersion: "2.20.1", tagName: "codeql-bundle-v2.20.1" },
{ cliVersion: "2.20.0", tagName: "codeql-bundle-v2.20.0" },
]);
});
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersion]),
getRunnerLogger(true),
);
t.deepEqual(result, [
{ cliVersion: "2.20.1", tagName: "codeql-bundle-v2.20.1" },
{ cliVersion: "2.20.0", tagName: "codeql-bundle-v2.20.0" },
]);
},
);
test.serial(
"getEnabledVersionsWithOverlayBaseDatabases returns empty when no cached version is enabled",
"getEnabledVersionsWithOverlayBaseDatabases returns empty when no cached version is flag-enabled",
async (t) => {
sinon.stub(api, "getAutomationID").resolves("test/");
sinon.stub(api, "listActionsCaches").resolves([
@@ -1730,7 +1678,8 @@ test.serial(
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
new RecordingLogger(),
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersion]),
getRunnerLogger(true),
);
t.deepEqual(result, []);
},
@@ -1746,7 +1695,8 @@ const noLanguagesMacro = makeMacro({
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
rawLanguages,
new RecordingLogger(),
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersion]),
getRunnerLogger(true),
);
t.deepEqual(result, []);
t.assert(
@@ -1770,7 +1720,8 @@ test.serial(
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
new RecordingLogger(),
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersion]),
getRunnerLogger(true),
);
t.deepEqual(result, []);
},
@@ -1789,7 +1740,8 @@ test.serial(
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
new RecordingLogger(),
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersion]),
getRunnerLogger(true),
);
t.deepEqual(result, [
{ cliVersion: "2.20.2", tagName: "codeql-bundle-v2.20.2" },
@@ -1797,6 +1749,78 @@ test.serial(
},
);
test.serial(
"getEnabledVersionsWithOverlayBaseDatabases does not list caches when both gates are off",
async (t) => {
const listStub = sinon.stub(api, "listActionsCaches").resolves([]);
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
createFeatures([]),
getRunnerLogger(true),
);
t.deepEqual(result, []);
t.assert(
listStub.notCalled,
"Should not list Actions caches when both gating feature flags are off.",
);
},
);
test.serial(
"getEnabledVersionsWithOverlayBaseDatabases dry-run returns empty but lists caches",
async (t) => {
sinon.stub(api, "getAutomationID").resolves("test/");
const listStub = sinon.stub(api, "listActionsCaches").resolves([
{
key: await fakeOverlayBaseCacheKey("javascript", "2.20.1", "abc-1-1"),
},
]);
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
createFeatures([Feature.OverlayAnalysisMatchCodeqlVersionDryRun]),
getRunnerLogger(true),
);
t.deepEqual(
result,
[],
"Dry-run should return an empty list so the caller falls back.",
);
t.assert(
listStub.calledOnce,
"Dry-run should still list Actions caches to populate the diagnostic.",
);
},
);
test.serial(
"getEnabledVersionsWithOverlayBaseDatabases match flag wins over dry-run",
async (t) => {
sinon.stub(api, "getAutomationID").resolves("test/");
sinon.stub(api, "listActionsCaches").resolves([
{
key: await fakeOverlayBaseCacheKey("javascript", "2.20.1", "abc-1-1"),
},
]);
const result = await setupCodeql.getEnabledVersionsWithOverlayBaseDatabases(
overlayMatchEnabledVersions,
["javascript"],
createFeatures([
Feature.OverlayAnalysisMatchCodeqlVersion,
Feature.OverlayAnalysisMatchCodeqlVersionDryRun,
]),
getRunnerLogger(true),
);
t.deepEqual(result, [
{ cliVersion: "2.20.1", tagName: "codeql-bundle-v2.20.1" },
]);
},
);
/** The CLI version that the toolcache cleanup tests download. */
const CLEANUP_CLI_VERSION = "2.21.0";
/** The bundle version that the toolcache cleanup tests download. */
@@ -1825,15 +1849,11 @@ function createToolcacheEntry(
*/
async function runDownloadCodeQL(
toolcacheRoot: string,
features: Feature[],
bundleVersion: string | undefined,
{
env = getTestEnv({
[ActionsEnvVars.RUNNER_ENVIRONMENT]: "github-hosted",
[ActionsEnvVars.RUNNER_TOOL_CACHE]: toolcacheRoot,
}),
}: {
env?: Env;
} = {},
env = getTestEnv({
[ActionsEnvVars.RUNNER_TOOL_CACHE]: toolcacheRoot,
}),
): Promise<{
codeqlFolder: string;
cleanupDiagnostic: toolsDownload.ToolcacheCleanupResult | undefined;
@@ -1844,6 +1864,7 @@ async function runDownloadCodeQL(
const { codeqlFolder } = await setupCodeql.downloadCodeQLBundle(
initAllState({
env,
features: createFeatures(features),
}),
{
bundle: {
@@ -1855,8 +1876,8 @@ async function runDownloadCodeQL(
cliVersion: CLEANUP_CLI_VERSION,
sourceType: "download",
toolsVersion: CLEANUP_CLI_VERSION,
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
},
SAMPLE_DOTCOM_API_DETAILS,
undefined, // tarVersion
toolcacheRoot, // tempDir
);
@@ -1881,9 +1902,11 @@ async function runDownloadCodeQL(
async function testToolcacheCleanup(
t: ExecutionContext<unknown>,
{
features,
runnerEnvironment,
setUp,
}: {
features: Feature[];
runnerEnvironment: string | undefined;
setUp?: (env: Env) => void;
},
@@ -1895,8 +1918,12 @@ async function testToolcacheCleanup(
) {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
if (runnerEnvironment === undefined) {
delete process.env[ActionsEnvVars.RUNNER_ENVIRONMENT];
} else {
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = runnerEnvironment;
}
const env = getTestEnv({
[ActionsEnvVars.RUNNER_ENVIRONMENT]: runnerEnvironment,
[ActionsEnvVars.RUNNER_TOOL_CACHE]: tmpDir,
});
setUp?.(env);
@@ -1916,8 +1943,9 @@ async function testToolcacheCleanup(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
features,
CLEANUP_BUNDLE_VERSION,
{ env },
env,
);
t.true(
@@ -1929,12 +1957,28 @@ async function testToolcacheCleanup(
});
}
test.serial(
"downloadCodeQLBundle does not clean up the toolcache when the feature flag is disabled",
async (t) => {
await testToolcacheCleanup(
t,
{ features: [], runnerEnvironment: "github-hosted" },
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"downloadCodeQLBundle does not clean up the toolcache when the runner is not GitHub-hosted",
async (t) => {
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "self-hosted",
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
@@ -1954,6 +1998,7 @@ test.serial(
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: undefined,
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
@@ -1966,11 +2011,12 @@ test.serial(
);
test.serial(
"downloadCodeQLBundle deletes other CodeQL bundles from the toolcache on a GitHub-hosted runner",
"downloadCodeQLBundle deletes other CodeQL bundles from the toolcache when enabled on a GitHub-hosted runner",
async (t) => {
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "github-hosted",
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
@@ -2004,6 +2050,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2033,6 +2080,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2095,7 +2143,7 @@ test.serial(
env: new ReadOnlyEnv({
[ActionsEnvVars.RUNNER_TOOL_CACHE]: injectedRoot,
}),
logger: new RecordingLogger(),
logger: getRunnerLogger(true),
});
t.deepEqual(result, {
@@ -2164,6 +2212,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
toolcacheRoot,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2186,6 +2235,7 @@ test.serial(
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "github-hosted",
setUp: (env) => {
env.set(EnvVar.HAS_SET_UP_CODEQL, "true");
@@ -2215,13 +2265,9 @@ test.serial(
);
const { codeqlFolder, cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
{
env: getTestEnv({
[ActionsEnvVars.RUNNER_ENVIRONMENT]: "github-hosted",
[EnvVar.HAS_SET_UP_CODEQL]: "true",
}),
},
getTestEnv({ [EnvVar.HAS_SET_UP_CODEQL]: "true" }),
);
t.true(fs.existsSync(staleDirectory));
@@ -2236,6 +2282,7 @@ test.serial(
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
delete process.env[EnvVar.HAS_SET_UP_CODEQL];
stubDownloadAndExtract();
@@ -2250,7 +2297,7 @@ test.serial(
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
new RecordingLogger(),
getRunnerLogger(true),
);
t.is(
@@ -2271,6 +2318,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
delete process.env[EnvVar.HAS_SET_UP_CODEQL];
const staleDirectory = createToolcacheEntry(
@@ -2289,8 +2337,8 @@ test.serial(
undefined, // rawLanguages
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
new RecordingLogger(),
createFeatures([Feature.CleanupToolcacheBundles]),
getRunnerLogger(true),
),
{ is: error },
);
@@ -2324,6 +2372,7 @@ test.serial(
const { codeqlFolder, cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
undefined, // bundleVersion
);
@@ -2357,6 +2406,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2392,6 +2442,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2405,7 +2456,7 @@ test.serial(
"isToolcacheOnWorkspaceFilesystem assumes a different filesystem when it cannot tell",
async (t) => {
await withTmpDir(async (tmpDir) => {
const logger = new RecordingLogger();
const logger = getRunnerLogger(true);
setupActionsVars(tmpDir, tmpDir);
t.true(toolsDownload.isToolcacheOnWorkspaceFilesystem(logger));
@@ -2441,6 +2492,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
toolcacheRoot,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
+85 -43
View File
@@ -183,16 +183,29 @@ async function findOverridingToolsInCache(
/**
* Returns the sorted set of enabled versions that have cached overlay-base databases for the
* given languages.
* given languages, or an empty list if neither the `OverlayAnalysisMatchCodeqlVersion` nor the
* `OverlayAnalysisMatchCodeqlVersionDryRun` feature flag is enabled. When only the dry-run flag
* is enabled, this performs the lookup and emits a telemetry diagnostic with the version that
* would have been chosen, but still returns an empty list so the caller falls back.
*/
export async function getEnabledVersionsWithOverlayBaseDatabases(
defaultCliVersion: CodeQLDefaultVersionInfo,
rawLanguages: string[] | undefined,
features: FeatureEnablement,
logger: Logger,
): Promise<CodeQLVersionInfo[]> {
if (rawLanguages === undefined || rawLanguages.length === 0) {
return [];
}
const isEnabled = await features.getValue(
Feature.OverlayAnalysisMatchCodeqlVersion,
);
const isDryRun =
!isEnabled &&
(await features.getValue(Feature.OverlayAnalysisMatchCodeqlVersionDryRun));
if (!isEnabled && !isDryRun) {
return [];
}
let cachedVersions: string[] | undefined;
try {
@@ -213,35 +226,66 @@ export async function getEnabledVersionsWithOverlayBaseDatabases(
}
const cachedVersionsSet = new Set(cachedVersions);
return defaultCliVersion.enabledVersions.filter((v) =>
const overlayVersions = defaultCliVersion.enabledVersions.filter((v) =>
cachedVersionsSet.has(v.cliVersion),
);
if (overlayVersions.length === 0) {
return [];
}
const isCachedVersionDifferent =
overlayVersions[0].cliVersion !==
defaultCliVersion.enabledVersions[0].cliVersion;
if (isCachedVersionDifferent) {
addNoLanguageDiagnostic(
undefined,
makeTelemetryDiagnostic(
"codeql-action/overlay-aware-default-codeql-version",
"Overlay-aware default CodeQL version selection",
{
cachedVersions,
enabledVersions: defaultCliVersion.enabledVersions.map(
(v) => v.cliVersion,
),
isDryRun,
overlayAwareVersion: overlayVersions[0].cliVersion,
},
),
);
}
if (isDryRun) {
logger.debug(
`Overlay-aware default CodeQL version selection is running in dry-run mode. Would have used version ${overlayVersions[0].cliVersion}.`,
);
return [];
}
return overlayVersions;
}
/**
* Resolves the newest enabled default CLI version that has a cached overlay-base database for the
* relevant languages, if this is a Code Scanning analysis of a pull request, multiple default
* versions are enabled, and such a version exists. Otherwise, falls back to the newest enabled
* default CLI version.
* relevant languages, if running a Code Scanning analysis for a pull request and one exists.
* Otherwise, falls back to the newest enabled default CLI version.
*/
async function resolveDefaultCliVersion(
defaultCliVersion: CodeQLDefaultVersionInfo,
rawLanguages: string[] | undefined,
useOverlayAwareDefaultCliVersion: boolean,
features: FeatureEnablement,
logger: Logger,
): Promise<CodeQLVersionInfo> {
if (
!useOverlayAwareDefaultCliVersion ||
// Listing the Actions cache can't change which version we use if only one version is enabled.
defaultCliVersion.enabledVersions.length === 1 ||
!isAnalyzingPullRequest()
) {
if (!useOverlayAwareDefaultCliVersion || !isAnalyzingPullRequest()) {
return defaultCliVersion.enabledVersions[0];
}
const overlayVersions = await getEnabledVersionsWithOverlayBaseDatabases(
defaultCliVersion,
rawLanguages,
features,
logger,
);
if (overlayVersions.length > 0) {
@@ -364,7 +408,7 @@ export async function getCodeQLSource(
);
}
bundle = await getLatestNightlyBundle(
{ env: getEnv(), logger },
{ env: getEnv(), features, logger },
rawLanguages,
otherLanguagePacksReason,
variant,
@@ -438,6 +482,7 @@ export async function getCodeQLSource(
defaultCliVersion,
rawLanguages,
useOverlayAwareDefaultCliVersion,
features,
logger,
);
cliVersion = version.cliVersion;
@@ -460,6 +505,7 @@ export async function getCodeQLSource(
defaultCliVersion,
rawLanguages,
useOverlayAwareDefaultCliVersion,
features,
logger,
);
cliVersion = version.cliVersion;
@@ -605,14 +651,17 @@ export async function getCodeQLSource(
const action = { env: getEnv(), logger };
const platform = getBundlePlatform();
const perLanguageBundleLanguage = getPerLanguageBundleLanguage(action, {
rawLanguages,
otherLanguagePacksReason,
cliVersion,
compressionMethod,
platform,
variant,
});
const perLanguageBundleLanguage = await getPerLanguageBundleLanguage(
{ ...action, features },
{
rawLanguages,
otherLanguagePacksReason,
cliVersion,
compressionMethod,
platform,
variant,
},
);
// Resolves the combined or per-language bundle URL for the requested release.
const resolveBundleURL = (language?: BuiltInLanguage) =>
@@ -669,6 +718,7 @@ export async function getCodeQLSource(
compressionMethod,
sourceType: "download",
toolsVersion: resolvedVersion ?? "unknown",
apiDetails,
};
}
@@ -697,7 +747,6 @@ async function tryGetFallbackToolcacheVersion(
// be able to stub this function and have other functions in this file use that stub.
export const downloadCodeQL = async function (
source: CodeQLDownloadSource,
apiDetails: api.GitHubApiDetails,
tarVersion: tar.TarVersion | undefined,
tempDir: string,
logger: Logger,
@@ -718,11 +767,8 @@ export const downloadCodeQL = async function (
if (searchParams.has("token")) {
logger.debug("CodeQL tools URL contains an authorization token.");
} else {
authorization = api.getAuthorizationHeaderFor(
logger,
apiDetails,
codeqlURL,
);
const client = new api.InstanceApiClient(source.apiDetails);
authorization = client.getAuthorizationHeaderFor(logger, codeqlURL);
}
const toolcacheDestination = getToolcacheDestination({ logger }, source);
@@ -790,7 +836,7 @@ function getToolcacheDestination(
}
/**
* Reclaims disk space by deleting the CodeQL tools from the toolcache on GitHub-hosted runners.
* Reclaims disk space by deleting the CodeQL tools from the toolcache, if enabled.
*
* On GitHub-hosted runners the toolcache shares a filesystem with the workspace, so tools left in
* the toolcache take up space that the analysis could use instead. This holds wherever we extract
@@ -798,8 +844,9 @@ function getToolcacheDestination(
*/
async function tryDeleteToolcacheBundles({
env,
features,
logger,
}: ActionState<["Logger", "ReadOnlyEnv"]>): Promise<void> {
}: ActionState<["Logger", "ReadOnlyEnv", "FeatureFlags"]>): Promise<void> {
// A step that has already set up CodeQL may hand out a path into the toolcache that a later step
// runs, so only the first step to set it up can know that nothing else relies on the toolcache.
if (env.getOptional(EnvVar.HAS_SET_UP_CODEQL) !== undefined) {
@@ -810,7 +857,11 @@ async function tryDeleteToolcacheBundles({
return;
}
if (!isGitHubHostedRunner(env) || !isToolcacheOnWorkspaceFilesystem(logger)) {
if (
!isGitHubHostedRunner() ||
!isToolcacheOnWorkspaceFilesystem(logger) ||
!(await features.getValue(Feature.CleanupToolcacheBundles))
) {
return;
}
@@ -929,9 +980,8 @@ export async function setupCodeQLBundle(
break;
case "download": {
const result = await downloadCodeQLBundle(
{ env: getEnv(), logger },
{ env: getEnv(), features, logger },
source,
apiDetails,
zstdAvailability.version,
tempDir,
);
@@ -965,9 +1015,8 @@ export async function setupCodeQLBundle(
* @returns The extraction directory and download timings.
*/
export async function downloadCodeQLBundle(
action: ActionState<["Logger", "ReadOnlyEnv"]>,
action: ActionState<["Logger", "ReadOnlyEnv", "FeatureFlags"]>,
source: CodeQLDownloadSource,
apiDetails: api.GitHubApiDetails,
tarVersion: tar.TarVersion | undefined,
tempDir: string,
): Promise<{
@@ -981,13 +1030,7 @@ export async function downloadCodeQLBundle(
const startTime = performance.now();
try {
return await downloadCodeQL(
source,
apiDetails,
tarVersion,
tempDir,
logger,
);
return await downloadCodeQL(source, tarVersion, tempDir, logger);
} catch (e) {
if (
bundle.kind !== "per-language" ||
@@ -1003,7 +1046,6 @@ export async function downloadCodeQLBundle(
...source,
bundle: { kind: "combined", url: bundle.combinedBundleURL },
},
apiDetails,
tarVersion,
tempDir,
logger,
@@ -1040,7 +1082,7 @@ function getTempExtractionDir(tempDir: string) {
* Records the combined bundle URL from that release for use if the selected asset is missing.
*/
async function getLatestNightlyBundle(
action: ActionState<["Logger", "ReadOnlyEnv"]>,
action: ActionState<["Logger", "ReadOnlyEnv", "FeatureFlags"]>,
rawLanguages: string[] | undefined,
otherLanguagePacksReason: string | undefined,
variant: util.GitHubVariant,
@@ -1056,7 +1098,7 @@ async function getLatestNightlyBundle(
: "gzip";
const platform = getBundlePlatform();
const language = getPerLanguageBundleLanguage(action, {
const language = await getPerLanguageBundleLanguage(action, {
rawLanguages,
otherLanguagePacksReason,
cliVersion: undefined,
+26 -25
View File
@@ -8,7 +8,6 @@ import {
parseRepositoryNwo,
type RepositoryNwoStr,
} from "../repository";
import { type InstanceURL } from "../util";
/** The NWO of the standard CodeQL Action repository. */
export const CODEQL_DEFAULT_ACTION_REPOSITORY = "github/codeql-action";
@@ -48,20 +47,14 @@ export function getCodeQLActionRepository(
}
/**A download source is a pair of strings. */
export type DownloadSource = [InstanceURL, RepositoryNwoStr];
/** GitHub.com, and the canonical Action. */
export const DEFAULT_DOWNLOAD_SOURCE: DownloadSource = [
GITHUB_DOTCOM_URL,
CODEQL_DEFAULT_ACTION_REPOSITORY,
];
export type DownloadSource = [api.GitHubApiDetails, RepositoryNwoStr];
/** Decides if the two provided download sources are the same. */
const isSameDownloadSource = (
[srcUrl, srcRepo]: DownloadSource,
[otherUrl, otherRepo]: DownloadSource,
[srcInstance, srcRepo]: DownloadSource,
[otherInstance, otherRepo]: DownloadSource,
) => {
return srcUrl === otherUrl && srcRepo === otherRepo;
return srcInstance.url === otherInstance.url && srcRepo === otherRepo;
};
/** A download URL is represented as a string. */
@@ -104,13 +97,21 @@ export async function getCodeQLAssetDownloadURL(
): Promise<DownloadURL> {
const codeQLActionRepository = getCodeQLActionRepository(action);
// We construct the default GitHub.com source here, because `getDotcomApiDetails`
// may retrieve the `token` input if we are currently running on GitHub.com and
// we want to avoid doing that statically.
const defaultDownloadSource: DownloadSource = [
api.getDotcomApiDetails(action.env),
CODEQL_DEFAULT_ACTION_REPOSITORY,
];
const potentialDownloadSources: DownloadSource[] = [
// This GitHub instance, and this Action.
[apiDetails.url, codeQLActionRepository],
[apiDetails, codeQLActionRepository],
// This GitHub instance, and the canonical Action.
[apiDetails.url, CODEQL_DEFAULT_ACTION_REPOSITORY],
[apiDetails, CODEQL_DEFAULT_ACTION_REPOSITORY],
// GitHub.com, and the canonical Action.
DEFAULT_DOWNLOAD_SOURCE,
defaultDownloadSource,
];
// We now filter out any duplicates.
@@ -123,34 +124,34 @@ export async function getCodeQLAssetDownloadURL(
},
);
for (const [instanceURL, repository] of uniqueDownloadSources) {
for (const [instance, repository] of uniqueDownloadSources) {
// If we've reached the final case, short-circuit the API check since we know the asset exists and is public.
if (
isSameDownloadSource(DEFAULT_DOWNLOAD_SOURCE, [instanceURL, repository])
) {
if (isSameDownloadSource(defaultDownloadSource, [instance, repository])) {
break;
}
const { owner: repositoryOwner, repo: repositoryName } =
parseRepositoryNwo(repository);
try {
const release = await api.getApiClient().rest.repos.getReleaseByTag({
owner: repositoryOwner,
repo: repositoryName,
tag: tagName,
});
const release = await api
.getApiClient(action.env, instance)
.rest.repos.getReleaseByTag({
owner: repositoryOwner,
repo: repositoryName,
tag: tagName,
});
for (const asset of release.data.assets) {
if (asset.name === assetName) {
action.logger.info(
`Found ${assetKind} ${assetName} in ${repository} on ${instanceURL} with URL ${asset.url}.`,
`Found ${assetKind} ${assetName} in ${repository} on ${instance.apiURL} with URL ${asset.url}.`,
);
return asset.url;
}
}
} catch (e) {
action.logger.info(
`Looked for ${assetKind} ${assetName} in ${repository} on ${instanceURL} but got error ${e}.`,
`Looked for ${assetKind} ${assetName} in ${repository} on ${instance.apiURL} but got error ${e}.`,
);
}
}
+71 -42
View File
@@ -8,12 +8,12 @@ import { ActionState } from "./action-common";
import * as actionsUtil from "./actions-util";
import * as apiClient from "./api-client";
import * as defaults from "./defaults.json";
import { ActionsEnvVars } from "./environment";
import { ActionsEnvVars, type Env } from "./environment";
import { setUpFeatureFlagTests } from "./feature-flags/testing-util";
import { UnvalidatedObject, validateSchema } from "./json";
import { makeFromSchema } from "./json/testing-util";
import { BuiltInLanguage } from "./languages";
import { Logger } from "./logging";
import { getRunnerLogger, Logger } from "./logging";
import { BundlePlatform, getBundlePlatform } from "./platform";
import * as repository from "./setup/repository";
import * as startProxyExports from "./start-proxy";
@@ -25,6 +25,7 @@ import {
initAllState,
makeMacro,
makeTestToken,
mockApiDetails,
RecordingLogger,
SAMPLE_DOTCOM_API_DETAILS,
setupTests,
@@ -67,7 +68,7 @@ test("getCredentials prefers registriesCredentials over registrySecrets", async
]);
const credentials = startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
registrySecrets,
registryCredentials,
undefined,
@@ -84,7 +85,7 @@ test("getCredentials throws an error when configurations are not an array", asyn
t.throws(
() =>
startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
registryCredentials,
undefined,
@@ -103,7 +104,7 @@ test("getCredentials throws error when credential is not an object", async (t) =
t.throws(
() =>
startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
testCredential,
undefined,
@@ -124,7 +125,7 @@ test("getCredentials throws error when credential is missing type", async (t) =>
t.throws(
() =>
startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
testCredential,
undefined,
@@ -147,7 +148,7 @@ test("getCredentials throws error when credential missing host and url", async (
t.throws(
() =>
startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
testCredential,
undefined,
@@ -161,7 +162,7 @@ test("getCredentials throws error when credential missing host and url", async (
test("getCredentials filters by language when specified", async (t) => {
const credentials = startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
toEncodedJSON(mixedCredentials),
BuiltInLanguage.java,
@@ -172,7 +173,7 @@ test("getCredentials filters by language when specified", async (t) => {
test("getCredentials returns all for a language when specified", async (t) => {
const credentials = startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
toEncodedJSON([...mixedCredentials, gitSourceCredential]),
BuiltInLanguage.go,
@@ -192,7 +193,7 @@ test("getCredentials returns all goproxy_servers for Go when specified", async (
];
const credentials = startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
toEncodedJSON(multipleGoproxyServers),
BuiltInLanguage.go,
@@ -221,7 +222,7 @@ test("getCredentials returns all maven_repositories for Java when specified", as
];
const credentials = startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
toEncodedJSON(multipleMavenRepositories),
BuiltInLanguage.java,
@@ -239,7 +240,7 @@ test("getCredentials returns all credentials when no language specified", async
const credentialsInput = toEncodedJSON(mixedCredentials);
const credentials = startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
credentialsInput,
undefined,
@@ -269,7 +270,7 @@ test("getCredentials throws an error when non-printable characters are used", as
t.throws(
() =>
startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
credentialsInput,
undefined,
@@ -300,7 +301,7 @@ for (const oidcSchemaInfo of startProxyExports.oidcSchemas) {
t.throws(
() =>
startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
credentialsInput,
undefined,
@@ -324,7 +325,7 @@ test("getCredentials accepts OIDC configurations", (t) => {
);
const credentials = startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
toEncodedJSON(oidcConfigurations),
BuiltInLanguage.csharp,
@@ -514,7 +515,7 @@ test("getCredentials validates 'replaces-base' correctly", async (t) => {
]);
const credentials = startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
credentialsInput,
BuiltInLanguage.java,
@@ -533,7 +534,7 @@ test("getCredentials validates 'replaces-base' correctly", async (t) => {
};
t.throws(() =>
startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
toEncodedJSON([{ ...baseInvalid, "replaces-base": null }]),
BuiltInLanguage.java,
@@ -541,7 +542,7 @@ test("getCredentials validates 'replaces-base' correctly", async (t) => {
);
t.throws(() =>
startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
toEncodedJSON([{ ...baseInvalid, "replaces-base": 123 }]),
BuiltInLanguage.java,
@@ -549,7 +550,7 @@ test("getCredentials validates 'replaces-base' correctly", async (t) => {
);
t.throws(() =>
startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
toEncodedJSON([{ ...baseInvalid, "replaces-base": "true" }]),
BuiltInLanguage.java,
@@ -565,7 +566,7 @@ test("getCredentials returns only ALWAYS_ENABLED_REGISTRY_TYPE credentials for A
]);
const credentials = startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
credentialsInput,
BuiltInLanguage.actions,
@@ -597,7 +598,7 @@ test("getCredentials always returns ALWAYS_ENABLED_REGISTRY_TYPE credentials for
// Test all languages.
for (const language of Object.values(BuiltInLanguage)) {
const credentials = startProxyExports.getCredentials(
new RecordingLogger(),
getRunnerLogger(true),
undefined,
credentialsInput,
language,
@@ -613,18 +614,23 @@ test("getProxyPackage - includes platform in name", (t) => {
}
});
function mockGetApiClient(endpoints: any) {
return (
sinon
.stub(apiClient, "getApiClient")
// eslint-disable-next-line @typescript-eslint/no-unsafe-argument
.returns({ rest: endpoints } as any)
);
function mockGetApiClient(env: Env, endpoints: any) {
const getRequiredInput = mockApiDetails(SAMPLE_DOTCOM_API_DETAILS, env);
const getApiClient = sinon
.stub(apiClient, "getApiClient")
// eslint-disable-next-line @typescript-eslint/no-unsafe-argument
.returns({ rest: endpoints } as any);
return { getRequiredInput, getApiClient };
}
type ReleaseAssets = Array<{ name: string; url?: string }>;
function mockGetReleaseByTag(expectedTag?: string, assets?: ReleaseAssets) {
function mockGetReleaseByTag(
env: Env,
expectedTag?: string,
assets?: ReleaseAssets,
) {
const getReleaseByTag =
assets === undefined
? sinon.stub().rejects()
@@ -645,7 +651,7 @@ function mockGetReleaseByTag(expectedTag?: string, assets?: ReleaseAssets) {
return {
getReleaseByTag,
getApiClient: mockGetApiClient({ repos: { getReleaseByTag } }),
...mockGetApiClient(env, { repos: { getReleaseByTag } }),
};
}
@@ -671,7 +677,6 @@ test.serial(
"getDownloadUrl logs unknown platforms/arch and defaults to linux64",
async (t) => {
const logger = new RecordingLogger();
mockGetReleaseByTag();
await withTmpDir(async (tempDir) => {
const features = mockOfflineFeatures(tempDir, logger);
@@ -681,6 +686,7 @@ test.serial(
logger,
features,
});
mockGetReleaseByTag(state.env);
// Stub this to always return `true` so we get the default repo for the tests.
sinon.stub(actionsUtil, "isRunningLocalAction").returns(true);
@@ -710,11 +716,11 @@ test.serial(
"getDownloadUrl returns default bundle URL for linked version",
async (t) => {
const logger = new RecordingLogger();
mockGetReleaseByTag();
await withTmpDir(async (tempDir) => {
const features = mockOfflineFeatures(tempDir, logger);
const state = initAllState({ logger, features });
mockGetReleaseByTag(state.env);
// Stub this to always return `true` so we get the default repo for the tests.
sinon.stub(actionsUtil, "isRunningLocalAction").returns(true);
@@ -750,7 +756,11 @@ test.serial(
sinon.stub(actionsUtil, "isRunningLocalAction").returns(true);
for (const assets of testAssets) {
const stubs = mockGetReleaseByTag(defaults.bundleVersion, assets);
const stubs = mockGetReleaseByTag(
state.env,
defaults.bundleVersion,
assets,
);
const info = await startProxyExports.getDownloadUrl(
state,
SAMPLE_DOTCOM_API_DETAILS,
@@ -768,6 +778,7 @@ test.serial(
// mockGetReleaseByTag stubs the global getApiClient; since we are calling
// that in a loop, we need to manually restore it here
stubs.getApiClient.restore();
stubs.getRequiredInput.restore();
}
});
},
@@ -783,7 +794,7 @@ test.serial("getDownloadUrl returns matching release asset", async (t) => {
url: "url-we-want",
},
];
const stubs = mockGetReleaseByTag(defaults.bundleVersion, assets);
const stubs = mockGetReleaseByTag(state.env, defaults.bundleVersion, assets);
await withTmpDir(async (tempDir) => {
state.env.set(ActionsEnvVars.RUNNER_TEMP, tempDir);
@@ -925,7 +936,8 @@ test.serial(
"getProxyBinaryPath - returns path from tool cache if available",
async (t) => {
const logger = new RecordingLogger();
mockGetReleaseByTag();
sinon.stub(apiClient, "getApiDetails").returns(SAMPLE_DOTCOM_API_DETAILS);
await withTmpDir(async (tempDir) => {
const toolcachePath = "/path/to/proxy/dir";
@@ -935,8 +947,11 @@ test.serial(
sinon.stub(actionsUtil, "isRunningLocalAction").returns(true);
const features = mockOfflineFeatures(tempDir, logger);
const state = initAllState({ logger, features });
mockGetReleaseByTag(state.env);
const result = await startProxyExports.getProxyBinaryPath(
initAllState({ logger, features }),
state,
SAMPLE_DOTCOM_API_DETAILS,
);
@@ -956,7 +971,7 @@ test.serial(
const logger = new RecordingLogger();
const state = initAllState({ logger });
const downloadUrl = "url-we-want";
const stubs = mockGetReleaseByTag(defaults.bundleVersion, [
const stubs = mockGetReleaseByTag(state.env, defaults.bundleVersion, [
{
name: startProxyExports.getProxyPackage(getTestPlatform(state)),
url: downloadUrl,
@@ -965,9 +980,16 @@ test.serial(
const toolcachePath = "/path/to/proxy/dir";
const find = sinon.stub(toolcache, "find").returns("");
const apiDetails = {
auth: "",
url: "",
apiURL: "",
} satisfies apiClient.GitHubApiDetails;
const client = new apiClient.InstanceApiClient(apiDetails);
const getAuthorizationHeaderFor = sinon
.stub(apiClient, "getAuthorizationHeaderFor")
.stub(client, "getAuthorizationHeaderFor")
.returns(undefined);
sinon.stub(apiClient, "InstanceApiClient").returns(client);
const archivePath = "/path/to/archive";
const downloadTool = sinon
.stub(toolcache, "downloadTool")
@@ -1007,7 +1029,7 @@ test.serial(
t.is(result.version, defaults.cliVersion);
checkExpectedLogMessages(t, logger.messages, [
`Found private registry proxy ${startProxyExports.getProxyPackage(getTestPlatform(state))} in test/repo on https://github.com with URL ${downloadUrl}`,
`Found private registry proxy ${startProxyExports.getProxyPackage(getTestPlatform(state))} in test/repo on https://api.github.com with URL ${downloadUrl}`,
]);
},
);
@@ -1039,14 +1061,21 @@ test.serial(
headers: {},
url: "GET /repos/:owner/:repo/releases/tags/:tag",
});
mockGetApiClient({ repos: { getReleaseByTag } });
mockGetApiClient(state.env, { repos: { getReleaseByTag } });
await withTmpDir(async (tempDir) => {
const toolcachePath = "/path/to/proxy/dir";
const find = sinon.stub(toolcache, "find").returns("");
const apiDetails = {
auth: "",
url: "",
apiURL: "",
} satisfies apiClient.GitHubApiDetails;
const client = new apiClient.InstanceApiClient(apiDetails);
const getAuthorizationHeaderFor = sinon
.stub(apiClient, "getAuthorizationHeaderFor")
.stub(client, "getAuthorizationHeaderFor")
.returns(undefined);
sinon.stub(apiClient, "InstanceApiClient").returns(client);
const archivePath = "/path/to/archive";
const downloadTool = sinon
.stub(toolcache, "downloadTool")
@@ -1107,7 +1136,7 @@ test.serial(
});
checkExpectedLogMessages(t, logger.messages, [
`Found private registry proxy ${startProxyExports.getProxyPackage(getTestPlatform(state))} in test/repo on https://github.com with URL url-we-want.`,
`Found private registry proxy ${startProxyExports.getProxyPackage(getTestPlatform(state))} in test/repo on https://api.github.com with URL url-we-want.`,
]);
},
);
+4 -4
View File
@@ -4,9 +4,9 @@ import * as toolcache from "@actions/tool-cache";
import { ActionState } from "./action-common";
import {
getAuthorizationHeaderFor,
getGitHubVersion,
GitHubApiDetails,
InstanceApiClient,
type GitHubApiDetails,
} from "./api-client";
import * as artifactScanner from "./artifact-scanner";
import * as defaults from "./defaults.json";
@@ -423,6 +423,7 @@ export async function getProxyBinaryPath(
logger.startGroup("Setting up authentication proxy");
try {
const client = new InstanceApiClient(apiDetails);
const proxyFileName = getProxyFilename();
const proxyInfo = await getDownloadUrl(action, apiDetails);
@@ -437,9 +438,8 @@ export async function getProxyBinaryPath(
`Authentication proxy is not in the toolcache. Will try to download it from '${proxyInfo.url}'.`,
);
const authorization = getAuthorizationHeaderFor(
const authorization = client.getAuthorizationHeaderFor(
logger,
apiDetails,
proxyInfo.url,
);
const temp = await downloadProxy(logger, proxyInfo.url, authorization);
+11 -10
View File
@@ -6,6 +6,7 @@ import * as actionsUtil from "./actions-util";
import { Config } from "./config-utils";
import { EnvVar, RegistryProxyVars } from "./environment";
import { BuiltInLanguage } from "./languages";
import { getRunnerLogger } from "./logging";
import { ToolsSource } from "./setup-codeql";
import type { Registry } from "./start-proxy";
import {
@@ -153,7 +154,7 @@ test.serial("createStatusReportBase", async (t) => {
languages: [BuiltInLanguage.java, BuiltInLanguage.swift],
}),
{ numAvailableBytes: 100, numTotalBytes: 500 },
new RecordingLogger(),
getRunnerLogger(false),
"failure cause",
"exception stack trace",
);
@@ -202,7 +203,7 @@ test.serial("createStatusReportBase - empty configuration", async (t) => {
new Date("May 19, 2023 05:19:00"),
{},
{ numAvailableBytes: 100, numTotalBytes: 500 },
new RecordingLogger(),
getRunnerLogger(false),
);
if (t.truthy(statusReport)) {
@@ -224,7 +225,7 @@ test.serial("createStatusReportBase - partial configuration", async (t) => {
languages: ["go"],
},
{ numAvailableBytes: 100, numTotalBytes: 500 },
new RecordingLogger(),
getRunnerLogger(false),
);
if (t.truthy(statusReport)) {
@@ -247,7 +248,7 @@ test.serial("createStatusReportBase_firstParty", async (t) => {
new Date("May 19, 2023 05:19:00"),
createTestConfig({}),
{ numAvailableBytes: 100, numTotalBytes: 500 },
new RecordingLogger(),
getRunnerLogger(false),
"failure cause",
"exception stack trace",
)
@@ -263,7 +264,7 @@ test.serial("createStatusReportBase_firstParty", async (t) => {
new Date("May 19, 2023 05:19:00"),
createTestConfig({}),
{ numAvailableBytes: 100, numTotalBytes: 500 },
new RecordingLogger(),
getRunnerLogger(false),
"failure cause",
"exception stack trace",
)
@@ -280,7 +281,7 @@ test.serial("createStatusReportBase_firstParty", async (t) => {
new Date("May 19, 2023 05:19:00"),
createTestConfig({}),
{ numAvailableBytes: 100, numTotalBytes: 500 },
new RecordingLogger(),
getRunnerLogger(false),
"failure cause",
"exception stack trace",
)
@@ -296,7 +297,7 @@ test.serial("createStatusReportBase_firstParty", async (t) => {
new Date("May 19, 2023 05:19:00"),
createTestConfig({}),
{ numAvailableBytes: 100, numTotalBytes: 500 },
new RecordingLogger(),
getRunnerLogger(false),
"failure cause",
"exception stack trace",
)
@@ -313,7 +314,7 @@ test.serial("createStatusReportBase_firstParty", async (t) => {
new Date("May 19, 2023 05:19:00"),
createTestConfig({}),
{ numAvailableBytes: 100, numTotalBytes: 500 },
new RecordingLogger(),
getRunnerLogger(false),
"failure cause",
"exception stack trace",
)
@@ -329,7 +330,7 @@ test.serial("createStatusReportBase_firstParty", async (t) => {
new Date("May 19, 2023 05:19:00"),
createTestConfig({}),
{ numAvailableBytes: 100, numTotalBytes: 500 },
new RecordingLogger(),
getRunnerLogger(false),
"failure cause",
"exception stack trace",
)
@@ -410,7 +411,7 @@ const testCreateInitWithConfigStatusReport = makeMacro({
new Date("May 19, 2023 05:19:00"),
config,
{ numAvailableBytes: 100, numTotalBytes: 500 },
new RecordingLogger(),
getRunnerLogger(false),
"failure cause",
"exception stack trace",
);
+3 -2
View File
@@ -3,8 +3,9 @@ import * as stream from "stream";
import test from "ava";
import { getRunnerLogger } from "./logging";
import { extractTarZst } from "./tar";
import { RecordingLogger, setupTests } from "./testing-utils";
import { setupTests } from "./testing-utils";
import { withTmpDir } from "./util";
setupTests(test);
@@ -16,7 +17,7 @@ test("extractTarZst rejects if the input stream errors", async (t) => {
archive,
path.join(tmpDir, "dest"),
{ type: "gnu", version: "1.34" },
new RecordingLogger(),
getRunnerLogger(true),
);
archive.destroy(
+26 -23
View File
@@ -14,6 +14,7 @@ import * as sinon from "sinon";
import { ActionState, StateFeature } from "./action-common";
import { ActionsEnv, getActionVersion } from "./actions-util";
import * as actionsUtil from "./actions-util";
import { AnalysisKind } from "./analyses";
import * as apiClient from "./api-client";
import { GitHubApiDetails } from "./api-client";
@@ -30,7 +31,7 @@ import {
featureConfig,
FeatureEnablement,
} from "./feature-flags";
import { joinMessageStrings, Loggable, LoggableError, Logger } from "./logging";
import { Logger } from "./logging";
import { OverlayDatabaseMode } from "./overlay/overlay-database-mode";
import { getBundlePlatform } from "./platform";
import { ActionName } from "./status-report";
@@ -52,6 +53,23 @@ export const SAMPLE_DOTCOM_API_DETAILS = {
apiURL: "https://api.github.com",
};
export function mockApiDetails(
apiDetails: Required<GitHubApiDetails>,
env: Env = getEnv(),
) {
// This is a workaround to mock `api.getApiDetails()` since it doesn't seem to be possible to
// mock this directly. The difficulty is that `getApiDetails()` is called locally in
// `api-client.ts`, but `sinon.stub(api, "getApiDetails")` only affects calls to
// `getApiDetails()` via an imported `api` module.
const getRequiredInput = sinon.stub(actionsUtil, "getRequiredInput");
getRequiredInput.withArgs("token").returns(apiDetails.auth);
env.set("GITHUB_SERVER_URL", apiDetails.url);
env.set("GITHUB_API_URL", apiDetails.apiURL ?? "");
return getRequiredInput;
}
export const LINKED_CLI_VERSION = {
cliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
@@ -658,13 +676,12 @@ export class RecordingLogger implements Logger {
constructor(private readonly logToConsole: boolean = true) {}
private addMessage(level: LogLevel, message: LoggableError): void {
const joinedMessage = joinMessageStrings(message);
this.messages.push({ type: level, message: joinedMessage });
private addMessage(level: LogLevel, message: string | Error): void {
this.messages.push({ type: level, message });
if (this.logToConsole) {
// eslint-disable-next-line no-console
console.debug(joinedMessage);
console.debug(message);
}
}
@@ -685,19 +702,19 @@ export class RecordingLogger implements Logger {
return true;
}
debug(message: Loggable) {
debug(message: string) {
this.addMessage("debug", message);
}
info(message: Loggable) {
info(message: string) {
this.addMessage("info", message);
}
warning(message: LoggableError) {
warning(message: string | Error) {
this.addMessage("warning", message);
}
error(message: LoggableError) {
error(message: string | Error) {
this.addMessage("error", message);
}
@@ -1032,7 +1049,6 @@ export function createTestConfig(overrides: Partial<Config>): Config {
overlayModeSetExplicitly: false,
repositoryProperties: {},
enableFileCoverageInformation: true,
successfullyInitialised: false,
} satisfies Config,
overrides,
);
@@ -1043,16 +1059,3 @@ export function makeTestToken(length: number = 36) {
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
return chars.repeat(Math.ceil(length / chars.length)).slice(0, length);
}
/**
* Get a real, newly created CodeQL instance for testing. The instance refers to
* a non-existent placeholder codeql command, so tests that use this function
* should also stub the toolrunner.ToolRunner constructor.
*/
export async function getCodeQLForTesting(
cmd = "codeql-for-testing",
logger: Logger = new RecordingLogger(),
env: Env = getEnv(),
): Promise<codeql.CodeQL> {
return codeql.getCodeQLForCmd(logger, cmd, false, env);
}
+8 -7
View File
@@ -10,8 +10,9 @@ import { https } from "follow-redirects";
import nock from "nock";
import * as sinon from "sinon";
import { getRunnerLogger } from "./logging";
import * as tar from "./tar";
import { RecordingLogger, setupTests } from "./testing-utils";
import { setupTests } from "./testing-utils";
import { downloadAndExtract } from "./tools-download";
import * as util from "./util";
@@ -33,7 +34,7 @@ test.serial(
undefined,
{},
undefined,
new RecordingLogger(),
getRunnerLogger(true),
);
t.assert(Number.isInteger(statusReport.downloadDurationMs));
@@ -71,7 +72,7 @@ test.serial(
undefined,
{},
{ type: "gnu", version: "1.34" },
new RecordingLogger(),
getRunnerLogger(true),
);
t.assert(Number.isInteger(statusReport.downloadDurationMs));
@@ -104,7 +105,7 @@ test.serial(
undefined,
{},
{ type: "gnu", version: "1.34" },
new RecordingLogger(),
getRunnerLogger(true),
),
{
instanceOf: util.HTTPError,
@@ -148,7 +149,7 @@ test.serial(
undefined,
{},
{ type: "gnu", version: "1.34" },
new RecordingLogger(),
getRunnerLogger(true),
);
t.assert(Number.isInteger(statusReport.downloadDurationMs));
@@ -188,7 +189,7 @@ test.serial(
undefined,
{},
{ type: "gnu", version: "1.34" },
new RecordingLogger(),
getRunnerLogger(true),
);
t.is(
@@ -236,7 +237,7 @@ test.serial(
undefined,
{},
{ type: "gnu", version: "1.34" },
new RecordingLogger(),
getRunnerLogger(true),
);
t.assert(Number.isInteger(statusReport.totalDurationMs));
Loaded 100 of 114 files, more files were not shown because too many files have changed in this diff. Show more