Compare commits

..
51 changed files with 999 additions and 1071 deletions

No files matched your search

+2 -15
View File
@@ -11,12 +11,9 @@ The CodeQL Action is used in GitHub Actions workflows to run CodeQL scans using
## Review instructions
- Prioritise identifying critical issues, such as logic bugs, over comments on documentation or typos.
- When wording review comments, be helpful and friendly. Assume that the PR author has written the code with the best of intentions. Never word suggestions as commands or using the imperative. Examples of what you should never write: "Preserve that non-fatal behaviour here" or "Please preserve that non-fatal behaviour here".
- When writing feedback, assume that the PR author is competent and is able to understand whether the point you are making is valid or not, why you are raising it, and how to best address it. While you can suggest how to address a point you are making, don't phrase it as a requirement or assume that it is the only solution.
- Always word your comments constructively as suggestions for improvements, for example: "It would make sense to ..." or "Could you add ...?" or "Would it make sense to ...?"
- When wording review comments, be helpful and friendly. Assume that the PR author has written the code with the best of intentions. Word your comments constructively as suggestions for improvements. Do not word suggestions as commands.
- If you want to comment on a change that you believe will fail a CI check, do not present the CI failure you expect as a fact. Instead, write that you think a change "may" lead to a failure in CI. Suggest that, if such a failure manifests, the changes you are commenting on may be the place responsible for the failure and are worth looking at.
- If a suggestion you make is suitable for a follow-up, such as a refactoring that doesn't change the behaviour or fixing a typo in a comment, mention that it can be addressed in a later PR rather than blocking this one or prefix your comment with "Minor: ".
- If a suggestion you make is suitable for a follow-up, such as a refactoring that doesn't change the behaviour or fixing a typo in a comment, mention that it can be addressed in a later PR rather than blocking this one.
- If a change is a net improvement, for example because it improves on an existing limitation of existing code, do not complain about pre-existing problems that remain. You may comment on them, but you should make it clear that the thing you are commenting on is not new by writing e.g. "Not new in this PR, but [..]" followed by your description of the issue and a suggestion that it could be improved at the same time with e.g. "Consider whether this is worth addressing as part of this PR as well."
## Generated code
@@ -25,16 +22,6 @@ The main codebase of the CodeQL Action is written in TypeScript and compiled to
GitHub Actions workflows in the `.github/workflows` directory whose filenames start with two underscores (e.g. `__all-platform-bundle.yml`) are automatically generated using the `pr-checks/sync.sh` script from template files in the `pr-checks/checks` directory. Therefore, you do not need to review files in the `.github/workflows` directory that start with two underscores. However, you should review changes to the `pr-checks` directory as well as workflows in the `.github/workflows` directory that do not start with underscores.
## Feature flags
Some functionality in the code may be gated behind feature flags, which are controlled server-side. If the functionality that is gated behind them is user-facing, then this should be documented in a change note at the time that the gated functionality is added. See `CHANGELOG.md` for existing notes that mention "experimental" for examples. These typically include information about when users may expect to see the feature rolled out.
When a feature flag that gates some functionality is removed, this does not need to be documented in a change note since the feature is already rolled out and the notes for the release which introduced the feature already describe it. The only exception is functionality which will start having an effect on GHES as a result of the feature flag removal:
- Most functionality gated behind feature flags does not affect GHES.
- We can make functionality available on GHES by setting `defaultValue` to `true` first and waiting for this to have had an effect on GHES. No change note is needed if a feature flag is removed where `defaultValue` was `true`.
- When removing a feature flag where `defaultValue` was `false` and the functionality starts to affect GHES, a change note may be appropriate.
## Release process
Changes that are merged to the `main` branch are not used in production straight away. Instead, changes from `main` must first be released using the release process, which has to be manually triggered by a staff member. Therefore, it is possible for further changes to be made to the code after some changes have been merged to `main` but before they are released. Once the release process automation is started, a series of PRs are automatically generated:
+47 -13
View File
@@ -48,15 +48,6 @@ jobs:
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install Node.js
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 20.x
cache: npm
- name: Install newer npm
run: npm install -g npm@11.19.1
- name: Install dependencies
run: npm ci
- name: Prepare test
id: prepare-test
uses: ./.github/actions/prepare-test
@@ -90,13 +81,56 @@ jobs:
name: diagnostics-export-${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: '${{ runner.temp }}/results/javascript.sarif'
retention-days: 7
- name: Check diagnostics appear in SARIF
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
env:
SARIF_PATH: '${{ runner.temp }}/results/javascript.sarif'
run: |
cd ../action
npx tsx ./pr-checks/check-diagnostics.ts
with:
script: |
const fs = require('fs');
function checkStatusPageNotification(n) {
const expectedMessage = 'Plaintext message';
if (n.message.text !== expectedMessage) {
core.setFailed(`Expected the status page diagnostic to have the message '${expectedMessage}', but found '${n.message.text}'.`);
}
if (n.locations.length !== 1) {
core.setFailed(`Expected the status page diagnostic to have exactly 1 location, but found ${n.locations.length}.`);
}
}
const sarif = JSON.parse(fs.readFileSync(process.env['SARIF_PATH'], 'utf8'));
const run = sarif.runs[0];
const toolExecutionNotifications = run.invocations[0].toolExecutionNotifications;
const statusPageNotifications = toolExecutionNotifications.filter(n =>
n.descriptor.id === 'lang/diagnostics/example' && n.properties?.visibility?.statusPage
);
if (statusPageNotifications.length !== 1) {
core.setFailed(
'Expected exactly one status page reporting descriptor for this diagnostic in the ' +
`'runs[].invocations[].toolExecutionNotifications[]' SARIF property, but found ` +
`${statusPageNotifications.length}. All notification reporting descriptors: ` +
`${JSON.stringify(toolExecutionNotifications)}.`
);
}
checkStatusPageNotification(statusPageNotifications[0]);
const notifications = run.tool.driver.notifications;
const diagnosticNotification = notifications.filter(n =>
n.id === 'lang/diagnostics/example' && n.name === 'lang/diagnostics/example' &&
n.fullDescription.text === 'Diagnostic name'
);
if (diagnosticNotification.length !== 1) {
core.setFailed(
'Expected exactly one notification for this diagnostic in the ' +
`'runs[].tool.driver.notifications[]' SARIF property, but found ` +
`${diagnosticNotification.length}. All notifications: ` +
`${JSON.stringify(notifications)}.`
);
}
core.info('Finished diagnostic export test');
env:
CODEQL_ACTION_EXPORT_DIAGNOSTICS: true
CODEQL_ACTION_TEST_MODE: true
+1
View File
@@ -192,5 +192,6 @@ jobs:
exit 1
fi
env:
CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES: true
CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI: true
CODEQL_ACTION_TEST_MODE: true
+5 -7
View File
@@ -2,9 +2,6 @@ name: PR Checks
on:
push:
branches:
- "main"
- "releases/v*"
pull_request:
merge_group:
types: [checks_requested]
@@ -148,11 +145,12 @@ jobs:
- name: Fetch base commit
id: fetch-base
# Skip this (and subsequent steps) if the PR originates from a fork, since this workflow
# won't have the necessary permissions to post comments on the PR.
# Forks and Dependabot PRs don't have permission to write comments, so skip the repo size
# check in those cases.
if: >-
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.repository
github.event.pull_request.head.repo.full_name == github.repository &&
github.event.pull_request.user.login != 'dependabot[bot]'
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
@@ -211,7 +209,7 @@ jobs:
if: >-
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.repository &&
github.actor != 'dependabot[bot]' &&
github.event.pull_request.user.login != 'dependabot[bot]' &&
needs.other-checks.result == 'success'
permissions:
contents: read
+1 -1
View File
@@ -4,7 +4,7 @@ See the [releases page](https://github.com/github/codeql-action/releases) for th
## [UNRELEASED]
- Update default CodeQL bundle version to [2.27.3](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.27.3). [#4256](https://github.com/github/codeql-action/pull/4256)
No user facing changes.
## 4.39.0 - 15 Oct 2026
+1 -4
View File
@@ -153,10 +153,7 @@ inputs:
required: false
trap-caching:
description: >-
Explicitly enable or disable TRAP caching instead of using the automatically determined default.
TRAP caching is disabled by default when improved incremental analysis is enabled,
because improved incremental analysis of a pull request only extracts changed files,
which rarely benefit from TRAP caching.
Explicitly enable or disable TRAP caching rather than respecting the feature flag for it.
required: false
dependency-caching:
description: >-
+4 -4
View File
@@ -1,6 +1,6 @@
{
"bundleVersion": "codeql-bundle-v2.27.3",
"cliVersion": "2.27.3",
"priorBundleVersion": "codeql-bundle-v2.27.2",
"priorCliVersion": "2.27.2"
"bundleVersion": "codeql-bundle-v2.27.2",
"cliVersion": "2.27.2",
"priorBundleVersion": "codeql-bundle-v2.27.1",
"priorCliVersion": "2.27.1"
}
+257 -271
View File
File diff suppressed because it is too large. Load diff
-77
View File
@@ -1,77 +0,0 @@
import fs from "fs";
import * as core from "@actions/core";
import * as sarif from "../src/sarif";
function checkStatusPageNotification(n) {
const expectedMessage = "Plaintext message";
if (n.message.text !== expectedMessage) {
core.setFailed(
`Expected the status page diagnostic to have the message '${expectedMessage}', but found '${n.message.text}'.`,
);
}
if (n.locations.length !== 1) {
core.setFailed(
`Expected the status page diagnostic to have exactly 1 location, but found ${n.locations.length}.`,
);
}
}
const sarifPath = process.env["SARIF_PATH"];
if (sarifPath === undefined) {
throw new Error("The path to a SARIF file must be set in 'SARIF_PATH'.");
}
const log = JSON.parse(fs.readFileSync(sarifPath, "utf8")) as sarif.Log;
const run = log.runs[0];
if (run.invocations === undefined) {
throw new Error("run.invocations is undefined");
}
const toolExecutionNotifications =
run.invocations[0].toolExecutionNotifications;
if (toolExecutionNotifications === undefined) {
throw new Error("toolExecutionNotifications is undefined");
}
const statusPageNotifications = toolExecutionNotifications.filter(
(n) =>
n.descriptor?.id === "lang/diagnostics/example" &&
n.properties?.visibility?.statusPage,
);
if (statusPageNotifications.length !== 1) {
core.setFailed(
"Expected exactly one status page reporting descriptor for this diagnostic in the " +
`'runs[].invocations[].toolExecutionNotifications[]' SARIF property, but found ` +
`${statusPageNotifications.length}. All notification reporting descriptors: ` +
`${JSON.stringify(toolExecutionNotifications)}.`,
);
}
checkStatusPageNotification(statusPageNotifications[0]);
const notifications = run.tool.driver.notifications;
if (notifications === undefined) {
throw new Error("notifications is undefined");
}
const diagnosticNotification = notifications.filter(
(n) =>
n.id === "lang/diagnostics/example" &&
n.name === "lang/diagnostics/example" &&
n.fullDescription?.text === "Diagnostic name",
);
if (diagnosticNotification.length !== 1) {
core.setFailed(
"Expected exactly one notification for this diagnostic in the " +
`'runs[].tool.driver.notifications[]' SARIF property, but found ` +
`${diagnosticNotification.length}. All notifications: ` +
`${JSON.stringify(notifications)}.`,
);
}
core.info("Finished diagnostic export test");
+47 -5
View File
@@ -5,7 +5,6 @@ versions:
- nightly-latest
env:
CODEQL_ACTION_EXPORT_DIAGNOSTICS: true
installNode: true
steps:
- uses: ./../action/init
id: init
@@ -33,10 +32,53 @@ steps:
name: diagnostics-export-${{ matrix.os }}-${{ matrix.version }}.sarif.json
path: "${{ runner.temp }}/results/javascript.sarif"
retention-days: 7
- name: Check diagnostics appear in SARIF
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
env:
SARIF_PATH: "${{ runner.temp }}/results/javascript.sarif"
run: |
cd ../action
npx tsx ./pr-checks/check-diagnostics.ts
with:
script: |
const fs = require('fs');
function checkStatusPageNotification(n) {
const expectedMessage = 'Plaintext message';
if (n.message.text !== expectedMessage) {
core.setFailed(`Expected the status page diagnostic to have the message '${expectedMessage}', but found '${n.message.text}'.`);
}
if (n.locations.length !== 1) {
core.setFailed(`Expected the status page diagnostic to have exactly 1 location, but found ${n.locations.length}.`);
}
}
const sarif = JSON.parse(fs.readFileSync(process.env['SARIF_PATH'], 'utf8'));
const run = sarif.runs[0];
const toolExecutionNotifications = run.invocations[0].toolExecutionNotifications;
const statusPageNotifications = toolExecutionNotifications.filter(n =>
n.descriptor.id === 'lang/diagnostics/example' && n.properties?.visibility?.statusPage
);
if (statusPageNotifications.length !== 1) {
core.setFailed(
'Expected exactly one status page reporting descriptor for this diagnostic in the ' +
`'runs[].invocations[].toolExecutionNotifications[]' SARIF property, but found ` +
`${statusPageNotifications.length}. All notification reporting descriptors: ` +
`${JSON.stringify(toolExecutionNotifications)}.`
);
}
checkStatusPageNotification(statusPageNotifications[0]);
const notifications = run.tool.driver.notifications;
const diagnosticNotification = notifications.filter(n =>
n.id === 'lang/diagnostics/example' && n.name === 'lang/diagnostics/example' &&
n.fullDescription.text === 'Diagnostic name'
);
if (diagnosticNotification.length !== 1) {
core.setFailed(
'Expected exactly one notification for this diagnostic in the ' +
`'runs[].tool.driver.notifications[]' SARIF property, but found ` +
`${diagnosticNotification.length}. All notifications: ` +
`${JSON.stringify(notifications)}.`
);
}
core.info('Finished diagnostic export test');
@@ -14,6 +14,7 @@ operatingSystems:
- stable-v2.21.4
- stable-v2.22.4
env:
CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES: true
CODEQL_ACTION_RESOLVE_SUPPORTED_LANGUAGES_USING_CLI: true
installGo: true
installDotNet: true
+4 -5
View File
@@ -12,8 +12,8 @@ import {
import { computeAutomationID } from "./api-client";
import { EnvVar } from "./environment";
import { getRunnerLogger } from "./logging";
import { getTestEnv, RecordingLogger, setupTests } from "./testing-utils";
import { withTmpDir } from "./util";
import { RecordingLogger, setupTests } from "./testing-utils";
import { initializeEnvironment, withTmpDir } from "./util";
setupTests(test);
@@ -228,9 +228,8 @@ test.serial(
);
test.serial("initializeEnvironment", (t) => {
const env = getTestEnv();
env.initialise("1.2.3");
t.deepEqual(env.getOptional(EnvVar.VERSION), "1.2.3");
initializeEnvironment("1.2.3");
t.deepEqual(process.env[EnvVar.VERSION], "1.2.3");
});
test.serial("fixCodeQualityCategory", (t) => {
+3 -6
View File
@@ -238,7 +238,7 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
let databaseUploadResults: DatabaseUploadResult[] = [];
try {
action.env.initialise(actionsUtil.getActionVersion());
util.initializeEnvironment(actionsUtil.getActionVersion());
// Make inputs accessible in the `post` step, details at
// https://github.com/github/codeql-action/issues/2553
@@ -361,7 +361,6 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
uploadResults = await postProcessAndUploadSarif(
logger,
features,
config,
uploadKind,
checkoutPath,
outputDir,
@@ -434,10 +433,8 @@ async function run(action: ActionState<["Base", "Logger", "Env", "Actions"]>) {
}
// We don't upload results in test mode, so don't wait for processing
if (util.shouldSkipSarifUpload()) {
logger.debug(
"SARIF upload is disabled and nothing was uploaded, so we do not need to wait for processing to finish.",
);
if (util.isInTestMode()) {
logger.debug("In test mode. Waiting for processing is disabled.");
} else if (
uploadResults?.[analyses.AnalysisKind.CodeScanning] !== undefined &&
actionsUtil.getRequiredInput("wait-for-processing") === "true"
+33 -58
View File
@@ -7,7 +7,7 @@ import { ProxyAgent } from "undici";
import * as actionsUtil from "./actions-util";
import * as api from "./api-client";
import { DO_NOT_RETRY_STATUSES } from "./api-client";
import { ActionsEnvVars, RegistryProxyVars, getEnv } from "./environment";
import { ActionsEnvVars, RegistryProxyVars } from "./environment";
import { GITHUB_DOTCOM_API_URL, GITHUB_DOTCOM_URL } from "./github";
import {
callee,
@@ -21,7 +21,7 @@ import * as util from "./util";
setupTests(test);
test.beforeEach(() => {
getEnv().initialise(actionsUtil.getActionVersion());
util.initializeEnvironment(actionsUtil.getActionVersion());
});
test.serial("getApiClient", async (t) => {
@@ -308,70 +308,45 @@ test.serial(
test("getAuthorizationHeaderFor - returns undefined if there's no token", (t) => {
const logger = new RecordingLogger();
t.is(
api.getAuthorizationHeaderFor(
logger,
{ ...SAMPLE_DOTCOM_API_DETAILS, auth: undefined },
`${GITHUB_DOTCOM_URL}/github`,
),
undefined,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
{ ...SAMPLE_DOTCOM_API_DETAILS, auth: "" },
`${GITHUB_DOTCOM_URL}/github`,
),
undefined,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
{ ...SAMPLE_DOTCOM_API_DETAILS, auth: " " },
`${GITHUB_DOTCOM_URL}/github`,
),
undefined,
);
for (const authValue of [undefined, "", " "]) {
const client = new api.InstanceApiClient({
...SAMPLE_DOTCOM_API_DETAILS,
auth: authValue,
});
t.is(
client.getAuthorizationHeaderFor(logger, `${GITHUB_DOTCOM_URL}/github`),
undefined,
);
}
});
test("getAuthorizationHeaderFor - returns header value with token if url is allowed", (t) => {
const logger = new RecordingLogger();
const client = new api.InstanceApiClient({
...SAMPLE_DOTCOM_API_DETAILS,
});
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`${GITHUB_DOTCOM_URL}/github`,
),
`token ${SAMPLE_DOTCOM_API_DETAILS.auth}`,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`${GITHUB_DOTCOM_API_URL}/github`,
),
`token ${SAMPLE_DOTCOM_API_DETAILS.auth}`,
);
for (const url of [
`${GITHUB_DOTCOM_URL}/github`,
`${GITHUB_DOTCOM_API_URL}/github`,
]) {
t.is(
client.getAuthorizationHeaderFor(logger, url),
`token ${SAMPLE_DOTCOM_API_DETAILS.auth}`,
);
}
});
test("getAuthorizationHeaderFor - returns undefined if url is not allowed", (t) => {
const logger = new RecordingLogger();
const client = new api.InstanceApiClient({
...SAMPLE_DOTCOM_API_DETAILS,
});
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`https://localhost/github`,
),
undefined,
);
t.is(
api.getAuthorizationHeaderFor(
logger,
SAMPLE_DOTCOM_API_DETAILS,
`https://api.localhost/github`,
),
undefined,
);
for (const url of [
`https://localhost/github`,
`https://api.localhost/github`,
]) {
t.is(client.getAuthorizationHeaderFor(logger, url), undefined);
}
});
+86 -38
View File
@@ -18,9 +18,11 @@ import {
RegistryProxyVars,
getEnv,
exportEnvVar,
ActionsEnvVars,
} from "./environment";
import {
getCurrentInstance,
getDotcomInstance,
GITHUB_DOTCOM_URL,
parseGitHubUrl,
type GitHubApiCombinedDetails,
@@ -120,6 +122,66 @@ export function makeProxyRequestOptions(
/** The type of GitHub API client we use. */
export type ApiClient = InstanceType<typeof githubUtils.GitHub>;
/**
* Encapsulates a {@link GitHubApiCombinedDetails} along with operations which ensure that any credentials
* are only used with the GitHub instance that is represented by the {@link GitHubApiCombinedDetails} value.
*/
export class InstanceApiClient {
private readonly client: ApiClient;
private readonly apiDetails: GitHubApiCombinedDetails;
private readonly authorizationHeader: string | undefined;
constructor(
apiDetails: GitHubApiCombinedDetails,
clientOptions: CreateApiClientOptions = {},
) {
this.apiDetails = apiDetails;
this.client = createApiClientWithDetails(apiDetails, clientOptions);
// Compute the Authorization header value if we have a token.
const token = this.apiDetails.auth?.trim();
if (token !== undefined && token !== "") {
this.authorizationHeader = `token ${token}`;
}
}
/**
* In most cases, we use an {@link ApiClient} which automatically handles auth for requests
* made with it and this method is not needed and should not be used in that setting.
*
* In some cases, we need to make requests that can't go through an {@link ApiClient}, but still
* require auth. For example, this is the case when using the `toolcache` module to download
* files. For these cases, this method gets a value for the `Authorization` header for a request
* to `url`; or `undefined` if the `Authorization` header should not be set for `url`.
*
* @param logger The logger to use for debugging messages.
* @param url The URL for which we want to add an `Authorization` header.
*/
public getAuthorizationHeaderFor(
logger: Logger,
url: string,
): string | undefined {
// We don't have to check the `url` if we don't have a token to begin with.
if (this.authorizationHeader === undefined) {
return undefined;
}
// We only want to provide an authorization header if we are downloading
// from the same GitHub instance the Action is running on.
// This avoids leaking Enterprise tokens to dotcom.
if (
url.startsWith(`${this.apiDetails.url}/`) ||
(this.apiDetails.apiURL && url.startsWith(`${this.apiDetails.apiURL}/`))
) {
logger.debug(`Providing an authorization token.`);
return this.authorizationHeader;
}
logger.debug(`Not using an authorization token.`);
return undefined;
}
}
/** Options for `createApiClientWithDetails`. */
interface CreateApiClientOptions {
allowExternal?: boolean;
@@ -172,8 +234,8 @@ export function getApiDetails(env: ReadOnlyEnv = getEnv()): GitHubApiDetails {
/**
* Constructs an {@link ApiClient} using {@link apiDetails}.
*
* If {@link apiDetails} is not specified, then the GitHub instance URLs and auth token
* are taken from {@link env}.
* If {@link apiDetails} is not specified, then the GitHub instance URLs are taken from {@link env}
* and the auth token is taken from the `token` input.
*
* If {@link env} is not specified, then the process environment is used.
*
@@ -187,6 +249,28 @@ export function getApiClient(
return createApiClientWithDetails(apiDetails ?? getApiDetails(env));
}
/*
* Gets a `GitHubApiDetails` object describing the GitHub Dotcom instance.
*
* @param env The environment variables to read.
*/
export function getDotcomApiDetails(env: ReadOnlyEnv): GitHubApiDetails {
// If we are currently running on dotcom, then just call `getApiDetails` which will
// construct the appropriate `GitHubApiDetails` from the environment and include the
// value of the `token` input. The `token` input relates to the GitHub instance that
// we are running on only. Therefore, if we are not currently running on dotcom,
// then we construct `GitHubApiDetails` without the `token` value and using the
// hard-coded instance and API URLs.
if (env.getRequired(ActionsEnvVars.GITHUB_SERVER_URL) === GITHUB_DOTCOM_URL) {
return getApiDetails(env);
} else {
return {
auth: undefined,
...getDotcomInstance(),
};
}
}
export function getApiClientWithExternalAuth(
apiDetails: GitHubApiCombinedDetails,
proxy?: ProxyAgent,
@@ -194,42 +278,6 @@ export function getApiClientWithExternalAuth(
return createApiClientWithDetails(apiDetails, { allowExternal: true, proxy });
}
/**
* Gets a value for the `Authorization` header for a request to `url`; or `undefined` if the
* `Authorization` header should not be set for `url`.
*
* @param logger The logger to use for debugging messages.
* @param apiDetails Details of the GitHub API we are using.
* @param url The URL for which we want to add an `Authorization` header.
*
* @returns The value for the `Authorization` header or `undefined` if it shouldn't be populated.
*/
export function getAuthorizationHeaderFor(
logger: Logger,
apiDetails: GitHubApiDetails,
url: string,
): string | undefined {
// We don't have to check the `url` if we don't have a token to begin with.
const token = apiDetails.auth?.trim();
if (token === undefined || token === "") {
return undefined;
}
// We only want to provide an authorization header if we are downloading
// from the same GitHub instance the Action is running on.
// This avoids leaking Enterprise tokens to dotcom.
if (
url.startsWith(`${apiDetails.url}/`) ||
(apiDetails.apiURL && url.startsWith(`${apiDetails.apiURL}/`))
) {
logger.debug(`Providing an authorization token.`);
return `token ${token}`;
}
logger.debug(`Not using an authorization token.`);
return undefined;
}
let cachedGitHubVersion: GitHubVersion | undefined = undefined;
export async function getGitHubVersionFromApi(
+4 -11
View File
@@ -10,7 +10,7 @@ import { getGitHubVersion } from "./api-client";
import { determineAutobuildLanguages, runAutobuild } from "./autobuild";
import { getCodeQL } from "./codeql";
import { Config, getConfig } from "./config-utils";
import { EnvVar, type Env } from "./environment";
import { EnvVar } from "./environment";
import { Language } from "./languages";
import { Logger } from "./logging";
import {
@@ -26,6 +26,7 @@ import {
checkDiskUsage,
checkGitHubVersionInRange,
ConfigurationError,
initializeEnvironment,
wrapError,
} from "./util";
@@ -37,7 +38,6 @@ interface AutobuildStatusReport extends StatusReportBase {
}
async function sendCompletedStatusReport(
env: Env,
config: Config | undefined,
logger: Logger,
startedAt: Date,
@@ -45,7 +45,7 @@ async function sendCompletedStatusReport(
failingLanguage?: string,
cause?: Error,
) {
env.initialise(getActionVersion());
initializeEnvironment(getActionVersion());
const status = getActionsStatus(cause, failingLanguage);
const statusReportBase = await createStatusReportBase(
@@ -129,7 +129,6 @@ async function run({
`We were unable to automatically build your code. Please replace the call to the autobuild action with your custom build steps. ${error.message}`,
);
await sendCompletedStatusReport(
env,
config,
logger,
startedAt,
@@ -142,13 +141,7 @@ async function run({
env.export(EnvVar.AUTOBUILD_DID_COMPLETE_SUCCESSFULLY, "true");
await sendCompletedStatusReport(
env,
config,
logger,
startedAt,
languages ?? [],
);
await sendCompletedStatusReport(config, logger, startedAt, languages ?? []);
}
/** Defines the `autobuild` Action. */
+3
View File
@@ -1,3 +1,4 @@
import type { GitHubApiDetails } from "./github";
import { BuiltInLanguage, parseBuiltInLanguage } from "./languages";
import { BundlePlatform } from "./platform";
import type { CompressionMethod } from "./tar";
@@ -17,6 +18,8 @@ export type CodeQLBundle =
export interface CodeQLDownloadSource {
/** Distinguishes downloads from local archives and cached installations. */
sourceType: "download";
/** Information about the GitHub instance that contains the asset. */
apiDetails: GitHubApiDetails;
/** The bundle to download. */
bundle: CodeQLBundle;
/** The compression format of the bundle archive. */
+6 -15
View File
@@ -21,7 +21,6 @@ import {
import type { Config } from "./config-utils";
import * as defaults from "./defaults.json";
import { DocUrl } from "./doc-url";
import { getEnv } from "./environment";
import { BuiltInLanguage } from "./languages";
import { getRunnerLogger } from "./logging";
import { ToolsSource } from "./setup-codeql";
@@ -35,16 +34,18 @@ import {
makeVersionInfo,
createTestConfig,
makeMacro,
mockApiDetails,
} from "./testing-utils";
import { ToolsDownloadStatusReport } from "./tools-download";
import * as util from "./util";
import { initializeEnvironment } from "./util";
setupTests(test);
let stubConfig: Config;
test.beforeEach(() => {
getEnv().initialise("1.2.3");
initializeEnvironment("1.2.3");
stubConfig = createTestConfig({
languages: [BuiltInLanguage.cpp],
@@ -126,19 +127,6 @@ function mockReleaseApi({
});
}
function mockApiDetails(apiDetails: Required<GitHubApiDetails>) {
// This is a workaround to mock `api.getApiDetails()` since it doesn't seem to be possible to
// mock this directly. The difficulty is that `getApiDetails()` is called locally in
// `api-client.ts`, but `sinon.stub(api, "getApiDetails")` only affects calls to
// `getApiDetails()` via an imported `api` module.
sinon
.stub(actionsUtil, "getRequiredInput")
.withArgs("token")
.returns(apiDetails.auth);
process.env["GITHUB_SERVER_URL"] = apiDetails.url;
process.env["GITHUB_API_URL"] = apiDetails.apiURL || "";
}
async function stubCodeql(): Promise<codeql.CodeQL> {
const codeqlObject = await codeql.getCodeQLForTesting();
sinon.stub(codeqlObject, "getVersion").resolves(makeVersionInfo("2.17.6"));
@@ -290,6 +278,7 @@ for (const { cliVersion, tagName, expectedToolcacheVersion } of [
async (t) => {
await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const result = await installIntoToolcache({
cliVersion,
@@ -470,6 +459,7 @@ test.serial(
await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
await installIntoToolcache({
tagName: "codeql-bundle-20200601",
@@ -517,6 +507,7 @@ test.serial(
await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
await installIntoToolcache({
tagName: "codeql-bundle-20200601",
+26 -31
View File
@@ -4,7 +4,6 @@ import * as path from "path";
import * as core from "@actions/core";
import * as toolrunner from "@actions/exec/lib/toolrunner";
import * as yaml from "js-yaml";
import * as semver from "semver";
import {
CommandInvocationError,
@@ -20,7 +19,6 @@ import type { ActionConfig } from "./config/action-config";
import { appendExtraQueryExclusions, type Config } from "./config-utils";
import { DocUrl } from "./doc-url";
import { EnvVar, getEnv, exportEnvVar } from "./environment";
import { getErrorMessage, isDiskConfigurationError } from "./errors";
import {
CodeQLDefaultVersionInfo,
Feature,
@@ -36,9 +34,7 @@ import { ToolsDownloadStatusReport } from "./tools-download";
import { ToolsFeature, isSupportedToolsFeature } from "./tools-features";
import { shouldEnableIndirectTracing } from "./tracer-config";
import * as util from "./util";
import { BuildMode, CleanupLevel } from "./util";
export { isDiskConfigurationError } from "./errors";
import { BuildMode, CleanupLevel, getErrorMessage } from "./util";
type Options = Array<string | number | boolean>;
@@ -74,11 +70,6 @@ export interface CodeQL {
* Get a string containing the semver version of the CodeQL executable.
*/
getVersion(): Promise<VersionInfo>;
/**
* Determines whether the CLI installation represented by this {@link CodeQL} instance
* has at least the {@link requiredVersion}.
*/
isVersionAtLeast(requiredVersion: string): Promise<boolean>;
/**
* Print version information about CodeQL.
*/
@@ -283,6 +274,26 @@ const GHES_MOST_RECENT_DEPRECATION_DATE = "2026-09-24";
/** The CLI verbosity level to use for extraction in debug mode. */
const EXTRACTION_DEBUG_MODE_VERBOSITY = "progress++";
/**
* Decides whether `e` is a disk-related error outside of our control
* that should be classified as a `ConfigurationError`.
*
* @param e The error to check.
* @returns True if the error should be treated as a `ConfigurationError` or false if not.
*/
export function isDiskConfigurationError(e: unknown): boolean {
if (!(e instanceof Error)) {
return false;
}
return (
// out of disk space
e.message.includes("ENOSPC") ||
// access denied
e.message.includes("EACCES")
);
}
/**
* Set up CodeQL CLI access.
*
@@ -391,9 +402,9 @@ export function setCodeQL(codeql: Partial<CodeQL>): void {
cachedCodeQL = createStubCodeQL(codeql);
}
function resolveFunction<K extends keyof CodeQL, T extends CodeQL[K]>(
function resolveFunction<T>(
partialCodeql: Partial<CodeQL>,
methodName: K,
methodName: string,
defaultImplementation?: T,
): T {
if (typeof partialCodeql[methodName] !== "function") {
@@ -403,7 +414,7 @@ function resolveFunction<K extends keyof CodeQL, T extends CodeQL[K]>(
const dummyMethod = () => {
throw new Error(`CodeQL ${methodName} method not correctly defined`);
};
return dummyMethod as unknown as T;
return dummyMethod as T;
}
return partialCodeql[methodName] as T;
}
@@ -420,19 +431,6 @@ export function createStubCodeQL(partialCodeql: Partial<CodeQL>): CodeQL {
getVersion: resolveFunction(partialCodeql, "getVersion", async () => ({
version: "1.0.0",
})),
isVersionAtLeast: resolveFunction(
partialCodeql,
"isVersionAtLeast",
async (requiredVersion) => {
return (
!!partialCodeql.getVersion &&
semver.gte(
(await partialCodeql.getVersion()).version,
requiredVersion,
)
);
},
),
printVersion: resolveFunction(partialCodeql, "printVersion"),
supportsFeature: resolveFunction(
partialCodeql,
@@ -536,9 +534,6 @@ async function getCodeQLForCmd(
}
return result;
},
async isVersionAtLeast(requiredVersion: string) {
return semver.gte((await this.getVersion()).version, requiredVersion);
},
async printVersion() {
// Reuse the cached version information rather than invoking the CLI again.
core.info(JSON.stringify(await this.getVersion(), null, 2));
@@ -1033,7 +1028,7 @@ async function getCodeQLForCmd(
// CodeQL object is created.
if (
checkVersion &&
!(await codeql.isVersionAtLeast(CODEQL_MINIMUM_VERSION))
!(await util.codeQlVersionAtLeast(codeql, CODEQL_MINIMUM_VERSION))
) {
throw new util.ConfigurationError(
`Expected a CodeQL CLI with version at least ${CODEQL_MINIMUM_VERSION} but got version ${
@@ -1043,7 +1038,7 @@ async function getCodeQLForCmd(
} else if (
checkVersion &&
process.env[EnvVar.SUPPRESS_DEPRECATED_SOON_WARNING] !== "true" &&
!(await codeql.isVersionAtLeast(CODEQL_NEXT_MINIMUM_VERSION))
!(await util.codeQlVersionAtLeast(codeql, CODEQL_NEXT_MINIMUM_VERSION))
) {
const result = await codeql.getVersion();
core.warning(
+67 -45
View File
@@ -2126,7 +2126,7 @@ test.serial("getPrimaryAnalysisConfig - Code Scanning + Code Quality", (t) => {
});
test.serial(
"isTrapCachingEnabled: explicit true input overrides self-hosted default",
"isTrapCachingEnabled: explicit input true is respected",
async (t) => {
return await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -2134,40 +2134,12 @@ test.serial(
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns("true");
t.true(configUtils.isTrapCachingEnabled(OverlayDatabaseMode.None));
});
},
);
test.serial(
"isTrapCachingEnabled: explicit true input overrides overlay defaults",
async (t) => {
return await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, path.join(tmpDir, "hostedtoolcache"));
sinon
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns("true");
for (const overlayDatabaseMode of [
OverlayDatabaseMode.Overlay,
OverlayDatabaseMode.OverlayBase,
]) {
t.true(configUtils.isTrapCachingEnabled(overlayDatabaseMode));
}
});
},
);
test.serial(
"isTrapCachingEnabled: explicit false input overrides hosted default",
async (t) => {
return await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, path.join(tmpDir, "hostedtoolcache"));
sinon
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns("false");
t.false(configUtils.isTrapCachingEnabled(OverlayDatabaseMode.None));
t.true(
await configUtils.isTrapCachingEnabled(
createFeatures([]),
OverlayDatabaseMode.None,
),
);
});
},
);
@@ -2181,13 +2153,18 @@ test.serial(
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns(undefined);
t.false(configUtils.isTrapCachingEnabled(OverlayDatabaseMode.None));
t.false(
await configUtils.isTrapCachingEnabled(
createFeatures([]),
OverlayDatabaseMode.None,
),
);
});
},
);
test.serial(
"isTrapCachingEnabled: enabled on hosted runner without overlay analysis",
"isTrapCachingEnabled: enabled on hosted runner by default",
async (t) => {
return await withTmpDir(async (tmpDir) => {
const hostedToolCache = path.join(tmpDir, "hostedtoolcache");
@@ -2196,13 +2173,18 @@ test.serial(
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns(undefined);
t.true(configUtils.isTrapCachingEnabled(OverlayDatabaseMode.None));
t.true(
await configUtils.isTrapCachingEnabled(
createFeatures([]),
OverlayDatabaseMode.None,
),
);
});
},
);
test.serial(
"isTrapCachingEnabled: disabled on hosted runner with overlay analysis",
"isTrapCachingEnabled: enabled on hosted runner when overlay enabled but feature flag off",
async (t) => {
return await withTmpDir(async (tmpDir) => {
const hostedToolCache = path.join(tmpDir, "hostedtoolcache");
@@ -2211,12 +2193,52 @@ test.serial(
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns(undefined);
for (const overlayDatabaseMode of [
OverlayDatabaseMode.Overlay,
OverlayDatabaseMode.OverlayBase,
]) {
t.false(configUtils.isTrapCachingEnabled(overlayDatabaseMode));
}
t.true(
await configUtils.isTrapCachingEnabled(
createFeatures([]),
OverlayDatabaseMode.Overlay,
),
);
});
},
);
test.serial(
"isTrapCachingEnabled: disabled on hosted runner when overlay enabled and feature flag on",
async (t) => {
return await withTmpDir(async (tmpDir) => {
const hostedToolCache = path.join(tmpDir, "hostedtoolcache");
setupActionsVars(tmpDir, hostedToolCache);
sinon
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns(undefined);
t.false(
await configUtils.isTrapCachingEnabled(
createFeatures([Feature.OverlayAnalysisDisableTrapCaching]),
OverlayDatabaseMode.Overlay,
),
);
});
},
);
test.serial(
"isTrapCachingEnabled: enabled on hosted runner when overlay is None even with feature flag on",
async (t) => {
return await withTmpDir(async (tmpDir) => {
const hostedToolCache = path.join(tmpDir, "hostedtoolcache");
setupActionsVars(tmpDir, hostedToolCache);
sinon
.stub(actionsUtil, "getOptionalInput")
.withArgs("trap-caching")
.returns(undefined);
t.true(
await configUtils.isTrapCachingEnabled(
createFeatures([Feature.OverlayAnalysisDisableTrapCaching]),
OverlayDatabaseMode.None,
),
);
});
},
);
+16 -12
View File
@@ -82,6 +82,7 @@ import {
GitHubVersion,
ConfigurationError,
BuildMode,
codeQlVersionAtLeast,
cloneObject,
isDefined,
checkDiskUsage,
@@ -651,7 +652,10 @@ async function runnerHasSufficientMemory(
logger: Logger,
): Promise<boolean> {
if (
await codeql.isVersionAtLeast(CODEQL_VERSION_REDUCED_OVERLAY_MEMORY_USAGE)
await codeQlVersionAtLeast(
codeql,
CODEQL_VERSION_REDUCED_OVERLAY_MEMORY_USAGE,
)
) {
logger.debug(
`Skipping memory check for overlay analysis because CodeQL version is at least ${CODEQL_VERSION_REDUCED_OVERLAY_MEMORY_USAGE}.`,
@@ -895,7 +899,7 @@ async function validateOverlayDatabaseMode(
);
return new Failure(OverlayDisabledReason.IncompatibleBuildMode);
}
if (!(await codeql.isVersionAtLeast(CODEQL_OVERLAY_MINIMUM_VERSION))) {
if (!(await codeQlVersionAtLeast(codeql, CODEQL_OVERLAY_MINIMUM_VERSION))) {
logger.warning(
`Cannot build an ${overlayDatabaseMode} database because ` +
`the CodeQL CLI is older than ${CODEQL_OVERLAY_MINIMUM_VERSION}. ` +
@@ -940,14 +944,10 @@ async function validateOverlayDatabaseMode(
});
}
/**
* Determines whether TRAP caching is enabled for the current run.
*
* @returns Whether TRAP caches should be downloaded and uploaded.
*/
export function isTrapCachingEnabled(
export async function isTrapCachingEnabled(
features: FeatureEnablement,
overlayDatabaseMode: OverlayDatabaseMode,
): boolean {
): Promise<boolean> {
// If the workflow specified something, always respect that.
const trapCaching = getOptionalInput("trap-caching");
if (trapCaching !== undefined) return trapCaching === "true";
@@ -955,8 +955,12 @@ export function isTrapCachingEnabled(
// On self-hosted runners which may have slow network access, disable TRAP caching by default.
if (!looksLikeHostedRunner()) return false;
// Overlay analysis supersedes TRAP caching, so disable TRAP caching by default when it is active.
if (overlayDatabaseMode !== OverlayDatabaseMode.None) {
// If overlay analysis is enabled, then disable TRAP caching since overlay analysis supersedes it.
// This change is gated behind a feature flag.
if (
overlayDatabaseMode !== OverlayDatabaseMode.None &&
(await features.getValue(Feature.OverlayAnalysisDisableTrapCaching))
) {
return false;
}
@@ -1298,7 +1302,7 @@ export async function initConfig(
logger,
);
if (isTrapCachingEnabled(config.overlayDatabaseMode)) {
if (await isTrapCachingEnabled(features, config.overlayDatabaseMode)) {
const { trapCaches, trapCacheDownloadTime } = await downloadCacheWithTime(
inputs.codeql,
config.languages,
+12 -3
View File
@@ -15,7 +15,6 @@ import {
cleanupAndUploadDatabases,
uploadBundledDatabase,
} from "./database-upload";
import { getEnv } from "./environment";
import { Feature } from "./feature-flags";
import * as gitUtils from "./git-utils";
import { BuiltInLanguage } from "./languages";
@@ -32,12 +31,18 @@ import {
setupActionsVars,
setupTests,
} from "./testing-utils";
import { CleanupLevel, GitHubVariant, HTTPError, withTmpDir } from "./util";
import {
CleanupLevel,
GitHubVariant,
HTTPError,
initializeEnvironment,
withTmpDir,
} from "./util";
setupTests(test);
test.beforeEach(() => {
getEnv().initialise("1.2.3");
initializeEnvironment("1.2.3");
});
const testRepoName: RepositoryNwo = { owner: "github", repo: "example" };
@@ -594,6 +599,7 @@ test.serial(
const nwo: RepositoryNwo = { owner: "test", repo: "foo" };
const language: BuiltInLanguage = BuiltInLanguage.java;
const env = getTestEnv();
await t.notThrowsAsync(async () => {
await uploadBundledDatabase(
nwo,
@@ -602,11 +608,14 @@ test.serial(
dbPath,
content.length,
SAMPLE_DOTCOM_API_DETAILS,
env,
);
});
// Check that our stubs were called and that they received the expected arguments.
t.is(getApiClient.callCount, 1);
t.deepEqual(getApiClient.args[0], [env, SAMPLE_DOTCOM_API_DETAILS]);
t.is(fetchStub.callCount, 1);
const args = fetchStub.args[0];
+6 -4
View File
@@ -10,6 +10,7 @@ import {
} from "./api-client";
import { type CodeQL } from "./codeql";
import { Config } from "./config-utils";
import type { ReadOnlyEnv } from "./environment";
import { Feature } from "./feature-flags";
import * as gitUtils from "./git-utils";
import { parseGitHubUrl } from "./github";
@@ -266,11 +267,12 @@ export async function uploadBundledDatabase(
bundledDb: string,
bundledDbSize: number,
apiDetails: GitHubApiDetails,
env: ReadOnlyEnv = util.getEnv(),
): Promise<number> {
// Ensure that the `apiDetails` that `getApiClient` uses match the ones we were given here,
// because we use `apiDetails.auth` below which must be used with the same instance that
// `apiDetails` refers to.
const client = getApiClient(util.getEnv(), apiDetails);
// It is important that the `apiDetails` that `getApiClient` uses match the ones we were given,
// because we use `apiDetails.url` below to construct the URL to upload the database to. That
// URL will be specific to the GitHub instance given by `apiDetails` and so the client must match it.
const client = getApiClient(env, apiDetails);
const uploadsUrl = new URL(parseGitHubUrl(apiDetails.url));
uploadsUrl.hostname = `uploads.${uploadsUrl.hostname}`;
+4 -4
View File
@@ -1,6 +1,6 @@
{
"bundleVersion": "codeql-bundle-v2.27.3",
"cliVersion": "2.27.3",
"priorBundleVersion": "codeql-bundle-v2.27.2",
"priorCliVersion": "2.27.2"
"bundleVersion": "codeql-bundle-v2.27.2",
"cliVersion": "2.27.2",
"priorBundleVersion": "codeql-bundle-v2.27.1",
"priorCliVersion": "2.27.1"
}
+10 -32
View File
@@ -165,21 +165,9 @@ export enum EnvVar {
*/
SARIF_DUMP_DIR = "CODEQL_ACTION_SARIF_DUMP_DIR",
/**
* Whether to skip generating a failed SARIF file. Intended for testing purposes.
* This setting is more specific than `CODEQL_ACTION_TEST_MODE` and
* `CODEQL_ACTION_SKIP_SARIF_UPLOAD`, which imply this option.
*
* Setting this to `false` takes precedence over `CODEQL_ACTION_TEST_MODE: true` or
* `CODEQL_ACTION_SKIP_SARIF_UPLOAD: true`.
*/
SKIP_GENERATE_FAILED_SARIF = "CODEQL_ACTION_SKIP_GENERATE_FAILED_SARIF",
/**
* Whether to skip uploading SARIF results to GitHub. Intended for testing purposes.
* This setting is more specific than `CODEQL_ACTION_TEST_MODE`, which implies this option.
*
* Setting this to `false` takes precedence over `CODEQL_ACTION_TEST_MODE: true`.
*/
SKIP_SARIF_UPLOAD = "CODEQL_ACTION_SKIP_SARIF_UPLOAD",
@@ -293,8 +281,8 @@ export function getOptionalEnvVar(paramName: string): string | undefined {
* in tests, while clearly signalling in regular code that the consumer of the `ReadOnlyEnv` instance
* will only read from it.
*/
export class ReadOnlyEnv {
constructor(protected readonly vars: Record<string, string | undefined>) {}
export class ReadOnlyEnv<T extends string | undefined = string | undefined> {
constructor(protected readonly vars: Record<string, T>) {}
/** Clones the object while detaching the underlying environment from the original. */
public clone(): this {
@@ -302,7 +290,7 @@ export class ReadOnlyEnv {
}
/** Gets a copy of the underlying environment. */
public get(): Record<string, string | undefined> {
public get(): Record<string, T> {
return { ...this.vars };
}
@@ -317,7 +305,7 @@ export class ReadOnlyEnv {
}
/** Gets the entries of the underlying `ProcessEnv`. */
public entries(): Array<[string, string | undefined]> {
public entries(): Array<[string, T]> {
return Object.entries(this.vars);
}
@@ -347,17 +335,19 @@ export class ReadOnlyEnv {
* Use `ReadOnlyEnv` instead if you only plan to read from the environment.
* This type allows writing to the environment.
*/
export class Env extends ReadOnlyEnv {
export class Env<
T extends string | undefined = string | undefined,
> extends ReadOnlyEnv<T> {
private changed: boolean = false;
/** Sets an environment variable. */
public set(name: string, value: string | undefined): void {
public set(name: string, value: T): void {
this.vars[name] = value;
this.changed = true;
}
/** Sets all environment variables given by `vars`. */
public setAll(vars: Record<string, string | undefined>): void {
public setAll(vars: Record<string, T>): void {
for (const [key, val] of Object.entries(vars)) {
this.set(key, val);
}
@@ -371,7 +361,7 @@ export class Env extends ReadOnlyEnv {
* @param name The name of the environment variable to set and export.
* @param val The value to set and export for the environment variable.
*/
public export(name: string, val: string): void {
public export(name: string, val: T): void {
// Setting the environment variable for this instance is always OK, including
// in tests, since we use fresh `Env` instances whenever needed. This allows
// tests to pass that rely on that part of the `core.exportVariable` behaviour.
@@ -383,18 +373,6 @@ export class Env extends ReadOnlyEnv {
}
}
/**
* Set some initial environment variables that we can set even without
* knowing what version of CodeQL we're running.
*/
public initialise(version: string) {
this.export(EnvVar.FEATURE_MULTI_LANGUAGE, "false");
this.export(EnvVar.FEATURE_SANDWICH, "false");
this.export(EnvVar.FEATURE_SARIF_COMBINE, "true");
this.export(EnvVar.FEATURE_WILL_UPLOAD, "true");
this.export(EnvVar.VERSION, version);
}
/** Gets a value indicating whether `set` was called at least once. */
public hasChanged(): boolean {
return this.changed;
-73
View File
@@ -1,73 +0,0 @@
import * as json from "../json";
/**
* If {@link error} is not already an instance of {@link Error}, then this function constructs
* an {@link Error} with a string representation of {@link error} as the message.
*/
export function wrapError(error: unknown): Error {
return error instanceof Error ? error : new Error(String(error));
}
/**
* Returns an appropriate message for the error.
*
* If the error is an {@link Error} instance, this returns the error message without
* an `Error: ` prefix.
*/
export function getErrorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
/**
* Decides whether `e` is a disk-related error outside of our control
* that should be classified as a `ConfigurationError`.
*
* @param e The error to check.
* @returns True if the error should be treated as a `ConfigurationError` or false if not.
*/
export function isDiskConfigurationError(e: unknown): boolean {
if (!(e instanceof Error)) {
return false;
}
return (
// out of disk space
e.message.includes("ENOSPC") ||
// access denied
e.message.includes("EACCES")
);
}
/**
* An Error class that indicates an error that occurred due to
* a misconfiguration of the action or the CodeQL CLI.
*/
export class ConfigurationError extends Error {}
/** Minimal representation of errors resulting from HTTP requests. */
export class HTTPError extends Error {
public status: number;
constructor(message: string, status: number) {
super(message);
this.status = status;
}
}
/**
* Tries to convert {@link arg} to a {@link HTTPError}, but returns `undefined` if
* {@link arg} is not like a {@link HTTPError}.
*/
export function asHTTPError(arg: any): HTTPError | undefined {
if (!json.isObject<any>(arg) || !json.isString(arg.message)) {
return undefined;
}
if (Number.isInteger(arg.status)) {
return new HTTPError(arg.message, arg.status as number);
}
// See https://github.com/actions/toolkit/blob/acb230b99a46ed33a3f04a758cd68b47b9a82908/packages/tool-cache/src/tool-cache.ts#L19
if (Number.isInteger(arg.httpStatusCode)) {
return new HTTPError(arg.message, arg.httpStatusCode as number);
}
return undefined;
}
+2 -3
View File
@@ -4,7 +4,6 @@ import * as path from "path";
import test from "ava";
import * as defaults from "./defaults.json";
import { getEnv } from "./environment";
import {
Feature,
featureConfig,
@@ -27,12 +26,12 @@ import {
setupTests,
stubFeatureFlagApiEndpoint,
} from "./testing-utils";
import { GitHubVariant, withTmpDir } from "./util";
import { GitHubVariant, initializeEnvironment, withTmpDir } from "./util";
setupTests(test);
test.beforeEach(() => {
getEnv().initialise("1.2.3");
initializeEnvironment("1.2.3");
});
test.serial(
+18 -1
View File
@@ -74,6 +74,11 @@ export enum Feature {
AllowMergeConfigFiles = "allow_merge_config_files",
/** Controls whether we allow multiple values for the `analysis-kinds` input. */
AllowMultipleAnalysisKinds = "allow_multiple_analysis_kinds",
/**
* Controls whether we delete CodeQL bundles that we are not going to use from the toolcache
* before downloading a different bundle, in order to reclaim disk space.
*/
CleanupToolcacheBundles = "cleanup_toolcache_bundles",
CleanupTrapCaches = "cleanup_trap_caches",
/** Whether to allow the `config-file` input to be specified via a repository property. */
ConfigFileRepositoryProperty = "config_file_repository_property",
@@ -103,6 +108,8 @@ export enum Feature {
OverlayAnalysisCodeScanningRuby = "overlay_analysis_code_scanning_ruby",
OverlayAnalysisCpp = "overlay_analysis_cpp",
OverlayAnalysisCsharp = "overlay_analysis_csharp",
/** Disable TRAP caching when overlay analysis is enabled. */
OverlayAnalysisDisableTrapCaching = "overlay_analysis_disable_trap_caching",
OverlayAnalysisGo = "overlay_analysis_go",
OverlayAnalysisJava = "overlay_analysis_java",
OverlayAnalysisJavascript = "overlay_analysis_javascript",
@@ -214,6 +221,11 @@ export const featureConfig = {
envVar: "CODEQL_ACTION_ALLOW_MULTIPLE_ANALYSIS_KINDS",
minimumVersion: undefined,
},
[Feature.CleanupToolcacheBundles]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CLEANUP_TOOLCACHE_BUNDLES",
minimumVersion: undefined,
},
[Feature.CleanupTrapCaches]: {
defaultValue: false,
envVar: "CODEQL_ACTION_CLEANUP_TRAP_CACHES",
@@ -367,6 +379,11 @@ export const featureConfig = {
minimumVersion: CODEQL_OVERLAY_MINIMUM_VERSION_RUBY,
},
// Other overlay-related feature flags
[Feature.OverlayAnalysisDisableTrapCaching]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_DISABLE_TRAP_CACHING",
minimumVersion: undefined,
},
[Feature.OverlayAnalysisMatchCodeqlVersion]: {
defaultValue: false,
envVar: "CODEQL_ACTION_OVERLAY_ANALYSIS_MATCH_CODEQL_VERSION",
@@ -582,7 +599,7 @@ class OfflineFeatures implements FeatureEnablement {
// Never use this feature if the CLI version explicitly can't support it.
const minimumVersion = config.minimumVersion;
if (codeql && minimumVersion) {
if (!(await codeql.isVersionAtLeast(minimumVersion))) {
if (!(await util.codeQlVersionAtLeast(codeql, minimumVersion))) {
this.logger.debug(
`Feature ${feature} is disabled because the CodeQL CLI version is older than the minimum ` +
`version ${minimumVersion}.`,
+2 -2
View File
@@ -8,7 +8,7 @@ import {
LoggedMessage,
setupTests,
} from "../testing-utils";
import { getEnv, GitHubVariant, withTmpDir } from "../util";
import { GitHubVariant, initializeEnvironment, withTmpDir } from "../util";
import {
assertAllFeaturesHaveDefaultValues,
@@ -18,7 +18,7 @@ import {
setupTests(test);
test.beforeEach(() => {
getEnv().initialise("1.2.3");
initializeEnvironment("1.2.3");
});
test("OfflineFeatures makes no API requests", async (t) => {
+1 -15
View File
@@ -1,19 +1,5 @@
import { ActionsEnvVars, type ReadOnlyEnv } from "./environment";
import { ConfigurationError } from "./errors";
export enum GitHubVariant {
/** [GitHub.com](https://github.com) */
DOTCOM = "GitHub.com",
/** [GitHub Enterprise Server](https://docs.github.com/en/enterprise-server@latest/admin/overview/about-github-enterprise-server) */
GHES = "GitHub Enterprise Server",
/** [GitHub Enterprise Cloud with data residency](https://docs.github.com/en/enterprise-cloud@latest/admin/data-residency/about-github-enterprise-cloud-with-data-residency) */
GHEC_DR = "GitHub Enterprise Cloud with data residency",
}
export type GitHubVersion =
| { type: GitHubVariant.DOTCOM }
| { type: GitHubVariant.GHEC_DR }
| { type: GitHubVariant.GHES; version: string };
import { ConfigurationError } from "./util";
/** A string representing a GitHub instance address. */
export type InstanceURL = string;
-1
View File
@@ -793,7 +793,6 @@ async function testFailedSarifUpload(
}
t.true(
uploadFiles.calledOnceWith(
config,
sinon.match.string,
sinon.match.string,
category,
+14 -35
View File
@@ -44,7 +44,6 @@ import {
getRequiredEnvParam,
parseMatrixInput,
Result,
shouldSkipGenerateFailedSarif,
shouldSkipSarifUpload,
Success,
wrapError,
@@ -116,7 +115,7 @@ async function prepareFailedSarif(
const jobName = getRequiredEnvParam("GITHUB_JOB");
const matrix = parseMatrixInput(actionsUtil.getRequiredInput("matrix"));
if (shouldSkipGenerateFailedSarif()) {
if (shouldSkipSarifUpload()) {
return new Failure({
upload_failed_run_skipped_because: "SARIF upload is disabled",
});
@@ -227,7 +226,6 @@ async function maybeUploadFailedSarif(
logger.info(`Uploading failed SARIF file ${failedSarif.sarifFile}`);
const uploadResult = await uploadLib.uploadFiles(
config,
failedSarif.sarifFile,
failedSarif.checkoutPath,
failedSarif.category,
@@ -241,14 +239,6 @@ async function maybeUploadFailedSarif(
logger,
{ isUnsuccessfulExecution: true },
);
// If SARIF upload is disabled for testing, `uploadFiles` will not have uploaded the failed
// SARIF file and instead will have written it to disk. Upload it as a workflow artifact so that
// we can inspect it in validation jobs.
if (uploadResult.sarifID && shouldSkipSarifUpload()) {
await uploadSarifArtifact(logger, "test-mode-sarif", failedSarif);
}
return uploadResult
? { ...uploadResult.statusReport, sarifID: uploadResult.sarifID }
: {};
@@ -267,34 +257,23 @@ async function maybeUploadFailedSarifArtifact(
}
const failedSarif = failedSarifResult.value;
return uploadSarifArtifact(logger, "sarif-artifact", failedSarif);
}
/** Uploads a failed SARIF file as workflow artifact. */
async function uploadSarifArtifact(
logger: Logger,
prefix: string,
failedSarif: FailedSarifInfo,
): Promise<UploadFailedSarifResult> {
logger.startGroup(`Uploading failed SARIF file as artifact`);
logger.info(`Uploading '${failedSarif.sarifFile}' as artifact`);
logger.info(
`Uploading failed SARIF file ${failedSarif.sarifFile} as artifact`,
);
try {
const gitHubVersion = await getGitHubVersion();
const client = await getArtifactUploaderClient(logger, gitHubVersion.type);
const gitHubVersion = await getGitHubVersion();
const client = await getArtifactUploaderClient(logger, gitHubVersion.type);
const suffix = getArtifactSuffix(actionsUtil.getOptionalInput("matrix"));
const name = sanitizeArtifactName(`${prefix}-${suffix}`);
await client.uploadArtifact(
name,
[path.normalize(failedSarif.sarifFile)],
path.normalize(path.dirname(failedSarif.sarifFile)),
);
const suffix = getArtifactSuffix(actionsUtil.getOptionalInput("matrix"));
const name = sanitizeArtifactName(`sarif-artifact-${suffix}`);
await client.uploadArtifact(
name,
[path.normalize(failedSarif.sarifFile)],
path.normalize(".."),
);
return { sarifID: name };
} finally {
logger.endGroup();
}
return { sarifID: name };
}
/**
-3
View File
@@ -76,9 +76,6 @@ async function run(action: ActionState<["Base", "Logger", "Env"]>) {
// Try to load the "failed" configuration file instead which may have been written by the `init` step
// with a minimal configuration for this post-action.
if (config === undefined) {
logger.debug(
"Will try to load the configuration file from an aborted 'init' step, since there is no regular one.",
);
config = getAbortedInitConfig(action, tmpDir);
}
+4 -2
View File
@@ -88,10 +88,12 @@ import {
checkDiskUsage,
checkForTimeout,
checkGitHubVersionInRange,
codeQlVersionAtLeast,
DEFAULT_DEBUG_ARTIFACT_NAME,
DEFAULT_DEBUG_DATABASE_NAME,
getCodeQLMemoryLimit,
getThreadsFlagValue,
initializeEnvironment,
ConfigurationError,
wrapError,
checkActionVersion,
@@ -224,7 +226,7 @@ async function run(
let toolsVersion: string;
try {
actionState.env.initialise(getActionVersion());
initializeEnvironment(getActionVersion());
// Make inputs accessible in the `post` step.
persistInputs();
@@ -684,7 +686,7 @@ async function run(
`${EnvVar.JAVA_EXTRACTOR_MINIMIZE_DEPENDENCY_JARS} is already set to '${process.env[EnvVar.JAVA_EXTRACTOR_MINIMIZE_DEPENDENCY_JARS]}', so the Action will not override it.`,
);
} else if (
(await codeql.isVersionAtLeast(CODEQL_VERSION_JAR_MINIMIZATION)) &&
(await codeQlVersionAtLeast(codeql, CODEQL_VERSION_JAR_MINIMIZATION)) &&
config.dependencyCachingEnabled &&
config.buildMode === BuildMode.None &&
config.languages.includes(BuiltInLanguage.java)
+2 -1
View File
@@ -38,6 +38,7 @@ import {
checkDiskUsage,
checkForTimeout,
checkGitHubVersionInRange,
initializeEnvironment,
ConfigurationError,
wrapError,
checkActionVersion,
@@ -107,7 +108,7 @@ async function run(
let toolsVersion: string;
try {
actionState.env.initialise(getActionVersion());
initializeEnvironment(getActionVersion());
const apiDetails = {
auth: getRequiredInput("token"),
+87 -33
View File
@@ -11,13 +11,7 @@ import * as sinon from "sinon";
import * as actionsUtil from "./actions-util";
import * as api from "./api-client";
import * as diagnostics from "./diagnostics";
import {
ActionsEnvVars,
Env,
EnvVar,
getEnv,
ReadOnlyEnv,
} from "./environment";
import { ActionsEnvVars, Env, EnvVar, ReadOnlyEnv } from "./environment";
import { Feature } from "./feature-flags";
import { BuiltInLanguage } from "./languages";
import { getRunnerLogger } from "./logging";
@@ -38,17 +32,24 @@ import {
getTestEnv,
initAllState,
makeMacro,
mockApiDetails,
mockBundleDownloadApi,
setupActionsVars,
setupTests,
} from "./testing-utils";
import * as toolsDownload from "./tools-download";
import { getErrorMessage, GitHubVariant, HTTPError, withTmpDir } from "./util";
import {
getErrorMessage,
GitHubVariant,
HTTPError,
initializeEnvironment,
withTmpDir,
} from "./util";
setupTests(test);
test.beforeEach(() => {
getEnv().initialise("1.2.3");
initializeEnvironment("1.2.3");
});
function stubDownloadAndExtract() {
@@ -155,6 +156,7 @@ test.serial(
compressionMethod: "gzip",
sourceType: "download",
toolsVersion: "1.2.3",
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
} satisfies setupCodeql.CodeQLDownloadSource);
});
},
@@ -221,6 +223,7 @@ for (const {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const source = await setupCodeql.getCodeQLSource(
"linked",
SAMPLE_DEFAULT_CLI_VERSION,
@@ -255,6 +258,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const source = await setupCodeql.getCodeQLSource(
"latest",
SAMPLE_DEFAULT_CLI_VERSION,
@@ -298,6 +302,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const result = await setupCodeql.setupCodeQLBundle(
"linked",
SAMPLE_DOTCOM_API_DETAILS,
@@ -431,6 +436,7 @@ test.serial(
compressionMethod: "zstd",
sourceType: "download",
toolsVersion: expectedVersion,
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
} satisfies setupCodeql.CodeQLToolsSource);
const result = await setupCodeql.setupCodeQLBundle(
@@ -479,6 +485,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir, { GITHUB_EVENT_NAME: "dynamic" });
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const source = await setupCodeql.getCodeQLSource(
undefined,
@@ -503,6 +510,7 @@ test.serial(
compressionMethod: "zstd",
sourceType: "download",
toolsVersion: expectedVersion,
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
} satisfies setupCodeql.CodeQLToolsSource);
// Afterwards, ensure that we see the expected messages in the log.
@@ -596,7 +604,7 @@ test.serial(
undefined, // rawLanguages
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
createFeatures([Feature.CleanupToolcacheBundles]),
getRunnerLogger(true),
);
@@ -624,6 +632,7 @@ for (const toolsInput of ["nightly", "nightly-latest"]) {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const source = await setupCodeql.getCodeQLSource(
toolsInput,
SAMPLE_DEFAULT_CLI_VERSION,
@@ -649,6 +658,7 @@ for (const toolsInput of ["nightly", "nightly-latest"]) {
cliVersion: undefined,
compressionMethod: "zstd",
toolsVersion: "0.0.0-30260213",
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
} satisfies setupCodeql.CodeQLDownloadSource);
t.true(latestNightlyRequest.calledOnce);
});
@@ -801,6 +811,7 @@ for (const date of ["20200101", "30260213"]) {
cliVersion: undefined,
compressionMethod: "zstd",
toolsVersion: `0.0.0-${date}`,
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
} satisfies setupCodeql.CodeQLDownloadSource);
await t.throwsAsync(
@@ -945,6 +956,7 @@ const toolcacheInputFallbackMacro = makeMacro({
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
for (const [k, v] of Object.entries(environment)) {
process.env[k] = v;
@@ -1165,6 +1177,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const source = await setupCodeql.getCodeQLSource(
undefined,
PER_LANGUAGE_CLI_VERSION,
@@ -1207,6 +1220,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
for (const { otherLanguagePacksReason, features } of [
// The per-language feature is disabled.
{ otherLanguagePacksReason: undefined, features: createFeatures([]) },
@@ -1276,7 +1290,14 @@ for (const fallback of [false, true]) {
request: { fetch: fetchRelease },
});
sinon.stub(api, "getApiClient").value(() => client);
const authorizationSpy = sinon.spy(api, "getAuthorizationHeaderFor");
const spyClient = new api.InstanceApiClient(SAMPLE_DOTCOM_API_DETAILS);
const authorizationSpy = sinon.spy(
spyClient,
"getAuthorizationHeaderFor",
);
sinon.stub(api, "InstanceApiClient").returns(spyClient);
const extractStub = stubDownloadAndExtract();
if (fallback) {
extractStub.onFirstCall().rejects(new HTTPError("Not Found", 404));
@@ -1286,6 +1307,7 @@ for (const fallback of [false, true]) {
setupActionsVars(tmpDir, tmpDir, {
GITHUB_ACTION_REPOSITORY: "codeql-testing/action-fork",
});
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
const result = await setupCodeql.setupCodeQLBundle(
undefined,
SAMPLE_DOTCOM_API_DETAILS,
@@ -1308,9 +1330,9 @@ for (const fallback of [false, true]) {
t.is(extractStub.firstCall.args[0], assetURL);
t.is(extractStub.lastCall.args[0], fallback ? combinedURL : assetURL);
t.is(authorizationSpy.callCount, extractStub.callCount);
t.is(authorizationSpy.firstCall.args[2], assetURL);
t.is(authorizationSpy.firstCall.args[1], assetURL);
t.is(
authorizationSpy.lastCall.args[2],
authorizationSpy.lastCall.args[1],
fallback ? combinedURL : assetURL,
);
t.is(extractStub.lastCall.args[3], "token token");
@@ -1375,7 +1397,10 @@ for (const bundle of ["per-language", "combined", "fallback"] as const) {
};
});
const addDiagnostic = sinon.stub(diagnostics, "addNoLanguageDiagnostic");
const features = createFeatures([Feature.PerLanguageBundles]);
const features = createFeatures([
Feature.PerLanguageBundles,
Feature.CleanupToolcacheBundles,
]);
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
@@ -1550,6 +1575,7 @@ for (const error of [
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
await t.throwsAsync(
setupCodeql.setupCodeQLBundle(
undefined,
@@ -1823,15 +1849,11 @@ function createToolcacheEntry(
*/
async function runDownloadCodeQL(
toolcacheRoot: string,
features: Feature[],
bundleVersion: string | undefined,
{
env = getTestEnv({
[ActionsEnvVars.RUNNER_ENVIRONMENT]: "github-hosted",
[ActionsEnvVars.RUNNER_TOOL_CACHE]: toolcacheRoot,
}),
}: {
env?: Env;
} = {},
env = getTestEnv({
[ActionsEnvVars.RUNNER_TOOL_CACHE]: toolcacheRoot,
}),
): Promise<{
codeqlFolder: string;
cleanupDiagnostic: toolsDownload.ToolcacheCleanupResult | undefined;
@@ -1842,6 +1864,7 @@ async function runDownloadCodeQL(
const { codeqlFolder } = await setupCodeql.downloadCodeQLBundle(
initAllState({
env,
features: createFeatures(features),
}),
{
bundle: {
@@ -1853,8 +1876,8 @@ async function runDownloadCodeQL(
cliVersion: CLEANUP_CLI_VERSION,
sourceType: "download",
toolsVersion: CLEANUP_CLI_VERSION,
apiDetails: SAMPLE_DOTCOM_API_DETAILS,
},
SAMPLE_DOTCOM_API_DETAILS,
undefined, // tarVersion
toolcacheRoot, // tempDir
);
@@ -1879,9 +1902,11 @@ async function runDownloadCodeQL(
async function testToolcacheCleanup(
t: ExecutionContext<unknown>,
{
features,
runnerEnvironment,
setUp,
}: {
features: Feature[];
runnerEnvironment: string | undefined;
setUp?: (env: Env) => void;
},
@@ -1893,8 +1918,12 @@ async function testToolcacheCleanup(
) {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
if (runnerEnvironment === undefined) {
delete process.env[ActionsEnvVars.RUNNER_ENVIRONMENT];
} else {
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = runnerEnvironment;
}
const env = getTestEnv({
[ActionsEnvVars.RUNNER_ENVIRONMENT]: runnerEnvironment,
[ActionsEnvVars.RUNNER_TOOL_CACHE]: tmpDir,
});
setUp?.(env);
@@ -1914,8 +1943,9 @@ async function testToolcacheCleanup(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
features,
CLEANUP_BUNDLE_VERSION,
{ env },
env,
);
t.true(
@@ -1927,12 +1957,28 @@ async function testToolcacheCleanup(
});
}
test.serial(
"downloadCodeQLBundle does not clean up the toolcache when the feature flag is disabled",
async (t) => {
await testToolcacheCleanup(
t,
{ features: [], runnerEnvironment: "github-hosted" },
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
t.true(fs.existsSync(staleDirectory));
t.true(fs.existsSync(destinationDirectory));
t.is(cleanupDiagnostic, undefined);
},
);
},
);
test.serial(
"downloadCodeQLBundle does not clean up the toolcache when the runner is not GitHub-hosted",
async (t) => {
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "self-hosted",
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
@@ -1952,6 +1998,7 @@ test.serial(
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: undefined,
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
@@ -1964,11 +2011,12 @@ test.serial(
);
test.serial(
"downloadCodeQLBundle deletes other CodeQL bundles from the toolcache on a GitHub-hosted runner",
"downloadCodeQLBundle deletes other CodeQL bundles from the toolcache when enabled on a GitHub-hosted runner",
async (t) => {
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "github-hosted",
},
({ cleanupDiagnostic, destinationDirectory, staleDirectory }) => {
@@ -2002,6 +2050,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2031,6 +2080,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2162,6 +2212,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
toolcacheRoot,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2184,6 +2235,7 @@ test.serial(
await testToolcacheCleanup(
t,
{
features: [Feature.CleanupToolcacheBundles],
runnerEnvironment: "github-hosted",
setUp: (env) => {
env.set(EnvVar.HAS_SET_UP_CODEQL, "true");
@@ -2213,13 +2265,9 @@ test.serial(
);
const { codeqlFolder, cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
{
env: getTestEnv({
[ActionsEnvVars.RUNNER_ENVIRONMENT]: "github-hosted",
[EnvVar.HAS_SET_UP_CODEQL]: "true",
}),
},
getTestEnv({ [EnvVar.HAS_SET_UP_CODEQL]: "true" }),
);
t.true(fs.existsSync(staleDirectory));
@@ -2234,6 +2282,7 @@ test.serial(
async (t) => {
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
delete process.env[EnvVar.HAS_SET_UP_CODEQL];
stubDownloadAndExtract();
@@ -2269,6 +2318,7 @@ test.serial(
await withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
mockApiDetails(SAMPLE_DOTCOM_API_DETAILS);
process.env[ActionsEnvVars.RUNNER_ENVIRONMENT] = "github-hosted";
delete process.env[EnvVar.HAS_SET_UP_CODEQL];
const staleDirectory = createToolcacheEntry(
@@ -2287,7 +2337,7 @@ test.serial(
undefined, // rawLanguages
undefined, // otherLanguagePacksReason
false, // useOverlayAwareDefaultCliVersion
createFeatures([]),
createFeatures([Feature.CleanupToolcacheBundles]),
getRunnerLogger(true),
),
{ is: error },
@@ -2322,6 +2372,7 @@ test.serial(
const { codeqlFolder, cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
undefined, // bundleVersion
);
@@ -2355,6 +2406,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2390,6 +2442,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
tmpDir,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
@@ -2439,6 +2492,7 @@ test.serial(
const { cleanupDiagnostic } = await runDownloadCodeQL(
toolcacheRoot,
[Feature.CleanupToolcacheBundles],
CLEANUP_BUNDLE_VERSION,
);
+14 -21
View File
@@ -718,6 +718,7 @@ export async function getCodeQLSource(
compressionMethod,
sourceType: "download",
toolsVersion: resolvedVersion ?? "unknown",
apiDetails,
};
}
@@ -746,7 +747,6 @@ async function tryGetFallbackToolcacheVersion(
// be able to stub this function and have other functions in this file use that stub.
export const downloadCodeQL = async function (
source: CodeQLDownloadSource,
apiDetails: api.GitHubApiDetails,
tarVersion: tar.TarVersion | undefined,
tempDir: string,
logger: Logger,
@@ -767,11 +767,8 @@ export const downloadCodeQL = async function (
if (searchParams.has("token")) {
logger.debug("CodeQL tools URL contains an authorization token.");
} else {
authorization = api.getAuthorizationHeaderFor(
logger,
apiDetails,
codeqlURL,
);
const client = new api.InstanceApiClient(source.apiDetails);
authorization = client.getAuthorizationHeaderFor(logger, codeqlURL);
}
const toolcacheDestination = getToolcacheDestination({ logger }, source);
@@ -839,7 +836,7 @@ function getToolcacheDestination(
}
/**
* Reclaims disk space by deleting the CodeQL tools from the toolcache on GitHub-hosted runners.
* Reclaims disk space by deleting the CodeQL tools from the toolcache, if enabled.
*
* On GitHub-hosted runners the toolcache shares a filesystem with the workspace, so tools left in
* the toolcache take up space that the analysis could use instead. This holds wherever we extract
@@ -847,8 +844,9 @@ function getToolcacheDestination(
*/
async function tryDeleteToolcacheBundles({
env,
features,
logger,
}: ActionState<["Logger", "ReadOnlyEnv"]>): Promise<void> {
}: ActionState<["Logger", "ReadOnlyEnv", "FeatureFlags"]>): Promise<void> {
// A step that has already set up CodeQL may hand out a path into the toolcache that a later step
// runs, so only the first step to set it up can know that nothing else relies on the toolcache.
if (env.getOptional(EnvVar.HAS_SET_UP_CODEQL) !== undefined) {
@@ -859,7 +857,11 @@ async function tryDeleteToolcacheBundles({
return;
}
if (!isGitHubHostedRunner(env) || !isToolcacheOnWorkspaceFilesystem(logger)) {
if (
!isGitHubHostedRunner() ||
!isToolcacheOnWorkspaceFilesystem(logger) ||
!(await features.getValue(Feature.CleanupToolcacheBundles))
) {
return;
}
@@ -978,9 +980,8 @@ export async function setupCodeQLBundle(
break;
case "download": {
const result = await downloadCodeQLBundle(
{ env: getEnv(), logger },
{ env: getEnv(), features, logger },
source,
apiDetails,
zstdAvailability.version,
tempDir,
);
@@ -1014,9 +1015,8 @@ export async function setupCodeQLBundle(
* @returns The extraction directory and download timings.
*/
export async function downloadCodeQLBundle(
action: ActionState<["Logger", "ReadOnlyEnv"]>,
action: ActionState<["Logger", "ReadOnlyEnv", "FeatureFlags"]>,
source: CodeQLDownloadSource,
apiDetails: api.GitHubApiDetails,
tarVersion: tar.TarVersion | undefined,
tempDir: string,
): Promise<{
@@ -1030,13 +1030,7 @@ export async function downloadCodeQLBundle(
const startTime = performance.now();
try {
return await downloadCodeQL(
source,
apiDetails,
tarVersion,
tempDir,
logger,
);
return await downloadCodeQL(source, tarVersion, tempDir, logger);
} catch (e) {
if (
bundle.kind !== "per-language" ||
@@ -1052,7 +1046,6 @@ export async function downloadCodeQLBundle(
...source,
bundle: { kind: "combined", url: bundle.combinedBundleURL },
},
apiDetails,
tarVersion,
tempDir,
logger,
+2 -1
View File
@@ -6,6 +6,7 @@ import * as sinon from "sinon";
import * as actionsUtil from "../actions-util";
import { ActionsEnvVars } from "../environment";
import { callee, getTestEnv, setupTests } from "../testing-utils";
import { initializeEnvironment } from "../util";
import { getCodeQLActionRepository } from "./repository";
@@ -15,7 +16,7 @@ test.serial("getCodeQLActionRepository", async (t) => {
const initialEnv = getTestEnv({
[ActionsEnvVars.RUNNER_TEMP]: path.dirname(__dirname),
});
initialEnv.initialise("1.2.3");
initializeEnvironment("1.2.3", initialEnv);
const target = callee(getCodeQLActionRepository)
.withArgs()
+27 -25
View File
@@ -2,7 +2,7 @@ import { ActionState } from "../action-common";
import { isRunningLocalAction } from "../actions-util";
import * as api from "../api-client";
import { ActionsEnvVars } from "../environment";
import { GITHUB_DOTCOM_URL, type InstanceURL } from "../github";
import { GITHUB_DOTCOM_URL } from "../github";
import {
isRepositoryNwoStr,
parseRepositoryNwo,
@@ -47,20 +47,14 @@ export function getCodeQLActionRepository(
}
/**A download source is a pair of strings. */
export type DownloadSource = [InstanceURL, RepositoryNwoStr];
/** GitHub.com, and the canonical Action. */
export const DEFAULT_DOWNLOAD_SOURCE: DownloadSource = [
GITHUB_DOTCOM_URL,
CODEQL_DEFAULT_ACTION_REPOSITORY,
];
export type DownloadSource = [api.GitHubApiDetails, RepositoryNwoStr];
/** Decides if the two provided download sources are the same. */
const isSameDownloadSource = (
[srcUrl, srcRepo]: DownloadSource,
[otherUrl, otherRepo]: DownloadSource,
[srcInstance, srcRepo]: DownloadSource,
[otherInstance, otherRepo]: DownloadSource,
) => {
return srcUrl === otherUrl && srcRepo === otherRepo;
return srcInstance.url === otherInstance.url && srcRepo === otherRepo;
};
/** A download URL is represented as a string. */
@@ -103,13 +97,21 @@ export async function getCodeQLAssetDownloadURL(
): Promise<DownloadURL> {
const codeQLActionRepository = getCodeQLActionRepository(action);
// We construct the default GitHub.com source here, because `getDotcomApiDetails`
// may retrieve the `token` input if we are currently running on GitHub.com and
// we want to avoid doing that statically.
const defaultDownloadSource: DownloadSource = [
api.getDotcomApiDetails(action.env),
CODEQL_DEFAULT_ACTION_REPOSITORY,
];
const potentialDownloadSources: DownloadSource[] = [
// This GitHub instance, and this Action.
[apiDetails.url, codeQLActionRepository],
[apiDetails, codeQLActionRepository],
// This GitHub instance, and the canonical Action.
[apiDetails.url, CODEQL_DEFAULT_ACTION_REPOSITORY],
[apiDetails, CODEQL_DEFAULT_ACTION_REPOSITORY],
// GitHub.com, and the canonical Action.
DEFAULT_DOWNLOAD_SOURCE,
defaultDownloadSource,
];
// We now filter out any duplicates.
@@ -122,34 +124,34 @@ export async function getCodeQLAssetDownloadURL(
},
);
for (const [instanceURL, repository] of uniqueDownloadSources) {
for (const [instance, repository] of uniqueDownloadSources) {
// If we've reached the final case, short-circuit the API check since we know the asset exists and is public.
if (
isSameDownloadSource(DEFAULT_DOWNLOAD_SOURCE, [instanceURL, repository])
) {
if (isSameDownloadSource(defaultDownloadSource, [instance, repository])) {
break;
}
const { owner: repositoryOwner, repo: repositoryName } =
parseRepositoryNwo(repository);
try {
const release = await api.getApiClient().rest.repos.getReleaseByTag({
owner: repositoryOwner,
repo: repositoryName,
tag: tagName,
});
const release = await api
.getApiClient(action.env, instance)
.rest.repos.getReleaseByTag({
owner: repositoryOwner,
repo: repositoryName,
tag: tagName,
});
for (const asset of release.data.assets) {
if (asset.name === assetName) {
action.logger.info(
`Found ${assetKind} ${assetName} in ${repository} on ${instanceURL} with URL ${asset.url}.`,
`Found ${assetKind} ${assetName} in ${repository} on ${instance.apiURL} with URL ${asset.url}.`,
);
return asset.url;
}
}
} catch (e) {
action.logger.info(
`Looked for ${assetKind} ${assetName} in ${repository} on ${instanceURL} but got error ${e}.`,
`Looked for ${assetKind} ${assetName} in ${repository} on ${instance.apiURL} but got error ${e}.`,
);
}
}
+51 -22
View File
@@ -8,7 +8,7 @@ import { ActionState } from "./action-common";
import * as actionsUtil from "./actions-util";
import * as apiClient from "./api-client";
import * as defaults from "./defaults.json";
import { ActionsEnvVars } from "./environment";
import { ActionsEnvVars, type Env } from "./environment";
import { setUpFeatureFlagTests } from "./feature-flags/testing-util";
import { UnvalidatedObject, validateSchema } from "./json";
import { makeFromSchema } from "./json/testing-util";
@@ -25,6 +25,7 @@ import {
initAllState,
makeMacro,
makeTestToken,
mockApiDetails,
RecordingLogger,
SAMPLE_DOTCOM_API_DETAILS,
setupTests,
@@ -613,18 +614,23 @@ test("getProxyPackage - includes platform in name", (t) => {
}
});
function mockGetApiClient(endpoints: any) {
return (
sinon
.stub(apiClient, "getApiClient")
// eslint-disable-next-line @typescript-eslint/no-unsafe-argument
.returns({ rest: endpoints } as any)
);
function mockGetApiClient(env: Env, endpoints: any) {
const getRequiredInput = mockApiDetails(SAMPLE_DOTCOM_API_DETAILS, env);
const getApiClient = sinon
.stub(apiClient, "getApiClient")
// eslint-disable-next-line @typescript-eslint/no-unsafe-argument
.returns({ rest: endpoints } as any);
return { getRequiredInput, getApiClient };
}
type ReleaseAssets = Array<{ name: string; url?: string }>;
function mockGetReleaseByTag(expectedTag?: string, assets?: ReleaseAssets) {
function mockGetReleaseByTag(
env: Env,
expectedTag?: string,
assets?: ReleaseAssets,
) {
const getReleaseByTag =
assets === undefined
? sinon.stub().rejects()
@@ -645,7 +651,7 @@ function mockGetReleaseByTag(expectedTag?: string, assets?: ReleaseAssets) {
return {
getReleaseByTag,
getApiClient: mockGetApiClient({ repos: { getReleaseByTag } }),
...mockGetApiClient(env, { repos: { getReleaseByTag } }),
};
}
@@ -671,7 +677,6 @@ test.serial(
"getDownloadUrl logs unknown platforms/arch and defaults to linux64",
async (t) => {
const logger = new RecordingLogger();
mockGetReleaseByTag();
await withTmpDir(async (tempDir) => {
const features = mockOfflineFeatures(tempDir, logger);
@@ -681,6 +686,7 @@ test.serial(
logger,
features,
});
mockGetReleaseByTag(state.env);
// Stub this to always return `true` so we get the default repo for the tests.
sinon.stub(actionsUtil, "isRunningLocalAction").returns(true);
@@ -710,11 +716,11 @@ test.serial(
"getDownloadUrl returns default bundle URL for linked version",
async (t) => {
const logger = new RecordingLogger();
mockGetReleaseByTag();
await withTmpDir(async (tempDir) => {
const features = mockOfflineFeatures(tempDir, logger);
const state = initAllState({ logger, features });
mockGetReleaseByTag(state.env);
// Stub this to always return `true` so we get the default repo for the tests.
sinon.stub(actionsUtil, "isRunningLocalAction").returns(true);
@@ -750,7 +756,11 @@ test.serial(
sinon.stub(actionsUtil, "isRunningLocalAction").returns(true);
for (const assets of testAssets) {
const stubs = mockGetReleaseByTag(defaults.bundleVersion, assets);
const stubs = mockGetReleaseByTag(
state.env,
defaults.bundleVersion,
assets,
);
const info = await startProxyExports.getDownloadUrl(
state,
SAMPLE_DOTCOM_API_DETAILS,
@@ -768,6 +778,7 @@ test.serial(
// mockGetReleaseByTag stubs the global getApiClient; since we are calling
// that in a loop, we need to manually restore it here
stubs.getApiClient.restore();
stubs.getRequiredInput.restore();
}
});
},
@@ -783,7 +794,7 @@ test.serial("getDownloadUrl returns matching release asset", async (t) => {
url: "url-we-want",
},
];
const stubs = mockGetReleaseByTag(defaults.bundleVersion, assets);
const stubs = mockGetReleaseByTag(state.env, defaults.bundleVersion, assets);
await withTmpDir(async (tempDir) => {
state.env.set(ActionsEnvVars.RUNNER_TEMP, tempDir);
@@ -925,7 +936,8 @@ test.serial(
"getProxyBinaryPath - returns path from tool cache if available",
async (t) => {
const logger = new RecordingLogger();
mockGetReleaseByTag();
sinon.stub(apiClient, "getApiDetails").returns(SAMPLE_DOTCOM_API_DETAILS);
await withTmpDir(async (tempDir) => {
const toolcachePath = "/path/to/proxy/dir";
@@ -935,8 +947,11 @@ test.serial(
sinon.stub(actionsUtil, "isRunningLocalAction").returns(true);
const features = mockOfflineFeatures(tempDir, logger);
const state = initAllState({ logger, features });
mockGetReleaseByTag(state.env);
const result = await startProxyExports.getProxyBinaryPath(
initAllState({ logger, features }),
state,
SAMPLE_DOTCOM_API_DETAILS,
);
@@ -956,7 +971,7 @@ test.serial(
const logger = new RecordingLogger();
const state = initAllState({ logger });
const downloadUrl = "url-we-want";
const stubs = mockGetReleaseByTag(defaults.bundleVersion, [
const stubs = mockGetReleaseByTag(state.env, defaults.bundleVersion, [
{
name: startProxyExports.getProxyPackage(getTestPlatform(state)),
url: downloadUrl,
@@ -965,9 +980,16 @@ test.serial(
const toolcachePath = "/path/to/proxy/dir";
const find = sinon.stub(toolcache, "find").returns("");
const apiDetails = {
auth: "",
url: "",
apiURL: "",
} satisfies apiClient.GitHubApiDetails;
const client = new apiClient.InstanceApiClient(apiDetails);
const getAuthorizationHeaderFor = sinon
.stub(apiClient, "getAuthorizationHeaderFor")
.stub(client, "getAuthorizationHeaderFor")
.returns(undefined);
sinon.stub(apiClient, "InstanceApiClient").returns(client);
const archivePath = "/path/to/archive";
const downloadTool = sinon
.stub(toolcache, "downloadTool")
@@ -1007,7 +1029,7 @@ test.serial(
t.is(result.version, defaults.cliVersion);
checkExpectedLogMessages(t, logger.messages, [
`Found private registry proxy ${startProxyExports.getProxyPackage(getTestPlatform(state))} in test/repo on https://github.com with URL ${downloadUrl}`,
`Found private registry proxy ${startProxyExports.getProxyPackage(getTestPlatform(state))} in test/repo on https://api.github.com with URL ${downloadUrl}`,
]);
},
);
@@ -1039,14 +1061,21 @@ test.serial(
headers: {},
url: "GET /repos/:owner/:repo/releases/tags/:tag",
});
mockGetApiClient({ repos: { getReleaseByTag } });
mockGetApiClient(state.env, { repos: { getReleaseByTag } });
await withTmpDir(async (tempDir) => {
const toolcachePath = "/path/to/proxy/dir";
const find = sinon.stub(toolcache, "find").returns("");
const apiDetails = {
auth: "",
url: "",
apiURL: "",
} satisfies apiClient.GitHubApiDetails;
const client = new apiClient.InstanceApiClient(apiDetails);
const getAuthorizationHeaderFor = sinon
.stub(apiClient, "getAuthorizationHeaderFor")
.stub(client, "getAuthorizationHeaderFor")
.returns(undefined);
sinon.stub(apiClient, "InstanceApiClient").returns(client);
const archivePath = "/path/to/archive";
const downloadTool = sinon
.stub(toolcache, "downloadTool")
@@ -1107,7 +1136,7 @@ test.serial(
});
checkExpectedLogMessages(t, logger.messages, [
`Found private registry proxy ${startProxyExports.getProxyPackage(getTestPlatform(state))} in test/repo on https://github.com with URL url-we-want.`,
`Found private registry proxy ${startProxyExports.getProxyPackage(getTestPlatform(state))} in test/repo on https://api.github.com with URL url-we-want.`,
]);
},
);
+4 -4
View File
@@ -4,9 +4,9 @@ import * as toolcache from "@actions/tool-cache";
import { ActionState } from "./action-common";
import {
getAuthorizationHeaderFor,
getGitHubVersion,
GitHubApiDetails,
InstanceApiClient,
type GitHubApiDetails,
} from "./api-client";
import * as artifactScanner from "./artifact-scanner";
import * as defaults from "./defaults.json";
@@ -423,6 +423,7 @@ export async function getProxyBinaryPath(
logger.startGroup("Setting up authentication proxy");
try {
const client = new InstanceApiClient(apiDetails);
const proxyFileName = getProxyFilename();
const proxyInfo = await getDownloadUrl(action, apiDetails);
@@ -437,9 +438,8 @@ export async function getProxyBinaryPath(
`Authentication proxy is not in the toolcache. Will try to download it from '${proxyInfo.url}'.`,
);
const authorization = getAuthorizationHeaderFor(
const authorization = client.getAuthorizationHeaderFor(
logger,
apiDetails,
proxyInfo.url,
);
const temp = await downloadProxy(logger, proxyInfo.url, authorization);
+18
View File
@@ -14,6 +14,7 @@ import * as sinon from "sinon";
import { ActionState, StateFeature } from "./action-common";
import { ActionsEnv, getActionVersion } from "./actions-util";
import * as actionsUtil from "./actions-util";
import { AnalysisKind } from "./analyses";
import * as apiClient from "./api-client";
import { GitHubApiDetails } from "./api-client";
@@ -52,6 +53,23 @@ export const SAMPLE_DOTCOM_API_DETAILS = {
apiURL: "https://api.github.com",
};
export function mockApiDetails(
apiDetails: Required<GitHubApiDetails>,
env: Env = getEnv(),
) {
// This is a workaround to mock `api.getApiDetails()` since it doesn't seem to be possible to
// mock this directly. The difficulty is that `getApiDetails()` is called locally in
// `api-client.ts`, but `sinon.stub(api, "getApiDetails")` only affects calls to
// `getApiDetails()` via an imported `api` module.
const getRequiredInput = sinon.stub(actionsUtil, "getRequiredInput");
getRequiredInput.withArgs("token").returns(apiDetails.auth);
env.set("GITHUB_SERVER_URL", apiDetails.url);
env.set("GITHUB_API_URL", apiDetails.apiURL ?? "");
return getRequiredInput;
}
export const LINKED_CLI_VERSION = {
cliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
+1 -2
View File
@@ -10,7 +10,6 @@ import { https } from "follow-redirects";
import nock from "nock";
import * as sinon from "sinon";
import * as errors from "./errors";
import { getRunnerLogger } from "./logging";
import * as tar from "./tar";
import { setupTests } from "./testing-utils";
@@ -165,7 +164,7 @@ test.serial(
test.serial(
"downloadAndExtract handles an unknown status as a non-HTTP error",
async (t) => {
const asHTTPError = sinon.spy(errors, "asHTTPError");
const asHTTPError = sinon.spy(util, "asHTTPError");
await util.withTmpDir(async (tmpDir) => {
sinon.stub(process, "platform").value("linux");
const archivePath = path.join(tmpDir, "codeql-bundle.tar.zst");
+2 -3
View File
@@ -10,18 +10,17 @@ import * as analyses from "./analyses";
import { AnalysisKind, CodeQuality, CodeScanning } from "./analyses";
import * as api from "./api-client";
import * as diffUtils from "./diff-informed-analysis-utils";
import { getEnv } from "./environment";
import { getRunnerLogger, Logger } from "./logging";
import * as sarif from "./sarif";
import { setupTests } from "./testing-utils";
import * as uploadLib from "./upload-lib";
import { UploadPayload } from "./upload-lib/types";
import { GitHubVariant, withTmpDir } from "./util";
import { GitHubVariant, initializeEnvironment, withTmpDir } from "./util";
setupTests(test);
test.beforeEach(() => {
getEnv().initialise("1.2.3");
initializeEnvironment("1.2.3");
});
test.serial("validateSarifFileSchema - valid", (t) => {
+10 -34
View File
@@ -12,7 +12,7 @@ import * as analyses from "./analyses";
import * as api from "./api-client";
import { getGitHubVersion, wrapApiConfigurationError } from "./api-client";
import { CodeQL, getCodeQL } from "./codeql";
import type { ActionConfig } from "./config/action-config";
import { getConfig } from "./config-utils";
import { readDiffRangesJsonFile } from "./diff-informed-analysis-utils";
import { ActionsEnvVars, EnvVar, exportEnvVar } from "./environment";
import { FeatureEnablement } from "./feature-flags";
@@ -94,19 +94,15 @@ async function shouldDisableCombineSarifFiles(
return true;
}
/**
* Takes a list of paths to sarif files and combines them together using the
* CLI `github merge-results` command when all SARIF files are produced by
* CodeQL. Otherwise, it will fall back to combining the files in the action.
*
* @returns The contents of the combined sarif file.
*/
// Takes a list of paths to sarif files and combines them together using the
// CLI `github merge-results` command when all SARIF files are produced by
// CodeQL. Otherwise, it will fall back to combining the files in the action.
// Returns the contents of the combined sarif file.
async function combineSarifFilesUsingCLI(
sarifFiles: string[],
gitHubVersion: GitHubVersion,
features: FeatureEnablement,
logger: Logger,
config: ActionConfig<"codeQLCmd" | "tempDir"> | undefined,
): Promise<Partial<sarif.Log>> {
const env = util.getEnv();
@@ -144,6 +140,7 @@ async function combineSarifFilesUsingCLI(
let codeQL: CodeQL;
let tempDir: string = actionsUtil.getTemporaryDirectory();
const config = await getConfig(tempDir, logger);
if (config !== undefined) {
codeQL = await getCodeQL(logger, config.codeQLCmd);
tempDir = config.tempDir;
@@ -231,14 +228,6 @@ function getAutomationID(
return api.computeAutomationID(analysis_key, environment);
}
/** Gets the path to a saved payload file for {@link analysisKind}. */
export function getSavedPayloadPath(analysisKind: analyses.AnalysisKind) {
return path.join(
actionsUtil.getTemporaryDirectory(),
`payload-${analysisKind}.json`,
);
}
/**
* Upload the given payload.
* If the request fails then this will retry a small number of times.
@@ -253,7 +242,10 @@ export async function uploadPayload(
logger.info("Uploading results");
if (util.shouldSkipSarifUpload()) {
const payloadSaveFile = getSavedPayloadPath(analysis.kind);
const payloadSaveFile = path.join(
actionsUtil.getTemporaryDirectory(),
`payload-${analysis.kind}.json`,
);
logger.info(
`SARIF upload disabled by an environment variable. Saving to ${payloadSaveFile}`,
);
@@ -607,7 +599,6 @@ export interface PostProcessingResults {
export async function postProcessSarifFiles(
logger: Logger,
features: FeatureEnablement,
config: ActionConfig<"codeQLCmd" | "tempDir"> | undefined,
checkoutPath: string,
sarifPaths: string[],
category: string | undefined,
@@ -632,7 +623,6 @@ export async function postProcessSarifFiles(
gitHubVersion,
features,
logger,
config,
);
} else {
const sarifPath = sarifPaths[0];
@@ -693,7 +683,6 @@ export async function writePostProcessedFiles(
* to.
*/
export async function uploadFiles(
config: ActionConfig<"codeQLCmd" | "tempDir"> | undefined,
inputSarifPath: string,
checkoutPath: string,
category: string | undefined,
@@ -708,7 +697,6 @@ export async function uploadFiles(
return uploadSpecifiedFiles(
sarifPaths,
config,
checkoutPath,
category,
features,
@@ -722,7 +710,6 @@ export async function uploadFiles(
*/
async function uploadSpecifiedFiles(
sarifPaths: string[],
config: ActionConfig<"codeQLCmd" | "tempDir"> | undefined,
checkoutPath: string,
category: string | undefined,
features: FeatureEnablement,
@@ -732,7 +719,6 @@ async function uploadSpecifiedFiles(
const processingResults: PostProcessingResults = await postProcessSarifFiles(
logger,
features,
config,
checkoutPath,
sarifPaths,
category,
@@ -871,16 +857,6 @@ export async function waitForProcessing(
): Promise<void> {
logger.startGroup("Waiting for processing to finish");
try {
// If SARIF upload is disabled and we reach this point, then the file was written to disk and
// there is nothing to wait for. We still allow ourselves to get to this point so that we can
// see in the log that we would have waited for processing to finish if the upload had happened.
if (util.shouldSkipSarifUpload()) {
logger.info(
"Nothing to wait for, because SARIF upload is disabled by an environment variable and the SARIF was saved to disk.",
);
return;
}
const client = api.getApiClient();
// Do an initial wait because processing will always take a minimum of 2-3 seconds
+7 -14
View File
@@ -5,7 +5,6 @@ import * as actionsUtil from "./actions-util";
import { getActionVersion, getTemporaryDirectory } from "./actions-util";
import * as analyses from "./analyses";
import { getGitHubVersion } from "./api-client";
import { getConfig } from "./config-utils";
import { initFeatures } from "./feature-flags";
import { Logger } from "./logging";
import { getRepositoryNwo } from "./repository";
@@ -24,6 +23,7 @@ import {
ConfigurationError,
checkActionVersion,
checkDiskUsage,
initializeEnvironment,
shouldSkipSarifUpload,
wrapError,
} from "./util";
@@ -54,13 +54,11 @@ async function sendSuccessStatusReport(
}
}
async function run(action: ActionState<["Base", "Env", "Logger"]>) {
async function run({ startedAt, logger }: ActionState<["Base", "Logger"]>) {
// To capture errors appropriately, keep as much code within the try-catch as
// possible, and only use safe functions outside.
const logger = action.logger;
try {
action.env.initialise(getActionVersion());
initializeEnvironment(getActionVersion());
const gitHubVersion = await getGitHubVersion();
checkActionVersion(getActionVersion(), gitHubVersion);
@@ -68,19 +66,18 @@ async function run(action: ActionState<["Base", "Env", "Logger"]>) {
// Make inputs accessible in the `post` step.
actionsUtil.persistInputs();
const tempDir = getTemporaryDirectory();
const repositoryNwo = getRepositoryNwo();
const features = initFeatures(
gitHubVersion,
repositoryNwo,
tempDir,
getTemporaryDirectory(),
logger,
);
const startingStatusReportBase = await createStatusReportBase(
ActionName.UploadSarif,
"starting",
action.startedAt,
startedAt,
undefined,
await checkDiskUsage(logger),
logger,
@@ -94,13 +91,9 @@ async function run(action: ActionState<["Base", "Env", "Logger"]>) {
const checkoutPath = actionsUtil.getRequiredInput("checkout_path");
const category = actionsUtil.getOptionalInput("category");
// Load the configuration from a previous step if there is one. This allows us to reuse the CLI that
// may have been installed by a previous step.
const maybeConfig = await getConfig(tempDir, logger);
const uploadResults = await postProcessAndUploadSarif(
logger,
features,
maybeConfig,
"always",
checkoutPath,
sarifPath,
@@ -138,7 +131,7 @@ async function run(action: ActionState<["Base", "Env", "Logger"]>) {
// so we can't wait for that here.
}
await sendSuccessStatusReport(
action.startedAt,
startedAt,
codeScanningResult?.statusReport || {},
logger,
);
@@ -154,7 +147,7 @@ async function run(action: ActionState<["Base", "Env", "Logger"]>) {
const errorStatusReportBase = await createStatusReportBase(
ActionName.UploadSarif,
getActionsStatus(error),
action.startedAt,
startedAt,
undefined,
await checkDiskUsage(logger),
logger,
-6
View File
@@ -26,7 +26,6 @@ function mockPostProcessSarifFiles() {
const analysisConfig = getAnalysisConfig(analysisKind);
postProcessSarifFiles
.withArgs(
sinon.match.object,
sinon.match.any,
sinon.match.any,
sinon.match.any,
@@ -76,11 +75,9 @@ const postProcessAndUploadSarifMacro = makeMacro({
fs.writeFileSync(sarifFile, "");
}
const config = undefined;
const actual = await postProcessAndUploadSarif(
logger,
features,
config,
"always",
"",
testPath,
@@ -97,7 +94,6 @@ const postProcessAndUploadSarifMacro = makeMacro({
postProcessSarifFiles.calledWith(
logger,
features,
config,
sinon.match.any,
analysisKindResult.expectedFiles?.map(toFullPath) ??
fullSarifPaths,
@@ -227,7 +223,6 @@ test.serial(
const actual = await postProcessAndUploadSarif(
logger,
features,
undefined,
"never",
"",
tempDir,
@@ -258,7 +253,6 @@ test.serial(
const actual = await postProcessAndUploadSarif(
logger,
features,
undefined,
"never",
"",
tempDir,
-3
View File
@@ -1,6 +1,5 @@
import { UploadKind } from "./actions-util";
import * as analyses from "./analyses";
import type { ActionConfig } from "./config/action-config";
import { FeatureEnablement } from "./feature-flags";
import { Logger } from "./logging";
import * as upload_lib from "./upload-lib";
@@ -27,7 +26,6 @@ export type UploadSarifResults = Partial<
export async function postProcessAndUploadSarif(
logger: Logger,
features: FeatureEnablement,
config: ActionConfig<"codeQLCmd" | "tempDir"> | undefined,
uploadKind: UploadKind,
checkoutPath: string,
sarifPath: string,
@@ -47,7 +45,6 @@ export async function postProcessAndUploadSarif(
const postProcessingResults = await upload_lib.postProcessSarifFiles(
logger,
features,
config,
checkoutPath,
sarifFiles,
category,
+1 -68
View File
@@ -11,7 +11,7 @@ import * as sinon from "sinon";
import * as api from "./api-client";
import { EnvVar } from "./environment";
import { getRunnerLogger } from "./logging";
import { getTestEnv, setupTests } from "./testing-utils";
import { setupTests } from "./testing-utils";
import * as util from "./util";
setupTests(test);
@@ -493,70 +493,3 @@ test("Failure.orElse returns the default value for a failure result", (t) => {
const result = new util.Failure(new Error("test error"));
t.is(result.orElse("default value"), "default value");
});
test("shouldSkipGenerateFailedSarif", (t) => {
const env = getTestEnv();
// Don't skip by default
t.is(util.shouldSkipGenerateFailedSarif(env), false);
// Skip in test mode
env.setAll({
[EnvVar.TEST_MODE]: "true",
[EnvVar.SKIP_SARIF_UPLOAD]: undefined,
[EnvVar.SKIP_GENERATE_FAILED_SARIF]: undefined,
});
t.is(util.shouldSkipGenerateFailedSarif(env), true);
// Skip if explicitly requested
env.setAll({
[EnvVar.TEST_MODE]: undefined,
[EnvVar.SKIP_SARIF_UPLOAD]: undefined,
[EnvVar.SKIP_GENERATE_FAILED_SARIF]: "true",
});
t.is(util.shouldSkipGenerateFailedSarif(env), true);
// Skip if not uploading SARIF
env.setAll({
[EnvVar.TEST_MODE]: undefined,
[EnvVar.SKIP_SARIF_UPLOAD]: "true",
[EnvVar.SKIP_GENERATE_FAILED_SARIF]: undefined,
});
t.is(util.shouldSkipGenerateFailedSarif(env), true);
// Don't skip if explicitly requested
env.setAll({
[EnvVar.TEST_MODE]: "true",
[EnvVar.SKIP_SARIF_UPLOAD]: "true",
[EnvVar.SKIP_GENERATE_FAILED_SARIF]: "false",
});
t.is(util.shouldSkipGenerateFailedSarif(env), false);
});
test("shouldSkipSarifUpload", (t) => {
const env = getTestEnv();
// Don't skip by default
t.is(util.shouldSkipSarifUpload(env), false);
// Skip in test mode
env.setAll({
[EnvVar.TEST_MODE]: "true",
[EnvVar.SKIP_SARIF_UPLOAD]: undefined,
});
t.is(util.shouldSkipSarifUpload(env), true);
// Skip if explicitly requested
env.setAll({
[EnvVar.TEST_MODE]: undefined,
[EnvVar.SKIP_SARIF_UPLOAD]: "true",
});
t.is(util.shouldSkipSarifUpload(env), true);
// Don't skip if explicitly requested
env.setAll({
[EnvVar.TEST_MODE]: "true",
[EnvVar.SKIP_SARIF_UPLOAD]: "false",
});
t.is(util.shouldSkipSarifUpload(env), false);
});
+86 -28
View File
@@ -18,24 +18,22 @@ import type { Config } from "./config-utils";
import {
EnvVar,
getRequiredEnvParam,
isInTestMode,
exportEnvVar,
getEnv,
type ReadOnlyEnv,
type Env,
} from "./environment";
import { ConfigurationError, getErrorMessage, wrapError } from "./errors";
import { GitHubVariant, type GitHubVersion } from "./github";
import type { Language } from "./languages";
import type { Logger } from "./logging";
import * as json from "./json";
import { Language } from "./languages";
import { Logger } from "./logging";
// Re-export for backwards compatibility to avoid updating a lot of imports elsewhere.
export * from "./errors";
export {
getRequiredEnvParam,
getOptionalEnvVar,
getEnv,
isInTestMode,
} from "./environment";
export { GitHubVariant, type GitHubVersion } from "./github";
/**
* The name of the file containing the base database OIDs, as stored in the
@@ -49,6 +47,9 @@ const BASE_DATABASE_OIDS_FILE_NAME = "base-database-oids.json";
*/
const BROKEN_VERSIONS = ["0.0.0-20211207"];
/** A string representing a GitHub instance. */
export type InstanceURL = string;
/**
* Default name of the debugging artifact.
*/
@@ -469,6 +470,20 @@ const CODEQL_ACTION_WARNED_ABOUT_VERSION_ENV_VAR =
let hasBeenWarnedAboutVersion = false;
export enum GitHubVariant {
/** [GitHub.com](https://github.com) */
DOTCOM = "GitHub.com",
/** [GitHub Enterprise Server](https://docs.github.com/en/enterprise-server@latest/admin/overview/about-github-enterprise-server) */
GHES = "GitHub Enterprise Server",
/** [GitHub Enterprise Cloud with data residency](https://docs.github.com/en/enterprise-cloud@latest/admin/data-residency/about-github-enterprise-cloud-with-data-residency) */
GHEC_DR = "GitHub Enterprise Cloud with data residency",
}
export type GitHubVersion =
| { type: GitHubVariant.DOTCOM }
| { type: GitHubVariant.GHEC_DR }
| { type: GitHubVariant.GHES; version: string };
export function checkGitHubVersionInRange(
version: GitHubVersion,
logger: Logger,
@@ -537,6 +552,54 @@ export function assertNever(value: never): never {
throw new ExhaustivityCheckingError(value);
}
/**
* Set some initial environment variables that we can set even without
* knowing what version of CodeQL we're running.
*/
export function initializeEnvironment(version: string, env: Env = getEnv()) {
env.export(EnvVar.FEATURE_MULTI_LANGUAGE, "false");
env.export(EnvVar.FEATURE_SANDWICH, "false");
env.export(EnvVar.FEATURE_SARIF_COMBINE, "true");
env.export(EnvVar.FEATURE_WILL_UPLOAD, "true");
env.export(EnvVar.VERSION, version);
}
export class HTTPError extends Error {
public status: number;
constructor(message: string, status: number) {
super(message);
this.status = status;
}
}
/**
* An Error class that indicates an error that occurred due to
* a misconfiguration of the action or the CodeQL CLI.
*/
export class ConfigurationError extends Error {}
export function asHTTPError(arg: any): HTTPError | undefined {
if (!json.isObject<any>(arg) || !json.isString(arg.message)) {
return undefined;
}
if (Number.isInteger(arg.status)) {
return new HTTPError(arg.message, arg.status as number);
}
// See https://github.com/actions/toolkit/blob/acb230b99a46ed33a3f04a758cd68b47b9a82908/packages/tool-cache/src/tool-cache.ts#L19
if (Number.isInteger(arg.httpStatusCode)) {
return new HTTPError(arg.message, arg.httpStatusCode as number);
}
return undefined;
}
export async function codeQlVersionAtLeast(
codeql: CodeQL,
requiredVersion: string,
): Promise<boolean> {
return semver.gte((await codeql.getVersion()).version, requiredVersion);
}
export function getBaseDatabaseOidsFilePath(
config: ActionConfig<"dbLocation">,
): string {
@@ -621,30 +684,11 @@ export function isGoodVersion(versionSpec: string) {
return !BROKEN_VERSIONS.includes(versionSpec);
}
/**
* Returns whether we specifically want to skip generating a failed SARIF.
*/
export function shouldSkipGenerateFailedSarif(
env: ReadOnlyEnv = getEnv(),
): boolean {
const skipGenerateFailedSarif = env.getOptional(
EnvVar.SKIP_GENERATE_FAILED_SARIF,
);
return (
skipGenerateFailedSarif !== "false" &&
(shouldSkipSarifUpload(env) || skipGenerateFailedSarif === "true")
);
}
/**
* Returns whether we specifically want to skip uploading SARIF files.
*/
export function shouldSkipSarifUpload(env: ReadOnlyEnv = getEnv()): boolean {
const skipSarifUpload = env.getOptional(EnvVar.SKIP_SARIF_UPLOAD);
return (
skipSarifUpload !== "false" &&
(env.isSkippingUploadsInTests() || skipSarifUpload === "true")
);
export function shouldSkipSarifUpload(): boolean {
return isInTestMode() || process.env[EnvVar.SKIP_SARIF_UPLOAD] === "true";
}
/**
@@ -823,6 +867,20 @@ export function parseMatrixInput(
}
}
export function wrapError(error: unknown): Error {
return error instanceof Error ? error : new Error(String(error));
}
/**
* Returns an appropriate message for the error.
*
* If the error is an `Error` instance, this returns the error message without
* an `Error: ` prefix.
*/
export function getErrorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
export function prettyPrintPack(pack: Pack) {
return `${pack.name}${pack.version ? `@${pack.version}` : ""}${
pack.path ? `:${pack.path}` : ""
-10
View File
@@ -2,16 +2,6 @@
Change-notes are Markdown files used to document user-facing changes. When making a change that affects users, create a Markdown file here that describes the change. During the next release, the change-note files in `unreleased-change-notes/` will automatically be added to `CHANGELOG.md`.
### What do document
Some guidelines to help you decide whether a change needs a change note:
- New functionality that is gated behind a feature flag should be documented when it is added, highlighting that the feature will only be rolled out later. Ideally, this should also include an estimate for when the feature is expected to become available.
- When a feature flag is removed and the gated functionality becomes the default behaviour, this does not need to be documented.
- Changes that may require users to make changes to their workflows must be documented.
If in doubt, see `CHANGELOG.md` for what we have included previously.
### Change-note file format
Change-note files must follow a certain format so that they can be automatically validated and processed. Failure to follow the format will result in a failed PR check.