Skip to content

Add github-codeql-tools repository property for tools input#3766

Draft
Copilot wants to merge 8 commits intomainfrom
copilot/add-tools-input-source-repository
Draft

Add github-codeql-tools repository property for tools input#3766
Copilot wants to merge 8 commits intomainfrom
copilot/add-tools-input-source-repository

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 23, 2026

Large organizations downloading a pinned CodeQL CLI version on every analysis run can hit rate limits. This adds a github-codeql-tools repository property that lets org admins set the tools source at org level, avoiding per-run downloads.

What changes

New repository property: github-codeql-tools

  • Org admins can set this on their repositories (e.g., github-codeql-tools: toolcache)
  • When set, it acts as a default tools input — explicit workflow-level tools input always takes precedence
  • toolcache value works without requiring the AllowToolcacheInput feature flag or a dynamic workflow trigger, since the org admin is explicitly opting in

Implementation

  • Added RepositoryPropertyName.TOOLS = "github-codeql-tools" to the existing property enum/type system in src/feature-flags/properties.ts
  • Threaded a toolsInputFromRepositoryProperty flag through the call chain: initCodeQLsetupCodeQLsetupCodeQLBundlegetCodeQLSource
  • In getCodeQLSource, toolcache with this flag set bypasses the feature-flag/dynamic-workflow guard and emits distinct log messages referencing the repository property name rather than tools: toolcache
  • init-action.ts resolves the effective tools input: workflow input wins; property is used only when no explicit input is given

Risk assessment

High risk: Not fully under a feature flag — the new code path activates when the repository property is set.

Which use cases does this change impact?

Workflow types:

  • Advanced setup - Impacts users who have custom CodeQL workflows.
  • Managed - Impacts users with dynamic workflows (Default Setup, Code Quality, ...).

Products:

  • Code Scanning - The changes impact analyses when analysis-kinds: code-scanning.
  • Code Quality - The changes impact analyses when analysis-kinds: code-quality.

Environments:

  • Dotcom - Impacts CodeQL workflows on github.com and/or GitHub Enterprise Cloud with Data Residency.
  • GHES - Impacts CodeQL workflows on GitHub Enterprise Server.

How did/will you validate this change?

  • Unit tests - I am depending on unit test coverage (i.e. tests in .test.ts files).

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

  • Rollback - Change can only be disabled by rolling back the release or releasing a new version with a fix.

The repository property must be explicitly set by an org admin; no existing workflows are affected unless they set github-codeql-tools.

How will you know if something goes wrong after this change is released?

  • Telemetry - I rely on existing telemetry or have made changes to the telemetry.
    • Dashboards - I will watch relevant dashboards for issues after the release.

Are there any special considerations for merging or releasing this change?

  • No special considerations - This change can be merged at any time.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Consider adding a changelog entry for this change.
  • Confirm the readme and docs have been updated if necessary.

⌨️ Start Copilot coding agent tasks without leaving your editor — available in VS Code, Visual Studio, JetBrains IDEs and Eclipse.

Copilot AI changed the title [WIP] Add repository property for tools input in CodeQL Action Add github-codeql-tools repository property for tools input Mar 23, 2026
Copilot AI requested a review from oscarsj March 23, 2026 17:31
@github-actions github-actions bot added the size/M Should be of average difficulty to review label Mar 24, 2026
@oscarsj oscarsj force-pushed the copilot/add-tools-input-source-repository branch from 5c6b9e8 to e74c1ee Compare March 24, 2026 17:42
@oscarsj oscarsj requested a review from mbg March 24, 2026 17:58
@oscarsj oscarsj marked this pull request as ready for review March 24, 2026 18:01
@oscarsj oscarsj requested a review from a team as a code owner March 24, 2026 18:01
Copilot AI review requested due to automatic review settings March 24, 2026 18:01
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new org-managed repository property (github-codeql-tools) that can supply a default tools input value (when the workflow doesn’t set one), enabling organizations to opt into using the toolcache without per-run CLI downloads.

Changes:

  • Introduces RepositoryPropertyName.TOOLS = "github-codeql-tools" and parses it from the repository properties API.
  • Resolves an effective tools input in init-action (workflow input wins; otherwise fall back to repository property) and threads an origin flag through to CodeQL setup.
  • Allows tools=toolcache to bypass the existing feature-flag/dynamic-workflow restriction when the value came from the repository property, with targeted log messages and unit tests.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/feature-flags/properties.ts Adds the new repository property name and parser typing for github-codeql-tools.
src/feature-flags/properties.test.ts Tests loading/parsing of the new github-codeql-tools property.
src/init-action.ts Resolves effective tools input (workflow vs repository property) and passes origin flag into initCodeQL.
src/init.ts Threads toolsInputFromRepositoryProperty into setupCodeQL.
src/codeql.ts Threads toolsInputFromRepositoryProperty into setupCodeQLBundle.
src/setup-codeql.ts Adds toolsInputFromRepositoryProperty parameter and uses it to allow toolcache without FF/dynamic checks.
src/setup-codeql.test.ts Adds tests for toolcache behavior when enabled via repository property (including empty toolcache fallback).
lib/* Generated JS output corresponding to the TS changes (not reviewed).

Comment on lines 281 to 290
export async function getCodeQLSource(
toolsInput: string | undefined,
defaultCliVersion: CodeQLDefaultVersionInfo,
apiDetails: api.GitHubApiDetails,
variant: util.GitHubVariant,
tarSupportsZstd: boolean,
features: FeatureEnablement,
logger: Logger,
toolsInputFromRepositoryProperty = false,
): Promise<CodeQLToolsSource> {
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toolsInputFromRepositoryProperty is introduced on getCodeQLSource, but it’s only used to special-case the toolcache guard + log messages. If the repository property is ever used for other tools values (e.g. linked, nightly, or a URL), the current logs in other branches still say they were requested by 'tools: …', which can be misleading when no workflow input was provided. Consider generalizing this to an “tools input origin” (workflow vs repo property) and using it consistently in all user-facing log messages about requested tools.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member

@mbg mbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR needs work. As CCR has already identified, the new logic is not coherently applied to all parts of the CodeQL Action.

Beyond that, we have some high-level design questions to answer before settling on an implementation. I have discussed these in my details comments, but in summary they are:

  • The most important is whether we want the CodeQL Action repo properties to serve as customisation options for managed workflows (like Default Setup) or enterprise policy enforcement (e.g. to force all analyses across an organisation to use particular queries, CLI versions, etc.
    • For this change specifically, this would determine whether the repo property (if set) overrides any explicit tools input or not. Or whether we would allow some way of specificing in the repo property value whether it should or not. (e.g. like the prefix + for queries).
  • We need to decide on the interaction between this, the FF, and whether the workflow is dynamic. See my comment there.

Once we have decided on all these aspects, then we should also add a changelog entry here and update the public docs for the new property.

Comment on lines +302 to +310
// Determine the effective tools input.
// The explicit `tools` workflow input takes precedence. If none is provided,
// fall back to the 'github-codeql-tools' repository property (if set).
const toolsWorkflowInput = getOptionalInput("tools");
const toolsPropertyValue: string | undefined =
repositoryPropertiesResult.orElse({})[RepositoryPropertyName.TOOLS];
const effectiveToolsInput = toolsWorkflowInput ?? toolsPropertyValue;
const toolsInputFromRepositoryProperty =
toolsWorkflowInput === undefined && toolsPropertyValue !== undefined;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. All of this should be refactored into its own function with appropriate tests.
  2. We need to think very carefully about the semantics we want here. Right now, an explicit tools input takes precedence over the repo property.
  • That probably makes sense since we don't have an explicit tools input in the Default Setup workflow. However, this then creates an implicit dependency / assumption that we won't add one to the workflow template there.
  • Users with advanced workflows can tune the tools input directly and don't need the repo property.
  • We will want to be deliberate (and consistent) in whether repository properties serve as "overrides for managed workflows" or "policy enforcement". E.g. could we imagine that an enterprise might want to force the use of a particular CLI version?

Let's discuss this offline and more widely in the team.

Copy link
Copy Markdown
Member

@oscarsj oscarsj Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as for 2:

  • I think an explicit tools input in advanced setup workflows takes precedence over repo propierties, and as you suggest default setup workflows should not have the possibility of setting it (no tools input there)
  • I don't see this as policy enforcement, in fact i struggle to think on a situation where an enterprise would want to enforce a specific version, why would they? My idea is around "bulk enablement": setting this for default setup workflows for all repos would make enterprise-level management easier, but still allow individual developers/repos to set up their own configs

That been said , I'm not sure whether we can decide over the semantics of this on our own. @jonjanego is this something Product would have a saying about?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this as policy enforcement, in fact i struggle to think on a situation where an enterprise would want to enforce a specific version

That point wasn't necessarily about this specific property, but about our use of repository properties in general. We want all of our repository properties to behave in a coherent way. So if others serve as an enterprise policy enforcement tool (e.g. to force particular queries to run), then this should be consistent for all properties. (Or have a coherent syntax/naming convention for "enforcement" vs "default".)

I think we decided elsewhere that we don't want the repository properties to serve as enforcement tool, since there are other features which are designed for that.

Comment on lines 405 to +412
const allowToolcacheValueFF = await features.getValue(
Feature.AllowToolcacheInput,
);
const allowToolcacheValue =
allowToolcacheValueFF && (isDynamicWorkflow() || util.isInTestMode());
toolsInputFromRepositoryProperty ||
(allowToolcacheValueFF && (isDynamicWorkflow() || util.isInTestMode()));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need to rethink this logic:

  • The FF is here to gate this functionality, so that we can disable it if it is causing problems. We should either also gate this new functionality behind the FF or remove the FF.
  • We need to think about whether we want to restrict this to just dynamic workflows. See my other point about our overall strategy with respect to repository properties ("customising managed workflows" vs "enterprise policy enforcement").
  • With the current logic, the repo property would unconditionally affect all workflows that don't have an explicit tools input. That includes advanced workflows without a tools input. Is that what we want? E.g. if an organisation wants to force tools: toolcache for Default Setup, then they couldn't do this without also forcing it for all advanced workflows that don't currently have an explicit tools input (no explicit tools input is generally the default).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree with your observations:

  • I'll gate this behind the same FF.
  • I suggest to restrict this to dynamic workflows (IIUC these are Default Setup, right?)
  • I lean towards this affecting advanced workflows that don't have specific tools input , too

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dynamic workflows are all GitHub-managed workflows. That's mainly Default Setup, but also Code Quality, CSRA, ...

I lean towards this affecting advanced workflows that don't have specific tools input , too

Just to be clear, my point was not that we would want this to affect advanced workflows that don't have a specific tools input, but that the implementation in the PR would have this (potentially undesirable) effect. No tools input is the default that causes the CodeQL Action to determine the latest, stable CLI version that's available, and doesn't mean that it should be overriden. Note also that your 2nd and 3rd point contradict each other.


Ideally, we'd have a way for customers to express whether they want the repo props to apply to all CodeQL workflows, or just Default Setup etc.

For the repo property that controls extra queries, we don't restrict it to just dynamic workflows. So I suppose we should be consistent with tools.

tarSupportsZstd: boolean,
features: FeatureEnablement,
logger: Logger,
toolsInputFromRepositoryProperty = false,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider whether it would be better to set toolsInput to the computed input, instead of having an extra toolsInputFromRepositoryProperty value.

Using only toolsInput would simplify the logic and reduce scope for errors here. (See my other comment about the enablement logic.)

Keeping toolsInputFromRepositoryProperty allows more specific log messages below. However, if we have a log message when computing the tools value that says Settings tools: ${value} based on repository property (or similar), then we wouldn't need to log that in each case here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think my suggestion to have an early log message about the source of the value when we compute the effective tools value makes more sense than having all the individual log messages state this.

[`Ignoring 'tools: toolcache' because the feature is not enabled.`],
);

test.serial(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tests here would not be necessary if we used the existing toolsInput argument.

Copy link
Copy Markdown
Member

@mbg mbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments on the latest commit.

* @returns An object containing the effective tools input and whether it came from repository property
*/
export function resolveToolsInput(
repositoryProperties: Record<string, any>,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a RepositoryProperties type. Why the less precise type here?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using it creates cyclic dependency

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this function doesn't have to be in this file - in fact it would probably be better off somewhere else, since it's not primarily an Actions helper

const initStatusReport: InitStatusReport = {
...statusReportBase,
tools_input: getOptionalInput("tools") || "",
tools_input: effectiveToolsInput || "",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure whether this is a good idea or not.

On one hand, we do want to know what the tools value is and it makes sense for the "effective" value to be reported in the telemetry.

On the other hand, this field is named tools_input and it could mislead us into thinking that this value is always the input to tools: in the workflow. There's also no indication in the telemetry what the source of the value is.

I'd probably suggest we either keep this as tools_input: getOptionalInput("tools") and add a computed_tools_input field with the effectiveToolsInput value or we add a tools_input_source field that's either set to input or property.

repositoryPropertiesResult.orElse({})[RepositoryPropertyName.TOOLS];
const effectiveToolsInput = toolsWorkflowInput ?? toolsPropertyValue;
const resolvedToolsInput = resolveToolsInput(
repositoryPropertiesResult.orElse({}),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the definition of the repositoryProperties constant up and then use that here instead of repositoryPropertiesResult.orElse({}).

*/
export function resolveToolsInput(
repositoryProperties: Record<string, any>,
toolsPropertyName: string,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this a parameter of this function? It's always RepositoryPropertyName.TOOLS. I can't think of a reason where we'd want this to be something else either.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, importing that variable would create a circular dep

const initStatusReport: InitStatusReport = {
...statusReportBase,
tools_input: getOptionalInput("tools") || "",
tools_input: effectiveToolsInput || "",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same point as for the init telemetry.

Comment on lines +131 to +134
const repositoryPropertiesResult = await loadPropertiesFromApi(
logger,
repositoryNwo,
);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use loadRepositoryProperties instead of loadPropertiesFromApi.

Comment on lines +164 to +165
const toolsInputFromRepositoryProperty =
resolvedToolsInput.toolsInputFromRepositoryProperty;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only used once, use resolvedToolsInput.toolsInputFromRepositoryProperty directly below?

Comment on lines +457 to +458
logger.warning(
`Ignoring ${getToolsInputOriginDescription(toolsInput, toolsInputFromRepositoryProperty)} because the feature is not enabled.`,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this should be a warning, since there's not really anything the user can do about it.

@oscarsj oscarsj marked this pull request as draft April 15, 2026 13:41
status,
logger,
);
if (response) {
} else if (status === "complete") {
break;
} else if (status === "failed") {
if (!response) {
@github-actions github-actions bot added size/L May be hard to review and removed size/M Should be of average difficulty to review labels Apr 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L May be hard to review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants