Skip to content

Commit dc03557

Browse files
committed
Merge branch 'main' into rb/summarize-loads-v2
2 parents 3af3772 + 296c0a7 commit dc03557

File tree

210 files changed

+1391
-7739
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

210 files changed

+1391
-7739
lines changed
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
/**
2+
* @name Usage of unsupported external library API
3+
* @description A call to an unsuppported external library API.
4+
* @kind problem
5+
* @problem.severity recommendation
6+
* @tags meta
7+
* @id csharp/meta/unsupported-external-api
8+
* @precision very-low
9+
*/
10+
11+
private import csharp
12+
private import semmle.code.csharp.dispatch.Dispatch
13+
private import semmle.code.csharp.dataflow.internal.FlowSummaryImpl as FlowSummaryImpl
14+
private import semmle.code.csharp.dataflow.internal.NegativeSummary
15+
private import Telemetry.ExternalApi
16+
17+
from DispatchCall c, ExternalApi api
18+
where
19+
c = api.getACall() and
20+
not api.isUninteresting() and
21+
not api.isSupported() and
22+
not api instanceof FlowSummaryImpl::Public::NegativeSummarizedCallable
23+
select c, "Call to unsupported external API $@.", api, api.toString()

docs/codeql/codeql-cli/analyzing-databases-with-the-codeql-cli.rst

Lines changed: 94 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,13 @@ For information about writing queries to run with ``database analyze``, see
1515

1616
Before starting an analysis you must:
1717

18-
- :doc:`Set up the CodeQL CLI <getting-started-with-the-codeql-cli>` so that it can find the queries
19-
and libraries included in the CodeQL repository.
18+
- :doc:`Set up the CodeQL CLI <getting-started-with-the-codeql-cli>` to run commands locally.
2019
- :doc:`Create a CodeQL database <creating-codeql-databases>` for the source
2120
code you want to analyze.
2221

22+
The simplest way to run ``codeql database analyze`` is using CodeQL packs. You can also
23+
run the command using queries from a local checkout of the CodeQL repository,
24+
which you may want to do if you want to customize the CodeQL core queries.
2325

2426
Running ``codeql database analyze``
2527
------------------------------------
@@ -34,7 +36,7 @@ When you run ``database analyze``, it:
3436

3537
You can analyze a database by running the following command::
3638

37-
codeql database analyze <database> --format=<format> --output=<output> <queries>
39+
codeql database analyze <database> --format=<format> --output=<output> <query-specifiers>...
3840

3941

4042
You must specify:
@@ -52,8 +54,8 @@ You must specify:
5254

5355
You can also specify:
5456

55-
- ``...<query-specifications>``: a list of queries to run over your database. This
56-
is a list of arguments. Where each argument can be:
57+
- ``<query-specifiers>...``: a space-separated list of queries to run over your database. This
58+
is a list of arguments, where each argument can be:
5759

5860
- a path to a query file
5961
- a path to a directory containing query files
@@ -62,7 +64,7 @@ You can also specify:
6264
- with an optional version range
6365
- with an optional path to a query, directory, or query suite inside the pack
6466

65-
If omitted, the default query suite for the language of the database being analyzed will be used. For more information, see the :ref:`examples <database-analyze-examples>` below.
67+
If omitted, the default query suite for the language of the analyzed database will be used. For the complete syntax of query specifiers, see :ref:`"Specifying which queries to run in a CodeQL pack"<specifying-which-queries>`.
6668

6769
- ``--sarif-category``: an identifying category for the results. Used when
6870
you want to upload more than one set of results for a commit.
@@ -95,55 +97,100 @@ You can also specify:
9597
For full details of all the options you can use when analyzing databases, see
9698
the `database analyze reference documentation <../manual/database-analyze>`__.
9799

98-
.. _database-analyze-examples:
99100

100-
Examples
101-
--------
101+
.. _specifying-which-queries:
102102

103-
The following examples assume your CodeQL databases have been created in a
104-
directory that is a sibling of your local copies of the CodeQL and CodeQL for Go
105-
repositories.
103+
Specifying which queries to run in a CodeQL pack
104+
------------------------------------------------
106105

107-
Running a single query
108-
~~~~~~~~~~~~~~~~~~~~~~
106+
Query specifiers are used by ``codeql database analyze`` and other commands that operate on a set of queries.
107+
The complete form of a query specifier is``scope/name@range:path``, where:
109108

110-
To run a single query over a CodeQL database for a JavaScript codebase,
111-
you could use the following command from the directory containing your database::
109+
- ``scope/name`` is the qualified name of a CodeQL pack.
110+
- ``range`` is a `semver range <https://docs.npmjs.com/cli/v6/using-npm/semver#ranges>`_.
111+
- ``path`` is a file system path to a single query, a directory containing queries, or a query suite file.
112112

113-
codeql database analyze <javascript-database> ../ql/javascript/ql/src/Declarations/UnusedVariable.ql --format=csv --output=js-analysis/js-results.csv
113+
When you specify a ``scope/name``, the ``range`` and ``path`` are
114+
optional. If you omit a ``range`` then the latest version of the
115+
specified pack is used. If you omit a ``path`` then the default query suite
116+
of the specified pack is used.
114117

115-
This command runs a simple query that finds potential bugs related to unused
116-
variables, imports, functions, or classes---it is one of the JavaScript
117-
queries included in the CodeQL repository. You could run more than one query by
118-
specifying a space-separated list of similar paths.
118+
The ``path`` can be one of: a ``.ql`` query file, a directory
119+
containing one or more queries, or a ``.qls`` query suite file. If
120+
you omit a pack name, then you must provide a ``path``,
121+
which will be interpreted relative to the working directory
122+
of the current process. Glob patterns are not supported.
119123

120-
The analysis generates a CSV file (``js-results.csv``) in a new directory
121-
(``js-analysis``).
124+
If you specify both a ``scope/name`` and ``path``, then the ``path`` cannot
125+
be absolute. It is considered relative to the root of the CodeQL
126+
pack.
122127

123-
You can also run your own custom queries with the ``database analyze`` command.
124-
For more information about preparing your queries to use with the CodeQL CLI,
125-
see ":doc:`Using custom queries with the CodeQL CLI <using-custom-queries-with-the-codeql-cli>`."
128+
Example query specifiers
129+
~~~~~~~~~~~~~~~~~~~~~~~~
126130

127-
If you do not have the CodeQL repository checked out, you can execute the same queries by specifying the query pack name and the path to the queries::
131+
* ``codeql/python-queries`` - All the queries in the default query suite of the latest version of the ``codeql/python-queries`` pack.
132+
* ``codeql/[email protected]`` - All the queries in the default query suite of version ``1.2.3`` of the ``codeql/python-queries`` pack.
133+
* ``codeql/python-queries@~1.2.3`` - All the queries in the default query suite of the latest version of the ``codeql/python-queries`` pack that is >= ``1.2.3`` and < ``1.3.0``.
134+
* ``codeql/python-queries:Functions`` - All queries in the ``Functions`` directory in the latest version of the ``codeql/python-queries`` pack.
135+
* ``codeql/[email protected]:Functions`` - All queries in the ``Functions`` directory in version 1.2.3 of the ``codeql/python-queries`` pack.
136+
* ``codeql/[email protected]:codeql-suites/python-code-scanning.qls`` - All queries in the ``codeql-suites/python-code-scanning.qls`` directory in version 1.2.3 of the ``codeql/python-queries`` pack.
137+
* ``suites/my-suite.qls`` - All queries in the ``suites/my-suite.qls`` file relative to the current working directory.
128138

129-
codeql database analyze --download <javascript-database> codeql/javascript-queries:Declarations/UnusedVariable.ql --format=csv --output=js-analysis/js-results.csv
139+
.. pull-quote::
140+
141+
Tip
142+
143+
The default query suite of the standard CodeQL query packs are ``codeql-suites/<lang>-code-scanning.qls``. Several other useful query suites can also be found in the ``codeql-suites`` directory of each pack. For example, the ``codeql/cpp-queries`` pack contains the following query suites:
144+
145+
* ``cpp-code-scanning.qls`` - Standard Code Scanning queries for C++. The default query suite for this pack.
146+
* ``cpp-security-extended.qls`` - Queries from the default ``cpp-code-scanning.qls`` suite for C++, plus lower severity and precision queries.
147+
* ``cpp-security-and-quality.qls`` - Queries from ``cpp-security-extended.qls``, plus maintainability and reliability queries.
130148

131-
Use the ``--download`` flag to download the query pack if it isn't yet available locally.
149+
You can see the sources for these query suites in the `CodeQL repository <https://github.com/github/codeql/tree/main/cpp/ql/src/codeql-suites>`__. Query suites for other languages are similar.
150+
151+
Examples of running database analyses
152+
---------------------------------------------
153+
154+
The following examples show how to run ``database analyze`` using CodeQL packs, and how to use a local checkout of the CodeQL repository. These examples assume your CodeQL databases have been created in a directory that is a sibling of your local copies of the CodeQL repository.
132155

133156
.. _run-query-pack:
134157

135-
Running a CodeQL pack
136-
~~~~~~~~~~~~~~~~~~~~~
158+
Running a CodeQL query pack
159+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
137160

138161
.. include:: ../reusables/beta-note-package-management.rst
139162

140163
To run an existing CodeQL query pack from the GitHub Container registry, you can specify one or more
141-
pack names and use the ``--download`` flag::
164+
pack names::
142165

143166
codeql database analyze <database> microsoft/[email protected] github/security-queries --format=sarifv2.1.0 --output=query-results.sarif --download
144167

145-
The ``analyze`` command above runs the default suite from ``microsoft/coding-standards v1.0.0`` and the latest version of ``github/security-queries`` on the specified database.
146-
For further information about default suites, see ":ref:`Publishing and using CodeQL packs <publishing-and-using-codeql-packs>`".
168+
This command runs the default query suite of two CodeQL query packs: ``microsoft/coding-standards`` version 1.0.0 and the latest version of ``github/security-queries`` on the specified database. For further information about default suites, see ":ref:`Publishing and using CodeQL packs <publishing-and-using-codeql-packs>`".
169+
170+
The ``--download`` flag is optional. Using it will ensure the query pack is downloaded if it isn't yet available locally.
171+
172+
Running a single query
173+
~~~~~~~~~~~~~~~~~~~~~~
174+
175+
To run a single query over a CodeQL database for a JavaScript codebase,
176+
you could use the following command from the directory containing your database::
177+
178+
codeql database analyze --download <javascript-database> codeql/javascript-queries:Declarations/UnusedVariable.ql --format=csv --output=js-analysis/js-results.csv
179+
180+
This command runs a simple query that finds potential bugs related to unused
181+
variables, imports, functions, or classes---it is one of the JavaScript
182+
queries included in the CodeQL repository. You could run more than one query by
183+
specifying a space-separated list of similar paths.
184+
185+
The analysis generates a CSV file (``js-results.csv``) in a new directory (``js-analysis``).
186+
187+
Alternatively, if you have the CodeQL repository checked out, you can execute the same queries by specifying the path to the query directly::
188+
189+
codeql database analyze <javascript-database> ../ql/javascript/ql/src/Declarations/UnusedVariable.ql --format=csv --output=js-analysis/js-results.csv
190+
191+
You can also run your own custom queries with the ``database analyze`` command.
192+
For more information about preparing your queries to use with the CodeQL CLI,
193+
see ":doc:`Using custom queries with the CodeQL CLI <using-custom-queries-with-the-codeql-cli>`."
147194

148195
Running all queries in a directory
149196
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -163,16 +210,20 @@ recursively, so any queries contained in subfolders will also be executed.
163210
pack's default queries in the analysis, or run one of the
164211
code scanning query suites.
165212

166-
For example, to execute all Python queries contained in the ``Functions`` directory you would run::
213+
For example, to execute all Python queries contained in the ``Functions`` directory in the
214+
``codeql/python-queries`` query pack you would run::
215+
216+
codeql database analyze <python-database> codeql/python-queries:Functions --format=sarif-latest --output=python-analysis/python-results.sarif --download
217+
218+
Alternatively, if you have the CodeQL repository checked out, you can execute the
219+
same queries by specifying the path to the directory directly::
167220

168221
codeql database analyze <python-database> ../ql/python/ql/src/Functions/ --format=sarif-latest --output=python-analysis/python-results.sarif
169222

170223
When the analysis has finished, a SARIF results file is generated. Specifying ``--format=sarif-latest`` ensures
171224
that the results are formatted according to the most recent SARIF specification
172225
supported by CodeQL.
173226

174-
.. _including-query-help-for-custom-codeql-queries-in-sarif-files:
175-
176227
Running a subset of queries in a CodeQL pack
177228
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
178229

@@ -224,33 +275,21 @@ For more information about CodeQL packs, see :doc:`About CodeQL Packs <about-cod
224275
Running query suites
225276
~~~~~~~~~~~~~~~~~~~~
226277

227-
To run a query suite over a CodeQL database for a C/C++ codebase,
278+
To run a query suite on a CodeQL database for a C/C++ codebase,
228279
you could use the following command from the directory containing your database::
229280

230-
codeql database analyze <cpp-database> cpp-code-scanning.qls --format=sarifv2.1.0 --output=cpp-results.sarif
281+
codeql database analyze <cpp-database> codeql/cpp-queries:codeql-suites/cpp-code-scanning.qls --format=sarifv2.1.0 --output=cpp-results.sarif --download
231282

232-
The analysis generates a file in the v2.1.0 SARIF format that is supported by all versions of GitHub.
233-
This file can be uploaded to GitHub by executing ``codeql github upload-results`` or the code scanning API.
283+
This command downloads the ``codeql/cpp-queries`` CodeQL query pack, runs the analysis, and generates a file in the SARIF version 2.1.0 format that is supported by all versions of GitHub. This file can be uploaded to GitHub by executing ``codeql github upload-results`` or the code scanning API.
234284
For more information, see `Analyzing a CodeQL database <https://docs.github.com/en/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system#analyzing-a-codeql-database>`__
235285
or `Code scanning API <https://docs.github.com/en/rest/reference/code-scanning>`__ in the GitHub documentation.
236286

237287
CodeQL query suites are ``.qls`` files that use directives to select queries to run
238288
based on certain metadata properties. The standard CodeQL packs have metadata that specify
239289
the location of the query suites used by code scanning, so the CodeQL CLI knows where to find these
240290
suite files automatically, and you don't have to specify the full path on the command line.
241-
For more information, see ":ref:`About CodeQL packs <standard-codeql-packs>`."
291+
For more information, see ":ref:`Creating CodeQL query suites <creating-codeql-query-suites>`."
242292

243-
The standard query suites are stored at the following paths in
244-
the CodeQL repository::
245-
246-
ql/<language>/ql/src/codeql-suites/<language>-code-scanning.qls
247-
248-
and at the following path in the CodeQL for Go repository::
249-
250-
ql/src/codeql-suites/go-code-scanning.qls
251-
252-
The repository also includes the query suites used by `LGTM.com <https://lgtm.com>`__.
253-
These are stored alongside the query suites for code scanning with names of the form: ``<language>-lgtm.qls``.
254293

255294
For information about creating custom query suites, see ":doc:`Creating
256295
CodeQL query suites <creating-codeql-query-suites>`."
@@ -265,11 +304,11 @@ If the analysis found fewer results for standard queries than you expected, revi
265304
Integrating a CodeQL pack into a code scanning workflow in GitHub
266305
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
267306

268-
.. include:: ../reusables/beta-note-package-management.rst
269-
270307
You can use CodeQL query packs in your code scanning setup. This allows you to select query packs published by various sources and use them to analyze your code.
271308
For more information, see "`Using CodeQL query packs in the CodeQL action <https://docs.github.com/en/code-security/secure-coding/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-codeql-query-packs/>`_" or "`Downloading and using CodeQL query packs in your CI system <https://docs.github.com/en/code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system#downloading-and-using-codeql-query-packs>`_."
272309

310+
.. _including-query-help-for-custom-codeql-queries-in-sarif-files:
311+
273312
Including query help for custom CodeQL queries in SARIF files
274313
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
275314

java/kotlin-extractor/src/main/kotlin/KotlinFileExtractor.kt

Lines changed: 13 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@ package com.github.codeql
33
import com.github.codeql.comments.CommentExtractor
44
import com.github.codeql.utils.*
55
import com.github.codeql.utils.versions.functionN
6-
import com.github.codeql.utils.versions.getIrStubFromDescriptor
76
import com.github.codeql.utils.versions.isUnderscoreParameter
87
import com.semmle.extractor.java.OdasaOutput
98
import org.jetbrains.kotlin.backend.common.extensions.IrPluginContext
@@ -1788,7 +1787,8 @@ open class KotlinFileExtractor(
17881787

17891788
private fun extractCall(c: IrCall, callable: Label<out DbCallable>, stmtExprParent: StmtExprParent) {
17901789
with("call", c) {
1791-
val target = tryReplaceSyntheticFunction(c.symbol.owner)
1790+
val owner = getBoundSymbolOwner(c.symbol, c) ?: return
1791+
val target = tryReplaceSyntheticFunction(owner)
17921792

17931793
// The vast majority of types of call want an expr context, so make one available lazily:
17941794
val exprParent by lazy {
@@ -2965,15 +2965,7 @@ open class KotlinFileExtractor(
29652965
tw.writeCallableEnclosingExpr(id, callable)
29662966
tw.writeStatementEnclosingExpr(id, exprParent.enclosingStmt)
29672967

2968-
val owner = if (e.symbol.isBound) {
2969-
e.symbol.owner
2970-
}
2971-
else {
2972-
logger.warnElement("Unbound enum value, trying to use enum entry stub from descriptor", e)
2973-
2974-
@OptIn(ObsoleteDescriptorBasedAPI::class)
2975-
getIrStubFromDescriptor() { it.generateEnumEntryStub(e.symbol.descriptor) }
2976-
} ?: return
2968+
val owner = getBoundSymbolOwner(e.symbol, e) ?: return
29772969

29782970
val vId = useEnumEntry(owner)
29792971
tw.writeVariableBinding(id, vId)
@@ -3150,15 +3142,7 @@ open class KotlinFileExtractor(
31503142
// automatically-generated `public static final MyObject INSTANCE`
31513143
// field that we are accessing here.
31523144
val exprParent = parent.expr(e, callable)
3153-
val c = if (e.symbol.isBound) {
3154-
e.symbol.owner
3155-
}
3156-
else {
3157-
logger.warnElement("Unbound object value, trying to use class stub from descriptor", e)
3158-
3159-
@OptIn(ObsoleteDescriptorBasedAPI::class)
3160-
getIrStubFromDescriptor() { it.generateClassStub(e.symbol.descriptor) }
3161-
} ?: return
3145+
val c = getBoundSymbolOwner(e.symbol, e) ?: return
31623146

31633147
val instance = if (c.isCompanion) useCompanionObjectClassInstance(c) else useObjectClassInstance(c)
31643148

@@ -3271,6 +3255,15 @@ open class KotlinFileExtractor(
32713255
}
32723256
}
32733257

3258+
private inline fun <D: DeclarationDescriptor, reified B: IrSymbolOwner> getBoundSymbolOwner(symbol: IrBindableSymbol<D, B>, e: IrExpression): B? {
3259+
if (symbol.isBound) {
3260+
return symbol.owner
3261+
}
3262+
3263+
logger.errorElement("Unbound symbol found, skipping extraction of expression", e)
3264+
return null
3265+
}
3266+
32743267
private fun extractSuperAccess(irType: IrType, callable: Label<out DbCallable>, parent: Label<out DbExprparent>, idx: Int, enclosingStmt: Label<out DbStmt>, locId: Label<out DbLocation>) =
32753268
tw.getFreshIdLabel<DbSuperaccess>().also {
32763269
val type = useType(irType)

0 commit comments

Comments
 (0)