Merge rc/1.18 into next.

change-notes/1.18/analysis-cpp.md

@@ -1,40 +1,28 @@
 # Improvements to C/C++ analysis
 
-## General improvements
-
-> Changes that affect alerts in many files or from many queries
-> For example, changes to file classification
-
 ## New queries
 
 | **Query**                   | **Tags**  | **Purpose**                                                        |
 |-----------------------------|-----------|--------------------------------------------------------------------|
-| Upcast array used in pointer arithmetic | reliability, correctness, external/cwe/cwe-119 | Finds undefined behavior caused by doing pointer arithmetic on an array of objects that has been cast to an array of a supertype. |
+| Upcast array used in pointer arithmetic (`cpp/upcast-array-pointer-arithmetic`) | reliability, correctness, external/cwe/cwe-119 | Finds undefined behavior caused by doing pointer arithmetic on an array of objects that has been cast to an array of a supertype. |
 
 ## Changes to existing queries
 
 | **Query**                  | **Expected impact**    | **Change**                                                       |
 |----------------------------|------------------------|------------------------------------------------------------------|
-| Self comparison | Fewer false positive results | Range checks of the form `x == (T)x` are no longer flagged unless they are guaranteed to have the same result on all platforms. |
-| [Nested loops with same variable] | Fewer false positive results | Results where the loop variable is a member of a class or struct now account for the object. |
-| [For loop variable changed in body] | Fewer false positive results | Results where the loop variable is a member of a class or struct now account for the object. |
-| [Local variable hides global variable] | Fewer false positive results | Results for parameters are now only reported if the name of the global variable is the same as the name of the parameter as used in the function definition (not just a function declaration). |
-| [Memory may not be freed] | More correct results | This query now models calls to `realloc` more accurately. |
-| Wrong number of arguments to formatting function | Fewer false positive results | Some false positives related to custom printf-like functions have been fixed. |
-| Wrong number of arguments to formatting function | Clear separation between results of high and low severity | This query has been split into two queries: a high-severity query named [Too few arguments to formatting function] and a low-severity query named [Too many arguments to formatting function]. |
-| [Too few arguments to formatting function] | More correct and fewer false positives results | This query now understands positional format arguments as supported by some libraries. |
-| [Too many arguments to formatting function] | More correct and fewer false positives results | This query now understands positional format arguments as supported by some libraries. |
-| [Variable used in its own initializer] | Fewer false positive results | Results where a macro is used to indicate deliberate uninitialization are now excluded |
-| [Assignment where comparison was intended] | Fewer false positive results | Results are no longer reported if the variable is not yet defined. |
-| [Comparison where assignment was intended] | More correct results | "This query now includes results where an overloaded `operator==` is used in the wrong context. |
-| [User-controlled data in arithmetic expression] | More correct results | Increment / decrement / addition assignment / subtraction assignment operations are now understood as arithmetic operations in this query. |
-| [Uncontrolled data in arithmetic expression] | More correct results | Increment / decrement / addition assignment / subtraction assignment operations are now understood as arithmetic operations in this query. |
-| [Use of extreme values in arithmetic expression] | More correct results | Increment / decrement / addition assignment / subtraction assignment operations are now understood as arithmetic operations in this query. |
-| [Use of extreme values in arithmetic expression] | Fewer false positives | The query now considers whether a particular expression might cause an overflow of minimum or maximum values only. |
+| Assignment where comparison was intended (`cpp/assign-where-compare-meant`) | Fewer false positive results | Results are no longer reported if the variable is not yet defined. |
+| Comparison where assignment was intended (`cpp/compare-where-assign-meant`) | More results | This query now includes results where an overloaded `operator==` is used in the wrong context. |
+| For loop variable changed in body (`cpp/loop-variable-changed`)  | Fewer false positive results | Results where the loop variable is a member of a class or struct now account for the object. |
+| Local variable hides global variable (`cpp/local-variable-hides-global-variable`) | Fewer false positive results | Results for parameters are now only reported if the name of the global variable is the same as the name of the parameter as used in the function definition (not just a function declaration). |
+| Nested loops with same variable (`cpp/nested-loops-with-same-variable`) | Fewer false positive results | Results where the loop variable is a member of a class or struct now account for the object. |
+| Self comparison (`cpp/comparison-of-identical-expressions`) | Fewer false positive results | Range checks of the form `x == (T)x` are no longer flagged unless they are guaranteed to have the same result on all platforms. |
+| Too few arguments to formatting function (`cpp/wrong-number-format-arguments`) | More precise results | This was previously known as "Wrong number of arguments to formatting function". It now focuses only on functions calls that are missing arguments, which tend to be more severe. See the next row for the new query that reports lower-severity alerts for calls with too many arguments. In addition, both queries now understand positional format arguments as supported by some libraries, and some false positive results for custom printf-like functions have been fixed.|
+| Too many arguments to formatting function (`cpp/too-many-format-arguments`) | More precise results | This new query was created by splitting the old "Wrong number of arguments to formatting function" query (see row above). It reports function calls with too many arguments.  |
+| User-controlled data in arithmetic expression (`cpp/tainted-arithmetic`) | More results | The query is extended to analyze increment, decrement, addition-assignment, and subtraction-assignment operations. |
+| Variable used in its own initializer (`cpp/use-in-own-initializer`) | Fewer false positive results | Results where a macro is used to indicate deliberate uninitialization are now excluded. |
+|Uncontrolled data in arithmetic expression (`cpp/uncontrolled-arithmetic`) | More results | The query is extended to analyze increment, decrement, addition-assignment, and subtraction-assignment operations. |
 
 ## Changes to QL libraries
 
-* Fixes for aggregate initializers using designators:
-  * `ClassAggregateLiteral.getFieldExpr()` previously assumed initializer expressions appeared in the same order as the declaration order of the fields, causing it to associate the expressions with the wrong fields when using designated initializers. This has been fixed.
-  * `ArrayAggregateLiteral.getElementExpr()` previously assumed initializer expressions appeared in the same order as the corresponding array elements, causing it to associate the expressions with the wrong array elements when using designated initializers. This has been fixed.
-* `Element.getEnclosingElement()` no longer includes macro accesses in its results. To explore parents and children of macro accesses, use the relevant member predicates on `MacroAccess` or `MacroInvocation`.
+* The `ClassAggregateLiteral.getFieldExpr()` and `ArrayAggregateLiteral.getElementExpr()` predicates incorrectly assumed that initializer expressions appeared in the same order as the declaration order of the elements. This resulted in the association of the expressions with the wrong elements when designated initializers were used. This has been fixed.
+* Results for the `Element.getEnclosingElement()` predicate no longer included macro accesses. To explore parents and children of macro accesses, use the relevant member predicates on `MacroAccess` or `MacroInvocation`.

change-notes/1.18/analysis-csharp.md

@@ -1,11 +1,5 @@
 # Improvements to C# analysis
 
-> NOTES
->
-> Please describe your changes in terms that are suitable for
-> customers to read. These notes will have only minor tidying up
-> before they are published as part of the release notes.
-
 ## General improvements
 
 * Control flow analysis has been improved for `catch` clauses with filters.
@@ -14,30 +8,53 @@
 
 | **Query**                   | **Tags**  | **Purpose**                                                        |
 |-----------------------------|-----------|--------------------------------------------------------------------|
-| Arbitrary file write during zip extraction ("Zip Slip") (`cs/zipslip`) | security, external/cwe/cwe-022  | Identifies zip extraction routines which allow arbitrary file overwrite vulnerabilities.
+| Arbitrary file write during zip extraction ("Zip Slip") (`cs/zipslip`) | security, external/cwe/cwe-022  | Identifies zip extraction routines which allow arbitrary file overwrite vulnerabilities. |
 | Local scope variable shadows member (`cs/local-shadows-member`) | maintainability, readability | Replaces the existing queries Local variable shadows class member (`cs/local-shadows-class-member`), Local variable shadows struct member (`cs/local-shadows-struct-member`), Parameter shadows class member (`cs/parameter-shadows-class-member`), and Parameter shadows struct member (`cs/parameter-shadows-struct-member`). |
 
 ## Changes to existing queries
 
 | **Query**                  | **Expected impact**    | **Change**                                                       |
 |----------------------------|------------------------|------------------------------------------------------------------|
-| [Constant condition](https://help.semmle.com/wiki/display/CSHARP/Constant+condition) (`cs/constant-condition`) | More results | The query has been generalized to cover both Null-coalescing left operand is constant (`cs/constant-null-coalescing`) and Switch selector is constant (`cs/constant-switch-selector`). |
+| Constant condition (`cs/constant-condition`) | More results | The query has been generalized to report alerts for the old queries Null-coalescing left operand is constant (`cs/constant-null-coalescing`) and Switch selector is constant (`cs/constant-switch-selector`). |
 | Exposing internal representation (`cs/expose-implementation`) | Different results | The query has been rewritten, based on the [equivalent Java query](https://help.semmle.com/wiki/display/JAVA/Exposing+internal+representation). |
-| Local variable shadows class member(`cs/local-shadows-class-member`) | No results | The query has been replaced by Local scope variable shadows member  (`cs/local-shadows-member`). |
-| Local variable shadows struct member (`cs/local-shadows-struct-member`) | No results | The query has been replaced by Local scope variable shadows member  (`cs/local-shadows-member`). |
-| [Missing Dispose call on local IDisposable](https://help.semmle.com/wiki/display/CSHARP/Missing+Dispose+call+on+local+IDisposable) (`cs/local-not-disposed`) | Fewer results | The query identifies more cases where the local variable may be disposed by a library call. |
-| [Nested loops with same variable](https://help.semmle.com/wiki/display/CSHARP/Nested+loops+with+same+variable) (`cs/nested-loops-with-same-variable`) | Fewer results | Results are no longer highlighted in nested loops that share the same condition, and do not use the variable after the inner loop. |
-| Null-coalescing left operand is constant (`cs/constant-null-coalescing`) | No results | The query has been removed, as it is now covered by Constant condition (`cs/constant-condition`). |
-| Parameter shadows class member (`cs/parameter-shadows-class-member`) | No results | The query has been replaced by Local scope variable shadows member  (`cs/local-shadows-member`). |
-| Parameter shadows struct member (`cs/parameter-shadows-struct-member`) | No results | The query has been replaced by Local scope variable shadows member  (`cs/local-shadows-member`). |
-| [Potentially incorrect CompareTo(...) signature](https://help.semmle.com/wiki/display/CSHARP/Potentially+incorrect+CompareTo%28...%29+signature) (`cs/wrong-compareto-signature`) | Fewer results | Results are no longer highlighted in constructed types. |
-| Switch selector is constant (`cs/constant-switch-selector`) | No results | The query has been removed, as it is now covered by Constant condition (`cs/constant-condition`). |
-| [Useless upcast](https://help.semmle.com/wiki/display/CSHARP/Useless+upcast) (`cs/useless-upcast`) | Fewer results | The query has been improved to cover more cases where upcasts may be needed. |
+| Local variable shadows class member (`cs/local-shadows-class-member`) | No results | The query has been replaced by the new query: Local scope variable shadows member  (`cs/local-shadows-member`). |
+| Local variable shadows struct member (`cs/local-shadows-struct-member`) | No results | The query has been replaced by the new query: Local scope variable shadows member  (`cs/local-shadows-member`). |
+| Missing Dispose call on local IDisposable (`cs/local-not-disposed`) | Fewer false positive results | The query identifies more cases where the local variable may be disposed by a library call. |
+| Nested loops with same variable (`cs/nested-loops-with-same-variable`) | Fewer false positive results | Results are no longer highlighted in nested loops that share the same condition, and do not use the variable after the inner loop. |
+| Null-coalescing left operand is constant (`cs/constant-null-coalescing`) | No results | The query has been removed, as alerts for this problem are now reported by the new query: Constant condition (`cs/constant-condition`). |
+| Parameter shadows class member (`cs/parameter-shadows-class-member`) | No results | The query has been replaced by the new query: Local scope variable shadows member  (`cs/local-shadows-member`). |
+| Parameter shadows struct member (`cs/parameter-shadows-struct-member`) | No results | The query has been replaced by the new query: Local scope variable shadows member  (`cs/local-shadows-member`). |
+| Potentially incorrect CompareTo(...) signature (`cs/wrong-compareto-signature`) | Fewer false positive results | Results are no longer highlighted in constructed types. |
+| Switch selector is constant (`cs/constant-switch-selector`) | No results | The query has been removed, as alerts for this problem are now reported by the new query: Constant condition (`cs/constant-condition`). |
+| Useless upcast (`cs/useless-upcast`) | Fewer false positive results | The query has been improved to cover more cases where upcasts may be needed. |
 
 ## Changes to code extraction
 
-* *Series of bullet points*
+* The `into` part of `join` clauses is now extracted.
+* The `when` part of constant cases is now extracted.
+* Fixed a bug where `while(x is T y) ...` was not extracted correctly.
 
 ## Changes to QL libraries
 
-* A new non-member predicate `mayBeDisposed()` can be used to determine if a variable is potentially disposed inside a library. It will analyse the CIL code in the library to determine this.
+* A new non-member predicate `mayBeDisposed()` can be used to determine if a variable is potentially disposed inside a library. It will analyze the CIL code in the library to determine this.
+* The predicate `getCondition()` has been moved from `TypeCase` to `CaseStmt`. It is now possible to get the condition of a `ConstCase` using its `getCondition()` predicate.
+* Several control flow graph entities have been renamed (the old names are deprecated but are still available in this release for backwards compatibility):
+  - `ControlFlowNode` has been renamed to `ControlFlow::Node`.
+  - `CallableEntryNode` has been renamed to `ControlFlow::Nodes::EntryNode`.
+  - `CallableExitNode` has been renamed to `ControlFlow::Nodes::ExitNode`.
+  - `ControlFlowEdgeType` has been renamed to `ControlFlow::SuccessorType`.
+  - `ControlFlowEdgeSuccessor` has been renamed to `ControlFlow::SuccessorTypes::NormalSuccessor`.
+  - `ControlFlowEdgeConditional` has been renamed to `ControlFlow::SuccessorTypes::ConditionalSuccessor`.
+  - `ControlFlowEdgeBoolean` has been renamed to `ControlFlow::SuccessorTypes::BooleanSuccessor`.
+  - `ControlFlowEdgeNullness` has been renamed to `ControlFlow::SuccessorTypes::NullnessSuccessor`.
+  - `ControlFlowEdgeMatching` has been renamed to `ControlFlow::SuccessorTypes::MatchingSuccessor`.
+  - `ControlFlowEdgeEmptiness` has been renamed to `ControlFlow::SuccessorTypes::EmptinessSuccessor`.
+  - `ControlFlowEdgeReturn` has been renamed to `ControlFlow::SuccessorTypes::ReturnSuccessor`.
+  - `ControlFlowEdgeBreak` has been renamed to `ControlFlow::SuccessorTypes::BreakSuccessor`.
+  - `ControlFlowEdgeContinue` has been renamed to `ControlFlow::SuccessorTypes::ContinueSuccessor`.
+  - `ControlFlowEdgeGotoLabel` has been renamed to `ControlFlow::SuccessorTypes::GotoLabelSuccessor`.
+  - `ControlFlowEdgeGotoCase` has been renamed to `ControlFlow::SuccessorTypes::GotoCaseSuccessor`.
+  - `ControlFlowEdgeGotoDefault` has been renamed to `ControlFlow::SuccessorTypes::GotoDefaultSuccessor`.
+  - `ControlFlowEdgeException` has been renamed to `ControlFlow::SuccessorTypes::ExceptionSuccessor`.
+
+> You should update any custom queries that use these entities to ensure that they continue working when the old names are removed in a future release.