Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 38 additions & 38 deletions Testing Glossary.csv
Original file line number Diff line number Diff line change
@@ -1,46 +1,46 @@
Name,Subject(s),Definition,Parent(s),Synonym(s),Notes
Unit Testing,Code (modules/functions),"""Testing the individual modules [of a program]"" [17, p. 438].",N/A,"Module Testing, Component Testing [16, p. 444]","Because of the definition of ""component"" (""a part of a system that can be tested in isolation"" [19], add to terms), this seems to apply to the testing of both modules and specific functions."
Integration Testing,Code (module interfaces),"""Testing the composition of modules"" done incrementally [17, pp. 438-439].",N/A,N/A,
System Testing,Code (most or all),"""Test[ing] the whole system against the user documentation and requirements specification"" [17, p. 439].",N/A,N/A,
Acceptance Testing,"Code, documentation (requirements)","Similar to system testing that is ""often performed under supervision of the user organization"" that focuses on usability [17, p. 439] and the needs of the customer(s) [16, p. 492].",N/A,N/A,
Installation Testing,"Code, documentation (installation)","Testing that focuses on the portability of the product, especially ""in an environment different from the one in which is has been developed"" [17, p. 439].",N/A,N/A,"Not one of the four levels of testing identified by the IEEE standard [16, p. 445]."
Bottom-up Testing,Code (most or all),"Integration testing that uses test drivers: “tool[s] that generate[] the test environment for a component to be tested” [17, p. 410].",Integration Testing,N/A,"""Test driver"" should be added to general terminology page."
Top-down Testing,Code (most or all),"Integration testing that uses test stubs: tools that ""simulate[] the function of a component not yet available"" [17, p. 410] by providing ""fake"" values to a given module to be tested [15, p. 110].",Integration Testing,N/A,"""Test stub"" should be added to general terminology page."
Big Bang Testing,Code (most or all),"The process of “integrat[ing] all modules in a single step and test[ing] the resulting system[]” [16, p. 489].",Integration Testing,N/A,
Sandwich Testing,Code (most or all),"The process of performing bottom-up and top-down testing by ""by defining a certain target layer in the hierarchy of the modules"" and working towards it from either end [16, p. 491].",Integration Testing,N/A,
Unit Testing,Code (modules/functions),"""Testing the individual modules [of a program]"" (van Vliet, 2000, p. 438).",N/A,"Module Testing, Component Testing (Peters and Pedrycz, 2000, p. 444)","Because of the definition of ""component"" (""a part of a system that can be tested in isolation"" (International Software Testing Qualifications Board, 2022), add to terms), this seems to apply to the testing of both modules and specific functions."
Integration Testing,Code (module interfaces),"""Testing the composition of modules"" done incrementally (van Vliet, 2000, pp. 438-439).",N/A,N/A,
System Testing,Code (most or all),"""Test[ing] the whole system against the user documentation and requirements specification"" (van Vliet, 2000, p. 439).",N/A,N/A,
Acceptance Testing,"Code, documentation (requirements)","Similar to system testing that is ""often performed under supervision of the user organization"" that focuses on usability (van Vliet, 2000, p. 439) and the needs of the customer(s) (Peters and Pedrycz, 2000, p. 492).",N/A,N/A,
Installation Testing,"Code, documentation (installation)","Testing that focuses on the portability of the product, especially ""in an environment different from the one in which is has been developed"" (van Vliet, 2000, p. 439).",N/A,N/A,"Not one of the four levels of testing identified by the IEEE standard (Peters and Pedrycz, 2000, p. 445)."
Bottom-up Testing,Code (most or all),"Integration testing that uses test drivers: “tool[s] that generate[] the test environment for a component to be tested” (van Vliet, 2000, p. 410).",Integration Testing,N/A,"""Test driver"" should be added to general terminology page."
Top-down Testing,Code (most or all),"Integration testing that uses test stubs: tools that ""simulate[] the function of a component not yet available"" (van Vliet, 2000, p. 410) by providing ""fake"" values to a given module to be tested (Patton, 2006, p. 110).",Integration Testing,N/A,"""Test stub"" should be added to general terminology page."
Big Bang Testing,Code (most or all),"The process of “integrat[ing] all modules in a single step and test[ing] the resulting system[]” (Peters and Pedrycz, 2000, p. 489).",Integration Testing,N/A,
Sandwich Testing,Code (most or all),"The process of performing bottom-up and top-down testing by ""by defining a certain target layer in the hierarchy of the modules"" and working towards it from either end (Peters and Pedrycz, 2000, p. 491).",Integration Testing,N/A,
Specification Testing,"Code, documentation (requirements)",,N/A,Static Black-Box Testing,Is this the most common name?
Terminology Checklist,Documentation,"Identifying specifications that are potentially unrealistic, vague, or incomplete [15, p. 61].",Specification Testing,N/A,
Coverage-Based Testing,Tests (coverage),"Testing derived from requirements ""depicted as a graph, where the nodes denote elementary requirements and the edges denote relations between [them]"" [17, p. 425].",Specification Testing,N/A,
Functional Testing,Code,"The process of ""entering inputs, receiving outputs, and checking the results"" [15, p. 64].",N/A,"Dynamic Black-Box Testing, Behavioural Testing",Is this the most common name?
Exploratory Testing,Code,"Exploring software to determine its features which are then tested [15, p. 65].",Functional Testing,N/A,"An alternative to dynamic black-box testing when a specification is not available [15, p. 65]."
Equivalence Partitioning,Tests (effficiency),"The process of dividing the infinite set of test cases into a finite set that is just as effective [15, p. 67].",Functional Testing,Equivalence Classing,"The opposite of this, ""exhaustive testing"", is ""probably not feasible"" [16, p. 461]."
Data Testing,Code,"The process of ""checking that information the user inputs [and] results"", both final and intermediate, ""are handled correctly"" [15, p. 70].","Functional Testing, Structural Testing",N/A,Could this be more specific?
Boundary Condition Testing,Code,"Testing using data ""at the edge[s] of the planned operational limits of the software"" [15, p. 72].",Data Testing,N/A,"Boundary conditions should be tested using ""the valid data just inside the boundary, …the last possible valid data, and …the invalid data just outside the boundary"" [15, p. 73] as well as the data on the boundary itself [16, p. 460]."
Terminology Checklist,Documentation,"Identifying specifications that are potentially unrealistic, vague, or incomplete (Patton, 2006, p. 61).",Specification Testing,N/A,
Coverage-Based Testing,Tests (coverage),"Testing derived from requirements ""depicted as a graph, where the nodes denote elementary requirements and the edges denote relations between [them]"" (van Vliet, 2000, p. 425).",Specification Testing,N/A,
Functional Testing,Code,"The process of ""entering inputs, receiving outputs, and checking the results"" (Patton, 2006, p. 64).",N/A,"Dynamic Black-Box Testing, Behavioural Testing",Is this the most common name?
Exploratory Testing,Code,"Exploring software to determine its features which are then tested (Patton, 2006, p. 65).",Functional Testing,N/A,"An alternative to dynamic black-box testing when a specification is not available (Patton, 2006, p. 65)."
Equivalence Partitioning,Tests (effficiency),"The process of dividing the infinite set of test cases into a finite set that is just as effective (Patton, 2006, p. 67).",Functional Testing,Equivalence Classing,"The opposite of this, ""exhaustive testing"", is ""probably not feasible"" (Peters and Pedrycz, 2000, p. 461)."
Data Testing,Code,"The process of ""checking that information the user inputs [and] results"", both final and intermediate, ""are handled correctly"" (Patton, 2006, p. 70).","Functional Testing, Structural Testing",N/A,Could this be more specific?
Boundary Condition Testing,Code,"Testing using data ""at the edge[s] of the planned operational limits of the software"" (Patton, 2006, p. 72).",Data Testing,N/A,"Boundary conditions should be tested using ""the valid data just inside the boundary, …the last possible valid data, and …the invalid data just outside the boundary"" (Patton, 2006, p. 73) as well as the data on the boundary itself (Peters and Pedrycz, 2000, p. 460)."
Buffer Overrun Testing,Code,,Boundary Condition Testing,N/A,Is this a meaningful distinction?
Sub-Boundary Condition Testing,Code,"Testing for boundary conditions ""that are internal to the software [but] aren’t necessarily apparent to an end user"" [15, p. 75].",Data Testing,N/A,
Sub-Boundary Condition Testing,Code,"Testing for boundary conditions ""that are internal to the software [but] aren’t necessarily apparent to an end user"" (Patton, 2006, p. 75).",Data Testing,N/A,
"Default, Empty, Blank, Null, Zero, and None Testing",Code,,Data Testing,N/A,Does this (and other types of data testing) have a better name? Does it make sense for these to be separate from data testing?
"Invalid, Wrong, Incorrect, and Garbage Data Testing",Code,,Data Testing,N/A,
Syntax-Driven Testing,Code,"Testing based on ""the syntax or constraint of input domains defined in requirement specification"" [21, p. 260].",Data Testing,N/A,
Decision Table-Based Testing,Code,"Testing based on conditions of requirements [16, pp. 448, 450].",Data Testing,N/A,Could this be more specific?
State Testing,Code,"The process of testing ""a program’s states and the transitions between them"" [15, p. 79].",Functional Testing,N/A,
Logic Flow Testing,Code,"Testing for correct state variables following different transitions [15, p. 83].",State Testing,N/A,Uses a state transition diagram (add to gen. term. page).
Performance Testing,Code,"""Identify[ing] weak points of a software system and quantify[ing] its shortcomings” [16, p. 447].",State Testing,N/A,Could this be more specific?
Syntax-Driven Testing,Code,"Testing based on ""the syntax or constraint of input domains defined in requirement specification"" (Intana et al., 2020, p. 260).",Data Testing,N/A,
Decision Table-Based Testing,Code,"Testing based on conditions of requirements (Peters and Pedrycz, 2000, pp. 448, 450).",Data Testing,N/A,Could this be more specific?
State Testing,Code,"The process of testing ""a program’s states and the transitions between them"" (Patton, 2006, p. 79).",Functional Testing,N/A,
Logic Flow Testing,Code,"Testing for correct state variables following different transitions (Patton, 2006, p. 83).",State Testing,N/A,Uses a state transition diagram (add to gen. term. page).
Performance Testing,Code,"""Identify[ing] weak points of a software system and quantify[ing] its shortcomings” (Peters and Pedrycz, 2000, p. 447).",State Testing,N/A,Could this be more specific?
Race Conditions and Bad Timing Testing,Code,,State Testing,N/A,Does this have a better name?
Repetition Testing,Code,"Testing ""the same operation over and over"", potentially up to ""thousands of attempts"" [15, p. 86].",State Testing,N/A,
Stress Testing,Code,"Testing ""the software under less-than-ideal conditions"" [15, p. 86].",State Testing,N/A,
Load Testing,Code,"Testing the software with as large of a load as possible (e.g., large inputs, many peripherals) [15, p. 86].",State Testing,N/A,
Structural Analysis,Code,"""The process of carefully and methodically reviewing the software design, architecture, or code for bugs without executing it"" [15, p. 92].",N/A,"Static White-Box Testing, Glass Box Testing",Is this the most common name?
Repetition Testing,Code,"Testing ""the same operation over and over"", potentially up to ""thousands of attempts"" (Patton, 2006, p. 86).",State Testing,N/A,
Stress Testing,Code,"Testing ""the software under less-than-ideal conditions"" (Patton, 2006, p. 86).",State Testing,N/A,
Load Testing,Code,"Testing the software with as large of a load as possible (e.g., large inputs, many peripherals) (Patton, 2006, p. 86).",State Testing,N/A,
Structural Analysis,Code,"""The process of carefully and methodically reviewing the software design, architecture, or code for bugs without executing it"" (Patton, 2006, p. 92).",N/A,"Static White-Box Testing, Glass Box Testing",Is this the most common name?
,,,,,
Structural Testing,Code,"Testing that uses ""information …from seeing what the code does and how it works"" [15, p. 105].",N/A,Dynamic White-Box Testing,Is this the most common name?
Code Coverage,"Code, Tests (coverage)","""[T]est[ing] the program’s states and the program’s flow among them"" [15, p. 117].",Structural Testing,"Control-Flow Coverage [17, pp. 421]",
Structural Testing,Code,"Testing that uses ""information …from seeing what the code does and how it works"" (Patton, 2006, p. 105).",N/A,Dynamic White-Box Testing,Is this the most common name?
Code Coverage,"Code, Tests (coverage)","""[T]est[ing] the program’s states and the program’s flow among them"" (Patton, 2006, p. 117).",Structural Testing,"Control-Flow Coverage (van Vliet, 2000, pp. 421)",
Statement/Line Coverage,"Code, Tests (coverage)",,Code Coverage,N/A,Line vs. statement coverage?
Branch Coverage,"Code, Tests (coverage)","Attempting to, ""at each branching node in the control graph, …[choose] all possible branches …at least once"" [17, p. 421].",Code Coverage,N/A,
Path Coverage,"Code, Tests (coverage)","""Attempting to cover all the paths in the software"" [15, p. 119].",Code Coverage,N/A,"""Path"" should be added to gen. term. page."
Condition Coverage,"Code, Tests (coverage)","Branch coverage that ""takes the extra conditions on the branch statements into account"" (e.g., all possible inputs to a Boolean expression) [15, p. 120].",Code Coverage,"Multiple Condition Coverage, Extended Branch Coverage",
Fault Seeding,"Code, Tests (coverage)","The introduction of faults to estimate the number of undiscovered faults in the system [17, p. 427].",Structural Testing,N/A,
Mutation Testing,"Code, Tests (coverage)","Running sets of tests on mutants of a system to maximize the number of mutants identified by a given test set [17, p. 429].",Structural Testing,N/A,"Could this be more specific? ""Mutant"" should be defined."
Strong Mutation Testing,"Code, Tests (coverage)","Mutation testing at the program level [17, p. 429].",Mutation Testing,N/A,
Weak Mutation Testing,"Code, Tests (coverage)","Mutation testing at the component level [17, p. 429].",Mutation Testing,N/A,
Gray-Box Testing,Code,"Testing ""the software as a black-box"" alongside limited information about ""what makes the software work"" [15, p. 218].",N/A,N/A,
Regression Testing,Code,"Retesting code using existing tests ""to make sure there are no unwanted changes [to the software’s behaviour]"" [16, p. 481].",,N/A,
Metamorphic Testing,Code,"Testing based on relations between changes to the input(s) and changes to the output(s) [22, pp. 67-68].",,N/A,
Branch Coverage,"Code, Tests (coverage)","Attempting to, ""at each branching node in the control graph, …[choose] all possible branches …at least once"" (van Vliet, 2000, p. 421).",Code Coverage,N/A,
Path Coverage,"Code, Tests (coverage)","""Attempting to cover all the paths in the software"" (Patton, 2006, p. 119).",Code Coverage,N/A,"""Path"" should be added to gen. term. page."
Condition Coverage,"Code, Tests (coverage)","Branch coverage that ""takes the extra conditions on the branch statements into account"" (e.g., all possible inputs to a Boolean expression) (Patton, 2006, p. 120).",Code Coverage,"Multiple Condition Coverage, Extended Branch Coverage",
Fault Seeding,"Code, Tests (coverage)","The introduction of faults to estimate the number of undiscovered faults in the system (van Vliet, 2000, p. 427).",Structural Testing,N/A,
Mutation Testing,"Code, Tests (coverage)","Running sets of tests on mutants of a system to maximize the number of mutants identified by a given test set (van Vliet, 2000, p. 429).",Structural Testing,N/A,"Could this be more specific? ""Mutant"" should be defined."
Strong Mutation Testing,"Code, Tests (coverage)","Mutation testing at the program level (van Vliet, 2000, p. 429).",Mutation Testing,N/A,
Weak Mutation Testing,"Code, Tests (coverage)","Mutation testing at the component level (van Vliet, 2000, p. 429).",Mutation Testing,N/A,
Gray-Box Testing,Code,"Testing ""the software as a black-box"" alongside limited information about ""what makes the software work"" (Patton, 2006, p. 218).",N/A,N/A,
Regression Testing,Code,"Retesting code using existing tests ""to make sure there are no unwanted changes [to the software’s behaviour]"" (Peters and Pedrycz, 2000, p. 481).",,N/A,
Metamorphic Testing,Code,"Testing based on relations between changes to the input(s) and changes to the output(s) (Kanewala and Yueh Chen, 2019, pp. 67-68).",,N/A,
2 changes: 1 addition & 1 deletion back.tex
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
%------------------------------------------------------------------------------
% Bibliography
%------------------------------------------------------------------------------
\printbibliography[heading=bibintoc]
\bibliography{references}

%------------------------------------------------------------------------------
% Appendix
Expand Down
Loading