Skip to content

DOC Document generated column support in the ORM #775

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
emteknetnz merged 2 commits into from
Jul 17, 2025
Merged

DOC Document generated column support in the ORM #775

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
43fe9dd
DOC Document generated column support in the ORM
GuySartorelli Jul 11, 2025
d65770c
DOC Add ellipses to indicate skipping bits for brevity
GuySartorelli Jul 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions en/02_Developer_Guides/00_Model/04_Data_Types_and_Casting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ use SilverStripe\ORM\DataObject;

class Player extends DataObject
{
// ...
Copy link
Member Author

@GuySartorelli GuySartorelli Jul 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding these to indicate we're skipping some best practices (e.g. table_name) for the sake of brevity.

private static $db = [
'PlayerNumber' => 'Int',
'FirstName' => 'Varchar(255)',
Expand Down Expand Up @@ -50,6 +51,7 @@ Most `DBField` subclasses will be validated using a [`FieldValidator`](api:Silve
| `Enum` | [`DBEnum`](api:SilverStripe\ORM\FieldType\DBEnum) | An enumeration of a set of strings that can store a single value | Must be one of the defined values |
| `Float` | [`DBFloat`](api:SilverStripe\ORM\FieldType\DBFloat) | A floating point number | Must be numeric |
| `ForeignKey` | [`DBForeignKey`](api:SilverStripe\ORM\FieldType\DBForeignKey) | A special `Int` field used for foreign keys in `has_one` relationships | Must be an int |
| `Generated` | [`DBGenerated`](api:SilverStripe\ORM\FieldType\DBGenerated) | A column which has its value generated by the database itself, usually based on other column values | N/A |
| `HTMLFragment` | [`DBHTMLText`](api:SilverStripe\ORM\FieldType\DBHTMLText) | A variable-length string of up to 2MB, designed to store HTML. Doesn't process [shortcodes](/developer_guides/extending/shortcodes/) | Must be a string |
| `HTMLText` | [`DBHTMLText`](api:SilverStripe\ORM\FieldType\DBHTMLText) | A variable-length string of up to 2MB, designed to store HTML. Processes [shortcodes](/developer_guides/extending/shortcodes/) | Must be a string |
| `HTMLVarchar` | [`DBHTMLVarchar`](api:SilverStripe\ORM\FieldType\DBHTMLVarchar) | A variable-length string of up to 255 characters, designed to store HTML. Can process [shortcodes](/developer_guides/extending/shortcodes/) with additional configuration | String must not be longer than specified length |
Expand Down Expand Up @@ -92,6 +94,7 @@ use SilverStripe\ORM\DataObject;

class Car extends DataObject
{
// ...
private static $db = [
'Wheels' => 'Int(4)',
'Condition' => 'Enum("New,Fair,Junk", "Fair")',
Expand All @@ -103,6 +106,66 @@ class Car extends DataObject
> [!NOTE]
> `Enum` fields will use the first defined value as the default if you don't explicitly declare one. In the example above, the default value would be "New" if it hadn't been declared.

## Generated columns

Generated columns are database columns where the value is generated inside the database, rather than being set by a user. They're usually based on other columns in the database and can be either generated when the record is updated and stored, or generated when requested in which case they aren't stored.

Some good use cases for generated columns include:

- sorting in a gridfield on a complex summary field: It's common to use a getter method to get some value derived from your database fields (e.g. a discounted price), and use that in `summary_fields`. With a generated column, you can remove the getter method and the [`GridFieldSortableHeader`](api:SilverStripe\Forms\GridField\GridFieldSortableHeader) component will be able to sort using the column.
- using functional indexes: generated columns can be included in indexes, allowing you to sort or filter by complex expressions in an efficient way.
- data integrity: unlike generating values inside an `onBeforeWrite()` method, generated column values will be correct even if you update the record with raw SQL. You also don't need to manually update values for historic records (e.g. when using [versioning](https://docs.silverstripe.org/en/6/developer_guides/model/versioning/#versioning)) even if you change the logic that determines the value.
- reduce repetition: instead of using a complex expression in multiple different places, you can just give the expression a name with a generated column.

The format for adding a generated column is `Generated("<datatype>", "<expression>", "<STORED|VIRTUAL>")`. Let's break that down:

|argument|explanation|example|
|---|---|---|
|`<datatype>`|The injector specification for a `DBField` instance that represents your generated field in the database and when getting its value, etc.|`Varchar(255)`, `Boolean`, etc|
|`<expression>`|The SQL expression used to generate values for this field. Best practice is to use ANSI quotes around column names. You need to add more escape characters as you might expect, e.g. to represent a FQCN in a string literal you will need something like `\'App\\\\\\\\Model\\\\\\\\Car\'`.|`\\"Price\\" * (1.0 - \\"Discount\\")`|
| `<STORED\|VIRTUAL>` | Whether the value is calculated when the record is updated and then stored in the database (use `STORED`), or calculated only when your query includes it (use `VIRTUAL`). |`STORED`|

> [!TIP]
> For lengthy or complex expressions, it is often best to use a [nowdoc](https://www.php.net/manual/en/language.types.string.php#language.types.string.syntax.nowdoc) multi-line string to make the expression easier to read.
>
> A nowdoc will also treat backslashes literally, which can be especially useful when dealing with FQCN in the expression.

If you had a car model and wanted to calculate a discounted price using a generated column, you could do something like this:

```php
namespace App\Model;

use SilverStripe\ORM\DataObject;

class Car extends DataObject
{
// ...
private static $db = [
'Condition' => 'Enum("New,Fair,Junk", "Fair")',
'Price' => 'Currency',
// Discount the car for a fixed percentage based on the condition of the car
'Discount' => <<<'SPEC'
Copy link
Member Author

@GuySartorelli GuySartorelli Jul 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See silverstripe/silverstripe-assets#694 (comment) for why I used SPEC as the delimiter.

Generated(
"Percentage",
"CASE WHEN (\"Condition\"='Junk') THEN 0.75
WHEN (\"Condition\"='Fair') THEN 0.25
ELSE 0
END",
"STORED"
)
SPEC,
'DiscountPrice' => 'Generated("Currency", "\\"Price\\" * (1.0 - \\"Discount\\")", "VIRTUAL")',
Copy link
Member

@emteknetnz emteknetnz Jul 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably makes sense to use a heredoc for Discount and keep as a one liner for DiscountPrice to show both ways - see silverstripe/silverstripe-framework#11774 (comment) for example of the heredoc syntax

Copy link
Member Author

@GuySartorelli GuySartorelli Jul 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.
Used a nowdoc because it's more appropriate here - we don't want any string interpolation.

];
}
```

> [!NOTE]
> You cannot set values for generated columns. If you use form field scaffolding, a read-only form field will be scaffolded for all of your generated columns.
>
> Some SQL servers will simply ignore the value, but others will throw an error. If an error is thrown, the ORM will throw a [`GeneratedColumnValueException`](api:SilverStripe\ORM\Connect\GeneratedColumnValueException) exception.
>
> If that happens when calling [`DataObject::write()`](api:SilverStripe\ORM\DataObject::write()), the exception will be caught and a [`ValidationException`](api:SilverStripe\Core\Validation\ValidationException) will be thrown instead. The CMS catches any `ValidationException` and displays them as user friendly validation errors in edit forms.

## Formatting output

The data type does more than set up the correct database schema. They can also define methods and formatting helpers for
Expand All @@ -120,6 +183,7 @@ use SilverStripe\ORM\FieldType\DBField;

class Player extends DataObject
{
// ...
public function getName()
{
return DBField::create_field('Varchar', $this->FirstName . ' ' . $this->LastName);
Expand Down Expand Up @@ -206,6 +270,7 @@ use SilverStripe\ORM\DataObject;

class Player extends DataObject
{
// ...
private static $casting = [
'Name' => 'Varchar',
];
Expand Down Expand Up @@ -285,6 +350,7 @@ use SilverStripe\ORM\DataObject;
*/
class Product extends DataObject
{
// ...
private static $db = [
'Title' => 'Varchar(255)',
//cost in pennies/cents
Expand Down
23 changes: 23 additions & 0 deletions en/08_Changelogs/6.1.0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ title: 6.1.0 (unreleased)
- [Composite indexes for `default_sort`](#sort-indexes)
- [Database query caching](#database-query-caching)
- [Drop indexes in `indexes` configuration](#drop-indexes)
- [Generated column support](#generated-columns)
- [Non-blocking file-based sessions](#non-blocking-sessions)
- [Password strength feedback](#password-strength-feedback)
- [Other new features and enhancements](#other-new)
Expand Down Expand Up @@ -99,6 +100,27 @@ If you are using the [`DataObjectSchema::databaseIndexes()`](api:SilverStripe\OR

See the [indexes documentation](/developer_guides/model/indexes/) for more details.

### Generated column support {#generated-columns}

Generated columns are database columns where the value is generated inside the database, rather than being set by a user. They're usually based on other columns in the database and can be either generated when the record is updated and stored, or generated when requested in which case they aren't stored.

Some good use cases for generated columns include:

- sorting in a gridfield on a complex summary field: It's common to use a getter method to get some value derived from your database fields (e.g. a discounted price), and use that in `summary_fields`. With a generated column, you can remove the getter method and the [`GridFieldSortableHeader`](api:SilverStripe\Forms\GridField\GridFieldSortableHeader) component will be able to sort using the column.
- using functional indexes: generated columns can be included in indexes, allowing you to sort or filter by complex expressions in an efficient way.
- data integrity: unlike generating values inside an `onBeforeWrite()` method, generated column values will be correct even if you update the record with raw SQL. You also don't need to manually update values for historic records (e.g. when using [versioning](https://docs.silverstripe.org/en/6/developer_guides/model/versioning/#versioning)) even if you change the logic that determines the value.
- reduce repetition: instead of using a complex expression in multiple different places, you can just give the expression a name with a generated column.
Copy link
Member

@emteknetnz emteknetnz Jul 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what this means?

Normally you'd just have a getMyValue() method on a model? i.e. logic in a single place?

Copy link
Member Author

@GuySartorelli GuySartorelli Jul 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you have multiple SQL queries that use the same complex expression, you can replace those complex expressions with a single named generated column.

Copy link
Member

@emteknetnz emteknetnz Jul 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not really registering with me

What would an example of this be using the ORM?

Copy link
Member Author

@GuySartorelli GuySartorelli Jul 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I have some record.
  • I want to filter by an expression in a template, so I add a controller method that does a custom SQL query that uses that expression. Even something simple like CONCAT("FieldOne", "FieldTwo").
  • In another area of the code that's completely separate from the above controller, I am using the same DataObject model. I want to filter or sort by that same expression.

Instead of having CONCAT("FieldOne", "FieldTwo") in both places, I can just add a generated column named FieldConcat and use that field name instead.


See [data types and casting](/developer_guides/model/data_types_and_casting/#generated-columns) for details about using generated columns.

#### Support in other SQL servers

Generated column support has been added to the MySQL database connector in `silverstripe/framework`, but is not guaranteed to work for other database servers.

If you maintain a module that adds support for another database server, you'll need to implement the new [`DBSchemaManager::makeGenerated()`](SilverStripe\ORM\Connect\DBSchemaManager::makeGenerated()) and [`DBSchemaManager::needRebuildColumn()`](SilverStripe\ORM\Connect\DBSchemaManager::needRebuildColumn()) methods to support generated columns.

You likely also need to adjust your implementation of [`DBSchemaManager::alterTable()`](SilverStripe\ORM\Connect\DBSchemaManager::alterTable()) to drop and recreate columns (instead of just updating their schema in place) based on `$advancedOptions['rebuildCols']`, and [`DBSchemaManager::fieldList()`](SilverStripe\ORM\Connect\DBSchemaManager::fieldList()) to normalise the representation of generated columns.

### Non-blocking file-based sessions {#non-blocking-sessions}

The default file-based session handler for PHP holds a lock on the session file while the session is open. This means that multiple concurrent requests from the same user have to wait for one another to finish processing after a session has been started. This includes AJAX requests.
Expand Down Expand Up @@ -134,6 +156,7 @@ This functionality will not work in your front-end forms out of the box as the r

- New [`FileIDHelper::VARIANT_SEPARATOR`](api:SilverStripe\Assets\FilenameParsing\FileIDHelper::VARIANT_SEPARATOR) improves discoverability of the string that separates variant names from original file names in the asset system.
- When executing commands via the [Sake CLI application](/developer_guides/cli/sake/), both the [`Sake`](api:SilverStripe\Cli\Sake) instance and the `Symfony\Component\Console\Command\Command` instance are added to the dependency injector. See [accessing sake from outside a command](/developer_guides/cli/sake/#sake-injector) for more details.
- The [`File`](api:SilverStripe\Assets\File) class now has a `IsFolder` generated column which is included in a new index. This makes sorting files in the asset admin faster.
- Previously there were a large number of unnecessary AJAX requests made to fetch the form schema for the search form for sections of the CMS that are searchable such as the site tree i.e. the list of pages on `/admin/page`. This has been fixed so these requests are only made when the filter button is clicked. Note this enhancement was originally released as a patch for CMS 5.4.
- There have been a number of other smaller performance enhancements that have been included in this release. Some of these enhacements were also released as patch releases for CMS 5.4.

Expand Down