DOC Document generated column support in the ORM #775

GuySartorelli · 2025-07-11T04:10:40Z

Issue

Add generated column with index to speed up AssetAdmin queries silverstripe-assets#693

en/02_Developer_Guides/00_Model/04_Data_Types_and_Casting.md

michalkleiner · 2025-07-11T05:31:22Z

It also seems the index support is only available from MySQL 8, should we mention that?

GuySartorelli · 2025-07-13T21:16:06Z

It also seems the index support is only available from MySQL 8, should we mention that?

The earliest supported version of MySQL is 8.0 as of CMS 6.0.0 so I don't think that needs to be mentioned.

michalkleiner

Reads well

en/02_Developer_Guides/00_Model/04_Data_Types_and_Casting.md

emteknetnz · 2025-07-15T01:19:38Z

en/08_Changelogs/6.1.0.md

+
+- sorting in a gridfield on a complex summary field: It's common to use a getter method to get some value derived from your database fields (e.g. a discounted price), and use that in `summary_fields`. With a generated column, you can remove the getter method and the [`GridFieldSortableHeader`](api:SilverStripe\Forms\GridField\GridFieldSortableHeader) component will be able to sort using the column.
+- using functional indexes: generated columns can be included in indexes, allowing you to sort or filter by complex expressions in an efficient way.
+- data integrity: unlike generating values inside an `onBeforeWrite()` extension hook, generated column values will be correct even if you update the record with raw SQL.


Suggested change

- data integrity: unlike generating values inside an `onBeforeWrite()` extension hook, generated column values will be correct even if you update the record with raw SQL.

- data integrity: unlike generating values inside an `onBeforeWrite()` extension hook, generated column values will be correct even if you update the record with raw SQL.

Probably also restoring old versions of records and archived records which have had columns added since the old record was created

I put the diff in a diff checker and there's no change in your suggested change?

Opps

Add something like this (feel free to alter)

This also ensures restored and archived records, with may have new columns since the record was created, remain accurate.

emteknetnz · 2025-07-15T01:20:31Z

en/08_Changelogs/6.1.0.md

+- sorting in a gridfield on a complex summary field: It's common to use a getter method to get some value derived from your database fields (e.g. a discounted price), and use that in `summary_fields`. With a generated column, you can remove the getter method and the [`GridFieldSortableHeader`](api:SilverStripe\Forms\GridField\GridFieldSortableHeader) component will be able to sort using the column.
+- using functional indexes: generated columns can be included in indexes, allowing you to sort or filter by complex expressions in an efficient way.
+- data integrity: unlike generating values inside an `onBeforeWrite()` extension hook, generated column values will be correct even if you update the record with raw SQL.
+- reduce repetition: instead of using a complex expression in multiple different places, you can just give the expression a name with a generated column.


I don't understand what this means?

Normally you'd just have a getMyValue() method on a model? i.e. logic in a single place?

If you have multiple SQL queries that use the same complex expression, you can replace those complex expressions with a single named generated column.

Still not really registering with me

What would an example of this be using the ORM?

I have some record.

I want to filter by an expression in a template, so I add a controller method that does a custom SQL query that uses that expression. Even something simple like CONCAT("FieldOne", "FieldTwo").

In another area of the code that's completely separate from the above controller, I am using the same DataObject model. I want to filter or sort by that same expression.

Instead of having CONCAT("FieldOne", "FieldTwo") in both places, I can just add a generated column named FieldConcat and use that field name instead.

emteknetnz · 2025-07-15T01:21:41Z

en/08_Changelogs/6.1.0.md

+
+Generated column support has been added to the MySQL database connector in `silverstripe/framework`, but is not guaranteed to work for other database servers.
+
+If you maintain a module that adds support for another database server, you'll need to implement the new [`DBSchemaManager::makeGenerated()`](SilverStripe\ORM\Connect\DBSchemaManager::makeGenerated()) and [`DBSchemaManager::needRebuildColumn()`](SilverStripe\ORM\Connect\DBSchemaManager::needRebuildColumn()) methods.


Suggested change

If you maintain a module that adds support for another database server, you'll need to implement the new [`DBSchemaManager::makeGenerated()`](SilverStripe\ORM\Connect\DBSchemaManager::makeGenerated()) and [`DBSchemaManager::needRebuildColumn()`](SilverStripe\ORM\Connect\DBSchemaManager::needRebuildColumn()) methods.

If you maintain a module that adds support for another database server, you'll need to implement the new [`DBSchemaManager::makeGenerated()`](SilverStripe\ORM\Connect\DBSchemaManager::makeGenerated()) and [`DBSchemaManager::needRebuildColumn()`](SilverStripe\ORM\Connect\DBSchemaManager::needRebuildColumn()) methods to support generated columns.

Presumbably you don't HAVE to update the the module, because there's no BC changes?

You do HAVE to update the module to support generated columns if you want to support generated columns lol. I think "to support generated columns" is very heavily implied but I'll add it to remove any hint of confusion.

emteknetnz · 2025-07-15T22:15:18Z

en/08_Changelogs/6.1.0.md

+
+- sorting in a gridfield on a complex summary field: It's common to use a getter method to get some value derived from your database fields (e.g. a discounted price), and use that in `summary_fields`. With a generated column, you can remove the getter method and the [`GridFieldSortableHeader`](api:SilverStripe\Forms\GridField\GridFieldSortableHeader) component will be able to sort using the column.
+- using functional indexes: generated columns can be included in indexes, allowing you to sort or filter by complex expressions in an efficient way.
+- data integrity: unlike generating values inside an `onBeforeWrite()` extension hook, generated column values will be correct even if you update the record with raw SQL.


Opps

Add something like this (feel free to alter)

This also ensures restored and archived records, with may have new columns since the record was created, remain accurate.

emteknetnz · 2025-07-15T22:18:54Z

en/08_Changelogs/6.1.0.md

+- sorting in a gridfield on a complex summary field: It's common to use a getter method to get some value derived from your database fields (e.g. a discounted price), and use that in `summary_fields`. With a generated column, you can remove the getter method and the [`GridFieldSortableHeader`](api:SilverStripe\Forms\GridField\GridFieldSortableHeader) component will be able to sort using the column.
+- using functional indexes: generated columns can be included in indexes, allowing you to sort or filter by complex expressions in an efficient way.
+- data integrity: unlike generating values inside an `onBeforeWrite()` extension hook, generated column values will be correct even if you update the record with raw SQL.
+- reduce repetition: instead of using a complex expression in multiple different places, you can just give the expression a name with a generated column.


Still not really registering with me

What would an example of this be using the ORM?

emteknetnz · 2025-07-15T23:10:53Z

en/02_Developer_Guides/00_Model/04_Data_Types_and_Casting.md

+        // Discount the car for a fixed percentage based on the condition of the car
+        // phpcs:ignore Generic.Files.LineLength.TooLong
+        'Discount' => 'Generated("Percentage", "CASE WHEN (\\"Condition\\"=\'Junk\') THEN 0.75 WHEN (\\"Condition\\"=\'Fair\') THEN 0.25 ELSE 0 END", "STORED")',
+        'DiscountPrice' => 'Generated("Currency", "\\"Price\\" * (1.0 - \\"Discount\\")", "VIRTUAL")',


Probably makes sense to use a heredoc for Discount and keep as a one liner for DiscountPrice to show both ways - see silverstripe/silverstripe-framework#11774 (comment) for example of the heredoc syntax

Done.
Used a nowdoc because it's more appropriate here - we don't want any string interpolation.

GuySartorelli · 2025-07-16T00:47:52Z

en/02_Developer_Guides/00_Model/04_Data_Types_and_Casting.md

+        'Condition' => 'Enum("New,Fair,Junk", "Fair")',
+        'Price' => 'Currency',
+        // Discount the car for a fixed percentage based on the condition of the car
+        'Discount' => <<<'SPEC'


See silverstripe/silverstripe-assets#694 (comment) for why I used SPEC as the delimiter.

We're skipping some best practice e.g. setting the table_name for the sake of brevity.

GuySartorelli · 2025-07-16T00:56:17Z

en/02_Developer_Guides/00_Model/04_Data_Types_and_Casting.md


 class Player extends DataObject
 {
+    // ...


Adding these to indicate we're skipping some best practices (e.g. table_name) for the sake of brevity.

* DOC Document generated column support in the ORM * DOC Add ellipses to indicate skipping bits for brevity We're skipping some best practice e.g. setting the table_name for the sake of brevity.

GuySartorelli mentioned this pull request Jul 6, 2025

Add generated column with index to speed up AssetAdmin queries silverstripe/silverstripe-assets#693

Closed

GuySartorelli force-pushed the pulls/6/generated-columns branch from 5166210 to 2b724a2 Compare July 11, 2025 04:39

michalkleiner reviewed Jul 11, 2025

View reviewed changes

en/02_Developer_Guides/00_Model/04_Data_Types_and_Casting.md Outdated Show resolved Hide resolved

GuySartorelli force-pushed the pulls/6/generated-columns branch from 2b724a2 to 271b222 Compare July 13, 2025 22:31

michalkleiner approved these changes Jul 13, 2025

View reviewed changes

emteknetnz requested changes Jul 15, 2025

View reviewed changes

GuySartorelli force-pushed the pulls/6/generated-columns branch from 271b222 to 261d34d Compare July 15, 2025 02:29

emteknetnz requested changes Jul 15, 2025

View reviewed changes

DOC Document generated column support in the ORM

43fe9dd

GuySartorelli force-pushed the pulls/6/generated-columns branch from 261d34d to 43fe9dd Compare July 16, 2025 00:46

GuySartorelli commented Jul 16, 2025

View reviewed changes

DOC Add ellipses to indicate skipping bits for brevity

d65770c

We're skipping some best practice e.g. setting the table_name for the sake of brevity.

GuySartorelli commented Jul 16, 2025

View reviewed changes

emteknetnz approved these changes Jul 17, 2025

View reviewed changes

emteknetnz merged commit c14e0a9 into silverstripe:6 Jul 17, 2025
3 checks passed

emteknetnz deleted the pulls/6/generated-columns branch July 17, 2025 05:45

	- data integrity: unlike generating values inside an `onBeforeWrite()` extension hook, generated column values will be correct even if you update the record with raw SQL.
	- data integrity: unlike generating values inside an `onBeforeWrite()` extension hook, generated column values will be correct even if you update the record with raw SQL.


		Generated column support has been added to the MySQL database connector in `silverstripe/framework`, but is not guaranteed to work for other database servers.

		If you maintain a module that adds support for another database server, you'll need to implement the new [`DBSchemaManager::makeGenerated()`](SilverStripe\ORM\Connect\DBSchemaManager::makeGenerated()) and [`DBSchemaManager::needRebuildColumn()`](SilverStripe\ORM\Connect\DBSchemaManager::needRebuildColumn()) methods.

DOC Document generated column support in the ORM #775

DOC Document generated column support in the ORM #775

Uh oh!

Conversation

GuySartorelli commented Jul 11, 2025

Issue

Uh oh!

Uh oh!

michalkleiner commented Jul 11, 2025

Uh oh!

GuySartorelli commented Jul 13, 2025

Uh oh!

michalkleiner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants