Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
32b571a
Optimize relationship fetching with batch population and performance …
cursoragent Aug 6, 2025
8da5e64
Fix relationship batch processing to handle Document objects correctly
cursoragent Aug 6, 2025
f9a0a2d
Fix relationship access to use array syntax consistently
cursoragent Aug 6, 2025
0307073
Fix recursive batch processing in relationship population
cursoragent Aug 6, 2025
865eff5
Fix multi-level relationship processing using existing depth mechanism
cursoragent Aug 6, 2025
fdf2937
Fix nested relationship detection using fetch stack instead of depth
cursoragent Aug 6, 2025
986f5db
Use batch processing at all levels consistently
cursoragent Aug 6, 2025
c01c352
Update all single-document methods to use batch relationship processing
cursoragent Aug 6, 2025
dfd1637
Fix relationship cycle detection by adding collection attribute to re…
cursoragent Aug 6, 2025
f5e9c28
Fix relationship assignment - set to null when related document not f…
cursoragent Aug 6, 2025
d3e2b21
Fix relationship stack management - use Document objects directly lik…
cursoragent Aug 6, 2025
f43a9c2
TEMP: Disable cycle detection to test if that's causing the issue
cursoragent Aug 6, 2025
ca82060
MAJOR FIX: Handle Document objects vs string IDs in relationship grou…
cursoragent Aug 6, 2025
a41b7ce
TEMP: Simplify cycle detection to only check max depth to test if com…
cursoragent Aug 6, 2025
3d227ed
Improve cycle detection: Use property-based comparison instead of Doc…
cursoragent Aug 6, 2025
f6cecd6
Revert to simpler cycle detection - only check max depth
cursoragent Aug 6, 2025
8b58a58
REDESIGN: Implement breadth-first cycle detection for batch processing
cursoragent Aug 6, 2025
a4d0222
FIX: Make cycle detection less restrictive for breadth-first processing
cursoragent Aug 6, 2025
31873c7
SIMPLIFY: Much more permissive cycle detection for breadth-first proc…
cursoragent Aug 6, 2025
dedc488
EXPERIMENT: Remove cycle detection entirely for breadth-first processing
cursoragent Aug 6, 2025
0c68374
EXPERIMENT: Pure breadth-first processing with minimal cycle prevention
cursoragent Aug 6, 2025
db23d76
OPTIMIZE: Map-based duplicate prevention replaces cycle detection
cursoragent Aug 6, 2025
0a3ef8c
ULTIMATE OPTIMIZATION: Skip already-populated relationships + map-bas…
cursoragent Aug 6, 2025
34511fe
DEBUG: Add extensive logging to track relationship population issues
cursoragent Aug 7, 2025
52723c2
DEBUG: Add var_dump for animal relationships specifically
cursoragent Aug 7, 2025
bffb3be
MAJOR FIX: Handle multiple documents referencing same related ID in o…
cursoragent Aug 7, 2025
f6a05fc
CLEANUP: Remove excessive animal-specific debug output
cursoragent Aug 7, 2025
52628e2
Add detailed debugging to relationship population
cursoragent Aug 7, 2025
cd6b022
Add simple relationship test to debug basic functionality
cursoragent Aug 7, 2025
1bbc241
Add comprehensive debugging to relationship population
cursoragent Aug 7, 2025
7f65e74
Add extensive debugging to relationship population flow
cursoragent Aug 7, 2025
2eb50a8
Add focused debugging for nested relationship selection flow
cursoragent Aug 7, 2025
400f860
Fix method signatures for breadth-first batch processing
cursoragent Aug 8, 2025
e0f7d28
Add debugging for relationship population flow and fix early returns
cursoragent Aug 8, 2025
a5710d7
Merge remote-tracking branch 'origin/main' into optimize-relationship…
abnegate Aug 27, 2025
2556b52
Merge remote-tracking branch 'origin/main' into optimize-relationship…
abnegate Sep 26, 2025
f18a985
Merge remote-tracking branch 'origin/main' into optimize-relationship…
abnegate Oct 1, 2025
ff40787
Fix population
abnegate Oct 1, 2025
c7cfab6
Clean up
abnegate Oct 1, 2025
741ef0c
Fix stan
abnegate Oct 1, 2025
1af779f
Add benchmark
abnegate Oct 1, 2025
5127fb9
Fix depth early exit
abnegate Oct 1, 2025
cfd14b7
Simplify back reference removal
abnegate Oct 1, 2025
272cad8
Chunk large data for relationship finds
abnegate Oct 2, 2025
cc3cbcd
Update benchmark
abnegate Oct 2, 2025
dff915b
Increase limit
abnegate Oct 2, 2025
18926bf
Fix limits
abnegate Oct 2, 2025
cefe7c6
Fix unit tests
abnegate Oct 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions bin/cli.php
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
require_once '/usr/src/code/vendor/autoload.php';

use Utopia\CLI\CLI;
use Utopia\CLI\Console;

ini_set('memory_limit', '-1');

Expand All @@ -11,5 +12,13 @@
include 'tasks/load.php';
include 'tasks/index.php';
include 'tasks/query.php';
include 'tasks/relationships.php';

$cli
->error()
->inject('error')
->action(function ($error) {
Console::error($error->getMessage());
});

$cli->run();
3 changes: 3 additions & 0 deletions bin/relationships
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh

php /usr/src/code/bin/cli.php relationships $@
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Quote forwarded CLI arguments.

$@ without quotes re-splits arguments containing spaces, so the CLI task receives mangled input. Wrap it in quotes.

-php /usr/src/code/bin/cli.php relationships $@
+php /usr/src/code/bin/cli.php relationships "$@"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
php /usr/src/code/bin/cli.php relationships $@
php /usr/src/code/bin/cli.php relationships "$@"
🤖 Prompt for AI Agents
In bin/relationships around line 3, the script forwards CLI args using $@
unquoted which re-splits arguments containing spaces; change the invocation to
pass the arguments quoted (i.e. use "$@") so each original argument is preserved
and forwarded intact to cli.php.

120 changes: 59 additions & 61 deletions bin/tasks/index.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,105 +10,103 @@
use Utopia\CLI\CLI;
use Utopia\CLI\Console;
use Utopia\Database\Adapter\MariaDB;
use Utopia\Database\Adapter\Mongo;
use Utopia\Database\Adapter\MySQL;
use Utopia\Database\Adapter\Postgres;
use Utopia\Database\Database;
use Utopia\Database\PDO;
use Utopia\Mongo\Client;
use Utopia\Validator\Boolean;
use Utopia\Validator\Text;

/**
* @Example
* docker compose exec tests bin/index --adapter=mysql --name=testing
*/

$cli
->task('index')
->desc('Index mock data for testing queries')
->param('adapter', '', new Text(0), 'Database adapter')
->param('name', '', new Text(0), 'Name of created database.')
->action(function ($adapter, $name) {
->param('sharedTables', false, new Boolean(true), 'Whether to use shared tables', true)
->action(function (string $adapter, string $name, bool $sharedTables) {
$namespace = '_ns';
$cache = new Cache(new NoCache());

switch ($adapter) {
case 'mongodb':
$client = new Client(
$name,
'mongo',
27017,
'root',
'example',
false
);

$database = new Database(new Mongo($client), $cache);
break;

case 'mariadb':
$dbHost = 'mariadb';
$dbPort = '3306';
$dbUser = 'root';
$dbPass = 'password';

$pdo = new PDO("mysql:host={$dbHost};port={$dbPort};charset=utf8mb4", $dbUser, $dbPass, MariaDB::getPDOAttributes());

$database = new Database(new MariaDB($pdo), $cache);
break;

case 'mysql':
$dbHost = 'mysql';
$dbPort = '3307';
$dbUser = 'root';
$dbPass = 'password';

$pdo = new PDO("mysql:host={$dbHost};port={$dbPort};charset=utf8mb4", $dbUser, $dbPass, MySQL::getPDOAttributes());
$dbAdapters = [
'mariadb' => [
'host' => 'mariadb',
'port' => 3306,
'user' => 'root',
'pass' => 'password',
'dsn' => static fn (string $host, int $port) => "mysql:host={$host};port={$port};charset=utf8mb4",
'adapter' => MariaDB::class,
'pdoAttr' => MariaDB::getPDOAttributes(),
],
'mysql' => [
'host' => 'mysql',
'port' => 3307,
'user' => 'root',
'pass' => 'password',
'dsn' => static fn (string $host, int $port) => "mysql:host={$host};port={$port};charset=utf8mb4",
'adapter' => MySQL::class,
'pdoAttr' => MySQL::getPDOAttributes(),
],
'postgres' => [
'host' => 'postgres',
'port' => 5432,
'user' => 'postgres',
'pass' => 'password',
'dsn' => static fn (string $host, int $port) => "pgsql:host={$host};port={$port}",
'adapter' => Postgres::class,
'pdoAttr' => Postgres::getPDOAttributes(),
],
];

if (!isset($dbAdapters[$adapter])) {
Console::error("Adapter '{$adapter}' not supported");
return;
}

$database = new Database(new MySQL($pdo), $cache);
break;
$cfg = $dbAdapters[$adapter];

default:
Console::error('Adapter not supported');
return;
}
$pdo = new PDO(
($cfg['dsn'])($cfg['host'], $cfg['port']),
$cfg['user'],
$cfg['pass'],
$cfg['pdoAttr']
);

$database->setDatabase($name);
$database->setNamespace($namespace);
$database = (new Database(new ($cfg['adapter'])($pdo), $cache))
->setDatabase($name)
->setNamespace($namespace)
->setSharedTables($sharedTables);

Console::info("greaterThan('created', ['2010-01-01 05:00:00']), equal('genre', ['travel'])");
Console::info("Creating key index 'createdGenre' on 'articles' for created > '2010-01-01 05:00:00' and genre = 'travel'");
$start = microtime(true);
$database->createIndex('articles', 'createdGenre', Database::INDEX_KEY, ['created', 'genre'], [], [Database::ORDER_DESC, Database::ORDER_DESC]);
$time = microtime(true) - $start;
Console::success("{$time} seconds");
Console::success("Index 'createdGenre' created in {$time} seconds");

Console::info("equal('genre', ['fashion', 'finance', 'sports'])");
Console::info("Creating key index 'genre' on 'articles' for genres: fashion, finance, sports");
$start = microtime(true);
$database->createIndex('articles', 'genre', Database::INDEX_KEY, ['genre'], [], [Database::ORDER_ASC]);
$time = microtime(true) - $start;
Console::success("{$time} seconds");
Console::success("Index 'genre' created in {$time} seconds");

Console::info("greaterThan('views', 100000)");
Console::info("Creating key index 'views' on 'articles' for views > 100000");
$start = microtime(true);
$database->createIndex('articles', 'views', Database::INDEX_KEY, ['views'], [], [Database::ORDER_DESC]);
$time = microtime(true) - $start;
Console::success("{$time} seconds");
Console::success("Index 'views' created in {$time} seconds");

Console::info("search('text', 'Alice')");
Console::info("Creating fulltext index 'fulltextsearch' on 'articles' for search term 'Alice'");
$start = microtime(true);
$database->createIndex('articles', 'fulltextsearch', Database::INDEX_FULLTEXT, ['text']);
$time = microtime(true) - $start;
Console::success("{$time} seconds");
Console::success("Index 'fulltextsearch' created in {$time} seconds");

Console::info("contains('tags', ['tag1'])");
Console::info("Creating key index 'tags' on 'articles' for tags containing 'tag1'");
$start = microtime(true);
$database->createIndex('articles', 'tags', Database::INDEX_KEY, ['tags']);
$time = microtime(true) - $start;
Console::success("{$time} seconds");
});

$cli
->error()
->inject('error')
->action(function (Exception $error) {
Console::error($error->getMessage());
Console::success("Index 'tags' created in {$time} seconds");
});
Loading