Limit in nested scope for hasMany relations

[fabripeco]

Hi,

I'm wondering if the 'limit' param works if added to the scope of nested relations, to limit the number of items of a hasMany relation. I have a strange behaviour in the response.
I have a BusinessPartner model with hasMany relation to Addresses

@hasMany(() => BuspartnerAddress, {keyTo: 'clientId', name: 'addresses'})
addresses: BuspartnerAddress[];

I want to get the businessPartners with only the first address for each one. My filter param looks like

{
  "order": ["name ASC"],
  "where": {
    "bustype": "CUSTOMER"
  },
  "include": [
    {
      "relation": "addresses",
      "scope": {
        "skip": 0,
        "limit": 1,
        "where": {
          "deleted": false
        }
      }
    }
  ]
}

The where clause in the scope of the include works well, but the limit: 1 param returns the address for only one businessPartner, not one for each businessPartner. If I increase the value of the limit, e.g. 10, the response returns max 10 address, differently distribuited on the businessPartners.
It looks like the query on the related models was performed only once and not for each 'parent' instance (businessPartner).

Am I wrong in something? Is this the intended behaviour?

I'm using a postgresql connector and this is my ecosystem (Nodejs v12.16.2 npm v6.14.4 - Postresql 12)

├── @loopback/authentication@4.2.3
├── @loopback/boot@2.2.0
├── @loopback/context@3.7.0
├── @loopback/core@2.5.0
├── @loopback/cron@0.2.7
├── @loopback/openapi-v3@3.3.1
├── @loopback/repository@2.4.0
├── @loopback/rest@4.0.0
├── @loopback/rest-explorer@2.2.0
├── @loopback/service-proxy@2.2.0
├── loopback-connector-postgresql@3.9.1
├── loopback-connector-rest@3.7.0

Thank you very much

PS: maybe related to #3453

[agnes512]

@fabripeco thank you reporting this. I can reproduce it on my end. And you're right, it is because of the way we query related data. In here, we query all related models with all target foreign keys. i.e when it fetches related models, it has filter:

{
  limit:10
  where:{
    userId:{inq: [1, 2, ...],
    deleted: false
}

instead of two separate queries.

The workaround I can think about atm is to manipulate the results D:
To fix the issue, we might need to do something similar to #5592, or change the way we query related models.

[fabripeco]

Ok, thanks for your reply.
Maybe another option for you could be to query the related models with only the where param (and sort and fields params) on the target repository and afterwards to accomplish the pagination params in memory (skip and limit). In this way you can keep a single query on the target repository. Just an idea.
Thank you!

[kpayson]

Maintainers's note: A separate issue, #6924, has been created.

I am having the same issue I think. In my schema Tenants have multiple Clients and Clients have multiple Providers. When I write a query for tenants that includes Clients and includes the Providers inside the Clients I get a max depth exceeded error.

  public async exportTenantData(
    @param.path.number('tenantId') tenantId: number
  ): Promise<any> {

    const inclusions = [
      {
        relation: 'clients',
        scope: {
          include: [
            {
              relation: 'providers',
              scope: {
                fields: { id: true }
              }
            },
          ]
        }
      },
    ];

    const filter = { include: inclusions };
    return this.tenantRepository.findById(tenantId, filter);
  }

[stale]

This issue has been marked stale because it has not seen activity within six months. If you believe this to be in error, please contact one of the code owners, listed in the CODEOWNERS file at the top-level of this repository. This issue will be closed within 30 days of being stale.

[stale]

This issue has been closed due to continued inactivity. Thank you for your understanding. If you believe this to be in error, please contact one of the code owners, listed in the CODEOWNERS file at the top-level of this repository.

[collaorodrigo7]

This should not have been closed w/o a fix. Also, This filter worked fine on LB3.
I am happy to take a look and try to help to solve it but I will need some guidance.
As a workaround, I first query the source model and then run multiple queries to get the related model includes for each id of the source model.
Is this how lb3 used to do it?
@agnes512 do you have any ideas on the best approach for this?

[collaorodrigo7]

Thanks, @achrinza
Do you have any suggestions on how we could implement it, and if what I mentioned is how LB3 used to do it and if the goal will be to make it like that?

[achrinza]

Apologies for letting it go stale; I'm not familiar with the LB3 codebase so unfortunately I can't provide info on how it's been done historically, but I think your idea makes sense given the current architecture of the @loopback/repository package.

If I'm not mistaken. this is the part of the code that needs changing:

loopback-next/packages/repository/src/relations/has-many/has-many.inclusion-resolver.ts

Lines 69 to 75 in cb29cbe

	const targetsFound = await findByForeignKeys(
	targetRepo,
	targetKey,
	sourceIds,
	scope,
	options,
	);

We can see it passes all foreign key instances into findByForeignKeys. As you've suggested, we could change this to instead run a new query for each foreign key and compare the result tally after each query with the user-specified limit.

Though I think we should introduce a new filter type (i.e. parentLimit? subLimit?, though I'm open to suggestions) so as to not break backwards compatibility; Both limit and the new parentLimt should be able to work together and independently.