Alternative to Observe for Sideways Data Loading

[sebmarkbage]

For reference. We're exploring an alternative to the observe function proposed in #3398.

Instead of having a separate function that defines a list of dependencies on Observables, we could allow the render function itself trigger a bunch of side-effects that registers the currently executing component with some Observables.

render() {
  var data = request(this.props.url);
  if (data === null) return <div>Loading...</div>;
  return <div>{data}</div>;
}

Side-effects! 😲

The implementation of request wouldn't return an Observable. Instead it would tell React that the currently executing component has a dependency on this Observable and needs to rerender anytime the subscription fires. It returns the last value received by this particular Observable... somehow.

function request(url) {
  return React.readAndSubscribe(Observable.fetch(url));
}

If an Observable is no longer subscribed to in the next render pass, it is automatically unsubscribed. Perhaps the Observable API is not ideal for this model.

[jimfb]

I've posted a PR (#3920) that implements the underlying API that would power these new semantics, mostly to continue/inform the discussion.

[gaearon]

I have mixed feelings about this. But I have to say I like that this keeps API surface minimal and doesn't introduce new intermediate states (e.g. componentDidReceivePropsAndNowGonnaUpdateObservedData).

[staltz]

render() {
  var data = request(this.props.url);
  if (data === null) return <div>Loading...</div>;
  return <div>{data}</div>;
}

• Mixing of concerns (render doesn't just render)
• Magic: it looks like data = request() is synchronous, but the real logic is async. Might make people rage when they can't match their expected abstraction with the actual functionality of the tool.

needs to rerender anytime the subscription fires

Subscriptions don't "fire", Observables do. Subscriptions are sinks. If Subscriptions are to send any data forward to somewhere, then you are looking for a second Observable (as a function of the first), not a Subscription.

function request(url) {
  return React.readAndSubscribe(Observable.fetch(url));
}

There is a problem here because you can't keep the "last value received by this Observable" because everytime you call readAndSubscribe() you give it a new Observable, namely Observable.fetch(url). You can solve this by making Observable.fetch() have internally a persistent registry of url -> Observable, but that sounds really bad, it's global shared state hidden as framework magic. The core problem is that you need to separate Observable creation from Observable consumption, but the suggested implementation of request(url) conflates them. "last value received" should be an Observable, because, well, you "can observe" the last value.

I recommend fully embracing Observables as they are, by Erik Meijer, or dropping the whole idea.

[glenjamin]

It seems to me that this makes server rendering much harder, because you'd now need to call render() to extract data dependencies.

[gaearon]

You can solve this by making Observable.fetch() have internally a persistent registry of url -> Observable, but that sounds really bad, it's global shared state hidden as framework magic.

I think a version of this was also present in the original observe hook proposal from #3398:

This ordering is important since it allows the provider of data to do reference counting of their cache. I.e. I can cache data for as long as nobody listens to it. If I unsubscribed immediately, then the reference count would go down to zero before I subscribe to the same data again.

In the original proposal, the lifetime of the observable was "until new props/state come in", and in this proposal, its lifetime is "until the next value comes in" which I agree doesn't really make sense for Observables. You can see that #3920 does not use Observables at all, presumably for this reason. One might of course turn an observable factory into a “data tracker provider” a la #3290 but it doesn't seem to have any additional value over just using data tracker directly.

That said, I'm concerned about DataTracker suggested in #3920 as I don't understand how it can be compatible with immutable data and plain object (or ImmutableJS) models.

At this point I'm starting to think that sideways data injection is solved well by higher-order components and doesn't need to be in the core. I'd like to learn what makes React team think otherwise. I think even seems that the proposal from #3920 could be implemented in the userland as a higher order component that just forceUpdates its child. Is this not so? I actually already asked this question and it got answered, oops!

[jimfb]

@glenjamin It doesn't make server-side rendering any harder, because you can just re-run render on the client to discover the dependencies. In fact, we do re-run render on the client anyway (to setup refs and callbacks and other component data structures), so we aren't making things any worse.

[glenjamin]

@jimfb If i'm following correctly, it doesn't give you a way to collect data dependencies to be resolved server-side, without calling renderToString() multiple times on the server?

[jimfb]

@gaearon Correct, there would be no benefit to using observables, unless your system already uses observables, in which case, by all means, do so! Observables propagate state on write, whereas a react render utilizes data on read. While observables are perfectly compatible with React, the model is actually the reverse of what React does internally, so, yeah, that's why the observable API doesn't add any additional value when talking with the react core.

@gaearon With regards to your point about immutable/plain javascript, this is super important and one we've give lots of thought! This system actually plays super nicely with both immutable and plain javascript, and you are encouraged to use immutable/plain javascript as much as possible! If you don't use the data tracker, everything behaves exactly the same as it behaves today. Sometimes, however (eg. large cyclic graphs), immutable data starts to run into troubles and the recommended solution is to break up the cycle by having keys that you lookup in a map (effectively introducing mutability into your immutable graph). This API allows the data store implementor to do that, without exposing where the mutable break-points are located. If you are able to effectively implement your app with immutable data, you are encouraged to do that, and you won't ever learn about this api! But when that isn't practical, you would be able to call into this API for better performance and sideways data loading. However, please don't feel obligated to ditch plain/immutable javascript just because we add a new API! We still love immutable and plain js!

[jimfb]

@glenjamin Ah, I see what you're asking. Either API (observables or this) will require some minor nonisomorphism (is that a word?). With this API, you SSR datastores would need to return the data eagerly (eagerly fetching isn't usually a problem on SSR because your database (source of truth) lives near by), and with Observables your SSR datastore would need to close all observable streams after loading but not close them on the client in order to receive updates from the server. So we aren't really making the situation any worse than the other proposals.

[gaearon]

@jimfb

Thanks for addressing this! So it is a bit like shouldComponentUpdate in reverse. It would not be the primary means of subscribing to data changes, but could be used for finer-grained subscriptions in large trees. Is this correct?

In the Flux architecture, what would be the subscription point? A Store? Would the people writing Stores be exposed to this API, or would rather the people writing Flux libraries be encouraged to use it? Would this be an encouraged pattern at all?

[jimfb]

@gaearon Yes, exactly, it's like shouldComponentUpdate in reverse! It allows datastores to provide "hints" to a component that "data they depend on has changed, and they might want to rerender". It does the rerenders in an optimal way to be maximally efficient (in some cases, even more efficient than you can be with immutablejs). See @josephsavona's #3920 (comment) for details.

This API would probably be used by people writing datastores, but potentially also by people writing flux libraries. If this is the API we choose, this pattern would be recommended for sideways data loading (ie. you shouldn't setup subscriptions to flux stores manually). All sideways data loading is still "discouraged" (ie. people SHOULD TRY to find a way to use just immutable props), but if you're sideways loading data anyway, it would be recommended that you let React track that data. Of course, you could still always do the tracking manually, just like today, if you felt the need to do your own thing.

One of the motivations is that: There is an oft quoted statistic that something like 80% of mixins are trying to implement sideways data loading. I'm not sure where that statistic came from, but it is certainly true that sideways data loading is one of the main thing mixins are used for, and it's also one of the more error prone things that people do in React, and we want to simplify that. See the motivation for Observe (#3398).

[slorber]

After reading the 3 proposals I don't understand everything of them but I think React should not try to solve this problem and keep it simple: UI = f(state).

However just a crazy idea: have you thought about a render function that could return a promise?

For example:

render: function() {
  var renderSync = <Spinner/>;
  var renderAsync = getUser(this.props.userId).then(user => <div>{user.name}</div>)
  return React.asyncRender(renderSync, renderAsync);
}

Not sure it is a good idea but I guess it wouldn't be a problem for isomorphic apps for React to wait for all render promises to complete...