[Feature Request] Stream of custom events

[arathunku]

👋 Hi!

I believe this would be a feature request, and I'm unclear whether issue tracker is the correct place to raise this?

Context

Let's say we've 2-3 heavy DB queries and I want to create a report that's available as app. Ideally, automatically refetch the data at some interval or when user clicks a button.

In edit mode, I can select most of the cells as "evaluate automatically" and it's works nicely, but it doesn't work for apps.

Actual solution as workaround

At this moment, I get around this by creating "UI" module that's appending data info in a single frame, but this mode of writing a notebook that also works as an app leaves a lot to be desired. This is also very difficult to explain to people that are just starting with Elixir and want to publish their notebook as an app.

Example:

# one cell, it might also be a more complex form, other type of user input

load_btn = Kino.Control.button("Load data") |> Kino.render()

# another cell
frame = Kino.Frame.new() |> Kino.render()

defmodule UI do
  def show(frame) do
    # very heavy db query 
    results = [1, 2, 3]
    Kino.Frame.render(
      frame, Kino.Markdown.new("#{Enum.join(results, ", ")} #{DateTime.utc_now()}")
    )
  end
end

load_btn
|> Kino.listen(fn _event -> 
  UI.show(frame)
end)
UI.show(frame)
Kino.nothing()

## another cell !

# another cell, wants to access "results" too!
# another cell, it must be reloaded "load_btn" is clicked!

defmodule AnotherUI do
  def show(frame, results) do
    Kino.Frame.render(
      frame, Kino.Markdown.new("Different view #{Enum.join(results, ", ")} #{DateTime.utc_now()}")
    )
  end
end

another_frame = Kino.Frame.new() |> Kino.render()

load_btn
|> Kino.listen(fn _event ->
  # no way to get "results"?
  AnotherUI.show(another_frame, [4, 5, 6])
end)
AnotherUI.show(another_frame, [4, 5, 6])
Kino.nothing()

Expected solution

Kino already has API for streams via Kino.Control.stream/1. I would love to simplify the code above to something like:

load_btn = Kino.Control.button("Load data") |> Kino.render()
event = Kino.Control.event(:results)

load_btn 
|> Kino.listen(fn _event -> 
  # very heavy db query 
  results = [1, 2, 3]
  Kino.send(event, data: [results: results]) 
end)

# another cell

event
# or  [event, ...other events] |> Kino.Control.stream()
|> Kino.animate(fn event -> 
  Kino.Markdown.new("Some view #{Enum.join(event.data.results, ", ")} #{DateTime.utc_now()}")
end)

## another cell !

event
|> Kino.animate(fn event -> 
  Kino.Markdown.new("Different view #{Enum.join(event.data.results, ", ")} #{DateTime.utc_now()}")
end)

---

It's also very likely that I'm doing something wrong and this is already possible but given the validation here I don't think it's easy to do nor obvious from the documentation.

[josevalim]

I think #426 would give a bit more of structure to what you want to do. But you can also achieve it today by starting a process and having the process subscribe to the events and then update the frames as appropriate.

My concern with the pubsub style approach you mention is the possibility of races (if events are emitted before the animate subscribes?).

[jonatanklosko]

@arathunku you could invoke all handles from the same listener, no? What am I missing? :)

[arathunku]

You're both, of course, right. I'm aware of that too and you're not missing anything!

Unfortunetly, this isn't the reality of how I see people write notebooks at place where I work.

People start with creating and documenting cells. Usually markdown -> Elixir cell -> markdown -> elixir/smart cell -> markdown, this all works fine. They typically mark cells to be evaluated automatically, this is still fine... great UX, very beginner-friendly. Then, "let me just share this as app" with option to reload data initial data/some form at the beginning... This is a moment where I typically get asked for support on how to do that because it's like drawing the rest of the owl meme and they're lost. 😄

My idea with events was that it would be easy to explain, "feel" like rest of the events(input, interval, form) Kino may handle. Maybe in reality, I'm looking for a solution in apps to reevaluate cells and have other cells in apps reevaluate automatically too.

@josevalim, regarding possibility of races -> I think that's expected and this is already a case with current events from forms/inputs if Kino.listen is in other cells.

[jonatanklosko]

@arathunku actually, if you mark cells as reevaluating automatically, they get reevaluate in the app also. Which Livebook version are you using?

[arathunku]

🤔 @jonatanklosko I'm using main branch from about 4 weeks ago so almost v0.14 before it was released. I don't think it's a problem with the updates but structure and flow of creating notebooks that may at some point get shared as app. Example notebook:

<!-- livebook:{"app_settings":{"access_type":"public","auto_shutdown_ms":5000,"output_type":"rich","slug":"issue-469"}} -->

# Prototype idea

```elixir
Mix.install([{:kino, "~> 0.14.0"}])
```

## Section

<!-- livebook:{"reevaluate_automatically":true} -->

```elixir
Kino.Markdown.new(
  "Testing automatic eval in app model: #{DateTime.utc_now()}. Only rich outputs"
)
```

<!-- livebook:{"reevaluate_automatically":true} -->

```elixir
# heavy db query
# how to reload this cell in app with rich outputs only that would trigger notification in other apps?
{:ok, results} = {:ok, [1, 2, 3, 4, 5]}
```

<!-- livebook:{"reevaluate_automatically":true} -->

```elixir
Kino.Markdown.new("#{Enum.join(results, ", ")} #{DateTime.utc_now()}")
```

Right now, when editing, it's very intuitve for users that when they reevaluate the cell with DB query, any cell below it with a reference to variable will get reevaluated, smart cells included. How can app trigger this behaviour? 🤔

Right now my approach and recommendation was what's in the initial post - reactive frames listening to interval/button click.

[jonatanklosko]

@arathunku ok, I see what you mean. You have that baseline notebook and then want to convert to an app, and currently there is no app equivalent of reevaluating the query cell, so you need to rework it into a form.

I think we can solve this with an input button. It would be similar to the button control, but it would have a value, which is the number of clicks (bump-only counter). With that we can implement reload functionality by showing the button and doing Kino.Input.read(button) in every cell that should be recomputed on click.

So your notebook would become:

<!-- livebook:{"app_settings":{"access_type":"public","auto_shutdown_ms":5000,"output_type":"rich","slug":"issue-469"}} -->

# Prototype idea

```elixir
Mix.install([{:kino, "~> 0.14.0"}])
```

## Section

<!-- livebook:{"reevaluate_automatically":true} -->

```elixir
Kino.Markdown.new(
  "Testing automatic eval in app model: #{DateTime.utc_now()}. Only rich outputs"
)
```

<!-- livebook:{"reevaluate_automatically":true} -->

```elixir
reload_button = Kino.Input.button("Reload data")
```

```elixir
_ = Kino.Input.read(reload_button)

# heavy db query
# how to reload this cell in app with rich outputs only that would trigger notification in other apps?
{:ok, results} = {:ok, [1, 2, 3, 4, 5]}
```

<!-- livebook:{"reevaluate_automatically":true} -->

```elixir
Kino.Markdown.new("#{Enum.join(results, ", ")} #{DateTime.utc_now()}")
```

You can even read the button in multiple cells, so that clicking it would evaluate all of them.

This is a little bit indirect, but should do the job. What do you think?

[arathunku]

@jonatanklosko this would 100% solve the problem but adding "button" in input module just for this use case sounds... wrong? I'm unsure what other use cases are there.

One alternative that maybe feels more intiutive is empty form with just "submit"? I believe <form><button type="submit">Reload</button></form> is valid syntax, and then it would be a matter of reading it.

<!-- livebook:{"reevaluate_automatically":true} -->

```elixir
# may be expanded later with additional inputs!
# this currently doesn't work, form wants at least 1 input
form = Kino.Control.form([], submit: "Reload data")
```

<!-- livebook:{"reevaluate_automatically":true} -->

```elixir
# new functionality
_form = Kino.Control.read(form)
```

[josevalim]

There is a conceptual difference though. Controls are not read, only inputs are. But maybe that's something we can change as part of this work and maybe we can have a single unified module?

[jonatanklosko]

Forms are fundamentally different, because each user has their own values of the inputs, so "reading a form" is not a thing. The use case for forms is processing submissions asynchronously, and possibly rendering results to the individual user (though that depends on what you are doing).

On the other hand, Kino.Input elements rendered directly are shared across all users, so at each time they have a particular value, hence we can "read" those. And when the read value changes we can invalidate a cell and reevaluate it.

I think this use case is a good enough justification for input button, because it effectively is a bridge for the app user to reevaluate cells.

[jonatanklosko]

FWIW I would love to unify inputs and forms, but it's two fundamentally different models (shared vs not shared computation), so that's why we have both.

[arathunku]

There is a conceptual difference though. Controls are not read, only inputs are.

Oh, #TIL! Given examples in Kino.Control.stream/1` I assumed it would be similar.

button/1 in Kino.Input still feels wrong to me, particularly if we already have Kino.Control.button/1, this would be confusing.

[jonatanklosko]

Technically we could deprecate Kino.Control.button/1, because subscribing works for inputs also. Or we could use a different naming like Kino.Input.counter/1, except in this case we don't really need to show the value, so it's literally a button.

Just to be clear on the distinction, Kino.Control is about events, which are handled asynchronously, not related to the notebook evaluation. On the other hand, Kino.Input is about inputs shared across users, so it's almost like notebook parameters, which you can read as part of regular evaluation.

I agree that Kino.Input.button/1 feels a bit weird, but I can't think of a better way of forcing evaluation from the app UI (technically we could have a reevaluate element, but I wouldn't go this route). In other words, I think it's a good functionality to add, it's more of a question if we can make the API more clear.

[josevalim]

It probably makes sense to deprecate Control.button given we could use an empty form? And we delegate it to the input one during the deprecation?

[jonatanklosko]

@josevalim not even because of form, you can already do that, but rather because we support subscribing to inputs, so it would be like this:

button = Kino.Control.button("Button")
Kino.listen(button, &IO.inspect/1)
#=> %{type: :click, origin: "4mb6p7xbajyendmo"}

button = Kino.Input.button("Button")
Kino.listen(button, &IO.inspect/1)
#=> %{type: :change, value: 1, origin: "4mb6p7xbajyendmo"}

The event is different, so we want to be graceful with the deprecation, but you should be able to replace the control with input in all places.

[jonatanklosko]

The button input is essentially a stateful button control.