The Great Migration

[ErikBjare]

There are currently a few migrations planned, here I'll try to document them so that we can have some general discussion and clarity on what migration debts have to be repaid.

• bundle: aw-server-python to aw-server-rust

  • This is the big one. aw-server-rust has been properly battle-tested by now (especially in aw-android) and is ready to become the default. We're currently (April 2020, v0.9) shipping it in the bundle as an alternative server that the user can choose to start with aw-qt.
  • Practically, all of the migrations below depend on this one.

• some watchers: 'unknown' hostnames to correct hostname

  • affects: aw-watcher-web, aw-android
  • currently create buckets with hostname "unknown", leading to them being assumed to have to belong to a certain hostname when querying.
  • new behavior (proposal): watchers use the hostname from the server's /info API endpoint when creating the bucket.
  • new behavior (proposal): watchers can create buckets with a magic hostname which automatically uses the server hostname. It's more convenient for watcher developers than the above proposal.
  • new behavior (proposal): watchers no longer have hostname in their name, they are assumed to be owned by the server they have requested to create a bucket for. All buckets in /api/0/buckets are local and "external" buckets from other devices can be on a seperate endpoint.
  • planned migration: Change buckets with a certain signature to the local server hostname.

• aw-server: hostnames to device IDs (Change how hostname/devices work #302)

  • hostnames are a bad way to identify an instance, especially so as hostnames sometimes change (transient hostnames) for network reasons.
  • planned migration: migrate to using device_id.

• some watchers: non-unique bucket IDs

  • we need a fix for this to allow having buckets from several hosts with identical IDs (aw-watcher-web-chrome), but we don't yet have one.

• some watchers: old bucket types (discussed in Rework the buckettype names and data formats #201)

  • We have been trying to adhere to a better standard for bucket type names (namespaces), which some of the older watchers don't use (currentwindow, afkstatus).
  • Before we perform this migration we might want to improve the docs on all the known and somewhat standard bucket types.

• aw-android: bucket type and id

  • aw-android currently uses currentwindow as its bucket type. This is not appropriate, as the event data schema is different (no title, for example).
  • affects: aw-android and aw-webui
  • best performed by: aw-android? aw-server-rust?

• aw-server-python: default datastore from peewee to sqlite

  • indefinitely postponed in favor of switching to aw-server-rust

There are more info about these in issues that I haven't yet referenced here.

[johan-bjareholt]

some watchers: old bucket types

My assumed plan for this is:

1. Fix so the webui supports both new and old names
2. Make aw-watcher-afk/aw-watcher-window/aw-watcher-android use the new names
3. Make aw-server-rust have a migration which changes the old names for the new names
4. (Optional) Make newly created buckets with the old name be replaced with the new name. This is necessary for imports of old buckets to work.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[johan-bjareholt]

Updated your post with another point

• new behavior (proposal): watchers no longer have hostname in their name, they are assumed to be owned by the server they have requested to create a bucket for. All buckets in /api/0/buckets are local and "external" buckets from other devices can be on a seperate endpoint.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[timwsuqld]

not stale. @ErikBjare do we have a tag to prevent things like this being marked as stale?

[petrroll]

Hi @ErikBjare I've just recently found out about AW Watcher and first off, amazing work!

But I found the rules system a bit lacking, especially if ones wants to have custom watchers influence what category gets selected.

My use case would be to have a virtual destkop name watcher (https://github.com/petrroll/aw-watcher-win-vdesktop) and force categorization of all windows if "Personal" virtual destkop is active as personal (even if it's Visual studio, which would otherwise be categorized as work).

The best solution I came up with was to extend the categorization rules engine with following:

• Each rule has a boolean: If selected, to consider all children rules, this rule's rule must match
• Each rule has a priority, by default the priorities are 10^ but can be overriten (this replaces the "deepest rule wins)
• Each rule allow specification of "on what data it wants to match", here I'm not sure if one would specifiy which bucket types or which "event keys", but in either case, the default would again be what it is. With an option to override it.

That way I could have the normal hierarchy for work stuff and then have another tree of rules with topmost rule matching on my custom watcher's bucket / event for virtual destkop name "Personal" and everything under that would have high prio but would have to match the root's rule. And would be categorized as personal, not as work.

BUT I'm not sure if a change of such magnitude would be accepted upstream, and also where to actually do it. As far as I saw it'd have to span the server and the frontend, but there are two servers... So would it have to implemented in both? Or is there any chance the migration to the rest server will ever finish?

:)

I'd be happy to help one way or the other.

Cheeers!