Make ActorHandles pickleable, also make proper ActorHandle and ActorC…

[robertnishihara]

…lass classes.

This is intended to cleanup some of the Python actor code.

Note that should enable named actors via #1424 (comment).

[robertnishihara]

Is this key in Redis used anywhere?

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5231/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5242/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5243/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5246/
Test FAILed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5261/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5262/
Test PASSed.

[robertnishihara]

I'm using this method to compute actor handle IDs for "forked" actor handles. However, if a user pickles and unpickles the actor handle, then I'm generating the actor handle ID randomly.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5264/
Test PASSed.

[robertnishihara]

Maybe the right way to do actor handle ID generation (for pickled handles) is the following. Suppose we have

actor_handle = Actor.remote()

# Defining this remote function, which closes over actor_handle, will
# cause actor_handle to be pickled. The remote function will be broadcast
# to all the workers and unpickled there.
@ray.remote
def f():
    actor_handle.method.remote()

f.remote()

Instead of generating a random actor handle ID on each worker when the remote function is broadcast, maybe we should instead leave the actor handle in an "uninitialized" state, and then whenever a f.remote() task executes, we generate a new actor handle ID deterministically from the actor ID and the current task ID. In other words, we want it to be roughly equivalent to calling f.remote(actor_handle). Doing this will cause a proliferation of actor handles, so we'll need to make sure that is handled efficiently.

cc @stephanie-wang

[stephanie-wang]

@robertnishihara, I like this suggestion. Just to clarify, what is the use case for closing over the actor handle vs. passing the actor handle into the f.remote() call?

[robertnishihara]

@stephanie-wang one use case is that you want to use the actor handle on a different driver (e.g., "named actors".

Another use case, I think, in RLlib/Tune is that certain user-defined functions have to have pre-specified signatures and the user isn't really able to just pass in an actor handle because that would require changing the signature and piping it through a bunch of RLlib abstractions, but they can define a function that just closes over the actor handle.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5277/
Test PASSed.