API
Clients should preferably use the ALExANDRA middleware! All API methods are available here, plus some nice-to-haves such as client-centric consistency when using version vectors!
FReD uses a gRPC API. The main benefit is that language libraries can easily be generated by using the protoc compiler. Compare that to JSON + HTTP + REST, where you need to roll your language-specific code, or we would need to provide different libraries.
For test purposes, the compiled Go and Python language files are already available in ./proto/middleware. To compile your own language bindings, read more about protobuf here.
The API provides endpoints to interact with FReD on five different aspects:
- Data
- Keygroup Management
- Replica Management
- User Management
- Trigger Node Management
Data
Data in FReD keygroups can be read, updated/appended and deleted.
Keygroups can be configured to be either mutable or immutable. Data in immutable keygroups can only be appended, existing keys cannot be updated or deleted. You need to specify a key in the form of a 64-bit unsigned integer, e.g., a timestamp.
Data in mutable keygroups cannot be appended as there is no concept of key increments, but you can insert and update data at specified keys with the update operation. If you update a key that does not exist yet, it will be created. You can use the delete operation to delete a key from the store.
Data keys can be any string that matches the Regex pattern ^[a-zA-Z0-9]+$, i.e., they must be alphanumeric. Data values can be any string, although the protobuf encoding is limited to UTF-8 (as far as we know…).
Keygroups
Keygroups can be created and deleted. Keygroups cannot be updated in their configuration as they have no mutable attributes: the mutability of their data can only be specified when creating a new keygroup.
Keygroup names must be alphanumeric. Additionally, you can specify whether a new keygroup should be mutable or not, i.e., if the data in the keygroup can be modified or deleted.
Furthermore, keygroups support data expiry. Each appended, updated, or created key-value pair in the keygroup will be deleted after expiration of the data. Note that we cannot guarantee that the data will be deleted exactly after expiring as this depends on the garbage collection of the underlying storage backends.
Expiration durations are set per keygroup and per replica FReD node. The same keygroup can have different expiration duration on different FReD nodes. For example, you may want your data to be removed on lightweight edge nodes but to be persisted in the Cloud. Set an expiry of 0 to disable expiration.
If you set an expiry during keygroup creation, this expiry will only apply to the FReD node you are asking to create the keygroup. This node will also automatically become the first replica node for that keygroup.
Replica Management
Every FReD node is a possible replica node for every keygroup. A key concept of FReD is that this replication is configurable by application.
The API allows the GetAllReplica and GetReplica endpoints so that clients can access information about all available FReD nodes in the cluster. Each FReD node can be uniquely identified with its ID.
The GetKeygroupReplica command returns a list of replica nodes for a given keygroup including the expiry settings for each replica node. Replica nodes for keygroups can also be added by providing a node ID, keygroup name, and expiry. Additionally, replicas can also be removed from keygroups.
User Management
FReD supports a simple authentication and authorization procedure. More information about this system can be found below.
The API allows adding and removing roles for users for keygroups with the AddUser and RemoveUser endpoints. You can specify a user ID, keygroup name, and role for that user in that keygroup.
Trigger Node Management
The API also allows adding, reading, and removing triggers for keygroups.
Triggers are defined per keygroup per FReD node. If you add a trigger for a keygroup by sending the corresponding API request to a FReD node, only this node will send data updates to this trigger node. As data is replicated between all FReD members of a keygroup, all updates will eventually also reach the trigger node. This makes it easier to reason about communication between FReD nodes and trigger nodes and foregoes duplicate transmissions.
However, if you remove a replica node for a keygroup, all keygroup triggers configured on that FReD node will also be removed.