$darkmode
Elektra 0.11.0
|
Currently, a range of Elektra plugins are implementing some sort of change tracking for configuration data. This includes, but is not limited to, the internalnotification and dbus plugins. In the near future, Elektra shall also be extended with session recording.
KDB itself also has some rudimentary change tracking (via the keyNeedsSync
flag) to determine whether kdbSet
needs to actually do something.
These competing change tracking implementations create multiple problems:
kdbSet
might write a config file but notification is not sent outFor KeySet
we need to track which of the keys:
For Key
we need to track:
system:/
user:/
dir:/
meta:/
spec:/
Change tracking must:
kdbGet
and kdbSet
as per this decisionWe only want to track changes that are done by the user, not changes that are done by plugins. I.e. the scope of change tracking is on what happens outside of kdbGet
and kdbSet
.
The library libelektra-core
must be kept minimal.
false
<->0
and true
<->1
system:/background
is stored with value false
system:/background
with value 0
(after conversion by plugin)false
false
!= 0
false
to 0
system:/background
has been changedWe already store which keys have been returned by kdbGet
for each backend within KDB. Currently, those are shallow copies. To be useful for changetracking, we need to perform deep copies instead. As keys and keysets are already utilizing copy-on-write, this would not add too much memory overhead.
A problem with this approach is that the internally stored keys are recreated as new instances every time kdbGet
is called.
We already decided that we want to have an internal deep-duped keyset of all the keys we returned. See internal cache decision.
The difference to backendData->keys
is that this cache is not recreated each time kdbGet
is called.
Have a global keyset with deep-duped keys that is purely used for changetracking and nothing else.
When something changes for the first time, store the original value as a metakey to every key. Not yet clear how we handle changes to metadata then. Also not really possible for keysets.
The idea here is that we extend the KeySet
and Key
structs with additional fields.
The tracking itself would be done within the ks*
and key*
methods, after checking if it is enabled. It would also transparently work for metadata, as metadata itself is implemented as a keyset with keys.
Downsides of this approach:
libelektra-core
which may violate the constraint above. It may, however, be debatable whatever 'minimal' means in this context.KeySet
due to the low amount of keysets in typical applications, it may be noticable for Key
. On a 64-bit system we'd add 16 bytes to it. 8 bytes for the pointer to the original value, 8 bytes for the size of the original value. To put this in perspective, the current size of the Key
struct is 64 bytes, so we'd add 25% overhead to an empty key. However, this percentage will be much lower in a real-world application, as the usefulness of an empty key is very low.libelektra-kdb
would need to mark the keys as original (after transformations etc.)Implement all the logic for changetracking directly within libelektra-kdb
.
Implement changetracking as a hooks plugin that will be called within kdbGet
and kdbSet
accordingly.
The following hooks will be needed:
tracking/get
: will be called at the end of kdbGet
, directly before the result is returned.tracking/set
: will be called at the beginning of kdbSet
.tracking/changeset
: compute the changeset for the requested parent key and return it.As every hook plugin can define its own contract, in theory all storage forms mentioned in the previous chapter should be possible to implement. We could just point the plugin to backendsData->keys
or the internal cache if we go down that route.
The API should be usable by plugins and by applications utilizing Elektra. It does not matter whether the changetracking is implemented as part of libelektra-kdb
or as a separate plugin. The API may look something like this:
This solution only makes sense if changetrackig is implemented as part of a seperate plugin. It will be a bit challenging to use for applications, as it would require that applications have access to the plugin contracts.
The changetracking plugin needs to export at least functions for the following things in its contract:
Plugin and application developers can declare that changetracking is required via a contract. Store deep-duped copy-on-write returned keys in a separate keyset, which we might also use as internal cache. The whole changetracking logic lives within libelektra-kdb
. We provide an API for developers with libelektra-kdb
.
This decision meets all the constraints.
We are not altering the behavior of already existing functions, so everything is transparent for applications using the public Elektra API. As changetracking is an opt-in feature, it will create zero runtime overhead and negligible memory overhead in usecases where it is not needed. There is only a single implementation and storage, so no duplication for each plugin that wants change tracking. As we are storing our own tracking data, all sequences of kdbGet
and kdbSet
should work.
struct _KDB
needs to be extended to contain the keyset for changetrackingkeyNeedsSync
flag. It also includes switching all notification plugins to use the new API.