.TH "md_doc_dev_architecture" 3elektra "Sat Jun 5 2021" "Version 0.9.6" "Elektra" \" -*- nroff -*- .ad l .nh .SH NAME md_doc_dev_architecture \- Architecture In this document we start to explain the implementation of Elektra\&. There are several follow-up documents which explain all details of: .PP .IP "\(bu" 2 \fBerror handling\fP, .IP "\(bu" 2 \fBdata structures\fP, and .IP "\(bu" 2 finally the \fBcore algorithm\fP\&. .PP .PP We discuss problems and the solution space so that the reader can understand the rationale of how problems were solved\&. .PP To help readers to understand the algorithm that glues together the plugins, we first describe some details of the \fBdata structures\fP\&. Full knowledge of the \fBalgorithm\fP is not presumed to be able to develop most plugins (with the exception of \fCthe resolver\fP)\&. .PP Further important concepts are explained in: .PP .IP "\(bu" 2 \fBbootstrapping\fP .IP "\(bu" 2 \fBgranularity\fP .IP "\(bu" 2 \fBsync-flag\fP .PP .SH "API" .PP The aim of the Elektra Initiative is to design and implement a powerful API for configuration\&. When the project started, we assumed that this goal was easy to achieve, but dealing with the semantics turned out to be a difficult problem\&. For the implementation, an ambitious solution is required because of the necessary modularity to implement flexible backends as introduced in Elektra\&. But also the design of a good API has proved to be much more difficult than expected\&. .SS "Changes in the APIs" From Elektra 0\&.7 to Elektra 0\&.8, we changed the API of Elektra as little as possible\&. It should be mentioned that \fCKeySet\fP is now always sorted by name\&. The function \fCksSort()\fP is now depreciated and was removed\&. The handling of removed keys was modified\&. Additionally, the API for metadata has fundamentally changed, but the old interface still works\&. These changes will be described in \fBimplementation of metadata\fP\&. However, the implementation of Elektra changed radically as discussed in \fBalgorithm\fP\&. .SS "API Design" API Design presents a critical craft every programmer should be aware of\&. We will shortly present some of the main design issues that matter and show how Elektra has solved them\&. .PP A design goal is to detect errors early\&. As easy as it sounds, as difficult it is to actually achieve this goal\&. Elektra tries to avoid the problem by checking data being inserted into \fCKey\fP and \fCKeySet\fP\&. Elektra catches many errors like invalid key names soon\&. Elektra allows plugins to check the configuration before it is written into the key database so that problematic values are never stored\&. .PP 'Hard to use it wrong' tends to be a more important design objective than 'Easy to use it right'\&. Searching for a stupid bug costs more time than falling into some standard traps which are explained in documentation\&. In Elektra, the data structures are robust and some efforts were taken to make misuse unlikely\&. .PP Another fundamental principle is that the API must hide implementation details and should not be optimized towards speed\&. In Elektra, the actual process of making configuration permanent is completely hidden\&. .PP 'Off-by-one confusion' is a topic of its own\&. The best is to stick to the conventions the programming language gives\&. For returning sizes of strings, it must be clear whether a terminating `'\\0'` is included or not\&. All such decisions must be consistent\&. In Elektra the terminating null is always included in the size\&. .PP The interface must be as small as possible to tackle problems addressed by the library\&. Internal and external APIs must be separated\&. Internal APIs in libraries shall be declared as \fCstatic\fP to prevent its export\&. In Elektra, internal names start with \fCelektra\fP opposed to the external names starting with \fCkey\fP, \fCks\fP or \fCkdb\fP\&. .PP Elektra always passes user context pointers, but never passes or receives a full data structure by value\&. It is impossible to be ABI compatible otherwise\&. Elektra is restrictive in what it returns (strong postconditions), but as liberal as possible for what comes in (preconditions are avoided where possible)\&. In Elektra even null pointers are accepted for any argument\&. .PP 'Free everything you allocate' is a difficult topic in some cases\&. If Elektra cannot free space or other resources after every call, it provides a \fCclose()\fP function\&. Everything will be freed\&. The tool \fBValgrind\fP with \fBMemcheck\fP helps us locate problems\&. The whole test suite runs without any memory problems\&. The user is responsible for deleting all created \fCKey\fP and \fCKeySet\fP objects and closing the \fCKDB\fP handle\&. .PP As a final statement, we note that the Unix philosophy should always be considered: 'Do only one thing, but do it in the best way\&. Write it that way that programs work together well\&.' .SH "Modules" .PP Elektra’s core can be compiled with a C compiler conforming to the ISO/IEC 9899:1999 standard: .PP .IP "\(bu" 2 One line comments, .IP "\(bu" 2 inline functions, .IP "\(bu" 2 \fCsnprintf()\fP .IP "\(bu" 2 inttypes\&.h and .IP "\(bu" 2 variable declaration at any place .PP .PP are used in addition to what is already defined in the standard ISO/IEC 9899:1990, called \fBC99\fP in the following text\&. Functions not conforming to C99 are considered to be not portable enough for Elektra and are separated into plugins\&. But there is one notable exception: it must be the core's task to load plugins\&. Unfortunately, C99 does not know anything about modules\&. \fBPOSIX\fP (Portable Operating System Interface) provides \fCdlopen()\fP, but other operating systems have dissimilar APIs for that purpose\&. They sometimes behave differently, use other names for the libraries and have incompatible error reporting systems\&. Because of these requirements Elektra provides a small internal API to load such modules independently from the operating system\&. This API also hides the fact that modules must be loaded dynamically if they are not available statically\&. .PP Plugins are usually realized with modules\&. Modules and libraries are technically the same in most systems\&. (One exception is macOS\&.) After the module is loaded, the special function plugin factory is searched for\&. This function returns a new plugin\&. With the plugin factory the actual plugins are created\&. .SS "Static Loading" For the static loading of modules, the modules must be built-in\&. With \fCdlopen(const char* file)\fP POSIX provides a solution to look up such symbols by passing a null pointer for the parameter \fCfile\fP\&. Non-POSIX operating systems may not support this kind of static loading\&. Therefore, Elektra provides a C99 conforming solution for that problem: a data structure stores the pointers to the plugin factory of every plugin\&. The build system generates the source file of this data structure because it depends on built-in plugins\&. .PP Elektra distinguishes internally between modules and plugins\&. Several plugins can be created out of a single module\&. During the creation process of the plugin, dynamic information - like the configuration or the data handle - is added\&. .SS "API" The API of \fBlibloader\fP consists of the following functions: .PP Interface of Module System: .PP .PP .nf int elektraModulesInit (KeySet *modules, Key *error); elektraPluginFactory elektraModulesLoad (KeySet *modules, const char *name, Key *error); int elektraModulesClose (KeySet *modules, Key *error); .fi .PP .PP \fCelektraModulesInit()\fP initializes the module cache and calls necessary operating system facilities if needed\&. \fCelektraModulesLoad()\fP does the main work by either returning a pointer to the plugin factory from cache or loading it from the operating system\&. The plugin factory creates plugins that do not have references to the module anymore\&. \fCelektraModulesClose()\fP cleans up the cache and finalises all connections with the operating system\&. .PP Not every plugin is loaded by \fClibloader\fP\&. For example, the \fIversion plugin\fP, which exports version information, is implemented internally\&. .SH "Mount Point Configuration" .PP \fCkdb mount\fP creates a \fBmount point configuration\fP as shown in the example below\&. \fCfstab\fP is a unique name within the mount point configuration provided by the administrator\&. .PP Example for a mount point configuration: .PP .PP .nf system:/elektra/mountpoints system:/elektra/mountpoints/fstab system:/elektra/mountpoints/fstab/config system:/elektra/mountpoints/fstab/config/path=fstab system:/elektra/mountpoints/fstab/config/struct=list FStab system:/elektra/mountpoints/fstab/config/struct/FStab system:/elektra/mountpoints/fstab/config/struct/FStab/device system:/elektra/mountpoints/fstab/config/struct/FStab/dumpfreq system:/elektra/mountpoints/fstab/config/struct/FStab/mpoint system:/elektra/mountpoints/fstab/config/struct/FStab/options system:/elektra/mountpoints/fstab/config/struct/FStab/passno system:/elektra/mountpoints/fstab/config/struct/FStab/type system:/elektra/mountpoints/fstab/errorplugins system:/elektra/mountpoints/fstab/errorplugins/#5#resolver#resolver# system:/elektra/mountpoints/fstab/getplugins system:/elektra/mountpoints/fstab/getplugins/#0#resolver system:/elektra/mountpoints/fstab/getplugins/#5#fstab#fstab# system:/elektra/mountpoints/fstab/mountpoint /fstab system:/elektra/mountpoints/fstab/setplugins system:/elektra/mountpoints/fstab/setplugins/#0#resolver system:/elektra/mountpoints/fstab/setplugins/#1#struct#struct# system:/elektra/mountpoints/fstab/setplugins/#2#type#type# system:/elektra/mountpoints/fstab/setplugins/#3#path#path# system:/elektra/mountpoints/fstab/setplugins/#3#path#path#/config system:/elektra/mountpoints/fstab/setplugins/#3#path#path#/config/path/allow=proc tmpfs none system:/elektra/mountpoints/fstab/setplugins/#5#fstab system:/elektra/mountpoints/fstab/setplugins/#7#resolver .fi .PP .PP Let us look at the subkeys below the key \fCsystem:/elektra/mountpoints/fstab\fP: .PP .IP "\(bu" 2 \fBconfig\fP: Everything below \fCconfig\fP is the system's configuration of the backend\&. Every plugin within the backend will find this configuration directly below \fCsystem\fP in its \fBplugin configuration\fP\&. For example, .PP .PP .PP .nf system:/elektra/mountpoints/fstab/config/struct/FStab/mpoint .fi .PP .PP will be translated to .PP .PP .nf system:/struct/FStab/mpoint .fi .PP .PP and inserted into the plugin configuration for all plugins in the \fCfstab\fP backend\&. .PP It is the place where configuration can be provided for every plugin of a backend\&. The contract checker deduces this configuration to satisfy the contract for a plugin\&. Fstab, for example, claims in a contract that it needs 'struct'\&. But the struct plugin needs a configuration to work properly\&. Fstab will provide this configuration\&. The \fBcontract checker\fP writes out the configuration looking like the one in this example\&. .PP .IP "\(bu" 2 \fBconfig/path\fP: is a common setting needed by the resolver plugin\&. It is the relative path to a filename that is used by this backend\&. On Unix systems, the resolver would determine the name \fC/etc/fstab\fP for system configuration\&. .IP "\(bu" 2 \fBmount point\fP: is a key that represents the mount point\&. Its value is the location where the backend is mounted\&. If a mount point has an entry for both the user and the system hierarchy, it is called \fBcascading mount point\fP\&. A cascading mount point differs from two separate mount points because internally only one backend is created\&. In the example, the mount point \fC/fstab\fP means that the backend handles both \fCuser:/fstab\fP and \fCsystem:/fstab\fP\&. If the mount point is \fC/\fP, the backend will be mounted to all namespaces except \fCspec\fP, including both \fCuser\fP and \fCsystem\fP\&. .IP "\(bu" 2 \fBerrorplugins\fP: presents a list of all plugins to be executed in the error case of \fC\fBkdbSet()\fP\fP which will be explained in \fBerror situation\fP\&. .IP "\(bu" 2 \fBgetplugins\fP: is a list of all plugins used when reading the configuration from the key database\&. They are executed in \fC\fBkdbGet()\fP\fP\&. .IP "\(bu" 2 \fBsetplugins\fP: contains a list of all plugins used when storing configuration\&. They are executed in \fC\fBkdbSet()\fP\fP\&. .PP .PP Each of the plugins inside the three lists may have the subkey \fCconfig\fP\&. The configuration below this subkey provides plugin specific configuration\&. This configuration appears in the user's configuration of the plugin\&. Configuration is renamed properly\&. For example, the key .PP .PP .nf system:/elektra/mountpoints/fstab/setplugins/#3#path#path#/config/path/allow .fi .PP .PP is transformed to .PP .PP .nf user:/path/allow .fi .PP .PP and appears in the plugin configuration of the path plugin inside the fstab backend\&. .SS "Referencing" The same plugin often must occur in more than one place within a backend\&. The most common use case is a plugin that has to be executed for both \fC\fBkdbGet()\fP\fP and \fC\fBkdbSet()\fP\fP\&. It must be the same plugin if it preserves state between the executions\&. .PP Other plugins additionally have to handle error or success situations\&. One example of exceptional intensive use is the resolver plugin\&. It is executed twice in \fC\fBkdbSet()\fP\fP\&. In \fC\fBkdbGet()\fP\fP it is also used as shown above\&. .PP .IP "\(bu" 2 \fC#n\fP: introduces a new plugin from the module \fCname\fP which cannot be referenced later\&. The cypher \fCn\fP appoints the actual placement of the plugin\&. .IP "\(bu" 2 \fC#n##