Tue, 31 Oct 2017 00:00:00 GMT

0.8.20 Release

guid: 547d48e2-c044-4a8e-9d32-ca6b6fb914d9

author: Markus Raab

pubDate: Thu, 31 Oct 2017 23:08:07 +0200

shortDesc: New Website, puppet-libelektra, New Plugins

What is Elektra?

Elektra serves as a universal and secure framework to access configuration settings in a global, hierarchical key database. For more information, visit https://libelektra.org.

Highlights

This is by far the largest release in Elektra's history. In 2813 commits, 19 authors changed 1714 files with 92462 insertions(+) and 21532 deletions(-). The highlights are:

libelektra.org: new website and puppet-libelektra

plugin+bindings for Haskell and Ruby

improved shell completion

new plugins: yamlcpp, camel, mini, date, file, range, multifile, xerces, ipaddr

libelektra.org

Unfortunately this release was delayed. The reason for the delay is that our community server (build server, web site,...) was compromised and we needed to reinstall everything from scratch.

We took advantage of the situation, and reinstalled everything properly managed by puppet-libelektra. With puppet-libelektra, you can directly set keys, specifications (validations), and even mount new configuration files from within Puppet.

Our community server is now completely managed by libelektra.

Thanks to Bernhard Denner, for rescuing us from the difficult situation, especially for the sprint shortly before the release.

As already already announced in December 2016 we completely reimplemented our website. Now all our websites are available via https. This release is the first one that includes the source code of the website and its snippet sharing functionality.

The backend for this snippet sharing website uses Elektra itself: both for its configuration and for the configuration snippets.

Thanks again to Marvin Mall for the awesome website.

Haskell and Ruby

The Ruby binding, created by Bernhard Denner, was greatly improved and now includes libtools bindings. It is the first binding that goes beyond Elektra's main API. Bernhard Denner also added many examples that demonstrate how you can take advantage of the Ruby bindings.

Armin Wurzinger created a new binding for the functional language Haskell. He also added support for Haskell plugins. Due to generic CMake and C Code, plugins can be written exclusively in Haskell, without any glue code. Several Haskell examples already exist. The Haskell support is currently experimental.

Shell Completion

René Schwaiger added completion support for Fish in this release. We also extended our support for other shells: The new tool kdb complete suggests how to complete an Elektra path. It considers mountpoints and also takes bookmarks into account. Thanks to Armin Wurzinger for creating this useful utility. Our Zsh and fish completions already take advantage of kdb complete. Thanks to Sebastian Bachmann for taking the time to update the zsh completions.

New Plugins

See plugin overview to get an overview of the ever-growing number of plugins.

The yamlcpp plugin and camel plugin add first support for YAML.

The mini plugin is yet another minimal INI plugin.

Thanks to René Schwaiger.

The date plugin supports validation of dates according to three standards:

RFC2822

ISO8601

POSIX

The multifile plugin allows us to integrate many configuration files via globbing with a single mount command. It supports .d configuration directories as often used today.

The file plugin interprets the content of a file as configuration value.

The ipaddr plugin adds support for IP address validation on systems that do not support getaddrinfo.

Thanks to Thomas Waser for creating these useful plugins.

The xerces plugin supplants the xmltool plugin and allows us to use XML files not following a specific schemata. Attributes are mapped to Elektra's metadata, multiple keys with the same names are mapped to arrays.

Thanks to Armin Wurzinger.

Documentation

The documentation was greatly improved within this release.

Added "Hello, Elektra" and logging tutorial, thanks to René Schwaiger

extended FAQ

Christoph Weber (@krit0n) improved some tutorials

options are passed to PDFLaTeX compiler, thanks to René Schwaiger

small fixes, thanks to Dominik Hofer

fix many spelling mistakes, use American english, fix formatting, fix+add links, unify title style, fix code blocks, add titles and fix the PDF manual a big thanks to René Schwaiger

Features

We added even more functionality, which could not make it to the highlights:

DBUS support for qt-gui (listening to configuration changes): qt-gui gets a viewer-mode where configuration settings are immediately updated via DBus notifications, thanks to Raffael Pancheri With the new qt-gui and newer qt releases (~5.7) the qtquick experience is much smoother, for example, the tree view does not collapse on syncs anymore.

Armin Wurzinger greatly improved the JNA binding. The build system now uses Maven to build them. Armin also added Doxygen documentation and a script to test the JNA binding using Randoop.

The improved curlget plugin, is now able to upload configuration files, thanks to Thomas Waser and Peter Nirschl (CMake fixes).

New command kdb rmmeta, thanks to Bernhard Denner

crypto plugin and fcrypt plugin

The configuration option gpg/key was renamed to encrypt/key

The plugins now make sure that you configured them properly by validating key IDs

thanks to Peter Nirschl

fcrypt plugin:

The plugin now list available GPG keys when config is missing

You can now specify signatures via the configuration option sign/key

New text mode, enabled by default (disable by setting fcrypt/textmode to 0)

New option fcrypt/tmpdir allows you to specify the output directory of gpg

If you want to learn how to use the plugin please check out our new ASCII cast

thanks to Peter Nirschl

Thomas Waser added useful scripts:

mount-list-all-files to list all mounted files.

mountpoint-info to provide more info about mountpoints.

stash to stash away Elektra's configuration, to be restored using restore.

backup to backup Elektra's configuration.

restore to restore a backup or stash.

check-env-dep allows users to check if environment has influence on configuration settings.

change-resolver-symlink allows users to change the default resolver.

change-storage-symlink allows users to change the default storage.

limit min/max depth for kdb ls (-mM), thanks to Armin Wurzinger.

conditionals: allow multiple assigns and conditions

base64 also works as filter for binary data (not just encrypted data), thanks to René Schwaiger

csvstorage plugin: compatibility with RFC 4180, thanks to Thomas Waser

gitresolver plugin: improvements and update of libgit version, thanks to Thomas Waser

curlget plugin: also allow uploading of configuration files, thanks to Thomas Waser

Compatibility

As always, the ABI and API of kdb.h is fully compatible, i.e. programs compiled against an older 0.8 version of Elektra will continue to work (ABI) and you will be able to recompile programs without errors (API).

We added explicit to some C++ constructors in libtools and internally moved some typedefs. ModulesPluginDatabase now has protected members (instead of private). This might break code in special cases, but should not affect binary compatibility. As always we tested for binary compatibility. This time we had to revert some changes to keep libelektra-tools ABI compatible.

Furthermore:

scripts now work on macOS (readlink and sed), thanks to Armin Wurzinger and René Schwaiger

generalized error #60, "invalid line encountered"

added new errors #164 - #187

added private headerfiles kdbglobal.h, kdbinvoke.h

Notes for Package Maintainer

These notes are of interest for people maintaining packages of Elektra:

LICENSE.md now contains only the BSD 3-Clause License (without any additional non-license text)

AUTHORS renamed to AUTHORS.md

NEWS.md is now a generated file containing all news concatenated

CMake 2.8.8 is no longer supported, CMake 3.0 is now needed

fix macOS RPATH, remove old policies, thanks to René Schwaiger

new option BUILD_DOCSET to build DocSet, thanks to René Schwaiger

new option ENABLE_OPTIMIZATIONS for OPMPHM preparation work, thanks to Kurt Micheli For this release, please keep ENABLE_OPTIMIZATIONS turned off. Currently the flag increases memory usage, without being faster.

add TARGET_TOOL_DATA_FOLDER for installation of tool data

The following files are new:

Libs: libelektra-utility.so ,libelektra4j-0.8.20.pom.xml, libelektra-invoke (needed by plugins: curlget, gitresolver, dini, blockresolver, multifile)

Plugins: libelektra-camel.so, libelektra-date.so, libelektra-file.so, libelektra-ipaddr.so, libelektra-mini.so, libelektra-multifile.so, libelektra-range.so, libelektra-xerces.so, libelektra-yamlcpp.so

Tools: backup, mount-list-all-files, mountpoint-info, restore, stash, update-snippet-repository

Tests: change-resolver-symlink, change-storage-symlink, check-env-dep, check_bashisms, check_doc, check_meta, testmod_camel, testmod_crypto_openssl, testmod_date, testmod_file, testmod_ipaddr, testmod_jni, testmod_mini, testmod_range, testmod_simpleini, testmod_xerces, testmod_yamlcpp, testtool_plugindatabase, test_utility

The following files were removed: testmod_curlget, testmod_dpkg, testmod_profile, testmod_shell, testmod_spec, test_opmphm_vheap, test_opmphm_vstack

The following files were renamed: libelektra-1.jar → libelektra4j-0.8.19.jar

In the Debian branch of the git repo, we now build upon the work of Pino Toscano. The branch allows building Debian packages of the release for Debian Stretch and Jessie.

Thanks to Pino Toscano for the high-quality packages.

Notes for Elektra's Developers

These notes are of interest for people developing Elektra:

Added macros, thanks to René Schwaiger:

ELEKTRA_NOT_NULL is an assertion against null pointers

ELEKTRA_MALLOC_ERROR sets an error when allocation failed

ELEKTRA_STRINGIFY to quote a macro value

ELEKTRA_PLUGIN_STATUS_ERROR, ELEKTRA_PLUGIN_STATUS_SUCCESS, ELEKTRA_PLUGIN_STATUS_NO_UPDATE for return values of plugins.

ELEKTRA_STRINGIFY used throughout, thanks to René Schwaiger

use (void) instead of () (added -Wstrict-prototypes)

new positions for global plugins, thanks to Mihael Pranjic

Kurt Micheli added generateKeySet to randomly generate large key sets

add vagrant and docker support, thanks to Christoph Weber (@krit0n)

improve support for CLion, NetBeans and oclint

portability improvements for logger, thanks to René Schwaiger

metadata consistency check within source repo, thanks to Thomas Waser

ELEKTRA_PLUGIN_EXPORT accepts macro as argument

fallthroughs in switch statements are now marked with FALLTHROUGH

introduce print_result to unify test output, thanks to René Schwaiger

export validateKey as preparation for type plugin

Other

Various other changes happened in the code repository:

kdb: errors are more colorful, add infos to report issues, catch signals for kdb tools to print errors on crashes, use $EDITOR if sensible-editor and editor is not found. René Schwaiger fixed preposition and format of the messages.

added Spanish translation for qt-gui thanks to AdanGQ (@pixelead0)

augeas plugin: error messages improved, export genconf (for WebUI to list all lenses)

improvements for CentOS and Debian Packages, thanks to Sebastian Bachmann

Travis improvements, fixes, and many build variants added, thanks to Mihael Pranjic and René Schwaiger

ronn now always uses UTF-8 as encoded and is no longer required as essential dependency to get man pages, thanks to René Schwaiger

GitHub now recognizes that we have a BSD licence, thanks to René Schwaiger

uninstallation Script also uninstalls directories and Python files, thanks to René Schwaiger

Kurt Micheli created a benchmark tool to generate large KeySets

added/reformatted use cases, thanks to Daniel Bugl

Thomas Wahringer prepared for a new theme on the website

Arfon Smith updated metadata for Elektra's journal entry

Quality

In this release we had a focus on quality improvements:

fixed all remaining ASAN problems, thanks to René Schwaiger and Armin Wurzinger (some tests are excluded when compiled with ASAN)

fix many compilation warnings, thanks to René Schwaiger and Armin Wurzinger

fixed many potential out-of-bound errors, thanks to René Schwaiger

fixed warnings of Clang's static analyzers, thanks to René Schwaiger

fixed cppcheck warnings, thanks to Armin Wurzinger

fixed strict prototypes, thanks to Armin Wurzinger

fixed and use scan-build (clang)

fixed potential memory leaks on errors

added assertions

generate Java API tests with randoop which revealed bugs in jna bindings that were fixed, thanks to Armin Wurzinger

Numerous fixes in the shell recorder, which does regression tests on Elektra's tutorials and READMEs, thanks to René Schwaiger and Thomas Waser

Fixes

Many problems were resolved with the following fixes:

kdb file: never print errors, thanks to René Schwaiger

plugin mathcheck: fixed regex #1094, thanks to Thomas Waser

dbus: properly do unref and document how to integrate DBus, thanks to Kai-Uwe Behrmann

dbus accepts announce=once which is used for kdb mount-openicc It protects against message floods in large configuration files, thanks to Kai-Uwe Behrmann for reporting

plugin desktop: fix crash if DESKTOP_SESSION is missing

shell-recorder: many fixes and improvements, thanks to Thomas Waser and René Schwaiger

fix getopt positional parameters, thanks to Armin Wurzinger

resolver: avoid silent errors of fchown/fchmod

plugin fcrypt: fixes in file name handling to make leaks less likely (still needs tmpfs to be secure!), thanks to Peter Nirschl

plugin jni: fix segfaults on errors, plugin is nevertheless tagged as experimental due to other problems

plugin type: reject integers if garbage follows

kdb: fix memleak when listing plugins

many spelling fixes and fix typo in source of qt-gui: thanks to klemens (ka7)

dpkg, fix file leakage, thanks to Armin Wurzinger

plugin line: only skip parentKey if present

plugin resolver: avoid failure after commit for files that cannot be removed

plugin simpleini: handle more errors, make format parameter more robust thanks to Bernhard Denner

plugin crypto: fix compilation errors for openssl versions on Debian 9, thanks to Peter Nirschl

kdb mv: fail without keys also in recurse mode

fix bashism, thanks to Armin Wurzinger

qtgui: fix crash on unhandled exception on binary values, thanks to Raffael Pancheri

Outlook

We are currently working on following topics:

Migration of LCDproc, OpenIcc, machinekit, ... to Elektra.

A reimplementation of Elektra's core API in Rust (next to implementation in C).

A user interface which generates restricted input fields based on the specification.

YAML as configuration file format (next to INI, XML, JSON, TCL, ...).

An mmap persistent cache.

Improvements for the specification language.

New APIs to be directly used by applications.

An order-preserving minimal hash for O(1) lookup and iteration.

Mainloop migration for notifications (currently only DBus, to be extended to Redis, ZeroMq).

Improvements on the Website and snippet sharing to also handle misconfiguration.

Get It!

You can download the release from here or GitHub

The hashsums are:

name: elektra-0.8.20.tar.gz

size: 4740032

md5sum: 0e906f1a1677a8bfb31d144e1eaeb3cf

sha1: 5e33c49ae6e3b890c9267288fb9f321289910eb5

sha256: e9cbc796e175685ccb6221f1dd5ea5c43832f545c40557c32b764ff5d567b312

The release tarball is also available signed by me using gpg from here or GitHub

Already built API documentation can be found online or GitHub.

Stay tuned!

Subscribe to the RSS feed to always get the release notifications.

For any questions and comments, please contact the issue tracker on GitHub or me by email using elektra@markus-raab.org.

Permalink to this NEWS entry

For more information, see https://libelektra.org

Best regards, Markus