LCFG Annual Review 2013

On Thursday 5th December 2013 instead of our normal monthly Deployers Meeting we will be holding an Annual Review session.

All users of LCFG are encouraged to attend this meeting to hear about what has been happening over the last year and what developments they can look forwards to in the next year. This is also an excellent opportunity to raise issues that are important to you, put forward ideas for future developments you would like to see and chat about all things LCFG!

If you have any topics you are particularly keen to have discussed then please edit this page and add them to the General Discussion section below with a brief summary.

This will start at 2pm and we aim to be finished by 5pm. It will be held in room 2.33 of the Informatics Forum (note that this is NOT the usual room).

The rough outline agenda is:

After the meeting there will be an informal gathering in a local pub followed by some food, everyone is welcome to come along.

Upstream

LCFG Client Update

The major work for this year was a refactoring of the LCFG client code base. This follows the succesful work done on the server last year. Full details of the project plan are available, Stephen also blogged about the project.

• Cleaned up code, improved maintainability
• New test suite
• Eradicated hard-wired paths (can use rdxprof as a normal user)
• Fixed a couple of long-standing bugs:
  • Handles DNS changes for servers
  • Does not block if network port is unavailable
• New generic profile support

Informatics are likely to be switching to the V3 client early in 2014. We may be able to maintain the capability to run with the V2 client for the short-term but it's unlikely to see further support.

The next phase of work has now begun. We are working on dropping the dependency on the ancient, and very unloved, W3C XML modules and moving to the standard, well-supported XML::LibXML modules. This should be completed within the next couple of months.

Generic Profiles

Once we have updated to the V3 LCFG client we will be able to enable the new support for "generic profiles". This new feature is designed to support two particular scenarios:

roaming machines

A roaming machine may regularly change hostname and/or domain name. Previously the LCFG client expected the LCFG profile name to follow the FQDN. We can now support consistently following the same LCFG profile no matter what the hostname is currently. Note that this prevents the receiving of "new profile notifications" but the client still regularly polls anyway.

many machines with single profile

Often sites have large sets of identical hosts which do not require any host specific configuration (that comes separately from dhcp, for example). Previously there would have to be a separate profile for each machine which might just be needless duplication, there is now support for the use of a single LCFG profile. Note that this completely breaks the use of spanning maps for managing your network infrastructure but that might not be a problem.

Auto-builder

To help component authors get their software built and submitted into the main lcfg package repository we have added support for auto-building whenever a new version is tagged in the subversion repository (i.e. using one of the major, minor or micro commands for the lcfg-reltool app, . This is done using a subversion hook which checks out the version and makes an SRPM which is submitted to the Informatics build-farm. This is primarily aimed at those who do not have DICE accounts and do not have direct access to our PkgForge system. It can be easily enabled by setting the autobuild parameter in the build section of the lcfg.yml file.

Component Changes

Changes to approximately 60 components, 9 of which are new this year:

• lcfg-accessconf
• lcfg-k5login
• lcfg-limits
• lcfg-lsyncd
• lcfg-ocsreports
• lcfg-opennebula
• lcfg-rsnapshot
• lcfg-sensu
• lcfg-update

Smaller stuff:

• updaterpms now reports deletion progress
• ngeneric can configure portreserve and run portrelease
• om - new perl API LCFG::Om::Command

General Discussion

• Components logging to syslog instead of direct to files by adding support for that to ngeneric.
• ngeneric support for other stuff - before methods are called it could set env variables and/or obtain kerberos tickets from keytabs
• Importing resources from databases.
• "Lightweight" use of LCFG, no boot component, only managing a few things on the OS.
• Header refactoring
• Switching to git
• SL7
• File component deleting files no longer in the profile.
• XFS, ext4?
• systemd
• Software Collections

Notes from Discussion

SL7

There was a fair amount of discussion regarding systemd. We will need a new component, if only to configure systemd for the daemons that are not managed with components. It was suggested that the "boot" information for components could be moved into the ngeneric namespace, not clear if this would be of much benefit since we still need to able to configure systemd anyway. A new systemd component will probably not take over the whole init process in the way as the current boot component. It was noted that all components which directly call scripts in /etc/init.d will need to be updated to use the "service" command. This could be done now as that works the same way with upstart.

The question of using XFS as the default partition type was raised. It seems likely that this will be the RHEL recommended FS. There was general agreement that we should follow RHEL if possible. Informatics will have to check if this is compatible with AFS but that wouldn't stop the default in the LCFG layer being XFS.

There was discussion of the default filesystem layout. Particularly in the MDP world the third partition is used for /var and the current 5GB default size is not large enough. It was noted that EL7 could introduce a new FHS layout with new partitions being required. We will discuss this further once we see the first beta release.

Component Frameworks

Various improvements to the LCFG component framework (ngeneric) were proposed. We identified a general class of problems we would like to solve which would include tasks such as setting environment variables and initialising kerberos credentials before component methods are called. This would be best done with a pluggable architecture so that different groups can develop what they require without having to rely on Informatics staff to develop (or at least integrate) the features.

It was mentioned that it is currently quite difficult to develop new functionality because we have always tried to keep the shell and Perl frameworks equivalent. The shell framework is already causing us quite a few problems with issues like file-descriptors not being closed before daemons are started. There was a general agreement that we would like to deprecate the shell framework and declare a ban on creating new shell-based components. It was also agreed that we should aim to remove the shell framework entirely for SL8 (which is probably 3 years away). This would free us up to redevelop the Perl framework and add new features which handle most of the common requirements.

There was also agreement that we should prefer the Perl Template Toolkit template style over the older LCFG-style templates. Currently the default with sxprof and LCFG::Template is to use LCFG-style templates. The proposal is that we switch this around so that the Perl TT is the default. The plan is to change this behaviour with the SL7 release. Older style templates would still be supported for SL7 with an aim to drop support with SL8.

There was some discussion over the idea of having a new python framework so that components can be written in that language. This would need some thought on the best design for the API rather than just blindly copying the Perl framework. We also need to be sure this doesn't cause us the same sort of problems as supporting Perl and shell. This isn't going to happen soon but possibly an aim for the SL8 release would be achievable.

Lightweight LCFG

The idea of taking a "lightweight" approach to LCFG was discussed. The idea being that a system would just have the client and a very small number of components installed rather than the whole machine being fully managed. Something similar was done a few years ago so we know it is possible. It is likely that systemd should make this easier than currently possible. Probably just needs a bit of effort to structure the headers appropriately.

This led onto a discussion of whether it would be possible to support LCFG on Debian/Ubuntu. There is no technical reason for this to not be possible but it requires someone to have the necessary spare time. Particularly we would need buildtools support. Also, lots of components would need configuration file locations to be moved into resources (that's something we've been working on for years already).

Security

There was discussion of how to make the delivery of XML profiles to the clients more secure. It was noted that the current client "profile fetcher" still needs more work. This shouldn't be too much effort now we have the refactored V3 client. We really want a fetcher class which supports the use of pluggable authentication handlers. That way different groups can use their own preferred authentication mechanisms (e.g. Kerberos or certificate-based) rather than the default user/pass system.

It would be nice to be able to secure access to the profile on the client so that only authorised users could run queries. This is harder and would probably need consolehelper-style PAM support to be added to qxprof and other tools. That's not likely to happen any time soon... One suggestion was that we could add support for securely retrieving resource values from remote services such as wallet, ldap, postgres. This would mean passwords and other secrets would not have to be stored directly in the profile.

File Component

A number of groups would like to see a change in behaviour for the LCFG file component so that it deleted files when they are no longer managed. Kenny already has a patch to do this so there is not much work involved other than applying the patch, building the packages and testing them.

The agreement was that this would be tested in the MDP world first and once we're happy that it is working as desired it would be pushed into the LCFG-level. As this is a major change in behaviour we would prefer to have it become the default only for SL7 onwards with SL6 staying with the old file component.

This is a general problem which many components need to solve. Currently many components are copying and pasting code rather than sharing functionality. We should look at adding support for a "file registry" system to the LCFG component framework. This would allow both the removal of files which are no longer managed and an optional full-cleanup stage on complete removal of the component package.

Proposals Summary

To summarise, the main proposals are listed below. Timescales are based around the delivery of the next major platforms. SL7 will arrive in early 2014. We expect SL8 approximately 3 years ahead (i.e. some time in 2017).

LCFG Core

• Shell components are now deprecated, we will not accept any new ones in the core.
• Support for Shell components will be dropped in SL8.
• New support for writing components in Python will be added.
• The LCFG-style templates are deprecated, the default style in SL7 will be Perl Template Toolkit
• Support for LCFG-style templates will be dropped in SL8.
• Support will be added to the file component in SL7 for deleting files when resources are removed.
• Look at adding support for a generic "file registry" system which can be used by all components which need to track the files that have been created.
• Work on improving the end-to-end security of delivering LCFG resource data.

SL7

• Create new systemd component
• Consider XFS for default filesystem type
• Consider partition layout
• Update components to use "service" command

Misc

• Consider support for lightweight LCFG

-- Main.squinney - 2013-10-11

• MDP_Review_and_Preview_2014.pdf: MDP Review 2013 Preview 2014.