Articles / A Solution to the Problem o…

A Solution to the Problem of Configuration in Linux

What is configuration in Linux? What do experienced system administrators do when they need to, for example, modify the access rights to a Web site or change the network settings for their server? Invariably, they're going to login into the machine and edit a text file using a Unix editor such as vi or Emacs. That by itself isn't that bad, but depending on which application you want to configure and which Linux distribution you happen to be using, the location of the file you need to edit (and maybe even the format of the file) could be completely unknown.

This is the current state of configuration in Linux. It is a bad state. Configuration of system-level properties, such as network interfaces and hostname, varies widely from one distribution to the next. System-level applications, such as Apache and Samba, each have a different method of configuration. Some tools and distributions provide applications to ease configuration, while some only provide a man page for the configuration file and leave it up to the user to edit. Nearly the only positive thing about Linux configuration is that it almost exclusively uses human-readable, human-editable text files.

This article is not going to talk about the existing state of configuration, however. This topic has been well discussed already. Instead, it will describe what it could be. It is a kick-off to a 9-month, 2-person project to address configuration in Linux. Hopefully, it won't be limited to 9 months or 2 persons, though. We invite your feedback and participation as we go.

Goals/Requirements

In order to develop a system that people will use and give direction to the project, some goals need to be defined. The primary goal is to set up a clear architecture in which various components of the system are clearly defined and separated. This is what this document will attempt to do. The following are some more specific goals, categorized into user requirements, data requirements, process requirements, and interface requirements.

User Requirements

  • The system should be easy enough to use and provide enough help that new Linux users can use it.
  • The system should be flexible and powerful enough that even advanced users will use it.
  • Administrators should be able to delegate access to various components of system configuration to other users (i.e., you should not need to give out the "root" password to give someone access).
  • Application developers who want to make their software configurable by this system should not need to modify their existing applications or make their applications dependent on it.
  • The system should be uniform across distributions. For example, a script written to update network configuration on Red Hat should work properly if run on Debian, because the interface is the same.
  • Internationalization and localization should be stressed throughout the project, so users from anywhere in the world can use their own languages to configure their systems.

Data Requirements

  • Configuration data should be stored in the native configuration files for the applications they configure. This will allow the system to work with existing configuration applications, as well as allowing the users to hand-edit the files if necessary.
  • Other data used by the system should be stored in human-readable, human-editable files.
  • Mechanisms should exist to keep secure data secure and to backup/restore data.

Process Requirements

  • The most obvious processes that should be available are methods to "get" and "set" configuration data.
  • Some configuration systems may need additional user-defined actions, such as an "activate" action which tells the appropriate application to reload itself so the new configuration info is used.

Interface Requirements

  • The system should support the ability for many different type of applications to access the configuration data. Interfaces should include a commandline program, a GUI using GNOME or KDE, a Web interface, and perhaps even an LDAP interface.
  • Programming interfaces between components should allow the different components to be written in any programming language.
  • User interface elements for configuration (e.g., the forms and wizards described below) should be defined at a level below the frontend, so multiple interfaces can use the same elements.

A Three-Tier Architecture

The architecture will be divided into many frontends, a middle layer, and many backends. The frontends will provide the user interface. This will allow commandline utilities that can be used in scripts, GUI interfaces for machines that have X installed, and remote interfaces such as Web and LDAP. The middle layer will provide shared functionality between the frontend interface and the backends. It could also be given capabilities like logging changes, controlling access, and caching configuration information. Finally, the backends will be the programs that actually read and manipulate the native configuration files. There may be backends for reading XML files and Windows INI-style files, as well as specialized backends for reading more complex configuration files (such as Apache's).

The Three-Tier Architecture
Frontends
Commandline Utilities GUI Web LDAP Specialized Tools (e.g., a "go back" utility)
Middle layer
Access control, logging, caching, and any other shared functionality, as appropriate.
Backends
Simple text files INI-style configuration XML configuration Specialized configuration (e.g., Apache, Sendmail)

How the layers communicate is something to be decided in the design phase, but it should allow each frontend and each backend to be written in any particular programming language that is best suited for the task. Perl, for instance, would probably be best suited for most of the backends, since it can easily parse and modify text files.

Modeling the Configuration Data as a Directory

One major feature of this three-tier design is that frontends are designed to handle configuring anything. In other words, the frontends know nothing about the programs and distributions they are configuring; they only provide an easy-to-use and consistent view of the configuration. How can you tell which frontend you want to modify, such as the list of users that are allowed to access one of your Samba shares? This is where the directory approach comes in.

Configuration will be modeled as a directory (or hierarchy), in a way similar to how GConf and the Windows registry handle configuration. Applications and system settings will be put into logical groups. Backends will be responsible for handling configuration "underneath" a particular branch of the directory.

A proposed hierarchy
Path Description
/ All configuration for the local machine is done under the "root". This root could be placed into another directory, which would be important for remote administration.
/apps This contains configuration defining how some system applications run. These are not daemons; they don't run all the time. Examples might include sudo and Debian's apt-get.
/daemons This contains system applications, such as Apache, Sendmail, and Samba. These are applications that normally run in the background, and are equivalent to "services" in MS Windows.
/daemons/samba This is where the backend of Samba would exist. Anything under this path would be handled by the Samba backend, which would be responsible for mapping the configuration of smb.conf into a directory.
/hardware This contains properties to configure hardware (e.g., sound card modules and network card IO addresses and IRQ numbers).
/network This configures network interfaces (e.g., the IP address and default gateway).
/system This contains system-level settings, such as hostname, default runlevel, and boot-loader settings.
/users This manages the addition/modification/deletion of system users and their properties.

Particular configuration locations are not fixed. Users should be allowed to rename, move around, or group system applications if they choose. As applications are installed, they will be placed in a suitable default location. Localization should be considered as well, so applications can be grouped using words from a user's native language. Users should also be able to create second copies of some applications, in case they want to run more than one version of the same application and they have more than one configuration file to edit.

Forms, Wizards, and Metadata

The system described so far does not present an interface much different from the registry in Windows. If you want to make a modification, you must navigate through a hierarchy and know the name of the attribute you want to add or modify. This is probably worse than editing the configuration files; with configuration files, you at least have the comments that were built in. This is where forms, wizards, and metadata come in.

Forms

Forms will give users a window in which they can see the important information and edit it using text boxes, list boxes, buttons, and other common controls. The information will be presented in an organized way and may be split into multiple tabs or pages if there is a lot of data to present. Forms can probably be best thought of as providing an interface like Webmin in order to modify configuration.

Forms will be defined using an implementation-independent language, such as XML Forms. The same form definition files could be used to create dialog boxes and Web pages. Forms will be distributed with a program that supports the configuration system, or they may be provided by a third party. As new versions of a particular distribution or system application are released, the forms may need to be updated to support new features.

Here is a screenshot of a prototype of how a Samba share form might look. Notice how the properties for this share are organized into four tabs, one of which is a generic "Other" tab which is capable of showing all properties. On the "General", "Permissions", and "Users" tabs, the application will show a form based on an XML form definition file that describes a layout of controls and bounds the data on the form to certain attributes of the Samba share.

Prototype screenshot

Wizards

While forms allow users to manipulate existing configuration data, wizards will give them opportunities to add things to the configuration. Take for example, adding a Samba share. It requires not just changing an attribute, but rather adding a whole section to the configuration file. Wizards will ask the user all the information needed to create something, then perform the steps necessary to do it. Wizards overlap somewhat with forms, because they will probably ask for the same information that the forms manipulate. Wizards, like forms, can probably be defined using XML Forms.

Wizards could also be created to install entire services. If integrated into a particular distribution, you might be able to use a wizard to add Samba to your computer. The wizard would use the facilities provided by the distribution to download and install Samba onto your computer, then create an appropriate default configuration file.

Metadata

Finally, metadata is information about the configuration data. Metadata could include the following items:

  • Descriptions of configuration items (e.g., excerpts from the application's manpages).
  • Data types and constraints (e.g., this attribute must be a filename; another attribute can be any string). XML Schema may provide a language for this.
  • Default values.
  • Comments in the configuration files as parsed by the backend.
  • Access Control Lists, for defining who has what type of access to the configuration.
  • A History Log, for keeping track of what changes were made and who made them.

Of course, metadata will be available to the frontends. The frontends with a higher level of user interaction may wish to display descriptions of the items and the comments associated with them.

Access Control

Access control defines who has access to read and/or change particular parts of the configuration. It will allow the root user to delegate control of the configuration to other users. This is important if more than one person is to administer a system, because a person who only needs to manipulate user accounts may not want access to the whole system.

Access control could potentially be quite flexible. In the traditional Unix filesystem, files and directories have three types of permissions which can be applied to the owner of the file, a group, and everyone. While pretty flexible, this has certain limitations, such as difficulty in giving two groups different levels of access while still denying access to everyone else. An access control language for Unix configuration could be much more flexible, allowing access to any number of users and groups and allowing very detailed types of access.

Access control is also useful when you start talking about logging, because it allows users to make changes as themselves instead of as root.

Logging

As users make changes to the configuration, they should be logged. The log could contain a date, the change that was made, and the user that made the change. If a remote access method is used, the IP address of the connecting machine could also be recorded.

A rich logging system would allow the creation of utilities that can "go back" to previous states of configuration. If someone made some changes that messed up the system, a "go back" utility could be used to revert the configuration to the state it was in just before those changes were made.

This "go back" utility would exist as a frontend, just like the commandline interface and the GUI interface. It wouldn't know anything about how to manipulate the configuration files; it would only know how to access the list of changes that were made and revert them.

Remote Administration

In the world of Unix, managing a computer remotely is easy, thanks to the wonders of SSH and remote X. For many users, these methods may be enough, but for administrators who manage a lot of servers, we may be able to provide better solutions.

Imagine a single program that you could use to browse multiple machines and edit their configurations, one which could handle connecting to the appropriate servers and providing the proper credentials to access the configuration.

Furthermore, imagine that when there's a new exploit for BIND, you do a search for all BIND programs (on all the servers in your organization), disable them all in one keystroke, apply the patch, and re-enable them.

Configuration Data Distribution

Remote administration could be taken to the next step by storing multiple copies of a computer's configuration on different servers. The primary source of the data would remain the computer that the configuration data is for, but secondary copies could automatically be kept on "peer" servers. This provides the unique ability to access configuration data when the server in question is down.

Furthermore, what if you were allowed to modify the configuration data when the computer is down? The modified data would be kept on the "peer" computer, and when the original computer came up, it would have to synchronize. The synchronization could lead to critical changes, so it might be best to synchronize before any daemons started.

These ideas about remote administration and data distribution are quite idealistic (and outside the initial scope of this project), but will give the project something to aim for.

Where to Start

At this point, we'd like to get feedback from the Linux and Open Source community. We'd like to answer questions like:

  • Is this something worth striving for?
  • Is it necessary to start from scratch, or can one of the existing configuration programs (e.g., LinuxConf and WebMin) incorporate some of these ideas?

Addressing details like what programming languages should be used and how to interface the various components is not necessary right now. These things will be decided later, during the design phase.

We'd also like you to visit our project site at SourceForge, http://config4gnu.sf.net/.

Recent comments

11 Apr 2003 08:21 Avatar skiingyac

PLEASE! READ THIS BEFORE POSTING
As one of the developers of the project, I'd like to clarify a few things that, judging by the comments so far, some people still seem to be grossly misunderstanding what our project is.


First off, our system is in an alpha stage at this point and we have proof-of-concept applications working very well with it, you should check them out at http://config4gnu.sf.net (http://config4gnu.sf.net)to get a better idea of what we've got.


Now to clear up some misconceptions:


Our system is an abstraction layer, *just like other config tools*. It is not a replacement for native config files, it does not require apps to use it, and it certainly does not intend to force a single way of doing things. It is just a tool that people can use to edit existing, plain text files one day, and the next day to edit them by hand if you wish. The system will provide an XML-based cache of configuration which you can use for various purposes. This XML cache will be kept up to date with the native config files, and NOT the other way aound. The beauty of our system is that you get an XML-based, structured interface to the native config files. Its just an interface, not necessarily a replacement.


Our system currently allows application developers to read their configuration easily using our API, though it does not require it. It is not a registry, it is an interface. If you want to write your own parser or your own file format, go ahead. We'll try to handle your special formats, but we'd like to keep the number of config file formats to a minimum if possible.


We intend the system to be for people who more or less know what they're doing. Someone made a good point that even if you have been a sysadmin for 20 years, there are just some apps that you don't configure often enough to remember all the details needed to hand edit a config file without consulting a fair amount of documentation.


Of course there are reasons why someone who doesn't know squat about unix should not be able to configure an important server in a production environment. Thats up to the boss. If they want to depend on someone being able to use our tool for their business to stay alive, they've got bigger problems.


*More importantly*, what if someone wants to mess with configuration on a *testing* machine? Why should it be so hard for them to learn?


Finally, here is a working example of how you could edit your Samba configuration with a perl script:


#!/usr/bin/perl
use config4gnu;


my $samba = config4gnu::get_configuration_root->resolve_path("/Daemons/Samba Configuration");


my $global_section = $samba->resolve_path(".samba-global");


my $wins_setting = $global_section->resolve_path(".wins_server");


print "Your samba's wins server is " . $wins_setting->get_string_value;


This example script asks the config4gnu library to load the samba configuration file (i.e., /etc/samba/smb.conf) into XML, locate specific nodes in it, and retrieve the output. Modifying it is just as easy, and the change is saved back in the native config file without changing ANYTHING else (whitespace, comments, blank lines, etc.).

14 Feb 2003 22:04 Avatar arschlesinger

The best idea for Linux since the inception of the Kernel
If this framework is implemented so that only open standards are used, particularly XML, then the project should grow fast. If people can distribute their programs for Linux, instead of for the Red Hat distros and for Slack(which almost never happens) and for Debian, then the Linux community can still be as diverse as it is now, but a VERY important standard will exist, also. The ability to easlily install software on Linux and configure services, just like on Windows is probably the most important issue and problem for Linux, and the largest drawback for users to move from Windows to Linux.
The reason Windows is so successful is that people can use the GUI programs(all in the "control panel") to configure everything that they probably will ever need to. If they do need to do something that they can't in the control panel, then they can go into the Registry and do it. The problem with Windows(that myself, as well as every Linux user hates) is that if, while you are configuring something in the control panel, you make a mistake, you had better have made a backup because Windows will crash. Basically Windows is easy but not reliable. I think that I can safely say that even the average user wants both these days(especially because he/she is sick of crashes, restarts, and the blus screen.)
The proposal here is to create a registry for Linux, which I think should be a standard filestructure for all Linux Distros. The difference between the Windows Registry and Linux one is that nothing could be done to this "registry" to crash the system because in Linux, everything is modular. This goes back to the reason why it is so hard to use.

summary:
the reason Linux is hard to use is because it is highly modular; so much so that it differs across any distro. The modularity scares many users because their newly installed program could be anywhere in the huge, hard to understand filesystem of /usr and /etc. What average user is going to know what is in the /etc folder? This project will effectively bring the modularity(which is a 100% benefit to linux and in no way a drawback to it) and inherent stability of Linux under a much needed and (hopefully) open standard. This standard will bring consistency across every linux, and will help win (hopefully lots of) converts over from Windows.

Basically, a user can do everything on Linux that they can on Windows (except play some awesome games, but that's why we all have PS2!) if not better than they can on Windows...but how are they gonna get these dream programs, and better yet, how are they gonna figure out how they work?

The only thing that linux is missing today is a reliable and standard control panel. From there, people will start to appreciate the uniqueness of every distro of Linux.

-Aaron

02 Feb 2003 16:27 Avatar kdan

Re: it's just a BAD idea.
Re: Leakage, I believe he's talking about <a
href="http://www.joelonsoftware.com/articles/LeakyAbstractions.html">Leaky
Abstractions, a very useful concept to
understand why a config adminsitration layers
won't solve all your headaches (though it might
ease some of them).


Daniel

21 Jan 2003 04:45 Avatar stevenwise

Standardized Configuration. Since when is Linux supposed to be standard?
I remember once (it would seem not long ago) reading a very good article by Gual Duval of Mandrake about the cons of standardizing one GUI for Linux, like deja-vu reading this article brought me right back to that day like it was yesterday. Since it's infancy over a decade ago GNU/Linux has always been about 2 things Freedom and Flexibility. With a past being as such, it's no wonder any discussion of having a standard would be such a hot topic. The problem is in the several pages of Technical Jargon nowhere does it mention the wants of the user. The fact of the matter is the typical Linux user wants just about as much &quot;Standardization&quot; as old Bill wants to open up to the wonderful world of GNU soft. Big or small Linux will be what it always has been, mainly a community of disgruntled Windoze users who are tierd of being told how to do it. So Why, Oh Why after all of these years would we turn back the clock and sell out to a standard? To attract a few more of Bill's gurus that barely know the difference between the terms 'point' and 'click'? Anyone that thinks standardization is the secret to Windoze success has spent the last about 5 years looking through a one sided pane of glass, but for those of us that have actually followed the world of software know it is a sheer combination of unfair domination and clever marketing. With Linux (intentionally) missing these concepts it is hard to see how standardization will do little more than upset allready dedicated users. Where are we to ensure freedom in Linux if we don't do it right at the source?? Ask yourself this question, How does one sail freely down a river once it's source has been capped by a huge dam? This is essentially the same thing, is it not?
As for a standard in programming let us take a look at DirectX, a fine example of standardization if there ever was one. Once upon a time there were three ways to do things when it came to Windoze game programming, the right way, the wrong way and the ingenious way. Nowdays it's hard to find a game that doesn't make use of this handy little tool. While on the surface, this may seem like a easy convenient way to do things, this half-hazard way of marching all our young ambitious programmers down this same narrow worn path is dangerous. It is not only a matter of freedom, but this tunnelvision way of buisness stunts advancement and development of other potentially better ideas. Is this the path we really wish to set Linux down???
If anyone wants to devote their time and skill to a program to simplify Linux development or tools to help the new enthusiastic Linux user this would certainly be a worthwhile if not welcome project, but do me and my Linux box a favor, Let's leave the &quot;Standardization&quot; for the Windoze gurus.

17 Jan 2003 07:44 Avatar adembaba

Re: it's just a BAD idea.

> I hope i'm not paraphrasing other people's comments too much, as a lot of
> valid points that I agree with have been made but this is purely and simply
> a bad idea.


Is it, really?


> As has already been mentioned the current system allows greatest flexibility
> and configurations that are actually well suited to the application they
> partner with. If you want a pointy-clicky-dont-have-to-use-my-brain


This is the kind of mindless bigotry that drives me up the wall..
If you think you have all the time to *learn* (as if the knowledge you
gain from someone's half-baked config solution can be called that),
go ahead; learn all that essentially useless stuff. But, please don't
even attempt to tell us that it is anything but a never-ending workaround.

A config file is a config file is a config file. It is supposed to be
something passive and comprehensible. Not a compilable script (if you
need a script, put it in the compiled code for gods sake), or a symbol
soup.

If you want to hold on to your freedom, be my guest: Do it in the
source code.

And, what leakage are you talking about?

If you are not supposed to edit config file in between admin sessions
you will not, that's it. Either the config software will not let you do
it, or it will at least warn you. What have better than that at the
moment?

About root level access? I believe you can Telnet over SSH etc (as secure
as that is), why can't you think of a similar solution?

And, finally, about storing your config files in one location... I see,
I have finally found someone who mirrors his/her every single byte at
a remote location --not just RAID50 :-)


> Anyway, these are my opinions, for what they are worth ;).


You are definitely not part of the solution ;-)

Cheers,
Adem

Screenshot

Project Spotlight

Kigo Video Converter Ultimate for Mac

A tool for converting and editing videos.

Screenshot

Project Spotlight

Kid3

An efficient tagger for MP3, Ogg/Vorbis, and FLAC files.