Articles / A Solution to the Problem o...

A Solution to the Problem of Configuration in Linux

What is configuration in Linux? What do experienced system administrators do when they need to, for example, modify the access rights to a Web site or change the network settings for their server? Invariably, they're going to login into the machine and edit a text file using a Unix editor such as vi or Emacs. That by itself isn't that bad, but depending on which application you want to configure and which Linux distribution you happen to be using, the location of the file you need to edit (and maybe even the format of the file) could be completely unknown.

This is the current state of configuration in Linux. It is a bad state. Configuration of system-level properties, such as network interfaces and hostname, varies widely from one distribution to the next. System-level applications, such as Apache and Samba, each have a different method of configuration. Some tools and distributions provide applications to ease configuration, while some only provide a man page for the configuration file and leave it up to the user to edit. Nearly the only positive thing about Linux configuration is that it almost exclusively uses human-readable, human-editable text files.

This article is not going to talk about the existing state of configuration, however. This topic has been well discussed already. Instead, it will describe what it could be. It is a kick-off to a 9-month, 2-person project to address configuration in Linux. Hopefully, it won't be limited to 9 months or 2 persons, though. We invite your feedback and participation as we go.

Goals/Requirements

In order to develop a system that people will use and give direction to the project, some goals need to be defined. The primary goal is to set up a clear architecture in which various components of the system are clearly defined and separated. This is what this document will attempt to do. The following are some more specific goals, categorized into user requirements, data requirements, process requirements, and interface requirements.

User Requirements

  • The system should be easy enough to use and provide enough help that new Linux users can use it.
  • The system should be flexible and powerful enough that even advanced users will use it.
  • Administrators should be able to delegate access to various components of system configuration to other users (i.e., you should not need to give out the "root" password to give someone access).
  • Application developers who want to make their software configurable by this system should not need to modify their existing applications or make their applications dependent on it.
  • The system should be uniform across distributions. For example, a script written to update network configuration on Red Hat should work properly if run on Debian, because the interface is the same.
  • Internationalization and localization should be stressed throughout the project, so users from anywhere in the world can use their own languages to configure their systems.

Data Requirements

  • Configuration data should be stored in the native configuration files for the applications they configure. This will allow the system to work with existing configuration applications, as well as allowing the users to hand-edit the files if necessary.
  • Other data used by the system should be stored in human-readable, human-editable files.
  • Mechanisms should exist to keep secure data secure and to backup/restore data.

Process Requirements

  • The most obvious processes that should be available are methods to "get" and "set" configuration data.
  • Some configuration systems may need additional user-defined actions, such as an "activate" action which tells the appropriate application to reload itself so the new configuration info is used.

Interface Requirements

  • The system should support the ability for many different type of applications to access the configuration data. Interfaces should include a commandline program, a GUI using GNOME or KDE, a Web interface, and perhaps even an LDAP interface.
  • Programming interfaces between components should allow the different components to be written in any programming language.
  • User interface elements for configuration (e.g., the forms and wizards described below) should be defined at a level below the frontend, so multiple interfaces can use the same elements.

A Three-Tier Architecture

The architecture will be divided into many frontends, a middle layer, and many backends. The frontends will provide the user interface. This will allow commandline utilities that can be used in scripts, GUI interfaces for machines that have X installed, and remote interfaces such as Web and LDAP. The middle layer will provide shared functionality between the frontend interface and the backends. It could also be given capabilities like logging changes, controlling access, and caching configuration information. Finally, the backends will be the programs that actually read and manipulate the native configuration files. There may be backends for reading XML files and Windows INI-style files, as well as specialized backends for reading more complex configuration files (such as Apache's).

The Three-Tier Architecture
Frontends
Commandline Utilities GUI Web LDAP Specialized Tools (e.g., a "go back" utility)
Middle layer
Access control, logging, caching, and any other shared functionality, as appropriate.
Backends
Simple text files INI-style configuration XML configuration Specialized configuration (e.g., Apache, Sendmail)

How the layers communicate is something to be decided in the design phase, but it should allow each frontend and each backend to be written in any particular programming language that is best suited for the task. Perl, for instance, would probably be best suited for most of the backends, since it can easily parse and modify text files.

Modeling the Configuration Data as a Directory

One major feature of this three-tier design is that frontends are designed to handle configuring anything. In other words, the frontends know nothing about the programs and distributions they are configuring; they only provide an easy-to-use and consistent view of the configuration. How can you tell which frontend you want to modify, such as the list of users that are allowed to access one of your Samba shares? This is where the directory approach comes in.

Configuration will be modeled as a directory (or hierarchy), in a way similar to how GConf and the Windows registry handle configuration. Applications and system settings will be put into logical groups. Backends will be responsible for handling configuration "underneath" a particular branch of the directory.

A proposed hierarchy
Path Description
/ All configuration for the local machine is done under the "root". This root could be placed into another directory, which would be important for remote administration.
/apps This contains configuration defining how some system applications run. These are not daemons; they don't run all the time. Examples might include sudo and Debian's apt-get.
/daemons This contains system applications, such as Apache, Sendmail, and Samba. These are applications that normally run in the background, and are equivalent to "services" in MS Windows.
/daemons/samba This is where the backend of Samba would exist. Anything under this path would be handled by the Samba backend, which would be responsible for mapping the configuration of smb.conf into a directory.
/hardware This contains properties to configure hardware (e.g., sound card modules and network card IO addresses and IRQ numbers).
/network This configures network interfaces (e.g., the IP address and default gateway).
/system This contains system-level settings, such as hostname, default runlevel, and boot-loader settings.
/users This manages the addition/modification/deletion of system users and their properties.

Particular configuration locations are not fixed. Users should be allowed to rename, move around, or group system applications if they choose. As applications are installed, they will be placed in a suitable default location. Localization should be considered as well, so applications can be grouped using words from a user's native language. Users should also be able to create second copies of some applications, in case they want to run more than one version of the same application and they have more than one configuration file to edit.

Forms, Wizards, and Metadata

The system described so far does not present an interface much different from the registry in Windows. If you want to make a modification, you must navigate through a hierarchy and know the name of the attribute you want to add or modify. This is probably worse than editing the configuration files; with configuration files, you at least have the comments that were built in. This is where forms, wizards, and metadata come in.

Forms

Forms will give users a window in which they can see the important information and edit it using text boxes, list boxes, buttons, and other common controls. The information will be presented in an organized way and may be split into multiple tabs or pages if there is a lot of data to present. Forms can probably be best thought of as providing an interface like Webmin in order to modify configuration.

Forms will be defined using an implementation-independent language, such as XML Forms. The same form definition files could be used to create dialog boxes and Web pages. Forms will be distributed with a program that supports the configuration system, or they may be provided by a third party. As new versions of a particular distribution or system application are released, the forms may need to be updated to support new features.

Here is a screenshot of a prototype of how a Samba share form might look. Notice how the properties for this share are organized into four tabs, one of which is a generic "Other" tab which is capable of showing all properties. On the "General", "Permissions", and "Users" tabs, the application will show a form based on an XML form definition file that describes a layout of controls and bounds the data on the form to certain attributes of the Samba share.

Prototype screenshot

Wizards

While forms allow users to manipulate existing configuration data, wizards will give them opportunities to add things to the configuration. Take for example, adding a Samba share. It requires not just changing an attribute, but rather adding a whole section to the configuration file. Wizards will ask the user all the information needed to create something, then perform the steps necessary to do it. Wizards overlap somewhat with forms, because they will probably ask for the same information that the forms manipulate. Wizards, like forms, can probably be defined using XML Forms.

Wizards could also be created to install entire services. If integrated into a particular distribution, you might be able to use a wizard to add Samba to your computer. The wizard would use the facilities provided by the distribution to download and install Samba onto your computer, then create an appropriate default configuration file.

Metadata

Finally, metadata is information about the configuration data. Metadata could include the following items:

  • Descriptions of configuration items (e.g., excerpts from the application's manpages).
  • Data types and constraints (e.g., this attribute must be a filename; another attribute can be any string). XML Schema may provide a language for this.
  • Default values.
  • Comments in the configuration files as parsed by the backend.
  • Access Control Lists, for defining who has what type of access to the configuration.
  • A History Log, for keeping track of what changes were made and who made them.

Of course, metadata will be available to the frontends. The frontends with a higher level of user interaction may wish to display descriptions of the items and the comments associated with them.

Access Control

Access control defines who has access to read and/or change particular parts of the configuration. It will allow the root user to delegate control of the configuration to other users. This is important if more than one person is to administer a system, because a person who only needs to manipulate user accounts may not want access to the whole system.

Access control could potentially be quite flexible. In the traditional Unix filesystem, files and directories have three types of permissions which can be applied to the owner of the file, a group, and everyone. While pretty flexible, this has certain limitations, such as difficulty in giving two groups different levels of access while still denying access to everyone else. An access control language for Unix configuration could be much more flexible, allowing access to any number of users and groups and allowing very detailed types of access.

Access control is also useful when you start talking about logging, because it allows users to make changes as themselves instead of as root.

Logging

As users make changes to the configuration, they should be logged. The log could contain a date, the change that was made, and the user that made the change. If a remote access method is used, the IP address of the connecting machine could also be recorded.

A rich logging system would allow the creation of utilities that can "go back" to previous states of configuration. If someone made some changes that messed up the system, a "go back" utility could be used to revert the configuration to the state it was in just before those changes were made.

This "go back" utility would exist as a frontend, just like the commandline interface and the GUI interface. It wouldn't know anything about how to manipulate the configuration files; it would only know how to access the list of changes that were made and revert them.

Remote Administration

In the world of Unix, managing a computer remotely is easy, thanks to the wonders of SSH and remote X. For many users, these methods may be enough, but for administrators who manage a lot of servers, we may be able to provide better solutions.

Imagine a single program that you could use to browse multiple machines and edit their configurations, one which could handle connecting to the appropriate servers and providing the proper credentials to access the configuration.

Furthermore, imagine that when there's a new exploit for BIND, you do a search for all BIND programs (on all the servers in your organization), disable them all in one keystroke, apply the patch, and re-enable them.

Configuration Data Distribution

Remote administration could be taken to the next step by storing multiple copies of a computer's configuration on different servers. The primary source of the data would remain the computer that the configuration data is for, but secondary copies could automatically be kept on "peer" servers. This provides the unique ability to access configuration data when the server in question is down.

Furthermore, what if you were allowed to modify the configuration data when the computer is down? The modified data would be kept on the "peer" computer, and when the original computer came up, it would have to synchronize. The synchronization could lead to critical changes, so it might be best to synchronize before any daemons started.

These ideas about remote administration and data distribution are quite idealistic (and outside the initial scope of this project), but will give the project something to aim for.

Where to Start

At this point, we'd like to get feedback from the Linux and Open Source community. We'd like to answer questions like:

  • Is this something worth striving for?
  • Is it necessary to start from scratch, or can one of the existing configuration programs (e.g., LinuxConf and WebMin) incorporate some of these ideas?

Addressing details like what programming languages should be used and how to interface the various components is not necessary right now. These things will be decided later, during the design phase.

We'd also like you to visit our project site at SourceForge, http://config4gnu.sf.net/.

RSS Recent comments

28 Sep 2002 01:50 wiresoft

Wiresoft ANA (self-managing Linux)
Many of the concepts introduced here are actually delivered today by the ANA (automated network agent) systems management service. The approach they have taken is to centralize the management service infrastructure and use remote admin protocols to manage distributed Linux servers and devices. Configuration, provisioning, updating, etc. is covered, as well as back-up, recovery, monitoring, alerting, optimization, etc.

28 Sep 2002 01:57 wiresoft

Re: Wiresoft ANA (self-managing Linux)

Freshmeat project page:
freshmeat.net/projects...

28 Sep 2002 02:05 schlaffi

hmmm, is linux configuration that simple?
The "configuration of linux" is easier as you may think, as long as the developer provides well formed man files .... Imho, you missed an important point about these configuration files: The developer has good reasons to design a configuration file in a particular way, since it should address the following:

-easy to parse
-no errors are possible like "could not start: broken configuration file"
-it reflects the needs of the thing that should be configured.

In special, the last point needs e.g. to develop some script-like configuration file (as modules+devfs or shell profiles) that are beyond all generalizied interface.

28 Sep 2002 02:23 sheister

it's growing!
two words: code bloat

28 Sep 2002 02:40 gregb4242

Work smarter, not harder
I completely agree that Linux configuration needs to be more standardized but I think your aproach to the problem is too brute force trying to write backends for all the different config file formats.

Develop a standard XML DTD for config files and publish the standard. The reason there are thousands of config formats is because there is no standard way to write a config file other than what is easiest to parse for the type of program using the config. After you develop the DTD, develop a very easy to use library to expose the config settings to the program using the config. This is the key. If you make a format/library that makes the coders job easier, you are home free.

Then use the big advantage of OSS and start submitting patches to existing projects that don't seem inclinded to switch config formats. Once the pump is primed enough the rest will take care of itself.

28 Sep 2002 02:46 rjmbos

Already exist
See:

Gnome Setup Tools

28 Sep 2002 04:22 pelam

Re: Already exist
The whole discussion lacks some encouraging spirit methinks :/

Great! Maybe these 2 projects can share (or rip) each others code or just benefit from the competition. Maybe even whole components (backends?) could be interoperable. This new project does not seem to have fixed interfaces yet, so maybe they could embrace (and extend?) GST's XML interface?

I see this configuration thing as a large obstacle between saving some friends from the darkness of the ... you know.

Any holistic solution could work miracles here. No matter if some feel it is, bloated and unjustly dependant on programs XYZ. Relative bloating won't even be a problem, since you can alwatys do "dpkg --purge", and edit files manually.

As a Perlist, I think the acceptance of different languages for eg. back-end's is great. I guess I could write some Perl back-ends for familiar formats, in a day. (Also CPAN already harbors modules for many formats)

Be creative, be nice, :)

Pelam

28 Sep 2002 05:12 yeti2

doomed to fail
If your, or any other, configuration system won't be actively supported by developers of the packages in question, it's doomed to fail. Such a system may work only if the packages share common configuration backend (as in GNOME).

Wizzards, being a kind of templated sample conmfigurations, can work. But that's little gain from such a grandiose project.

Regarding general configuration tool, you have two basic problems, the first being conceptual, the second practical.

1) When configuring complex things, a knowledge of what one is doing is required. You just can't sit down, launch a config tool and configure properly mysql privileges, apache mod_rewrite, samba logon scripts, or a firewall. Thus the user will need to read the documentation anyway. And the documentation speaks about package config files, not about your tool. Thus the user needs to learn both. Also, as you can't eliminate the necessity of hand-editing of config files completely, and the most probable case one will need to return back to hand-editing of config files is when one gets into troubles, when something doesn't work as expected, or not at all. Of course, who used you configuration tool will be unable to do so, or he will have to learn all the config file language right now, instead of learning it gradually, starting from configuring simple things (this is usually easy, because the config files are well documented in package documentation as well as inside the config files themselves).

2) A package-xx.y.z configuration file needs to be accepted by package-xx.y.z, package-xx.y.z specific configuration tool needs to support package-xx.y.z. But your tool will needs to support all versions. And not just simply apache-1.3 and apache-2.0, but it needs to know a particular option appeared in apache-1.3.12, thus if we are configuring apache-1.3.11, it should not be offered. It also needs to read my /usr/share/mysql/charsets/Index to learn my mysql knows about utf8adnocase charset and offer me it in configuration. And it needs to learn configuration options of my extra apache modules, somehow. Briefly speaking, it needs to correctly parse all configuration files of all possible versions of a given package, with any additional modules. This will lead to a kind of unmaintainable bloatware, becoming more and more broken with every new version of either to-be-configured packages or the tool itself.

28 Sep 2002 06:51 pelam

Re: doomed to fail

Some "buts" follow... I don't think the authors of this intended to make all-in-one solution for everybody and every software. A good subset of useful programs with good subset of available features and doc snippets can go far in helping people to efficiently manage their boxes.

Also if something like this will ever become popular, maybe new programs will start to come with simple optional interface to it.

If this was included as a package in some distribution, the package version could be correlated with other package versions. Also the number of possible packages and versions would be smaller.

Also many everyday programs, are not in the midle of heavy development. Some programs maintain stable config files and backwards compatibility event through development.

Since their back-ends are 2-way, I could help a friend by finetuning his mysql installation manually, after which he can still modify everyday things cosily from a nice GUI.

Even if he was forced to learn details of mysql config files and privilege tables, he still would not need to learn similar things about samba. Hopefully, not that same instant at least.

I admit, that full complexity of program configuration can't be translated into user-friendly interface (unless some PL is considered user-friendly). ;) Expressive power of some everyday config files might turn out to be equivalent to Turing Machines and other programming formalisms.

Luckily, this does not rule out the possibilty that a high percentage of everyday system administration could fall into much more confined class.

With kind regards,

Pelam

28 Sep 2002 07:10 mordecai

Good idea, but...
I think it's a great idea, really I do. I'd love to see well-documented XML files for configuring everything.

But (and not wishing to start a flame war) SuSE does something similar with YAST, and for me it is the biggest pain in the backside imaginable. I hate it. The thought of auto-generated files that you can only change in the way that the GUI designer lets you fills me with horror.

What might be better is a proper set of documentation for each one (which the LDP is doing very nicely) and man pages/info pages for each package.

28 Sep 2002 08:03 yeti2

Re: doomed to fail

OK, I'll summarize the basic objection once again. What is it supposed to do, when I have either newer, or older, or patched version of the package, or some not-so-standard modules? Please note this is quite a common situation. Even when the packages in question are relatively stable, new configuration options appear quite regularly (see e.g. changes in the 1.3 apache branch). Is a disto vendor supposed to release a new version of configuration tool with every other update of any package? If you support 20 packages and each changes only twice a year, you still need one release of the configuration tool every 9 days (in average).

What it should do when it encounters something it doesn't understand in the config file? Omit it? This is asking for disaster. Copy it literarly to the new config file? This is proably safer than the first possibility, but not much as it's not exactly clear what and where to copy.

So, as I said before (and you said too), basic setup can be handled this way (wizzards, ...). It's possible to create a tool writing feasible configuration files. But reading them and mixing the configuration tool and hand-editting is not likely to work, except for trivial cases.

28 Sep 2002 09:29 pelam

Older related article on Freshmeat
Worth reading if you are thinking about these problems.

How to Fix the Unix Configuration Nightmare - Matthew Arnison (freshmeat.net/articles...)

He has some valid points about earlier attempts on this same subject and about future.

Regards,

Pelam

28 Sep 2002 09:35 jeffcovey

Re: Older related article on Freshmeat
Of course, if you look closely, you'll notice that article is already
linked from this one. :)

28 Sep 2002 11:06 pelam

Re: doomed to fail
There is always a point, where this (it's called Configuration 4 GNU (C4G) :), has to throw it's hands in the air and admit it has no clue about a given file. However it can try to do it gracefully and it can also try to push that point as far as possible. Anyway, C4G will probably have to check software versions and inform the user if it is not familiar with given version.

If it can't fully understand a file, it could offer to revert the file to previous version. It could also try to identify the weird bits and ask the user for a policy about them. Maybe it could even give the user searches to documentation of the particular package version.

If a more sophisticated user or program added the weird bits manually it/he could have also added special comments telling both C4G and regular user what to think about them. Some config files have some syntactical structure that could also help with these kind of decisions.

Many users also only upgrade their system with full releases of given distro (+ security upgrades hopefully). Then it could be expected that C4G and the all packages around are fairly well in sync.

Also, C4G will hopefully have modular enough architecture that only small part of it will need to be changed/upgraded when any given package is changed. Maybe there could even be some-kind of online service that given specific software versions, would provide new versions of some C4G modules and metadata.

Regards,

Pelam

PS. Apologies for everybody about that duplicate link I posted above :) I thought I was so clever to stumble upon it :I

28 Sep 2002 11:40 jc2000

What about cross application configuration?
It would be great if it would be possible to find a generic logical model for all configuration information of an application.

e.g. it should be possible to import existing apache configuration data, enable "exec cgi scripts" for "/myhomepage", and write the data back; and - oh, I dont like apache any more, I would like to switch to any other supported webserver and keep my configuration information. Ok, not every webserver supports the same features, but it should use all supported features.

28 Sep 2002 11:42 pelam

Re: doomed to fail
One more thing came to mind. C4G could add special sections to all configuration files as soon as it first sees them and user approves. These sections could be marked with special comments.

There could be predefined places for eg. things to keep, things to ask user about, security critical stuff and stuff that can be removed (and archived) if need be.

Btw. Thanks for this discussion.

Yours,

Pelam

28 Sep 2002 13:51 ajg

Re: Wiresoft ANA (self-managing Linux)
I would much prefer to occasionally edit a configuration file than pay $1000 for something to do it for me... I expect I'm not alone here... :)

28 Sep 2002 14:28 CherniyVolk

Ok, this is justified elitism if you ask me.

Ever since the lamers have come to discover us Linux/BSD users having fun in the corner, there have been many attempts to send our communities down the drain.

I learned how to use Linux and BSD, the MAIN APPEAL of these operating systems is that they are 100% customizable!
That includes the implications that MY SYSTEM IS MINE, and
very little like the next guys!

So, I had to adapt to this philosophy so can YOU! Don't tell
a thirty year old technology it needs to change becuase all of
your loser buddies coming from windows do not realize if you bring over that crap from Windows, then Linux WILL no longer be better than Microsoft Windows!! You people are just stupid, and pissed that you cannot understand the material so you and some of your MCSE buddies or your lame CS degree figures you have clout in this market arena and start throwing ideas to corrupt a working system. Why the hell [from one of the BSD mailling lists] FreeBSD/OpenBSD needs a GUI based installer would be totally degrading the quality and functionality of the current installer. Some friggin newbie was crying because RedHat had a GNOME installer and some flavour of BSD didn't. The GUI, ease of use does not, never has and never will equate to "better" or "advanced". The functionality of the working code is what constitutes what is better. Not "How stupid a person can be, and still use this device". Frankly, I don't give a damn about the stupid people, you all can rot in hell and suffer the restraints of Microsoft Windows for all I care. Don't come here dragging your Microsoft problems with you, remember why you converted in the first place.

28 Sep 2002 15:11 datamoist

Config file as File System
Turning configuration files into a whole filesystem could be an end in itself. Consider:
- Hans Reiser wants to make the filesystem intelligent so the an XML document could look like a directy of files. He will put the XML-to-directory-and-back translation into the filesystem. No code for you to write. (Poor Hans, doing our work. ;)
- Logging and archiving configurations would be a job for tar (or CVS). No new code to write. (ok, maybe a cron job and a nice 'restore configureation' GUI that calls untar.)
- Access control (ACLs) already exist, and will eventually become standard (just like journaling file systems have). Neither the application nor the C4G program will need any special permissions-checking code -- the filesystem will do it. GUIs may be needed to help the user delegate tasks, but the hardend sysadmin will just use chmod or chacl.
- Task automation (such as "delete the /tmp share from samba") don't need parse code anymore. These tasks can now be done with simple shell scripts.
- Remote administration would be a simple network mount. No new protocols, no new code. Accessing and changing data when the computer is down is alreay supported by file systems like coda and intermezzo. Until those become more robust and standardized, a file-sync program like unison can be used. (The advantage over normal config files: fine-grained sync reduces the conflicts).

28 Sep 2002 15:44 smallberries

Re: Work smarter, not harder
XML isn't very good for plain configuration files. This might not be the standard gee-whiz XML is great answer, but it is true. I've used lots of XML, but I've moved away from using it in human-driven configuration.

XML is great for data, it is great for machines reading and exchanging data. It is a pain for humans. XML might have a place as an intermediate form like a parse tree, but people don't think in XML, machines do.

28 Sep 2002 16:13 dentar

Don't save state!!
The problem with Linuxconf style configurators is that the configuration program attempts to "save state" elsewhere other than the configuration files themselves. This causes problems in a multi-administrator environment or in an instance where the admin wants to edit the file by hand. Next time the admin wants to go into Linuxconf (for example) to fix something not related, the Linuxconf program tries to sync up everything, thus throwing off what the administrator did with the file by hand.

28 Sep 2002 17:30 ariejan

Toybox
That's why, in my opinion, Linux is just a toy-O/S. Everyone builds his own Linux, and doesn't pay any attention to compatibility with other distributions, or any other standard there is.

I'm a fanatic FreeBSD user, and the best thing about FreeBSD is its purity. No code bloat, just a very fast and stable kernel, and everything is in places where it's logical to store them.

Maybe it's too late for Linux distro's to come clean, and produce some sane standards? I'll just keep to FreeBSD.

28 Sep 2002 17:41 wiresoft

Re:
There is a real need for the automation of configuration, as well as other management tasks, within Linux. So I agree that opening this discussion is a good idea, and disagree that it is in conflict with Linux's advantages for customization, as well as that it is not a valuable service (worth $1000 plus dollars per year, per server). The problem with the relative complexity of Linux configuration really shows up when provisioning systems, and again at times when changes need to be made. It takes about the same amount of time as windows based systems, but requires much more expertise. This is the PROBLEM that needs to be solved soon. IBM estimates that there will be over 1 billion systems by 2010, and we will need over 200 million admins to manage them. Most of systems will be Linux. We currently have about 15 million good admins in the world today. We will never train enough in time, so AUTOMATION of systems management is a must to avoid a co$t crisis. I don't believe configuration should be standardized, simply the common variations cataloged and accessible to automation engines that perform config changes. These automation engines could behave like web services and use onyl standard communications protocols to manage systems, like SSH, WSDL/SOAP, etc.

28 Sep 2002 18:17 Ritchiey

Sounds good
Would you make the storage of the settings pluggable? Would you consider using LDAP as your store?

PCs and servers set up within a large (or small) organisation may be best served by keeping the configuration in an LDAP enabled directory (eg OpenLDAP). It's a standards based replicable directory. The administrator doesn't have to go to a machine or even connect to a machine to configure it.

It may also make sense to incorporate the possiblity of configuring other devices. Eg Unix servers, Windows servers and workstations, printers, etc) from the same console/store.

28 Sep 2002 19:22 HandyCannon

Webmin
www.webmin.com

Er, what's wrong with webmin? It's gui in the sense
it works in a browser, which also makes the front
end cross platform. It isn't just tied to Linux either,
and works across all Linux distros afaik.

Why re-invent the wheel....

28 Sep 2002 23:03 matman

Re: Work smarter, not harder

> XML isn't very good for plain
> configuration files. This might not be
> the standard gee-whiz XML is great
> answer, but it is true. I've used lots
> of XML, but I've moved away from using
> it in human-driven configuration.

I agree that XML is not as easily editable as some formats, however, it does offer some other features that other formats do not offer.

o It's editable by hand (although not as editable as some other formats)
o It's very easy to parse (since there are XML parsers for most languages)
o It's flexable enough to express most configuration options
o It is flexable enough to include inline hints that are easy for an editor to display nicely
o It is easy to validate configuration data since you can validate against a DTD
o Using namespaces you could probably embed configuration structures not offered by the format, that could still be handled reasonably by a 'config editor'
o The config could be quickly turned into docs
o The markup language could be defined such that one could split config up into multiple files, so that each file could have permissions set differently (some sort of includes)
o It is quick and easy to query XML for values - you can check for config values without implementing parser after parser and worrying about bugs

And, I agree with the original poster - offering developers an easy way to get at configuration that is also easy for third party utilities to get at is the way to go. Working togeather is the way to improve things - working around everyone else is not effective, and won't work out. Lets solve a problem, not work around it.

29 Sep 2002 09:40 gsobalvarro

Make it easy for lazy people? Nahhh!
Use FreeBSD as an example and take it from there. This is for people who know the insides of their systems and not for superficial types who just wan't a slam-bang-thank-you-ma'm OS. FreeBSD requires a "relationship", a commitment, and not just a one night stand. This is not for lazy asses. The market wants people who know what they are doing, not people who know how to use a GUI. If you want dancing clips, Einstein and doggies, stick to XP! Guess that's what all the envy is about.

29 Sep 2002 13:06 tavisbarr

Re: Already exist

XST had/has a lot of potential and I was really sad to see Ximian drop the project. It may have been overly ambitious in trying to support other unices (e.g., Solaris, HP-UX) in addition to Linux. But it had a clean separation of backend and frontend, as well as the ability to save old versions of the configuration, that gave its underlying structure a lot of potential. Although it has been developed for Gnome, there is no reason a KDE or web frontend couldn't be made, and in fact the developers have mentioned that they secretly hope someone might try such an undertaking. Although the proposed project here has the advantage of using a filesystem-like layout for configuration information, this does not seem like a big enough advantage to justify the duplicated effort.

Anyway, I agree with the authors that confusing and inconsistent configuration tools, not a lack of applications or features, are the biggest barrier to getting non-technical users to enjoy Linux. Mandrake shows some promise but RedHat is just awful -- even 8.0, while showing some improvement, has its setup tools lying all over the place (Some under System, some under Setup, some under "Start Here" -- which is not where I'd look for configuration tools unless I knew to) and lacks things like a decent firewall setup.

Whatever you guys end up doing, duplicated or not, will be a big help if you manage to bring it to completion.

29 Sep 2002 20:57 skiingyac

Re: Work smarter, not harder
Yes, there is an obvious lack of a standard in this area, although there are several common config formats. Our goal is to provide parsers and "unparsers" to allow *everything* (comments, whitespace, etc.) to be transformed to and from a standard XML config format.

It goes without saying that our system would be able to read a config format written directly in the XML format we create. However, we also realize that, as some of the others who replied indicate, XML can be a pain and can be hard to read. By automatically detecting when the native config format has changed, we will re-create the XML as necessary. Essentially, this gives the benefits of XML without many headaches. Of course, it also goes without saying that part of the project may include creating a library which can be used by the programs to read its configuration from XML, Apache-style, INI-style, or whatever.

The strengths of this project come in because there is a single standard format (XML) and a single middle layer which everything must pass through. This means that specialized config files can be used simply by creating a parser and unparser to transfer the configuration to and from XML.

29 Sep 2002 21:15 skiingyac

Re: doomed to fail
Well, first off a main goal of the project will be to create files for current versions of popular software (apache, samba, php, qmail, etc.) Once these initial files are made, it is our hope (and we will work hard on making this happen) that the software maintainers or distro maintainers will include these files in the package with the software. This way, when user installs version x.y.z of software A on distro 34, the installer (whether its a tarball w/, an rpm, a .deb, an ebuild, whatever) will put the necessary files in the right place. If this were to happen, the headache would be gone. It is also important to know that our goal is to have these files consist of XML which is read by our system, and the XML would just define the configuration options and how they should be displayed, so that any reasonably up-to-date version of our software should be able to handle it. If we can't get software maintainers to include the file, we'll make a web-based database which the system can automatically check and get the necessary files from.

All the configuration will be in XML. If a particular configuration directive isn't recognized, it will be marked as such and will likely be displayed as a generic text entry field (although we're open to better suggestions). Since neither using our system or editing the config files by hand will exclude doing the other, extremely special situations won't be a problem as you can resort to doing it by hand. Keep in mind that both of us use a text editor to edit 99% of our config files and are more or less happy with that. We just think that there could (and should) be more options.

To help people understand the configuration, we would like to provide context-sensitive documentation inside the config interfaces where possible. For example, I plan to write a parser that will take things like samba's man page (which is *excellent*) and create an XML representation of it to allow our middle layer to associate the documentation with each directive. This, of course, needs to be mostly automatic, since I know that I definitely don't want to write lots of XML documentation by hand for every release of every project if I can help it.

29 Sep 2002 21:26 skiingyac

Re: Good idea, but...
I don't want to repeat myself, but you are mostly right that there will be auto-generated files that you can only edit the way the designer of the meta-configuration files wants you to. However, there will be a way for you to use the GUI to modify any configuration options they forgot about, and more importantly, the auto-generated XML files are only intended to be edited by the system itself. The modified XML is then sent to the backend where the original config file is re-constructed with the necessary changes and everything else preserved (whitespace, comments, etc).

You should still hand-edit the native config files if necessary. Few people like editing XML by hand very much, and with good reason. Personally, I'm pretty sure that I won't want to use this system more than 25-50% of the time, if that. For lots of things it is easier for me to whip open a text editor and hit a few keys without having to bother with a mouse. But for that 25-50%, it will (if successful) save people a lot of headaches.

29 Sep 2002 21:30 skiingyac

Re: What about cross application configuration?
Yes, that is an excellent idea for an extension of this system. If you could specify the machine's IP address(es) in a single place and hit the "Update globally" button, that would be great, provided it really worked of course :)

29 Sep 2002 21:41 skiingyac

Re: Ok, this is justified elitism if you ask me.
While I have some reservations about responding, I'd like to clear up a few things. First, both of us use Gentoo, which must be installed without a fancy GUI installer. In fact, I didn't even get X on my one box until after about 3 days later, after everything else was up and running (since I only have 1 monitor anyway...) Gentoo can do this not because it sacrifices configurability or speed, but because it is "slick". Much is automatic neither at the expense of abstracting things to an insanely high level or of pinning you into a little box that you can't get out of. Configuration in general should be equally slick. People who use Gentoo will know what I mean.

As far as your bashing of people you know nothing about, the last time I used windows was to see how a website looked in IE. You may say who cares how it looks in IE, to which I respond that over 50% of my client's visitors do. It would be nice if everyone used the same browser, but if you renounce everyone not using what you are, then you're doing to other people exactly what you're complaining about.

People like different things. There's nothing wrong with that. If you don't want to use our system, rest assured that even if you find yourself at a box which previously had only been configured by our system, you wouldn't even notice.

29 Sep 2002 21:53 skiingyac

Re: Sounds good
LDAP should not be the actual store for the data. The config data will remain in its native format to allow hand-editing, backward compatibility, etc. We did think about using LDAP as one of the interfaces, which could work well since it is designed to be able to edit such data. The LDAP server would merely read auto-generated XML made by our system and pretend to be actually doing things, when in reality it is just sending the modified XML back to our system. LDAP would be a good choice to offer, although due to the additional requirements (an LDAP server and knowledge of LDAP), it is currently not one of the primary interfaces and so will be developed later (or if you want to help, go for it!)

29 Sep 2002 22:01 skiingyac

Re: Config file as File System
Again, this could be a possible alternate interface, which would certainly make configuration very accessibly at the command line and using already existing tools. This isn't a primary interface because it ties it to that specific OS, but it could easily be added later. Still, the native configuration needs to be parsed to and from XML in order to get into the XML-to-filesystem code, so this type of an extension would work nicely as an add-on interface. We hope to make libraries available so that such a thing could be done. I think it is a great idea, and would definitely work nicely over a network.

Initially, our system will probably just rely on standard file permissions to determine whether a user can or can't update a configuration file, since that makes things simple. We want to make a durable core which allows these sorts of things to be easily added on, but we do agree that its important to keep things simple and not re-invent anything if we can help it.

29 Sep 2002 22:11 skiingyac

Re: Don't save state!!
I should point out that Linuxconf says on its website (sorry, I can't find where at the moment) that for some time (a few years I believe) it hasn't done that. Pretty much everyone agrees that this is a bad idea. All of the XML files used by our system are automatically generated as needed and the native config files are always treated as the authoritative copy.

Last I knew linuxconf did try to restart various services pretty much every time I used it for un-related things. I think that is a pain, and we'll be sure to avoid that except when necessary. For example, if someone changes a samba config directive, our system may restart samba automatically, remind the user to restart it, ask the user whether or not to restart it, or just shut up and assume the user knows that (this could depend on the user's preferences).

29 Sep 2002 22:19 skiingyac

Re: Webmin
There are some tools that are good at a lot of things. Webmin is one of them. Webmin has some flaws which detract from its usefulness to some extent. First, it is only web-based, which although that is probably the best interface to pick if you're going to focus on a single one, picking only one can also be bad.

Second, webmin requires a module be written for each new software, and that this module be added by the user. Our system will (ideally) handle this automatically as version-appropriate files will be included with each software package.

Third, webmin doesn't make it terribly easy to allow people to write new modules. There are some things built in, but not nearly enough. There's little reason why someone should have to write more than a basic text file to explain how a piece of software (or part of the system) is configured. We want our system to be easy and straightforward to use for everyone involved, without keeping people too far from the details. For example, we may want to display someplace in most of our interfaces a note explaining what actual config file the user is editing should they want to edit it by hand.

Our system is specific to the task of configuration, but it will be general enough so that it won't need to be re-written to add support for new things. The process will be very simple, and may even be mostly automatic.

29 Sep 2002 22:32 skiingyac

Re: Already exist
We really appreciate your input pelam, your posts are very encouraging. We have looked at various configuration tools as there are a lot out there, many of them in the alpha or pre-alpha stage. Unfortunately, all of them seem to lack the generality or standardization that we are hoping to create. I think it would be cool if we created plugins/modules for other tools like webmin and linuxconf that allowed those systems to use our system to edit configuration, or vice versa. We want to make everything we create very accessibly.

As for the problem of bloat, we're hoping that will be avoided by the fact that no information about specific software is going to be included in our actual code. That way, when someone is editing samba's configuration, only the general core, the specific backend, the users's chosen frontend, and the XML files for samba will be needed. A user may or may not have the files on his/her system needed to allow him/her to configure apache with our system, but either way they won't be loaded.

Multiple languages is of course always good and will hopefully encourage lots of participation. Right now, *all* of our parsers and unparers are written in perl (and all are very short). They just transfer the XML on stdout to the our prototype GUI (a C program). Pretty much any language with XML support (which leaves out very few) will work. Thanks for the tip on CPAN parsing modules, we'll look into that. :) Are you interested in helping out in any way? If so, contact either me or Jason, we'd love to hear from you.

29 Sep 2002 22:52 bevege

Re: Make it easy for lazy people? Nahhh!
Attitudes like this is what is holding Linux & Unix back from the mainstream desktop. Most people (like me) don't have the time it takes to become a unix/linux guru. It's not about being lazy or dumb it's about getting a job done as quickly and efficiently as possible. This just isn't the case for most things in linux/Unix. I don't want a relationship with my computer. I want it to be easy to manage and work reliably. Preferably in hours and not days or weeks.

I'm no Microsoft lover but programs like this (and some reasonable documentation) will only help Linux/Unix become more mainstream. For those of us who have better things to do than have relationships with our computers that is. Keep up the good work. And btw, please write some good documentation.

Unix/Linux Newbie

> Use FreeBSD as an example and take it
> from there. This is for people who know
> the insides of their systems and not for
> superficial types who just wan't a
> slam-bang-thank-you-ma'm OS. FreeBSD
> requires a "relationship", a
> commitment, and not just a one night
> stand. This is not for lazy asses.
> The market wants people who know what
> they are doing, not people who know how
> to use a GUI. If you want dancing
> clips, Einstein and doggies, stick to
> XP! Guess that's what all the envy is
> about.

30 Sep 2002 05:42 fremar

Add support for integrity checkers + script generation
I certainly hope you will pull it off to devellop such a project.

And even though you have already burried yourself under an enourmous workload I want to add a request: build in some form of support for integrity checkers (e.g. tripwire). One thing which has kept my from using any of the existing tools like linuxconf or webmin is that I always get some tripwire error messages about changed files. Of course, checking the logs will usually give a clue as to why some configuration file has been altered, but clues are not enough where system integrety and security is concerned, I want an exact listing of the files that have been altered.
This might even be implemented in the form of an additional backend which takes the list of modified files and feeds it to the tool that recalculated the checksums and updates the database; all with proper authentication of course.

Now that I think of it, I can add a 2nd request right now: might it be possible to have gui or web frontends generate a list of commands to have the command-line tools re-create the same change? This would be ideal to automate configuration changes for clusters or groups of similar workstations: configure a service like samba on one of them through the gui and have a script to make the same changes on all the other machines. In my optinion this will be more powerful than a centralized configuration solution since the same script can also be applied to new installations (e.g.a post-install script for a kickstart installation).

30 Sep 2002 10:15 noselasd

GConf
Seems like this article describe GConf. Widely used in Gnome, but not gnome dependant.

30 Sep 2002 17:02 ged

You gotta laugh!

Some people say it is impossible; some say it already exists.

I'll just say this, the article gives the goals:
The system should be easy enough [for] new Linux users.

The system should be flexible and powerful enough [for] advanced users.

I don't think there is any existing solution that meets just those two goals. I don't think meeting them should be impossible (though it will be hard). And I do think doing this would be fantastic for the GNU/Linux community. Best of luck!

Ged

30 Sep 2002 23:45 skiingyac

Re: GConf
We did look at GConf among other projects, hoping that we could build on some existing code. One of GConf's main flaws is that the method for adding support for a new thing to configure involves writing a lot of C code and various other things, which we felt was too cumbersome.

We want to make things simpler, and the architecture for GConf and most of the other tools wasn't flexible enough to accomplish what we're looking to do. Unfortunately (depending on how you look at it), it looks like we'll end up writing lots of new code in order to accomplish this. The upside is that since we're putting tons of effort into planning, soliciting comments, and really thinking this through, the initial release should turn out pretty close to what we're going for.

01 Oct 2002 00:19 skiingyac

Re: Add support for integrity checkers + script generation
Integration with tripwire/aide/etc is certainly not a pressing goal, but it is a great idea. We're planning to have the middle layer log changes in some way(including what files were changed) and there are a lot of good reasons to make optional components available which do other things with config changes (modification detection, copying to multiple boxes, creating a script to get from state A to state B, etc.). To do things like managing users, changing hard drive settings, etc. it may turn out that this type of change to set of commands translation is necessary.

You have some good ideas, and we will try to make it so that even if we aren't able to implement and stabilize all the requested features, it will be relatively easy for people to add them later since our system will be as modular as possible.

01 Oct 2002 10:04 r0b0

Data
Though I'm still quite skeptical about a real need for such a system, I'm going to add a few points to the discussion. And maybe once the succcess of the project will beat my skepticism :)

The functional and architectual model (here and on the project website) look good to me. I just miss a data-design.

What will be the structure of data flowing through the system? You only speak about text fields, integers and such non-structured entities. But in real life and in real configuration schemes, the information in configuration files is more structured.

Look at apache for example. The configuration options like DocumentRoot or ErrorLog can be specified for the whole server, for one virtual server, for one directory etc. Parameters in subdirectories are inherited from their parent directories. Some configuretion options are not allowed or have no meaning for directories and can only be specified globally. Values of some configuration options include other configuration options... I guess you see the point.

Another look at the data: I propose the low-level data entity should be a chunk of a diff file to the orignal configuration file. This approach has proved to fit for the need of editing files by various players - the user editing the configuration file directly, your system or maybe even some other configuration system used on the machine. The main pros - consistency (it's harder to damage a file with a diff than by rewriting it from scratch), auditing (when you log a diff-file, then you can easily see who did what and why).

And by precisely defining data-structure, you can precisely define interfaces. A great pro for a modular system like yours.

Luckily, XML is very flexible for use with structured data, so there should be no problem with the implementation.

Hope this brings something for you,

R.

01 Oct 2002 15:51 skiingyac

Re: Data
The actual XML data is organized by nested <section> tags, which accomplishes the necessary grouping. The individual parameter/value pairs are inside one or more levels of <section> tags. Currently, we don't have a parser for apache style config files, but this would be a goal. Even with INI style files (like samba), the same issues rear their head. For example the guest ok directive can be used to globally allow guest access to samba shares, and any sections inherit this default value unless it is explicitly over-ridden. From my experience, apache works in the same way (more or less).

At the very least, we will need to provide a way for the system to be aware of default values, which will be inherited by all sections including the highest or global section. If carefully written, this inheritance could easily be extended & handled as the XML is processed (either to or from the top-level) by the middle layer. It is going to be a tough problem to tackle, but as you mentioned, I think the nature of XML will help make it easier. Properly handling defaults & inheritance will be very important for the project's success since otherwise it won't be very useful for a lot of situations.

As far as using only diffs of the new and current config files, we can currently completely reconstruct the config file *exactly* (byte for byte identical) as it was from our XML representation of the data. If we can do this, and we checked to make sure the file hadn't been changed by one user as another was editing it by hand, we should be ok. Either way, since we can re-create the exact (or slightly modified) config file, producing a diff of the two wouldn't be difficult either, and may be a good option (among others) for logging. Thanks for your input!

02 Oct 2002 01:16 juahonen

Re: Make it easy for lazy people? Nahhh!

> Attitudes like this is what is holding
> Linux & Unix back from the mainstream
> desktop. Most people (like me) don't
> have the time it takes to become a
> unix/linux guru. It's not about being
> lazy or dumb it's about getting a job
> done as quickly and efficiently as
> possible. This just isn't the case for
> most things in linux/Unix. I don't want
> a relationship with my computer. I want
> it to be easy to manage and work
> reliably. Preferably in hours and not
> days or weeks.

That's not the way to do it. Blaming others from "keeping Linux back" just because you don't want to commit to making an "easier" system possible. If you don't have time for Linux, quit using it. It is not for the weak of heart.

And why should Linux be put to the mainstream desktop? Home users have difficulties with even Windows. Who will help them when they're trying to modify their Linux installation? Corporate desktop is another matter. Corporations have system administrators who will do the necessary configuration for you.

But if you fear to throw yourself into a relationship, consider some alternative. Linux needs the commitment -- otherwise how would you manage when you screw up your system. Reinstall? And do all the configuration *again*. You're left to that if you don't know how the system works.

02 Oct 2002 10:19 Avatar sparre

The Dotfile Generator

Have you looked at how the Dotfile Generator attempts
to solve the same problems?

Have you considered how to handle configuration file formats that are close to complete programming languages?

02 Oct 2002 11:44 pythoneer

Worthy Project, but for whom?
Configuring anything (computer, car, bookshelf, etc) can be a difficult and time consuming and may require expertise in the applicable field. However, how often do you reconfigure you Linux box? Obviously this answer varies enourmously.

For the home hobbist, it may be hourly and the convience of remote GUI is a godsend. For others, the inconvience of once a week/month/year going and logging on directly to administer a change is a small price to pay for the lack of daily security headaches.

The issues of configuration and ongoing maintenance should already be solved long before a system is implemented into a "production" environment, in most cases it probably boils down to 'change this line in this file here' as the only modification ever done on an ongoing basis, despite the fact that the machine itself may have several hundred customised/standard configuration files.

I'm lucky enough to use Linux for work and play, but my approach to configuring it for work and play is completely different.

04 Oct 2002 15:00 djnietzsch

Re: Make it easy for lazy people? Nahhh!

This is a really bizarre post. The poster seems to
suggest that there is something noble about using
unnecessarily difficult systems. Things should be exactly
as difficult as they need to be and no more. For
example, command line programs have a standardized
(more or less) way of expressing parameters. You type
the name of the command, some options, and some
arguments. It's basically the same format for every
program. And if you haven't memorized every option,
you can just bring up a manpage that describes all of
the options and details (again in a way that is fairly
uniform from program to program). Some people would
say this is not easy, and sometimes they're right. Some
things get pretty cryptic. But it is straightforward.
There's not much guesswork about format or what to
expect in general. Configuration files seem to be a
different matter. There are as many formats as services
and there aren't a lot of universally applicable
principles. This means that if you want to do some
nontrivial configuration of something you've never
configured before, you need to learn the format and
syntax of the configuration files in question without
much of a starting place. If there were a universal
format/system for creating and editing these files you
wouldn't have to start from scratch and things would
be straightforward. That doesn't make them easy.
Some matters of configuration will still require some
care, subtlety, and knowledge. I would propose that
one should only have to know what program does to
use (use in a broad sense that includes configuration
and whatever else might go into the program) it.
Knowledge of how to use it should follow easily from
universal principles that apply to all programs within
the domain we're considering.
Your relationship with your computer (yikes) should be
a good one. To carry the analogy a step further, your
computer should not be emotionally needy or
enigmatic. It should be straightforward to deal with.
You should not feel like you are a good person for
putting up with it.

12 Oct 2002 10:40 gsobalvarro

Re: Make it easy for lazy people? Nahhh!
Stick to M$ products if they are not too hard for you. You seem to be an end-user so if these work for you don't delve into anything "hard". That way you won't have any commitment to anything. All you are interested in is getting your work done, right? Slam-bang.. et al.
Most Linux and FreeBSD versions are easy to install and will give users something they can work with on a day to day basis from the start. If all you want are tools to get your job done they are there. I just don' see what is hard or difficult about it. Some people never get the hang of using a computer or an office suite. They should stick to pencil & paper and a calculator. Computers are not for everyone.

16 Oct 2002 15:06 kop

Plugins
The definition of what a certain config-file format looks like (referred to in the article as metadata) should (naturally) be stored in separate files for each, so that people could create and distribute these format definitions in a simple way.

I'd say that for a system like this to ever grow popular, it needs to be heavily based on the "plugin view" in general. Creating new plugs/defs should require no awkward object file compilation etc.

On projects like this, BLOAT is alway waiting just around the corner; A good design would enable features such as "History Log" thorough a simple RCS-plugin.

20 Oct 2002 01:33 jordg

API/Library
I believe that for this project to succeed a well defined configuration API/Library needs to be created for as many languages as possible. For a start we would target:
C
C++
bash and other shells
perl
php
java
KDE
Gnome
tk
and any other languages we can think of. Then one by one in cooperation with each project bring them over to the config libraries. I admit that this may be asking a lot from some projects to cooperate but I believe most will. Maybe a translation API for to cater for the odd project that does not wish to use the new API.

It may be necessary to have dual formatted conf files in the interim. One that the legacy program reads and one that uses the API with a config parser to write it out as the legacy config file. This is not desirable but until the config library is ported to that particular program it wiull suffice as an interim measure.

Today when writing a program it is required to write an entire parser to read and verify config files. With the library already built it then becomes a few simple calls to it. Required paramers, dependancies, options etc,etc are all handled easily and consistantly with the new libraries.

The exact format of the configuration files is yet to be decided. Some cry XML here and others RDP there. This must be decided first on the merits and carry on from there. Personally it does not matter to me what is used, but that it is agreed upon soon. I would hope that once a discussion specifically for deciding the file format is completed that we can all move to the next stage. (It really does not matter what we use as long as it will do the job). Anyone follow the unixconfig mailing list. That is where it died. Come on if we can agree on this quicker we can progress quicker.

Once the file format is decided a clear and precise definition for types, dependancies, requirements, etc. needs to be formally documented and agreed upon.

OK, now write the Libraries for each of the languages that wish to be supported. The API should be the same for each language (as it can be). Write reference code and examples for each language with good documentation (Now there's another ....), and testing etc.

We then can target certain projects and cooperate in the rewrite of the configuration sections of code to use the standard libraries that we have provided.

Then dependacies between programs, global variables such as (HOSTNAME) can be accessed by all programs. etc, etc.

This can extend to enterprise management systems.

The GUI bit should then be the easy part. But it is important that the GUI parts are developed along the way in several reference languages so that on release date we have a product.

It would be great to have the configuration system define the GUI components and layout without having to write the code to do it. The easier this is the better. The configuration can define the checkboxes, textfields, comboboxes, tables, help, dependancies, requires, ordering tabs, wizards, etc, etc. There should be little need on the GUI end to write a great deal of code. We should also include a cursor driven configuration system or even a purely text based configurator. It is all in the API for each situation.

This project will need the cooperation and support from many areas of the community. It will require acceptance of decisions that are made on merit. There is definately more than one way to do this. More important is that we use a standard. Maybe in the future some will wonder why we did it this way or that but as long as the standard is adhered to there is no big deal in changing it.

It would be great if members of other projects joined this one and shared there wealth of knowledge. This project is necessary for unix to go ahead.

Lets do it!

22 Oct 2002 13:41 swissroll

Looking forward to this - My 2 cents worth
Anybody still reading this ?? Anyway.

To answer your first question:
Is this something worth striving for? In my opinion, most definitely yes.

A few comments on the spec so far: As is already obvious from some of the feedback, not everybody thinks newbies should be allowed to use Linux. Some appear to suggest the XFree is a complete waste of time .... I'm definatly not one of them.
From my point of view, as a developer, there are some apps which I toy arround with and reconfigure weekly. I know the config files for these inside out. Then there are those apps and daemons which get configured once a year when I feel I need to change something. I definatly need all the help I can get with these. What I mean is: there is no such thing as new user/advanced user. There are just the apps you know, and there are those you don't. Maybe rewording the spec to this effect may help avoid some of the flaming that's been going on (and undoubtedly will carry on). no suggestion at the moment.

As far as features I'd like to see:
- on the GUI end (or for that matter from the command line), built in documentation for all fields.
- on the back end: ability to specify validation info. Ideally this would allow not only simple field validation but also relational validation (a=1 therefore b > 6) !!! That's going to be fun
- ability to export/import subtrees (purpose: to take the config info over to another machine, to be able to play around with certain elements and then restore them to previous state - without afecting any other changes made in the mean time, or even to allow the config section to be posted on a mailing list when asking for help)
- the ability for user to configure their own application preferences (i.e. the dot files which are in the user's home directory) - not sure if this is explicitly included in the draft.

24 Oct 2002 08:58 burple6

Has anyone looked at NetMAX?
Hi, I wanted to mention that there is a commercial product that does a pretty good job at most of the things discussed in this article.
The product is called NetMAX and is a web-based administration frontend to a Red Hat-based distro.
Though some of the things on the list are not currently implemented, I am sure it would be a trivial thing to add them if there was an interest.
Anyway, thought I would mention it.

29 Oct 2002 06:21 nholtz

COAS
Caldera made a start at something like this several years ago with COAS (www.coas.org (www.coas.org)). I think the last time anything was done on this was late 1999. Might be worth looking at, as there
are some technical notes (and code) there. Also might be useful to find out why it didn't fly (I suspect it was the competition from linuxconf and webmin as much as anything).

Interested me as Python was to be the source language, which sounds like a good idea.

29 Oct 2002 07:14 mdekkers

Re: Webmin

> There are some tools that are good at a
> lot of things. Webmin is one of them.
> Webmin has some flaws which detract from
> its usefulness to some extent. First,
> it is only web-based, which although
> that is probably the best interface to
> pick if you're going to focus on a
> single one, picking only one can also be
> bad.
>
> Second, webmin requires a module be
> written for each new software, and that
> this module be added by the user. Our
> system will (ideally) handle this
> automatically as version-appropriate
> files will be included with each
> software package.
>
> Third, webmin doesn't make it terribly
> easy to allow people to write new
> modules. There are some things built
> in, but not nearly enough. There's
> little reason why someone should have to
> write more than a basic text file to
> explain how a piece of software (or part
> of the system) is configured. We want
> our system to be easy and
> straightforward to use for everyone
> involved, without keeping people too far
> from the details. For example, we may
> want to display someplace in most of our
> interfaces a note explaining what actual
> config file the user is editing should
> they want to edit it by hand.
>
> Our system is specific to the task of
> configuration, but it will be general
> enough so that it won't need to be
> re-written to add support for new
> things. The process will be very
> simple, and may even be mostly
> automatic.

Hi,

I agree with what you say, mostly, but what I don't think there is anything within the webmin architecture that precludes you from implementing the stuff you miss.

As I understand it, the webmin engine does most of what you need, and it would make a lot of sense for your considerable talents to be joined with the webmin krewe, rather then starting (yet another) configuration project.

Another thing I missed is how this system can be used to manage large, distributed networks, and what facilities there are to make this work.

03 Nov 2002 16:07 hansvb

importance of a standardised data structure

Introduction

This doc is a reply on freshmeat.net/articles.... Please note that i recognize and agree on the concerns that you have on the linux way of configuration.

In short i would like to express the importance of a standardised data structure. Also i think it is very impotant to contantly remind program devellopers to avoid unneeded complexity.

I hope you understand my point below and find it a positive contribution.

General

In my percetion a configuration consists of three things:
- A data structue

- and a user interface

- A application interface

The data structure
The structure

About the data structure: We've seen two way's of not doing things, the windows registry and the linux way. Why are these a screw-up? Because the lack of a standardised structure. If you have a standardised data structure then your'e halfway, and, it doesn't seem that hard to do. Also it is not really importand how you do it, as long as it is standardised everybody can find whatever they need.

Let me explain this. First you design a structure. The data has to be seperated into levels. f.i.

Root, Vendor, Application, Data Group, Data Item, Value.

By defining all elements of the data structure, app designers don't need to "invent" or interpret standards for them selves.

The implementation

As soon as the data structure is clear you can start thinking about the implementation of the data structure. (Design and implementation are relativily independend). The data structure can be implemented using several techniques:

- Structured Text (Linux conf, Windows ini)

- XML

- Database

The internal structure should be standised as detailed as possible and have freely available access tools.

The database implementation f.i. could be mysql, a lichtweight, reliable and fast database. Discourage the use of "a different kind of database for each app", this really sucks. Even if mysql is to big for you, it will not be to big if you add-up a few apps. Other databases may be just as good. Important is that all apps use the same one whenever possible.

Mixing implementation techniques

It is very well possible to mix these techniques, however, a few things must be kept in mind:

- The data structure may not be violated

- The mapping from one implementation method to another must be standardised (Documentend)

- Mixing implementation methods may make things more complex.

- If data exists on multiple locations there must be a standardised way to keep data consistant

Examples

An example: A vendor called "AVendor" creates a program called "XApp". He could choose to do the following:

/etc/AVendor/XApp/XApp.conf

/etc/AVendor/shared/X.conf

/home/bob/.etc/AVendor/XApp/XApp.sav

or

/etc/AVendor/XApp.xml

/etc/AVendor/shared.xml

/home/bob/.etc/AVendor/XApp.xml

Suppose a certain vendor has a complex suite of applications and wishes to use a database. Then you could get:

/etc/AVendor.database

/home/bob/.etc/AVendor/XApp.xml

Vendor?

Vendor is used to be able to group certain applications. Examples of vendors are:

- redhat

- linux (or system)

- mandrake

- gnu

- myapps

A pitfall in choosing a vendor name is that f.i. redhad would put every app of his distribution in /etc/redhat. If someone later on dowloads an app then it would go in /etc/gnu. So i think that the data location should be determined by the app writer and not changed by a certain distribution.

Only proper standardisation and documentation can prevent misinterpretation of the data structure design.

OS dependent data

What to do with os dependent settings. It might be usefull to give guidelines for this. Maybe keep the data separate, use standardised notation and translation.

The user interface

The user interface is a complex topic. Normally i would not say that the userinterface should be a representation of the data structure. In this case i think we should try to let the two resemble as much a possible.

The configuration file datastructure should be a logical one that is easily understood. Not just a bunch of names and values in random order. In fact, these files are a (simple) user interface, otherwise they would be binary.

What has been said about the data structure is also valid for the user interface. It is not important how you implement it, it is important that the way of using it is standardised. So if you see the data files themselve as the simpelest user interface it is logical that all "higher" user interfaces sould have the same structure but are easyer to use.

The API

Ehh... I don't have anything to say about this so I won't.

Comment on "A Solution to the Problem of Configuration in Linux"

I note that you are clearly focussing on a certain app while my focus is more to standardising the data and way of working. This is an important difference. I think a good app to manage the settings is a good thing but then again you may be overtaken left and right by others such as linuxconf etc. I think it is more important right now to get the basis right. The app you have in mind could be serving as a test to see if the data structure is choosen right.

Also you are focussing on the use of text based config files. This is not in conflict with what i write, I even think this is good whenever this solves the problem, but my view goes further. If someone decdes he needs xml or a database then you've lost him. My proposal encourages to keep following the standard and use the same logics whithin those techniques and makes sure everyone can access this data.

The three tier architecure.

I think it is good to keep this topic in mind but having a widely accepted, standardized data structure is the first thing to worry about.

You directory structure is flatter than what i propose. I think my structure has an advantage towards commercial software vendors. I think these should also enbrace our proposals.

Further, you directory structure uses "deamons". I have learned that unix has only files and processes. A deamon is nothing more than a proces running in the background. Many of them can be executed as a normal app. I would try to match closely to the Unix philosophy and not try to determine a border between programs and deamons like windows does. You might get burned if you do.

Configuration locations are not fixed: Arrrrrrrggh.... Of course they should be. The app should be able to read from other locations but there is a standard location that all Linux distrubutions use. If you deviate from that you sould better have a clear reason for it.

Forms, Wizards

No, I don't agree, the Windows registry is a complete mess that has partially been deliberately been made unreadable. This database has been written with comercial motives. I repeat, this is a screwup because of the lack of clearity towards its users. Of course, remarks in files are a big advantage but you can do that in a database too (the forms you mention) so there is no real point in discussing the implementation technique other than complexity and performance.

Wizards

To me the main task of a wizzard is

- To give comment on each option

- To hide advanced options

- To prevent entering unrealistic values

- To prevent conflics

Please evaluate each wizzard on these rules. After using a wizzard you should get a working situation.

Metadata

The data should be based on standards. Be ware of the microsoft effect. Every dummy is going to add metadata to his setup. Look at what they do with the registry, every stupid program that needs 5 lines of data to init has reg entries and needs to reboot.

We have 5 lines of defence against complexity

Rule 1: Keep everyting as simple as possible, but not simpeler. (That quote's not mine)

Rule 2: Clear data structures

Rule 3: Logical configuration

Rule 4: Documentation

Rule 5: Documentation (Also document the setup data)

You have to have a real complex program to need to use metadata. Use it only when this kind of complexity is needed. Many things can be done with straight forward configuration and documentation.

Access Control

Access Control should be clear. The current linux system puts user accessable data in the home dir and system data in /etc. This is straightforward and simple.

Doing access control within one configuration area or even in one file wil cause more complexity so avoid making it fancy. If you need to go away from the linux file access rights then you end up with a database solution with rule based access. This confilcs with "Everybody should be able to use it" There's no such thing with rule based access controll.

The bottom line

Yes, redo the linux config but start at the basis. Standardise the data, then go to the app layer. Keep in mind that the data usually lives longer than an app.

Be ware of inviting complexity. People tend to use what you offer them. I think there should be a clear statement about avoiding unneeded complexity and a few samples of how to configure simple and less simple programs.

Greetings, Hans van Buitenen

04 Nov 2002 08:28 richardrusso

Some Suggestions: Precursors & Parentheses
I'm coming to this discussion late, sorry if I restate someone else's opinions.

I'd like to point out that there are a lot of other systems with similar goals, some very successful. Take webmin (www.webmin.com), for example. It has a wide range of plugins which add to the list of programs that webmin can configure. Its UI is a webpage, served over https by an internal server. HTML is nice for developing the kinds of UIs needed by this program (forms), so new forms are developed quickly.

Where webmin is practical, the next example is theoretically nice: Emacs M-x customize. Emacs customization groups provide a general, form-based way to configure Emacs extensions. The documentation, including samples, is right there in front of you. I'd say that if a new Emacs user could figure out how to get into the customization groups, there'd be no problem configuring the favorite extensions. This system has some things that you aren't assuming, however, like some standard ways in which all programs read in their configuration (through emacs-lisp variables).

I really respect this project, I think it's a great idea. Check out these systems so that you could possibly incorporate their good points.

btw, I saw in the author info that you were at the Honolulu ICPC? I think I remember seeing Messiah on the roster. I competed with the team from UCF there.

05 Nov 2002 19:51 kurtolsen

Re: Make it easy for lazy people? Nahhh!

> Use FreeBSD as an example and take it
> from there. This is for people who know
> the insides of their systems and not for
> superficial types who just wan't a
> slam-bang-thank-you-ma'm OS. FreeBSD
> requires a "relationship", a
> commitment, and not just a one night
> stand. This is not for lazy asses.
> The market wants people who know what
> they are doing, not people who know how
> to use a GUI. If you want dancing
> clips, Einstein and doggies, stick to
> XP! Guess that's what all the envy is
> about.

I have to disagree, an operating system is a simple thing spawning a million options. As a longtime developer I must say that anything that makes the OS easy to install and configure is the way to go because the majority of folks want to run applications.

13 Nov 2002 15:44 starkfist

a GUI using GNOME or KDE ???????
Or a better idea might be to create the gui in a toolkit like qt or gtk, and not a "desktop enviroment/application suite" like Gnome and KDE.
An even better idea would be to staticly compile a fltk gui.
But if you did that, you would never expirience the fun of not being able to configure your system because of gnome-scrollkeeper errors. Yah, I know if that happens you can still just edit the text file, YUP! thats right. But thats not fair, The Newbie, WinIdiots can't handle that, they might have to read a manual instead of clicking on things until it works.
oh, if you need a gui config. Webmin rocks!

14 Nov 2002 19:14 venomx

Scope of project
while I have to agree that standard configuration is needed in linux, I think that the scope of the project is too broad. IMHO a much better approach is to attempt to create a standard set of tools (library) that handles parameters and values indexed by key. While there are certainly complex configuration formats in applications, I believe that they are the exception rather than the rule.

If a sufficient library were created that offered 80-90% of the needs of programmers it would simplify the configuration of all of those applications that don't require advanced configuration formats. I also believe that a simple key, parameter, value format can be adapted to more advanced files albiet with a little work. This should be the job of the application to handle the various complexities that arise such as parameter contexts and overrides, etc.

The simpler the better. I believe that once a good toolset is in place people will want to use it. Key, parameter, values models data structures accurately. A key item that offers flexibility is the muli-valued parameter using a delimiter. Quoted strings should be used to enable delimiters to be used in a config:

[ KEY ]
param1 = something, "something else, blah,blah", another

Hierarchies can be created using dot notation:

[Global]
param1 = something
param2 = something, "something else, something else,", one more thing

[ Some local key ]
param1 = foo
param2 = bar
param3 = baz

[ Some local key.overrides ]
param1 = something different

Etc. So contexts are created using dot notation rather than braces. Each level in the heirarchy that is associated with a given key is up to the application to handle. After all the config file should just store parameters and values. If we need to handle default values, we can add another section:

[ Some key.defaults ]
param1 = default foo
param2 = default bar
param3 = default baz

Comments can be stored easily and associated with either some key or parameter. This way the documentation is in the config file. If for instance a GUI were to be made, it can simply pull the comments out of the config file and display them.
For similar generic items you can create a key that using numeric increments like for modelines in a xconfig file.

[ Modelines ]
total modes = 4
mode1= ...
mode2= ...
mode3= ...
mode4= ...

The toolset should not need to be aware of this situation, the application should interpret each modeline by
I just don't think the problem requires massive amounts of engineering. This solution is simple, it works and can be adapted. If we find out for some reason that a particular requires some complex format that can't be adapted (I refuse to believe that with enough creativity a config format can't be adapted somehow), then extend the format in an intuitive way to allow more advanced configuration. This way you can get 80-90% coverage for the simple config files and still allow advanced functionality using the same library.

I believe that this is the key to a uniform config system. Once everyone is using the same library, then you can create a generic framework to localize configuration of all these various config files by extending the toolset to include a simple registration system allowing a single app to version configurations, save, backup, alter, use ACL's etc. But the first step is to "market" this flexible toolset to developers.

Start small, why spend massive amounts of time engineering something when this solution is so trivially simple it's possible in such a shorter time frame. Once this phase is accomplished, you can create another toolset (library) that will parse the config files and build a GUI automatically by reading all keys, creating the menu hierarchy from the dot notation, reading all parameters and putthing them into a GUI list control, extracting the comments, and anything with a *.default can be plugged in. Heck you could even create an *.option if you needed in order to dynamically generate all of the possible values for a specific parameter. This can all be done in various languages with ease. A C API could be created, a set of simple C++ classes, perl modules, python modules. Porting something like this is far easier than some 3 tier system with XML and all that jazz.

I personally would rather spend my time on something like this rather than some massive engineering effort to try to unify already existing config files. This is just much simpler than a 3 tier approach. Just my $1.50 here :)

22 Nov 2002 10:33 sankeld

Why this can't work.

The Law of Leaky Abstractions (www.joelonsoftware.com...) applies here. Unix will always be fundamentally limited by its complexity. A system like the one presented is bound to break. The current unix way of doing things is every program has a different kind of configuration file and the format of the file may change between versions. The maintainters of this system must update the versions of their sofware every time one of the things it configures for changes.

In turn, this project would not really help any sysadmin. Anyone who runs djbdns better know the configuration format, anyone who runs qmail better understand the /var/qmail heirchy. The system presented is bound to be leaky so an understanding of the underlying configuration files is required.

When K&R and friends invented UNIX, they invented a language for it called C. Neither C nor the UNIX libraries had special facilities for configuration files, just files in general. If someone was to change this paradigm, the system would no longer be UNIX and it wouldn't be, in any decent way, compatible with UNIX.

So, yes, I think this idea couldn't work in any practicle way. What it does show is a fundamental limitation to UNIX that new operating system designers should take into account.

22 Nov 2002 19:42 AstroDrabb

Re: Make it easy for lazy people? Nahhh!

> Use FreeBSD as an example and take it
> from there. This is for people who know
> the insides of their systems and not for
> superficial types who just wan't a
> slam-bang-thank-you-ma'm OS. FreeBSD
> requires a "relationship", a
> commitment, and not just a one night
> stand. This is not for lazy asses.
> The market wants people who know what
> they are doing, not people who know how
> to use a GUI. If you want dancing
> clips, Einstein and doggies, stick to
> XP! Guess that's what all the envy is
> about.

This is just the wrong attitude. I am a developer and
most of my day is spent working or thinking about
computers. However, most of the world does not
revolve around computers. We are the
minority and they are the majority. A PC is meant to be
a tool. And tools are meant to perform certain tasks.
Let's look at an example of a mechanic. If a mechanic
had two tools on his table and one was great but
needed to have 32 switches turned on while pulling 5
levers while the other tool was ready to go, which tool
would he choose? Even if the one that was ready to go
was less powerful and broke half the time, it still allows
the mechanic to get his job done faster. This is what
people want! Linux has come very far. I want to see it
go all the way. There is no reason that Linux cannot
take the desktop market with some more work.

The Linux community needs to go for the ease of use
features now. Until that happens, the average Joe Blow
will be happy to use the closed source propriety garbage
feed to them. Why, because it allows them to do what
they want without needing to become a geek like you or
me. Since Linux has come so far in such a short time, I
think it is now time to finish it.

Let Linux reign.

22 Nov 2002 20:07 AstroDrabb

API, API, API!
This system sounds great. As a developer, the only way I see this "catching on" is if there is a simple api. The system <b>needs</b> to be applicaiton oriented and not system oriented. I am not fond of the windows registry, however, I do like the fact that it is applicatioin oriented. One application can go in and save it's configuration data without affecting any other program. Most importantly, there is no need for a user to know it is there. If a "power user" wants to get in and tweak, well then let them get in an tweak. There should be no reason for a "normal" home user to have to edit config files, registries, xml files, etc to use a program. When I write software for Linux or windows, I try to make them need no user intervention to setup. Linux needs to take this approach as well. One good selling point for your project will be if you can come up with a simple api, that will be a breeze for a programmer to store their settings in. If a certain apps needs more complicated settings, then allow that to be possible through an api, but don't make it the only api. If I have an app that only needs to store a string, then I want a simple call like<br>
save_string_setting("MyApp", MYVERSION, "key name", "some value");

Then at program startup it should be something as simple as<br>

char *myValue;
myValue = get_string_setting("MyApp", MYVERSION, "key name");

Notice that memory allocation would best be handled by the API and at program termination a call to cleanup() would free any memory. I could live with managing my own memory thou.

Best of luck.

04 Dec 2002 10:37 albertfuller

your solution
you have a great idea and I like much of what you have to say.

I would just like to add my thoughts on the design level: I feel you need to confront the variability of Linux distros and file layout. The question is how you can add an app and a definition of its configuration elements (even with choice lists if possible) in a straight forward way to this system; and at the same time your system could support multiple states based on machine.

It's no good for the user to effectively endup using your tool as a blank scheme where user has to add in new objects (eg., mySpiffProg), in need of configuration, and the definition specifics to get some configuration interface; and then configure the object. At this point, just go straight to configuring the object arbitrarily.

The issue I feel is one of developing a model of distributed knowledge (that's essentially what a Linux-universal config tool would be) where your tool is the implementation of that model.

Now this model must do 2 things: (1) seperate knowledge requirements from utilization and (2) seperate knowledge requirements from knowledge need.

This model can exist as a public online database of configurations, and your tool if it could get/create/poll the current environment to determine all available objects and then query your public db to download the required configurations. Any requests for configurations that the public db did not have would be logged for later resolution.

Possibly users who had a configuration to submit to the public db, could do so through your tool which would then accept item to be validated before going into distribution.

example stop:
I have just installed your-config-tool; it polls my system and determines there are 3000 configurable objects (ok I run SuSE Linux) -- your-config-tool would need to determine which configurable objects are already properly configured and which are not -- then I could download those configs I had selected from the picklist of objects in need of configuration. I would then get those selected configurations loaded in my machine and what was not available in the db was logged by the db for future acquisition. I even get a chance to have you guys poll my system for configs that you are in need of (as in thousands of thousands of people want my my freeware program mySpiffProg dispite its tricky config setup) and/or one I directly submit.

Now, I jsut got me a reved-up bash or emacs configuration along with a nice config for wine, apache and squid (all version relevant of course). Your db uploaded 3 configs (to by polling my system and one by user submission - I just want to get mySpiffProg out there!)

********

Ok, you see the point I am trying to make. I see the issue essentially one of information distribution in a distributed community where users can add to the pool of configurations as well as get configurations a la carte... This is what I think addresses your idea of "universal configuration." "IT'S NOT JUST A TOOL, BUT A WAY TO linux-BLISS)

I have posed my take on the issue at the widest level, to see what the basic interaction could be like given your "universalization" on the config theme

have fun
and good luck

05 Dec 2002 16:55 smithdev

Re: Why this can't work.
I agree with what you say. I was going to take a more database-ish stance, and point out that what we need to do here is protect the information from the administrator and the administrator from the information.

It might be fun to write a Postgres database that knows how to register applications, their configuration file locations, and abstracts their formats to the point you could hit them with SQL (!?!)

Such a hypothetical implementation/maintenance nightmare, while instructive as an exercise, would fail the admin-panic test: you very likely couldn't boot from a floppy in an emergency and get anything useful out of it.

There just aren't any silver bullets. Have we considered the LSB? Perhaps this project could be included in the LSB so that the community will be more likely to migrate towards it.

16 Dec 2002 09:54 webfishrune

it's just a BAD idea.


I hope i'm not paraphrasing other people's
comments too
much, as a lot of valid points that I agree with
have been made but this is purely and simply a
bad idea.

As has already been mentioned the current system
allows greatest flexibility and configurations
that are actually well suited to the application
they partner with. If you want a
pointy-clicky-dont-have-to-use-my-brain
environment the try Windoze (or maybe MacOS?) and
you'll be happy for ever.

Three of the best things about the UNIX/Linux
philosophy are:

1) There is more than one way to do it - AKA the
perl motto. We don't want standarisation - why do
you think a lot of us chose linux in the first
place - because we can do what WE not someone
else wants. Which brings me to point 2:

2) UNIX/Linux allows us not just to customise
WHAT we do but how we do it. By unifiying
configuration and having configurations modified
by a software layer we are potentialy removing
our freedom and control over our systems and the
way they behave.

3) Linux always has been about learning. If you
don't know how to configure something LEARN. The
information is out there, it may be hard at first
but the end result will be a better, more stable
system that fits your needs exactly. We have been
forced by Microsoft and others for years to
accept the way they want things done as "The
Standard". Instead, we should be encouraging
freedom of information, freedom of configuration,
freedom of choice. This is one of the only true
preventive measures to avoid monopolisation of
our software and systems - insist on being able
to do it HOW YOU WANT TO. It's your right.

I also agree with the theories on Leakage, which
effectively make this project almost redundant.

If any of you have tried to use Linuxconf et.al.
you will soon realise they are of little
practical use and make it difficult to recover
your configuration by hand should the program
screw up. In fact, if you edit a file by hand
in-between admin sessions, you can get into no
end of trouble. In fact, it can generate a lot of
hassle and wasted time just because of the fact
that the user, especially the inexperienced one,
will blindly believe that their configuration
interface is handling everything for them when in
many cases it may well not be doing this
effectively.

Also, this can pose a major security risk.
Despite secuirty measures in place, it is REALLY
unwise to offer effectively root level access via
a web page, an open network port or any type of
front end. I would say this does not fit with
good practice and is not conducive to knowing
what your box is doing all the time - which is
actually a legal requirement in a lot of
countries as you are responsible for your
systems' actions.

Another point that is worth noting is this:
Storing all of your configuration in one place is
bad. If you kill your configuration interface or
your config database, you kill your machine, at
least partially. One of the great things about
unix systems is that software is seperate. Apache
can die without taking out your mailserver.
Joining appplications together in any way is a
bad idea as it causes a single point of failure.
You only have to take a look at the Windows
Registry system and how easily registry and
profile information can get corrupted to prove
this.


Anyway, these are my opinions, for what they are
worth ;).

rune

Webfish Linux Project
webfish-linux.sourcefo...

23 Dec 2002 16:10 forceuser

Re: it's just a BAD idea.

it's not horrible in concept, but i think we should concentrate on something that is both easier to impliment (excuse the spelling), and easier for the smaller machines out there with no room for individual config tools to use. We should concentrate on creating a unified place for ALL configuration to go, and a unified format for All config files. that way, a unified system configurator would be easy to create that could expand for everything that has configuration simply by look ing in the directory. this seems impossible, but look at this:
you have a format, say
[variable name] = [set number/option] -- [special extension or whatever]

hint = [whatever you want a configurator to say about the input]

then you could easily make a simple graphical configurator that gives you a variable, a box to set it, and notes on what to set it to.it would be simple, universal, and easy to manipulate from text, not to mention liteweight.

just my opinion, but hey. -The BubbbleMeister

27 Dec 2002 23:18 ccat

Re: Why this can't work.

> The Law of Leaky Abstractions applies
> here. Unix will always be fundamentally
> limited by its complexity. A system
> like the one presented is bound to
> break. The current unix way of doing
> things is every program has a different
> kind of configuration file and the
> format of the file may change between
> versions. The maintainters of this
> system must update the versions of their
> sofware every time one of the things it
> configures for changes.
>

I read the URL and I can't see any connection between the examples of Leaky Abstractions. Except "the world is not as perfect as you thought, oh well...". Anyway, I have to point that runlevels and init scripts are optional. They are the good way for a sysadmin will manage his server, but on the other hand linux can be as simple as AUTOEXEC.BAT CONFIG.SYS. And even simpler.

There is no complexity in UNIX. Again: UNIX is terribly simple. The current configuration systems are hacks by sysadmins.

30 Dec 2002 04:40 ptb

Remote vs. local
Remote configuration is a different problem from local configuration. I suggest you tackle the two separately. If a remote configuration tool has access to your remote middle-layers it may eventually be able to help the admin out, but first put the middle layers in place on the boxes, then worry about accessing them remotely.

01 Jan 2003 17:54 mccabec

Making an app work in the config framework is the app developers problem.
Many of these comments seem to revolve around the difficulty of making the config framework fit k's of different file formats or what happens when the formats change between versions, etc. The developer of the application needs to give their app the ability to be managed by the framework. So, Apache says, "Our config can be managed by your-config-app." You open your-config-app and point to Apache's config file, which is self-defining, and the hierarchy of Apache options opens in your-config-app. No need for standard config file locations and the "leakage" is the responsibility of the developer.

Just one thing to which I can't help but respond specifically: The specifications of this project explicitly called for the retainment of native config file formats. None of the beauty of *nix's configurability is lost; it's simply exposed.

07 Jan 2003 11:08 McLaurin

Re: Work smarter, not harder
I like the idea of XML for this a lot, and it does look as if the way for Linux to displace Windoze is to make configuration a little easier. I don't think there's any hope for imposing or extracting standards upon or from new projects, but if a good basic compliance framework were defined, we might persuade developers to comply, or even to state their config data in the framework, as an addition, and of course to make use of it if supplied. There's a bunch of suitable stuff embedded, IMHO, in the standard Red Hat rpm approach.

17 Jan 2003 07:44 adembaba

Re: it's just a BAD idea.

> I hope i'm not paraphrasing other people's comments too much, as a lot of
> valid points that I agree with have been made but this is purely and simply
> a bad idea.

Is it, really?

> As has already been mentioned the current system allows greatest flexibility
> and configurations that are actually well suited to the application they
> partner with. If you want a pointy-clicky-dont-have-to-use-my-brain

This is the kind of mindless bigotry that drives me up the wall..
If you think you have all the time to *learn* (as if the knowledge you
gain from someone's half-baked config solution can be called that),
go ahead; learn all that essentially useless stuff. But, please don't
even attempt to tell us that it is anything but a never-ending workaround.

A config file is a config file is a config file. It is supposed to be
something passive and comprehensible. Not a compilable script (if you
need a script, put it in the compiled code for gods sake), or a symbol
soup.

If you want to hold on to your freedom, be my guest: Do it in the
source code.

And, what leakage are you talking about?

If you are not supposed to edit config file in between admin sessions
you will not, that's it. Either the config software will not let you do
it, or it will at least warn you. What have better than that at the
moment?

About root level access? I believe you can Telnet over SSH etc (as secure
as that is), why can't you think of a similar solution?

And, finally, about storing your config files in one location... I see,
I have finally found someone who mirrors his/her every single byte at
a remote location --not just RAID50 :-)

> Anyway, these are my opinions, for what they are worth ;).

You are definitely not part of the solution ;-)

Cheers,
Adem

21 Jan 2003 04:45 stevenwise

Standardized Configuration. Since when is Linux supposed to be standard?
I remember once (it would seem not long ago) reading a very good article by Gual Duval of Mandrake about the cons of standardizing one GUI for Linux, like deja-vu reading this article brought me right back to that day like it was yesterday. Since it's infancy over a decade ago GNU/Linux has always been about 2 things Freedom and Flexibility. With a past being as such, it's no wonder any discussion of having a standard would be such a hot topic. The problem is in the several pages of Technical Jargon nowhere does it mention the wants of the user. The fact of the matter is the typical Linux user wants just about as much "Standardization" as old Bill wants to open up to the wonderful world of GNU soft. Big or small Linux will be what it always has been, mainly a community of disgruntled Windoze users who are tierd of being told how to do it. So Why, Oh Why after all of these years would we turn back the clock and sell out to a standard? To attract a few more of Bill's gurus that barely know the difference between the terms 'point' and 'click'? Anyone that thinks standardization is the secret to Windoze success has spent the last about 5 years looking through a one sided pane of glass, but for those of us that have actually followed the world of software know it is a sheer combination of unfair domination and clever marketing. With Linux (intentionally) missing these concepts it is hard to see how standardization will do little more than upset allready dedicated users. Where are we to ensure freedom in Linux if we don't do it right at the source?? Ask yourself this question, How does one sail freely down a river once it's source has been capped by a huge dam? This is essentially the same thing, is it not?
As for a standard in programming let us take a look at DirectX, a fine example of standardization if there ever was one. Once upon a time there were three ways to do things when it came to Windoze game programming, the right way, the wrong way and the ingenious way. Nowdays it's hard to find a game that doesn't make use of this handy little tool. While on the surface, this may seem like a easy convenient way to do things, this half-hazard way of marching all our young ambitious programmers down this same narrow worn path is dangerous. It is not only a matter of freedom, but this tunnelvision way of buisness stunts advancement and development of other potentially better ideas. Is this the path we really wish to set Linux down???
If anyone wants to devote their time and skill to a program to simplify Linux development or tools to help the new enthusiastic Linux user this would certainly be a worthwhile if not welcome project, but do me and my Linux box a favor, Let's leave the "Standardization" for the Windoze gurus.

02 Feb 2003 16:27 kdan

Re: it's just a BAD idea.
Re: Leakage, I believe he's talking about Leaky
Abstractions, a very useful concept to
understand why a config adminsitration layers
won't solve all your headaches (though it might
ease some of them).


Daniel

14 Feb 2003 22:04 arschlesinger

The best idea for Linux since the inception of the Kernel
If this framework is implemented so that only open standards are used, particularly XML, then the project should grow fast. If people can distribute their programs for Linux, instead of for the Red Hat distros and for Slack(which almost never happens) and for Debian, then the Linux community can still be as diverse as it is now, but a VERY important standard will exist, also. The ability to easlily install software on Linux and configure services, just like on Windows is probably the most important issue and problem for Linux, and the largest drawback for users to move from Windows to Linux.
The reason Windows is so successful is that people can use the GUI programs(all in the "control panel") to configure everything that they probably will ever need to. If they do need to do something that they can't in the control panel, then they can go into the Registry and do it. The problem with Windows(that myself, as well as every Linux user hates) is that if, while you are configuring something in the control panel, you make a mistake, you had better have made a backup because Windows will crash. Basically Windows is easy but not reliable. I think that I can safely say that even the average user wants both these days(especially because he/she is sick of crashes, restarts, and the blus screen.)
The proposal here is to create a registry for Linux, which I think should be a standard filestructure for all Linux Distros. The difference between the Windows Registry and Linux one is that nothing could be done to this "registry" to crash the system because in Linux, everything is modular. This goes back to the reason why it is so hard to use.

summary:
the reason Linux is hard to use is because it is highly modular; so much so that it differs across any distro. The modularity scares many users because their newly installed program could be anywhere in the huge, hard to understand filesystem of /usr and /etc. What average user is going to know what is in the /etc folder? This project will effectively bring the modularity(which is a 100% benefit to linux and in no way a drawback to it) and inherent stability of Linux under a much needed and (hopefully) open standard. This standard will bring consistency across every linux, and will help win (hopefully lots of) converts over from Windows.

Basically, a user can do everything on Linux that they can on Windows (except play some awesome games, but that's why we all have PS2!) if not better than they can on Windows...but how are they gonna get these dream programs, and better yet, how are they gonna figure out how they work?

The only thing that linux is missing today is a reliable and standard control panel. From there, people will start to appreciate the uniqueness of every distro of Linux.

-Aaron

11 Apr 2003 08:21 skiingyac

PLEASE! READ THIS BEFORE POSTING
As one of the developers of the project, I'd like to clarify a few things that, judging by the comments so far, some people still seem to be grossly misunderstanding what our project is.

First off, our system is in an alpha stage at this point and we have proof-of-concept applications working very well with it, you should check them out at config4gnu.sf.net (config4gnu.sf.net)to get a better idea of what we've got.

Now to clear up some misconceptions:

Our system is an abstraction layer, *just like other config tools*. It is not a replacement for native config files, it does not require apps to use it, and it certainly does not intend to force a single way of doing things. It is just a tool that people can use to edit existing, plain text files one day, and the next day to edit them by hand if you wish. The system will provide an XML-based cache of configuration which you can use for various purposes. This XML cache will be kept up to date with the native config files, and NOT the other way aound. The beauty of our system is that you get an XML-based, structured interface to the native config files. Its just an interface, not necessarily a replacement.

Our system currently allows application developers to read their configuration easily using our API, though it does not require it. It is not a registry, it is an interface. If you want to write your own parser or your own file format, go ahead. We'll try to handle your special formats, but we'd like to keep the number of config file formats to a minimum if possible.

We intend the system to be for people who more or less know what they're doing. Someone made a good point that even if you have been a sysadmin for 20 years, there are just some apps that you don't configure often enough to remember all the details needed to hand edit a config file without consulting a fair amount of documentation.

Of course there are reasons why someone who doesn't know squat about unix should not be able to configure an important server in a production environment. Thats up to the boss. If they want to depend on someone being able to use our tool for their business to stay alive, they've got bigger problems.

*More importantly*, what if someone wants to mess with configuration on a *testing* machine? Why should it be so hard for them to learn?

Finally, here is a working example of how you could edit your Samba configuration with a perl script:

#!/usr/bin/perl
use config4gnu;

my $samba = config4gnu::get_configuration_root->resolve_path("/Daemons/Samba Configuration");

my $global_section = $samba->resolve_path(".samba-global");

my $wins_setting = $global_section->resolve_path(".wins_server");

print "Your samba's wins server is " . $wins_setting->get_string_value;

This example script asks the config4gnu library to load the samba configuration file (i.e., /etc/samba/smb.conf) into XML, locate specific nodes in it, and retrieve the output. Modifying it is just as easy, and the change is saved back in the native config file without changing ANYTHING else (whitespace, comments, blank lines, etc.).

Screenshot

Project Spotlight

Octopussy

A Perl/XML log analyzer, alerter, and reporter.

Screenshot

Project Spotlight

Aspose.Pdf for Java

A Java component to create PDF documents.