Articles / Lessons in Packaging Linux …

Lessons in Packaging Linux Applications

This article records our experiences with packaging an application for many distributions and shows areas in which packagers, Linux distributions, and developers can improve coordination for better and easier distribution. We look at communication problems, packaging errors, package dependencies, menu entries, and bug tracking systems.

On packaging

For Free Software, more users means more potential developers. With tvtime, we made an effort to ensure high quality packages for all major Linux distributions. This article describes some of our findings when packaging for ALT Linux, Debian GNU/Linux, Gentoo Linux, Mandrake Linux, Red Hat Linux, Slackware Linux, and SuSE Linux.

We present some of the lessons we learned and some of our assumptions that we found to be false. We believe our project is typical in that it is fairly young and unknown, and so we believe that many other developers and packagers may benefit from these notes.

Packager-developer communication is poor

We originally thought that finding packagers for our application would be difficult. However, we soon discovered that packages were being made, but we were rarely notified. While some of these packagers had just built RPMs for their own personal use, we found that many had made packages which they provided to the community or contributed to their distributions. Our Mandrake packager explained that he never contacted us because he never had any problems with our package.

Our advice to application developers:

  1. Routinely use Google to search for your own application. You will likely find packages made by users.
  2. Check the contents of packages. We mostly use Debian, but have become quite familiar with tools such as rpm2cpio, as well as the .spec and .ebuild file formats. Make sure that no files are missing.
  3. The "packager" tag in RPM files has different semantics depending on the distribution. Be sure to look in the spec file and see who the maintainer is, not just the person who touched it last. The email addresses listed in the changelog should be trusted over any other tags.
  4. If you find a package that looks official, try looking for it on the distribution QA site. This may give more detailed contact and bug reporting information. Both Debian and Mandrake have QA sites.

For packagers:

  1. We strongly recommend that you include your email address somewhere in the package. This is especially awkward in the case of Slackware packages or Gentoo ebuilds which are often missing this information. Developers may wish to contact you to help fix errors in your packaging.

The lack of email messages or other notifications from packagers was by far our most surprising observation, and we believe that this issue deserves more awareness. While many packagers have assured us that they usually do send an email to the developers, we hope that developers will make package searching a part of their own QA process.

References:

  1. http://qa.debian.org/
  2. http://qa.mandrakesoft.com/

Errors in packaging

We have learned to watch for some specific errors in the packages during a new release of our software. We believe that these issues are usually noticed more easily by the developer than the packager. Developers should:

  1. Check for missing documentation. GNU autoconf provides no standard location for HTML documentation or example files, so packagers must install this separately. New documentation files can easily be missed.
  2. Check the package description and URLs. If these have been updated to mention new features, ensure that the package information matches.
  3. Watch for out-of-date dependency information. Packagers tended to find new dependencies, but forgot to check for removed ones.

Dependency problems

A problem for packaging any program is how to handle non-standard dependencies. There were three main problems we encountered:

  1. If you depend on library packages not in the standard installation of a major distribution, distribute the packages for those from your Web site.
  2. If the dependency is optional, some packages may be built without it. Make sure that both you and your users know about crippled packages.
  3. Watch for incompatible library packages. For us, it was lirc, the Linux Infrared Remote Control library. It seems there are incompatible packages of the lirc client library being distributed for Red Hat.

Clearly, there is never an ideal solution for these sorts of problems, and it's hard to give advice about what to do with them. We do not believe that the responsibility for these sorts of difficulties can lie completely with the packager. We believe that applications themselves can often be improved by having runtime (vs. compiletime) options and even by avoiding dependencies that are problematic for users. However, in order to make these judgements appropriately, feedback is required from packagers on multiple distributions.

Menu entries and .desktop files

The most common thing that packagers of tvtime added to our packages was a menu entry or .desktop file. We discovered that by consolidating the combined input from our packagers, we could help ensure that our menu entries had a similar look and feel across distributions and menu systems.

The .desktop file format

The desktop entry file format standard is used by both GNOME and KDE for their menu entries. However, installation locations and conventions for both the .desktop files and associated icon files have not been standardized until recently. freedesktop.org is leading this effort with standards for menu entries and how applications can locate them. This standard is used in KDE 3.2 and GNOME 2, but the rollout of these systems is not yet complete.

We found that understanding what standards exist and how we should use them was quite challenging. Where should the .desktop files be placed by a .spec file or Debian installation script, do we need to install them separately for a GNOME or KDE setup, and how is this affected by user upgrades? We lacked any Best Current Practices information from desktop environments or distributions.

Here is a list of where we decided to place our .desktop files and icons for installation on modern distributions. We don't believe this list is sufficient and would like to expand it to include more common cases.

  • $datadir/applications/ for our .desktop file and $datadir/icons/hicolor/size/apps/ for our icons. This follows the freedesktop standard.
  • $datadir/pixmaps for our icons. This fixes icons on older versions of GNOME and on Mandrake.
  • On SuSE, we link to our .desktop file from /opt/kde3/share/applnk/ and include our icons in /opt/kde3/share/icons/.
  • On Debian, we link to our .desktop file from /usr/share/applnk/.

References:

  1. http://www.freedesktop.org/standards/desktop-entry-spec/
  2. http://www.freedesktop.org/standards/menu-spec/

The Debian menu system

Debian created its own menu format for applications which predates the freedesktop work. This menu format is also used by Mandrake, but it is slightly modified. ALT Linux uses the same modified system from Mandrake.

The format is simple, but less powerful than .desktop files. The most notable disadvantage of this format is the lack of internationalized names. Our entry for Debian is:

  ?package(tvtime):needs=x11 section=Apps/Viewers \
    title="tvtime" command="/usr/bin/tvtime" \
    icon="/usr/share/pixmaps/tvtime.xpm"

For both Mandrake and ALT Linux, we use:

  ?package(tvtime):needs="X11" section="Multimedia/Video" \
    title="tvtime" command="/usr/bin/tvtime" icon="tvtime.png" \
    longtitle="High quality television application"

The section titles are completely different between Debian and Mandrake. For our application, a TV viewer, the Mandrake sections are much more descriptive. Also, it seems that Debian requires icons in the xpm format.

Since the freedesktop initiative is so promising, these legacy menu systems may soon disappear, so we recommend that maintaining these menu entries be done by the Mandrake and Debian packagers. However, we strongly recommend that the upstream ensure that both the Mandrake and Debian menu entries are as close to each other as possible.

References for these files from Mandrake and Debian are here:

  1. http://www.linux-mandrake.com/en/howtos/mdk-rpm/advanced.html
  2. http://www.linux-mandrake.com/en/howtos/mdk-rpm/mdk-menu-structure.html
  3. http://www.debian.org/doc/packaging-manuals/menu.html/
  4. http://www.debian.org/doc/packaging-manuals/menu-policy/

Hearing about bugs and dealing with them

Probably the most difficult problem for users and developers alike is the handling of bug reports. This includes both bugs in the application and in the packaging. We found only Debian to provide us a method of automatically receiving all bug reports for our application (through http://packages.qa.debian.org/ and the package tracking system). However, Red Hat does allow you to subscribe by maintainer.

Many believe that the distribution QA infrastructure should provide the first level of tech support in all cases, and users should always be instructed to seek support first from their distributions. We believe this is irresponsible. Most packagers and distribution QA teams, like developers, are volunteers with time constraints, and application developers need to be aware of packagers and their packages since they will likely have to help support them.

We would like to see packagers inform developers about the user experience of their packages. At minimum, we would like to know:

  1. How do users install your package in their distribution?
  2. Where is the documentation installed? Debian uses /usr/share/doc/tvtime/, Mandrake uses /usr/share/doc/tvtime-0.9.10/, and Official Red Hat packages use /usr/share/doc/tvtime-0.9.10-5/, replacing 5 with the release number of the package.
  3. Is there someplace users should go to report bugs in your package?
  4. How can we subscribe to be notified of bugs?

This information can go a long way towards helping developers provide better support for their applications.

References:

  1. http://packages.qa.debian.org/
  2. http://www.debian.org/doc/developers-reference/ch-resources.en.html#s-pkg-tracking-system
  3. https://bugzilla.redhat.com/bugzilla/userprefs.cgi?tab=email

Conclusion

We have presented some of our experiences in packaging our application and consolidating our user experience across many Linux distributions. We hope this will help packagers, distributors, and other developers be more aware of the problems and possible solutions for distributing Linux software effectively.

We also believe that packagers can do more to tell developers what might make their jobs easier. We found that through a small amount of coordination, we could consolidate some common additions done by different packages, reducing work for the packagers. We believe that this significantly reduces the number of distribution-specific bugs.

Acknowledgements

Our thanks go to Oliver Baltzer, Mike Harris, Sam Hocevar, Max Kalika, Simon Law, David Lehn, Guillaume Rousse, and Matt Zimmerman for their comments on this document.

Recent comments

12 Dec 2004 23:11 Avatar psypete

Re: Dependancy Issues


> A lot of dependancy issues will

> magically go away when you compile your

> binaries as static.

>

> Sure, your binary file will become

> large, but not significant enough for a

> user to decide not to use your package.

>

> Have any of you had experiences with

> this kind of thing?

Yes. For one thing, if a security hole or bug is found in a library used by the application, you must recompile the library AND the application to fix it. It also presents complications in having different applications compiled with different versions of libraries (Application A depends on Application B, but Application B was compiled with earlier versions of Libraries A & B, so it does not support the operation Application A needs, so you must upgrade Libraries A & B and Application B, ...). Not to mention those bigger binaries take up more RAM unlike the dynamically-linked libraries which are only loaded into RAM once for multiple apps.

Statically linking your apps basically works against all the reasons they thought up dynamic linking in the first place. Don't do it.

23 Jan 2004 03:17 Avatar satryoko

Re: Lessons in Packaging Linux Applications
as you know...my favorite distro is redhat. may be it is sound subjective because i've used it since 1998 and it's very user friedly. Package is very interesting thing, because for me, choosing an approriate OS, sometime can be see when we do the installing. I think, if the distro make the package simpler and easier, alotof people will choose it. And that what microsoft do the best. :P. if linux can make it, wow, the great OS with Great installer, it will make linux really unbreakable.

the only barriers linux and the user is, linux is very difficult. that's we have to end it now.


>
> % As a side note, one of the main
> reasons
> % my favorite distro (Slackware) is my
> % favorite distro is because building
> % packages seems so much simpler and
> % easier than other distributions I've
> % used. I hope that package creation
> can
> % be kept reasonably simple.
> % [...]
> % Creating packages should not become
> some
> % arcane art that only a 'priesthood of
> % packagers' can accomplish properly.
> It
> % needs to be something anyone can do,
> % even if their package is only for
> their
> % own particular system.
>
> Have you seen
> http://checkinstall.izto.org? I use it
> to create my own debian
> packages: ./configure; make; make
> install
> and you have a .deb package. The nice
> thing is you
> can add dependencies to them, and last
> time I
> upgraded with apt-get upgrade, my
> custom
> made packages were replaced correctly
> with new
> binary versions that were available.
>
> Really fantastic. Do get the development
> version
> for dependencies and other advanced uses
> which
> still have not made it to the stable
> branch.


04 Dec 2003 11:54 Avatar sankeld

Dependancy Issues
A lot of dependancy issues will magically go away when you compile your binaries as static.

Sure, your binary file will become large, but not significant enough for a user to decide not to use your package.

Have any of you had experiences with this kind of thing?

23 Sep 2003 15:48 Avatar nightwriter

Re: DeMorgan to the rescue?

>
> % We have the ability to do a logical
> and,
> % greater than, less than and exclusion
> % but why the or is missing I don't
> know.
> % (and yes I've written to the rpm.org
> % people.)
>
>
> First, I assume you mean logical NOT
> when you say exclusion.
>
> If you can do AND and NOT, why can't you
> simulate OR?
>
> A OR B = NOT ( ( NOT A ) AND ( NOT B )
> )
>
> It certainly doesn't look as clean, but
> it is logically equivalent.
>
> Of course, I don't know anything about
> RPM .spec files. Does exclusion mean
> NOT? Is it possible to string together
> NOTs and ANDs? If yes to both, then you
> should be able to recreate the above
> logic.


Actually I really did mean exclusion. Or perhaps a better choice would have been conflict creation. In a spec file I can say that if X is installed Y cannot be installed at the same time. For example if proftpd is installed wu-ftpd could not be installed without first removing proftpd. The logical not is really an implied not in that by using the exclusion characteristic or the > and

23 Sep 2003 12:05 Avatar robolange

DeMorgan to the rescue?

> We have the ability to do a logical and,
> greater than, less than and exclusion
> but why the or is missing I don't know.
> (and yes I've written to the rpm.org
> people.)


First, I assume you mean logical NOT when you say exclusion.

If you can do AND and NOT, why can't you simulate OR?

A OR B = NOT ( ( NOT A ) AND ( NOT B ) )

It certainly doesn't look as clean, but it is logically equivalent.

Of course, I don't know anything about RPM .spec files. Does exclusion mean NOT? Is it possible to string together NOTs and ANDs? If yes to both, then you should be able to recreate the above logic.

Screenshot

Project Spotlight

Kigo Video Converter Ultimate for Mac

A tool for converting and editing videos.

Screenshot

Project Spotlight

Kid3

An efficient tagger for MP3, Ogg/Vorbis, and FLAC files.