Articles / Making it Easier to Announc…

Making it Easier to Announce New Software

When you want to announce changes in your project to several Web sites, you go to the first one and fill in a form with your new info. Then you go to the next one and do it again. And again with the next one. And again. And again. Doc O'Leary thinks he has a solution. Please read to the bottom for my comments and a couple of questions I'd like to ask everyone.

Introduction

In the (good ol') days when FTP and archie were king, it was fairly simple for developers to spread their offerings far and wide. I had scripts set up to drop the right files in the right locations, and it didn't much matter if there were two or twenty archives.

Enter the Web, and the focus shifts from pushing software out to archives in favor of pulling people into Web sites. I think that's a good thing, because it puts more information into the users' hands, but in the process, developers have lost the ability to easily push anything out. Instead, we have to manually go to a number of tracking sites (the more the better, usually), set up accounts, and edit essentially the same information on all those sites.

My long-winded question is essentially this: Is there any interest in automating this process? I currently have a property list (easily made available in plist or XML format (and simple to convert to other formats, if necessary)) I can use to build the dynamic pages of my site which contains all or nearly all the information that is gathered at various software tracking sites. If a general software description file format can be agreed on, simply making that file available would give sites all the information they need to update their database entries. No fuss, no muss. Minimizing the administrative efforts will really lower the barrier of entry for all sites.

Reference Files

BetterConsole is my newest piece of software, recently released, which has brought this issue to the surface. You can find my SPIF files for it in two formats:

ASCII plist
XML plist

The plist format might not be particularly easy to parse on non-NeXT/Apple systems, so I would be willing to write a converter that puts out a format that is easier to parse.

Discussion of SPIF format

Keep in mind that this is a work in progress and represents only a first pass effort at a format that contains sufficient information to satisfy most tracking software. Most tracking sites seem to consider a basic piece of software to have eight attributes: author, description, version, system, license, price, category, and package.

author

The author attribute identifies who owns the software. It does this via contact information by assigning values to the sub-attributes name (e.g., Joe Programmer) and location (e.g., http://www.someisp.com/~joe). Other sub-attributes such as email could also be added, though most tracking sites currently only seem to ask for the name and URL of the author.

description

The description attribute identifies the software in increasing levels of detail. Currently that is done with 3 sub-attributes: name, short, and long.

version

The version attribute identifies the version of the packaged software. This is done with 2 sub-attributes: revision, and status.

system

The system attribute identifies what operating systems the software is for. This is done with 2 sub-attributes: name, and version. It may be a good idea to make this an array instead, as some software may run on many different systems (the workaround would be to make a SPIF file for each system). It may also be a good idea to add further system requirements (RAM, HD, etc.), but that does not seem to be a major consideration from a tracking point of view.

license

The license attribute identifies the license the software is distributed under. This is done with 2 sub-attributes: name, and location.

purchase

The purchase attribute gives information on purchasing the software. This is done with 2 sub-attributes: price, and location.

category

The category attribute identifies how the software should be organized. In looking at the various tracking sites, there was really no consistency in the arrangement or naming of software categories. Additionally, most sites had additional fields for keyword descriptions of software (for search purposes). I'm hoping these features can be subsumed by this one category attribute. It should be considered a prioritized list of organization and search keywords. The software that scans the file should be able to look at this list and determine where it fits with all the other software that is being tracked. If it fails, I suppose it would be up to the tracker to either adjust the scanning software to be more robust or inform the developer of the error.

package

Leaving the most complicated for last, the package attribute identifies all the files that are associated with this software. Example sub-attributes are info, binary, and source. Each of these identifies a document that is related to this particular piece of software. That is done by further breaking that file information down to location, size, and checksum. I also included contact information, just in case the contact for, say, the binary might be different from the contact for the source, but I'm not sure that's really necessary.

[You can watch the original version of this document at http://www.subsume.com/spif/ for updates about the file format proposal. -- Ed.]


Editor's Comments

Doc makes a good point -- hackers who would rather be coding have to take the time to submit announcements about their work both to general announcement locations (like freshmeat and comp.os.linux.announce) and to locations that match their specific area of interest (gtk.org or linuxgames.com). This is redundant and time-consuming work, and it would be good to reduce it as much as possible. The question is, how?

A pull proposal

As Doc points out, there are two ways of getting your information through -- you can push it, or you can let people pull it. His proposal for letting sites pull the information from you makes it possible that you would only need to keep one piece of information up-to-date at all the sites on which you want your project listed -- the URL of your project info file. If you want your site listed on freshmeat, gnu.org, and kde.org, you could just give them the URL, and, at a regular interval, each site would have a bot check if the file has changed. If it has, it would compare the file with the information in the site's database and submit any differences as change requests to be reviewed by the site's staff. If the regular interval is taking too long, you could click a button on the site to ask it to check your info file immediately. You could even have an "info file URL" field in the info file, and you wouldn't have to go to the sites to keep the URL up-to-date. When you changed servers, you would put your files up at the new location and change the field in the file at the old location. The URL to the new info file would be submitted as a change request like any other. After allowing enough time for everyone to catch up, you could just remove the old files.

One big thing missing from his first draft is an "announcement" field. When you release a new version, you want it to show up on freshmeat's front page. Using Doc's scheme, you should simply be able to change the necessary fields in your info file, including one that lists what's new in this version, ready to appear in an announcement on freshmeat's front page, in the newsletter, on the newsgroups, etc.

This brings us to a problem I see in doing this. Ask any of the freshmeat staff, and they'll tell you that the number of items that are submitted and approved without any changes is quite small. Sometimes, there are errors in spelling and grammar. Sometimes, it's not clear what the contributor is trying to say, and we have to work it out with him or her. Sometimes, there are just changes that have to be made to make the submission match our editorial policies -- for example, we don't allow the name of the project to appear in the short description, and we insist that it appear in the long description (preferably in the first few words), we don't allow HTML in descriptions, etc.

Now, let's say you make changes to your info file, and we pick them up as change requests. What do we do?

Option one:
We make the changes we need to make, and approve the submission. Unfortunately, your info file still contains the original version; in a few hours, our bot will notice the difference and submit your version again. This will quickly drive us insane.
Option two:
We write to you and ask you to change your info file. This will either:
  1. Delay your announcement unreasonably (you'll wake up the next morning and realize your announcement never went up because we sent you a message pointing out that you spelled Linux with a small "l"), or
  2. Make us lower our standards, and let lots of misspellings through rather than make people wait. (<== Not going to happen.)

Even when we get past that point and your info file is acceptable to us, you'll check your mail in an hour and see messages from two other sites saying that they need you to change x and y. When you change x, site number 3 will be unhappy, and when you change y, site number 1 will be unhappy. At this point, you'll wish you were just going to Web pages and filling out forms again.

The issues of what options to include in the file format can be overcome. Everyone who wants to take part in the system can get together and flame each other until they work it out. Dealing with policy issues and the editorial needs of all the sites is not as easy.

You might end up having tag attributes to accommodate different sites:

<announcement site="freshmeat.net">
    (text acceptable to freshmeat.)
</announcement>
<announcement site="linuxdoc.org">
    (text acceptable to the LDP.)
</announcement>

, etc. Whatever the solution, the problem would have to be dealt with. One size is not going to fit all.

A push proposal

Another idea that comes up from time to time is that of letting people submit information by email. Again, you would have a standard format for the information, only now it would be sent to the sites, where a script would parse it and submit the parts of it as change requests as needed.

The advantage to this is that you no longer have multiple sites trying to get you to change your info file to match their needs. They each receive your request and can contact you with any problems they have. You could have your XML info file in your build directory and have a rule in your makefile with a list of the addresses to which it should be sent and a command that will send it. Then:

make submit

I like that just for the coolness factor. :)

Your proposals

I have two questions for everyone:

  1. Think about the fields currently available for a freshmeat appindex entry and those available in Doc's proposal, in LSM files, etc. Do they provide space for all the information you need to give your audience about your project? What needs to be added or changed?
  2. What suggestions do you have for how your information could be more easily pushed or pulled to all the places you want to keep it up-to-date?


Doc O'Leary (droleary@subsume.com) is a COG in the machine of Subsume Technologies, Inc. (http://www.subsume.com/). He is lazy, and has thus been an advocate of free software since 1996 and of object-oriented development for nearly a decade.


T-Shirts and Fame!

We're eager to find people interested in writing editorials on software-related topics. We're flexible on length, style, and topic, so long as you know what you're talking about and back up your opinions with facts. Anyone who writes an editorial gets a freshmeat t-shirt from ThinkGeek in addition to 15 minutes of fame. If you think you'd like to try your hand at it, let jeff.covey@freshmeat.net know what you'd like to write about.

Recent comments

14 Nov 2000 15:38 Avatar evanlanglois

NO ! NO POLLING !! This is better IMHO ...
Polling is of course an evil thing!

A =MUCH= better idea IMHO, is that when you have a release, you take your description file and email it to the sites. Each announcement site would have an email account that just grok'd the file and updated the database. Then the authors only have to add the sites to an email list/group and send it out to all of em at once. This would stop silly polling while allowing the author to send out changes with a simple file-attach.

Also, to make sure we don't fake emails, I think a GPG/PGP signature would be called for. I know some people are gonna fuss if required, but I really think authenticated/signed email is gonna become a necessity anyway, might as well get used to it.

Those that don't like signatures would probably have to go to the site and manually log-in and then upload the description file, still a bit faster than going all over. It would also be nice if some of the major sites allowed the others to request the description files from their database by date and/or type or allow remote SQL searches.

I do like the idea of a standard format description file, and I do like the idea of adding dependency information up front. In fact, when browsing freshmeat, you should be able to click the dependency so you know where to get it if you don't have it.

Anyone else hate downloading something, finding out you need another file, and then going back on the net to find it. Then you download it compile it, install it, go back to what you were trying the first place, and then find another dependancy you need ... ARGH!

13 Nov 2000 04:42 Avatar oliviermacchioni

Trying to fix the mess in those packages...
I see one other piece of information which may be interesting.
Take for example the program MySQL, which has one appindex record in freshmeat (that makes perfect sense as the average user would see MySQL as one single unit, maybe two if you consider the client and the server side).
Check now the related Debian information (Debian is only a sample here, you could probably do the same for RedHat or Mandrake). The files which 'belong' to MySQL in Potato are :
- mysql-gpl-doc
- mysql-manual
- mysql-gpl-client
- mysql-doc
- mysql-server
- mysql-client

For some packages you can have libs, dev-packages, source packages... Which is enough to create a huge confusion in beginner's minds.

We're still talking about packages. Freshmeat has an idea of packages which is close to the user's idea of packages yet distributions have an idea of packages which is more programmer-oriented.

In my point of view it would be *very* interesting to have a link from the concept 'MySQL' to all the distribution packages which correspond to this concept, and even a link to other concepts like 'kmysql-php' which just don't make sense without MySQL. That's more or less the 'dependency' idea applied to packages.

We gotta keep on moving on this, it's an important step in merging the different packaging systems for Linux. It may still be an utopia but it's so important for users.

Am I the only one to think this way ?

13 Nov 2000 00:06 Avatar mrtinkerbell

pole rates
some one mentioned that if a project wasn't updated in a while it would eventualy be abandoned. Another idea was to specify how often to check back in the info file itself. I would sugest making the server figure it out itself. Each time it checks to find no update it can increase the wait interval. This would make it adapt to the update rate and would be nice because often when a project is started it has freqent changes and after it becomes more stable the updates are less fequent.

12 Nov 2000 10:11 Avatar guidod

KISS - keep it simple, stupid
I don't think it's necessary to discuss a standard or
interchangebility of the information. What is instead
a nuisiance is the login process after which
I am presented with the fields I have to change then.


"make announce" sounds cool anyway - to make it work
I do want freshmeat to send back a form sheet of the
latest announcement. The next time I am around to make
a re-announcement, I can simply adapt the fields as
long as the syntax is intuitive. And as long as it
is intuitive, I don't care about a specific
format or conversion tools - just edit the thing for
each site as they want it, as long as I am able to
cut and paste on my local computer. It
would even be sufficent if I would be allowed to
paste the form-information online into a single field.
Still I have to login - an authentication matter.


The authentication would be a bit complex with e-mail
however (I still don't have a pgp-key). Well, no need
for that if you go with the url+pull+trigger idea -
this means that I just setup a website once (which would
need authentication there), and then I push the announcement information in there - just a file anyway - and
simply send a trigger impuls to freshmeat so they look
up the url they have been made to know. And the trigger
impuls is as simple as a "make announcement-call". And
please, no need for micro-emails here - just make it
a special cgi-url at freshmeat to trigger the pull, there
are enough commandline utils that can do http-get.

12 Nov 2000 01:18 Avatar zvi

Copy the Media!
They copy one another. Web sites should also talk to each other...

So you could write a web service to allow a person to enter the necessary information, and then let the server make submissions to all the other news sites! Or, for a pull technology, it could accept "what has changed since date x" requests from participating free software sites. This would result in much less overall bandwidth being used.

Screenshot

Project Spotlight

Kigo Video Converter Ultimate for Mac

A tool for converting and editing videos.

Screenshot

Project Spotlight

Kid3

An efficient tagger for MP3, Ogg/Vorbis, and FLAC files.