Isobel is a framework to build complex information retrieval and analysis systems. Isobel can be functionally divided in two subsytems, Isobel Gatherer (the crawling and filtering subsystem) and Isobel Analyzer (the analysis subsystem). The two subsytems can also be used separately. Isobel Gatherer offers ready-to-use services like content fetching, scheduling, document format conversion, Hyperlink graph storage and analysis, content storage and indexing. A programmer may easily add new services. Isobel Analyzer uses the IBM UIMA architecture to reuse the analysis components developed for this architecture.
Ojax provides an OAI-PMH harvester to harvest metadata from multiple repositories and builds a single searchable Lucene index of this data. OJAX provides a dynamic user experience enabled by Ajax and full OpenSearch integration with the IE 7 and FireFox 2 search fields. It supports OpenSearch Discovery, OpenSearch Atom results, and OpenSearch Search Suggestions. Loosely coupled Web Services and OpenSearch Atom standard allow integration with a range of VLEs, CMSs, institutional repositories, and portals.
Python Web Graph Generator is a threaded Web graph (Power law random graph) generator. It can generate a synthetic Web graph of about one million nodes in a few minutes on a desktop machine. It supports both directed and undirected graphs. It implements a threaded variant of the RMAT algorithm. A little tweak can produce graphs representing social networks or community networks. It can also output connected components in a graph.