Sanzang is a compact and simple cross-platform machine translation system. It is especially useful for translating from the CJK languages (Chinese, Japanese, and Korean), and it is very suitable for working with ancient and otherwise difficult texts. Unlike most other machine translation systems, Sanzang is small and approachable. Any user can develop his or her own translation rules, and these rules are simply stored in a text file and applied at runtime.
nyu is a combination of modern academic approaches to parsing formal grammars from PEGs and expression grammars that represents the new state of the art in parser generators. nyu grammars are written in a powerful language based on PEGs (parsing expression grammars) but with modifications to allow both the AST and the parser to be specified intuitively in a single grammar. nyu outputs parsers that take advantage of the chilon::parser meta-programming library for C++. The generated parsers are almost as concise and readable as the input grammars, yet perform as well as hand-written C code. nyu ASTs are built using tuples, variant types, and lists, and allow self referential parsers and AST nodes to be manipulated. Advanced features such as hashed containers and grammar inheritance are also possible and well tested. nyu is currently powerful enough to deal with complex grammars and bootstraps its own parser.
Multi-byte Keyword Generator extracts meta keywords from multi-byte text. It is an enhanced version of the "Automatic Keyword Generator" class originally written by Ver Pangonilo. This version provides better word segmentation, the ability to handle multi-byte strings, and support for text in multiple languages.
jSmaTeP assists in the use of Java for processing import and export data by configuring a data structure rather than by programming it. The structure of the import data is specified in an XML file. jSmaTeP then generates a value object representing exactly one row or record in the import file based on a given XML data configuration. This means that if the import or export format changes, only the XML data configuration needs to be changed to match it.
xMarkup is a command line and GUI utility for multipurpose processing of a set of text files. It can be used to generate or edit the navigational cross-references within a set of HTML documents, analyze and convert the structure or content of SGML, XML, HTML, or text documents, split or merge text files with specified rules, analyze and extract data, generate scripts, and more. xMarkup supports a built-in procedural language which may be used to describe rules of the processing. This language is a simple dialect of the Icon programming language.
QuHelp is a command-line program that scans a directory with HTML formatted text files and builds a help site using that directory's content and a template. A default template is included, but any template can be specified. QuHelp adds a nice expandable tree view of the help site's contents, a navigator, permalinks, and other useful stuff for browsing the help site.
The Okapi project’s main purpose is to architect a set of building blocks for the creation of larger open source localization and translation tools. But many Okapi components are generic enough to be of interest to the text mining, natural language processing, and text retrieval communities. Okapi’s many text filters (HTML, Properties, XML (ITS XPath-based rules), OpenXML, ODF, Regex etc.) provide a straightforward way to access the text of multiple document formats. Its document events and pipeline can be made to integrate with other frameworks such as UIMA, LingPipe, OpenPipeline, OpenNLP, GATE, and Lucene. The advantage of Okapi’s text filters is that not only is text extracted, but all non-textual formatting is preserved. It is possible to decompose a document into events, process them via the pipeline, and then rebuild the input document without loss. Structural information can be added to Okapi document events so that tables, lists, links, titles etc. are grouped together and treated as a unit. This is useful when context based on a “universal” document structure is needed. The Okapi event model supports user configurable annotations, similar to UIMA, but simpler and more restricted in scope. User can annotate spans of text or add new resources such as translation memory matches, terminology, token types, or part of speech information.