EXCLAIM: Difference between revisions

Content deleted Content added

Inline

Revision as of 21:48, 18 April 2008

The EXtensible Cross-Linguistic Automatic Information Machine (EXCLAIM) is an integrated tool for cross-language information retrieval (CLIR), created at the University of California, Santa Cruz in early 2006. It is currently in a beta stage of development, with some support for more than a dozen languages. The lead developers are Justin Nuger and Jesse Saba Kirchner.

Early work on CLIR depended on manually constructed parallel corpora for each pair of languages. This method is labor-intensive compared to parallel corpora created automatically. A more efficient way of finding data to train a CLIR system is to use matching pages on the web which are written in different languages^[1].

EXCLAIM capitalizes on the idea of latent parallel corpora on the web by automating the alignment of such corpora in various domains. The most significant of these is Wikipedia itself, which includes articles in 250 languages. The role of EXCLAIM is to use semantics and linguistic analytic tools to align the information in these wikipedias so that they can be treated as parallel corpora. EXCLAIM is also extensible to incorporate information from many other sources, such as the Chinese Community Health Resource Center (CCHRC).

One of the main goals of the EXCLAIM project is to provide the kind of computational tools and CLIR tools for minority languages and endangered languages which are often available only for powerful or prosperous majority languages.

Current Status

EXCLAIM is in a beta state, with varying degrees of functionality for different languages. Support for CLIR using the Wikipedia dataset and the most current version of EXCLAIM (v.0.4), including full UTF-8 support and Porter stemming for the English component, is available for the following nineteen languages:

Amharic
Bengali
Gothic
Greek
Icelandic
Indonesian
Latvian
Malagasy
Nahuatl
Navajo
Quechua
Sardinian
Swahili
Tagalog
Tibetan
Turkish
Welsh
Wolof
Yiddish

Support using the Wikipedia dataset and an earlier version of EXCLAIM (v.0.3) is available for the following languages:

Dutch
Spanish

Current development efforts focus on developing support for Chinese, which has technical issues with segmentation and encoding as well as many available latent datasets in addition to the Wikipedia dataset. Chinese support will be the first for any language in EXCLAIM v.0.5, which incorporates the Trimming And Reformatting Modular System (TARMS) toolkit.

The EXCLAIM development plan calls for an integrated CLIR instrument usable searching from English for information in any of the supported languages, or searching from any of the supported languages for information in English when EXCLAIM 1.0 is released. Future versions will allow searching from any supported language into any other, and searching from and into multiple languages.

Notes and references

^ "Cross-Language Information Retrieval based on Parallel Texts and Automatic Mining of Parallel Texts in the Web" (PDF). ACM-SIGIR 1999. Retrieved 2006-12-2. {{cite web}}: Check date values in: |accessdate= (help)

External links

[1] "Cross-Language Information Retrieval based on Parallel Texts and Automatic Mining of Parallel Texts in the Web" (PDF). ACM-SIGIR 1999. Retrieved 2006-12-2. {{cite web}}: Check date values in: |accessdate= (help)

[1]

@@ Line 6: / Line 6: @@
 |title=Cross-Language Information Retrieval based on Parallel Texts and Automatic Mining of Parallel Texts in the Web
 |url=http://www.iro.umontreal.ca/%7Enie/Publication/nie-sigir99.pdf
+|publisher=ACM-SIGIR 1999
-|publisher=ACM-SIGIR 1999mnbbmnb.mnbnmb.nbnmbmj,bnjhnbnm nmbnmbmn.bkjmbkjkjkjnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn
 |accessdate=2006-12-2
 }}