Jump to content

Computational linguistics

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by Headbomb (talk | contribs) at 13:52, 23 July 2023 (IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

This article is about the scientific field. For the journal, see Computational Linguistics (journal).
Part of a series on
Linguistics
General linguistics
Applied linguistics
Theoretical frameworks
Topics
Portal

Computational linguistics has since 2020s became a near-synonym of either natural language processing or language technology, with deep learning approaches, such as large language models, having replaced most of the specific approaches previously used in the field.

History

After the failure of rule-based approaches in AI in general and in machine translation in particular, David Hays[1] coined the term in order to distinguish the field from AI and co-founded both the Association for Computational Linguistics (ACL) and the International Committee on Computational Linguistics (ICCL) in the 1970s and 1980s.

Origins

The field overlapped with artificial intelligence since the efforts in the United States in the 1950s to use computers to automatically translate texts from foreign languages, particularly Russian scientific journals, into English.[2] Since rule-based approaches were able to make arithmetic (systematic) calculations much faster and more accurately than humans, it was thought to be only a short matter of time before they could also begin to process language.[3]

At the time, it seemed that both the grammar of both languages, including both morphology (the grammar of word forms) and syntax (the grammar of sentence structure) of both languages need to be learnt first before one is able to translate between the two. To understand syntax, one had to also understand the semantics and the lexicon (or 'vocabulary'), and even something of the pragmatics of language use. What started as an effort to translate between languages evolved into a much wider field of natural language processing.[4]

Modelling language acquisition

The fact that during language acquisition, children are largely only exposed to positive evidence,[5] meaning that the only evidence for what is a correct form is provided, and no evidence for what is not correct,[6] was a limitation for the models at the time because the now available deep learning models were not available in late 1980s.[7]

It has been shown that languages can be learned with a combination of simple input presented incrementally as the child develops better memory and longer attention span,[8] which explained the long period of language acquisition in human infants and children.[8]

Robots has been used to test linguistic theories.[9] Enabled to learn as children might, models were created based on an affordance model in which mappings between actions, perceptions, and effects were created and linked to spoken words. Crucially, these robots were able to acquire functioning word-to-meaning mappings without needing grammatical structure.

Using the Price equation and Pólya urn dynamics, researchers have created a system which not only predicts future linguistic evolution but also gives insight into the evolutionary history of modern-day languages.[10]

Annotated corpora

In order to be able to meticulously study the English language, an annotated text corpus was much needed. The Penn Treebank[11] was one of the most used corpora. It consisted of IBM computer manuals, transcribed telephone conversations, and other texts, together containing over 4.5 million words of American English, annotated using both part-of-speech tagging and syntactic bracketing.[12]

Using computational methods, Japanese sentence corpora were analyzed and a pattern of log-normality was found in relation to sentence length.[13]

Chomsky's theories

Attempts have been made to find out how does an infant learn a "non-normal grammar" as theorized by Chomsky normal form without learning an "overgeneralized version" and "getting stuck".[6]

See also

References

  1. ^ "Deceased members". ICCL members. Archived from the original on 17 May 2017. Retrieved 15 November 2017.
  2. ^ John Hutchins: Retrospect and prospect in computer-based translation. Proceedings of MT Summit VII, 1999, pp. 30–44.
  3. ^ Arnold B. Barach: Translating Machine 1975: And the Changes To Come.
  4. ^ Natural Language Processing by Liz Liddy, Eduard Hovy, Jimmy Lin, John Prager, Dragomir Radev, Lucy Vanderwende, Ralph Weischedel
  5. ^ Bowerman, M. (1988). The "no negative evidence" problem: How do children avoid constructing an overly general grammar. Explaining language universals.
  6. ^ a b Braine, M.D.S. (1971). On two types of models of the internalization of grammars. In D.I. Slobin (Ed.), The ontogenesis of grammar: A theoretical perspective. New York: Academic Press.
  7. ^ Powers, D.M.W. & Turk, C.C.R. (1989). Machine Learning of Natural Language. Springer-Verlag. ISBN 978-0-387-19557-5.
  8. ^ a b Elman, Jeffrey L. (1993). "Learning and development in neural networks: The importance of starting small". Cognition. 48 (1): 71–99. doi:10.1016/0010-0277(93)90058-4. PMID 8403835. S2CID 2105042.
  9. ^ Salvi, G.; Montesano, L.; Bernardino, A.; Santos-Victor, J. (2012). "Language bootstrapping: learning word meanings from the perception-action association". IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics. 42 (3): 660–71. arXiv:1711.09714. doi:10.1109/TSMCB.2011.2172420. PMID 22106152. S2CID 977486.
  10. ^ Gong, T.; Shuai, L.; Tamariz, M. & Jäger, G. (2012). E. Scalas (ed.). "Studying Language Change Using Price Equation and Pólya-urn Dynamics". PLOS ONE. 7 (3): e33171. Bibcode:2012PLoSO...733171G. doi:10.1371/journal.pone.0033171. PMC 3299756. PMID 22427981.
  11. ^ Marcus, M. & Marcinkiewicz, M. (1993). "Building a large annotated corpus of English: The Penn Treebank" (PDF). Computational Linguistics. 19 (2): 313–330. Archived (PDF) from the original on 2022-10-09.
  12. ^ Taylor, Ann (2003). "1". Treebanks. Spring Netherlands. pp. 5–22.
  13. ^ Furuhashi, S. & Hayakawa, Y. (2012). "Lognormality of the Distribution of Japanese Sentence Lengths". Journal of the Physical Society of Japan. 81 (3): 034004. Bibcode:2012JPSJ...81c4004F. doi:10.1143/JPSJ.81.034004.

Further reading

Wikiversity has learning resources about Computational linguistics
Wikimedia Commons has media related to Computational linguistics.
Retrieved from "https://en.wikipedia.org/enwiki/w/index.php?title=Computational_linguistics&oldid=1166744896"