Jump to content

Stylometry: Difference between revisions

From Wikipedia, the free encyclopedia
Browse history interactively
← Previous editNext edit →
Content deleted Content added
VisualWikitext
Line 82: Line 82:


===1. Gaussian statistics===
===1. Gaussian statistics===
Stylometric data are distributed according the [[Zipf-Mandelbrot law]]. However, usage of Gaussian statistics is possible by adopting data transformation.<ref> Van Droogenbroeck F.J., 'An essential rephrasing of the Zipf-Mandelbrot law to solve authorship attribution applications by Gaussian statistics' (2019) [https://www.academia.edu/40029629/An_essential_rephrasing_of_the_Zipf-Mandelbrot_law_to_solve_authorship_attribution_applications_by_Gaussian_statistics]</ref>
Stylometric data are distributed according the [[Zipf-Mandelbrot law]]. The distribution is extremely spiky and [[leptokurtic]], reason why researchers had to turn their backs to statistics to solve e.g. authorship attribution problems. Nevertheless, usage of Gaussian statistics is perfectly possible by applying [[data transformation (statistics)|data transformation]].<ref> Van Droogenbroeck F.J., 'An essential rephrasing of the Zipf-Mandelbrot law to solve authorship attribution applications by Gaussian statistics' (2019) [https://www.academia.edu/40029629]</ref>


===2. Neural networks===
===2. Neural networks===

Revision as of 13:20, 10 August 2019

Part of a series on
Linguistics
General linguistics
Applied linguistics
Theoretical frameworks
Topics
Portal

Stylometry is the application of the study of linguistic style, usually to written language, but it has successfully been applied to music[1] and to fine-art paintings[2] as well.[3]

Stylometry is often used to attribute authorship to anonymous or disputed documents. It has legal as well as academic and literary applications, ranging from the question of the authorship of Shakespeare's works to forensic linguistics.

History

Stylometry grew out of earlier techniques of analyzing texts for evidence of authenticity, author identity, and other questions.

The modern practice of the discipline received major impetus from the study of authorship problems in English Renaissance drama. Researchers and readers observed that some playwrights of the era had distinctive patterns of language preferences, and attempted to use those patterns to identify authors in uncertain or collaborative works. Early efforts were not always successful: in 1901, one researcher attempted to use John Fletcher's preference for "⁠ ⁠’em", the contractional form of "them", as a marker to distinguish between Fletcher and Philip Massinger in their collaborations—but he mistakenly employed an edition of Massinger's works in which the editor had expanded all instances of "⁠ ⁠’em" to "them".[4]

The basics of stylometry were set out by Polish philosopher Wincenty Lutosławski in Principes de stylométrie (1890). Lutosławski used this method to build a chronology of Plato's Dialogues.

The development of computers and their capacities for analyzing large quantities of data enhanced this type of effort by orders of magnitude. The great capacity of computers for data analysis, however, did not guarantee quality output. In the early 1960s, Rev. A. Q. Morton produced a computer analysis of the fourteen Epistles of the New Testament attributed to St. Paul, which showed that six different authors had written that body of work. A check of his method, applied to the works of James Joyce, gave the result that Ulysses, Joyce's multi-perspective, multi-style masterpiece, was written by five separate individuals, none of whom apparently had any part in the crafting of Joyce's first novel, A Portrait of the Artist as a Young Man.[5]

In time, however, and with practice, researchers and scholars have refined their approaches and methods, to yield better results. One notable early success was the resolution of disputed authorship in twelve of The Federalist Papers by Frederick Mosteller and David Wallace.[6] While questions of initial assumptions and methodology still arise (and, perhaps, always will), few now dispute the basic premise that linguistic analysis of written texts can produce valuable information and insight. (Indeed, this was apparent even before the advent of computers: the successful application of a textual/linguistic approach to the Fletcher canon by Cyrus Hoy and others yielded clear results in the late 1950s and early '60s.)

Applications

Applications of stylometry include literary studies, historical studies, social studies, gender studies, and many forensic cases and studies.[7][8] It can also be applied to computer code.[9]

Current research

Modern stylometry draws heavily on the aid of computers for statistical analysis, artificial intelligence and access to the growing corpus of texts available via the Internet.[10] Software systems such as Signature[11] (freeware produced by Dr Peter Millican of Oxford University), JGAAP[12] (the Java Graphical Authorship Attribution Program—freeware produced by Dr Patrick Juola of Duquesne University), stylo[13][14] (an open-source R package for a variety of stylometric analyses, including authorship attribution, developed by Maciej Eder, Jan Rybicki and Mike Kestemont) and Stylene[15] for Dutch (online freeware by Prof Walter Daelemans of University of Antwerp and Dr Véronique Hoste of University of Ghent) make its use increasingly practicable, even for the non-expert.

Notable cases

Stylometry has been used in a number of high-profile cases. Matthew Jockers used stylometric techniques to analyse the Book of Mormon, concluding that Sidney Rigdon, and not Joseph Smith, was its author.[16] More recently, a critique by Schaalje et al. of the above paper by Jockers et al. was published in Literary and Linguistic Computing (now Digital Scholarship in the Humanities), showing the Jockers et al. application (using a closed set of candidate authors, rather than an open one) of nearest shrunken centroid (NSC) classification to be misguided. The closed-set method leads to nonsensical results when applied to other authorship attribution problems, as the Schaalje et al. paper shows. Schaalje and colleagues produced results from an open-set NSC method showing that Sidney Rigdon was a very unlikely author—a result that comports with documented history that Rigdon and Smith did not even know each other in 1829 or early 1830, when the Book of Mormon was dictated and published, making it manifestly impossible for Rigdon to have been the author. Schaalje and colleagues also demonstrated that both Joseph Smith and the men who acted as scribes were unlikely authors, while also revealing multiple distinct voices aligning with the main purported authors of the text.[17]

Simon Fuller and James O'Sullivan published a study claiming that bestselling author James Patterson does not do any writing in his co-authored novels.[18][19][20] According to O'Sullivan, his collaboration with former U.S. president Bill Clinton, The President is Missing, is an exception to this rule.[21]

Academic venues and events

Stylometric methods are discussed in several academic fields, mostly as a tangential field of application as with machine learning, natural language processing, and lexicography.

Forensic linguistics

The International Association of Forensic Linguists (IAFL) organises the Biennial Conference of the International Association of Forensic Linguists (13th edition in 2016 in Porto) and publishes The International Journal of Speech, Language and the Law with forensic stylistics as one of its central topics.

AAAI

The Association for the Advancement of Artificial Intelligence (AAAI) has hosted several events on subjective and stylistic analysis of text.[22][23][24]

PAN

PAN workshops (originally, plagiarism analysis, authorship identification, and near-duplicate detection, later more generally workshop on uncovering plagiarism, authorship, and social software misuse) organised since 2007 mainly in conjunction with information access conferences such as ACM SIGIR, FIRE, and CLEF. PAN formulates shared challenge tasks for plagiarism detection,[25] authorship identification,[26] author gender identification,[27] author profiling,[28] vandalism detection,[29] and other related text analysis tasks, many of which hinge on stylometry.

Case studies of interest

  • Around 1370 to 1070 BC, as recorded in the Book of Judges, one tribe identified members of another tribe in order to kill them by asking them to say the word Shibboleth which in the dialect of the intended victims sounded like "sibboleth".[30]
  • In 1439, Lorenzo Valla showed that the Donation of Constantine was a forgery, an argument based partly on a comparison of the Latin with that used in authentic 4th-century documents.
  • In 1952, the Swedish priest Dick Helander was elected bishop of Strängnäs. The campaign was competitive and Helander was accused of writing a series of a hundred-some anonymous libelous letters about other candidates to the electorate of the bishopric of Strängnäs. Helander was first convicted of writing the letters and lost his position as bishop but later partially exonerated. The letters were studied using a number of stylometric measures (and also typewriter characteristics) and the various court cases and further examinations, many contracted by Helander himself during the years up to his death in 1978 discussed stylometric methodology and its value as evidence in some detail.[31][32]
  • In 1975, after Ronald Reagan had served as governor of California, he began giving weekly radio commentaries syndicated to hundreds of stations. After his personal notes were made public on his 90th birthday in 2001, a study to determine which of those talks were written by him and which were written by various aides used stylostatistical methods.[33]
  • In 1996, the stylometric analysis of the controversial, pseudonymously authored book Primary Colors, performed by Vassar College professor Donald Foster[34] brought the field to the attention of a wider audience after correctly identifying the author as Joe Klein. (This case was only resolved after a handwriting analysis confirmed the authorship).
  • In 1996, stylometric methods were used to compare the Unabomber manifesto with letters written by one of the suspects, Theodor Kaczynski to his brother, which led to his apprehension and later conviction.[35]
  • In April 2015, researchers using stylometry techniques identified a play, Double Falsehood, as being the work of William Shakespeare.[36] Researchers analyzed 54 plays by Shakespeare and John Fletcher and compared average sentence length, studied the use of unusual words and quantified the complexity and psychological valence of its language.
  • In 2017, a group of linguists, computer scientists, and scholars analysed the authorship of Elena Ferrante. Based on a corpus created at University of Padua containing 150 novels written by 40 authors, they analyzed Ferrante's style based on seven of her novels. They were able to compare her writing style with 39 other novelists using, for example, stylo.[13] The conclusion was the same for all of them: Domenico Starnone is the secret hand behind Elena Ferrante.[37]
  • In 2018, Mark Glickman, senior lecturer in statistics at Harvard University worked with Ryan Song, a former statistics student at Harvard, and Jason Brown, a professor at Dalhousie University in Nova Scotia, applied stylometry to find that, most likely, The Beatles' song "In My Life" was composed by John Lennon, but with a 50% chance that Paul McCartney wrote the middle eight.[38]

Data and methods

Since stylometry has both descriptive use cases, used to characterise the content of a collection, and identificatory use cases, e.g. identifying authors or categories of texts, the methods used to analyse the data and features above range from those built to classify items into sets or to distribute items in a space of feature variation. Most methods are statistical in nature, such as cluster analysis and discriminant analysis, are typically based on philological data and features, and are fruitful application domains for modern machine learning approaches.

Whereas in the past, stylometry emphasized the rarest or most striking elements of a text, contemporary techniques can isolate identifying patterns even in common parts of speech. Most systems are based on lexical statistics, i.e. using the frequencies of words and terms in the text to characterise the text (or its author). In this context, unlike in information retrieval, the observed occurrence patterns of the most common words are more interesting than the topical terms which are less frequent.[39][40]

The primary stylometric method is the writer invariant: a property held in common by all texts, or at least all texts long enough to admit of analysis yielding statistically significant results, written by a given author. An example of a writer invariant is frequency of function words used by the writer.

In one such method, the text is analyzed to find the 50 most common words. The text is then broken into 5,000 word chunks and each of the chunks is analyzed to find the frequency of those 50 words in that chunk. This generates a unique 50-number identifier for each chunk. These numbers place each chunk of text into a point in a 50-dimensional space. This 50-dimensional space is flattened into a plane using principal components analysis (PCA). This results in a display of points that correspond to an author's style. If two literary works are placed on the same plane, the resulting pattern may show if both works were by the same author or different authors.

1. Gaussian statistics

Stylometric data are distributed according the Zipf-Mandelbrot law. The distribution is extremely spiky and leptokurtic, reason why researchers had to turn their backs to statistics to solve e.g. authorship attribution problems. Nevertheless, usage of Gaussian statistics is perfectly possible by applying data transformation.[41]

2. Neural networks

Neural networks, a special case of statistical machine learning methods, have been used to analyze authorship of texts. Text of undisputed authorship are used to train the neural network through processes such as backpropagation, where training error is calculated and used to update the process to increase accuracy. Through a process akin to non-linear regression, the network gains the ability to generalize its recognition ability to new texts to which it has not yet been exposed, classifying them to a stated degree of confidence. Such techniques were applied to the long-standing claims of collaboration of Shakespeare with his contemporaries Fletcher and Christopher Marlowe,[42][43] and confirmed the view, based on more conventional scholarship, that such collaboration had indeed taken place.

A 1999 study showed that a neural network program reached 70% accuracy in determining authorship of poems it had not yet analyzed. This study from Vrije Universiteit examined identification of poems by three Dutch authors using only letter sequences such as "den".[44]

A study used deep belief networks (DBN) for authorship verification model applicable for continuous authentication (CA).[45]

One problem with this method of analysis is that the network can become biased based on its training set, possibly selecting authors the network has more often analyzed.[44]

3. Genetic algorithms

The genetic algorithm is another machine learning technique used in stylometry. This involves a method that starts out with a set of rules. An example rule might be, "If but appears more than 1.7 times in every thousand words, then the text is author X". The program is presented with text and uses the rules to determine authorship. The rules are tested against a set of known texts and each rule is given a fitness score. The 50 rules with the lowest scores are thrown out. The remaining 50 rules are given small changes and 50 new rules are introduced. This is repeated until the evolved rules correctly attribute the texts.

4. Rare pairs

One method for identifying style is called "rare pairs", and relies upon individual habits of collocation. The use of certain words may, for a particular author, idiosyncratically entail the use of other, predictable words.

Authorship attribution in instant messaging

The diffusion of Internet has shifted the authorship attribution attention towards online texts (web pages, blogs, etc.) electronic messages (e-mails, tweets, posts, etc.), and other types of written information that are far shorter than an average book, much less formal and more diverse in terms of expressive elements such as colors, layout, fonts, graphics, emoticons, etc. Efforts to take into account such aspects at the level of both structure and syntax were reported in.[46] In addition, content-specific and idiosyncratic cues (e.g., topic models and grammar checking tools) were introduced to unveil deliberate stylistic choices.[47]

Standard stylometric features have been employed to categorize the content of a chat over instant messaging,[48] or the behavior of the participants,[49] but attempts of identifying chat participants are still few and early. Furthermore, the similarity between spoken conversations and chat interactions has been neglected while being a key difference between chat data and any other type of written information.

See also

Notes

  1. ^ Westcott, Richard (15 June 2006). "Making hit music into a science". BBC News.
  2. ^ "Internet Archive Wayback Machine". 2006-06-30. Archived from the original on June 30, 2006. Retrieved 2012-10-15. {{cite web}}: Cite uses generic title (help); Unknown parameter |deadurl= ignored (|url-status= suggested) (help)
  3. ^ Argamon, Shlomo, Kevin Burns, and Shlomo Dubnov, eds. The structure of style: algorithmic approaches to understanding manner and meaning. Springer Science & Business Media, 2010.
  4. ^ Samuel Schoenbaum, Internal evidence and Elizabethan dramatic authorship; an essay in literary history and method, p. 171.
  5. ^ Samuel Schoenbaum, Internal evidence and Elizabethan dramatic authorship; an essay in literary history and method, p. 196.
  6. ^ F. Mosteller; D. Wallace (1964). Inference and Disputed Authorship: The Federalist. Reading, MA: Addison-Wesley. {{cite book}}: Unknown parameter |lastauthoramp= ignored (|name-list-style= suggested) (help)
  7. ^ Chaski, Carole (2012). Author Identification in the Forensic Setting. Oxford University Press. doi:10.1093/oxfordhb/9780199572120.001.0001. ISBN 9780199572120. {{cite book}}: |journal= ignored (help)
  8. ^ Chaski, Carole (22 December 2005). Wecht, Cyril H.; Rago, John T. (eds.). Forensic Science and Law: Investigative Applications in Criminal, Civil and Family Justice. CRC Press. ISBN 978-1-4200-5811-6.
  9. ^ Claburn, Thomas (March 16, 2018). "FYI: AI tools can unmask anonymous coders from their binary executables". The Register. Retrieved August 2, 2018.
  10. ^ Argamon, Shlomo, Jussi Karlgren, and James G. Shanahan. Stylistic analysis of text for information access. Papers from the workshop held in conjunction with the 28th Annual International ACM Conference on Research and Development in Information Retrieval, August 13–19, 2005, Salvador, Bahia, Brazil. Swedish institute of computer science, 2005.
  11. ^ "The Signature Stylometric System". PhiloComp. Retrieved 2014-01-03.
  12. ^ "JGAAP". JGAAP. 2012-09-04. Retrieved 2012-10-15.
  13. ^ a b "The stylo for R package". Computational Stylistics Group. 2014-10-24. Retrieved 2014-10-24.
  14. ^ Eder, Maciej; Rybicki, Jan; Kestemont, Mike (2016). "Stylometry with R: a package for computational text analysis" (PDF). R Journal. 8 (1): 107–121. doi:10.32614/RJ-2016-007.
  15. ^ Daelemans, Walter; Hoste, Véronique (2013). STYLENE: an Environment for Stylometry and Readability Research for Dutch (Technical report). CLiPS Technical Report Series. ISSN 2033-3544. {{cite tech report}}: Unknown parameter |lastauthoramp= ignored (|name-list-style= suggested) (help)
  16. ^ Jockers, M. L.; Witten, D. M.; Criddle, C. S. (2008-09-29). "Reassessing authorship of the Book of Mormon using delta and nearest shrunken centroid classification". Literary and Linguistic Computing. 23 (4): 465–491. doi:10.1093/llc/fqn040. ISSN 0268-1145.
  17. ^ Schaalje, G. Bruce; Fields, Paul J.; Roper, Matthew; Snow, Gregory L. (2011-01-18). "Extended nearest shrunken centroid classification: A new method for open-set authorship attribution of texts of varying sizes". Digital Scholarship in the Humanities. 26 (1): 71–88. doi:10.1093/llc/fqq029. ISSN 2055-7671.
  18. ^ Fuller, Simon; O'Sullivan, James (2017). "Structure over Style: Collaborative Authorship and the Revival of Literary Capitalism". Digital Humanities Quarterly. 011 (1). Retrieved April 20, 2017.
  19. ^ Lane, Anthony (June 18, 2018). "Bill Clinton and James Patterson's Concussive Collaboration". The New Yorker. Retrieved 2018-06-07.
  20. ^ "Why you don't need to write much to be the world's bestselling author". The Conversation. April 3, 2017. Retrieved April 20, 2017.
  21. ^ O'Sullivan, James (2018-06-07). "Bill Clinton and James Patterson are co-authors – but who did the writing?". The Guardian. Retrieved 2018-06-07. {{cite web}}: Cite has empty unknown parameter: |dead-url= (help)
  22. ^ Yan Qu, James Shanahan, and Janyce Wiebe. "Exploring attitude and affect in text: Theories and applications." AAAI Spring Symposium Technical report SS-04-07. AAAI Press, Menlo Park, CA. 2004.
  23. ^ Jussi Karlgren, Björn Gambäck, and Pentti Kanerva. "Acquiring (and Using) Linguistic (and World) Knowledge for Information Access." (2002). AAAI Spring Symposium. Technical report SS-02-09. AAAI Press, Menlo Park, CA. 2002.
  24. ^ Shlomo Argamon, Shlomo Dubnov, and Julie Jupp. "Style and Meaning in Language, Art, Music, and Design" (2004). AAAI Fall Symposium. Technical report FS-04-07.
  25. ^ Potthast, Martin, Benno Stein, Alberto Barrón-Cedeño, and Paolo Rosso. "An evaluation framework for plagiarism detection." In Proceedings of the 23rd international conference on computational linguistics: Posters, pp. 997–1005. Association for Computational Linguistics, 2010.
  26. ^ Stamatatos, Efstathios, Walter Daelemans, Ben Verhoeven, Patrick Juola, Aurelio López-López, Martin Potthast, and Benno Stein. "Overview of the Author Identification Task at PAN 2014." In CLEF (Working Notes), pp. 877–897. 2014.
  27. ^ Rangel, Francisco, Paolo Rosso, Martin Potthast, and Benno Stein. "Overview of the 5th author profiling task at pan 2017: Gender and language variety identification in twitter." Working Notes Papers of the CLEF (2017).
  28. ^ Rangel Pardo, Francisco Manuel, Fabio Celli, Paolo Rosso, Martin Potthast, Benno Stein, and Walter Daelemans. "Overview of the 3rd Author Profiling Task at PAN 2015." In CLEF 2015 Evaluation Labs and Workshop Working Notes Papers, pp. 1–8. 2015.
  29. ^ Potthast, Martin, Benno Stein, and Teresa Holfeld. "Overview of the 1st International Competition on Wikipedia Vandalism Detection." In CLEF (Notebook Papers/LABs/Workshops). 2010.
  30. ^ Judges 12:5–6
  31. ^ Text processing text analysis and generation – text typology and attribution. Proceedings of Nobel symposium 51 / ed. by Sture Allén Stockholm : Almqvist & Wiksell international 1982 653 pp. Data linguistica ; 16 Nobel symposium ; 51 ISBN 91-22-00594-3
  32. ^ Karlgren, Jussi (2003). "Helander: An Authorship Attribution Case". Retrieved 4 October 2017.
  33. ^ Edoardo M. Airoldi; Stephen E. Fienberg; Kiron K. Skinner (July 2007). "Whose Ideas? Whose Words? Authorship of Ronald Reagan's Radio Addresses" (PDF). PS: Political Science & Politics. 40 (3): 501–506. CiteSeerX 10.1.1.190.5798. doi:10.1017/S1049096507070874.
  34. ^ Author Unknown by Gavin McNett Salon November 2, 2000
  35. ^ Belluck, Pam (April 10, 1996). "In Unabom Case, Pain for Suspect's Family". The New York Times. Archived from the original on August 10, 2017. Retrieved July 5, 2008. {{cite news}}: Unknown parameter |deadurl= ignored (|url-status= suggested) (help)
  36. ^ "Study finds a disputed Shakespeare play bears the master's mark". LATimes.com. 2015-04-10. Retrieved 2015-04-13.
  37. ^ Jacques Savoy. Elena Ferrante Unmasked. https://www.researchgate.net/publication/320131096_Elena_Ferrante_Unmasked
  38. ^ Peter Reuell. "You say John, I say Paul. But what does stylometry say?". https://news.harvard.edu/gazette/story/2018/09/harvard-statistician-examines-beatles-mystery/
  39. ^ Biber, Douglas. Variation across speech and writing. Cambridge University Press, 1991.
  40. ^ Karlgren, Jussi; Cutting, Douglass (1994). "Recognizing Text Genres with Simple Metrics Using Discriminant Analysis". Proceedings of the International Conference on Computational Linguistics.
  41. ^ Van Droogenbroeck F.J., 'An essential rephrasing of the Zipf-Mandelbrot law to solve authorship attribution applications by Gaussian statistics' (2019) [1]
  42. ^ [2] Neural Computation in Stylometry I: An Application to the Works of Shakespeare and Fletcher Matthews RAJ & Merriam TVN Lit Linguist Computing (1993) 8 (4): 203–209. doi: 10.1093/llc/8.4.203
  43. ^ [3]Neural Computation in Stylometry II: An Application to the Works of Shakespeare and Marlowe Merriam TVN & Matthews RAJ Lit Linguist Computing (1994) 9 (1): 1–6
  44. ^ a b JF HoornZ; SL Frank; W Kowalczyk; F van der Ham (2012-09-03). "Neural network identification of poets using letter sequences". Literary and Linguistic Computing. 14 (3): 311–338. doi:10.1093/llc/14.3.311. Retrieved 2012-10-15.
  45. ^ Brocardo, ML; Traore, I; Woungang, I; Obaidat, MS (2017). "Authorship verification using deep belief network systems". Int J Commun Syst. 30 (12): e3259. doi:10.1002/dac.3259.
  46. ^ de Vel, O.; Anderson, A.; Corney, M.; Mohay, G. (2001-12-01). "Mining e-Mail Content for Author Identification Forensics". SIGMOD Rec. 30 (4): 55–64. CiteSeerX 10.1.1.408.4231. doi:10.1145/604264.604272. ISSN 0163-5808.
  47. ^ Argamon, Shlomo; Koppel, Moshe; Pennebaker, James W.; Schler, Jonathan (2009-02-01). "Automatically Profiling the Author of an Anonymous Text". Commun. ACM. 52 (2): 119–123. CiteSeerX 10.1.1.136.9952. doi:10.1145/1461928.1461959. ISSN 0001-0782.
  48. ^ "Classification of Instant Messaging Communications for Forensics Analysis – TechRepublic". TechRepublic. Retrieved 2016-01-26.
  49. ^ Zhou, L.; Zhang, Dongsong (2004-01-01). Can online behavior unveil deceivers? – an exploratory investigation of deception in instant messaging. pp. 9 pp.–. doi:10.1109/HICSS.2004.1265079. ISBN 978-0-7695-2056-8. {{cite book}}: |journal= ignored (help)

References

Further reading

See also the academic journal Literary and Linguistic Computing (published by the University of Oxford) and the Language Resources and Evaluation journal.

Retrieved from "https://en.wikipedia.org/enwiki/w/index.php?title=Stylometry&oldid=910212391"