Jump to content

Glob (programming): Difference between revisions

From Wikipedia, the free encyclopedia
Browse history interactively
← Previous editNext edit →
Content deleted Content added
VisualWikitext
Citation bot (talk | contribs)
Bots
5,426,715 edits
Added date. | Use this bot. Report bugs. | Suggested by Dominic3203 | Category:Pattern matching | #UCB_Category 19/28
(47 intermediate revisions by 31 users not shown)
Line 1: Line 1:
{{Short description|Patterns used in computer programming}}
{{lowercase}}
{{lowercase}}
{{Use dmy dates|date=July 2019|cs1-dates=y}}
{{Use dmy dates|date=July 2019|cs1-dates=y}}
In [[computer programming]], '''glob''' patterns specify sets of filenames with [[wildcard characters]]. For example, the Unix [[Bash (Unix shell)|Bash shell]] command <syntaxhighlight lang="Bash" inline>mv *.txt textfiles/</syntaxhighlight> moves (<syntaxhighlight lang="Bash" inline>mv</syntaxhighlight>) all files with names ending in <code>.txt</code> from the current directory to the directory <code>textfiles</code>. Here, <code>*</code> is a wildcard standing for "any [[String (computer science)|string]] of characters" and <code>*.txt</code> is a glob pattern. The other common wildcard is the question mark (<code>?</code>), which stands for one character. For example, <syntaxhighlight lang="Bash" inline>mv ?.txt shorttextfiles/</syntaxhighlight> will move all files named with a single character followed by <code>.txt</code> from the current directory to directory <code>shorttextfiles</code>, while <syntaxhighlight lang="Bash" inline>??.txt</syntaxhighlight> would match all files whose name consists of 2 characters followed by <code>.txt</code>.
In [[computer programming]], '''glob''' ({{IPAc-en|g|l|ɒ|b}}) patterns specify sets of filenames with [[wildcard characters]]. For example, the Unix [[Bash (Unix shell)|Bash shell]] command <syntaxhighlight lang="Bash" inline>mv *.txt textfiles/</syntaxhighlight> moves all files with names ending in <code>.txt</code> from the current directory to the directory <code>textfiles</code>. Here, <code>*</code> is a wildcard and <code>*.txt</code> is a glob pattern. The wildcard <code>*</code> stands for "any [[String (computer science)|string]] of any length including empty, but excluding the path separator characters (<code>/</code> in unix and <code>\</code> in windows)".
The other common wildcard is the question mark (<code>?</code>), which stands for one character. For example, <syntaxhighlight lang="Bash" inline>mv ?.txt shorttextfiles/</syntaxhighlight> will move all files named with a single character followed by <code>.txt</code> from the current directory to directory <code>shorttextfiles</code>, while <syntaxhighlight lang="Bash" inline>??.txt</syntaxhighlight> would match all files whose name consists of 2 characters followed by <code>.txt</code>.


In addition to matching filenames, globs are also used widely for matching arbitrary strings ([[wildcard matching]]). In this capacity a common interface is <code>fnmatch</code>.
In addition to matching filenames, globs are also used widely for matching arbitrary strings ([[wildcard matching]]). In this capacity a common interface is <code>fnmatch</code>.
Line 9: Line 12:
The glob command, short for ''global'', originates in the earliest versions of Bell Labs' [[Unix]].<ref name="man7Unix1"/> The command interpreters of the early versions of Unix (1st through 6th Editions, 1969–1975) relied on a separate program to expand [[wildcard character]]s in unquoted arguments to a command: ''/etc/glob''. That program performed the expansion and supplied the expanded list of file paths to the command for execution.
The glob command, short for ''global'', originates in the earliest versions of Bell Labs' [[Unix]].<ref name="man7Unix1"/> The command interpreters of the early versions of Unix (1st through 6th Editions, 1969–1975) relied on a separate program to expand [[wildcard character]]s in unquoted arguments to a command: ''/etc/glob''. That program performed the expansion and supplied the expanded list of file paths to the command for execution.


Glob was originally written in the [[B (programming language)|B programming language]]. It was the first piece of mainline Unix software to be developed in a [[high-level programming language]].<ref name="reader">{{cite techreport |first1=M. D. |last1=McIlroy |author-link1=Doug McIlroy |year=1987 |url=http://www.cs.dartmouth.edu/~doug/reader.pdf |title=A Research Unix reader: annotated excerpts from the Programmer's Manual, 1971–1986 |series=CSTR |number=139 |institution=Bell Labs}}</ref> Later, this functionality was provided as a C [[library function]], <code>glob()</code>, used by programs such as the [[Bourne shell|shell]]. It is usually defined based on a function named <code>fnmatch()</code>, which tests for whether a string matches a given pattern - the program using this function can then iterate through a series of strings (usually filenames) to determine which ones match. Both functions are a part of [[POSIX]]: the functions defined in POSIX.1 since 2001, and the syntax defined in POSIX.2.<ref name=fnmatch3>{{man|3|fnmatch|Linux}}</ref><ref>{{man|3|glob|Linux}}</ref> The idea of defining a separate match function started with [[wildmat]] (wildcard match), a simple library to match strings against Bourne Shell globs.
Glob was originally written in the [[B (programming language)|B programming language]]. It was the first piece of mainline Unix software to be developed in a [[high-level programming language]].<ref name="reader">{{cite tech report |first1=M. D. |last1=McIlroy |author-link1=Doug McIlroy |year=1987 |url=http://www.cs.dartmouth.edu/~doug/reader.pdf |title=A Research Unix reader: annotated excerpts from the Programmer's Manual, 1971–1986 |series=CSTR |number=139 |institution=Bell Labs}}</ref> Later, this functionality was provided as a C [[library function]], <code>glob()</code>, used by programs such as the [[Bourne shell|shell]]. It is usually defined based on a function named <code>fnmatch()</code>, which tests for whether a string matches a given pattern - the program using this function can then iterate through a series of strings (usually filenames) to determine which ones match. Both functions are a part of [[POSIX]]: the functions defined in POSIX.1 since 2001, and the syntax defined in POSIX.2.<ref name=fnmatch3>{{man|3|fnmatch|Linux}}</ref><ref>{{man|3|glob|Linux}}</ref> The idea of defining a separate match function started with [[wildmat]] (wildcard match), a simple library to match strings against Bourne Shell globs.


Traditionally, globs do not match hidden files in the form of Unix [[dotfiles]]; to match them the pattern must explicitly start with <code>.</code>. For example, <code>*</code> matches all visible files while <code>.*</code> matches all hidden files.
Traditionally, globs do not match hidden files in the form of Unix [[dotfiles]]; to match them the pattern must explicitly start with <code>.</code>. For example, <code>*</code> matches all visible files while <code>.*</code> matches all hidden files.
Line 58: Line 61:
|}
|}


Normally, the path separator character ({{code|/}} on Linux/Unix, MacOS, etc. or {{code|\}} on Windows) will never be matched. Some shells, such as [[Bash (Unix shell)|Bash]] have functionality allowing users to circumvent this. <ref>[https://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching Bash Reference Manual</ref>
Normally, the path separator character ({{code|/}} on Linux/Unix, MacOS, etc. or {{code|\}} on Windows) will never be matched. Some shells, such as [[Bash (Unix shell)|Bash]] have functionality allowing users to circumvent this.<ref>https://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching {{Webarchive|url=https://web.archive.org/web/20180315115230/http://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching |date=2018-03-15 }} Bash Reference Manual</ref>


===Unix-like{{anchor|Unix}}===
===Unix-like{{anchor|Unix}}===
Line 94: Line 97:
Some shells (such as the [[C shell]] and [[Bash_(Unix_shell)|Bash]]) support additional syntax known as [[alternation (string expansion)|alternation]] or [[brace expansion]]. Because it is not part of the glob syntax, it is not provided in <code>case</code>. It is only expanded on the command line before globbing.
Some shells (such as the [[C shell]] and [[Bash_(Unix_shell)|Bash]]) support additional syntax known as [[alternation (string expansion)|alternation]] or [[brace expansion]]. Because it is not part of the glob syntax, it is not provided in <code>case</code>. It is only expanded on the command line before globbing.


The Bash shell also supports the following extensions:<ref>{{cite web |title=Bash globs |url=https://mywiki.wooledge.org/glob |website=greg's bash knowledgebase |access-date=25 November 2019}}</ref>
The Bash shell also supports the following extensions:<ref>{{cite web |title=Bash globs |url=https://mywiki.wooledge.org/glob |website=greg's bash knowledgebase |access-date=25 November 2019 |archive-date=2019-11-18 |archive-url=https://web.archive.org/web/20191118054712/http://mywiki.wooledge.org/glob |url-status=live }}</ref>
* Extended globbing (extglob): allows other pattern matching operators to be used to match multiple occurrences of a pattern enclosed in parentheses, essentially providing the missing [[kleene star]] and alternation for describing regular languages. It can be enabled by setting the {{code|extglob}} shell option. This option came from ksh93.<ref name="bashpat"/> The GNU fnmatch and glob has an identical extension.<ref name=fnmatch3/>
* Extended globbing (extglob): allows other pattern matching operators to be used to match multiple occurrences of a pattern enclosed in parentheses, essentially providing the missing [[kleene star]] and alternation for describing regular languages. It can be enabled by setting the {{code|extglob}} shell option. This option came from ksh93.<ref name="bashpat"/> The GNU fnmatch and glob has an identical extension.<ref name=fnmatch3/>
* globstar: allows <code>**</code> on its own as a name component to recursively match any number of layers of non-hidden directories.<ref name="bashpat"/> Also supported by the JS libraries and Python's glob.
* globstar: allows <code>**</code> on its own as a name component to recursively match any number of layers of non-hidden directories.<ref name="bashpat"/> Also supported by the [[JavaScript]] libraries and [[Python (programming language)|Python]]'s glob.


===Windows and DOS===
===Windows and DOS===
[[File:IBM PC DOS 1.0 screenshot.jpg|thumb|The [[dir (command)|{{code|dir}}]] command with a glob pattern in [[IBM PC DOS]]]]
[[File:IBM PC DOS 1.0 screenshot.png|thumb|The [[dir (command)|{{code|dir}}]] command with a glob pattern in [[IBM PC DOS]] 1.0.]]


The original [[DOS]] was a clone of [[CP/M]] designed to work on Intel's [[Intel 8088|8088]] and [[Intel 8086|8086]] processors. Windows shells, following DOS, do not traditionally perform any glob expansion in arguments passed to external programs. Shells may use an expansion for their own builtin commands:
The original [[DOS]] was a clone of [[CP/M]] designed to work on Intel's [[Intel 8088|8088]] and [[Intel 8086|8086]] processors. Windows shells, following DOS, do not traditionally perform any glob expansion in arguments passed to external programs. Shells may use an expansion for their own builtin commands:
Line 108: Line 111:
Windows and DOS programs receive a long command-line string instead of argv-style parameters, and it is their responsibility to perform any splitting, quoting, or glob expansion. There is technically no fixed way of describing wildcards in programs since they are free to do what they wish. Two common glob expanders include:<ref name="winWildcard"/>
Windows and DOS programs receive a long command-line string instead of argv-style parameters, and it is their responsibility to perform any splitting, quoting, or glob expansion. There is technically no fixed way of describing wildcards in programs since they are free to do what they wish. Two common glob expanders include:<ref name="winWildcard"/>


* The Microsoft C Runtime (msvcrt) command-line expander, which only supports {{code|?}} and {{code|*}}.<ref>{{cite web |title=Wildcard Expansion |url=https://docs.microsoft.com/en-us/cpp/cpp/wildcard-expansion |website=docs.microsoft.com |language=en-us}}</ref> Both [[ReactOS]] (crt/misc/getargs.c) and [[Wine (software)|Wine]] (msvcrt/data.c) contain a compatible open-source implementation of {{code|__getmainargs}}, the function operating under-the-hood, in their core CRT.
* The Microsoft C Runtime (msvcrt) command-line expander, which only supports {{code|?}} and {{code|*}}.<ref>{{cite web |title=Wildcard Expansion |url=https://docs.microsoft.com/en-us/cpp/cpp/wildcard-expansion |website=docs.microsoft.com |date=8 February 2022 |language=en-us}}</ref> Both [[ReactOS]] (crt/misc/getargs.c) and [[Wine (software)|Wine]] (msvcrt/data.c) contain a compatible open-source implementation of {{code|__getmainargs}}, the function operating under-the-hood, in their core CRT.
* The [[Cygwin]] and MSYS {{code|dcrt0.cc}} command-line expander, which uses the unix-style {{code|glob()}} routine under-the-hood, after splitting the arguments.
* The [[Cygwin]] and MSYS {{code|dcrt0.cc}} command-line expander, which uses the unix-style {{code|glob()}} routine under-the-hood, after splitting the arguments.


Most other parts of Windows, including the Indexing Service, use the MS-DOS style of wildcards found in CMD. A relic of the 8.3 filename age, this syntax pays special attention to dots in the pattern and the text (filename). Internally this is done using three extra wildcard characters, {{code|<>"}}. On the Windows API end, the {{tt|glob()}} equivalent is {{tt|FindFirstFile}}, and {{tt|fnmatch()}} corresponds to its underlying {{tt|RtlIsNameInExpression}}.<ref>[https://blogs.msdn.microsoft.com/jeremykuhne/2017/06/04/wildcards-in-windows/ Wildcards in Windows]. MSDN Devblog.</ref> (Another fnmatch analogue is {{tt|PathMatchSpec}}.) Both open-source msvcrt expanders use {{tt|FindFirstFile}}, so 8.3 filename quirks will also apply in them.
Most other parts of Windows, including the Indexing Service, use the MS-DOS style of wildcards found in CMD. A relic of the 8.3 filename age, this syntax pays special attention to dots in the pattern and the text (filename). Internally this is done using three extra wildcard characters, {{code|<>"}}. On the Windows API end, the {{tt|glob()}} equivalent is {{tt|FindFirstFile}}, and {{tt|fnmatch()}} corresponds to its underlying {{tt|RtlIsNameInExpression}}.<ref>[https://blogs.msdn.microsoft.com/jeremykuhne/2017/06/04/wildcards-in-windows/ Wildcards in Windows] {{Webarchive|url=https://web.archive.org/web/20191224091832/https://blogs.msdn.microsoft.com/jeremykuhne/2017/06/04/wildcards-in-windows/ |date=2019-12-24 }}. MSDN Devblog.</ref> (Another fnmatch analogue is {{tt|PathMatchSpec}}.) Both open-source msvcrt expanders use {{tt|FindFirstFile}}, so 8.3 filename quirks will also apply in them.


===SQL===
===SQL===
Line 161: Line 164:
The original Mozilla [[proxy auto-config]] implementation, which provides a glob-matching function on strings, uses a replace-as-RegExp implementation as above. The bracket syntax happens to be covered by regex in such an example.
The original Mozilla [[proxy auto-config]] implementation, which provides a glob-matching function on strings, uses a replace-as-RegExp implementation as above. The bracket syntax happens to be covered by regex in such an example.


Python's fnmatch uses a more elaborate procedure to change the pattern into a regular expression.<ref name=py>{{cite web |title=Lib/fnmatch.py |url=https://github.com/python/cpython/blob/3.4/Lib/fnmatch.py |publisher=Python |access-date=24 November 2019 |date=24 November 2019}}</ref>
Python's fnmatch uses a more elaborate procedure to transform the pattern into a regular expression.<ref name=py>{{cite web |title=Lib/fnmatch.py |url=https://github.com/python/cpython/blob/3.8/Lib/fnmatch.py |publisher=Python |access-date=10 November 2021 |date=20 January 2021 |archive-date=2021-11-10 |archive-url=https://web.archive.org/web/20211110204645/https://github.com/python/cpython/blob/3.8/Lib/fnmatch.py |url-status=live }}</ref>


== Other implementations==
== Other implementations==
Beyond their uses in shells, globs patterns also find use in a variety of programming languages, mainly to process human input. A glob-style interface for returning files or an fnmatch-style interface for matching strings are found in the following programming languages:
Beyond their uses in shells, globs patterns also find use in a variety of programming languages, mainly to process human input. A glob-style interface for returning files or an fnmatch-style interface for matching strings are found in the following programming languages:


* [[C Sharp (programming language)|C#]] has a library called <code>Glob</code> which can be installed using [[NuGet]].<ref name="dotnet-glob"/>
* [[C Sharp (programming language)|C#]] has multiple libraries available through [[NuGet]] such as <code>Glob</code>.<ref name="dotnet-glob"/> or <code>DotNet.Glob</code>.<ref name="dotnet.glob"/>
* [[D (programming language)|D]] has a <code>globMatch</code> function in the <code>std.path</code> module.<ref name="dlang"/>
* [[D (programming language)|D]] has a <code>globMatch</code> function in the <code>std.path</code> module.<ref name="dlang"/>
* [[JavaScript]] has a library called <code>minimatch</code> which is used internally by [[Npm (software)|npm]], and <code>micromatch</code>, a purportedly more optimized, accurate and safer globbing implementation used by [[Babel (compiler)|babel]] and yarn.<ref name="minimatch"/><ref name="micromatch"/>
* [[JavaScript]] has a library called <code>minimatch</code> which is used internally by [[Npm (software)|npm]], and <code>micromatch</code>, a purportedly more optimized, accurate and safer globbing implementation used by [[Babel (compiler)|Babel]] and yarn.<ref name="minimatch"/><ref name="micromatch"/>
* [[Go (programming language)|Go]] has a <code>Glob</code> function in the <code>filepath</code> package.<ref name="golang"/>
* [[Go (programming language)|Go]] has a <code>Glob</code> function in the <code>filepath</code> package.<ref name="golang"/>
* [[Java (programming language)|Java]] has a <code>Files</code> class containing methods that operate on glob patterns.<ref name="java"/>
* [[Java (programming language)|Java]] has a <code>Files</code> class containing methods that operate on glob patterns.<ref name="java"/>
* [[Haskell (programming language)|Haskell]] has a <code>Glob</code> package with the main module <code>System.FilePath.Glob</code>. The pattern syntax is based on a subset of [[Zsh]]’s. It tries to optimize the given pattern and should be noticeably faster than a naïve character-by-character matcher.<ref name="hs"/>
* [[Haskell (programming language)|Haskell]] has a <code>Glob</code> package with the main module <code>System.FilePath.Glob</code>. The pattern syntax is based on a subset of [[Zsh]]'s. It tries to optimize the given pattern and should be noticeably faster than a naïve character-by-character matcher.<ref name="hs"/>
* [[Perl]] has both a <code>glob</code> function (as discussed in [[Larry Wall]]'s book ''[[Programming Perl]]'') and a ''Glob'' extension which mimics the BSD glob routine.<ref name="pl"/> Perl's angle brackets can be used to glob as well: <code>&lt;*.log&gt;</code>.
* [[Perl]] has both a <code>glob</code> function (as discussed in [[Larry Wall]]'s book ''[[Programming Perl]]'') and a ''Glob'' extension which mimics the BSD glob routine.<ref name="pl"/> Perl's angle brackets can be used to glob as well: <code>&lt;*.log&gt;</code>.
* [[PHP]] has a <code>glob</code> function.<ref name="php"/>
* [[PHP]] has a <code>glob</code> function.<ref name="php"/>
* [[Python (programming language)|Python]] has a <code>glob</code> module in the standard library which performs wildcard pattern matching on filenames,<ref name="pyglob"/> and an <code>fnmatch</code> module with functions for matching strings or filtering lists based on these same wildcard patterns.<ref name=py/> [[Guido van Rossum]], author of the Python programming language, wrote and contributed a <code>glob</code> routine to [[Berkeley Software Distribution|BSD]] [[Unix]] in 1986.<ref name="isc-glob"/> There were previous implementations of <code>glob</code>, e.g., in the [[ex (text editor)|ex]] and [[File Transfer Protocol|ftp]] programs in previous releases of BSD.
* [[Python (programming language)|Python]] has a <code>glob</code> module in the standard library which performs wildcard pattern matching on filenames,<ref name="pyglob"/> and an <code>fnmatch</code> module with functions for matching strings or filtering lists based on these same wildcard patterns.<ref name=py/> [[Guido van Rossum]], author of the Python programming language, wrote and contributed a <code>glob</code> routine to [[Berkeley Software Distribution|BSD]] [[Unix]] in 1986.<ref name="isc-glob"/> There were previous implementations of <code>glob</code>, e.g., in the [[ex (text editor)|ex]] and [[File Transfer Protocol|ftp]] programs in previous releases of BSD.
* [[Ruby (programming language)|Ruby]] has a <code>glob</code> method for the <code>Dir</code> class which performs wildcard pattern matching on filenames.<ref name="rbdir"/> Several libraries such as Rant and Rake provide a <code>FileList</code> class which has a glob method or use the method <code>FileList.[]</code> identically.
* [[Ruby (programming language)|Ruby]] has a <code>glob</code> method for the <code>Dir</code> class which performs wildcard pattern matching on filenames.<ref name="rbdir"/> Several libraries such as Rant and Rake provide a <code>FileList</code> class which has a glob method or use the method <code>FileList.[]</code> identically.
* [[Rust (programming language)|Rust]] has multiple libraries that can match glob patterns.<ref>{{cite web|title=#glob - Lib.rs|url=https://lib.rs/keywords/glob|access-date=12 November 2021|website=lib.rs|archive-date=2021-11-12 |archive-url=https://web.archive.org/web/20211112115505/https://lib.rs/keywords/glob|url-status=live}}</ref>
* [[SQLite]] has a <code>GLOB</code> function.
* [[SQLite]] has a <code>GLOB</code> function.
* [[Tcl]] contains a globbing facility.<ref name="tcl"/>
* [[Tcl]] contains a globbing facility.<ref name="tcl"/>
Line 187: Line 191:
{{Reflist|refs=
{{Reflist|refs=
<ref name="man7Unix1">{{cite web |url=http://cm.bell-labs.com/cm/cs/who/dmr/man71.pdf |archive-url=https://web.archive.org/web/20000829224359/http://cm.bell-labs.com/cm/cs/who/dmr/man71.pdf |url-status=dead |archive-date=2000-08-29 |title=First Edition Unix manual 'Miscellaneous' section (PDF) |access-date=2011-05-11}}</ref>
<ref name="man7Unix1">{{cite web |url=http://cm.bell-labs.com/cm/cs/who/dmr/man71.pdf |archive-url=https://web.archive.org/web/20000829224359/http://cm.bell-labs.com/cm/cs/who/dmr/man71.pdf |url-status=dead |archive-date=2000-08-29 |title=First Edition Unix manual 'Miscellaneous' section (PDF) |access-date=2011-05-11}}</ref>
<ref name="posixglob">{{cite web |title=The Open Group Base Specifications Issue 7 IEEE Std 1003.1, 2013 Edition, 2.13. Pattern Matching Notation |url=http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13}}</ref>
<ref name="posixglob">{{cite web |title=The Open Group Base Specifications Issue 7 IEEE Std 1003.1, 2013 Edition, 2.13. Pattern Matching Notation |url=http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13 |access-date=2015-10-26 |archive-date=2014-04-27 |archive-url=https://web.archive.org/web/20140427082439/http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13 |url-status=live }}</ref>
<ref name="linuxglob7">{{cite web |title=Linux Programmer's Manual, GLOB(7) |url=http://man7.org/linux/man-pages/man7/glob.7.html}}</ref>
<ref name="linuxglob7">{{cite web |title=Linux Programmer's Manual, GLOB(7) |url=http://man7.org/linux/man-pages/man7/glob.7.html |access-date=2015-10-26 |archive-date=2015-10-31 |archive-url=https://web.archive.org/web/20151031215157/http://man7.org/linux/man-pages/man7/glob.7.html |url-status=live }}</ref>
<ref name="bashpat">{{cite web |website=Bash Reference Manual |title=Pattern Matching |url=https://www.gnu.org/software/bash/manual/html_node/Pattern-Matching.html}}</ref>
<ref name="bashpat">{{cite web |website=Bash Reference Manual |title=Pattern Matching |url=https://www.gnu.org/software/bash/manual/html_node/Pattern-Matching.html |access-date=2016-01-11 |archive-date=2016-02-11 |archive-url=https://web.archive.org/web/20160211205104/http://www.gnu.org/software/bash/manual/html_node/Pattern-Matching.html |url-status=live }}</ref>
<ref name="pwshcmdlet">{{cite web |title=Supporting Wildcard Characters in Cmdlet Parameters |url=https://docs.microsoft.com/en-us/powershell/developer/cmdlet/supporting-wildcard-characters-in-cmdlet-parameters|publisher=Microsoft Developer Network |website=[[Microsoft]]}}</ref>
<ref name="pwshcmdlet">{{cite web |title=Supporting Wildcard Characters in Cmdlet Parameters |url=https://docs.microsoft.com/en-us/powershell/scripting/developer/cmdlet/supporting-wildcard-characters-in-cmdlet-parameters|publisher=Microsoft Developer Network |website=[[Microsoft]]|date=18 December 2023 }}</ref>
<ref name="transact-sql-like">{{cite web |title=LIKE (Transact-SQL) |url=https://docs.microsoft.com/en-us/sql/t-sql/language-elements/like-transact-sql}}</ref>
<ref name="transact-sql-like">{{cite web |title=LIKE (Transact-SQL) |date=23 May 2023 |url=https://docs.microsoft.com/en-us/sql/t-sql/language-elements/like-transact-sql |access-date=2017-08-01 |archive-date=2017-08-02 |archive-url=https://web.archive.org/web/20170802044711/https://docs.microsoft.com/en-us/sql/t-sql/language-elements/like-transact-sql |url-status=live }}</ref>
<ref name="ABSGlob">The "Advanced Bash-Scripting Guide, Chapter 19.2: Globbing" (Mendel Cooper, 2003) has a concise set of examples of filename globbing patterns.</ref>
<ref name="ABSGlob">The "Advanced Bash-Scripting Guide, Chapter 19.2: Globbing" (Mendel Cooper, 2003) has a concise set of examples of filename globbing patterns.</ref>
<ref name="winWildcard">{{cite web |url=http://msdn.microsoft.com/en-us/library/e1w828yy.aspx |title=Wildcard Expansion |publisher=Microsoft Developer Network |year=2013}}</ref>
<ref name="winWildcard">{{cite web |url=http://msdn.microsoft.com/en-us/library/e1w828yy.aspx |title=Wildcard Expansion |publisher=Microsoft Developer Network |year=2013 |access-date=2013-10-16 |archive-date=2014-08-22 |archive-url=https://web.archive.org/web/20140822042708/http://msdn.microsoft.com/en-us/library/e1w828yy.aspx |url-status=live }}</ref>
<!-- unused ref <ref name="winWcExp">{{cite web |url=http://msdn.microsoft.com/en-us/library/8bch7bkk.aspx |title=Expanding Wildcard Arguments |publisher=Microsoft Developer Network |date=2013}}</ref> -->
<!-- unused ref <ref name="winWcExp">{{cite web |url=http://msdn.microsoft.com/en-us/library/8bch7bkk.aspx |title=Expanding Wildcard Arguments |publisher=Microsoft Developer Network |date=2013}}</ref> -->
<ref name="dlang">{{cite web |url=http://dlang.org/phobos/std_path.html#.globMatch |title=std.path - D Programming Language - Digital Mars |publisher=dlang.org |access-date=2014-09-08}}</ref>
<ref name="dlang">{{cite web |url=http://dlang.org/phobos/std_path.html#.globMatch |title=std.path - D Programming Language - Digital Mars |publisher=dlang.org |access-date=2014-09-08 |archive-date=2014-09-08 |archive-url=https://web.archive.org/web/20140908203040/http://dlang.org/phobos/std_path.html#.globMatch |url-status=live }}</ref>
<ref name="minimatch">{{Cite web |url=https://github.com/isaacs/minimatch |title=isaacs/minimatch |website=GitHub |access-date=2016-08-10}}</ref>
<ref name="minimatch">{{Cite web |url=https://github.com/isaacs/minimatch |title=isaacs/minimatch |website=GitHub |access-date=2016-08-10 |archive-date=2016-07-28 |archive-url=https://web.archive.org/web/20160728063829/https://github.com/isaacs/minimatch |url-status=live }}</ref>
<ref name="micromatch">{{Cite web |url=https://github.com/jonschlinkert/micromatch |title=jonschlinkert/micromatch |website=GitHub |access-date=2017-04-04}}</ref>
<ref name="micromatch">{{Cite web |url=https://github.com/jonschlinkert/micromatch |title=jonschlinkert/micromatch |website=GitHub |access-date=2017-04-04 |archive-date=2016-02-11 |archive-url=https://web.archive.org/web/20160211020457/https://github.com/jonschlinkert/micromatch |url-status=live }}</ref>
<ref name="golang">{{cite web |url=http://golang.org/pkg/path/filepath/#Glob |title=Package filepath - The Go Programming Language |publisher=Golang.org |access-date=2011-05-11}}</ref>
<ref name="golang">{{cite web |url=http://golang.org/pkg/path/filepath/#Glob |title=Package filepath - The Go Programming Language |publisher=Golang.org |access-date=2011-05-11 |archive-date=2011-05-25 |archive-url=https://web.archive.org/web/20110525035625/http://golang.org/pkg/path/filepath/#Glob |url-status=live }}</ref>
<ref name="java">{{cite web |url=http://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob |title=File Operations |publisher=Oracle |access-date=2013-12-16}}</ref>
<ref name="java">{{cite web |url=http://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob |title=File Operations |publisher=Oracle |access-date=2013-12-16 |archive-date=2013-09-20 |archive-url=https://web.archive.org/web/20130920172102/http://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob |url-status=live }}</ref>
<ref name="hs">{{cite web |url=http://hackage.haskell.org/package/Glob-0.7.4/docs/System-FilePath-Glob.html |title=Glob-0.7.4: Globbing library |access-date=2014-05-07}}</ref>
<ref name="hs">{{cite web |url=http://hackage.haskell.org/package/Glob-0.7.4/docs/System-FilePath-Glob.html |title=Glob-0.7.4: Globbing library |access-date=2014-05-07 |archive-date=2014-05-08 |archive-url=https://web.archive.org/web/20140508025811/http://hackage.haskell.org/package/Glob-0.7.4/docs/System-FilePath-Glob.html |url-status=live }}</ref>
<ref name="pl">{{cite web |url=http://perldoc.perl.org/File/Glob.html |title=File::Glob - Perl extension for BSD glob routine |publisher=perldoc.perl.org |access-date=2011-05-11}}</ref>
<ref name="pl">{{cite web |url=http://perldoc.perl.org/File/Glob.html |title=File::Glob - Perl extension for BSD glob routine |publisher=perldoc.perl.org |access-date=2011-05-11}}</ref>
<ref name="php">{{cite web |url=http://www.php.net/glob |title=glob - Manual |publisher=PHP |date=2011-05-06 |access-date=2011-05-11}}</ref>
<ref name="php">{{cite web |url=http://www.php.net/glob |title=glob - Manual |publisher=PHP |date=2011-05-06 |access-date=2011-05-11 |archive-date=2017-11-13 |archive-url=https://web.archive.org/web/20171113011131/http://php.net/glob |url-status=live }}</ref>
<ref name="pyglob">{{cite web |url=https://docs.python.org/library/glob.html |title=10.7. glob — Unix style pathname pattern expansion — Python v2.7.1 documentation |publisher=Docs.python.org |access-date=2011-05-11}}</ref>
<ref name="pyglob">{{cite web |url=https://docs.python.org/library/glob.html |title=10.7. glob — Unix style pathname pattern expansion — Python v2.7.1 documentation |publisher=Docs.python.org |access-date=2011-05-11 |archive-date=2011-05-16 |archive-url=https://web.archive.org/web/20110516205728/http://docs.python.org/library/glob.html |url-status=live }}</ref>
<ref name="isc-glob">{{cite web |url=http://www.isc.org/sources/devel/func/glob.txt |title='Globbing' library routine |access-date=2011-05-11 |archive-url=https://web.archive.org/web/20071219090708/http://www.isc.org/sources/devel/func/glob.txt |archive-date=2007-12-19}}</ref>
<ref name="isc-glob">{{cite web |url=http://www.isc.org/sources/devel/func/glob.txt |title='Globbing' library routine |access-date=2011-05-11 |archive-url=https://web.archive.org/web/20071219090708/http://www.isc.org/sources/devel/func/glob.txt |archive-date=2007-12-19}}</ref>
<ref name="rbdir">{{cite web |url=http://www.ruby-doc.org/core/classes/Dir.html#M000629 |title=Class: Dir |publisher=Ruby-doc.org |access-date=2011-05-11}}</ref>
<ref name="rbdir">{{cite web |url=http://www.ruby-doc.org/core/classes/Dir.html#M000629 |title=Class: Dir |publisher=Ruby-doc.org |access-date=2011-05-11 |archive-date=2011-05-15 |archive-url=https://web.archive.org/web/20110515140813/http://ruby-doc.org/core/classes/Dir.html#M000629 |url-status=live }}</ref>
<ref name="tcl">{{cite web |url=http://www.tcl.tk/man/tcl8.5/TclCmd/glob.htm |title=TCL glob manual page |access-date=2011-11-16}}</ref>
<ref name="tcl">{{cite web |url=http://www.tcl.tk/man/tcl8.5/TclCmd/glob.htm |title=TCL glob manual page |access-date=2011-11-16 |archive-date=2011-12-08 |archive-url=https://web.archive.org/web/20111208112250/http://www.tcl.tk/man/tcl8.5/TclCmd/glob.htm |url-status=live }}</ref>
<ref name="dotnet-glob">{{Cite web |url=https://github.com/kthompson/glob |title=kthompson/glob |website=GitHub |access-date=2020-11-06}}</ref>
<ref name="dotnet-glob">{{Cite web |url=https://github.com/kthompson/glob |title=kthompson/glob |website=GitHub |access-date=2020-11-06 |archive-date=2020-10-26 |archive-url=https://web.archive.org/web/20201026174625/https://github.com/kthompson/glob |url-status=live }}</ref>
<ref name="dotnet.glob">{{Cite web |url=https://github.com/dazinator/DotNet.Glob |title=dazinator/dotnet.glob |website=GitHub |access-date=2022-06-22 |archive-date=2022-06-22 |archive-url=https://web.archive.org/web/20220622181350/https://github.com/dazinator/DotNet.Glob |url-status=live }}</ref>
}}
}}


Revision as of 18:03, 25 October 2024

In computer programming, glob (/ɡlɒb/) patterns specify sets of filenames with wildcard characters. For example, the Unix Bash shell command mv *.txt textfiles/ moves all files with names ending in .txt from the current directory to the directory textfiles. Here, * is a wildcard and *.txt is a glob pattern. The wildcard * stands for "any string of any length including empty, but excluding the path separator characters (/ in unix and \ in windows)".

The other common wildcard is the question mark (?), which stands for one character. For example, mv ?.txt shorttextfiles/ will move all files named with a single character followed by .txt from the current directory to directory shorttextfiles, while ??.txt would match all files whose name consists of 2 characters followed by .txt.

In addition to matching filenames, globs are also used widely for matching arbitrary strings (wildcard matching). In this capacity a common interface is fnmatch.

Origin

A screenshot of the original 1971 Unix reference page for glob – the owner is dmr, short for Dennis Ritchie.

The glob command, short for global, originates in the earliest versions of Bell Labs' Unix.[1] The command interpreters of the early versions of Unix (1st through 6th Editions, 1969–1975) relied on a separate program to expand wildcard characters in unquoted arguments to a command: /etc/glob. That program performed the expansion and supplied the expanded list of file paths to the command for execution.

Glob was originally written in the B programming language. It was the first piece of mainline Unix software to be developed in a high-level programming language.[2] Later, this functionality was provided as a C library function, glob(), used by programs such as the shell. It is usually defined based on a function named fnmatch(), which tests for whether a string matches a given pattern - the program using this function can then iterate through a series of strings (usually filenames) to determine which ones match. Both functions are a part of POSIX: the functions defined in POSIX.1 since 2001, and the syntax defined in POSIX.2.[3][4] The idea of defining a separate match function started with wildmat (wildcard match), a simple library to match strings against Bourne Shell globs.

Traditionally, globs do not match hidden files in the form of Unix dotfiles; to match them the pattern must explicitly start with .. For example, * matches all visible files while .* matches all hidden files.

Syntax

The most common wildcards are *, ?, and […].

Wildcard Description Example Matches Does not match
* matches any number of any characters including none Law* Law, Laws, or Lawyer GrokLaw, La, or aw
*Law* Law, GrokLaw, or Lawyer. La, or aw
? matches any single character ?at Cat, cat, Bat or bat at
[abc] matches one character given in the bracket [CB]at Cat or Bat cat, bat or CBat
[a-z] matches one character from the (locale-dependent) range given in the bracket Letter[0-9] Letter0, Letter1, Letter2 up to Letter9 Letters, Letter or Letter10

Normally, the path separator character (/ on Linux/Unix, MacOS, etc. or \ on Windows) will never be matched. Some shells, such as Bash have functionality allowing users to circumvent this.[5]

Unix-like

On Unix-like systems *, ? is defined as above while […] has two additional meanings:[6][7]

Wildcard Description Example Matches Does not match
[!abc] matches one character that is not given in the bracket [!C]at Bat, bat, or cat Cat
[!a-z] matches one character that is not from the range given in the bracket Letter[!3-5] Letter1, Letter2, Letter6 up to Letter9 and Letterx etc. Letter3, Letter4, Letter5 or Letterxx

The ranges are also allowed to include pre-defined character classes, equivalence classes for accented characters, and collation symbols for hard-to-type characters. They are defined to match up with the brackets in POSIX regular expressions.[6][7]

Unix globbing is handled by the shell per POSIX tradition. Globbing is provided on filenames at the command line and in shell scripts.[8] The POSIX-mandated case statement in shells provides pattern-matching using glob patterns.

Some shells (such as the C shell and Bash) support additional syntax known as alternation or brace expansion. Because it is not part of the glob syntax, it is not provided in case. It is only expanded on the command line before globbing.

The Bash shell also supports the following extensions:[9]

  • Extended globbing (extglob): allows other pattern matching operators to be used to match multiple occurrences of a pattern enclosed in parentheses, essentially providing the missing kleene star and alternation for describing regular languages. It can be enabled by setting the extglob shell option. This option came from ksh93.[10] The GNU fnmatch and glob has an identical extension.[3]
  • globstar: allows ** on its own as a name component to recursively match any number of layers of non-hidden directories.[10] Also supported by the JavaScript libraries and Python's glob.

Windows and DOS

The dir command with a glob pattern in IBM PC DOS 1.0.

The original DOS was a clone of CP/M designed to work on Intel's 8088 and 8086 processors. Windows shells, following DOS, do not traditionally perform any glob expansion in arguments passed to external programs. Shells may use an expansion for their own builtin commands:

  • Windows PowerShell has all the common syntax defined as stated above without any additions.[11]
  • COMMAND.COM and cmd.exe have most of the common syntax with some limitations: There is no […] and for COMMAND.COM the * may only appear at the end of the pattern. It can not appear in the middle of a pattern, except immediately preceding the filename extension separator dot.

Windows and DOS programs receive a long command-line string instead of argv-style parameters, and it is their responsibility to perform any splitting, quoting, or glob expansion. There is technically no fixed way of describing wildcards in programs since they are free to do what they wish. Two common glob expanders include:[12]

  • The Microsoft C Runtime (msvcrt) command-line expander, which only supports ? and *.[13] Both ReactOS (crt/misc/getargs.c) and Wine (msvcrt/data.c) contain a compatible open-source implementation of __getmainargs, the function operating under-the-hood, in their core CRT.
  • The Cygwin and MSYS dcrt0.cc command-line expander, which uses the unix-style glob() routine under-the-hood, after splitting the arguments.

Most other parts of Windows, including the Indexing Service, use the MS-DOS style of wildcards found in CMD. A relic of the 8.3 filename age, this syntax pays special attention to dots in the pattern and the text (filename). Internally this is done using three extra wildcard characters, <>". On the Windows API end, the glob() equivalent is FindFirstFile, and fnmatch() corresponds to its underlying RtlIsNameInExpression.[14] (Another fnmatch analogue is PathMatchSpec.) Both open-source msvcrt expanders use FindFirstFile, so 8.3 filename quirks will also apply in them.

SQL

The SQL LIKE operator has an equivalent to ? and * but not […].

Common wildcard SQL wildcard Description
? _ matches any single character
* % matches any number of any characters including none

Standard SQL uses a glob-like syntax for simple string matching in its LIKE operator, although the term "glob" is not generally used in the SQL community. The percent sign (%) matches zero or more characters and the underscore (_) matches exactly one.

Many implementations of SQL have extended the LIKE operator to allow a richer pattern-matching language, incorporating character ranges ([…]), their negation, and elements of regular expressions.[15]

Compared to regular expressions

Globs do not include syntax for the Kleene star which allows multiple repetitions of the preceding part of the expression; thus they are not considered regular expressions, which can describe the full set of regular languages over any given finite alphabet.[16]

Common wildcard Equivalent regular expression
? .
* .*

Globs attempt to match the entire string (for example, S*.DOC matches S.DOC and SA.DOC, but not POST.DOC or SURREY.DOCKS), whereas, depending on implementation details, regular expressions may match a substring.

Implementing as regular expressions

The original Mozilla proxy auto-config implementation, which provides a glob-matching function on strings, uses a replace-as-RegExp implementation as above. The bracket syntax happens to be covered by regex in such an example.

Python's fnmatch uses a more elaborate procedure to transform the pattern into a regular expression.[17]

Other implementations

Beyond their uses in shells, globs patterns also find use in a variety of programming languages, mainly to process human input. A glob-style interface for returning files or an fnmatch-style interface for matching strings are found in the following programming languages:

  • C# has multiple libraries available through NuGet such as Glob.[18] or DotNet.Glob.[19]
  • D has a globMatch function in the std.path module.[20]
  • JavaScript has a library called minimatch which is used internally by npm, and micromatch, a purportedly more optimized, accurate and safer globbing implementation used by Babel and yarn.[21][22]
  • Go has a Glob function in the filepath package.[23]
  • Java has a Files class containing methods that operate on glob patterns.[24]
  • Haskell has a Glob package with the main module System.FilePath.Glob. The pattern syntax is based on a subset of Zsh's. It tries to optimize the given pattern and should be noticeably faster than a naïve character-by-character matcher.[25]
  • Perl has both a glob function (as discussed in Larry Wall's book Programming Perl) and a Glob extension which mimics the BSD glob routine.[26] Perl's angle brackets can be used to glob as well: <*.log>.
  • PHP has a glob function.[27]
  • Python has a glob module in the standard library which performs wildcard pattern matching on filenames,[28] and an fnmatch module with functions for matching strings or filtering lists based on these same wildcard patterns.[17] Guido van Rossum, author of the Python programming language, wrote and contributed a glob routine to BSD Unix in 1986.[29] There were previous implementations of glob, e.g., in the ex and ftp programs in previous releases of BSD.
  • Ruby has a glob method for the Dir class which performs wildcard pattern matching on filenames.[30] Several libraries such as Rant and Rake provide a FileList class which has a glob method or use the method FileList.[] identically.
  • Rust has multiple libraries that can match glob patterns.[31]
  • SQLite has a GLOB function.
  • Tcl contains a globbing facility.[32]

See also

References

  1. ^ "First Edition Unix manual 'Miscellaneous' section (PDF)" (PDF). Archived from the original (PDF) on 2000-08-29. Retrieved 2011-05-11.
  2. ^ McIlroy, M. D. (1987). A Research Unix reader: annotated excerpts from the Programmer's Manual, 1971–1986 (PDF) (Technical report). CSTR. Bell Labs. 139.
  3. ^ a b fnmatch(3) – Linux Programmer's Manual – Library Functions
  4. ^ glob(3) – Linux Programmer's Manual – Library Functions
  5. ^ https://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching Archived 2018-03-15 at the Wayback Machine Bash Reference Manual
  6. ^ a b "The Open Group Base Specifications Issue 7 IEEE Std 1003.1, 2013 Edition, 2.13. Pattern Matching Notation". Archived from the original on 2014-04-27. Retrieved 2015-10-26.
  7. ^ a b "Linux Programmer's Manual, GLOB(7)". Archived from the original on 2015-10-31. Retrieved 2015-10-26.
  8. ^ The "Advanced Bash-Scripting Guide, Chapter 19.2: Globbing" (Mendel Cooper, 2003) has a concise set of examples of filename globbing patterns.
  9. ^ "Bash globs". greg's bash knowledgebase. Archived from the original on 2019-11-18. Retrieved 2019-11-25.
  10. ^ a b "Pattern Matching". Bash Reference Manual. Archived from the original on 2016-02-11. Retrieved 2016-01-11.
  11. ^ "Supporting Wildcard Characters in Cmdlet Parameters". Microsoft. Microsoft Developer Network. 2023-12-18.
  12. ^ "Wildcard Expansion". Microsoft Developer Network. 2013. Archived from the original on 2014-08-22. Retrieved 2013-10-16.
  13. ^ "Wildcard Expansion". docs.microsoft.com. 2022-02-08.
  14. ^ Wildcards in Windows Archived 2019-12-24 at the Wayback Machine. MSDN Devblog.
  15. ^ "LIKE (Transact-SQL)". 2023-05-23. Archived from the original on 2017-08-02. Retrieved 2017-08-01.
  16. ^ Hopcroft, John E.; Motwani, Rajeev; Ullman, Jeffrey D. (2000). Introduction to Automata Theory, Languages, and Computation (2nd ed.). Addison-Wesley.
  17. ^ a b "Lib/fnmatch.py". Python. 2021-01-20. Archived from the original on 2021-11-10. Retrieved 2021-11-10.
  18. ^ "kthompson/glob". GitHub. Archived from the original on 2020-10-26. Retrieved 2020-11-06.
  19. ^ "dazinator/dotnet.glob". GitHub. Archived from the original on 2022-06-22. Retrieved 2022-06-22.
  20. ^ "std.path - D Programming Language - Digital Mars". dlang.org. Archived from the original on 2014-09-08. Retrieved 2014-09-08.
  21. ^ "isaacs/minimatch". GitHub. Archived from the original on 2016-07-28. Retrieved 2016-08-10.
  22. ^ "jonschlinkert/micromatch". GitHub. Archived from the original on 2016-02-11. Retrieved 2017-04-04.
  23. ^ "Package filepath - The Go Programming Language". Golang.org. Archived from the original on 2011-05-25. Retrieved 2011-05-11.
  24. ^ "File Operations". Oracle. Archived from the original on 2013-09-20. Retrieved 2013-12-16.
  25. ^ "Glob-0.7.4: Globbing library". Archived from the original on 2014-05-08. Retrieved 2014-05-07.
  26. ^ "File::Glob - Perl extension for BSD glob routine". perldoc.perl.org. Retrieved 2011-05-11.
  27. ^ "glob - Manual". PHP. 2011-05-06. Archived from the original on 2017-11-13. Retrieved 2011-05-11.
  28. ^ "10.7. glob — Unix style pathname pattern expansion — Python v2.7.1 documentation". Docs.python.org. Archived from the original on 2011-05-16. Retrieved 2011-05-11.
  29. ^ "'Globbing' library routine". Archived from the original on 2007-12-19. Retrieved 2011-05-11.
  30. ^ "Class: Dir". Ruby-doc.org. Archived from the original on 2011-05-15. Retrieved 2011-05-11.
  31. ^ "#glob - Lib.rs". lib.rs. Archived from the original on 2021-11-12. Retrieved 2021-11-12.
  32. ^ "TCL glob manual page". Archived from the original on 2011-12-08. Retrieved 2011-11-16.
Retrieved from "https://en.wikipedia.org/enwiki/w/index.php?title=Glob_(programming)&oldid=1253382850"