ctags-client-tools

Hints for developing a tool using ctags command and tags output

Version:0.0.0
Manual group:Universal-ctags
Manual section:7

SYNOPSIS

ctags [options] [file(s)]
etags [options] [file(s)]

DESCRIPTION

Client tool means a tool running the ctags command and/or reading a tags file generated by ctags command. This man page gathers hints for people who develop client tools.

PSEUDO-TAGS

Pseudo-tags, stored in a tag file, indicate how ctags generated the tags file: whether the tags file is sorted or not, which version of tags file format is used, the name of tags generator, and so on. The opposite term for pseudo-tags is regular-tags. A regular-tag is for a language object in an input file. A pseudo-tag is for the tags file itself. Client tools may use pseudo-tags as reference for processing regular-tags.

A pseudo-tag is stored in a tags file in the same format as regular-tags as described in tags(5), except that pseudo-tag names are prefixed with “!_”. For the general information about pseudo-tags, see “TAG FILE INFORMATION” in tags(5).

An example of a pseudo tag:

!_TAG_PROGRAM_NAME      Universal Ctags /Derived from Exuberant Ctags/

The value, “2”, associated with the pseudo tag “TAG_PROGRAM_NAME”, is used in the field for input file. The description, “Derived from Exuberant Ctags”, is used in the field for pattern.

Universal-ctags extends the naming scheme of the classical pseudo-tags available in Exuberant-ctags for emitting language specific information as pseudo tags:

!_{pseudo-tag-name}!{language-name}     {associated-value}      /{description}/

The language-name is appended to the pseudo-tag name with a separator, “!”.

An example of pseudo tag with a language suffix:

!_TAG_KIND_DESCRIPTION!C        f,function      /function definitions/

This pseudo-tag says “the function kind of C language is enabled when generating this tags file.” --pseudo-tags is the option for enabling/disabling individual pseudo-tags. When enabling/disabling a pseudo tag with the option, specify the tag name only “TAG_KIND_DESCRIPTION”, without the prefix (“!_”) or the suffix (“!C”).

Options for Pseudo-tags

--extras=+p (or --extras=+{pseudo})

Forces writing pseudo-tags.

ctags emits pseudo-tags by default when writing tags to a regular file (e.g. “tags’.) However, when specifying -o - or -f - for writing tags to standard output, ctags doesn’t emit pseudo-tags. --extras=+p or --extras=+{pseudo} will force pseudo-tags to be written.

--list-pseudo-tags

Lists available types of pseudo-tags and shows whether they are enabled or disabled.

Running ctags with --list-pseudo-tags option lists available pseudo-tags. Some of pseudo-tags newly introduced in Universal-ctags project are disabled by default. Use --pseudo-tags=... to enable them.

--pseudo-tags=[+|-]names|*

Specifies a list of pseudo-tag types to include in the output.

The parameters are a set of pseudo tag names. Valid pseudo tag names can be listed with --list-pseudo-tags. Surround each name in the set with braces, like “{TAG_PROGRAM_AUTHOR}”. You don’t have to include the “!_” pseudo tag prefix when specifying a name in the option argument for --pseudo-tags= option.

pseudo-tags don’t have a notation using one-letter flags.

If a name is preceded by either the ‘+’ or ‘-‘ characters, that tags’s effect has been added or removed. Otherwise the names replace any current settings. All entries are included if ‘*’ is given.

--fields=+E (or --fields=+{extras})

Attach “extras:pseudo” field to pseudo-tags.

An example of pseudo tags with the field:

!_TAG_PROGRAM_NAME      Universal Ctags /Derived from Exuberant Ctags/  extras:pseudo

If the name of a normal tag in a tag file starts with “!_”, a client tool cannot distinguish whether the tag is a regular-tag or pseudo-tag. The fields attached with this option help the tool distinguish them.

List of notable pseudo-tags

Running ctags with --list-pseudo-tags option lists available types of pseudo-tags with short descriptions. This subsection shows hints for using notable ones.

TAG_EXTRA_DESCRIPTION (new in Universal-ctags)

Indicates the names and descriptions of enabled extras:

!_TAG_EXTRA_DESCRIPTION       {extra-name}    /description/
!_TAG_EXTRA_DESCRIPTION!{language-name}       {extra-name}    /description/

If your tool relies on some extra tags (extras), refer to the pseudo-tags of this type. A tool can reject the tags file that doesn’t include expected extras, and raise an error in an early stage of processing.

An example of the pseudo-tags:

$ ctags --extras=+p --pseudo-tags='{TAG_EXTRA_DESCRIPTION}' -o - input.c
!_TAG_EXTRA_DESCRIPTION       anonymous       /Include tags for non-named objects like lambda/
!_TAG_EXTRA_DESCRIPTION       fileScope       /Include tags of file scope/
!_TAG_EXTRA_DESCRIPTION       pseudo  /Include pseudo tags/
!_TAG_EXTRA_DESCRIPTION       subparser       /Include tags generated by subparsers/
...

A client tool can know “{anonymous}”, “{fileScope}”, “{pseudo}”, and “{subparser}” extras are enabled from the output.

TAG_FIELD_DESCRIPTION (new in Universal-ctags)

Indicates the names and descriptions of enabled fields:

!_TAG_FIELD_DESCRIPTION       {field-name}    /description/
!_TAG_FIELD_DESCRIPTION!{language-name}       {field-name}    /description/

If your tool relies on some fields, refer to the pseudo-tags of this type. A tool can reject a tags file that doesn’t include expected fields, and raise an error in an early stage of processing.

An example of the pseudo-tags:

$ ctags --fields-C=+'{macrodef}' --extras=+p --pseudo-tags='{TAG_FIELD_DESCRIPTION}' -o - input.c
!_TAG_FIELD_DESCRIPTION       file    /File-restricted scoping/
!_TAG_FIELD_DESCRIPTION       input   /input file/
!_TAG_FIELD_DESCRIPTION       name    /tag name/
!_TAG_FIELD_DESCRIPTION       pattern /pattern/
!_TAG_FIELD_DESCRIPTION       typeref /Type and name of a variable or typedef/
!_TAG_FIELD_DESCRIPTION!C     macrodef        /macro definition/
...

A client tool can know “{file}”, “{input}”, “{name}”, “{pattern}”, and “{typeref}” fields are enabled from the output. The fields are common in languages. In addition to the common fields, the tool can known “{macrodef}” field of C language is also enabled.

TAG_FILE_ENCODING (new in Universal-ctags)
TBW
TAG_FILE_FORMAT
See also tags(5).
TAG_FILE_SORTED
See also tags(5).
TAG_KIND_DESCRIPTION (new in Universal-ctags)

Indicates the names and descriptions of enabled kinds:

!_TAG_KIND_DESCRIPTION!{language-name}        {kind-letter},{kind-name}       /description/

If your tool relies on some kinds, refer to the pseudo-tags of this type. A tool can reject the tags file that doesn’t include expected kinds, and raise an error in an early stage of processing.

Kinds are language specific, so a language name is always appended to the tag name as suffix.

An example of the pseudo-tags:

$ ctags --extras=+p --kinds-C=vfm --pseudo-tags='{TAG_KIND_DESCRIPTION}' -o - input.c
!_TAG_KIND_DESCRIPTION!C      f,function      /function definitions/
!_TAG_KIND_DESCRIPTION!C      m,member        /struct, and union members/
!_TAG_KIND_DESCRIPTION!C      v,variable      /variable definitions/
...

A client tool can know “{function}”, “{member}”, and “{variable}” kinds of C language are enabled from the output.

TAG_KIND_SEPARATOR (new in Universal-ctags)
TBW
TAG_OUTPUT_FILESEP (new in Universal-ctags)
TBW
TAG_OUTPUT_MODE (new in Universal-ctags)
TBW
TAG_PATTERN_LENGTH_LIMIT (new in Universal-ctags)
TBW
TAG_PROC_CWD (new in Universal-ctags)

Indicates the working directory of ctags during processing.

This pseudo-tag helps a client tool solve the absolute paths for the input files for tag entries even when they are tagged with relative paths.

An example of the pseudo-tags:

$ cat tags
!_TAG_PROC_CWD        /tmp/   //
main  input.c /^int main (void) { return 0; }$/;"     f       typeref:typename:int
...

From the regular tag for “main”, the client tool can know the “main” is at “input.c”. However, it is a relative path. So if the directory where ctags run and the directory where the client tool runs are different, the client tool cannot find “input.c” from the file system. In that case, TAG_PROC_CWD gives the tool a hint; “input.c” may be at “/tmp”.

TAG_PROGRAM_NAME
TBW

REDUNDANT-KINDS

TBW

MULTIPE-LANGUAGES FOR AN INPUT FILE

TBW

UTILIZING READTAGS

See readtags(1) to know how to use readtags. This section is for discussing some notable topics for client tools.

Build Filter/Sorter Expressions

Certain escape sequences in expressions are recognized by readtags. For example, when searching for a tag that matches a\?b, if using a filter expression like '(eq? $name "a\?b")', since \? is translated into a single ? by readtags, it actually searches for a?b.

Another problem is if a single quote appear in filter expressions (which is also wrapped by single quotes), it terminates the expression, producing broken expressions, and may even cause unintended shell injection. Single quotes can be escaped using '"'"'.

So, client tools need to:

  • Replace \ by \\
  • Replace ' by '"'"'

inside the expressions. If the expression also contains strings, " in the strings needs to be replaced by \".

Client tools written in Lisp could build the expression using lists. prin1 (in Common Lisp style Lisps) and write (in Scheme style Lisps) can translate the list into a string that can be directly used. For example, in EmacsLisp:

(let ((name "hi"))
  (prin1 `(eq? $name ,name)))
=> "(eq\\? $name "hi")"

The “?” is escaped, and readtags can handle it. Scheme style Lisps should do proper escaping so the expression readtags gets is just the expression passed into write. Common Lisp style Lisps may produce unrecognized escape sequences by readtags, like \#. Readtags provides some aliases for these Lisps:

  • Use true for #t.
  • Use false for #f.
  • Use nil or () for ().
  • Use (string->regexp "PATTERN") for #/PATTERN/. Use (string->regexp "PATTERN" :case-fold true) for #/PATTERN/i. Notice that string->regexp doesn’t require escaping “/” in the pattern.

Notice that even when the client tool uses this method, ' still needs to be replaced by '"'"' to prevent broken expressions and shell injection.

Parse Readtags Output

In the output of readtags, tabs can appear in all field values (e.g., the tag name itself could contain tabs), which makes it hard to split the line into fields. Client tools should use the -E option, which keeps the escape sequences in the tags file, so the only field that could contain tabs is the pattern field.

Client tools could then split the line using the following steps:

  • Find the first 2 tabs in the line, so we get the name and input field.
  • From the 2nd tab
    • If a “/” follows, then the pattern delimiter is “/”.
    • If a “?” follows, then the pattern delimiter is “?”.
    • If a number follows, then if after the end of the number is a “;/”, then the pattern delimiter is “/”; If it’s a “;?”, then the delimiter is “?”.
  • Find the 3rd tab, and count the delimiters between it and the 2nd tab. Notice that delimiters can be escaped, so only the ones with a even number (including 0) of backslashes before should be counted.
  • If there are even numbers of delimiters, then the 3rd tab is the end of the pattern. If not, keep searching tabs forward until this condition is satisfied.
  • From here, split the rest of the line into fields by tabs.

Then, the escape sequences in fields other than the pattern field should be translated. See “Proposal” in tags(5) to know about all the escape sequences. The pattern field needs no special treatment. It can be directly used by editors supporting Ex commands.