ctags regex for multiple declarations in one line - ctags

Question

I'm writing a .ctags file for a custom language... Like most languages, it allows for multiple variable declarations in one line.. i.e.:
int a, b, c;
I have a basic regex which recognizes 'a':
--regex-mylang=/^[ \t]*int[ \t]*([a-zA-Z_0-9]+)/\1/v,variable/
How do I modify this to have it match 'b' and 'c', as well? I can't find anything in ctags documentation that deals with multiple matches in a single line.

Answer 1

The latest Universal-ctags can capture them.
[jet#localhost]/tmp% cat input.x
int a, b, c;
[jet#localhost]/tmp% cat x.ctags
--langdef=X
--map-X=.x
--kinddef-X=v,var,variables
--_tabledef-X=main
--_tabledef-X=vardef
--_mtable-regex-X=main/int[ \t]+//{tenter=vardef}
--_mtable-regex-X=main/.//
--_mtable-regex-X=vardef/([a-zA-Z0-9]+)/\1/v/
--_mtable-regex-X=vardef/;//{tleave}
--_mtable-regex-X=vardef/.//
[jet#localhost]/tmp% u-ctags --options=x.ctags -o - ./input.x
a ./input.x /^int a, b, c;$/;" v
b ./input.x /^int a, b, c;$/;" v
c ./input.x /^int a, b, c;$/;" v
See https://docs.ctags.io/en/latest/optlib.html#advanced-pattern-matching-with-multiple-regex-tables for more details.

Answer 2

After going through this for a few hours, I'm convinced it can't be done. No matter what, the regular expression will only expand to one tag per line. Even if you put \1 \2 \3 ... as the expansion, that would just cause a tag consisting of multiple matches, instead of one tag per match.
It parses the C example correctly because inside the ctags source code it uses an actual code parser, not a regexp.

Answer 3

You're trying to do parsing with a regex, which is not generally possible. Parsing requires the equivalent of storing information on a stack, but a regular expression can embody only a finite number of different states.

Answer 4

it can be partialy done with the Universal Ctags and with the help of {_multiline=N} and {scope} flag. The N is group number which position is saved in generated tags file.
For more information look here: docs/optlib.rst
Configuration: mylang.ctags
--langmap=mylang:.txt
--regex-mylang=/^[[:blank:]]*(int)[[:blank:]]/\1/{placeholder}{scope=set}{_multiline=1}
--regex-mylang=/(;)/\1/{placeholder}{scope=clear}
--regex-mylang=/[[:blank:]]*([[:alnum:]]+)[[:blank:]]*,?/\1/v,variable/{_multiline=1}{scope=ref}
Test file: test.txt
void main() {
int a, b, c, d;
}
Generate tags with: ctags --options=mylang.ctags test.txt
Generated tags file:
!_TAG_FILE_FORMAT 2 /extended format; --format=1 will not append ;" to lines/
!_TAG_FILE_SORTED 1 /0=unsorted, 1=sorted, 2=foldcase/
!_TAG_OUTPUT_MODE u-ctags /u-ctags or e-ctags/
!_TAG_PROGRAM_AUTHOR Universal Ctags Team //
!_TAG_PROGRAM_NAME Universal Ctags /Derived from Exuberant Ctags/
!_TAG_PROGRAM_URL https://ctags.io/ /official site/
!_TAG_PROGRAM_VERSION 0.0.0 /cb4476eb/
a test.txt /^ int a, b, c, d;$/;" v
b test.txt /^ int a, b, c, d;$/;" v
c test.txt /^ int a, b, c, d;$/;" v
d test.txt /^ int a, b, c, d;$/;" v
int test.txt /^ int a, b, c, d;$/;" v
main test.txt /^void main() {$/;" v
void test.txt /^void main() {$/;" v

Answer 5

--regex-perl=/^\s*?use\s+(\w+[\w\:]*?\w*?)/\1/u,use,uses/
--regex-perl=/^\s*?require\s+(\w+[\w\:]*?\w*?)/\1/r,require,requires/
--regex-perl=/^\s*?has\s+['"]?(\w+)['"]?/\1/a,attribute,attributes/
--regex-perl=/^\s*?\*(\w+)\s*?=/\1/a,aliase,aliases/
--regex-perl=/->helper\(\s?['"]?(\w+)['"]?/\1/h,helper,helpers/
--regex-perl=/^\s*?our\s*?[\$#%](\w+)/\1/o,our,ours/
--regex-perl=/^=head1\s+(.+)/\1/p,pod,Plain Old Documentation/
--regex-perl=/^=head2\s+(.+)/-- \1/p,pod,Plain Old Documentation/
--regex-perl=/^=head[3-5]\s+(.+)/---- \1/p,pod,Plain Old Documentation/