can't figure out what's wrong with ctag regex for assembly macro extraction - ctags

This is part of an assembly code (u-boot, arch/arm/include/debug/8250.S)
.macro addruart, rp, rv, tmp
ldr \rp, =CONFIG_DEBUG_UART_PHYS
ldr \rv, =CONFIG_DEBUG_UART_VIRT
.endm
I wanted to extrac tags for this type of macro defintion. So I referred to How to write .ctags file for assembly extention?
But somehow my trial doesn't work.
I tried all these commands below but all failed to show the tag (addruart).
Suppose this 8250.S file is under /tmp and contains only those 4 lines. And actually there are two tabs for indentation of each line. and between .macro and adduart is a tab too.
ckim#ckim-ubuntu:~/U-BOOT/u-boot$ ctags --regex-Asm='/\.macro[\t ]+\(([a-zA-Z_0-9]+)\)/\1/m,macro/' -o - /tmp/8250.S
ckim#ckim-ubuntu:~/U-BOOT/u-boot$ ctags --regex-Asm='/\.macro[\t ][\t ]*\([a-zA-Z_0-9]+\)/\1/m,macro/' -o - /tmp/8250.S
ckim#ckim-ubuntu:~/U-BOOT/u-boot$ ctags --regex-Asm='/\.macro[\t ][\t ]*\([a-zA-Z_0-9][a-zA-Z_0-9]*\)/\1/m,macro/' -o - /tmp/8250.S
ckim#ckim-ubuntu:~/U-BOOT/u-boot$ ctags --regex-Asm='/\.macro[\t ]\([a-zA-Z_0-9][a-zA-Z_0-9]*\)/\1/m,macro/' -o - /tmp/8250.S
ckim#ckim-ubuntu:~/U-BOOT/u-boot$ ctags --regex-Asm='/\.macro\t\([a-zA-Z_0-9][a-zA-Z_0-9]*\)/\1/m,macro/' -o - /tmp/8250.S
ckim#ckim-ubuntu:~/U-BOOT/u-boot$ ctags --regex-Asm='/\.macro\([a-zA-Z_0-9][a-zA-Z_0-9]*\)/\1/m,macro/' -o - /tmp/8250.S
ckim#ckim-ubuntu:~/U-BOOT/u-boot$ ctags --regex-Asm='/\.macro\([a-zA-Z_0-9][a-zA-Z_0-9]*\)/\1/m,macro/' -o - /tmp/8250.S
ckim#ckim-ubuntu:~/U-BOOT/u-boot$
How should I do it?
my ctags version :
Exuberant Ctags 5.9~svn20110310, Copyright (C) 1996-2009 Darren Hiebert
Addresses: <dhiebert#users.sourceforge.net>, http://ctags.sourceforge.net
Optional compiled features: +wildcards, +regex

ckim#ckim-ubuntu:~/U-BOOT/u-boot$ ctags --regex-Asm='/\.macro[\t ]+\(([a-zA-Z_0-9]+)\)/\1/m,macro/' -o - /tmp/8250.S
This should be:
--regex-Asm='/\.macro[\t ]+([a-zA-Z_0-9]+)/\1/m,macro/'
If you don't want to tune the regex patterns, you can replace your ctags with another implementation.
Universal Ctags (https://ctags.io) is an unofficial fork of Exuberant Ctags.
The latest u-ctags recognizes macro definitions defined with gas's syntax.
You don't have to add regex options for extracting them.
$ cat /tmp/foo.S
cat /tmp/foo.S
.macro addruart, rp, rv, tmp
ldr \rp, =CONFIG_DEBUG_UART_PHYS
ldr \rv, =CONFIG_DEBUG_UART_VIRT
.endm
$ ./ctags --options=NONE -o - /tmp/foo.S
./ctags --options=NONE -o - /tmp/foo.S
ctags: Notice: No options will be read from files or environment
addruart /tmp/foo.S /^.macro addruart, rp, rv, tmp$/;" m
ldr /tmp/foo.S /^ldr \\rp, =CONFIG_DEBUG_UART_PHYS$/;" l
ldr /tmp/foo.S /^ldr \\rv, =CONFIG_DEBUG_UART_VIRT$/;" l
ldr is extracted unexpectedly. I have no idea how to fix this.

Related

GDB: No symbol files found in NASM assembled file

Forgive me as I'm pretty new here, but I am trying to debug the x86 assembly with gdb.
ex10.asm
global main
extern printf
section .data
msg db "Testing %i...", 0x0a, 0x00
main:
push ebp
mov ebp, esp
push 123
push msg
call printf
mov eax, 0
mov esp, ebp
pop ebp
ret
compiled and linked with below:
nasm -f elf32 -F dwarf -g ex10.asm -o ex10.o
gcc -m32 -gdwarf ex10.o -o ex10
ex10.o appears to have debug symbols
$ objdump --syms ./ex10.o | grep debug
00000000 l d .debug_info 00000000 .debug_info
00000000 l d .debug_abbrev 00000000 .debug_abbrev
00000000 l d .debug_line 00000000 .debug_line
ex10 appears to have no debug symbols
$ objdump --syms ./ex10 | grep debug
----returns nothing----
gdb ./ex10 returns below
$ gdb ./ex10
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
This GDB was configured as "x86_64-linux-gnu".
Reading symbols from ./ex10...
(No debugging symbols found in ./ex10)
After this, I'm not quite sure to look.Any suggestions or info i should provide?
also, nasm version
$ apt list --installed | grep nasm
nasm/focal,now 2.14.02-1 amd64 [installed]
Reproduced with NASM version 2.15.05.
Without section .text (suggested by ssbssa):
readelf -w ex10.o
Section '.debug_aranges' has no debugging data.
Section '.debug_pubnames' has no debugging data.
Section '.debug_info' has no debugging data.
Section '.debug_abbrev' has no debugging data.
Section '.debug_line' has no debugging data.
Section '.debug_frame' has no debugging data.
Section '.debug_loc' has no debugging data.
so naturally you get nothing in the final link output.
Adding section .text before main solves the problem.
Note: you are expecting output from objdump --syms ./ex10 | grep debug, but that is the wrong thing to expect:
there are no symbols named *debug* in your file
you should ~never look at ELF files with objdump. Use readelf instead.
If you insist on using objdump, do this:
objdump -g ex10 | grep debug
Contents of the .debug_aranges section (loaded from ex10):
Offset into .debug_info: 0x0
Contents of the .debug_info section (loaded from ex10):
Contents of the .debug_abbrev section (loaded from ex10):
Raw dump of debug contents of section .debug_line (loaded from ex10):

Enabling bash-completion disables the default behaviour of auto-completing filenames

My make version on macOS High Sierra Version 10.13.6 looks like this:
$ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-apple-darwin11.3.0
My Bash version looks like this:
$ bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)
Copyright (C) 2007 Free Software Foundation, Inc.
My current directory looks like this:
$ ls -1
Makefile
a.txt
b.txt
c.txt
My Makefile looks like this:
hello:
echo hello
%.txt: FORCE
cat "$#"
FORCE:
Case 1: Without bash-completion
When I do not have bash-completion installed, I can conveniently autocomplete filenames in the current directory as the target argument for make. I mean, if I type:
make a<TAB>
it would automatically complete to:
make a.txt
Case 2: With bash-completion
When I do have bash-completion (such as with brew install bash-completion), I can no longer use this behaviour.
If I type:
make a<TAB>
it does not autocomplete at all. It does autocomplete other targets now such as make he<TAB> and make FO<TAB> which was not possible earlier but I lose the autocomplete any arbitrary filenames.
Question
Why does installing bash-completion disable the default behaviour of autocompleting current filenames? I think that's a very convenient behaviour to have.
How can I get that behaviour back without removing bash-completion. If I can do this, I can have the best of both worlds. Use auto-completions provided by bash-completion as well as auto-complete filenames in the current directory.
To answer the first part of your question, I think most people would not want Bash to offer files in the working directory as suggested targets for make.
But you can customise the behaviour without too much trouble.
Some useful docs on Bash Tab Completion can be found here.
It looks like the source code for the Bash Completion used on Mac OS X is here. The line you care about is this one:
COMPREPLY=( $( compgen -W "$( make -qp $makef $makef_dir 2>/dev/null | \
awk -F':' '/^[a-zA-Z0-9][^$#\/\t=]*:([^=]|$)/ \
{split($1,A,/ /);for(i in A)print A[i]}' )" \
-- "$cur" ) )
You can see that the compgen builtin there generates the list of possible completions, based on the output of make -qp Makefile ..
You can hack this to work the way you want it to this way. First, copy the make-specific autocomplete file to your user bash completion file:
cp /usr/local/Cellar/bash-completion/1.3_3/etc/bash_completion.d/make ~/.bash_completion
Then edit that file to specify a slightly different behaviour. Here, I'll show the diff:
--- /usr/local/Cellar/bash-completion/1.3_3/etc/bash_completion.d/make 2019-02-06 18:35:00.000000000 +1100
+++ /Users/alexharvey/.bash_completion 2019-02-06 18:34:36.000000000 +1100
## -62,7 +62,7 ##
COMPREPLY=( $( compgen -W "$( make -qp $makef $makef_dir 2>/dev/null | \
awk -F':' '/^[a-zA-Z0-9][^$#\/\t=]*:([^=]|$)/ \
- {split($1,A,/ /);for(i in A)print A[i]}' )" \
+ {split($1,A,/ /);for(i in A)print A[i]}' ; ls * )" \
-- "$cur" ) )
fi
Notice all I did there was to append the output of ls * to the response returned to compgen so that it considers files in the working directory to be also possible tab completions.
And that's it. Start a new shell.

sed - how to replace line based on pattern?

I have a Macros file which has, amongst other things, variable definitions like these:
SLIBS+=# USERDEFINED $(shell $(NETCDF_PATH)/bin/nc-config --flibs)
CFLAGS:= -gopt -Mlist -time
and I am trying to replace the contents of both SLIBS+= and CFLAGS:= using sed.
For example, I am trying to replace
SLIBS+=# USERDEFINED $(shell $(NETCDF_PATH)/bin/nc-config --flibs)
with
SLIBS+= -L/sw/arcts/centos7/netcdf/4.4.1/pgi-16.4-hdf5-1.8.16/lib -lnetcdff -L/sw/arcts/centos7/netcdf/4.4.1/pgi-16.4-hdf5-1.8.16/lib -L/sw/arcts/centos7/hdf5/1.8.16-pgi-16.4/lib -lnetcdf -lnetcdf -llapack -lblas
by using this command:
sed -i "s:SLIBS+=:SLIBS+= -L/sw/arcts/centos7/netcdf/4.4.1/pgi-16.4-hdf5-1.8.16/lib -lnetcdff -L/sw/arcts/centos7/netcdf/4.4.1/pgi-16.4-hdf5-1.8.16/lib -L/sw/arcts/centos7/hdf5/1.8.16-pgi-16.4/lib -lnetcdf -lnetcdf -llapack -lblas:g" Macros
but the new content is being appended to the old content, instead of replacing the old content.
What am I doing wrong?
Also, how could I do the same with the CFLAGS variable, replacing
CFLAGS:= -gopt -Mlist -time
with
CFLAGS:= -gopt -Mlist -time -I/sw/arcts/centos7/netcdf/4.4.1/pgi-16.4-hdf5-1.8.16/include -I/sw/arcts/centos7/hdf5/1.8.16-pgi-16.4/include
?
You're just missing a regex subexpression to capture "whatever else is on the line after SLIBS+=". As a result, only exactly SLIBS+= is being replaced, and the text after that is remaining (so looks like its been appended).
Replacing your sed command with eg.
sed -i 's:SLIBS+=.*:SLIBS+= ... :g' Macros
should do the trick (note the added .* to match any number of other characters and the ' to prevent the shell escaping *).
This approach should work for your CFLAGS as well.

Pandoc: What are the available syntax highlighters?

Bullet point 18 of http://pandoc.org/demos.html#examples shows how to change the syntax highlighter used by giving an argument to --highlight-style. For example:
pandoc code.text -s --highlight-style pygments -o example18a.html
pandoc code.text -s --highlight-style kate -o example18b.html
pandoc code.text -s --highlight-style monochrome -o example18c.html
pandoc code.text -s --highlight-style espresso -o example18d.html
pandoc code.text -s --highlight-style haddock -o example18e.html
pandoc code.text -s --highlight-style tango -o example18f.html
pandoc code.text -s --highlight-style zenburn -o example18g.html
I am wondering if these are the only color schemes available. If not, how can I load a different syntax highlighter? Can I define my own?
Since pandoc 2.0.5, you can also use --print-highlight-style to output a theme file and edit it.
To me, the best way to use this option is to
Pick a pleasant available style
Output its theme file
Edit the theme file
Use it!
1. Available Styles
Pick your style, among the one already existing:
2. Output its theme file
Once you decided which style was the closest to your needs, you can output its theme file, using (for instance for pygments, the default style):
pandoc --print-highlight-style pygments
so that you can store this style in a file, using, e.g.,
pandoc --print-highlight-style pygments > my_style.theme
With some shells, especially on Windows, using redirected output can lead to encoding problems. If that happens, use this instead:
pandoc -o my_style.theme --print-highlight-style pygments
3. Edit the file
Using the Skylighting JSON Themes guide, edit the file according to your need / taste.
4. Use the file
In the right folder, just use
pandoc my_file.md --highlight-style my_style.theme -o doc.html
If your pandoc --version indicates a release of 1.15.1 (from Oct 15, 2015) or newer, then you can check if the --bash-completion parameter works for you to get a full list of available built-in highlighting styles.
Run
pandoc --bash-completion
If it works, you'll see a lot of output. And it will be useful well beyond the original question above...
If --bash-completion works, then put this line towards the end of your ${HOME}/.bashrc file (on Mac OS X or Linux -- doesn't work on Windows yet):
eval "$(pandoc --bash-completion)"
Once you open a new terminal, you can use the pandoc command with "tab completion":
pandoc --h[tab]
will yield
--help --highlight-style --html-q-tags
pandoc --hi[tab]
will yield
pandoc --highlight-style
Answer to original question:
Now punch the [tab] key one more time, and you'll see
espresso haddock kate monochrome pygments tango zenburn
It's the list of all available syntax highlighters. To shorten the precedure, you could also type
pandoc --hi[tab][tab]
to get the same result.
Usefulness of Pandoc's tab completion beyond original question:
Pandoc's bash tab completion also works for all other commandline switches:
pandoc -h[tab]
yields this -- a list of all possible command line parameters:
Display all 108 possibilities? (y or n)
--ascii --indented-code-classes --template
--asciimathml --jsmath --title-prefix
--atx-headers --katex --to
--base-header-level --katex-stylesheet --toc
--bash-completion --latex-engine --toc-depth
--biblatex --latex-engine-opt --trace
--bibliography --latexmathml --track-changes
--chapters --listings --variable
--citation-abbreviations --mathjax --verbose
--columns --mathml --version
--csl --metadata --webtex
--css --mimetex --wrap
--data-dir --natbib --write
--default-image-extension --no-highlight -A
--dpi --no-tex-ligatures -B
--dump-args --no-wrap -D
--email-obfuscation --normalize -F
--epub-chapter-level --number-offset -H
--epub-cover-image --number-sections -M
--epub-embed-font --old-dashes -N
--epub-metadata --output -R
--epub-stylesheet --parse-raw -S
--extract-media --preserve-tabs -T
--file-scope --print-default-data-file -V
--filter --print-default-template -c
--from --read -f
--gladtex --reference-docx -h
--help --reference-links -i
--highlight-style --reference-odt -m
--html-q-tags --section-divs -o
--id-prefix --self-contained -p
--ignore-args --slide-level -r
--include-after-body --smart -s
--include-before-body --standalone -t
--include-in-header --tab-stop -v
--incremental --table-of-contents -w
One interesting use case for Pandoc's tab completion is this:
pandoc --print-default-d[tab][tab]
gives the output list of completion for pandoc --print-default-data-file. This list gives you a uniq insight into what data files your instance of Pandoc will load when it is doing its work. For example you could investigate a detail of Pandoc's default ODT (OpenDocument Text file) output styling like this:
pandoc --print-default-data-file odt/content.xml \
| tr " " "\n" \
| tr "<" "\n" \
| grep --color "style"
The Pandoc README says:
--highlight-style=STYLE|FILE
Specifies the coloring style to be used in highlighted source code.
Options are pygments (the default), kate, monochrome,
breezeDark, espresso, zenburn, haddock, and tango.
For more information on syntax highlighting in pandoc, see
Syntax highlighting, below. See also
--list-highlight-styles.
Instead of a STYLE name, a JSON file with extension
.theme may be supplied. This will be parsed as a KDE
syntax highlighting theme and (if valid) used as the
highlighting style. To see a sample theme that can be
modified, pandoc --print-default-data-file default.theme.
The library skylighting (in older versions highlighting-kate) is used for the highlighting. If you don't like any of the provided color schemes, you can either:
Specify a .theme file as mentioned above,
when exporting to HTML, <span> tags are generated that you can style with your custom CSS, or
when exporting to LaTeX/PDF, you need to use a custom Pandoc LaTeX template and replace the $highlighting-macros$ part with your custom color definitions, as described in this issue.
If you are using Pandoc version 1.18 (released in October 2016) or later, a new answer is possible:
pandoc --list-highlight-languages
and
pandoc --list-highlight-styles
will give you all the info you were asking for.
Other new informational command line parameters added to v1.18 are:
pandoc --list-input-formats
pandoc --list-output-formats
pandoc --list-extensions

Sublime Text build file for C++11

I'm currently trying to set up my build system in Sublime Text to compile C++11 Code and additionally use libpqxx library for some Postgres database stuff.
Right now, without C++11, my g++ command looks like this:
"cmd": ["g++", "${file}", "-o", "${file_path}/${file_base_name}", "-I/usr/include/pqxx", "-lpqxx", "-lpq"]
Which works fine.
Now I add "-std=c++0x" to the line, and it doesn't work. However, I wrote a simple testprogram that has #include <regex> in it, startet out with a default build file and added the C++11 flag to look like this:
"cmd": ["g++", "-std=c++0x", "${file}", "-o", "${file_path}/${file_base_name}"]
It works fine.
I also tried running the build from terminal, using this line:
g++ Connect.cpp -o Connect -std=c++0x -I/usr/include/pqxx -lpqxx, -lpq
But with no luck either.
Further I tried different places for the -std=c++0x. Right after g++, after the input and output file names, after the include, after the libary linking. And I trief to alter the std flag to -std+gnu++11, -std=c++11, -std=gnu++0x. Nothing helped.
What am I doing wrong?
(by the way my GCC/G++ has version 4.9, so there shouldn't be any problems with C++11 itself. The testfile.cpp without lipqxx include and linking works as mentioned before)
The solution: -I/usr/include/pqxx needed to be taken out.
Indeed, the solution is to remove the -I/usr/include/pqxx. But why?
The file he wants to include is /usr/include/pqxx/pqxx.
He is including it with #include <pqxx/pqxx> (He didnt provide that information on stackoverflow). With this include statement he doesn't need the additional include path since /usr/include is in the standard path.
But why is the build failing?
% ~ cpp -Wall -std=c++11 -M test.cpp > test2
% ~ cpp -Wall -std=c++11 -I/usr/include/pqxx -M test.cpp > test
% ~ cat test2 | sed "s/ \\\\//g" | sed "s/ /\n/g" | sort > test2
% ~ cat test | sed "s/ \\\\//g" | sed "s/ /\n/g" | sort > test
% ~ diff test test2
235d234
<
270a270
> /usr/include/c++/4.9.0/array
404a405
> /usr/include/c++/4.9.0/tuple
496d496
< /usr/include/pqxx/tuple
The -M option outputs a makefile.compatible list of headerfiles. The rest is just reformating to have a nicer diff.
Result: with -I/usr/include/pqxx the pqxx/tuple is included instead of the standard c++ tuple. That causes the compilation errors he didnt provide here.

Resources