With go test -v pattern I can select and only run the tests that matches a pattern, but is there any way that I can list all test cases without run them. There is this case that the project just handed over to me has a lot of test case and I need to select some tests to run. I know greping the sources xxx_test.go is a way but is there more elegant way? I mean, each test has its meta data stored in some where, right? as the tests are required to be in a specific signature(func TestXXX(*testing.T)). This meta data can be used to do this kind of work.
Just saw this in the Go1.9 release notes (draft) and I think could be what you are looking for
The go test command accepts a new -list flag, which takes a regular
expression as an argument and prints to stdout the name of any tests,
benchmarks, or examples that match it, without running them.
There's no saved metadata, grepping is the only way pretty much.
If you have ack, you can easily use something like this:
➜ ack 'func Test[^(]+'
If you do not use ack as another answer mentions, you may simply grep -r "func Test" . | wc -l. This works because
grep for substring func Test, which should include every test func (it also includes TestMain)
flag -r tells grep to look in subdirectories recursively
wc -l will count the number of lines piped into it, which should be the number of test cases
Related
Is there a way to modify egrep -rha 'part1.*part2' to add something like: "if .* contains (not necessarily equals) string_x then pattern is not a match"? The problem is that string_x is present in every line so I can't -v it. It's okay to have this string before or after pattern, just not in the middle of it.
I'm assuming double .* with not string_x between them will get the job done, but it'll take a lot of time, plus I sometimes use .{n,m} wildcard, and in this case it would double the desired wildcard length. Maybe some sort of search termination every time it encounters string_x before part2?
Forget you ever heard about -r or any other option to let grep find files. There's a perfectly good tool for finding files with an extremely obvious name - find. Keep grep for what it's good at which is doing g/re/p. I can't imagine what the GNU guys were smoking when they decided to give grep options to find files, but hopefully they aren't now plotting to add options to sort files or pull content from web sites or print process data or do anything else that existing tools do perfectly well!
In this case you're looking for more than just g/re/p though so you should use awk:
awk '/part1.*part2/ && !/part1.*string_x.*part2/'
So the full script would be something like (untested since no sample input/output provided):
find . -type f -exec awk '/part1.*part2/ && !/part1.*string_x.*part2/' {} +
I want to find and replace the VALUE into a xml file :
<test name="NAME" value="VALUE"/>
I have to filter by name (because there are lot of lines like that).
Is it possible ?
Thanks for you help.
Since you tagged the question "bash", I assume that you're not trying to use an XML library (although I think an XML expert might be able to give you something like an XSLT processor command that solves this question very robustly), but that you're simply interested in doing search & replace from the commandline.
I am using perl for this:
perl -pi -e 's#VALUE#replacement#g' *.xml
See perlrun man page: Very shortly put, the -p switches perl into text processing mode, -i stands for "in-place", and -e let's you specify an expression to apply to all lines of input.
Also note (if you are not too familiar with that already) that you may use other characters than # (common ones are %, a comma, etc.) that don't clash with your search & replacement strings.
There is one small caveat: perl will read & write all files given on the commandline, even those that did not change. Thus, the files' modification times will be updated even if they did not change. (I usually work around that with some more shell magic, e.g. using grep -l or grin -l to select files for perl to work on.)
EDIT: If I understand your comments correctly, you also need help with the regular expression to apply. Let me briefly suggest something like this then:
perl -pi -e 's,(name="NAME" value=)"[^"]*",\1"NEWVALUE",g' *.xml
Related: bash XHTML parsing using xpath
You can use SED:
SED 's/\(<test name=\"NAME\"\) value=\"VALUE\"/\1 value=\"YourValue\"/' test.xml
where test.xml is the xml document containing the given node. This is very fragile, and you can work to make it more flexible if you need to do this substitution multiple times. For instance, the current statement is case sensitive, so it won't substitute the value on a node with the name="name", but you can add a case insensitivity flag to the end of the statement, like so:
('s/\(<test name=\"NAME\"\) value=\"VALUE\"/\1 value=\"YourValue\"/I').
Another option would be to use XSLT, but it would require you to download an external library. It's pretty versatile, and could be a viable option for more complex modifications to an XML document.
this is my very first post on Stackoverflow, and I should probably point out that I am EXTREMELY new to a lot of programming. I'm currently a postgraduate student doing projects involving a lot of coding in various programs, everything from LaTeX to bash, MATLAB etc etc.
If you could explicitly explain your answers that would be much appreciated as I'm trying to learn as I go. I apologise if there is an answer else where that does what I'm trying to do, but I have spent a couple of days looking now.
So to the problem I'm trying to solve: I'm currently using a selection of bioinformatics tools to analyse a range of genomes, and I'm trying to somewhat automate the process.
I have a few sequences with names that look like this for instance (all contained in folders of their own currently as paired files):
SOL2511_S5_L001_R1_001.fastq
SOL2511_S5_L001_R2_001.fastq
SOL2510_S4_L001_R1_001.fastq
SOL2510_S4_L001_R2_001.fastq
...and so on...
I basically wish to automate the process by turning these in to variables and passing these variables to each of the programs I use in turn. So for example my idea thus far was to assign them as wildcards, using the R1 and R2 (which appears in all the file names, as they represent each strand of DNA) as follows:
#!/bin/bash
seq1=*R1_001*
seq2=*R2_001*
On a rudimentary level this works, as it returns the correct files, so now I pass these variables to my first function which trims the DNA sequences down by a specified amount, like so:
# seqtk is the program suite, trimfq is a function within it,
# and the options -b -e specify how many bases to trim from the beginning and end of
# the DNA sequence respectively.
seqtk trimfq -b 10 -e 20 $seq1 >
seqtk trimfq -b 10 -e 20 $seq2 >
So now my problem is I wish to be able to append something like "_trim" to the output file which appears after the >, but I can't find anything that seems like it will work online.
Alternatively, I've been hunting for a script that will take the name of the folder that the files are in, and create a variable for the folder name which I can then give to the functions in question so that all the output files are named correctly for use later on.
Many thanks in advance for any help, and I apologise that this isn't really much of a minimum working example to go on, as I'm only just getting going on all this stuff!
Joe
EDIT
So I modified #ghoti 's for loop (does the job wonderfully I might add, rep for you :D ) and now I append trim_, as the loop as it was before ended up giving me a .fastq.trim which will cause errors later.
Is there any way I can append _trim to the end of the filename, but before the extension?
Explicit is usually better than implied, when matching filenames. Your wildcards may match more than you expect, especially if you have versions of the files with "_trim" appended to the end!
I would be more precise with the wildcards, and use for loops to process the files instead of relying on seqtk to handle multiple files. That way, you can do your own processing on the filenames.
Here's an example:
#!/bin/bash
# Define an array of sequences
sequences=(R1_001 R2_001)
# Step through the array...
for seq in ${sequences[#]}; do
# Step through the files in this sequence...
for file in SOL*_${seq}.fastq; do
seqtk trimfq -b 10 -e 20 "$file" > "${file}.trim"
done
done
I don't know how your folders are set up, so I haven't addressed that in this script. But the basic idea is that if you want the script to be able to manipulate individual filenames, you need something like a for loop to handle the that manipulation on a per-filename basis.
Does this help?
UPDATE:
To put _trim before the extension, replace the seqtk line with the following:
seqtk trimfq -b 10 -e 20 "$file" > "${file%.fastq}_trim.fastq"
This uses something documented in the Bash man page under Parameter Expansion if you want to read up on it. Basically, the ${file%.fastq} takes the $file variable and strips off a suffix. Then we add your extra text, along with the suffix.
You could also strip an extension using basename(1), but there's no need to call something external when you can use something built in to the shell.
Instead of setting variables with the filenames, you could pipe the output of ls to the command you want to run with these filenames, like this:
ls *R{1,2}_001* | xargs -I# sh -c 'seqtk trimfq -b 10 -e 20 "$1" > "${1}_trim"' -- #
xargs -I# will grab the output of the previous command and store it in # to be used by seqtk
Problem:
I have a list of filenames, filenames.txt:
Eg.
/usr/share/important-library.c
/usr/share/youneedthis-header.h
/lib/delete/this-at-your-peril.c
I need to rename or delete these files and I need to find references to these files in a project directory tree: /home/noob/my-project/ so I can remove or correct them.
My thought is to use bash to extract the filename: basename filename, then grep for it in the project directory using a for loop.
FILELISTING=listing.txt
PROJECTDIR=/home/noob/my-project/
for f in $(cat "$FILELISTING"); do
extension=$(basename ${f##*.})
filename=$(basename ${f%.*})
pattern="$filename"\\."$extension"
grep -r "$pattern" "$PROJECTDIR"
done
I could royally screw up this project -- does anyone see a flaw in my logic; better: do you see a more reliable scalable way to do this over a huge directory tree? Let's assume that revision control is off the table ( it is, in fact ).
A few comments:
Instead of
for f in $(cat "$FILELISTING") ; do
...
done
it's somewhat safer to write
while IFS= read -r f ; do
...
done < "$FILELISTING"
That way, your code will have no problem with spaces, tabs, asterisks, and so on in the filenames (though it still won't support newlines).
Your goal in separating f into extension and filename, and then reassembling them with \., seems to be that you want the filename to be treated as a literal string; right? Like, you're worried that grep will treat the . as meaning "any character" rather than as "one dot". A more general solution is to use grep's -F option, which tells it to treat the pattern as a fixed string rather than a regex:
grep -r -F "$f" "$PROJECTDIR"
Your introduction mentions using basename, but then you don't actually use it. Is that intentional?
If your non-use of basename is intentional, then filenames.txt really just contains a list of patterns to search for; you don't even need to write a loop, in this case, since grep's -f option tells it to take a newline-separated list of patterns from a file:
grep -r -F -f "$FILELISTING" "$PROJECTDIR"
You should back up your project, using something like tar -czf backup.tar.gz "$PROJECTDIR". "Revision control is off the table" doesn't mean you can't have a rollback strategy!
Edited to add:
To pass all your base-names to grep at once, in the hopes that it can do something smarter with them than just looping over them just as though the calls were separate, you can write something like:
grep -r -F "$(sed 's#.*/##g' "$FILELISTING")" "$PROJECTDIR"
(I used sed rather than while+basename for brevity's sake, but you can an entire loop inside the "$(...)" if you prefer.)
This is a job for an IDE.
You're right that this is a perilous task, and unless you know the build process and the search directories and the order of the directories, you really can't say what header is with which file.
Let's take something as simple as this:
# include "sql.h"
You have a file in the project headers/sql.h. Is that file needed? Maybe it is. Maybe not. There's also a /usr/include/sql.h. Maybe that's the one that's actually used. You can't tell without looking at the Makefile and seeing the order of the include directories which is which.
Then, there are the libraries that get included and may need their own header files in order to be able to compile. And, once you get to the C preprocessor, you really will have a hard time.
This is a task for an IDE (Integrated Development Environment). An IDE builds the project and tracks file and other resource dependencies. In the Java world, most people use Eclipse, and there is a C/C++ plugin for those developers. However, there are over 2 dozen listed in Wikipedia and almost all of them are open source. The best one will depend upon your environment.
I'm writing a simple command line gem.
The library that does the actual work was developed with rspec and so far that works.
I'm trying to test the command line portion with Aruba/Cucumber, but I've come across some strange behaviour.
Just to test this, I've got a the binary file to puts ARGV, and I've got test files in tmp/aruba
When I run bundle exec gem_name tmp/aruba/*.* I am presented with the list of shell expanded file names.
Now my features file has:
Given files to work on # I set up files in tmp/aruba in this step
When I run `gem_name *.*` # standard step
Then the output should contain "Wibble"
The last step is obviously going to fail, but it shows me a diff between what it expects and the actual output. Rather than seeing a list of shell expanded filenames, all I get is "*.*"
So I'm left in the position of having an app that actually works as expected, but I can't get the tests to pass. I could take the "." and generate the list of files from there, but then I'm writing extra production code just to get the app to work under test - which I don't think is the correct way to go about it. And all because shell expansion isn't happening.
If you look at my profile, you'll see that Ruby isn't my main bag, feel free to point me at any resources that I may have missed about this, but is this just me missing something, or expected behaviour that somebody knows how to work around?
After a little digging in the Aruba source I figured out that the When I run step ends up in a code block like this:
def run!(&block)
#process = ChildProcess.build(*shellwords(#cmd))
...
begin
#process.start
...
Further digging into ChildProcess ends up here:
def launch_process
...
begin
exec(*#args)
...
And therein lies the problem. exec does not do shell expansion when the argument list is split into multiple array elements:
If exec is given a single argument, that argument is
taken as a line that is subject to shell expansion before being
executed. If multiple arguments are given, the second and
subsequent arguments are passed as parameters to command with no
shell expansion.
However playing with shellwords a bit we find:
Shellwords.shellwords('gem_name *.*')
=> ["gem_name", "*.*"] # No good
Shellwords.shellwords('"gem_name *.*"')
=> ["gem_name *.*"] # Aha!
Therefore the solution might be as simple as:
When I run `"gem_name *.*"`
If that doesn't work then you are pretty much out of luck. I would suggest you expand the file names manually since you're not really testing shell expansion here - we know that works: you are testing multiple arguments.
Therefore you should instead do:
When I run `gem_name your_file1 your_file2 your_file3`