I can't read or apply any other commands like cat or strings on .txt files because it is not allowed. I need to read a file named flag.txt, but this file is also on the blacklist. So, is there any way to read *.txt using the head command? The head command is allowed.
blacklist=\
'flag\|<\|$\|"\|'"'"'\|'\
'cat\|tac\|*\|?\|less\|more\|pico\|nano\|edit\|hexdump\|xxd\|'\
'sed\|tail\|diff\|grep\|paste\|strings\|bas64\|sort\|uniq\|cut\|awk\|'\
'bzip\|gzip\|xz\|tar\|ar\|'\
'mv\|cp\|ln\|nl\|'\
'python\|perl\|sh\|cc\|g++\|php\|hd\|g++\|gcc\|curl\|tcp\|udp\|'\
'scp\|sftp\|wget\|nc\|netcat'
Thanks
do you want some alternative of the command head *.txt? if so, ls/findand xargs will help, but it can not identify .txt file, it will read all the file under the directory.
ls -1| xargs head
You can use the ` (backtick) in the following way:
head `ls -1`
Backtick has a very special meaning. Everything you type between
backticks is evaluated (executed) by the shell before the main command
So the command will do the following:
`ls -1` - will result with the file names
head - will show the start of the files listed in ls -1
More info about backtick can be found in this answer
If you need a glob that matches flag.txt but can use neither * not the string flag, you can use fl[a]g.txt instead. Then, to print the entire file using head, use -c and pass it the size of the file:
head -c $(stat -c '%s' fl[a]g.txt) fl[a]g.txt
Another approach would be to use the shell to read the file:
while IFS= read -r c; do echo $c; done < fl[a]g.txt
You could also just use paste:
paste fl[a]g.txt
Related
I need to create a file that lists all the files in a folder into a text file, along with a comma and the number 15 after. For example
My folder has video.mp4, video2.mp4, picture1.jpg, picture2.jpg, picture3.png
I need the text file to read as follows:
video.mp4,15
video2.mp4,15
picture1.jpg,15
picture2.jpg,15
picture3.png,15
No spaces, just filename.ext,15 on each line. I am using a raspberry pi. I am aware that the command ls > filename.txt would put all the file names into a folder, but how would I get a ,15 after every line?
Thanks
bash one-liner:
for f in *; do echo "$f,15" >> filename.txt; done
To avoid opening the output file on each iteration you may redirect the entire output with > filename.txt:
for f in *; do echo "$f,15"; done > filename.txt
$ printf '%s,15\n' *
picture1.jpg,15
picture2.jpg,15
picture3.png,15
video.mp4,15
video2.mp4,15
This will work if those are the only files in the directory. The format specifier %s,15\n will be applied to each of printf's arguments (the names in the current directory) and they will be outputted with ,15 appended (and a newline).
If there are other files, then the following would work too, regardless of whether there are files called like this or not:
$ printf '%s,15\n' video.mp4 video2.mp4 picture1.jpg picture2.jpg "whatever this is"
video.mp4,15
video2.mp4,15
picture1.jpg,15
picture2.jpg,15
whatever this is,15
Or, on all MP4, PNG and JPEG files:
$ printf '%s,15\n' *.mp4 *.jpg *.png
video.mp4,15
video2.mp4,15
picture1.jpg,15
picture2.jpg,15
picture3.png,15
Then redirect this to a file with printf ...as above... >output.txt.
If you're using Bash, then this will not make use of any external utility, as printf is built into the shell.
You need to do something like this:
#!/bin/bash
for i in $(ls folder_name); do
echo $i",15" >> filename.txt;
done
It's possible to do this in one line, however, if you want to create a script, consider code readability in the long run.
Edit 1: better solution
As #CristianRamon-Cortes suggested in the comments below, you should not rely on the output of ls because of the problems explained in this discussion: why not parse ls. As such, here's how you should write the script instead:
#!/bin/bash
cd folder_name
for i in *; do
echo $i",15" >> filename.txt;
done
You can skip the part cd folder_name if you are already in the folder.
Edit 2: Enhanced solution:
As suggested by #kusalananda, you'd better do the redirection after done to avoid opening the file in each iteration of the for loop, so the script will look like this:
#!/bin/bash
cd folder_name
for i in *; do
echo $i",15";
done > filename.txt
Just 1 command line using 2 msr commands recusively (-r) search specific files:
msr -rp your-dir1,dir2,dirN -l -f "\.(mp4|jpg|png)$" -PAC | msr -t .+ -o '$0,15' -PIC > save-file.txt
If you want to sort by time, add --wt to first command like: msr --wt -l -rp your-dirs
Sort by size? Add --sz but only the prior one is effective if use both --sz and --wt.
If you want to exclude some directory, add like: --nd "^(test|garbage)$"
remove tail \r\n in save-file.txt : msr -p save-file.txt -S -t "\s+$" -o "" -R
See msr.exe / msr.gcc48 etc in my open project https://github.com/qualiu/msr tools directory.
A solution without a loop:
ls | xargs -i echo {},15 > filename.txt
I have a new file that I created that has a list of all directories that have a particular file:
$ find . -name "bun.biscuts" > TREE.temp
This writes all of the correct info I need to the new temp file.
I am having trouble writing a bash script using sed to replace emails in the TREE.temp file.
This is what I have so far:
#!/bin/bash
#set -x
echo Start!
for bun.biscuts in (TREE.temp)
do
sed -i 's/EMAIL1/EMAIL2/g';
done
Any help would be amazing!
gniourf_gniourf's comment is right; you don't need to use a temporary file and loop to do this. But if you want to, you could do it like this:
while read; do
sed -i 's/EMAIL1/EMAIL2/g' "$REPLY"
done < TREE.temp
which performs the sed operation on the filename on each line of the file. $REPLY is the default variable name that each line is written to by read. You can change the variable name by doing read var instead - then each line will be written to $var.
You can use only one sed command:
sed -i.bak 's/EMAIL1/EMAIL2/g' TREE.temp
If you have node on the computer you can npm install -g rexreplace and then
rexreplace 'email1' 'email2' TREE.tmp
I have a few files that I want to copy and rename with the new file names generated by adding a fixed string to each of them.
E.g:
ls -ltr | tail -3
games.txt
files.sh
system.pl
Output should be:
games_my.txt
files_my.sh
system_my.pl
I am able to append at the end of file names but not before *.txt.
for i in `ls -ltr | tail -10`; do cp $i `echo $i\_my`;done
I am thinking if I am able to save the extension of each file by a simple cut as follows,
ext=cut -d'.' -f2
then I can append the same in the above for loop.
do cp $i `echo $i$ext\_my`;done
How do I achieve this?
You can use the following:
for file in *
do
name="${file%.*}"
extension="${file##*.}"
cp $file ${name}_my${extension}
done
Note that ${file%.*} returns the file name without extension, so that from hello.txt you get hello. By doing ${file%.*}_my.txt you then get from hello.txt -> hello_my.txt.
Regarding the extension, extension="${file##*.}" gets it. It is based on the question Extract filename and extension in bash.
If the shell variable expansion mechanisms provided by fedorqui's answer look too unreadable to you, you also can use the unix tool basename with a second argument to strip off the suffix:
for file in *.txt
do
cp -i "$file" "$(basename "$file" .txt)_my.txt"
done
Btw, in such cases I always propose to apply the -i option for cp to prevent any unwanted overwrites due to typing errors or similar.
It's also possible to use a direct replacement with shell methods:
cp -i "$file" "${file/.txt/_my.txt}"
The ways are numerous :)
I have different location, but they all have a pattern:
some_text/some_text/some_text/log/some_text.text
All locations don't start with the same thing, and they don't have the same number of subdirectories, but I am interested in what comes after log/ only. I would like to extract the .text
edited question:
I have a lot of location:
/s/h/r/t/log/b.p
/t/j/u/f/e/log/k.h
/f/j/a/w/g/h/log/m.l
Just to show you that I don't know what they are, the user enters these location, so I have no idea what the user enters. The only I know is that it always contains log/ followed by the name of the file.
I would like to extract the type of the file, whatever string comes after the dot
THe only i know is that it always contains log/ followed by the name
of the file.
I would like to extract the type of the file, whatever string comes
after the dot
based on this requirement, this line works:
grep -o '[^.]*$' file
for your example, it outputs:
text
You can use bash built-in string operations. The example below will extract everything after the last dot from the input string.
$ var="some_text/some_text/some_text/log/some_text.text"
$ echo "${var##*.}"
text
Alternatively, use sed:
$ sed 's/.*\.//' <<< "$var"
text
Not the cleanest way, but this will work
sed -e "s/.*log\///" | sed -e "s/\..*//"
This is the sed patterns for it anyway, not sure if you have that string in a variable, or if you're reading from a file etc.
You could also grab that text and store in a sed register for later substitution etc. All depends on exactly what you are trying to do.
Using awk
awk -F'.' '{print $NF}' file
Using sed
sed 's/.*\.//' file
Running from the root of this structure:
/s/h/r/t/log/b.p
/t/j/u/f/e/log/k.h
/f/j/a/w/g/h/log/m.l
This seems to work, you can skip the echo command if you really just want the file types with no record of where they came from.
$ for DIR in *; do
> echo -n "$DIR "
> find $DIR -path "*/log/*" -exec basename {} \; | sed 's/.*\.//'
> done
f l
s p
t h
I want to change extension of large number of files,
Example: "ABC.dat.length.doc.fasta"
Desired: ABC.mat
You can use bash parameter expansion in a for loop:
for file in *.length.doc.fasta
do
mv "$file" "${file%dat.length.doc.fasta}"mat
done
Another way to do it:
for file in *.length.doc.fasta
do
mv "$file" "${file%%.*}".mat
done
Alternatively to using bash or awk it is possible to use the rarely used command line tool rename:
rename .dat.length.doc.fasta .mat *.dat.length.doc.fasta
This replaces the first argument with the second argument for the files provided is subsequent arguments (here represented by the shell expansion of *.dat.length.doc.fasta).
prename will do this easily for you. Uses regular expressions to locate and rename files.
prename 's/(.*)\.dat\.length\.doc\.fasta/$1.mat/' *.dat.length.doc.fasta
You can do it in a single line using AWK:
ls -1 *.dat.length.doc.fasta | awk -F"." '{system("mv "$0" "$1".mat")}' -
Input:
$ ls
ABC.dat.length.doc.fasta
ABCD.dat.length.doc.fasta
ABCDE.dat.length.doc.fasta
do_not_touch.txt
Output:
$ ls
ABCDE.mat
ABCD.mat
ABC.mat
do_not_touch.txt
ls -1 *.dat.length.doc.fasta | nawk '{p=$0;gsub("dat.length.doc.fasta","mat");system("mv "p" "$0); }'