Ok so with the new High Sierra, I am trying to write a script to automatically delete there local snapshots that eat up HDD space. I know you can shrink using thinlocalsnapshots / 1000000000 4 but I feel like that is only a band-aid.
So what I am trying to do is extract the date 2018-02-##-###### from:
sudo tmutil listlocalsnapshots /
com.apple.TimeMachine.2018-02-15-170531
com.apple.TimeMachine.2018-02-15-181655
com.apple.TimeMachine.2018-02-15-223352
com.apple.TimeMachine.2018-02-16-000403
com.apple.TimeMachine.2018-02-16-013400
com.apple.TimeMachine.2018-02-16-033621
com.apple.TimeMachine.2018-02-16-063811
com.apple.TimeMachine.2018-02-16-080812
com.apple.TimeMachine.2018-02-16-090939
com.apple.TimeMachine.2018-02-16-100459
com.apple.TimeMachine.2018-02-16-110325
com.apple.TimeMachine.2018-02-16-122954
com.apple.TimeMachine.2018-02-16-141223
com.apple.TimeMachine.2018-02-16-151309
com.apple.TimeMachine.2018-02-16-161040
I have tried variations of
| awk '{print $ } (insert number after $)
along with
| cut -d ' ' -f 10-.
Please if you know what I am missing here I would greatly appreciate it
edit: Here is the script that will get rid of those pesky Local snapshots.If anyone is interested, Thanks again:
#! /bin/bash
dates=`tmutil listlocalsnapshots / | awk -F "." 'NR++1{print $4}'`
for dates in $dates
do
tmutil deletelocalsnapshots $dates
done
You were close:
somecommand | cut -d"." -f4-
# or
somecommand | awk -F"." '{print $4}'
You can also try sed, but cut is made for this.
1- awk: you can either specify the field separator with the -F option, or print a substring
awk -F. '{print $4}'
awk '{print substr($0,23)}'
2- cut: equivalently.
cut -d. -f4
cut -c23-
3- Pure bash (sloooooow!): same as above.
while IFS=. read s1 s2 s3 d; do echo "$d"; done
while read line; do echo "${line:23}"; done
In practice, with a small number of records as in your use case, speed is not an issue and even pure bash or regexps (as in other aswers) can be used. As the number of records grows, the higher speed of awk and cut becomes noticeable.
Using grep and a regex :
$ grep -oP '\d{4}-\d{2}-\d{2}-\d{6}$'
2018-02-15-170531
2018-02-15-181655
2018-02-15-223352
2018-02-16-000403
2018-02-16-013400
2018-02-16-033621
2018-02-16-063811
2018-02-16-080812
2018-02-16-090939
2018-02-16-100459
2018-02-16-110325
2018-02-16-122954
2018-02-16-141223
2018-02-16-151309
2018-02-16-161040
Related
I tried to automate the process of cleaning up various wordlists I am working with. This is the following code for it:
#!/bin/bash
# Removes spaces and duplicates in a wordlist
echo "Please be in the same directory as wordlist!"
read -p "Enter Worldlist: " WORDLIST
RESULT=$( awk '{print length, $0}' $WORDLIST | sort -n | cut -d " " -f2- )
awk '!(count[$0]++)' $RESULT > better-$RESULT
This is the error I recieve after running the program:
./wordlist-cleaner.sh: fork: Cannot allocate memory
First post, I hope I formatted it correctly.
You didn't describe your intentions or desired output, but I guess this may do what you want
awk '{print length, $0}' "$WORDLIST" | sort -n | cut -d " " -f2- | uniq > better-RESULT
Notice that it's better-RESULT instead of better-$RESULT as you don't want that as a filename.
Yeah okay I got it to run successfully. I was trying to clean up wordlists I was downloading of the net. I have some knowledge of the basic variable usage in Bash, but not enough of the text manipulation commands like sed or awk. Thanks for the support.
I have multiple urls as input
https://drive.google.com/a/domain.com/file/d/1OR9QLGsxiLrJIz3JAdbQRACd-G9ZfL3O/view?usp=drivesdk
https://drive.google.com/a/domain.com/file/d/1sEWMFqGW9p2qT-8VIoBesPlVJ4xvOzXD/view?usp=drivesdk
How can I create a sed command to simply return only the file ID
desired output:
1OR9QLGsxiLrJIz3JAdbQRACd-G9ZfL3O
1sEWMFqGW9p2qT-8VIoBesPlVJ4xvOzXD
Looks like I need to start between /d/ and stop at /view but I'm not quite sure how to do that.
I've tried? sed -e 's/d\(.*\)\/view/\1/'
I was able to do this with cut -d '/' -f 8
also awk -F/ '{print $8}' file worked, thanks!
Your command was almost right:
# Wrong
sed -e 's/d\(.*\)\/view/\1/'
# better, removing unmatched stuff including the / after the d
sed -e 's/.*d\/\(.*\)\/view.*/\1/'
# better: using # for making the command easier to read
sed -e 's#.*d/\(.*\)/view.*#\1#'
# Alternative:Using cut when you don't know which field /d/ is
some_straem | grep -Eo '/d/.*/view' | cut -d/ -f3
I'm trying to extract usernames from a text file in one per line format and from my research, it seems like the only way to do it is by spacing commands here's the format:
1 user 3
2 fusrfff 4
3 usrf 12
The only problem is because all of the users are different I can't define a static space amount. There's also the fact the UIDs (first numbers) go from 1-40k. There's a bunch of other information after the user group number too. Can anyone point me in the right direction? Thanks.
awk does not care about the amount of space between fields:
awk '{print $2}' your_file.txt
If you want to go with bash only, read does not care either:
while read uid username other_stuff; do
printf '%s\n' "$name"
done < your_file.txt
First replace spaces by one space. You can use sed 's/ +/ /g' or
tr -s " " < file.txt| cut -d" " -f2
This is using sed
$ cat file.txt| sed "s/ */ /g" | cut -d' ' -f2
user
fusrfff
usrf
I am new to Bash scripting, but do understand most of the basics. My scenario is as follows:
I have a server from which I get a load of data via cURL. This is parsed properly (XML format) and from these results I then extract the data I want. The cURL statement writes its output to a file called temp-rec-schedule.txt. The below code is what I use to get the values I want to use in further calculation.
MP=`cat temp-rec-schedule.txt | grep "<ns3:mediapackage" | awk -F' ' '{print $3}' | cut -d '=' -f 2 | awk -F\" '{print $(NF-1)}'`
REC_TIME=`cat temp-rec-schedule.txt | grep "<ns3:mediapackage" | awk -F' ' '{print $2}' | cut -d '=' -f 2 | awk -F\" '{print $(NF-1)}'`
So this all still work perfectly. The output of the above code is respectively (if written to two separate files):
MP output:
b1706f0d-2cf1-4fd6-ab60-ae4d08608f1f
fd578fcc-342c-4f6c-986a-794ccb1abd0c
ce9f40e9-8e2c-4654-ba1c-7f79d35a69fd
c31a2354-6f4b-4bfe-b51e-2bac80889897
df342d88-c660-490e-9da6-9c91a7966536
49083f88-4264-4629-80fb-fae480d0bb25
946121c7-4948-4254-9cb5-2457e1b99685
f7bd0cad-e8f5-4e3d-a219-650d07a4bb34
REC_TIME output:
2014-09-15T07:30:00Z
2014-09-19T08:58:00Z
2014-09-22T07:30:00Z
2014-10-13T07:30:00Z
2014-10-17T08:58:00Z
2014-10-20T07:30:00Z
2014-10-22T13:28:00Z
2014-10-27T07:30:00Z
What I want to do now is create a file where line1 from file1 is appended with line1 from file2. i.e. :
b1706f0d-2cf1-4fd6-ab60-ae4d08608f1f 2014-09-15T07:30:00Z
fd578fcc-342c-4f6c-986a-794ccb1abd0c 2014-09-19T08:58:00Z
and so on.
I am not really familiar with Perl, but do know a little bit about Bash, so if it is possible, I would like to do this in Bash.
Further, from here, I want to compare two files that contain the same MP variable, but have two different TIME values assigned: subtract the one value from the other, and calculate the amount of hours that have passed between. This is all to calculate the amount of hours that have passed between publishing a video on our system, and the start time of the recording. Basically:
File1's output: b1706f0d-2cf1-4fd6-ab60-ae4d08608f1f 2014-09-15T07:30:00Z
File2's output: b1706f0d-2cf1-4fd6-ab60-ae4d08608f1f 2014-09-15T09:30:00Z
The output of my script should yield a value of 2 hours.
How can I do this with Bash?
You're probably better off just using awk for the whole thing. Something like:
awk '/<ns3:medipacakge/{gsub("\"","");
split($3,mp,"=");
split($2,rt,"="); print mp[2],rt[2]}' temp-rec-schedule.txt
The answer to the first question is to write the output to two different files and then use paste.
grep "<ns3:mediapackage" temp-rec-schedule.txt | awk -F' ' '{print $3}' | cut -d '=' -f 2 | awk -F\" '{print $(NF-1)}' > MP_out.txt
grep "<ns3:mediapackage" temp-rec-schedule.txt | awk -F' ' '{print $2}' | cut -d '=' -f 2 | awk -F\" '{print $(NF-1)}' > REC_out.txt
paste MP_out.txt REC_out.txt
That being said (and as #WilliamPursell says in his comment on the OP) there is never a reason to string this series of commands together since awk can do all the things you are doing there with significantly less overhead and more flexibility.
I am trying to manipulate a text file. I've got most of it figured out myself, but I'm stumped with why sed seems to go into infinite loop mode. The text file can be downloaded from census.gov.
At the moment, I just want a list of states that I can throw into a for loop to do some state-specific processing. So far, I've got this. (I'm not a bash expert, suggestions are welcome.
sed 1d tables/ansi.csv | awk -F "," '{print $1}' | uniq | tr \n : | sed s/:/" "/g
I want to put this into $() to use the output in a for loop, but for some reason, sed is hanging up and not exiting. I actually need to add a couple of things to the final sed command, to properly format things, but I want to get this running correctly before I go any further.
In the end - I want something that looks like (just showing the first few):
"AL" "AK" "AZ" "AR" "CA" "CO" ....
Right now, sed returns more of less what I expect and returns (just showing the last few)
...."MP" "PR" "UM" "VI" "
But, rather than exiting, sed hangs and I have to Ctrl-C out of the script. If I remove the final sed statement, the little script runs as I would expect, without hanging.
So, why on earth is this hanging?
I would suggest putting the sed script inside quotes:
sed 1d tables/ansi.csv | awk -F "," '{print $1}' | uniq | tr '\n' : | sed 's/:/" "/g'
The reason that sed seems to "hang" may be that tr has removed the final newline which sed requires. By the way, the newline argument to tr needs to be quoted.
However, the whole thing can be done in AWK:
awk -F, 'NR > 1 {a[$1]=$1} END { delim=":"; num=asort(a); for (i=1;i<=num;i++) printf "\"%s\" ",a[i]; printf "\n"}' tables/ansi.csv
awk -F"," 'NR>1 && (!($1 in a)){print $1;a[$1]}' file|sort|awk '{printf "\"%s\" ",$1}'