I'm trying to extract user's crontabs so as to view them together. The initial problem I ran into was that the crontab file (From crontab -l) contains a lot of commented lines placed there by the system to explain the file's function. I stole a sed snippet to deal with this that deletes lines starting with comments and replaces the rest of lines following comments with blanks. (As best as I understand it.)
Here's an example crontab I'd like to capture:
0 0 5 * * /home/thornegarvin/myscript.sh
The sed code I'm using is: (With croneditor.temp containing the crontab)
sed '/^[[:blank:]]*#/d;s/#.*//' croneditor.temp
I think that the command is matching the *s in the file as comments and then deleting the line, but I'm not sure if that's why the command is failing.
I need a version of this command or another one entirely that works as I intended (Grabbing crontabs from the output of crontab -l).
Related
My problem consists mainly of ignorance towards the different linux commands. I've run into a wall as I'm at a loss of words to correctly look up the commands I need.
However, I'd like to take the string that schemer (github.com/thefryscorer/schemer" outputs and insert it into my Terminator config at the line which starts with "palette=", replacing the existing info.
The purpose is to set this to run at intervals to keep my cycling wallpaper list updated with my bash colors.
If you could point me towards a place to learn of such automation and usage of commands, I'd be grateful.
Running Cinnamon 2.6.13 on Arch Linux, I wrote this code that, from a directory defined, takes a random file and applies it as the wallpaper. Afterwards it runs schemer and copies the newly generated config into Terminators config directory.
#!/bin/bash
#Todo;
currentmonth=August2015
#directory of wallpaper
currentfilename="$(ls ~/Pictures/wallpapers/"$currentmonth" | shuf -n 1)"
#set $currentfilename to random filename from $currentmonth
gsettings set org.cinnamon.desktop.background picture-uri file:///home/cogitantium/Pictures/wallpapers/"$currentmonth"/"$currentfilename"
#set wallpaper as current filename from default directory.
~/Go/bin/schemer -term="terminator" ~/Pictures/wallpapers/"$currentmonth"/"$currentfilename" > ~/Scripts/temp/currentpalette
#generate palette and redirect output to temporary palette file
echo "$currentmonth - $currentfilename - $currentpalette"
currentpalette="$(cat temp/currentpalette)"
#set $currentpalette to currentpalette
touch "temp/config"
sed -i "s/^.*\bpalette\b.*$/$currentpalette/g" "temp/config"
#insert generated palette into terminator config
cp "temp/config" "~/.config/terminator/config"
It does contain some errors and behaves irregularly at times. Furthermore, Terminator doesn't seem to react to the changes, even after a killall. I'll update my answer, should I find a solution.
The two things you'll need are sed and cron.
sed is a stream editor. In particular, it uses regular expressions to allow you to search through text files and replace parts of your text.
In particular, if you have
#conf.config
palette=REPLACE_ME
other.var=numbers
You can say sed s/REPLACE_ME/replaced_text/g conf.config
And sed will replace the text in conf.config with that second argument ("replace text").
So that'll be in a bash script you write.
What you'll want to do then in regularly execute your script, which you do by setting up a cron job. This will regularly execute a file.
You can either put your shell script in any one of the /etc/cron folders etc/cron.hourly, etc/cron.daily, etc, or
enter crontab -e at the terminal to open your personal cron configuration file, for more fine-grained control over when your commands execute.
The format of a cron command is as follows:
minute hour day-of-month month day-of-week command
So you could execute a command (as is explained in the sources below) once a week on Monday afternoon with
30 17 * * 1 /path/to/command (30=min, 17=5pm, and 1 is Monday as Sunday is 0-indexed).
or every 15 minutes with
*/15 * * * * /path/to/command
And your command would be ~/scripts/myscript.sh or whatever bash script has your sed command.
So you could have your cron job run schemer, then in the same script, put that string in your config file. You'll probably need to reload your config file, and the command for that (to stick at the end of your script) is source [path_to_config]
And there you go. Run schemer regularly, grab the string, stick it in your config file, and reload the file. cron, some basic bash (as I am unfamiliar with schemer, or the nature of it's output beyond "it's a string"), sed, and source.
Sources for cron jobs, as I am less familiar with them than with sed:
https://help.ubuntu.com/community/CronHowto
https://askubuntu.com/questions/2368/how-do-i-set-up-a-cron-job
I have an output file (namely a log from screen) containing several control characters. Inside the screen, I have programs running that use control characters to refresh certain lines (examples would be top or anything printing progress bars).
I would like to output a tail of this file using PHP. If I simply read in that file and echo its contents (either using PHP functions or through calling tail, the output is messy and much more than these last lines as it also includes things that have been overwritten. If I instead run tail in the command line, it returns just what I want because the terminal evaluates the control characters.
So my question is: Is there a way to evaluate the control characters, getting the output that a terminal would show me, in a way that I could then use elsewhere (e.g., write to a file)?
#5gon12eder's answer got rid of some control characters (thanks for that!) but it did not handle the carriage return part that was even more important to me.
I figured out that I could just delete anything from the beginning of a line to the last carriage return inside that line and simply keep everything after that, so here is my sed command accomplishing that:
sed 's/^.*\r\([^\r]\+\)\r\?$/\1\r/g'
The output can then be further cleaned using #5gon12eder's answer:
cat screenlog.0 | sed 's/^.*\r\([^\r]\+\)\r\?$/\1\r/g' | sed 's,\x1B\[[0-9?;]*[a-zA-Z],,g'
Combined, this looks exactly like I wanted.
I'm not sure what you mean by “evaluating” the control characters but you could remove them easily.
Here is an example using sed but if you are already using PHP, its internal regex processing functionality seems more appropriate. The command
$ sed 's,\x1B\[[0-9?;]*[a-zA-Z],,g' file.dat
will dump the contents of file.dat to standard output with all ANSI escape sequences removed. (And I'm pretty sure that nothing else is removed except if your file contains invalid escape sequences in which case the operation is ill-defined anyway.)
Here is a little demo:
$ echo -e "This is\033[31m a \033[umessy \033[46mstring.\033[0m" > file.dat
$ cat file.dat
# The output of the above command is not shown to protect small children
# that might be browsing this site.
$ reset # your terminal
$ sed 's,\x1B\[[0-9?;]*[a-zA-Z],,g' file.dat
This is a messy string.
The less program has some more advanced logic built in to selectively replace some escape sequences. Read the man page for the relevant options.
Due to processes out of my control I need run multiple SH files which contains lengthy CURL commands. Problem is that whichever process created these commands seems to have included one line of whitespace at the very end. If I call it as is - it fails. If I physically open the file and hit backspace on the first full empty line and save the file - it works perfectly.
Any way to put some kind of command into the SH file so that it removes any unnecessary stuff?
More info would be helpful, but the following might work:
If you need to put something into each of the files that contain the curl commands as you mention, you could try putting exit as the last line of the curl script (also depends on how you're calling the 'curl files'
exit
If you can run a separate script against the files that have a blank line, perhaps sed the blank lines away?
sed -i s/^\s$// $fileWithLineOfSpaces
edit:
Or (after thinking about it), perhaps simply delete the last line of the file....
sed -i '$d' $file
I'm new to sed command. I was reading the build script for some source code and I found this for loop in it.
for x in '*.la'
do
sed -i -e 's|^\(libdir=.\).*\(/opt/toolchains\)|\1\2|' x
done
I'm not able to understand the function this for loop is doing. Can anyone help.
It's iterating over a series of files in the current directory ending with ".la" and for each file found, it's editing the contents using sed to convert lines of the form:
libdir=X[zero-or-more-chars]/opt/toolchains
(where X is any character) into lines of the form:
libdir=X/opt/toolchains.
In other words, it's removing the [zero-or-more-chars] part of those lines.
Actually, this looks buggy because I would expect the sed command to reference $x, not x. I have a feeling you lost the $ somehow in the copy/paste step (or perhaps it's simply a bug).
To process a bunch of data and get it ready to be inserted into our database, we generate a bunch of shell scripts. Each of them has about 15 lines, one for each table that the data is going. One a recent import batch, some of the import files failed going into one particular table. So, I have a bunch of shell scripts (about 600) where I need to comment out the first 7 lines, then rerun the file. There are about 6000 shell scripts in this folder, and nothing about a particular file can tell me if it needs the edit. I've got a list of which files that I pulled from the database output.
So how do I write a bash script (or anything else that would work better) to take this list of file names and for each of them, comment out the first 7 lines, and run the script?
EDIT:
#!/usr/bin/env sh
cmd1
cmd2
cmd3
cmd4
cmd5
cmd6
cmd7
cmd8
Not sure how readable that is. Basically, the first 7 lines (not counting the first line) need to have a # added to the beginning of them. Note: the files have been edited to make each line shorter and partially cut off copying out of VIM. But in the main part of each file, there is a line starting with echo, then a line starting with sqlldr
Using sed, you can specify a line number range in the file to be changed.
#!/bin/bash
while read line
do
# add a comment to beginning of lines 1 - 7 and rename the script
sed '3,9 s/^/#/' $line > $line.new
exec $line.new
done < "filelist.txt"
You may wish to test this before running it on all of those scripts...
EDIT: changed the lines numbers to reflect comments.
Roughly speaking:
#!/bin/sh
for file in "$#"
do
out=/tmp/$file.$$
sed '2,8s/^/#/' < $file > $out
$SHELL $out
rm -f $out
done
Assuming you don't care about checking for race conditions etc.
ex seems made for what you want to do.
For instance, for editing one file, with a here document:
#!/bin/sh
ex test.txt << END
1,12s/^/#/
wq
END
That'll comment out the first 12 lines in "test.txt". For your example you could try "$FILE" or similar (including quotes!).
Then run them the usual way, i.e. ./"$FILE"
edit: $SHELL "$FILE" is probably a better approach to run them (from one of the above commenters).
Ultimately you're going to want to use the linux command sed. Whatever logic you need to place in the script, you know. But your script will ultimately call sed. http://lowfatlinux.com/linux-sed.html