I have been using sylfilter for over a year now (it is available from http://sylpheed.sraoss.jp/sylfilter/) and it works great as a filtering tool (no complaints). However, I have been trying to use procmail with sylfilter, but have been having a lot of trouble.
The web page for the filter shows:
sylfilter ~/Mail/inbox/1234
as the example to classify a message.
The return values are as following:
0 junk (spam)
1 clean (non-spam)
2 uncertain
127 other errors
I have been trying to incorporate sylfilter with procmail but not with much success. The big issue as compared with some other spam tool like bogofilter is that sylfilter does not make any changes to the e-mail message itself
(unlike bogofilter, for which examples abound on the web, and which
puts in a X-Bogosity field in the message header). I want everything
that is classified as Junk to go to $HOME/Mail/Junk and everything that
is not to be further classified into folders such as procmail rules.
Perhaps the stuff that returns 2 can go to $HOME/Mail/uncertain.
Here is my latest attempt based on suggestions made in the Fedora mailing list.
:0 Wc
| /usr/bin/sylfilter /dev/stdin
:0 a
$HOME/Mail/Junk/.
However, this does not process the e-mail message using sylfilter (and
the logfile says "No input file." before going on to process the other
rules). So, I was wondering if anyone here knew of a similar case and knew the answer to this question.
I am not familiar with sylfilter, and the (somewhat vague) problem description makes me think there is something wrong with feeding it a message on standard input. But if you can make that work, the following is how you examine a program's exit code in Procmail.
:0
* ? sylfilter /dev/stdin
$HOME/Mail/Junk/.
# You should now have the exit code in $? if you want it for further processing
SYLSTATUS=$?
:0
* SYLSTATUS ?? ^^1^^
$HOME/Mail/INBOX/.
# ... etc
The condition succeeds if sylfilter returns a success (zero) exit code; if it fails, we fall through to subsequent recipes. We save $? to a named variable so that we can examine its value even if a subsequent recipe resets the system global $? by invoking some other external program.
By the by, you should not need to hard-code the path to sylfilter. If it's in a nonstandard location, amend the PATH at the beginning of your .procmailrc rather than littering your code with explicit paths to executables. So if it's in /usr/local/really/sf/sylfilter, you'd put
PATH=/usr/local/really/sf:$PATH
If you need the message in a temporary file, try something like this;
TMP=`mktemp -t sylf.XXXXXXXX`
TRAP='rm -f $TMP'
:0c
$TMP
:0
* ? sylfilter $TMP
$HOME/Mail/Junk/.
# etc as above
The mktemp command creates a unique temporary file. The TRAP assignment sets up a command sequence to run when Procmail terminates; this takes care of cleaning out the temporary file when we are done. Because we will be the only writer to this file, we don't care about locking while writing a copy of the message to this file.
For more nitty-gritty syntax details, see also http://www.iki.fi/era/procmail/quickref.html
Related
Old stuff:
Background:
- Ultimate goal is to put a script in my .bash_profile that warns me by changing text color if I'm typing a commit message and it gets too
long (yes I'm aware vim has something like this).
Progress:
- I found the read -n option which led me to write this:
while true; do
# This hits at the 53rd character
read -rn53 input
# I have commit aliased to gc so the if is just checking if I'm writing a commit
if [ "${input:0:2}" = "gc" ]; then
printf "\nMessage getting long"
fi
done
Question:
- However, running this takes the user out of the bash prompt. I need a way to do something like this while at a normal prompt. I can't find
information on anything like this. Does that mean it's not possible?
Or am I just going about it the wrong way?
New progress:
I found the bind -x option which led me to write this:
check_commit() {
if [ "${READLINE_LINE:0:13}" == 'git commit -m' ] && [ ${#READLINE_LINE} -gt 87 ]; then
echo "Commit is $((${#READLINE_LINE} - 87)) characters too long!"
fi
READLINE_LINE="$READLINE_LINE$1"
READLINE_POINT=$(($READLINE_POINT+1))
}
bind -x '"\"": check_commit "\""'
It listens for a double quote and if I'm writing a long commit message tells me how many characters I am over the limit. Also puts the character I typed into the current line since it is eaten by the bind.
New question:
Now I just need a way to put in a regex, character list or at least a variable instead of \" so I can listen on more keys (Yes, I'm aware bind -x probably wasn't intended to be used this way. I can check performance/footprint/stability myself). I tried "$char", "${char}", "$(char)" and a few other things, but none seem to work. What is the correct approach here ?
AFAIK, not possible in a sane way if you want this to happen during your normal prompt (when PROMPT_COMMAND and PS1 are evaluated). That would involved binding a custom compiled readline function for every insert-self and alike.
If you want this to happen in a script using prompt builtin, this is crudely possible with a loop of
read -e -i $(munge_buf $buf) -n $(buf_warn_len $buf) -p $(buf_warning $buf) buf
like commands. This will allow you to create munge_buf() to alter the currently typed text if needed, buf_warn_len() to calculate a new len to warn at (which may be very large if warning was already displayed), and buf_warn_msg() to derive a warning message based upon the buffer.
So one of our developers needs me to batch a bunch of information and process it into an OVA to be presented back for download. This is an easy process using the long method (ie writing to the filesystem), but the developers want a cleaner, streamlined solution that will scale better. They have therefore requested that I stream the entire processes which is proving difficult. Can someone please give me some direction. Here are the steps that need to be accomplished:
Get input from webserver (Webserver will pass these as stream eventually.)
Random password
XML file
Modify boot script on file system (ie insert random password generated by server)
Create ISO of XML file and boot script
Calculate the SHA1 sum of ISO
Append SHA1 sum of ISO to manifest file in OVF directory
Create OVA from OVF directory
Here is an example directory structure (I outlined this in / just for simplicity)
/--
|
|--ISO/
| |
| |--boot.sh (Where the random password gets inserted)
| |--config.xml (This is handed from the web server. Needs to stream from server)
|
|--OVF/
|
|--disk.vmdk
|--ovf.xml
|--manifest.mf (Contains SHA1 of all files in OVF directory)
|--boot.iso (This file will exist once created from ISO directory)
Here is what I have so far (I'll explain the issues afterwards. Yes... there are a lot of issues):
cat /ISO/boot.sh | sed "s%DEFAULT%RANDOM%" | mkisofs /ISO/* | echo "SHA1(boot.iso)= " && sha1sum >> manifest.mf | tar -cvf success.ova /OVF/*
NOTE
In boot.sh there is a variable set to DEFAULT like this (Just for testing purposes):
PASSWORD="DEFAULT"
NOTE
This is what a line in the manifest file should look like:
SHA1(boot.iso)= 5fbc0d70 BLAH BLAH BLAH a91c9121bb
So I've never tried to write an entire script in one stream before. Usually I write to the filesystem a lot as I go. The first issue I see with this is that sed is replacing the string, but what it's piping over to mkisofs will not be used as mkiosfs is just going to make an iso of what it finds in /ISO. I dont even know if you can pass something like that to mkisofs. Piping is sometimes weird to think about.
Next, I think mkisofs is ok because I didnt specify a file output, therefore it should output to stdout which will be passed to sha1sum, but and here is the next problem I see. I need to append some additional text to the file before the SHA1 sum gets added which kinda interrupts the stream.
Finally, the last problem I see is how to pass everything to be tar into OVA without writing to the filesystem first (writing to manifest.mf).
Oh and the last BIG problem which I should have mentioned first is the config.xml file. Right now im dealing with it as just a file. The dev guys want to pass it to this script as a stream as well. I dont have a clue how to handle that.
Any help would be greatly appreciated. These concepts are a little beyond my knowledge.
Thanks!
UPDATE 12/11/13 2:11PM EST
Testing each part individually right now. Will report findings below soon.
UPDATE 12/11/13 2:14PM EST
The following works:
cat /ISO/boot.sh | sed "s%DEFAULT%RANDOM%"
and produces the following output:
RANDOM="RANDOM"
Exactly as expected.
You are correct NeronLeVelu, I will have to come back later and look at sed more carefully when real random passwords are being generated. ie. Making sure proper characters are escaped. Right now though, I'm just testing the logic. I will worry about regex and escaping later. We have not even decided on random password yet. It's only temporary and will most likely be alphanumeric.
Moving onto next part. Still not sure how to take the output from sed (stdout) and use it to include in ISO creation without actually creating a file that gets written to the file system. It may not be possible without writing to file system. More to come soon
# for the password if it contain & \ and separator used in your sed (default is /)
Password4Sed="`echo \"${PASSWORD} | sed \"s/[\\/&]/\\\\&/g\"`"
# no need of a cat with a sed
sed "s/DEFAULT/${Password4Sed}/"/ISO/boot.sh > /tmp/mkisofs.input
Treat rest from this input and put some test to validate each step like empty crc value or mkisofs.input. This will help at runtime when production error occur
I am trying to redirect emails that match a particular pattern to a shell script which will create files containing the texts, with datestamped filenames.
First, here is the routine from .procmailrc that hands the emails off to the script:
:0c:
* Subject: ^Ingest_q.*
| /home/myname/procmail/process
and here is the script 'process':
#!/bin/bash
DATE=`date +%F_%N`
FILE=/home/myname/procmail/${DATE}_email.txt
while read line
do
echo "$line" 1>>"$FILE";
done
I have gotten very frustrated with this because I can pipe text to this script on the command line and it works fine:
mybox-248: echo 'foo' | process
mybox-249: ls
2013-07-31_856743000_email.txt process
The file contains the word 'foo.'
I have been trying to get an email text to get output as a date-stamped file for hours now, and nothing has worked.
(I've also turned logging on in my .procmailrc and that isn't working either -- I'm not trying to ask a second question by mentioning that, just wondering if that might provide some hint as to what I might be doing wrong ...).
Thanks,
GB
Quoting your attempt:
:0c:
* Subject: ^Ingest_q.*
| /home/myname/procmail/process
The regex is wrong, ^ only matches at beginning of line, so it cannot occur after Subject:. Try this instead.
:0c:process.lock
* ^Subject: Ingest_q
| /home/myname/procmail/process
I also specified a named lockfile; I do not believe Procmail can infer a lock file name from just a script name. As you might have multiple email messages being delivered at the same time, and you don't want their logging intermingled in the log file, using a lock file is required here.
Finally, the trailing .* in the regex is completely redundant, so I removed it.
(The olde Procmail mini-FAQ also addresses both of these issues.)
I realize your recipe is probably just a quick test before you start on something bigger, but the entire recipe invoking the process script can be completely replaced by something like
MAILDIR=/home/myname/procmail
DATE=`date +%F_%N`
:0c:
${DATE}_email.txt
This will generate Berkeley mbox format, i.e. each message should have a From_ pseudo-header before the real headers; if you are not sure whether this is already the case, you should probably use procmail -Yf- to make sure to make it so (otherwise there is really no way to tell where one message ends and another begins; this applies both to your original solution, and this replacement).
Because Procmail sees the file name you are delivering to, it can infer a lockfile name now, as a minor bonus.
Using MAILDIR to specify the directory is the conventional way to do this, but you can specify a complete path to an mbox file if you prefer, of course.
I have a few lines of code in Stata. I'd like the lines to be executed only if the .txt file to which the lines refer exist a priori. I am wondering whether there is a shell command that I can use for this that I can embed in an if statement.
For example might something like the following exist and be possible:
insheet using "file.txt" if ('file.txt')
My intent is to say insheet the file file.txt only if it exists. My concern is that the program would otherwise stop, fail, die, or whatever you call it due to a syntax error if I have that insheet statement but the file does not exist.
Immediate answer is No. There is nothing like that syntax for several reasons.
The if qualifier tests whether some condition is true separately for each observation and whether a file exists is not an appropriate condition for testing observation by observation.
The quite different if command tests once and once only whether something is true and might seem more appropriate. In practice it is not used for this purpose, but to learn more, see help ifcmd.
Stata has no special syntax based on paired identical single quotes ' '.
However, Stata provides a separate construct here
confirm file file.txt
In practice that is going to stop a do-file or program whenever the file does not exist and the file does not exist. A general scheme to catch the error is something like
capture confirm file file.txt
if _rc == 0 insheet using file.txt
else {
<code if the file does not exist>
}
capture is to be thought of as eating the return code from the confirm command. In general the return code _rc from any command is 0 if the command was valid and executed and some non-zero value otherwise. Sometimes one tests for a specific non-zero code. Experiment shows that file not found is return code 601. The main reason for looking up error codes (in [P] error) is to deliver official-looking error messages, but in practice knowing the zero/non-zero rule is the main detail under this heading.
The example here uses == to test for equality.
Note that insheet using file.txt is not strictly a syntax error if the file does not exist. As far as Stata's language is concerned, that is legal syntax. However, that is a fine distinction: it is an error in every ordinary sense.
(LATER) It would be possible to short-circuit the entire process
capture insheet using file.txt
if _rc != 0 {
<code if the file does not exist>
}
as in this case the non-existence of the file is the presumed explanation for any failure of the insheet command. If, however, the insheet call were more complicated, with a varlist and/or options, then failure of the command could arise for other reasons. So in general separating out a check for the existence of the file seems a better strategy.
The confirm command has what you're looking for.
capture confirm file "file.txt"
if !_rc { # if the file exists, confirm will return error code 0
insheet using "file.txt"
}
Alternatively, you could put a capture before the insheet command, which will catch the syntax error. Check the [P] manual for more on capture and confirm.
I'm writing a shell script which iterates over a set of variables, edits a source file line by line according to the current iteration value, then remakes, and finally calls the just compiled binary. After execution the old line is restored.
Here is a snippet:
#!/bin/sh
for i in 0..4; do
perl -i -pe "s/.*/{SUBS[$i]}/ if $. == ${LINE[$i]}" ${SOURCE}
make
./bin/myTool
perl -i -pe "s/.*/\/\/{SUBS[$i]}/ if $. == ${LINE[$i]}" ${SOURCE}
done
Basically I have about 10 mutually exclusive #define in a C++ source file, and I'm experimenting the effects of each. Since I'm lazy I'd like to make it an automated process, and here I stuck.
Sometimes it happens that the shell says:
`make: Nothing to be done for 'all'`
Now, I tried to diff the file before and after every perlinstruction and the files do appear correct... I can't figure why this happens and how to make it behave correct.
Any idea?
Thank you in advance.
It's probably looping too quickly for make to tell each iteration apart. Either remove the make products or add a delay of 2 seconds at the beginning or end of the loop.
Make only checks if the target timestamp is younger than the source timestamp. That's the only way it can know what needs to be updated. So, if you loop iterations take less than a second then make won't know that anything has changed.
You can either clean up at the top of each iteration or add a delay as Ignacio Vazquez-Abrams has noted.