Occasionally, we receive ZIP files from our suppliers that are seemingly corrupted. Attempting to list the contents of a ZIP will trigger an error like this:
$>unzip -qql JABL_VER_20120808_165910.zip
unzip: cannot find or open JABL_VER_20120808_165910.zip, JABL_VER_20120808_165910.zip.zip or JABL_VER_20120808_165910.zip.ZIP.
I did a quick read of the unzip man page and coded that snippet to trap the above error
EXIT=`echo $?`
case $EXIT in
> 0-1) echo "Unzip Complete.";;
> *) echo "Unzip Failed.";;
> esac
$>Unzip Failed.
It seems to work. However, there are cases like this one where the error is different:
$>unzip -qql JABL_VER_20120808_175915.zip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of JABL_VER_20120808_175915.zip or
JABL_VER_20120808_175915.zip.zip, and cannot find JABL_VER_20120808_175915.zip.ZIP, period.
Is there a "surefire" way to trap errors like these?
PS: Not sure if matters but the ZIP files are generated on MS Windows ; We use Red Hat.
exit code 1 isn't good either,
1 one or more warning errors were encountered, but processing completed successfully anyway. This includes zipfiles
where one or more files was skipped due to unsupported compression method or encryption with an unknown password.
you should try
unzip -t zipfilename
and only accept exit code 0
why do you have so many errors? ftp'd file in text mode?
Related
I have a bot on Marathi language wikipedia. The bot runs from Wikimedi's toolforge server. I have set up a cron job which generates two files: one .err and another is .out
Following is my the content from my cron file:
0 9 * * * jsub -release buster -N KiranBOT1 kiranbot1.sh
The job runs daily. I was away for around a week, and the .err file became around 500 megabytes in size. So I deleted it, and after one run when I checked the newly generated file, it was around 8 megabytes.
It doesn't generate multiple .err files, but it keeps adding on the details in the same file, which increases the file size tremendeously. I have created that bot in such a way, that even if I stop editing Wikipedia/stop checking the bot, even then the bot would keep doing its task. In such a scenario, the .err file size would just keep on increasing. I don't want that to happen.
I can live without these .err, and .out files. Is there a way to stop generating these files?
Thanks a lot in advance,
-usernamekiran.
Edit:
The file names are KiranBOT1.err, and KiranBOT1.out. I apologise, I should have mentioned this in my original question.
Original throwaway comment now posted as an answer.
The day you stop generating the logs is the day that you need them...
My suggestion would be create a dated daily pair of files (KiranBOT1_YYYYMMDD.err, and KiranBOT1_YYYYMMDD.out), and use logrotate to remove after (say) one week?
Check the full names of the files and do something like
ln -s /dev/null .err
ln -s /dev/null .out
echo "Warning: .err and .out are symlinks from /dev/null" > README
I am using sphinx for documentation. and have used the make html and make latexpdf through a bash script.
when running the bash script, and when make html is run, it shows ERROR in teh rst file after the Reading sources.
make clean
+ make clean
rm -rf _build/*
make html
+ make html
sphinx-build -b html -d _build/doctrees . _build/html
Running Sphinx v1.3b3
making output directory...
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 2 source files that are out of date
updating environment: 2 added, 0 changed, 0 removed
reading sources... [100%] index
/home/sphinx/Inder.rst:12:WARNING: Bullet list ends without a blank line; unexpected unindent.
/home/sphinx/Inder.rst:12: ERROR: Document may not end with a transition.
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
How can i catch this error in the bash script or send it to another variable.
i have tried grep ERROR with the make html, but it gives blank.
sphinx-build will probably print output on both stdout and stderr so you likely want to capture both streams.
You can do this like:
out=$(make html 2>&1)
if grep -q 'ERROR: ' <<< "$out"; then
# do some error handling
fi
I'm facing a quite weird failure after I successfully compiled a .chm file with HTML Help Workshop's hhc.exe.
I created a documentation of my source code with Doxygen. Doxygen creates if you enable GENERATE_HTMLHELP in the doxygen file:
If the GENERATE_HTMLHELP tag is set to YES then doxygen generates three
additional HTML index files: index.hhp, index.hhc, and index.hhk. The index.hhp
is a project file that can be read by Microsoft's HTML Help Workshop (see:
http://www.microsoft.com/en-us/download/details.aspx?id=21138) on Windows.
After Doxygen creates those files, I want to create the .chm file with HTML Help Workshop. For that reason I call hhc.exe in a CMD. After that I can open the .chm file and everything seems fine. But if I ask the CMD for the current errorlevel, it outputs:
"C:\Program Files (x86)\HTML Help Workshop\hhc.exe" index.hhp
Microsoft HTML Help Compiler 4.74.8702
Compiling C:\... folder path ...\index.chm
Compile time: 0 minutes, 0 seconds
37 Topics
346 Local links
12 Internet links
0 Graphics
Created C:\... folder path ...\index.chm, 88,740 bytes
Compression decreased file by 194,682 bytes.
echo %errorlevel%
1
Does anyone have an idea of what the h... is going wrong here and how I can avoid that problem that it's compiled successfully but still returns an error? How can I figure out what's the problem instead of an error "1"?
The problem is that my build server (TFS2015) returns the error and the build is failed.
HTML Help Workshop is installed with 2 main executables:
hhc.exe is the console version of the HTML Help Compiler.
hhw.exe is the Windows GUI version of HTML Help Compiler.
But the compilation of the HTML Help Project (hhp) to Compiled HTML (chm) is not directly done by those two executables.
Both use for compilation hha.dll - the HTML Help Author library - by calling the exported function HHA_CompileHHP.
The exported functions of this library are:
EditHhCtrlObject
EditHhCtrlScript
FreeFilterDIB
HHA_CompileHHP
LoadFilterImage
LoadJpeg
Microsoft has not published any documentation nor the function declarations of those functions as far as I know.
I suppose from some quick tests with hhc.exe that the function HHA_CompileHHP has BOOL as return type which is int and returns on success TRUE, i.e. the value 1, and on failure FALSE, i.e. the value 0. And it looks like hhc.exe uses this return value without inverting the value as exit/return code.
Therefore the errorlevel is 1 on success and 0 on failure.
The tests I made to verify my assumption:
Run HTML Help compiler with name of a project file which does not exist:
hhc.exe index_1.hhp
Unable to open index_1.hhp.
The exit code respectively errorlevel is 0. This error message is printed by hhc.exe because the error message can be found in hhc.exe.
Set read-only file attribute on already existing output file index.chm and run HTML Help compiler:
attrib +r index.chm & hhc.exe index.hhp & attrib -r index.chm
HHC5010: Error: Cannot open "C:... folder path ...\index.chm". Compilation stopped.
The exit code respectively errorlevel is 0. This error message is printed by hha.dll because this error message can be found only in hha.dll.
Rename a *.htm file explicitly specified in index.hhp and run HTML Help Compiler:
ren "file.htm" "renamed file.htm" & hhc.exe index.hhp & ren "renamed file.htm" "file.htm"
Microsoft HTML Help Compiler 4.74.8702
Compiling C:... folder path ...\index.chm
HHC5003: Error: Compilation failed while compiling file.htm.
The following files were not compiled:
file.htm
The exit code respectively errorlevel is 0. This error message is printed also by hha.dll because this error message can be found also only in hha.dll.
All error messages are written to handle STDOUT and not to STDERR as typical for console applications. There was never another value than 0 or 1 assigned to errorlevel which is the reason why I suppose the function HHA_CompileHHP returns a simple boolean value.
Conclusion:
The opposite as usual must be done to evaluate on success/failure of an HTML Help compilation for example by using in the batch file:
"%ProgramFiles(x86)%\HTML Help Workshop\hhc.exe" index.hhp
if not errorlevel 1 exit /B 1
In HTML Help Project file (*.hhp file) in section [OPTIONS] a log file can be specified with Error log file=... into which all messages output by HHA_CompileHHP are written additionally to printing them to STDOUT.
But in this case with Doxygen generating the *.hhp file it would be easier to redirect STDOUT in the batch file to a log file although this is also not really needed because most likely the Team Foundation Server is capturing the messages already to a log. (I don't have Team Foundation Server installed.)
HTML Workshop is a GUI program. When using CMD interactively it does not wait for GUI programs to exit. Therefore there is no error code.
Echo %errorlevel% interactively will never show an error code as well.
This two ways will
Dir && Echo Success || Echo Failure
Dir df:\ & Echo %errorlevel%
[See my answer at Trouble with renaming folders and sub folders using Batch to see what it means]
You will have to read HTML Workshop's documentation to see if it sets an errorlevel. Most GUI programs don't bother.
First post, so Hi! Let me start by saying I'm a total noob regarding programming. I understand very basic stuff, but when it comes to checking exit codes or what the adequate term is, I'm at a loss. Apparently my searchfoo is really weak in this area, I guess it's a question of terminology.
Thanks in advance for taking your time to reading this/answering my question!
Description: I found a script that converts/repack .cbr files to .cbz files. These files are basically your average rar and zip files, however renamed to another extension as they are used for (comic)book applications such as comicrack, qcomicbook and what not. Surprisingly enough there no cbr -> cbz converters out there. The advantages of .cbz is besides escaping the proprietary rar file format, that one can store the metadata from Comic Vine with e. g comictagger.
Issue: Sometimes the repackaging of the files doesn't end well and would hopefully be alleviated by a integrity check & another go. I modified said script slightly to use p7zip as it can both pack/unpack 7z, zip-files and some others, i. e great for options. p7zip can test the archive by:
7z t comicfile.cbz tmpworkingdir
I guess it's a matter of using if & else here(?) to check the integrity and then give it another go, if there are any error.
Question/tl;dr: What would be the "best"/adequate approach to add a integrity file check to the script below?
#!/bin/bash
#Source: http://comicrack.cyolito.com/forum/13-scripts/30013-cbr3cbz-rar-to-zip-conversion-for-linux
echo "Converting CBRs to CBZs"
# Set the "field separator" to something other than spaces/newlines" so that spaces
# in the file names don't mess things up. I'm using the pipe symbol ("|") as it is very
# unlikely to appear in a file name.
IFS="|"
# Set working directory where to create the temp dir. The user you are using must have permission
# to write into this directory.
# For performance reasons I'm using ram disk (/dev/shm/) in Ubuntu server.
WORKDIR="/dev/shm/"
# Set name for the temp dir. This directory will be created under WORDDIR
TEMPDIR="cbr2cbz"
# The script should be invoked as "cbr2cbz {directory}", where "{directory}" is the
# top-level directory to be searched. Just to be paranoid, if no directory is specified,
# then default to the current working directory ("."). Let's put the name of the
# directory into a shell variable called SOURCEDIR.
# Note: "$1" = "The first command line argument"
if test -z "$1"; then
SOURCEDIR=`pwd`
else
SOURCEDIR="$1"
fi
echo "Working from directory $SOURCEDIR"
# We need an empty directory to work in, so we'll create a temp directory here
cd "$WORKDIR"
mkdir "$TEMPDIR"
# and step into it
cd "$TEMPDIR"
# Now, execute a loop, based on a "find" command in the specified directory. The
# "-printf "$p|" will cause the file names to be separated by the pipe symbol, rather than
# the default newline. Note the backtics ("`") (the key above the tab key on US
# keyboards).
for CBRFILE in `find "$SOURCEDIR" -name "*.cbr" -printf "%p|while read line; do
# Now for the actual work. First, extract the base file name (without the extension)
# using the "basename" command. Warning: more backtics.
BASENAME=`basename $CBRFILE ".cbr"`
# And the directory path for that file, so we know where to put the finished ".cbz"
# file.
DIRNAME=`dirname $CBRFILE`
# Now, build the "new" file name,
NEWNAME="$BASENAME.cbz"
# We use RAR file's name to create folder for unpacked files
echo "Processing $CBRFILE"
mkdir "$BASENAME"
# and unpack the rar file into it
7z x "$CBRFILE" -O"$BASENAME"
cd "$BASENAME"
# Lets ensure the permissions allow us to pack everything
sudo chmod 777 -R ./*
# Put all the extracted files into new ".cbz" file
7z a -tzip -mx=9 "$NEWNAME" *
# And move it to the directory where we found the original ".cbr" file
mv "$NEWNAME" $DIRNAME/"$NEWNAME"
# Finally, "cd" back to the original working directory, and delete the temp directory
# created earlier.
cd ..
rm -r "$BASENAME"
# Delete the RAR file also
rm "$CBRFILE"
done
# At the end we cleanup by removing the temp folder from ram disk
cd ..
echo "Conversion Done"
rm -r "$TEMPDIR"
Oh the humanity, not posting more than two links before 10 reputation and I linked the crap out of OP.. [edit]ah.. mh-mmm.. there we go..
[edit 2] I removed unrar as an dependency and use p7zip instead, as it can extract rar-files.
You will need two checks:
7z t will test the integrity of the archive
You should also test the integrity of all the image files in the archive. You can use at tools like ImageMagick for this.
A simple test would be identify file but that might read only the header. I'd use convert file -resize 5x5 png:- > /dev/null
This scales the image down to 5x5 pixels, converts it to PNG and then pipes the result to /dev/null (discarding it). For the scaling, the whole image has to be read. If this command fails with an error, something is wrong with the image file.
I must write a basic script that takes input in the form of:
./chext 'param1' 'param2.xx'
and outputs:
param2.param1
specifically, I have to check to see if the file exists, and if it does, rename it with the new extension, and if it does not, generate the message:
param2.xx: No such file
specifically, I'm told that: "No other messages should be issued, including error messages from commands invoked by your script."
This works fine with examples like:
./chext 'com' 'aardvark.dat'
and
./chext 'ret' 'too common.exp'
but for:
./chext 'dat' 'ocelot.dat'
I'm getting the error (created by my professor) "generated unexpected messages"
Can anyone tell me what's so different about this example that my script isn't taking care of?
#!/bin/csh
set ext="$1"
set oldName="$2"
if (-r "$oldName") then
set newName=`echo "$oldName" | sed 's/\.[A-Za-z0-9]*$/'".$ext"'/g'`
mv "$oldName" "$newName"
else
echo "$oldName": No such file
endif
Thanks
You can't rename a file to the name it already has.