I want to extract a 7-Zip archive in a Python script. It works fine except that it spits out the extraction details (which is huge in my case).
Is there a way to avoid this verbose information while extracting? I did not find any "silent" command line option to 7z.exe.
My command is
7z.exe -o some_dir x some_archive.7z
I just came across this when searching for the same, but I solved it myself! Assuming the command is processed with Windows / DOS, a simpler solution is to change your command to:
7z.exe -o some_dir x some_archive.7z > nul
That is, direct the output to a null file rather than the screen.
Or you could pipe the output to the DOS "find" command to only output specific data, that is,
7z.exe -o some_dir x some_archive.7z | FIND "ing archive"
This would just result in the following output.
Creating archive some_archive.7z
or
Updating archive some_archive.7z**
My final solution was to change the command to
... some_archive.7z | FIND /V "ing "
Note double space after 'ing'. This resulted in the following output.
7-Zip 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18
Scanning
Updating some_archive.7z
Everything is Ok
This removes the individual file processing, but produces a summary of the overall operation, regardless of the operation type.
One possibility would be to spawn the child process with popen, so its output will come back to the parent to be processed/displayed (if desired) or else completely ignored (create your popen object with stdout=PIPE and stderr=PIPE to be able to retrieve the output from the child).
Like they said, to hide most of the screen-filling messages you could use ... some_archive.7z | FIND /V "Compressing" but that "FIND" would also remove the error messages that had that word. You would not be warned. That "FIND" also may have to be changed because of a newer 7-zip version.
7-zip has a forced verbose output, no silence mode, mixes stderr and stdout(*), doesn't save Unix permissions, etc. Those anti-standards behaviors together put "7-zip" in a bad place when being compared to "tar+bzip2" or "zip", for example.
(*) "Upstream (Igor Pavlov) does not want to make different outputs for messages, even though he's been asked several times to do so :(" http://us.generation-nt.com/answer/bug-346463-p7zip-stdout-stderr-help-166693561.html - "Igor Pavlov does not want to change this behaviour" http://sourceforge.net/tracker/?func=detail&aid=1075294&group_id=111810&atid=660493
7zip does not have an explicit "quiet" or "silent" mode for command line extraction.
One possibility would be to spawn the child process with popen, so its output will come back to the parent to be processed/displayed (if desired) or else completely ignored (create your popen object with stdout=PIPE and stderr=PIPE to be able to retrieve the output from the child).
Otherwise Try doing this:
%COMSPEC% /c "%ProgramFiles%\7-Zip\7z.exe" ...
Expanding on #Matthew 's answer and this answer https://superuser.com/questions/194659/how-to-disable-the-output-of-7-zip
I'm using FINDSTR instead of find so I can chain multiple lines to exclude and blank lines as well:
7za.exe a test1.zip .\foldertozip | FINDSTR /V /R /C:"^Compressing " /C:"Igor Pavlov" /C:"^Scanning$" /C:"^$" /C:"^Everything is Ok$"
/V: exclude
/R: regex
/C:"^Compressing " : begining of line, Compressing, 2 spaces
/C:"^Scanning$" : the word Scanning on its own on a line (begining/end)
/C:"^$" : a begining and end without anything in between, ie, a blank line
I'm using /C so that a space is a space, otherwise it's a separator between multiple words to exlude as in this simpler version:
FINDSTR /V "Compressing Pavlov Scanning Everytyhing"
(the same caveats exist, if the wording changes in a new version, or if a useful line starts with the word "Compressing ", it will not work as expected).
If you're running 7-zip.exe from Powershell, and you only want to see errors, then you could try something like this:
7-zip.exe u <Target> <Source> | Select-String "Error" -Context 10
This will only display the "Error" message line and the surrounding 10 lines (or whatever number) to capture the error specific output.
The | FIND is a good alternative to show what happened without displaying insignificant text.
Examining 7zip source I found hidden -ba switch that seems to do the trick. Unfortunately it is not finished. I managed to make it work with several modifications of sources but it's just a hack. If someone's interested, the option variable is called options.EnableHeaders and changes are required in CPP/7zip/UI/Console/Main.cpp file.
Alternatively you can poke 7Zip's author to finish the feature in tracker. There are several requests on this and one of them is here.
7-zip has not such an option. Plus the lines printed at each file compressed are supposed to display at the same spot without newline, erasing the previous one, which has a cool effect. Unfortunatly, in some contexts (Jenkins...) it produced several lines ☹️ flooding the console.
NUL (windows) is maybe one solution.
7-zip.exe -o some_dir x some_archive.7z>NUL
To show just the last 4 lines...
7z x -y some_archive.7z | tail -4
gives me:
Everything is Ok
Size: 917519
Compressed: 171589
The switch -y is to answer yes to everything (in my case to override existing files).
On Unix-like operating systems (Linux, BSD, etc.) the shell command 7z ... >/dev/null will discard all text written by 7z to standard output. That should cover all the status/informational messages written by 7z.
It seems that 7z writes error messages to standard error so if you do >/dev/null, error messages will still be shown.
As told by Fr0sT above, -ba switch outputs only valid things (at least in list option on which I was trying).
7z.exe l archive_name.zip
7z.exe l -ba archive_name.zip
made great difference, esp for parsing the output in scripts.
There is no need to modify anything, just use -ba switch in version19. This was also told bysomeone above. I'm putting as answer as I can't comment.
You can stop 7-Zip from displaying prompts by using the -y switch. This will answer yes to all prompts. Use this only when you are confident.
Related
I would like to use a windows command that is processing many files.
The syntax of the command is as follows & requires a separated list of filenames: command "file_1 file_2 file_3 file_4" output-file
I have to handle 1000s of files.
Is there any way generate the list of files automatically in the command line?
Something like:
command "(echo file_1.txt to file_1000.txt)" output-file
Thanks a lot!
If your question is "how do I create a list of files numbered 1 to 1000," then you can do this in PowerShell:
1..1000 | % { New-Item file_$_.txt }
Note that % is an alias for ForEach-Object. The $_ token means "current object from the pipeline" (i.e., the number 1-1000).
This question has many open-ended issues. It is unlikely that all files can be processed by a single command since cmd has a line length limit. However, you can process them one at a time. It is unclear what the output-file would contain.
FOR %A IN ("file_*") DO (command "%A")
If this is in a .bat file script, double the PERCENT character on the variable name.
FOR %%A IN ("file_*") DO (command "%%~A")
I've written a program that returns keycodes as integers for DOS
but i don't know how to get it's output as a variable.
Note: I'm using MS-DOS 7 / Windows 98, so i can't use FOR /F or SET /P
Does anyone know how i could do that?
A few solutions are described by Eric Pement here. However, for older versions of cmd the author was forced to use external tools.
For example, program tools like STRINGS by Douglas Boling, allows for following code:
echo Greetings! | STRINGS hi=ASK # puts "Greetings!" into %hi%
Same goes for ASET by Richard Breuer:
echo Greetings! | ASET hi=line # puts "Greetings!" into %hi%
One of alternative pure DOS solutions needs the program output to be redirected to the file (named ANSWER.DAT in example below) and then uses a specially prepared batch file. To cite the aforementioned page:
[I]n the batch file we need to be able to issue the command
set MYVAR={the contents of ANSWER.DAT go here}. This is a difficult task, since MS-DOS doesn't offer an easy way to prepend "set MYVAR=" to a file [...]
Normal DOS text files and batch files end all lines with two consecutive bytes: a carriage return (Ctrl-M, hex 0D, or ASCII 13) and a linefeed (Ctrl-J, hex 0A or ASCII 10). In the batch file, you must be able to embed a Ctrl-J in the middle of a line.
Many text editors have a way to do this: via a Ctrl-P followed by Ctrl-J (DOS EDIT with Win95/98, VDE), via a Ctrl-Q prefix (Emacs, PFE), via direct entry with ALT and the numeric keypad (QEdit, Multi-Edit), or via a designated function key (Boxer). Other editors absolutely will not support this (Notepad, Editpad, EDIT from MS-DOS 6.22 or earlier; VIM can insert a linefeed only in binary mode, but not in its normal text mode).
If you can do it, your batch file might look like this:
#echo off
:: assume that the datafile exists already in ANSWER.DAT
echo set myvar=^J | find "set" >PREFIX.DAT
copy PREFIX.DAT+ANSWER.DAT VARIAB.BAT
call VARIAB.BAT
echo Success! The value of myvar is: [%myvar%].
:: erase temp files ...
for %%f in (PREFIX.DAT ANSWER.DAT VARIAB.BAT) do del %%f >NUL
Where you see the ^J on line 3 above, the linefeed should be embedded at that point. Your editor may display it as a square box with an embedded circle.
I think I've looked sufficiently for this, but I haven't come up with much. Or rather I think I've found pieces, but I am not strong enough in scripting/BAT/Programming to make it all come together. I have an idea for how this should go, but I think I may be overcomplicating it a great deal. So I need help getting the final solution and perhaps some refinement in the code/process.
I am trying to create a BAT file or two that allows me to deauthenticate any mac-addresses stuck in Idle on our Cisco wireless routers. I've got my initial BAT file to login to the router (via plink) and to spit out an output file with all the mac-addresses in it.
Now that I have the output file though, I need to strip the first 14 characters from each line, concatenate them within another command and feed them back to the router to force deauthentication.
The output is random in content and lines, but it is formatted. So I know I can reliably say the first 14 chars of each line are what I need.
I found a couple things similar but I am too much of a newb to adapt them to my needs. But the string commands I am reading about
set str=
echo.%str%
set str=%str:~0,14%
echo.%str%
look like they are meant to be within the same file and don't span multiple or unknown numbers of lines. I may very well be wrong, again; newb.
After I get the first 14 chars though I also need to combine it into the middle of command. I've read about concatenating between files but it appears to output more like
one
potato
two
instead of
one potato two
and I need the form of the latter.
Hopefully by the end I'd have a file that looks like
one potato two
one carrot two
one apple two
one etc two
I think I can figure out how to send it back into the router if I can get the stripped and concatenated form.
So I guess the shorter version is, how do I take the first 14 letters of each line in a randomized output.txt file, combine it into the middle of a command (one insertHere two), and pass it out to op2.txt?
Thanks for any help, and I apologize if this isn't clear or is a repost of some kind. I'm at a basic level for all of these topics, so explanations of why things are the way they are would also be helpful. Reading through CMD /help topics has not really cleared anything up for me yet.
EDIT: Here is what I use to get the output file and what the output looks like.
#echo on
REM Log in to Cisco box via ssh
plink -v -ssh USERNAME#X.X.X.X -pw PWD -m "c:\temp\WirelessDump.txt" >> "C:\temp\output.txt"
pause
the -m argument pushes the following cisco command sho wir cl sum | inc Idle and then the >> pushes the info into output.txt which looks like this
10ae.xxxx.xxxx APe4c7.xxxx.xxxx 3 Idle 11n(2.4)
5c0a.xxxx.xxxx APc067.xxxx.xxxx 3 Idle 11n(2.4)
6809.xxxx.xxxx APe4c7.xxxx.xxxx 3 Idle 11n(2.4)
8019.xxxx.xxxx APc067.xxxx.xxxx 3 Idle 11n(2.4)
a826.xxxx.xxxx APe4c7.xxxx.xxxx 3 Idle 11n(2.4)
And I'd like to make my output look like
wir cli mac-address a826.xxxx.xxxx deauth forced
wir cli mac-address 10ae.xxxx.xxxx deauth forced
etc
#echo off
setlocal
for /f %%a in (C:\temp\output.txt) do >>"youroutputfilename.txt" echo wir cli mac-address %%a deauth forced
should process that data as you specify.
'for /f' reads each line of the file and assigns the first "token" to %%a. >> appends to a file as you are already aware. The rest of your text seems constant, so I've just included that verbatim.
I have the following command in a windows batch script
echo =%%k-16,INDIRECT.EXT^("'C:\Users\...\Analysis\[ObsStreamflow.xlsx]Sheet1'^!A%%k"^),INDIRECT.EXT^("'C:\Users\...\Analysis\[sim%%j.xlsx]Sheet1'^!B!val!"^),^=C%%k/1000,^=D%%k-B%%k,^=ABS^(E%%k^),^=(E%%k^)^^2,=^(B%%k-B10^),=Sqrt^(B%%k^),=SQRT^(D%%k^),=^(J%%k - B13^)^^2 >>t%%j.csv
where the omitted file path is 38 characters long (I don't think I'm hitting the line limits, but just in case this is the problem). This is a single line in my .bat file, shown here as multiple lines just to make things more readable.
The output is mostly correct, except that where I have ^^2, it just becomes 2 (so I have =(E1)2 and =(J1-B13)2. If I omit the Indirect.Ext text, and just have
echo =%%k-16,a1,b1,^=C%%k/1000,^=D%%k-B%%k,^=ABS^(E%%k^),^=(E%%k^)^^2,=^(B%%k-B10^),=Sqrt^(B%%k^),=SQRT^(D%%k^),=^(J%%k - B13^)^^2 >>t%%j.csv
it prints correctly, so the relevant comments show as =(E1)^2 and =(J1-B13)^2, which is what I am after.
I've not had any luck finding an answer, everything I have found just points to using ^^ to get echo to return ^. I cannot break this command into multiple lines, I need it to be a single row in csv format.
Any suggestions for a fix much appreciated, I only really need to use this for a week or so, don't need an elegant solution, just one that works. - I'm very new to bat scripts (and indeed programming in general), will keep trying different ideas in the mean time.
It's only the exclamation mark that creates the problems for you.
If at least one ! is in your line (and delayed expansion is enabled), then a second caret escape phase will be started.
In this phase quotes aren't regarded, only carets.
A small test
setlocal EnableDelayedExpansion
echo one^1
echo two^^2
echo two^^2 With exclam!
echo five^^^^^& With exclam!
Output
one1
two^2
two2 With exclam
four^& With exclam
So in your sample, you need five carets.
Four to create one caret and the last one to escape the ), as the escape of the special character is only once required.
Not sure what your specific problem is but you can use a trick in Windows to emulate echo -n (echo without a newline).
The commands:
<nul: >file.csv set /p junk=first field
<nul: >>file.csv set /p junk=,second field
>>file.csv echo ,third field
will result in a single line:
first field,second field,third field
That may make it easier for you to avoid the specific problem and, as a bonus, clean up your script so it's a little more readable (such as one field per script line).
It works because set /p var=prompt is the input command. It first outputs prompt without a newline then waits for the user to enter something, assigning it to the var environment variable.
By getting input from nul:, you basically give it an empty string so it doesn't wait. The prompt is output to file.csv without the newline.
In any case, for something this complex, I'd be bypassing cmd.exe for something a little more powerful such as the UNIX text processing tools under CygWin or MinGW (which require installation but are well worth it), or even VBScript scripts (which should be on Windows by default), where you can more easily control the output.
I discovered this quite by accident while looking for a file with a number in the name. When I type:
dir *number*
(where number represents any number from 0 to 9 and with no spaces between the asterisks and the number)
at the cmd.exe command prompt, it returns various files do not appear in any to fit the search criteria. What's weird, is that depending on the directory, some numbers will work and not others. An example is, in a directory associated with a website, I type the following:
dir *4*
and what is returned is:
Directory of C:\Ampps\www\includes\pages
04/30/2012 03:55 PM 153 inventory_list_retrieve.php
06/18/2012 11:17 AM 6,756 ix.html
06/19/2012 01:47 PM 257,501 jquery.1.7.1.js
3 File(s) 264,410 bytes
0 Dir(s) 362,280,906,752 bytes free
That just doesn't make any sense to me. Any clue?
The question is posed on stackOverflow because the DIR command is often combined with FOR in batch programs. The strange DIR behavior would seem to make batch programs potentially unreliable if they use the DIR command.
Edit: (additional note). Though much time has passed, I discovered another quirk with this that almost cost me a lot of work. I wanted to delete all .htm files in a particular directory tree. I realized just before doing it that *.htm matches .html files as well. Also, *.man matches .manifest, and there are probably others. Deleting all .html files in that particular directory would have been upsetting to say the least.
Wild cards at the command prompt are matched against both the long file name and the short "8.3" name if one is present. This can produce surprises.
To see the short names, use the /X option to the DIR command.
Note that this behavior is not in any way specific to the DIR command, and can lead to other (often unpleasant) surprises when a wild card matches more than expected on any command, such as DEL.
Unlike in *nix shells, replacement of a file pattern with the list of matching names is implemented within each command and not implemented by the shell itself. This can mean that different commands could implement different wild card pattern rules, but in practice this is quite rare as Windows provides API calls to search a directory for files that match a pattern and most programs use those calls in the obvious way. For programs written in C or C++ using the "usual" tools, that expansion is provided "for free" by the C runtime library, using the Windows API.
The Windows API in question is FindFirstFile() and its close relatives FindFirstFileEx(), FindNextFile(), and FindClose().
Oddly, although the documentation for FindFirstFile() describes its lpFileName parameter as "directory or path, and the file name, which can include wildcard characters, for example, an asterisk (*) or a question mark (?)" it never actually defines what the * and ? characters mean.
The exact meaning of the file pattern has history in the CP/M operating system dating from the early 1970s that strongly influenced (some might say "was directly copied" in place of "influenced" here) the design of MSDOS. This has resulted in a number of "interesting" artifacts and behaviors. Some of this at the DOS end of the spectrum is described at this blog post from 2007 where Raymond describes exactly how file patters were implemented in DOS.
Yep. You'll see that it also searches through short names if you try this:
dir /x *4*
(/x switch is for short names)
for filtering file names use :
dir /b | find "4"
A quote from RBerteig's answer:
Note that this behavior is not in any way specific to the DIR command,
and can lead to other (often unpleasant) surprises when a wild card
matches more than expected on any command, such as DEL.
The above is true even for the FOR command, which is very nasty.
for %A in (*4*) do #echo %A contains a 4
will also search the short names. The solution again would be to use FIND or FINDSTR to filter out the names in a more reliable manner.
for %A in (*) do #echo %A | >nul findstr 4 && echo %A contains a 4
Note - change %A to %%A if using the command within a batch file.
Combining FOR with FINDSTR can be a general purpose method to safely use any command that runs into problems with short file names. Simply replace ECHO with the problem command such as COPY or DEL.
Seems like dir command searches also short ( 8.3 manner ) file names under the hood.
When I call dir *1* this is what I get:
Volume in drive C is System
Volume Serial Number is F061-0B78
Directory of C:\Users\Piotrek\Desktop\Downloads
2012-05-20 17:33 23 639 040 gDEBugger-5_8.msi
2012-05-20 17:30 761 942 glew-1.7.0.zip
2012-05-20 17:11 9 330 176 irfanview_plugins_433_setup.exe
2012-05-24 20:17 4 419 192 SumatraPDF-2.1.1-install.exe
2012-05-15 22:55 3 466 248 TrueCrypt Setup 7.1a.exe
5 File(s) 1 127 302 494 bytes
There is a gDEBugger-5_8.msi file amongst listed ones, which apparently does not have any 1 character in it.
Everything becomes clear when I use /X switch with the dir command, which makes dir use 8.3 file names. Output from a dir /X *1* command:
Volume in drive C is System
Volume Serial Number is F061-0B78
Directory of C:\Users\Piotrek\Desktop\Downloads
2012-05-20 17:33 23 639 040 GDEBUG~1.MSI gDEBugger-5_8.msi
2012-05-20 17:30 761 942 GLEW-1~1.ZIP glew-1.7.0.zip
2012-05-20 17:11 9 330 176 IRFANV~1.EXE irfanview_plugins_433_setup.exe
2012-05-24 20:17 4 419 192 SUMATR~1.EXE SumatraPDF-2.1.1-install.exe
2012-05-15 22:55 3 466 248 TRUECR~1.EXE TrueCrypt Setup 7.1a.exe
5 File(s) 1 127 302 494 bytes
Quote from dir's help:
/X This displays the short names generated for non-8dot3 file
names. The format is that of /N with the short name inserted
before the long name. If no short name is present, blanks are
displayed in its place.