I am attempting to export some database data using the BCP Utility.
Here is my batch command so far:
BCP [table] out [file] -c -T -S [server] -t"¶" | 7z.exe a -si [archive name] -sdel
The BCP part works just fine:
BCP [table] out [file] -c -T -S [server] -t"¶"
However, for the 7-Zip part:
7z.exe a -si [archive name] -sdel
It works to a point. The original file is not removed, and I'd also like to encrypt the archive with 128 bit or 256 bit encryption with a password.
Any suggestions?
I found a work around solution with a small VB .NET script.
The script takes in a table name, runs BCP into a text file, runs 7 Zip with encryption options (https://sevenzip.osdn.jp/chm/cmdline/switches/method.htm#Zip), and a password, then deletes the original text file.
These commands are run using the Process() object functions.
That way I can loop through the tables I need placed in files easily.
It is not the Windows batch answer I was looking for, but it works.
Any other suggestions are still welcome.
Thanks!
BCP .... | 7z u -sidirData -pMyPassword -mhe outputFile.7z
^ ^ ^ ^ ^______________ The file that will be generated
| | | |___________________ Encrypt file names
| | |________________________________ Password used for encryption
| |___________________________________________ Name of stored file
|_____________________________________________ update/create container file
Note that there are no spaces between the switches and the values
Related
I am trying to upload a file in redis using command:
redis-cli -p <Port> -h <Host> -n <DB> -x set <key> < /tmp/file.json
The problem is : in redis value -
It is storing a \n at the end of line and I don't want this.
"{\"items\":{\"38749\":{\"buyone\":0,\"buytwo\":10},\"38712\":{\"buyone\":0,\"buytwo\":10},\"112775\":{\"buyone\":0,\"buytwo\":10},\"38721\":{\"buyone\":0,\"buytwo\":10},\"38720\":{\"buyone\":0,\"buytwo\":10},\"38714\":{\"buyone\":0,\"buytwo\":10},\"38726\":{\"buyone\":0,\"buytwo\":10},\"38733\":{\"buyone\":0,\"buytwo\":10},\"38729\":{\"buyone\":0,\"buytwo\":10},\"113708\":{\"buyone\":0,\"buytwo\":10},\"38731\":{\"buyone\":0,\"buytwo\":10},\"38745\":{\"buyone\":0,\"buytwo\":10},\"38732\":{\"buyone\":0,\"buytwo\":10},\"100074\":{\"buyone\":0,\"buytwo\":10},\"38730\":{\"buyone\":0,\"buytwo\":10},\"38719\":{\"buyone\":0,\"buytwo\":10},\"38723\":{\"buyone\":0,\"buytwo\":10},\"38755\":{\"buyone\":0,\"buytwo\":10},\"38760\":{\"buyone\":0,\"buytwo\":10}}}\n"
Since your file contains newlines, that's what gets stored in Redis.
You'll need to strip the newlines from your file before setting its contents to Redis. Depending on your OS, the method may vary. Here's a question about this: How do I remove newlines from a text file?
Finally got this working -
cat /tmp/up.json | redis-cli -n 20 --pipe
and up.json contents
set 'PACKAGES_CONFIG' '{"checkOfferFieldPrices":true,"showPrescInfoScreen":true,"showAddOnsScreen":true,"offerText":"1 + 1 with Lenskart Gold","bannerConfig":{"isVisible":true,"primaryText":"Hi %s, You are a GOLD Member!","secondaryText":"You are eligible for Buy 1 Get 1 offer on this order!"},"isExpandedByDefault":true,"isPreSelected":true,"displayBogoTabs": true, "defaultSelectedTabId": "buy2","tabConfig":[{"id":"buy1","title":"Buy 1","subtitle":"No Offer","enabled":true},{"id":"buy2","title":"Buy 2","subtitle":"Buy 1 Get 1 Free","enabled":true}]}'
I am currently using this command to do a restore in mysql of a gzipped file.
C:\...directory of gzip.exe >gunzip -c filename.gz | mysql -u.. -p.. -P.. -h dbname
I would like to extract files that are located in a directory that is different from the one in which gzip.exe is located.
How should i modify the instruction?
Change to the directory containing the .gz file and either 1) specify the path to gunzip.exe or 2) add the directory containing gunzip.exe to your PATH variable.
"C:\path\to\gunzip.exe" -c filename.gz | mysql -u.. -p.. -P.. -h dbname
I want to dump only a specific column on some text file using parquet-tools-1.8.1.jar.But not able to do so. I am trying below command. Please note my column name has forward slash.
parquet-tools-1.8.1.jar dump --column 'dir1/log1/job12121' '/hdfs-path/to/parquet file with space.parquet' > /home/local/parquet/output.text
Run
hadoop jar parquet-tools-1.8.1.jar parquet.tools.Main dump --column 'dir1/log1/job12121' '/hdfs-path/to/parquet file with space.parquet' > /home/local/parquet/output.text
Please use the following:
hadoop jar parquet-tools-1.8.1.jar dump -c dir1 log1 job12121 -m /hdfs-path/to/parquet file with space.parquet >> /home/local/parquet/output.text
Note:No single quotes for input arguments.
After running a set of tests with drmemory overnight I am trying to resolve the error stacks by providing pdb symbols. The pdb's come from a large samba-mapped repository and using _NT_SYMBOL_PATH at runtime slowed things down too much.
Does anyone know of a tool that post-processes results.txt and pulls new symbols (via NT_SYMBOL_PATH or otherwise) as required to produce more detailed stacks ? If not, any hints for adapting asan_symbolize.py to do this ?
https://llvm.org/svn/llvm-project/compiler-rt/trunk/lib/asan/scripts/asan_symbolize.py
What I came up with so far using dbghelp.dll is below. Works but could be better.
https://github.com/patraulea/postpdb
ok this Query does not pertain to use of windbg or doesn't have anything to do with _NT_SYMBOL_PATH
Dr.Memory is a memory diagnostic tool akin to valgrind and is based on Dynamorio instumentation framework usable on raw unmodified binaries
on windows you can invoke it like drmemory.exe calc.exe from a command prompt (cmd.exe)
as soon as the binary finishes execution a log file named results.txt is written to a default location
if you had setup _NT_SYMBOL_PATH drmemory honors it and resolves symbol information from prepulled symbol file (viz *.pdb) it does not seem to download files from ms symbol server it simply seems to ignore the SRV* cache and seems to use only the downstream symbol folder
so if the pdb file is missing or isnt downloaded yet
the results.txt will contain stack trace like
# 6 USER32.dll!gapfnScSendMessage +0x1ce (0x75fdc4e7 <USER32.dll+0x1c4e7>)
# 7 USER32.dll!gapfnScSendMessage +0x2ce (0x75fdc5e7 <USER32.dll+0x1c5e7>)
while if the symbol file was available it would show
# 6 USER32.dll!InternalCallWinProc
# 7 USER32.dll!UserCallWinProcCheckWow
so basically you need the symbol file for appplication in question
so as i commented you need to fetch the symbols for the exe in question
you can use symchk on a running process too and create a manifest file
and you can use symchk on a machine that is connected to internet
to download symbols and copy it to a local folder on a non_internet machine
and point _NT_SYMBOL_PATH to this folder
>tlist | grep calc.exe
1772 calc.exe Calculator
>symchk /om calcsyms.txt /ip 1772
SYMCHK: GdiPlus.dll FAILED - MicrosoftWindowsGdiPlus-
1.1.7601.17514-gdiplus.pdb mismatched or not found
SYMCHK: FAILED files = 1
SYMCHK: PASSED + IGNORED files = 27
>head -n 4 calcsyms.txt
calc.pdb,971D2945E998438C847643A9DB39C88E2,1
calc.exe,4ce7979dc0000,1
ntdll.pdb,120028FA453F4CD5A6A404EC37396A582,1
ntdll.dll,4ce7b96e13c000,1
>tail -n 4 calcsyms.txt
CLBCatQ.pdb,00A720C79BAC402295B6EBDC147257182,1
clbcatq.dll,4a5bd9b183000,1
oleacc.pdb,67620D076A2E43C5A18ECD5AF77AADBE2,1
oleacc.dll,4a5bdac83c000,1
so assuming you have fetched the symbols it would be easier to rerun the tests with a locally cached copies of the symbol files
if you have fetched the symbols but you cannot rerun the tests and have to work solely with the output from results.txt you have some text processing work (sed . grep , awk . or custom parser)
the drmemory suite comes with a symbolquery.exe in the bin folder and it can be used to resolve the symbols from results.txt
in the example above you can notice the offset relative to modulebase like
0x1c4e7 in the line # 6 USER32.dll!gapfnScSendMessage +0x1ce (0x75fdc4e7 {USER32.dll+0x1c4e7})
so for each line in results.txt you have to parse out the offset and invoke symbolquery on the module like below
:\>symquery.exe -f -e c:\Windows\System32\user32.dll -a +0x1c4e7
InternalCallWinProc+0x23
??:0
:\>symquery.exe -f -e c:\Windows\System32\user32.dll -a +0x1c5e7
UserCallWinProcCheckWow+0xb3
a simple test processing example from a result.txt and a trimmed output
:\>grep "^#" results.txt | sed s/".*<"//g
# 0 system call NtUserBuildPropList parameter #2
USER32.dll+0x649d9>)
snip
COMCTL32.dll+0x2f443>)
notice the comctl32.dll (there is a default comctl.dll in system32.dll and several others in winsxs you have to consult the other files like global.log to view the dll load path
symquery.exe -f -e c:\Windows\winsxs\x86_microsoft.windows.common-
controls_6595b64144ccf1df_6.0.7601.17514_none_41e6975e2bd6f2b2\comctl32.dll -a +0x2f443
CallOriginalWndProc+0x1a
??:0
symquery.exe -f -e c:\Windows\system32\comctl32.dll -a +0x2f443
DrawInsert+0x120 <----- wrong symbol due to wrong module (late binding
/forwarded xxx yyy reasons)
I am trying a code in shell script. while I am trying to convert the code from batch script to shell script I am getting an error.
BATCH FILE CODE
:: Create a file with all latest snapshots
FOR /F "tokens=5" %%a in (' ec2-describe-snapshots ^|find "SNAPSHOT" ^|sort /+64') do set "var=%%a"
set "latestdate=%var:~0,10%"
call ec2-describe-snapshots |find "SNAPSHOT"|sort /+64 |find "%latestdate%">"%EC2_HOME%\Working\SnapshotsLatest_%date-today%.txt"
CODE IN SHELL SCRIPT
#Create a file with all latest snapshots
FOR snapshot_date in $(' ec2-describe-snapshots | grep -i "SNAPSHOT" |sort /+64') do set "var=$snapshot_date"
set "latestdate=$var:~0,10"
ec2-describe-snapshots |grep -i "SNAPSHOT" |sort /+64 | grep "$latestdate">"$EC2_HOME%/SnapshotsLatest_$today_date"
I want to sort the snapshots according to dates and to save the snapshots that are created in latest date in a file.
SAMPLE OUTPUT OF ece-describe-snapshots:
SNAPSHOT snap-5e20 vol-f660 completed 2013-12-10T08:00:30+0000 100% 109030037527 10 2013-12-10: Daily Backup for i-2111 (VolID:vol-f9a0 InstID:i-2601)
It will contain records like this
I got this code :
latestdate=$(ec2-describe-snapshots | grep ^SNAPSHOT | sort -k 5 | awk '{print $5}')
ec2-describe-snapshots | grep SNAPSHOT.*$latestdate | > "$EC2_HOME/SnapshotsLatest_$today_date"
but getting this error :
grep: 2013-12-10T09:55:34+0000: No such file or directory
grep: 2013-12-11T04:16:49+0000: No such file or directory
grep: 2013-12-11T04:17:57+0000: No such file or directory
i have some snapshots made on amazon, i want to find the latest snapshots made on a date and then want to store them in a file. like date 2013-12-10 snapshots made on this date should be stored in file. Contents of snapshotslatest file should be
SNAPSHOT snap-c17f3 vol-f69a0 completed 2013-12-04T09:24:50+0000 100% 109030037527 10 2013-12-04: Daily Backup for Sanjay_Test_Machine (VolID:vol-f66409a0 InstID:i-26048111)
SNAPSHOT snap-c7d617f9 vol-3d335f6b completed 2013-12-04T09:24:54+0000 100% 109030037527 10 2013-12-04: Daily Backup for sacht_VPC (VolID:vol-3db InstID:i-ed6)
please not that if there are snapshots created on 2013-12-10, 2013-12-11, 2013-12-12. It means that the latest_date should be 2013-12-12 and all the snaphshot created on 2013-12-12 should be saved in file.
Any suggestion or lead is appreciated.
Neither the batch script nor the shell script you posted are a good starting point so let's start from scratch. Sorry, this is too big for a comment.
You want to find the latest snapshots made on a date and then want to store them in a file.
What does that mean?
Do the snapshot files have a timestamp in their name or in their content?
If not - UNIX does not store file creation timestamps so is a last-modified timestamp adequate?
Do you literally want to concatenate all of your snapshot files into one singe file or do you want to create a file that has a list of the snapshot file names?
Post some sample input (e.g. some snapshot file names and contents if that's where the timestamp is stored) and the expected output given that input.
Update your question to address all of the above, do not try to reply in a comment.
Minor issue, you don't need a pipe when re-directing output, so your line to save should be
ec2-describe-snapshots | grep SNAPSHOT.*$latestdate > "$EC2_HOME/SnapshotsLatest_$today_date"
Now the main issue here, is that the grep is messed up. I haven't worked with amazon snapshots, but judging by your example descriptions, you should be doing something like
latestdate=$(ec2-describe-snapshots | grep -oP "\d+-\d+-\d+" | sort -r | head -1)
This will get all the dates containing the form dddd-dd-dd from the file (I'm assuming the two dates in each snapshot line always match up), sort them in reverse order (latest first) and take the head which is the latest date, storing it in $latestdate.
Then to store all snapshots with the given date do something like
ec2-describe-snapshots | grep -oP "SNAPSHOT(.*?)$lastdateT(.*?)\)" > "$EC2_HOME/SnapshotsLatest_$today_date"
This will get all text starting with SNAPSHOT, containing the given date, and ending in a closing ")" and save it. Note, you may have to mess around with it a bit, if ")" can be present elsewhere.