postgres COPY (...) TO PROGRAM on Windows giving error - windows

I am trying to dump PostgreSQL tables using a COPY statement. I am using COPY instead of pg_dump because I need to perform a transformation on one of the columns as it comes out.
The statement I would like to execute is essentially this
COPY ( SELECT 'punt' ) TO PROGRAM '7z a -si C:/Users/Public/Documents/punt';
but I am getting the generic error
ERROR: program "7z a -si C:/Users/Public/Documents/punt" failed
DETAIL: child process exited with exit code 1
I am trying to pass the COPY output through 7zip similar to the fourth example in the COPY documentation in order to compress the output as the database is about 130 GB and I'm ideally working with SSDs. The following command works fine:
COPY ( SELECT 'punt' ) TO 'C:/Users/Public/Documents/punt';
with result COPY 1. I have executed the following command on the psql prompt and it also works which makes me fairly confident about syntax and permissions:
\! echo 'punt' | 7z a -si C:/Users/Public/Documents/punt
Am I missing something about the cmd pipeline or the 7zip executable permissions?
EDIT: Postgres version is 9.3.3
I'm still getting the same error after changing the dir permissions to NETWORK SERVICE based on Craig Ringer's answer. This, however, did not fix the problem and the error is identical. Even after changing the target directory's permissions giving full control to Everyone, I still got the same error. I also tried changing the permission on the 7zip executable and the 7zip Program Files Directory to no avail.
In experimentation, I ran the following, taking the input from copy out of the equation:
COPY ( select '1' ) TO PROGRAM 'echo punt | 7z a -si C:/Users/Public/Documents/punt';
and curiously got the failure message, but instead with exit code 255, which for 7zip has the meaning "User stopped the process". Can't seem to find a reason for this myself.

It'll be a permissions issue.
The PostgreSQL server runs as NETWORKSERVICE for 9.2 and above, or user postgres for 9.1 and below, by default. COPY runs server-side as the user PostgreSQL its self runs as.
Neither of these are likely to have the right to write to C:\Users\Public\Documents.
You'll need to grant the PostgreSQL service user the right to write there using the Windows security properties dialog, or choose another location it can write to.
It works with \! because psql is running as your user, which will have permissions to write there, and \! runs the command client-side as the user running psql.

Strikethrough the 7zip bit in my edit, the 255 error code is actually from cmd not finding the 7zip exe. I plopped 7zip.exe on C:/ (couldn't figure out escaping the space in Program Files) and ran
COPY ( SELECT 'punt' ) TO PROGRAM 'C:/7z a -si C:/Users/Public/Documents/punt';
and this worked. I'm guessing the NETWORKSERVICE user doesn't have the path environment variable loaded, but I'm not sure about that.
I set the permissions on C:/Users/Public/Documents back to its default and the COPY ... TO PROGRAM still worked so file permission wasn't an issue in this specific case. I would still look to Craig Ringer's suggestion first if I encountered a problem like this again.

CMD doesn't recognize a path with spaces.
COPY FROM PROGRAM doesn't have access to PATH variables.
Just wrap your PROGRAM_NAME like RESOURCE_NAME in double apostrophes and you are done:
COPY ( SELECT 'punt' ) TO PROGRAM '"C:/Program Files/bla/bla/7z.exe" a -si "C:/Users/Public/Documents/punt"';

Related

Proper syntax for psexec.exe hostname with filename to start

I'm trying to use psexec.exe to fire up Excel and then use Excel to open a specific .xlsx file.
I've just started learning psexec and I have written commands that work 'incrementally' in order to be sure my initial building blocks were correct along the way. Decided to actually start with NOTEPAD and a specific TEXT FILE to start:
(Please note, my goal is to execute this process on the local machine, not a remote machine, and for it to be Interactive, thus the -i switch)
This Works (just opens a blank Notepad window)
"C:\Users\Username\OneDrive - OrgName\Desktop\RunAs Test\psexec.exe" -i -u domain\username -p Password C:\Windows\system32\notepad.exe
This does not work - and I am positive the exe path and the file path are correct, I've tried them separately
"C:\Users\Username\OneDrive - OrgName\Desktop\RunAs Test\psexec.exe" -i -u domain\username -p Password C:\Windows\system32\notepad.exe \\server\data\mpsc-users\UserName\Test\sadf.txt
(It just returns the boilerplate 3 lines copyright, but with no mention of Error Code or results at all. Nothing else happens)
PsTools.chm is very clear on why it wont work
"Arguments to pass (note that file paths must be absolute paths on the
target system)"
"psexec.exe" -i \\somewhere.... C:\Windows\system32\notepad.exe c:\users\Isaac\desktop\hello.txt
should work (if the file exists on the remote system)

"ERROR: Invalid syntax. Default option is not allowed more than '2' time(s)." using SETX

I thought I had this working at one time but recently my command quit working. My research here and abroad is not turning up anything I have not attempted already. Perhaps there is some other variable I am missing that SETX is looking at?
My command:
setx /s servername PATH "%Path%";"c:\program files (x86)\java\jre7\bin";"c:\program files (x86)\java\jre7\bin\client" /m
I have tried the same command against a couple of Win2k8 servers to no avail. This has worked but is now producing the error above. I usually run this within a batch file against a list of several servers to push Java out. (Path to java is not being set unless we manually create the entries when remotely installed.)
The existing path on the server I am attempting to update is this:
%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%systemroot%\System32\WindowsPowerShell\v1.0\;C:\BMC\Patrol3\bin;C:\BMC\common\globalc\bin\Windows-x86;C:\BMC\common\globalc\bin\Windows-x86-64;C:\BMC\Patrol3\BEST1\9.5.00\bgs\bin
I did clean up a couple duplicate path entries. Aside from that I did not find anything unusual to me.
Edit:
Tweaking the command to remove some of the quotation marks does allow it to run but with that, it imports the path of the local system replacing the remote systems path and appends the desired path entries.
The revised command:
setx /s servername PATH "%Path%;c:\program files (x86)\java\jre7\bin;c:\program files (x86)\java\jre7\bin\client" /m
In hopes of fixing this new problem, I am attempting to run the command with psexec. Doing so brings me back to my original problem, "Default option is not allowed more than '2' time(s)". Depending on how I modify the quotes when using psexec I can get "To many Arguments" or command succeeded, path truncated which ends up being the local path being applied on the remote server once again.
The most promising of my attempts:
psexec \\servername -u domain\user -p p#ssw0rd "c:\windows\system32\setx" "Path "%Path%;c:\program files (x86)\java\jre7\bin;c:\program files (x86)\java\jre7\bin\client" /M"
Using the "-s" option on psexec produces the same error, just less detail.
I was able to update the path value on remote systems by creating a batch file to be executed remotely.
Contents of "setJava7path.cmd"
setx Path "%Path%";"%ProgramFiles(x86)%\java\jre7\bin";"%ProgramFiles(x86)%\java\jre7\bin\client" /m
To execute I utilized psexec which copied the batch file to the remote system and executed the command.
psexec \\systemname -c setJava7path.cmd
It adds / changes a line to my overall deployment script but that is livable.

Download zip file from web, unzipped and import to data set using sas

I want to download one zip file from web and unzipped and import that file to sas data set for further use.
For unzipping i use below sas code.. But result is nothing with no error.
data _null_;
x "cd C:\Program Files\7-Zip ";
X "7z.exe e C:\Users\Ravinder\Downloads\Compressed\*.zip -o C:\sasdata\New";
run;
Please help to do the same.
ravinder kumar
Have you tried running your command from a command prompt? Did it work? I would recommend running the command like this rather than changing the folder first:
"C:\Program Files\7-Zip\7z.exe" e C:\Users\Ravinder\Downloads\Compressed\*.zip -o C:\sasdata\New
When you run the above command from SAS use single quotes as your string delimiter for the command like so:
data _null_;
X '"C:\Program Files\7-Zip\7z.exe" e C:\Users\Ravinder\Downloads\Compressed\*.zip -o C:\sasdata\New';
run;
If it is still not working as expected from SAS, then the easiest way to debug IMO is to put the command in a batch file (.bat), and then at the end of the file put a line that simply says pause. The pause command will tell windows to keep the batch file window open until you close it. The window will give you full visibility of what happened when you ran the command. Try running the batch file from SAS instead of running the command directly.
Note that you may also need to specify the following SAS options if you want SAS to wait for the unzip to finish before your SAS program continues processing:
options noxwait xsync;
noxwait tells SAS to close the command window automatically once the command has run, and xsync tells SAS to wait for the command to finish running before continuing to process. Note that sometimes when extracting files, the command can finish but the file will still not exist on disk due to either network copying and/or caching. If you want to be absolutely sure, you should probably check for the existence of the file and/or tell sas to sleep() for a certain amount of time.
here is my problem solution.. File download to c:\sasdata location.
data _null_;
x "cd c:\sasdata";
x "curl -A 'Mozilla/4.0' -O http://nseindia.com/content/indices/histdata/CNX%20NIFTY&FRNAME-&FRNAME..csv";
run;

Batch file - Remote Copy and Execute issue

I have a batch file that part of it copies a folder and its content to a local directory and then continues through the batch process which then tries to execute the contents of the folder . locally run it works on all wintel os and copies the files from \networkshare\folder*.* to %systemdrive%\temp\ .
I have attempted to use e.g psexec #serverlist.txt -c batfile.bat -s -f -d to copy the file and then execute it remotely how ever the issue is that this will not copy the files when run remotely . I think its a authentication issues after you a have remotely executed the batch file the remote system will not allow me to access/authenticate the networkshare
i have tried xcopy , copy , robocopy .
AFAIK you can only authenticate against the next hop using implicit credentials. Connection attempts from the first hop to a second hop will fail, even if your user has the required permisssions. See e.g. here for an explanation.
Try this:
psexec #serverlist.txt -u %USERDOMAIN%\%USERNAME% -d -c batfile.bat
Use explicit credentials so you're authenticated against the remote host. With that the second hop will be the next hop for your authenticated session. Don't run the script as LOCAL SYSTEM (-s), because that account is restricted to local resources.
I've had this issue in the past. Instead of trying to copy/run the BAT file just run the script things from a UNC path if you are able to. I think the problem lies in the BAT file not actually able to run through PSEXEC and copy like you think. Does it give an exit code?
Here's an example I used a while ago to install Adobe reader. I tried to do it through a batch file but no dice. I could not get it to run within the batch file whether I copied it or not. I can't remember the exact reason, I think it had something with the way a batch file is called in the system and runs in some sort of local context. Don't quote me on that though as I can't remember the exact why.
Here is my code example:
psexec /accepteula \\%computer% -s cmd /c msiexec /i "\\UNC\Software\adobe\Adobe Reader 7.0.9.msi" TRANSFORMS="acrobat7.mst" /qn

Problems Installing PostgreSQL 9.2

I've been trying to install the 64bit version of PostgreSQL 9.2 for Windows on my machine (Windows 7 64bit) and get this error:
The environment variable COMPSPEC does not seem to point to the cmd.exe or there is a trailing semi colon present.
I've installed it as Administrator.
I disabled the antivirus (Microsoft Security Essentials) and the firewall.
Running:
"%COMSPEC%" /C "echo test ok"
returned test ok
I've checked my System Environment Variables for trailing semi colon and I couldn't find any.
I then installed the 32bit version and managed to get to the end of the install with a different error message stating: Problem running post-install step. Installation may not complete correctly Error reading the C:\Program Files (x86)\PostgreSQL\9.2\data\postgresql.conf but there is no postgresql.conf file in that directory. It did install the application and when I try to connect the server with the red X on it it says fail at the bottom and it won't connect after I type in my password.
How can I connect to this server connection?
ComSpec is a generic error message for any installation failure.
Identifying the problem
Navigate to below path
c:\Users\XXXXXX\AppData\Local\Temp
Open 'bitrock_installer_XXXX.log'
Check, if you are getting below error:
Script stderr:
'"C:\Users\XXXXX\AppData\Local\Temp\POSTGR~1\TEMP_C~1.BAT"' is not recognized as an internal or external command, operable program or batch file.
Error running
C:\Users\XXXXX\AppData\Local\Temp/postgresql_installer_47b21c4ea1/temp_check_comspec.bat :
'"C:\Users\XXXXX\AppData\Local\Temp\POSTGR~1\TEMP_C~1.BAT"' is not recognized as an internal or external command,
operable program or batch file.
This is a problem with '8.3 file names and directories' (e.g. '\Postgres Install' -> '\POSTGR~1')
Microsoft article on disabling 8.3 file names and directories: https://support.microsoft.com/en-gb/kb/121007
Solution:
Open command prompt in admin mode
Execute following command to change the format based on your drive or all drives
Sample commands:
fsutil 8dot3name set 1" - disable 8dot3 name creation on all volumes
fsutil 8dot3name set C: 1" - disable 8dot3 name creation on c:
Execute the installation as a user having admin privileges
After install, consider resetting the 8dot3name setting to default (2) to avoid unintended consequences
Hope it solves the problem!
Very easy fix:
Just open Advanced System Settings in Control Panel and create a new System Variable( in the System Variable instead of User Variable section).
In the variable name, enter ComSpec and then in the variable value , enter C:\Windows\system32\cmd.exe.
Alternative fix:
If you have already the ComSpec variable in the System Variable section, remove the ;at the end of it this should fix it.
It's not COMPSPEC it's just COMSPEC. Please show the output of:
echo %COMSPEC%
Note that COMSPEC could be set to something different in the Administrator account you're running the installer as. I'm not sure how to find that out, but it might appear in the PostgreSQL installer log, so please upload that and link to it in your post. See Reporting an installation error for info on where to get the installer log.
See the PostgreSQL for Windows FAQ entry Check the COMSPEC environment variable.
Here's a report I made suggesting that the installer should test for this explicitly and here's my blog post on the topic.
I got the same problem, and i found in the log:
Script stderr:
'C:\Users\S300' is not recognized as an internal or external command,
operable program or batch file.
Error running C:\Users\S300 (i5)\AppData\Local\Temp/postgresql_installer_56caeadbd6/temp_check_comspec.bat : 'C:\Users\S300' is not recognized as an internal or external command,
operable program or batch file.
I change in User Variables TEMP to D:\TEMP and TMP to D:\TEMP.
And Solved My Problem.
In my case , the Installer was in %USERPROFILE%\DownloadsP{ Windows download folder}, I moved the installer to desktop and ran again. weird it worked lol.
I had a similar problem. After installation, the data folder contained no postgres.conf file. It only contained a single folder named "pg_log".
I described the solution that I used here: Postgres Installation Error reading file postgresql.conf
Basically, it would be helpful to check if the user has full permissions for the postgres folder, and run "init_db" and "pg_ctl start" commands again. If the path contains a space character, try using a relative path for the pg_ctl data folder argument.
I'm running Windows Server 2003 R2, and I have been unable to resolve this problem with the installer, so I resorted to using the binary PostgreSQL package. Hopefully this will be an alternative for others who do not want to perform an OS reinstall.
First, some background (hopefully useful to the developers)
It started out with the postgres service failing to start (the server had been running reliably for over a year). I assumed it was a corrupted PostgreSQL installation, so I uninstalled and attempted to reinstall. I encountered the following error:
There has been an error.
The environment variable COMSPEC does not seem to point to the cmd.exe or there is a trailing semicolon present.
Please fix this variable and restart installation.
However, the COMSPEC variable is set properly, verified with:
echo %COMSPEC%
C:\WINDOWS\system32\cmd.exe
and:
"%COMSPEC%" /C "echo test ok"
test ok
Since this is Windows Server 2003, there is no UCA wrapper around the Administrator account, so that is not causing the problem.
Manual Installation
NET USER postgres /ADD
C:\pgsql\bin\initdb.exe -U postgres -A password -E utf8 -W -D C:\pgsql\data
runas /user:postgres "C:\pgsql\bin\pg_ctl -D C:/pgsql/data -l C:/pgsql/logfile.txt start"
just do it run as administrator and change the environment system variable
like create a new variable 'ComSpec' and value type 'C:\Windows\system32\cmd.exe'.
If the installer exe is on a network share that mapped drive might actually not be accessible to the installer as it runs as administrator. This can often happen in some virtual machine arrangements such as running windows in a parallels VM. Copy the installer to a local drive first and you won't have a problem.
What worked for me after trying to enter the commandline given her in cmd.exe
I found it was named cmd1.exe in system32.. so i copied the file and renamed it as cmd.exe and installation finished
Open Environment Variables, you can do this on Windows 7 by typing environment variables in the Search program and files bar when pressing the start button at the bottom left of the desktop. And create a new System Variable(in the 'System Variable' instead of 'User Variable' section).
In the variable name, enter ComSpec and then in the variable value , enter C:\Windows\system32\cmd.exe.
That's all. Hope it works!
Alternative fix:
If you already got the ComSpec variable in the System Variable section, remove the ; at the end of it this should fix it.
First find the path to cmd.exe(mostly it is C:\Windows\System32\cmd.exe).
Go to the enviornment variable and add this path to system variable path.And also create new variable in user variable called ComSpec and add this path as value. And you are ready to go.

Resources