Is there a way to prevent sqlloader from creating either the BAD file or the DISCARD file or both?
I read in a forum that setting the BAD parameter to NUL or NULL does that, but it doesn't seem to work. Something like:
sqlldr control=CONTROL.ctl data=DATA.txt bad=NUL
This ends up creating a file called NUL.bad (or NULL.bad depending on how you spell it)
Discard file should be optional, based on a discard predicate / discard parameter, but the Bad file is not an optional file - if you get rows rejected when loading, it will generate the file.
You could try send bad to the null depending (/dev/null in unix, NUL windows), did you set the right one for your OS?
Related
I have code where I'm writing to a file, and the next time I run the code after the code successfully runs, it gives me the following error:
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:'minicube_HE022222.fits'
So every single time I have to change the name of the fits file and then there are no errors. It's just really frustrating having to change the filename everytime I run the code. Here is my code snippet:
new_hdu = fits.HDUList([fits.PrimaryHDU(mini_data), fits.ImageHDU(mini_error)])
new_hdu[0].header = qso_header
new_hdu.writeto('minicube_HE022222.fits',overwrite=True)
new_hdu.close()
I get the error at:
new_hdu.writeto('minicube_HE022222.fits',overwrite=True)
I close the file after writing to it but that doesn't help either.
Any suggestions are appreciated.
Update:
Here is another portion of a code where this error occurs:
hdus=[]
hdus.append(fits.PrimaryHDU())
hdus.append(fits.ImageHDU(par[0,:,:],name='amp_Hb'))
hdus.append(fits.ImageHDU(par[1,:,:],name='amp_OIII5007'))
hdus.append(fits.ImageHDU(par[2,:,:],name='amp_OIII5007_br'))
hdus.append(fits.ImageHDU(par[3,:,:],name='amp_Hb_br'))
hdus.append(fits.ImageHDU(par[4,:,:],name='amp_Hb1'))
hdus.append(fits.ImageHDU(par[5,:,:],name='amp_Hb2'))
hdus.append(fits.ImageHDU(par[6,:,:],name='amp_Fe5018_1'))
hdus.append(fits.ImageHDU(par[7,:,:],name='amp_Fe5018_2'))
hdus.append(fits.ImageHDU(par[8,:,:],name='m'))
hdus.append(fits.ImageHDU(par[9,:,:],name='c'))
hdu = fits.HDUList(hdus)
hdu.writeto('subcube_par_HE12_lsq.fits',overwrite=True)
It's only at the 'xxx.writeto' where the error occurs. If there's another way I can write to a file or update the existing file with the new data, please let me know. Thanks
As this comment notes, the way file I/O works on Windows is such that you can't overwrite a file if you have that file already open in another process. Are you writing this file and opening it in another program? If you have that file open in any other program then you can't overwrite it.
Do you need to be able to make in-place updates the file while it is open in another program? If so that may still be possible, but you can't use HDUList.writeto() as that effectively deletes the existing file and replaces it with a new one (rather than updating the existing file in-place).
Also, how are you running this code? Is it in a script? You mentioned having to change the filename every time but you could design things such that you wouldn't have to. I noticed that you have the filename hard-coded in your code, and that can and should be fixed if you want to write a more versatile script. For example, you could accept the filename as command-line argument. You could also have the script append a number to the filename or something if the file already exists.
All that said, you should figure out why you have the same file open in multiple programs.
One small usage aside:
The new_hdu.close() in your example doesn't actually do anything. The HDUList.close() method only makes sense when you open an existing FITS file from disk. Here you're creating an HDUList (the data structure representing a FITS file) in memory, and calling the high-level HDUList.writeto() which opens a file, writes the in-memory data to that file, and the closes the file. So the .close() in this case is a no-op. I'm guessing maybe you added it to try to fix your problem, but it's actually not relevant.
Seems like it must be easy, but I just can't figure it out. How do you delete the very last character of a file using Ruby IO?
I took a look at the answer for deleting the last line of a file with Ruby but didn't fully understand it, and there must be a simpler way.
Any help?
There is File.truncate:
truncate(file_name, integer) → 0
Truncates the file file_name to be at most integer bytes long. Not available on all platforms.
So you can say things like:
File.truncate(file_name, File.size(file_name) - 1)
That should truncate the file with a single system call to adjust the file's size in the file system without copying anything.
Note that not available on all platforms caveat though. File.truncate should be available on anything unixy (such as Linux or OSX), I can't say anything useful about Windows support.
I assume you are referring to a text file. The usual way of changing such is to read it, make the changes, then write a new file:
text = File.read(in_fname)
File.write(out_fname, text[0..-2])
Insert the name of the file you are reading from for in_fname and the name of the file you are writing to for 'out_fname'. They can be the same file, but if that's the intent it's safer to write to a temporary file, copy the temporary file to the original file then delete the temporary file. That way, if something goes wrong before the operations are completed, you will probably still have either the original or temporary file. text[0..-2] is a string comprised of all characters read except for the last one. You could alternatively do this:
File.write(out_fname, File.read(in_fname, File.stat(in_fname).size-1))
Is the ReplaceFile Windows API a convenience function only, or does it achieve anything beyond what could be coded using multiple calls to MoveFileEx?
I'm currently in the situation where I need to
write a temporary file and then
rename this temporary file to the original filename, possibly replacing the original file.
I thought about using MoveFileEx with MOVEFILE_REPLACE_EXISTING (since I don't need a backup or anything) but there is also the ReplaceFile API and since it is mentioned under Alternatives to TxF.
This got me thinking: Does ReplaceFile actually do anything special, or is it just a convenience wrapper for MoveFile(Ex)?
I think the key to this can be found in this line from the documentation (my emphasis):
The replacement file assumes the name of the replaced file and its identity.
When you use MoveFileEx, the replacement file has a different identity. Its creation date is not preserved, the creator is not preserved, any ACLs are not preserved and so on. Using ReplaceFile allows you to make it look as though you opened the file, and modified its contents.
The documentation says it like this:
Another advantage is that ReplaceFile not only copies the new file data, but also preserves the following attributes of the original file:
Creation time
Short file name
Object identifier
DACLs
Security resource attributes
Encryption
Compression
Named streams not already in the replacement file
For example, if the replacement file is encrypted, but the
replaced file is not encrypted, the resulting file is not
encrypted.
Any app that wants to update a file by writing to a temp and doing the rename/rename/delete dance (handling all the various failure scenarios correctly), would have to change each time a new non-data attribute was added to the system. Rather than forcing all apps to change, they put in an API that is supposed to do this for you.
So you could "just do it yourself", but why? Do you correctly cover all the failure scenarios? Yes, MS may have a bug, but why try to invent the wheel?
NB, I have a number of issues with the programming model (better to do a "CreateUsingTemplate") but it's better than nothing.
I remember there was a WinAPI function which copied the "date modified" property of the previous file which was replaced with it or something like that? Perhaps anyone can tell me about it?
The problem occured when you used that function very frequently.
This is ReplaceFile (Windows 2000 and up):
The ReplaceFile function combines
several steps within a single
function. An application can call
ReplaceFile instead of calling
separate functions to save the data to
a new file, rename the original file
using a temporary name, rename the new
file to have the same name as the
original file, and delete the original
file. Another advantage is that
ReplaceFile not only copies the new
file data, but also preserves the
following attributes of the original
file:
Creation time
Short file name
Object identifier
DACLs
Encryption
Compression
Named streams not already
in the replacement file
Not too clear exactly what you want, but it seems your after SetFileTime to edit and GetFileTime to copy, combining the two you can do exactly as 'described/wanted'
Good morning/evening
how can I write something in a file and then copy its content into the current file?
for example I consult file1.pro then I have rule write something in file2.pro , after this rule finish its job I want append the content of the file2.pro int file1.pro .
when I tried to append into file1.pro directly , the data appear like undefined symbols ,I don't know why
please hellp me
thank you.
Specifics of the solution might depend on the Prolog dialect. Here I am using SWI-Prolog. SWI-Prolog allows you to open a file with open(SrcDest, Mode, Stream), where SrcDest will be your file name, Mode is read/write/append/update, and Stream is the "file descriptor" the system will return. The manual clarifies difference between appending and updating as follows: "Mode append opens the file for writing, positioning the file-pointer at the end. Mode update opens the file for writing, positioning the file-pointer at the beginning of the file without truncating the file."
To copy from one stream to another you should use copy_stream_data(Stream1,Stream2).
Finally, you should close the streams, otherwise the output file will be empty.
Putting everything together gives
copy(File1,File2) :- open(File1,read,Stream1), open(File2,write,Stream2),copy_stream_data(File1,File2),close(File1),close(File2).
If you need to rewrite the second file, just use update/append mode.