ioutil.WriteFile takes a perm argument - if the file to write doesn't already exist, it is created with permissions perm:
func WriteFile(filename string, data []byte, perm os.FileMode) error
In the general case, is there a recommended value to pass for the perm argument?
More specifically, I am writing a file which is a transformation of an existing file. Is it recommended to read the permissions of the input file (using os.Stat) and use the same permissions for the output file?
In the general case, is there a recommended value to pass for the perm argument?
0666. This is the value used by Go's os.Create, and is also the value of MODE_RW_UGO, used when a file is created by tools such as touch.
More specifically, I am writing a file which is a transformation of an existing file. Is it recommended to read the permissions of the input file (using os.Stat) and use the same permissions for the output file?
Other than pure cp, tools don't seem to do this. As above, tools that create new files generally just use 0666.
There is no standard permission that's "generally recommended". This is more of an OS question than a Go question.
Do you want your file to be executable?
Do you want it to be writable?
Do you want other accounts to be able to access it and with which permissions
If you're taking an existing file and transforming it to the output file, and it's logically OK for your application to have both files with the same permissions, then copying permissions sounds like a reasonable idea (unless you foresee security issues).
Related
I have code where I'm writing to a file, and the next time I run the code after the code successfully runs, it gives me the following error:
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:'minicube_HE022222.fits'
So every single time I have to change the name of the fits file and then there are no errors. It's just really frustrating having to change the filename everytime I run the code. Here is my code snippet:
new_hdu = fits.HDUList([fits.PrimaryHDU(mini_data), fits.ImageHDU(mini_error)])
new_hdu[0].header = qso_header
new_hdu.writeto('minicube_HE022222.fits',overwrite=True)
new_hdu.close()
I get the error at:
new_hdu.writeto('minicube_HE022222.fits',overwrite=True)
I close the file after writing to it but that doesn't help either.
Any suggestions are appreciated.
Update:
Here is another portion of a code where this error occurs:
hdus=[]
hdus.append(fits.PrimaryHDU())
hdus.append(fits.ImageHDU(par[0,:,:],name='amp_Hb'))
hdus.append(fits.ImageHDU(par[1,:,:],name='amp_OIII5007'))
hdus.append(fits.ImageHDU(par[2,:,:],name='amp_OIII5007_br'))
hdus.append(fits.ImageHDU(par[3,:,:],name='amp_Hb_br'))
hdus.append(fits.ImageHDU(par[4,:,:],name='amp_Hb1'))
hdus.append(fits.ImageHDU(par[5,:,:],name='amp_Hb2'))
hdus.append(fits.ImageHDU(par[6,:,:],name='amp_Fe5018_1'))
hdus.append(fits.ImageHDU(par[7,:,:],name='amp_Fe5018_2'))
hdus.append(fits.ImageHDU(par[8,:,:],name='m'))
hdus.append(fits.ImageHDU(par[9,:,:],name='c'))
hdu = fits.HDUList(hdus)
hdu.writeto('subcube_par_HE12_lsq.fits',overwrite=True)
It's only at the 'xxx.writeto' where the error occurs. If there's another way I can write to a file or update the existing file with the new data, please let me know. Thanks
As this comment notes, the way file I/O works on Windows is such that you can't overwrite a file if you have that file already open in another process. Are you writing this file and opening it in another program? If you have that file open in any other program then you can't overwrite it.
Do you need to be able to make in-place updates the file while it is open in another program? If so that may still be possible, but you can't use HDUList.writeto() as that effectively deletes the existing file and replaces it with a new one (rather than updating the existing file in-place).
Also, how are you running this code? Is it in a script? You mentioned having to change the filename every time but you could design things such that you wouldn't have to. I noticed that you have the filename hard-coded in your code, and that can and should be fixed if you want to write a more versatile script. For example, you could accept the filename as command-line argument. You could also have the script append a number to the filename or something if the file already exists.
All that said, you should figure out why you have the same file open in multiple programs.
One small usage aside:
The new_hdu.close() in your example doesn't actually do anything. The HDUList.close() method only makes sense when you open an existing FITS file from disk. Here you're creating an HDUList (the data structure representing a FITS file) in memory, and calling the high-level HDUList.writeto() which opens a file, writes the in-memory data to that file, and the closes the file. So the .close() in this case is a no-op. I'm guessing maybe you added it to try to fix your problem, but it's actually not relevant.
I have a general software security question.
I have an application. This application can receive a file as an input and write it to the disk.
This file is a text file and i want to make sure the user is not trying to write a script that will be saved on the disk.
FYI - operating system is Linux.
What i have so far is:
1) I can write it with limited permissions. (i do not think this is secure enough)
2) I want to verify this is not a script. << How can i do this?
Set a unprivileged user and write that file disabling execution and reading for other user i think it's the best option.
The second option is impracticable , but you can try to make things difficult to an attacker:
Remove some special delimiters, e.g. PHP's
Remove (if you can)some special characters used normally, e.g. {},<>, etc
Remove any non printablechar (for partially-avoiding any binaryfile)
Are there any rules for file extensions? For example, I wrote some code which reads and writes a byte pattern that is only understood by that specific programm. I'm assuming my anti virus programm won't be too happy if I give it the name "pleasetrustme.exe"... Is it gerally allowed to use those extensions? And what about the lesser known ones, like ".arw"?
You can use any file extension you want (or none at all). Using standard extensions that reflect the actual type of the file just makes things more convenient. On Windows, file extensions control stuff like how the files are displayed in Windows Explorer and what happens when you double click on it.
I wrote some code which reads and writes a byte pattern that is only
understood by that specific programm.
A file extension is only an indication of what type of data will be inside, never a guarantee that certain data formatted in a specific way will be inside the file.
For your own specific data structure it is of course always best to choose an extension that is not already in use for other file formats (or use a general extension like .dat or .bin maybe). This also has the advantage of being able to use an own icon without it being overwritten by other software using the same extension - or the other way around.
But maybe even more important when creating a custom (binary?) file format, is to provide a magic number as the first bytes of that file, maybe followed by a file header structure containing a version number etc. That way your own software can first check the header data to make sure it's the right type and version (for example: anyone could rename any file type to your extension, so your program needs to have a way to do some checks inside the file before reading the remaining data).
Seems like it must be easy, but I just can't figure it out. How do you delete the very last character of a file using Ruby IO?
I took a look at the answer for deleting the last line of a file with Ruby but didn't fully understand it, and there must be a simpler way.
Any help?
There is File.truncate:
truncate(file_name, integer) → 0
Truncates the file file_name to be at most integer bytes long. Not available on all platforms.
So you can say things like:
File.truncate(file_name, File.size(file_name) - 1)
That should truncate the file with a single system call to adjust the file's size in the file system without copying anything.
Note that not available on all platforms caveat though. File.truncate should be available on anything unixy (such as Linux or OSX), I can't say anything useful about Windows support.
I assume you are referring to a text file. The usual way of changing such is to read it, make the changes, then write a new file:
text = File.read(in_fname)
File.write(out_fname, text[0..-2])
Insert the name of the file you are reading from for in_fname and the name of the file you are writing to for 'out_fname'. They can be the same file, but if that's the intent it's safer to write to a temporary file, copy the temporary file to the original file then delete the temporary file. That way, if something goes wrong before the operations are completed, you will probably still have either the original or temporary file. text[0..-2] is a string comprised of all characters read except for the last one. You could alternatively do this:
File.write(out_fname, File.read(in_fname, File.stat(in_fname).size-1))
Is the ReplaceFile Windows API a convenience function only, or does it achieve anything beyond what could be coded using multiple calls to MoveFileEx?
I'm currently in the situation where I need to
write a temporary file and then
rename this temporary file to the original filename, possibly replacing the original file.
I thought about using MoveFileEx with MOVEFILE_REPLACE_EXISTING (since I don't need a backup or anything) but there is also the ReplaceFile API and since it is mentioned under Alternatives to TxF.
This got me thinking: Does ReplaceFile actually do anything special, or is it just a convenience wrapper for MoveFile(Ex)?
I think the key to this can be found in this line from the documentation (my emphasis):
The replacement file assumes the name of the replaced file and its identity.
When you use MoveFileEx, the replacement file has a different identity. Its creation date is not preserved, the creator is not preserved, any ACLs are not preserved and so on. Using ReplaceFile allows you to make it look as though you opened the file, and modified its contents.
The documentation says it like this:
Another advantage is that ReplaceFile not only copies the new file data, but also preserves the following attributes of the original file:
Creation time
Short file name
Object identifier
DACLs
Security resource attributes
Encryption
Compression
Named streams not already in the replacement file
For example, if the replacement file is encrypted, but the
replaced file is not encrypted, the resulting file is not
encrypted.
Any app that wants to update a file by writing to a temp and doing the rename/rename/delete dance (handling all the various failure scenarios correctly), would have to change each time a new non-data attribute was added to the system. Rather than forcing all apps to change, they put in an API that is supposed to do this for you.
So you could "just do it yourself", but why? Do you correctly cover all the failure scenarios? Yes, MS may have a bug, but why try to invent the wheel?
NB, I have a number of issues with the programming model (better to do a "CreateUsingTemplate") but it's better than nothing.