Which os.FileMode values should I provide to os.Mkdir as an argument to have a permission corresponding to drwxrwxr-x?
Daniel Farrell's answer covers the literal settings for modes, but if you're working on a Unix-like system, there are two other important points:
For most cases (most files), you should just use 0666.
For most of the remaining cases (most directories), you should just use 0777.
Certain Go code checkers complain about this, but they're just wrong. 😀
The reason to use 0666 (rw-rw-rw-) and 0777 (rwxrwxrwx) every time is that on Unix-like systems, newly created files are created under a umask setting. The protections that the binary program asks for, such as 0777, are always reduced.1 The reduction is based on this umask setting.
The most permissive umask setting is 0. The least is the rather unusable 0777. The common settings are 077, 007, 022, and 002.
Bits that are set in the umask get cleared in the underlying file permissions. So a umask setting of 022 means that whatever the program asks for, the file winds up being not writable by Group or Other. A program that creates a file using 0666 mode winds up with a file whose actual mode is 0644, or rw-r--r--.
Should the user wish to grant group write permissions, they can run umask 2. Now only the Other w bit will be cleared and this file will be mode rw-rw-r--.
A umask setting of 077 takes away ---rwxrwx: that is, my files and directories are now private to me only (and the super-user of course). One of 027 takes away ----w-rwx, so that newly created files are now 0640, and newly created directories (that use 0777) are now 0750, or rwxr-x--.
One not-entirely-rare exception to the above rules is that temporary files created in areas of "unknown security" (os.CreateTemp files for instance), or files that contain any sensitive data, should be created with mode 0600. They can then be written out without worrying about the data leaking (except to the super-user as usual), and then if they turn out not to be sensitive after all, the final file can be created with the correct permissions.
The really tricky case is when creating a temporary file of unknown security which you are then able to rename (rather than copying) to a file of known-to-be-lower security level, where you'd like to os.Chmod the file to the final mode that you'd get by applying the user's umask. But that's pretty rare.
1In the special case of umask = 0, no bits are removed. So the result is the original value. But that's what the user wanted: to have the reduction be a zero-valued reduction. We're still reducing the permissions, just with a degenerate case.
drwxrwxr-x
each set of rwx (r-x, etc) is represented by one digit in an octal string where 4 = r, 2 = w, and 1=x. So Read and write combine to make 6, read and execute (eg executables in /bin are 5, and read-only is 4.
For files, the most common mode is:
0644 owner gets read and write, group and other get only read
And for directories, typical is:
0755 owner gets read, write (create within) and execute (list contents) , group and other get only read.
In your case, you'd need to add 2 to the Group digit to allow any users in the directory's Group to also create files in that directory.
Related
I have an output/ directory. When the directory to be created is
prefix := "output/2021-11-12.19.51.50.047614/2011-12"
os.MkdirAll(prefix, 0644) fails. When it is
prefix := "output/2021-11-12.19.51.50.047614"
it works.
Why can't os.MkdirAll() create nested directories? That's its purpose. Is this a bug?
The mode argument to os.MkdirAll must include some executable (111) bits.
In general, the mode argument to os.MkdirAll should be 0777, at least on Unix-like systems. In some cases, it should be 0700. (For the same reason, newly created files should be made with 0666, not 0644; sometimes 0600 is appropriate.)
The Go runtime passes the mode down to the OS-level calls. On Unix-like systems, where the permissions actually make sense,1 the bits that you provide are then modified by clearing any bits set in the current umask setting. It's this umask that should take away group and other write permissions, resulting in the eventual 0755 permissions—unless, that is, the user wants to take away more permissions (resulting in, say, 0750 or 0700) or fewer (resulting in, say, 0775).
Remember that the three groups of three bits represent read (r), write (w), and execute (x) permissions: 7 is rwx, 6 is rw-, 5 is r-x, and so on. So 0777 represents rwxrwxrwx.
Unix-like systems require execute permission to name a file within a directory, so if you don't give yourself execute permission, you can no longer work with the directory.
1On systems with ACLs, user permissions make some sense and "other" might be reasonable as the default, but "group" is ill-defined. Systems that do offer ACLs usually have a lot finer granularity than the simple rwx controls anyway, though.
A lot of file permission getting changed. there is huge number of file and chmod -R taking more than 30 minutes. but most of files permission will be as needed.
Changing recursive so that if some file creates with wrong permission that gets fixed.
is there any other more perform way there?
Well, depending on the scenario there are at least three things that you can make:
Add the default ACL (and normal ACL) so some user/groups always have proper access.
Change file group and SETGID bit to group of users that must have access to the files. Also setting proper mask might be helpful.
Use audit subsystem, but with huge number of messages it is hard to maintain.
rsync - same size, content, owner, group, permissions, time - yet it wants to copy
Using macOS Mojave. Using the default filesystem in the source dir (apfs per df?), FAT32 in the dest dir (msdos per df).
Run out of ideas.
Try adding --modify-window=1. From the rsync man page:
--modify-window
When comparing two timestamps, rsync treats the timestamps as
being equal if they differ by no more than the modify-window
value. This is normally 0 (for an exact match), but you may
find it useful to set this to a larger value in some situations.
In particular, when transferring to or from an MS Windows FAT
filesystem (which represents times with a 2-second resolution),
--modify-window=1 is useful (allowing times to differ by up to 1
second).
In my Go application instead of writing to a file directly I would like to write to a temporary that is renamed into the final file when everything is done. This is to avoid leaving partially written content in the file if the application crashes.
Currently I use ioutil.TempFile, but the issue is that it creates the file with the 0600 permission, not 0666. Thus with typical umask values one gets the 0600 permission, not expected 0644 or 0660. This is not a problem is the destination file already exist as I can fix the permission on the temporary to much the existing ones, but if the file does not exist, then I need somehow to deduce the current umask.
I suppose I can just duplicate ioutil.TempFile implementation to pass 0666 into os.OpenFile, but that does not sound nice. So the question is there a better way?
I don't quite grok your problem.
Temporary files must be created with as tight permissions as possible because the whole idea of having them is to provide your application with secure means of temporary storing data which is too big to fit in memory (or to hand the generated file over to another process). (Note that on POSIX systems, where an opened file counts as a live reference to it, it's even customary to immediately remove the file while having it open so that there's no way to modify its data other than writing it from the process which created it.)
So in my opinion you're trying to use a wrong solution to your problem.
So what I do in a case like yours is:
Create a file with the same name as old one but with the ".temp" suffix appended.
Write data there.
Close, rename it over the old one.
If you feel like using a fixed suffix is lame, you can "steal" the implementation of picking a unique non-conflicting file name from ioutil.TempFile(). But IMO this would be overengeneering.
You can use ioutil.TempDir to get the folder where temporary files should be stored an than create the file on your own with the right permissions.
I want to write a program that allows or blocks processes while openning a file depending on a policy.
I could make a control by checking the name of the program. However, it would not be enough because user can change the name to pass the policy. i.e. let's say that policy doesn't allow a.exe to access txt files whereas b.exe is allowed. If user change a.exe with b.exe, i cannot block it.
On the other hand, verifying portable executable signature is not enough for me, because i don't care whether the executable signed or not. I just want to identify the executable that is wanted to execute even its name is changed.
For this type of case, what would you propose? Any solutions are welcome.
Thanks in advance
There are many ways to identify an executable file. Here is a simple list:
Name:
The most simple and straightforward approach is to identify a file by its name. But it is one of the easiest things to change, and you already ruled that out.
Date:
Files have an access, creation, and modification date, and they are managed by the operating system. They are not foolproof, or maybe not even accurate.
Also, they are very simple to change.
Version Information:
Since we are talking about executable files, then most executable files have version information attached to it. For example, original file name, file version, product version, company, description, etc. You can check these fields if you are sure the user cannot modify them by editing your executable. It doesn't require you to keep a database of allowed files. However, it does require you to have something to compare to, like company name, or a product name. Also, If someone made an executable with the same value
they can run instead of the allowed one and bypass your protection.
Location:
If the file is located in a specific place and is protected by file system access rights, and it cannot be changed, then you can use that. You can, for example, put the allowed files in a folder where the user (without admin rights) can only read/execute them, but not rename/move. Then identify the file with its location. If it is run from this location, then allow it, else block it. It is good as it doesn't need a database of
allowed/blocked files, it just compares the location, it it is a valid one, then allow, and you can keep adding and removing files to allowed locations
without affecting your program.
Size:
If the file has a specific file size, you can quickly check its size and compare it. But it is unreliable as files can be changed/patched and without any change in size. You can avoid that by also applying a CRC check to detect if the content of the file changed.
But, both size and CRC can be changed. Also, this requires you to have a list of file names and their sizes/CRC, and keeping it up to date.
Signature:
Deanna mentioned that you can self-sign your executable files. Then check if the signature matches yours and allow/deny based on that. This seems to be a good way if
it is okay for you to sign all the executable files you want to allow. It doesn't require you to keep an updated list of allowed files.
Hash:
arx also pointed out, that you can hash the files. It is one of the slowest methods, as it requires the file to be hashed every time it is executed, then compared to a list of files. But it very reliable as it can uniquely identify each file and hard to break. But, you will need to keep an up to date database of every file hash.
Finally, and depend on your needs and options, you can mix two or more ways together to get the results you want. Like, checking file name + location, etc.
I hope I covered most of things, but I'm sure there are more ways. Anyone can freely edit my post to include anything that I have missed.
I would recommend using the signature, if it has one, or the hash otherwise. Apps such as Office that update frequently are more likely to be signed, whereas smaller apps downloaded off the Internet are unlikely to ever be updated and so should have a consistent hash.