Saying this is how the world looks like before pivot_root,
/
/bin
/sbin
/etc
/lib
/proc
/sysfs
.....
When I call "pivot_root /newroot /newroot/oldroot", what happened to all these folders above?
We get a nude "/" and whole old world resides in "/oldroot"??
/
/oldroot
/oldroot/bin
/oldroot/sbin
/oldroot/lib
/oldroot/etc
But then who created /bin, /sbin, and /proc, /sysfs in the new world?
Before doing the pivot_root, one should care to do the "mount --move" for all the necessary /proc, /dev, etc... whatever required to be available.
Related
I have a directory containing a big number of sub-directories within.
I need to loop over all subdiretories and save it names (without a path!) as a distinct variable
for d in ${output}/*/
do
dir_name=${d%*/}
echo ${dir_name}
done
the problem of the current version that it gives me a full path of the directory instead. Here is the result of echo
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig992
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig993
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig994
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig995
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig996
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig997
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig998
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig999
With the dir_name=${d%*/}, you remove the trailing / only. You will want to remove everything upto the last / as well. Or try basename, which is perhapse a better option.
As in:
for d in /var/*/ ; do
dir_name=${d%/}
base=$(basename "$d")
echo "$d $dir_name ${dir_name##*/} $base"
done
which produces:
/var/adm/ /var/adm adm adm
/var/cache/ /var/cache cache cache
/var/db/ /var/db db db
/var/empty/ /var/empty empty empty
/var/games/ /var/games games games
/var/heimdal/ /var/heimdal heimdal heimdal
/var/kerberos/ /var/kerberos kerberos kerberos
/var/lib/ /var/lib lib lib
/var/lock/ /var/lock lock lock
/var/log/ /var/log log log
/var/mail/ /var/mail mail mail
/var/man/ /var/man man man
/var/named/ /var/named named named
/var/netatalk/ /var/netatalk netatalk netatalk
/var/run/ /var/run run run
/var/slapt-get/ /var/slapt-get slapt-get slapt-get
/var/spool/ /var/spool spool spool
/var/state/ /var/state state state
/var/tmp/ /var/tmp tmp tmp
/var/www/ /var/www www www
/var/yp/ /var/yp yp yp
(on my system).
Can you cd to that parent directory?
cd ${output}/
lst=( */ )
for d in "${lst[#]}"; do echo "${d*/}"; done
If that's not an option, then you can strip it each time.
lst=( ${output}/*/ )
for d in "${lst[#]}"; do dir="${d*/}"; echo "${dir##/}"; done
As a hybrid, you can sometimes use a trick of changing directory inside a subshell, as the cd is local to the subshell and "goes away" when it ends, but so do any assignments.
cd /tmp
( cd ${output}/; lst=( */ ); for d in "${lst[#]}"; do echo "${d*/}"; done )
# in /tmp here, lst array does not exist any more...
There are couple of folders in the root dir of HDFS:
dir1
subdir1
table1
table2
subdir2
dir2
subdir1
table1
table2
dir3
They all have subfolders that contain different Parquet files that are queried with Hive.
I can't load one of the subfolders (for example table1 inside dir2) even though the permissions look ok to me, I get the EXECUTE error when trying to load it.
The code is running in a Jupyter notebook.
Users are organized in groups.
I've added rwx permissions for the directory in question to the group by using the following command:
hdfs dfs -setfacl -R -m group:user_group:rwx /dir2/subdir2
The error I'm getting looks like this:
Cannot create staging directory 'hdfs://server:8020/dir2/subdir1/table1/.hive-staging_hive_2019-08-01_13-04-22': Permission denied: user=username, access=EXECUTE, inode="/dir2":hdfs:supergroup:drwxrwx---
I've added read and execute permissions on dir2 to the user group but the error persists. It looks to me from this error that somehow the default permissions are applied and they are ---
So, to summarize;
group has read and execute privileges on the root dir, and read, write and execute privileges on the table directories, but it keeps failing with permissions for root directory.
This is how the permissions look:
# file: /dir2
# owner: hdfs
# group: supergroup
user::rwx
user:some_group1:r-x
group::---
group:some_group2:rwx
group:user_group:r-x
group:hive:rwx
group:some_group3:r-x
group:some_group4:r-x
mask::rwx
other::---
default:user::rwx
default:user:some_group1:r-x
default:group::---
default:group:some_group2:rwx
default:group:hive:rwx
default:group:some_group3:r-x
default:group:some_group4:r-x
default:mask::rwx
default:other::---
# file: /dir2/subdir1/table1
# owner: some_user
# group: supergroup
user::rwx
user:some_group1:r-x
group::---
group:some_group2:rwx
group:user_group:rwx
group:hive:rwx
group:some_group3:r-x
group:some_group4:rwx
mask::rwx
other::---
default:user::rwx
default:user:some_group1:r-x
default:group::---
default:group:some_group2:rwx
default:group:user_group:rwx
default:group:hive:rwx
default:group:some_group3:r-x
default:group:some_group4:rwx
default:mask::rwx
default:other::---
The problem was eventually solved by creating new directories that replaced the old ones. The new directories were created with the correct user and credentials.
For example, I created subdir1_new, moved the data there, renamed subdir1 to subdir1_old and renamed subdir1_new to subdir1. Not a lot of folders were affected by this issue so it didn't take a long time.
I know it's not the actual solution, but I couldn't figure out what exactly was happening and this workaround did the trick.
I am really new to Bash Scripting so please bear with me if this question sounds stupid. I am also not too sure what to search on the internet.What should I do if I need to write a shell script to list any directory where one user's home directory can be modified by some other user? I am not able to understand what this 'modified by some other user means'.Please help. Thanks !
The very short answer to your question is: no script needed, simply:
ls -al /home
That will list for you all users and the respective permissions for each users home directory. Linux file permission are controlled by 10 bits that represent who has access and what, if any, special permissions are associated with a given file. The permissions bits are usually represented for discussion as drwxrwxrwx. The first, or special, bit meaning is as follows:
_: (unset) indicates a regular file with no special properties
d: directory,
l: link,
s: the directory is setuid/setgid
t: sticky bit
The next nine bits rwxrwxrwx (3 sets of rwx) control the access the owner group world has to the file in question. So who is the owner group or world? Let's look at an example from ls -al /home:
drwxr-xr-x 15 deborah users 4096 Mar 11 2011 deborah
Looking at the information we can separate the 10 bits and information as follow:
d rwx r-x r-x .. deborah users ..... deborah
| | | \ \ \
owner | world owner group filename
group
Above the special permission bit is a d which indicates that the filename (at the far right deborah) is a directory. The first set of 3 bit specifies that the owner (deborah) has read, write and execute permission on the file. Similarly, the next set of 3 specify that the group (users) has read and and execute permission but no write permission. NOTE: with a directory, the execute bit also control whether the (owner, group or world) can descend into the directory. In like manner, the world (everybody) has the same permission as group (users).
To manipulate the bits, you use the chmod (change mode) command. To manipulate the user or group, you use the chown (change owner) command. The chown command has simple basic usage, just specify the new owner and group separated by a colon :. For example to change the file shown above to be owned by user david and group samba the command would be chown david:samba filename
There are two ways to change the permissions or (mode) with chmod. You either specify the octal equivalent for special bit and the 3 sets of owner, group and world bits at once numerically. Example: to make the directory rwx for the user and group you would issue the command:
chmod 0775 filename # to set all permissions as desired at once
The 0 simply stating no special bit settings for the directory, the first 7 indicating the binary 111 (or rwx) for the user, the second 7 indicating the same for the group and the final 5 indicating the world should have (binary 101) r_x permissions. While not always required, it is recommended to provide the leading 0 even when there will be no change to the special permission bit to remove any ambiguity.
You can also use chmod with +/-/= r, w, x (for corresponding rwx bits) for u, g, or o user, group, or owner permissions (you can shorcut using a for all). To put it all together and set the mode the same as shown above using octal bit, you would simply do:
chmod g+w filename # to add the single write bit to group 'users'
Using this method, you may be required to make multiple calls to chmod to set all permission as required, but contrast using the octal permissions, you can set all permission fields in a single call.
Obviously there is much more to it than this, but for a good introduction, this should be enough to get you started managing permissions and ownership. (obviously this post also turned out way longer than initially anticipated, enjoy).
The:
where one user's home directory can be modified by some other user?
can be:
if the user1 is in the same group as user2 AND the home directory is group-writable, or
if the user has world-writable directory
You really need understand how unix-like permissions works. (or in wider context - how ACLs works in general)
For the (partial) solution (many ways - one of them is the next):
you can get the path of home directories from the /etc/passwd file.
can read them in a cycle, (filter the /etc/passwd with the cut command), and
test, if they're writable for you (for this, read the man page about the shell builtins if and the command test alias [.
I'm using code like following to monitor the whole file system:
fanotify_mark(fd,
FAN_MARK_ADD | FAN_MARK_MOUNT,
FAN_OPEN | FAN_EVENT_ON_CHILD,
AT_FDCWD, "/"
)
But I need write some tests, so, I want monitor just a specific dir, let say "/tmp/test_dir". The problem is when I change code this way:
fanotify_mark(fd,
FAN_MARK_ADD,
FAN_OPEN | FAN_EVENT_ON_CHILD,
AT_FDCWD, "/tmp/test_dir"
)
fanotify only watchs to events on "/tmp/test_dir" ignoring whatever happen in deeper folders.
For instance: If I open "/tmp/test_dir/aa/bb/cc/test_file.txt" fanotify detects nothing.
I'm missing some flag?
Problem solved.
fanotify isn't recursive. It only works that way when working on mounted directories. I did the following test:
mkdir /tmp/parent
mkdir -p /tmp/other/aa/bb/cc/dd
touch /tmp/other/aa/bb/cc/dd/test.txt
mount --bind /tmp/other /tmp/parent
then in code:
fanotify_mark(fd,
FAN_MARK_ADD | FAN_MARK_MOUNT,
FAN_OPEN | FAN_EVENT_ON_CHILD,
AT_FDCWD, "/tmp/parent"
)
and it's done. Now fanotify fire up events for test.txt file.
With fanotify, either monitor entire mount point of specified path (using FAN_MARK_MOUNT), or monitor files in a directory (not its sub-directory, without specifying FAN_MARK_MOUNT). You can set separate monitors for sub-directories to achieve this. see https://stackoverflow.com/a/20965660/2706918
I am currently running Cygwin on a target Windows Server 2003 machine to fire off a shell script that, among other things, creates a bunch of files on disc. However after the files are created I no longer have permissions to manipulate them through Windows.
When the files are created the owner is getting set to 'SYSTEM' and the permissions for Administrators/Creator Group/Creator Owner/system are set to only 'special permissions' and nothing else.
The permissions for Everyone and Users have Read & Execute, List folder contents and Read.
My problem is that I cannot delete/modify the files now through Windows. I would prefer to have something built into my scripts (either the shell script or something to call in Cygwin) that would allow Administrators full control on the folder and all contents.
My current workaround has been to either do file modifications through Cygwin but this is not preferable. I have also used setfacl -r -m default:other:rwx to add write permissions for the 'Users' group but it doesn't appear to have a recursive option and still doesn't give 'full control'
Is there a better way to use setfacl? Can I call the shell script using different/elevated permissions?
Results of getfacl on a newly created directory:
$ getfacl Directory/
# file: Directory/
# owner: SYSTEM
# group: root
user::rwx
group::r-x
group:Users:rwx
mask:rwx
other:r-x
default:user::rwx
default:group::r-x
default:group:Users:rwx
default:mask:rwx
default:other:r-x
You can try setting umask:
umask u=rwx,g=rwx,o=rwx
That should give user, group, and other read/write/execute on any newly created dirs.
If you only want the modified umask permanently, you can add it to your .bash_profile
Edit - Added example of mkdir before/after umask.
Here's the output of getfacl on a directory created before I set umask:
[/cygdrive/c/Documents and Settings/NOYB/Desktop]
==> getfacl test_wo_umask/
# file: test_wo_umask/
# owner: NOYB
# group: Domain Users
user::rwx
group::r-x
group:root:rwx
group:SYSTEM:rwx
mask:rwx
other:r-x
default:user::rwx
default:user:NOYB:rwx
default:group::r-x
default:group:root:rwx
default:group:SYSTEM:rwx
default:mask:rwx
default:other:r-x
Here's the output of getfacl on a directory created after I set umask:
[/cygdrive/c/Documents and Settings/NOYB/Desktop]
==> getfacl test_w_umask/
# file: test_w_umask/
# owner: NOYB
# group: Domain Users
user::rwx
group::rwx
group:root:rwx
group:SYSTEM:rwx
mask:rwx
other:rwx
default:user::rwx
default:user:NOYB:rwx
default:group::rwx
default:group:root:rwx
default:group:SYSTEM:rwx
default:mask:rwx
default:other:rwx