When using CLS to list files via LFTP, is there a way to show the calculated size of any folders that appear in the listing? Here's what I'm using:
cls -s -h --sort=date
And here's the result:
3.7G bigfile.mp4
4.0K some.folder/
4.0K another.folder/
1.1G anotherfile.psd
All folders are showing as being only 4.0K, which of course does not reflect the total size of the folder's contents. My Google-fu fails me on this one :/
cls shows folder size reported by the server. In this case it is the size of the folder itself without the size of nested files. To see the size of all nested files and directories, use du -h command.
Related
I want to move data from one volume to another. The folders and file sizes vary. Files can be up to 100 GB, but we can have also a lot of small files. If there is data in the destination volume at that particular folder, it can be overwritten.
So far, I tried (Code has been simplified for demonstration purposes)
(1)for root, directories, files in os.walk(src):
for file in files:
mv -v <src> <dest>
(2)hadoop distcp -overwrite -m100 <src> <dest>
Less than 10 GB, the mv option is faster. At 10 GB both options take approx 2 minutes transfer time.
I'm a newbie in windows scripting, so bear with me.
I want to find "test" folder from multiple folders. Such as
Search test folders from multiple paths:
/rbl.com/mailroot/test
/pml.com/mailroot/test
/pax.com/mailroot/
Want to get output like that:
test#rbl.com
test#pml.com
I have run this command to get results dir /S test > c:\results.txt
Output:
Volume in drive C has no label.
Volume Serial Number is 4A9E-06D4
Directory of C:\Program Files (x86)\Parallels\Plesk\Mail Servers\Mail Enable\Postoffices\rbl.com\MAILROOT
08/05/2016 12:36 AM <DIR> test
0 File(s) 0 bytes
Directory of C:\Program Files (x86)\Parallels\Plesk\Mail Servers\Mail Enable\Postoffices\pml.com\MAILROOT
08/05/2016 12:36 AM <DIR> test
0 File(s) 0 bytes
But how to extract the information like:
test#rbl.com
test#pml.com
This should do the trick
ls test -Directory -Recurse | % { "$($_.Name)#$($_.Parent.Parent.Name)" }
It searches your folders recursively for test folders, then prints out the folder's parent, its parent, and the folder's name as you want it formatted.
Suppose I have a text file, which contains data like this.
Below output generated from du - sh /home/*
1.5G user1
2.5G user2
And so on...
Now if I want that those files size be stored in an array and compared to 5 GB if the user is consuming more than 5 Gb. What can I do?
The du command shows the usage of each folder in home directory. So if i want myself to be notified that some user is consuming more than 5 GB. Because there is a long list of users. It will be tedious to identify each user's disk usage. I want a shell script to identify the usage for each directory in home. And then I will put mail function to notify myself for exceeded limits.
Note : Don't want to implement quota as I just want to monitor the usage.
Use du's -t (--threshold) option to specify you only want to know about directories with more than a certain amount of data in them:
$ du -sh -t 5G /home/*
If you're picky about precisely how big a gigabyte is, note that 5G uses multiples of 1024; you may prefer -t 5GB for multiples of 1000, or even -t 5000M to mix them.
For lots of users, you're probably better off writing that using -d 1 instead of -s to avoid the shell having to expand the * into a very long list:
$ du -h -d 1 -t 5G /home/
Windows 7/NTFS and later has both symbolic links and junctions, and they are subtly different. (See this excellent set of posts). For logging (and debugging) purposes, I need to be able to resolve the target of a junction on a remote file server. There have been some posts on this topic, but they apply to resolving a junction as the local machine see it.
On \\FileServer, we have a directory G:\Shared that is shared out as PublicShare, so it apears on the network as \\FileServer\PublicShare. Within that directory are sub-directories G:\Shared\SubDir1 and G:\Shared\SubDir2. Also within that directory is a junction G:\Shared\Junc that points to either G:\Shared\SubDir1 or G:\Shared\SubDir2. (The target can change.) Thus, on \\FileServer, one sees
\\FileServer\PublicShare\SubDir1
\\FileServer\PublicShare\SubDir2
\\FileServer\PublicShare\Junc
On a client machine where \\FileServer\PublicShare is mounted (mapped in Windows lingo) as M:\, one thus sees M:\SubDir1, M:\SubDir2, M:\Junc. If, on that client machine, you open a console (cmd.exe) and do dir M:\, Windows gives a nice listing which shows that M:\Junc is a junction and includes the target, G:\Shared\SubDirX, of the junction.
M:\>dir
Volume in drive M is XXXXXXXXX
Volume Serial Number is XXXX-XXXX
Directory of M:\
09/05/2014 07:30 PM <DIR> .
09/05/2014 07:30 PM <DIR> ..
09/05/2014 01:36 PM <JUNCTION> Junc [G:\Shared\SubDir1]
09/06/2014 12:55 PM <DIR> SubDir1
09/05/2014 05:15 PM <DIR> SubDir2
0 File(s) 0 bytes
3 Dir(s) 1,895,493,492,736 bytes free
M:\>
Thus, the target of such a junction is clearly available to a client. Does anyone know how to obtain this information programatically, e.g. which system api to call??? Thanks.
This does the trick:
fsutil reparsepoint query "M:\Junc"
and it seems very reliable.
If you don't want to parse the output of C:\Windows\System32\fsutil.exe, you could simply use DIR:
for /f "usebackq delims=[] tokens=2" %i in ( `dir M:\^|findstr JUNCTION` ) do set RES=%i
Notes:
M:\: you need to display the subfolders (and junctions) of the parent folder of the JUNCTION (in this case: M:\)
findstr: if you have several junctions and want to resolve a specific one, add a |findstr foldername to isolate the right line,
delims=[]: that will divide the output 09/05/2014 01:36 PM <JUNCTION> Junc [G:\Shared\SubDir1] into 3 tokens:
09/05/2014 01:36 PM <JUNCTION> Junc
G:\Shared\SubDir1
"<empty>"
tokens=2: only the second token ("G:\Shared\SubDir1") is of interest
%i: if you run that command in a .bat script, double the '%': %%i in and res=%%i
When I try to use du command, to see the size of my folders like this for
example:
du -H --max-depth=1 some-folder/
28M
11M
8.0K
4.2M
260K
896K
86M
7.9M
24K
8.6M
22M
14M
6.0M
60K
912K
365M total
,final size does not shows the real sum of the above numbers. Why the summary size is wrong?
The command you're executing shows the folders only.
Therefore any files in some-folder only added to the total but not listed individually.
Try this:
du -Hs some-folder/*
But note that if you have hidden files (ie. files with a dot prefix) won't be listed with this command either.