Every time I opened the directory of a software, the structures of the directory are always the same. My question is what the structure is and what's the purpose of each directory.
Related
Here's the scenario: We have a computer running Windows 10 which has a directory that's backed up nightly. The backups are done with a batch file utilizing Robocopy and scheduled via Windows. The parameters are as such that the backup will always add any new files or existing file edits into the destination, but it will never delete files from the destination that have been deleted in the source. It essentially archives all files which are in the source directory at the end of each day.
Here's the tricky part. The source directory is very large, and occasionally someone finds a duplicate file (or several duplicates of a file) in it. When that happens, we need to delete all but one copy of the file, and then we need to access the backup directory manually, locate the file there, and do the same. This is tedious and time-consuming as it's not rare for someone to notice an entire subdirectory full of files that exist 5+ times each.
What we're looking for is a way to scan the source directory and all subdirectories inside for duplicate files and remove all but one copy of them, and then a way to reflect that into the destination. I've assumed that we will not be able to use Robocopy to reflect the changes in the destination due to the nature of the backup script it's running, but we do have the ability to run any third-party software on the destination directory as well, essentially running an action in both directories to clean each of them of duplicate files.
On that note, I'm not against using third-party tools to make this cleaner or more efficient, I'm just not aware of any.
There is one way to solve this problem I was also suffering from this problem. but I found that how to use "BATCH" file
There are mainly 2 command
X_COPY
ROBO_COPY
According to your need here, (1)x_copy will be helpfull
xcopywill backup your specific file or folder even if you changed some megabytes data, it will copy the new data and will not be replaced on previous data it will make new copy.
HOW TO DO
Open NotePad and type
xcopy "source file" "destination" /y/e/d/c/f/h/i/z/j
And then save your notepad as ".bat" file
for more requirement use below url
https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/xcopy
I am about to design a win32 process that periodically polls the contents of a specific folder (let's call it "incoming") for the arrival of folders.
These new folders will be created occasionally by another process that copies a directory/folder tree from somewhere else into the the "incoming" folder. It may even be created by a user drag-dropping a folder in Windows/File Explorer.
When my process polls, it will notice the presence of a new folder and will delve several levels down in the folder tree and read the contents of some of files it finds there.
It is important that my process waits until the whole folder tree has finished copying before it attempts to read it, otherwise it may miss some files that have not been copied yet.
So is there a way for my process to be sure that the entire folder tree has been fully copied? For example, is there a lock or property that gets set on the top level folder until all lower level folders and files have finished copying?
For the purposes of a security test involving Windows servers, I would like to attempt uploading a Symbolic link to a Windows web application. However, based on the information officially available, it is unclear whether Windows hard links (Which I suppose are the same as NTFS junctions) exist as a file that can be copied from the hard disk the same way it does on Linux. It's vague, but I get the sense that NTFS junctions are some other kind of file system artifacts which is different than "regular" files - I can't find the documentation to confirm or deny this. I.E NTFS I want to know if NTFS supports the direct manipulation of the symlink record such that I could move the symlink to a different computer.
I am aware that Windows softlink files (.lnk) are not limited in this way, but they do not suit the purposes of the test.
My Aim is to copy a symlink off of a virtual machine, and then upload it to the server which I am testing.
Is this possible? (I am under the impression it is not.) From what I have seen absolutely every program on Windows would regard the hardlink as the destination file. Is there a way around this, perhaps by using a special editor to temporarily corrupt the file? If the symlink exists as a normal file on the file system can the symlink be altered so it can moved to a non-Windows OS for further use?
Let me know if this would be a better question for server fault. Since this is not directly about security, and is more of mundane technical problem in the service of a security exercise, I don't think it would fit on Stack Exchange security.
It's hard to provide a very direct answer. I work on a backup/repair/imaging project, and I copy whole disk images to a server via a web service - so, it's possible to do what you want, but there's a lot to consider.
Hardlinks
It is generally assumed that hardlinks cannot be distinguished from each other, however, there is a subtle difference between linked files and their "original" file. That difference is that queries to the $MFT (using USN-related arguments on the winapi function DeviceIOControl) will only return one of the files. This may be considered the original file. You can then call the winapi function NtQueryInformationFile to enumerate the hard links.
Symlinks and junctions are different animals...
You can know that a folder is a junction or a symlink, by getting the attributes from it. There's a ReparsePoint flag in the attributes if it's a junction or a symlink. BTW - the difference between junctions and symlinks is that the junction is a redirect to another location on the same volume, while a symlink is a redirect to an off-volume location. The redirect target is always another folder either way.
What's interesting is that both symlinks and junctions look and act like folders, while they are really files containing redirect information. When you open 'em, NTFS will normally look at the redirect, and open the redirect target. NTFS checks the permissions at the redirect target, so as an attack, this might not be a robust strategy.
When opening a junction/symlink, you can add a flag FILE_FLAG_OPEN_REPARSE_POINT. When you do this, NTFS does not perform the redirect, but opens the content, which is actually redirect information, and assuming you know the format of that information, it is possible to reconstruct the junction/symlink at the server. Note that the redirect may point to a location that may not exist, or may exist only temporarily. This is expected as some network resources may not always be available.
So, in short, it's possible to copy a junction or a symlink...while copying a hard link nominally means copying the file...with the foregoing subtleties in mind. You can create a hard link manually, too, as long as the target file exists.
With hardlinks, there's one interesting kink in the NTFS security picture. If a user has access to a file, and you create a hardlink to that file in a folder the user doesn't have access to, the user can still open that file using the path to the hard link. This is because the link and the original file are both pointing to the same file (and security info) on disk. Permissions changed on any of the links affect all the links. Without knowing this, you can inadvertently wreak havoc on a file system :-)
I know this is a bit helter-skelter, so let me summarize this way:
NTFS directory entries can be folders or files. Hardlinks are directory entries that all point to one file. Symlinks and junctions are really files that act like folders for most practical purposes (until you know how to get at the redirect info as described above).
AFAIR, NTFS (directory) junctions are actually symbolic links. The juctnion is implemented as a special file attribute called repars point that contains the link target.
Hardlinks, on the other hand, are implemented as direct references to the base MFT record of the target file and are stored as regular entries inside directory tree. You actually cannot distinguish a hardlink from the "original" file (every file and directory actually has at least one hardlink since it is contained somewhere within the directory tree).
If you wish to copy a symbolic link itself, you need to know that it is a symbolic link and extract the information about its target. File operations (except deletion and, probably, renamng) are redirected to the link target. So, you can, in general, copy a symbolic link by creating its exact copy in the destination area.
The actual question is, whether the interface you are using to perform the copy operation allows you to create symbolic links on the target.
I am looking for a bit of advice on how Windows file system differentiates between files that are copied(copy and pasted from another location) and files that are created (a new file created in a a folder).
A bit of background to this so it makes more sense: I have an application that is used to move files. The application will monitor a directory and when a file is placed in the directory it will move it elsewhere. However, I am having issues where the application will not pick up a file that is created within the monitored directory but will pick up files that have been created else where and are copied into the monitored directory.
Any advice on how Windows differentiates, or if it does at all, would be greatly appreciated.
This is running on Microsoft Windows Server 2008 R2 Standard. I can't dig into the code and see what is going on under the hood unfortunately, so need to get an idea of the difference if any there would be.
The filesystems don't know the operation of "copying" the file. Any copying is a sequence of file open/read/write/close operations. The same applies to moving to the different filesystem. Moving within the same filesystem, though, is an operation native to the filesystems and it can be done with one command to the filesystem.
Now about your problem. Most likely you catch the creation of the file (before the data is written), and when your application reacts, the file is still opened for writing. So you need to wait until the file is closed.
Depending on how you do monitoring, such waiting is done in different ways. In filesystem filters you wait for file close operation. With .NET FileSystemWatcher there's no way to track file close operation, but I saw a couple of tricks here on StackOverflow (don't have a link though, sorry).
A file existing in D: drive, from creation
The same file which was copied to E: drive
As you can see, the file which was copied to E: drive, has a creation time as the latest, when it was copied to and the modification time as the last modification time for that file in previous location.
So I guess this illustrates, how windows differentiates between copied files and created files.
I've been recently researching how I can manage source files in a project or multiple projects. I've read that Xcode has a built-in support for using svn, and will support git as well, both of which I found to be very useful.
The one thing I couldn't understand clearly is about Source Trees described in Xcode Project Management Guide. Here is my theory, but as I couldn't really verify this from anywhere (as far as I could tell), I would really like if someone could say what I'm missing, if any.
A Source Tree in Xcode preferences is more like a root of a source tree, which is a folder in my local file system.
I can use any files in any of my Xcode projects, even if the files are not in the project folders, if I can specify the files' location related to one of my source trees.
Now someone has the same project folder that is synchronized with mine. She has all files in the project folder, but the files referenced by a relative location to the source tree may exist out of the project folder.
But she has a source tree, with the same Setting Name to mine, (but absolutely in a different folder in her local file system), and if she has the file in the same relative location, then her Xcode can access the file without a problem.
So is this correct, and we use source trees because it enables us collaborating with files outside the project folder?
And even if the files outside the project folder is referenced by a relative path to the project folder itself (not to a source tree), if these files are all managed by SVN so they exist in the same relative location to the project folder in everyone's environments, then I wouldn't need source trees, right?
I never think I am an expert of Xcode, but it seems your question hasn't been answered for a while, so maybe it's worth commenting what I could say:
What you described is pretty much about it. Think is as an environmental variable of an operating system. Typically in a build system made by Autotools, for example, files are referenced by relative paths, such as $PROJECT_HOME/src/common/error.cpp. It doesn't matter where $PROJECT_HOME is in each user's local file system, as long as files are accessible by their relative paths to the user's $PROJECT_HOME directory.
And yes, you don't need to use source tree if the entire folder hierarchy used for a project is referenced by relative paths to the project home and somehow it is certain that everyone has the same files in the same location (for example, because a version control repository contains every files in a chunk as you said).
However, I think it's the best to keep all files in the project home folder, unless they are used across multiple projects, and therefore your version control repository only contains a single root directory (the project home) for your project. If there are files that are best to be shared by multiple projects, then I would have a separate repository for those files. In this case all of your coworkers must use the same protocol, say, having a source tree with the same setting name and put all project homes retrieved from your version control server directly under the source tree (so files outside a project home can be referenced in relation to the source tree for all programmers).
The most of my answer is kind of rephrasing what you already described, but that's how I use the source trees feature in Xcode myself. Maybe others can tell you more about it.