Many places here on StackOverflow, this question has been asked and answered. However, I have found most of those, while technically correct, leave out some specific details that not only explain what's actually going on, but can be valuable information when trying to debug file references or build issues.
I decided to post this here per Jeff Atwood's own comments on encouraging posting your own answers where he says...
It is not merely OK to ask and answer your own question, it is explicitly encouraged [...] I do it all the time!
As such, I'm adding a detailed explanation of what exactly happens under the hood with these items hoping this knowledge can help others as it has helped me, especially in regards to clearing up the all-too-common misconception of thinking as of Xcode 9, the project tree mirrors the on-disk folders, which isn't actually the case.
A Deep-Dive Into Groups and Folder References
As mentioned above, there are lots of answers here on SO around this topic, all which explain the basic differences. However, there are some subtle details that are omitted when it comes to behaviors and understanding of what's actually going on which can lead to a frustrating experience when your project or its changes aren't working as expected. As such, I'm tossing my hat into the ring with a more thorough explanation. Hopefully this will help avoid a lot of headaches.
Now, let's get our hands dirty!
Folder References
Folder references are simpler to grasp. They are simple pointers to actual on-disk folders. Whatever's in the folder on-disk will appear in the project tree.
You identify a folder reference by its icon, which will be blue.
You can include whatever is in that folder in your application by selecting the folder reference in your project tree, then choosing which target to include it in in the inspector.
Note: You can only choose the target at the Folder Reference level, not the individual file level. It's an all-or-nothing addition.
An important side-effect is while you can bundle resources into your application from a folder reference, you CANNOT build source code from a folder reference. Source code must be in groups!
Groups
Groups on the other hand are deceivingly more complex. In their simplest explanation, they are logical-only containers in your Xcode project that you use to store and organize relative pointers to your source code files. These are stored in the Xcode project only, not on disk.
You can identify a group in your project by a yellow folder.
Each item, whether a group itself or a file-reference within the group specifies a location and how it is related to the project's structure. This can be found in the 'Identity and Type' section of the inspector.
Of note...
Groups can reference a specific folder on disk, but don't have to
File references must point to a specific file on disk.
How those references are stored depends on the value for 'Location':
If 'Location' is set to 'Absolute Path' then the reference will contain the entire on-disk path to the item.
If it's set to 'Relative to group', then its first determined what the group currently resolves to, then the references are stored relative to that value. This will propagate recursively up through the parent groups.
There's also a subtle difference in the iconography. If the group points to an actual folder, you will just see the yellow folder. If however, the group doesn't reference a specific folder (i.e. it is logical-only) you will see a small triangle in the folder's icon.
Additionally, regardless of whether the group is connected or not, you can always see what the current physical location referenced by a group or any of its children by looking at the 'Full Path' property in your inspector.
Again, it's important to note that your logical 'groups' structure has nothing to do with the physical on-disk structure. People regularly make the mistake of thinking starting with Xcode 9, that statement isn't true, but read on to know why that misconception still exists.
As mentioned, groups do not represent actual on-disk folders. They are simple logical groupings purely for code organization. However, they can point to an actual on-disk folder. You set which folder that is via the little folder button under the Location dropdown in the 'Identity and Type' section of the inspector. You can also clear that association again by pressing the little x in the circle.
Now here's the tricky part to get. When dealing with groups that aren't pointing to actual folders, their entire purpose is simply to help you as a user organize your code in a structure that makes sense to you, and for Xcode to be able to resolve the actual paths on disk to those file references.
But what if you have a group that does point to a folder? Well now an entire new set of rules comes into play. These rules are what changed in Xcode 9.
After Xcode 9, modifying a group's name may modify the on-disk name, but not necessarily. It will only do so if the group name and physical folder name match prior to the rename. If they don't, they are for all intents and purposes 'disconnected'.
For instance, say you have a group in your project called 'Virtual' that points to a folder on-disk called Literal, like so...
Group On-disk Folder
----- --------------
Virtual -> Literal
If you rename the group 'Virtual' to 'Conceptual', nothing happens on-disk.
Conceptual -> Literal
If you then rename 'Conceptual' to 'Literal' so it matches the actual folder name on disk...
Literal -> Literal
...then again rename 'Literal' to 'Changed', both the group and the folder will be updated to 'Changed'.
Changed -> Changed
Note: This has nothing to do with where the folder is on-disk. This is only pertaining to the name itself, nothing else.
Where are we going?
As for where it is on disk, there too things are more complex than they seem. If you move a group that is not currently pointing to an actual on-disk folder, nothing happens except Xcode updates how it stores your project's items' relative paths in the project file, making them relative to the updated group structure.
However if you move a group that is currently pointing to a folder--even if its name doesn't match the folder on disk (that is a critical point and an oft-source of confusion about 'corrupted' project trees)--the physical on-disk folder it points to will be moved to the new location relative to whatever group you drag it under, along with all items in that folder on disk whether referenced in your project or not!
For instance, say you have this in your project structure...
Project
GroupA -> Points to \Code\Project\GroupA
GroupB -> Points to \Some\Really\Deep\Path\Somewhere\Else\On\The\Disk\Entirely\GroupB
And you drag GroupA so it's under GroupB in your project tree tree, like so...
Project
GroupB
GroupA
Since GroupA points to a physical folder on-disk (again, remember, this has nothing to do with whether the group name and directory name match, only that the group points to an actual directory), the on-disk directory and all of its contents whether referenced or not will actually physically move to
\Some\Really\Deep\Path\Somewhere\Else\On\The\Disk\Entirely\GroupB\GroupA
Again, you are moving it to be under whatever the target group's actual on-disk path is.
Now, if GroupB doesn't actually point to a folder, but GroupA does, then the final location resolves to where GroupB resolves to on disk, which means it considers all of its parent groups to determine the location.
Most Common Scenario
The good news is that for 95% of all use-cases--really, unless you are dealing with a legacy code-base or you actively alter a new project's structure--the physical on-disk folder structure will match the project structure, and all those folder references should be set 'Relative to Group' meaning the groups are essentially mirroring the file system. That is why people think starting with Xcode 9, you are modifying the file system, but again, you are not. You're still just modifying groups and file references. It's just Xcode is making assumptions that if your groups structure mirrors the physical disk and you rename/reorder them, chances are you also want the physical disk to update, so it does it for you. It's very convenient if albeit a little misleading.
With this new knowledge, if things get out of whack and aren't working as you're expecting them to, it may be time to check your file and group references and relative locations. Make sure your folder references are actually pointing to where you are expecting them to be, especially when upgrading older projects, and you should be able to get back on track pretty quickly.
Anyway, hope this better explains things. Sure helped us! :)
Related
We have a P4 repo where we have multiple source and compiled data files alongside each other.
Normally this wouldn't be a problem but I work across multiple teams, so there's certain folders where I need the source files, while in the vast majority of cases I don't (and it's literally terabytes of data).
Is it possible in P4 to setup a filter either on the stream or on the workspace, so that I have something along the line:
include root/
exclude *.max
exclude *.ma
exclude *.mb
include root/Engine/Data/Models/Reference/*.max
include root/Engine/Data/Models/Reference/*.ma
include root/Engine/Data/Models/Reference/*.mb
ie: exclude all .max and .maya files by default, but include them in the folder and subfolders of the Reference folder (bonus points if this can be done from the P4V UI).
I know there's a way to do this one by one file, but that's not a option since there's several hundred files in there and there's new ones added every few days by artists.
Thanks in advance
From a best-practices point of view (in terms of both conceptual simplicity of workspace mappings and performance of the Perforce server), it would be preferable to organize the depot in such a way that the source and compiled files are in separate folders, rather than only being separable by file extension and/or name. Presumably if the compiled files went into a "generated" folder you could simply map that entire folder as the general rule, add in the specific source folders you need the source for, and call it a day, without needing any tricky overlapping exclude/include logic.
Streams flatly do not allow the level of granularity you're describing; you can "ignore" an extension across the board, and you can include/exclude individual folders, but you can't mix and match those rules as you describe; this is a forcing function for simpler depot structures. Streams are meant to encourage/enforce best practices, and were built in part as a way to constrain users from building arbitrarily complex client views that have historically been shown to be difficult to support.
In a "classic" client view (i.e. a client where the View is constructed manually rather than being auto-generated based on a stream) you do still have that arbitrary level of flexibility, and can build mappings like:
//depot/root/... //my-client/...
-//depot/root/....max //my-client/....max
-//depot/root/....ma //my-client/....ma
-//depot/root/....mb //my-client/....mb
//depot/root/Engine/Data/Models/Reference/*.max //my-client/Engine/Data/Models/Reference/*.max
//depot/root/Engine/Data/Models/Reference/*.ma //my-client/Engine/Data/Models/Reference/*.ma
//depot/root/Engine/Data/Models/Reference/*.mb //my-client/Engine/Data/Models/Reference/*.mb
with the caveat that the performance does not scale arbitrarily -- in particular, combining a large number of rules like this with a similar set of rules in the protection table can significantly slow down server operations that need to join the two mappings (which is most of them).
I think the title is pretty explicit. I've read and read (and read) the search files doc, without success.
The problem is when I try to list the files that I can edit because I'm a contributor or manager of the shared drive they're located in: I have no direct permission on the files, I can edit them thanks to the inheritance.
I've tried to run the files.list request with q = 'myadress' in writers or 'myadress' in owners and it returns all the files I can edit except the case described above.
(Yes I set all the options relative to shared drive to true).
Are included in this search:
the files in a "regular" drive that I can edit
the files in a shared drive when I have an explicit "writer" permission on the file
Are excluded:
the files that I can edit because they're in a shared drive I'm manager of (or contributor)
Is there any way to get all those files in one shot?
edit: to make it clear, I'm only interested in the files that I can edit. I could list all the files visible to me, and then check in the response object if I can edit them or not, but that would mean listing way more files than necessary.
Thanks !
You are right, that's not possible
I believe I have similar results to you:
The q parameter
Search query terms
To search for files, use the q parameter in files: files.list.
In the article cited above, there are many parameters for files in your main drive, for example: writers, owners, readers.
Yet if I made any request with a query containing in writers or in owners, if I could edit a file only because I was a member of the parent shared drive then it would not show up. It would only show up if someone had gone into the individual document and given me edit permission.
I could use other q parameters such as fullText and that would return shared drive items no problem.
UPDATE: It seems Google are aware of this issue now, https://issuetracker.google.com/171363628. Go and put a ☆ on it to let them know it affects you.
Possible workarounds
You probably know these well, but I'll list them anyway for the benefit of future users.
Limit the fields
Reference
For example, using a request with the following parameters:
corpora = allDrives
includeItemsFromAllDrives = true
pageSize = 1000
supportsAllDrives = true
fields = files/id, files/name, files/capabilities/canEdit
key = [YOUR_API_KEY]
Which involves greatly limiting the fields returned for better performance and efficiency. In this case, only the id, name and a boolean for whether the user can edit are returned. In this request the pageSize is set to the maximum allowed.
Once you have this response, you could filter it in memory. As you suggested.
Make a few requests
I am not sure if you are running into quota issues, in which case, this option may not be practical for you.
If you are not averse to making a few calls to the API, is to identify the shared drives you have edit access to. By necessity you will have edit access on the files contained therein. Once you have identified the whether you can edit a certain shared drive. You can make a request of each shared drive and your main drive. Which depending on how many shared drives you have and how often they change, could be practical enough for you.
If the shared drives don't change very often at all, you could just keep a list of the shared drive IDs, which you could update periodically with an automatic script. So you are making one less request.
I'm developing a desktop app in `Electron' which allows the "non-pro" user to import (copy) images from their local drive into a project directory which they created earlier. Through the platform dialog (OSX or Windows), the user can select single or multiple images, or single or multiple directories, which could also include sub-directories.
I know how to handle the coding but I am stumped on a strategy to avoid naming conflicts, particularly as images may be coming from camera files which use a simple naming scheme, with batch imports from different camera sessions having the same names.
For a simple example, a user could pick both the "DCIM" directories below, or make selections from within each of the directories of files with the same name.
This is likely a very common programming issue and there must be some solutions which people smarter than me have come up with – but I don't know what this problem is called, in order to search for them.
The solution that I've seen is to look for a naming conflict, and then append something to the name of the thing being imported, before the ending. So you'll see files named foo.txt, foo-001.txt, foo-002.txt, and so on.
If you expect a great many conflicts, the appended text should be random, instead of sequential. That's because it takes 51 duplicate checks before settling on foo-050.txt but only 2.0000214334705... before settling on foo-kyc.txt. The performance difference can be quite noticeable after many conflicts for many files.
NTFS files can have object ids. These ids can be set using FSCTL_SET_OBJECT_ID. However, the msdn article says:
Modifying an object identifier can result in the loss of data from portions of a file, up to and including entire volumes of data.
But it doesn't go into any more detail. How can this result in loss of data? Is it talking about potential object id collisions in the file system, and does NTFS rely on them in some way?
Side node: I did some experimenting with this before I found that paragraph, and set the object id's of some newly created files, here's hoping that my file system's still intact.
I really don't think this can directly result in loss of data.
The only way I can imagine it being possible is if e.g. a backup program assumes that (1) every file has an Object Id, and (2) that the program is keeping track of all IDs at all times. In that case it might assume that an ID that is not in its database must refer to a file that should not exist, and it might delete the file.
Yeah, I know it sounds ridiculous, but that's the only way I can think of in which this might happen. I don't think you can lose data just by changing IDs.
They are used by distributed link tracking service which enables client applications to track link sources that have moved. The link tracking service maintains its link to an object only by using these object identifier (ID).
So coming back to your question,
Is it talking about potential object id collisions in the file system
?
I dont think so. Windows does provides us the option to set the object IDs using FSCTL_SET_OBJECT_ID but that doesnt bring the risk of ID collision.
Attempting to set an object identifier on an object that already has an object identifier will fail.
.. and does NTFS rely on them in some way?
Yes. Object identifiers are used to track files and directories. An index of all object IDs is stored on the volume. Rename, backup, and restore operations preserve object IDs. However, copy operations do not preserve object IDs, because that would violate their uniqueness.
How can this result in loss of data?
You wont get into a serious problem if you change(or rather set) object ID of user-created files(as you did). However, if a user(knowingly/unknowingly) sets object ID used by a shared object file/library, change will not be reflected as is.
Since Windows doesnt want everyone(but developers) to play with crutial library files, it issues a generic warning:
Modifying an object identifier can result in the loss of data from
portions of a file, up to and including entire volumes of data.
Bottom line: Change it if you know what you are doing.
There's another msn article on distributed link tracking and object identifiers.
Hope it helps!
EDIT:
Thanks to #Mehrdad for pointing out.I didnt mean object identifiers of DLLs themselves but ones which they use internally.
OLEACC(a dll), provides the Active Accessibility runtime and manages requests from Active Accessibility clients[source]. It use OBJID_QUERYCLASSNAMEIDX object identifier [ source ]
We have a small team running StarTeam. A constant source of frustration and problems is the handling of deleted files in StarTeam. It is obvious that Starteam keeps track of deleted files internally, but it does not seem to be possible to get any information about a file deletion.
So far, my only solution to find the timing of a delete is to perform a manual binary search using the 'compare' views. Is there any better way (the query for 'delete time' never seems to pick up any files).
The Audit tab (just to the right of File, ChangeRequest, etc.) is probably your best bet if you're just looking for who deleted what and when. The Audit tab also provides information about when items and folders were created, shared, or moved, as well as when View labels are attached/detached. Whenever someone has files unexpectedly appear or disappear, I direct them to the Audit tab first.
There is a server-side configuration setting for the length of time the audit data is retained (30 days by default, I believe). Since it is not retained forever, it isn't a good option for historical data. The number of audits can be quite large in active views.
If you're looking for something more than that or older than your audit retention time, go with Bubbafat's suggestion of the SDK and getDeletedTime/getDeletedUserID.
Comparing views (or rolling back a view to see the item again) is the only way I know how to do this in StarTeam without writing code.
If you are willing to write a little code the StarTeam API provides the Item.getDeletedTime and Item.getDeletedUserId methods (I believe these showed up in 2006).