I have two sepearate git repos: A and B. Some repository B files are already present in a subfolder of project A. My goal is to create patches for repo B and then applying them to the subfolder within repo A to conserve history of repo B while merging them. The issue is that a patch is unable to create new files. For example:
assuming this folder structure: /home/user/B/..bunch of directories and /home/user/A/ext/lib/B/..bunch of directories
cd /home/user/B
git format-patch "xx..xx" -o /home/user/A/ (create patch files)
cd /home/user/A
git apply -v --directory=ext/lib/B/ 0001-foo-12345.patch
works fine since the patch is not creating any new files or trying to access a folder which is present in B but not A
BUT
cd /home/user/A
git apply -v --directory=ext/lib/B/ 0002-foo2-6789.patch
does not work and throws this error:
Checking patch ext/lib/B/xyz/test.c...
error: ext/lib/B/xyz/test.c: No such file or directory.
I have tried the following commands so far:
git apply -v --directory=/home/user/A/lib/B/ --include=bb/cc --exclude=cc/ --exclude=bb/ --include=* 0002-foo2-6789.patch
git apply -v --directory=/home/user/A/lib/B/ --include=* --include=bb/cc --exclude=cc/ --exclude=bb/ 0002-foo2-6789.patch
git am --directory=/home/user/A/lib/B/ --include=* --include=bb/cc --exclude=cc/ --exclude=bb/ 0002-foo2-6789.patch
1.create patch file:
git diff --cached >> test.patch
2.use the patch:
git apply -p1 < test.patch
There are a number of ways to create patch files that will create new files. However, creating patches in repo B and applying them in repo A won't import the history of repo B into repo A. Is that what you mean by "conserve history of repo B while merging them"?
Example patch that causes git apply to create a new path:
diff --git a/b1.txt b/b1.txt
new file mode 100644 <-- lines specific to creating new files
index 0000000..12f00e9
--- /dev/null <-- lines specific to creating new files
+++ b/b1.txt
## -0,0 +1 ##
+contents
One way to create patches like this is to copy the files from repo B to their destination in repo A, git add any changed or new files, then use git diff --staged > my.patch to create a patch for all changed and new files. However, by then, the files are already in repo A, so there's little point in creating a such patch, and this also won't import repo B's history into repo A.
If you really want to merge repo B into a subdirectory of repo A and preserver the history of both, you're better off not using patches and looking at the top several answers here: How do you merge two Git repositories?
Related
I have a yaml file that has a field data.version which I want to detect changes from main branch.\
The yaml looks something like this:
# ...
data:
version: 1.2.3
# ...
There are more fields which are not relevant for this purpose.
I am writing a GitLab-CI script where I have my current commit checked out.
I am able to see the changes in general by using this command:\
git fetch origin main
git diff origin/main HEAD -- my_yaml_file
But this does not allow me to detect changes to this specific field...
Is there a way to get and parse the original file from main branch?
Note that I am trying to avoid checking out the entire repository on a temp directory just for that purpose :)
You can get a specific version of a file with git show
git show origin/main:my_yaml_file
After that you need to parse the yaml file to get the diff
For example using yq
git show origin/main:my_yaml_file|yq eval ".data.version"
Will give out the value of data.version
I forked a repo, then cloned it to my Mac into a /YATC directory. I had a previously-created Xcode project (TwitterTimeline) in another directory, which I copied into the /YATC directory. I did git add . in the /YATC directory, and only an empty TwitterTimeline directory was added to the repo. No other files were added. I later found out that there was already a .git directory in TwitterTimeline. I think Xcode must have created that, although I don't recall ever asking it to.
Anyway, I deleted the TwitterTimeline/.git directory. I went back up to /YATC and tried to do git add . there, and nothing happened. Meaning I immediately did git status, and it said there was nothing to commit. Then I went down to the TwitterTimeline directory and did git add ., and got the following:
Assertion failed: (item->nowildcard_len <= item->len && item->prefix <= item->len), function prefix_pathspec, file pathspec.c, line 308.
Abort trap: 6
What is this?
Not sure exactly what's happening but I was in the same situation
I moved one git repo inside another
deleted the .git dir thinking it would become a regular subdir
tried to git add -A .
got the weird error message
I got around it by renaming the subdir, doing git add from a parent dir, then renaming the subdir back to the original name. Somehow that seemed to get git back in a working state.
Basically the problem is known and it usually affects submodules. It's not Git bug, but basically instead of assert you should see the proper error that specific pathspec is part of the submodule.
The following Git patch from Stefan Beller fixes that problem:
--- a/pathspec.c
+++ b/pathspec.c
## -313,8 +313,23 ## static unsigned prefix_pathspec(struct pathspec_item *item,
}
/* sanity checks, pathspec matchers assume these are sane */
- assert(item->nowildcard_len <= item->len &&
- item->prefix <= item->len);
+ if (item->nowildcard_len > item->len ||
+ item->prefix > item->len) {
+ /* Historically this always was a submodule issue */
+ for (i = 0; i < active_nr; i++) {
+ struct cache_entry *ce = active_cache[i];
+ int ce_len = ce_namelen(ce);
+ int len = ce_len < item->len ? ce_len : item->len;
+ if (!S_ISGITLINK(ce->ce_mode))
+ continue;
+ if (!memcmp(ce->name, item->match, len))
+ die (_("Pathspec '%s' is in submodule '%.*s'"),
+ item->original, ce_len, ce->name);
+ }
+ /* The error is a new unknown bug */
+ die ("BUG: item->nowildcard_len > item->len || item->prefix > item->len)");
+ }
+
return magic;
}
One of the reason could be that the directory where you're adding the files is still registered as a submodule in the index, but actually it's not a valid git repository (e.g. it's missing .git directory).
So you should either:
initialize and update your submodules by:
git submodule init
git submodule update
Run from the parent directory where you had the error.
Make sure all non-initialized submodules have empty directories, so move any files out of there temporary.
or if you don't need this submodule, remove it:
git rm -f --cached subrepo
Run from the parent directory where you had the error.
Then try adding the files again.
See also:
[PATCH] pathspec: give better message for submodule related pathspec error
Re: [PATCH] pathspec: adjust prefixlen after striping trailing slash
assert failed in submodule edge case,
"item->nowildcard_len" in Google.
I got this problem too,Here is my way to fix this:
delete the .git folder in the submodule folder(if you did this before, go 2)
then excute follow command, directory is your submodule folder
git rm --cached directory
git add directory
then you can upload this folder like others
The only way i was able to get around this error, after
Deleting the .git folder
git deinit
git rm
going over these links git fatal pathspec submodule , deleting local repo , assert failed in submodule
was to rename directory and then add it again.
I have the .gitignore file with this code:
/app/cache/*
/app/logs/*
/app/bootstrap*
/vendor/*
/web/bundles/
/app/config/parameters.yml
but when I do :
$ git status
in any situation (before and after add and commit), I get a long text output like this:
...
# deleted: vendor/doctrine/orm/tools/sandbox/cli-config.php
# deleted: vendor/doctrine/orm/tools/sandbox/doctrine
# deleted: vendor/doctrine/orm/tools/sandbox/doctrine.php
# deleted: vendor/doctrine/orm/tools/sandbox/index.php
# deleted: vendor/doctrine/orm/tools/sandbox/xml/Entities.Address.dcm.xml
# deleted: vendor/doctrine/orm/tools/sandbox/xml/Entities.User.dcm.xml
# deleted: vendor/doctrine/orm/tools/sandbox/yaml/Entities.Address.dcm.yml
# deleted: vendor/doctrine/orm/tools/sandbox/yaml/Entities.User.dcm.yml
# modified: vendor/friendsofsymfony/user-bundle/FOS/UserBundle
# modified: vendor/gedmo/doctrine-extensions
# modified: vendor/herzult/forum-bundle/Herzult/Bundle/ForumBundle
# modified: vendor/kriswallsmith/assetic
# modified: vendor/symfony/property-access/Symfony/Component/PropertyAccess/.gitignore
# modified: vendor/symfony/property-access/Symfony/Component/PropertyAccess/StringUtil.php
# modified: vendor/symfony/symfony/CHANGELOG-2.1.md
...
The vendors directory is in .gitignore file, so I don't know what is happening.
I've tried with:
$ sudo git clean -dxf
but nothing changes.
Your vendor directory is checked in to the repo. To remove it, go to your git repo root and run:
git rm -r --cached vendor
This will recursively (due to -r flag) remove vendor from your git repo. The --cached flag will keep the local copy of vendor directory in tact. Note that if there are other devs working with the repo, their copy of the vendor directory will be removed and they will need to bundle install again.
Once you've untracked the directory in git, you can commit the change using:
git commit -m "untrack vendor directory"
Thereafter, .gitignore will happily ignore any changes within the vendor directory next time onwards.
Also, you don't need your entries in .gitignore to begin with a /. Use the / when you want to ensure that only files/folders in root directory are ignored, and any file in a subdirectory matching the pattern should not be ignored.
It looks like you already have files under vendor/* checked in. .gitignore ignores only untracked files. See also the first paragraph in man gitignore.
I am creating a svn diff patch, however it seems the image files are not getting included. The patch contain similar lines for each image file, as shown below:
Index: crimgeoprofile/code/jquery/css/ui-lightness/images/animated-overlay.gif
===================================================================
Cannot display: file marked as a binary type.
svn:mime-type = application/octet-stream
Index: crimgeoprofile/code/jquery/css/ui-lightness/images/animated-overlay.gif
===================================================================
--- crimgeoprofile/code/jquery/css/ui-lightness/images/animated-overlay.gif (revision 1510040)
+++ crimgeoprofile/code/jquery/css/ui-lightness/images/animated-overlay.gif (working copy)
I am using the following command to create a patch:
svn diff > test.diff
Any suggestions on how I can include image files will be appreciated.
SVN does not support to include binary files in diffs. As a side note: git does support binary files. The resulting patch file looks like this:
diff --git a/bin/windows/SDL_mixer.dll b/bin/windows/SDL_mixer.dll
new file mode 100644
index 0000000000000000000000000000000000000000..f48ee2da696f92b66940b91b52aa53c2
GIT binary patch
literal 160256
zcmd?S4SZD9)i*kmOyYopCrYBxf<%o9l`2uFL_&=TgA|RT7>j7Ev^CX7sg%wregu+E
z26K8G$kPW}+uD|hZFwrKv_*(YAs#UMf~XNJW(<Ug6wVlm;iDl0WbXgJ_BoSD06*UQ
z-h1DBFF(yWXYaMwUVE*z*Is+=k13j2?MQYw94`DHi#Z&%c=BJq{Qc}d<;Xr~#Ovoc
zRu6jXl3M4jZ(VZNLl6HbYtG!qzCU-??5yw3`oRw#^JRVK!K}IdA7nlJgRDunPtThD
So technically it is possible, it just doesn't work with svn. So if you desperately need a patch file including binaries, consider checking out svn using git. It's easy: git svn clone http://path/to/svn. Also works similar with svn://.... You can then create a git diff, and apply that diff to any target. The target does not need to be a git repository. git apply my.patch
With Suversion 1.9 you can use --git flag for include binary content to patch file, for example:
svn diff https://storage/svn/project/trunk --git -c 42 > patch-42.diff
Subversion 1.8 already have --git flag, but ignore binary content with it.
Unfortunately, svn diff does not handle binary data.
Check some of the answers from: subversion diff including new files
In particular: https://stackoverflow.com/a/2255846/9822
The Image files are getting included in your diff as indicated by the lines with --- and +++ but they are included as whole files in the patch - this is due in part the the problem of how to meaningfully display changes in binary data such as images in a text only format - unless you would like pages of hex differences, (such as fc -b a.gif b.gif would produce).
So you are told that the files have changed and it is up to you to decide how you would like to compare them - for image files one of the best comparisons of the significant differences is the human eye - you would not expect a revision control system to be able to tell you "This was a picture of a bald man frowning but now it is a pretty redhead cheerleader smiling" would you?
We have a large base of code that contains several shared projects, solution files, etc in one directory in SVN. We're migrating to Mercurial. I would like to take this opportunity to reorganize our code into several repositories to make cloning for branching have less overhead. I've already successfully converted our repo from SVN to Mercurial while preserving history. My question: how do I break all the different projects into separate repositories while preserving their history?
Here is an example of what our single repository (OurPlatform) currently looks like:
/OurPlatform
---- Core
---- Core.Tests
---- Database
---- Database.Tests
---- CMS
---- CMS.Tests
---- Product1.Domain
---- Product1.Stresstester
---- Product1.Web
---- Product1.Web.Tests
---- Product2.Domain
---- Product2.Stresstester
---- Product2.Web
---- Product2.Web.Tests
==== Product1.sln
==== Product2.sln
All of those are folders containing VS Projects except for the solution files. Product1.sln and Product2.sln both reference all of the other projects. Ideally, I'd like to take each of those folders, and turn them into separate Hg repos, and also add new repos for each project (they would act as parent repos). Then, If someone was going to work on Product1, they would clone the Product1 repo, which contained Product1.sln and subrepo references to ReferenceAssemblies, Core, Core.Tests, Database, Database.Tests, CMS, and CMS.Tests.
So, it's easy to do this by just hg init'ing in the project directories. But can it be done while preserving history? Or is there a better way to arrange this?
EDIT::::
Thanks to Ry4an's answer, I was able to accomplish my goal. I wanted to share how I did it here for others.
Since we had a lot of separate projects, I wrote a small bash script to automate creating the filemaps and to create the final bat script to actually do the conversion. What wasn't completely apparent from the answer, is that the convert command needs to be run once for each filemap, to produce a separate repository for each project. This script would be placed in the directory above a svn working copy that you have previously converted. I used the working copy since it's file structure best matched what I wanted the final new hg repos to be.
#!/bin/bash
# this requires you to be in: /path/to/svn/working/copy/, and issue: ../filemaplister.sh ./
for filename in *
do
extension=${filename##*.} #$filename|awk -F . '{print $NF}'
if [ "$extension" == "sln" -o "$extension" == "suo" -o "$extension" == "vsmdi" ]; then
base=${filename%.*}
echo "#$base.filemap" >> "$base.filemap"
echo "include $filename" >> "$base.filemap"
echo "C:\Applications\TortoiseHgPortable\hg.exe convert --filemap $base.filemap ../hg-datesort-converted ../hg-separated/$base > $base.convert.output.txt" >> "MASTERGO.convert.bat"
else
echo "#$filename.filemap" >> "$filename.filemap"
echo "include $filename" >> "$filename.filemap"
echo "rename $filename ." >> "$filename.filemap"
echo "C:\Applications\TortoiseHgPortable\hg.exe convert --filemap $filename.filemap ../hg-datesort-converted ../hg-separated/$filename > $filename.convert.output.txt" >> "MASTERGO.convert.bat"
fi
done;
mv *.filemap ../hg-conversion-filemaps/
mv *.convert.bat ../hg-conversion-filemaps/
This script looks at every file in an svn working copy, and depending on the type either creates a new filemap file or appends to an existing one. The if is really just to catch misc visual studio files, and place them into a separate repo. This is meant to be run on bash (cygwin in my case), but running the actual convert command is accomplished through the version of hg shipped with TortoiseHg due to forking/process issues on Windows (gah, I know...).
So you run the MASTERGO.convert.bat file, which looks at your converted hg repo, and creates separate repos using the supplied filemap. After it is complete, there is a folder called hg-separated that contains a folder/repo for each project, as well as a folder/repo for each solution. You then have to manually clone all the projects into a solution repo, and add the clones to the .hgsub file. After committing, an .hgsubstate file is created and you're set to go!
With the example given above, my .hgsub file looks like this for "Product1":
Product1.Domain = /absolute/path/to/Product1.Domain
Product1.Stresstester = /absolute/path/to/Product1.Stresstester
Product1.Web = /absolute/path/to/Product1.Web
Product1.Web.Tests = /absolute/path/to/Product1.Web.Tests
Once I transfer these repos to a central server, I'll be manually changing the paths to be urls.
Also, there is no analog to the initial OurPlatform svn repo, since everything is separated now.
Thanks again!
This can absolutely be done. You'll want to use the hg convert command. Here's the process I'd use:
convert everything to a single hg repository using hg convert with a source type of svn and a dest type of hg (it sounds like you've already done this step)
create a collection of filemap files for use with hg convert's --filemap option
run hg convert with source type hg and dest type hg and the source being the mercurial repo created in step one -- and do it for each of the filemaps you created in step two.
The filemap syntax is shown in the hg help convert output, but here's the gist:
The filemap is a file that allows filtering and remapping of files and
directories. Comment lines start with '#'. Each line can contain one of
the following directives:
include path/to/file
exclude path/to/file
rename from/file to/file
So in your example your filemaps would look like this:
# this is Core.filemap
include Core
rename Core .
Note that if you have an include that the exclusion of everything else is implied. Also that rename line ends in a dot and moves everything up one level.
# this is Core.Tests
include Core.Tests
rename Core.Tests .
and so on.
Once you've created the broken-out repositories for each of the new repos, you can delete the has-everything initial repo created in step one and start setting up your subrepo configuration in .hgsub files.