Append Problem
I am trying to read rules from a file and in a bash script set the rules, for this to work I need to append the svn:ignore rules to the directory.
I have an example set of data:
/js/blank.html
/js/index.php
/js/spacer.gif
If I try to run svn propedit svn:ignore js/ < "blank.html" or echo "test" | svn propedit svn:ignore js/ I get the following error:
Vim: Warning: Input is not from a terminal
Vim: Error reading input, exiting...
Vim: preserving files...
Vim: Finished.
svn: E200012: system('/usr/bin/editor svn-prop.tmp') returned 256
Is it possible to append rules to svn:ignore?
Alternatives
I know you can use propset to set a list of rules, one per line as per Ignore multiple specific files with svn but that is not the behaviour I am looking for as I would have to order my list in bash some how and make sure I do not overwrite any existing changes.
Interestingly this came up in 2005 but there was no outcome, maybe I should track him down.
If anyone knows how to use propset to append that would be useful too.
Thanks
proplist
I am automating these additions from a file, so I want to ignore all of these apart from local.xml.sample
ls -h errors/
404.php default enterprise processor.php
503.php design.xml local.xml.sample report.php
My propedit rules:
.htaccess
404.php
503.php
design.php
processor.php
report.php
design.xml
Proplist output:
svn proplist errors
Properties on 'errors':
svn:ignore
There are more complex examples I can give, but the baseline is that I am trying to automate all rules from a single file to create a reliable way of ignore all core files of a software package from my repository. I know I am using the wrong tool for the job, but management is management, I feel like I am hitting a nail with a piece of paper.
Related
I would like to write a script that restores a file, but preserving the changes that may be done after the backout file is created.
With more details: at some moment I create a backup of a file (file_orig). Do some changes to the original file as well(file_my_changes). After that, the original file can be changed again (file_additional_changes), but after the restore I want to have the backup file, plus the additional changes (file_orig + file_addtional_changes). In general backing out my changes only.
I am talking about grub.cfg file, so the expected possible changes will be adding or removing parts of a line.
Is it possible this to be done with a bash script?
I have 2 ideas:
Add some comments above the lines I am going to change, and then before the resotore if the line differ from the one from the backed out file, to read the comment, which will tell me what exactly to remove from the line;
If there is a way to display only the part of the line that differs from the file_orig and file_additional_changes, then to replace this line with the line from file_orig + the part that differs. But I am not sure if this is possible to be done at all.
Example"
line1: This is line1
line2: This is another line1
Is it possible to display only "another"?
Of course any other ideas are welcome!
Thank you!
Unclear, but perhaps if you're using a bash script you could run a diff on the 2 edited file and the last one and save that output someplace that you want to keep it? That would mean you have a copy of the changes.
Or just use git like everybody else.
One possibility would be to use POSIX commands patch and
diff.
Create the backup:
cp operational-file operational-file.001
Edit the operational file.
Create a patch from the differences:
diff -u operational-file.001 operational-file > operational-file.patch001
Copy the operational file again.
cp operational-file operational-file.002
Edit the operational file again.
Create a new patch
diff -u operational-file.002 operational-file > operational-file.patch002
If you need to recover but skip the changes from patch.001, then:
cp operational-file.001 operational-file
patch -i patch.002
This would apply just the second set of changes to the original file, as log as there's no overlap.
Consider using a version control system to keep records of the file changes. Consider using date/time stamps instead of version numbers on the file names.
I want to make a stat via svn,and the input and output like this:
input:svn url,statDate,endDate
output:
person1 20Lines
persion2 30Lines
eg: my.sh http://svn.xxx.com/trunck/xxx/xxx/test.php 2014-12-01 2014-12-10
I use the shell script,and my idea like follows:
1.svn co the whole project
2.get the changed files within a specific date based on the giving url
3.use the command 'svn blame [file url]' get the changed infos by person
but now I don't know how to get the changed files in a directory within a specific date
please give me any ideas,thanks!
You don't want to use svn status, as this only shows if a file has changed in the working copy, whereas you are looking for commit info. You can, however, use
svn log -vq -r {2014-12-01}:{2014-12-10} http://svn.xxx.com/trunck/xxx/xxx/test.php
for your purposes. The -v adds information on the changed paths, the -q suppresses the commit message and only shows the changed paths.
You can then parse this output for the changed paths, user, and revision, and then use svn diff to find out for a given path how many lines were changed by that user in the specified revision.
While trying to setup a dropbox folder with git, I saw a "Icon\r" file which is not created by me. I try to ignore it in the ~/.gitignore file. But adding Icon\r Icon\r\r Icon? won't work at all.
You can use vim as well.
vim .gitignore
in a new line write Icon, then
press ctrl+v and then press Enter
repeat step 3
save and exit (shortcut: ZZ)
Now you should have Icon^M^M and it's done :)
For a smarter use you could add it to your gitignore global config file in ~/.gitignore_global.
(This improves on the original answer, following a suggestion by robotspacer, according to hidn's explanation.)
The Icon? is the file of OS X folder icon. The "?" is a special character for double carriage return (\r\r).
To tell git to ignore it, open a terminal and navigate to your repository folder. Then type:
printf "Icon\r\r" >> .gitignore
If the file does not exist, it will be created and Icon\r\r will be its one line. If the file does exist, the line Icon\r\r will be appended to it.
"Icon[\r]" is probably a better alternative.
In vim, you just put Icon[^M], which is Icon[ followed by CtrlV, Enter then ].
The problem with "Icon\r\r" is EOL conversion.
The whole line is actually "Icon\r\r\n", counting line ending. Based on your setup, CRLF may be converted to LF on commit, so your repo will actually have "Icon\r\n". Say you sync the changes to another repo. You will get "Icon\r\n" in that working directory, which ignores Icon but not Icon^M. If you further edit .gitignore and commit it there, you will end up with "Icon\n" - completely losing \r.
I encountered this in a project where some develop on OS X while some on Windows. By using brackets to separate \r and the line ending, I don't have to repeat \r twice and I don't worry about EOL conversion.
The best place for this is in your global gitignore configuration file. You can create this file, access it, and then edit per the following steps:
>> git config --global core.excludesfile ~/.gitignore_global
>> vim ~/.gitignore_global
press i to enter insert mode
type Icon on a new line
while on the same line, ctrl + v, enter, ctrl + v, enter
press esc, then shift + ; then type wq then hit enter
Regarding Naming (and Quoting) Things: First, more people would benefit by knowing that ANSI-C Quoting can be used to unambiguously match the macOS icon file. Both Icon$'\r' or $'Icon\r' and work in Bash and Zsh and most other modern shells, I hope, such as Fish.
Keep Your .gitignore Editable: While I'm impressed by the byte-level manipulation offered by other answers here, these methods are brittle in practice. Simply put, programmers tend to use text editors, and many of these editors are configured to alter line endings when saving a file. (For example, see this VS Code discussion about line ending normalization.)
Do you want your careful byte editing undone by your editor? Of course not. So perhaps you find it practical and convenient to configure your editor so that it doesn't affect line endings. You might look into (a) editor-specific configuration settings; or (b) cross-editor configuration (i.e. EditorConfig).
But this gets complex and messy. If want a simpler, more flexible way, use this in your .gitignore file:
# .gitignore
Icon?
![iI]con[_a-zA-Z0-9]
Explanation for the patterns:
Use Icon? because the gitignore format does not support \r as an escape code.
Use [iI] because Git can be case sensitive.
Use [_a-zA-Z0-9] to catch many common ASCII characters; you may want to broaden this.
You can test that your gitignore patterns are working as expected with:
git check-ignore -v *
For example, for testing, with these files in a directory:
-rw-r--r--# Icon?
-rw-r--r-- icon8
drwxr-xr-x icons
-rw-r--r-- iconography
... the result of git check-ignore -v * is:
/Users/abc/.gitignore:3:Icon? "Icon\r"
/Users/abc/.gitignore:4:![iI]con[_a-zA-Z0-9] icon_
/Users/abc/.gitignore:4:![iI]con[_a-zA-Z0-9] icons
This is what you want.
Long Term Recommendation This problem would be trivial to fix if Git supported the \r escape in .gitconfig files. One could simply write:
# .gitignore
Icon[\r]
So I suggest we engage with the Git community and try to make this happen.
(If you do want to wade in and suggest a patch to Git, be sure to read first.)
References
From the gitignore documentation:
Otherwise, Git treats the pattern as a shell glob: "*" matches anything except "/", "?" matches any one character except "/" and "[]" matches one character in a selected range. See fnmatch(3) and the FNM_PATHNAME flag for a more detailed description.
Please see This linuxize.com article for good examples of the square bracket syntax and negation syntax in .gitignore files.
For those that want to dig deep and see how pattern matching has changed over time in the Git source code, you can run this search for uses of fnmatch in the git repository on GitHub.
The Icon? is the file of OSX folder icon. It turn out that \r is actually CRLF. So I use ruby to add the line to .gitignore file. Open terminal and navigate to home folder, then:
> irb
>> f = File.open(".gitignore", "a+") #<File:.gitignore>
>> f.write("Icon\r\r") # output a integer
>> f.close
>> exit
For me this worked in TextMate: Icon<CR><CR>. The <CR> is a carriage return character, which is at ctrl-alt-return on the keyboard. You can also find it in the standard Character Viewer app searching for cr. Please note that the <CR> is an invisible character, so it's only visible if the editor is set up to show them.
I'm posting just an update answer because the one above didn't work for me but actually simply adding Icon? in my .gitignore worked. If you look at your name file on your Finder, it is actually how it is displayed.
Icon[\r] did not work for me. I had to use the following in .gitignore...
Icon*
I also added Icon* to my Settings > Core > Ignored Names in Atom...
.git, .hg, .svn, .DS_Store, ._*, Thumbs.db, desktop.inis, Icon*
Add Icon? to your .gitignore file and save it. It should do the job.
Icon?
To avoid wasting time on such trivial issues, I recommend using gibo.
gibo dump macOS >> .gitignore
The result:
### Generated by gibo (https://github.com/simonwhitaker/gibo)
### https://raw.github.com/github/gitignore/e5323759e387ba347a9d50f8b0ddd16502eb71d4/Global/macOS.gitignore
# General
.DS_Store
.AppleDouble
.LSOverride
# Icon must end with two \r
Icon
# Thumbnails
._*
# Files that might appear in the root of a volume
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
.TemporaryItems
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
I have imported a huge hierarchy of maven projects into IntelliJ idea, now idea has created .iml projects on all different levels. I would like to svn:ignore these files.
There are several similar questions, for example this one: svn (subversion) ignore specific file types in all sub-directories, configured on root?
However: the answer is always to apply svn:ignore with the --recursive flag. That's not what I am looking for, I just want to ignore the files I created, not set a property on the hundreds of directories underneath.
Basically, what I have is the output of
svn status | grep .iml
which looks like this:
? foo/bar/bar.iml
? foo/baz/phleem.iml
? flapp/flapp.iml
etc.
What I would like to do is for each entry dir/project.iml to add svn:ignore *.iml to dir. I am guessing I have to pipe the above grep to sed or awk and from there to svn, but I am at a total loss as to the exact sed or awk command on one hand and the exact svn command (with which I won't override existing svn:ignore values) on the other hand.
Update: what I am also not looking for are solutions based on find, because possibly there are projects in this hierarchy where .iml files are in fact committed and I wouldn't want to interfere with those projects, so I'm looking to add the property only for .iml files my IDE has created.
You can set up your client to globally ignore given file extensions. Just add
global-ignores = *.iml
into your Subversion config file.
Update: If you want to only ignore iml files in the directories involved, you can try
svn status | grep '^\?.*\.iml' | sed 's=^? *=./=;s=/[^/]*$==' | xargs svn propset svn:ignore '*.iml'
We have a large base of code that contains several shared projects, solution files, etc in one directory in SVN. We're migrating to Mercurial. I would like to take this opportunity to reorganize our code into several repositories to make cloning for branching have less overhead. I've already successfully converted our repo from SVN to Mercurial while preserving history. My question: how do I break all the different projects into separate repositories while preserving their history?
Here is an example of what our single repository (OurPlatform) currently looks like:
/OurPlatform
---- Core
---- Core.Tests
---- Database
---- Database.Tests
---- CMS
---- CMS.Tests
---- Product1.Domain
---- Product1.Stresstester
---- Product1.Web
---- Product1.Web.Tests
---- Product2.Domain
---- Product2.Stresstester
---- Product2.Web
---- Product2.Web.Tests
==== Product1.sln
==== Product2.sln
All of those are folders containing VS Projects except for the solution files. Product1.sln and Product2.sln both reference all of the other projects. Ideally, I'd like to take each of those folders, and turn them into separate Hg repos, and also add new repos for each project (they would act as parent repos). Then, If someone was going to work on Product1, they would clone the Product1 repo, which contained Product1.sln and subrepo references to ReferenceAssemblies, Core, Core.Tests, Database, Database.Tests, CMS, and CMS.Tests.
So, it's easy to do this by just hg init'ing in the project directories. But can it be done while preserving history? Or is there a better way to arrange this?
EDIT::::
Thanks to Ry4an's answer, I was able to accomplish my goal. I wanted to share how I did it here for others.
Since we had a lot of separate projects, I wrote a small bash script to automate creating the filemaps and to create the final bat script to actually do the conversion. What wasn't completely apparent from the answer, is that the convert command needs to be run once for each filemap, to produce a separate repository for each project. This script would be placed in the directory above a svn working copy that you have previously converted. I used the working copy since it's file structure best matched what I wanted the final new hg repos to be.
#!/bin/bash
# this requires you to be in: /path/to/svn/working/copy/, and issue: ../filemaplister.sh ./
for filename in *
do
extension=${filename##*.} #$filename|awk -F . '{print $NF}'
if [ "$extension" == "sln" -o "$extension" == "suo" -o "$extension" == "vsmdi" ]; then
base=${filename%.*}
echo "#$base.filemap" >> "$base.filemap"
echo "include $filename" >> "$base.filemap"
echo "C:\Applications\TortoiseHgPortable\hg.exe convert --filemap $base.filemap ../hg-datesort-converted ../hg-separated/$base > $base.convert.output.txt" >> "MASTERGO.convert.bat"
else
echo "#$filename.filemap" >> "$filename.filemap"
echo "include $filename" >> "$filename.filemap"
echo "rename $filename ." >> "$filename.filemap"
echo "C:\Applications\TortoiseHgPortable\hg.exe convert --filemap $filename.filemap ../hg-datesort-converted ../hg-separated/$filename > $filename.convert.output.txt" >> "MASTERGO.convert.bat"
fi
done;
mv *.filemap ../hg-conversion-filemaps/
mv *.convert.bat ../hg-conversion-filemaps/
This script looks at every file in an svn working copy, and depending on the type either creates a new filemap file or appends to an existing one. The if is really just to catch misc visual studio files, and place them into a separate repo. This is meant to be run on bash (cygwin in my case), but running the actual convert command is accomplished through the version of hg shipped with TortoiseHg due to forking/process issues on Windows (gah, I know...).
So you run the MASTERGO.convert.bat file, which looks at your converted hg repo, and creates separate repos using the supplied filemap. After it is complete, there is a folder called hg-separated that contains a folder/repo for each project, as well as a folder/repo for each solution. You then have to manually clone all the projects into a solution repo, and add the clones to the .hgsub file. After committing, an .hgsubstate file is created and you're set to go!
With the example given above, my .hgsub file looks like this for "Product1":
Product1.Domain = /absolute/path/to/Product1.Domain
Product1.Stresstester = /absolute/path/to/Product1.Stresstester
Product1.Web = /absolute/path/to/Product1.Web
Product1.Web.Tests = /absolute/path/to/Product1.Web.Tests
Once I transfer these repos to a central server, I'll be manually changing the paths to be urls.
Also, there is no analog to the initial OurPlatform svn repo, since everything is separated now.
Thanks again!
This can absolutely be done. You'll want to use the hg convert command. Here's the process I'd use:
convert everything to a single hg repository using hg convert with a source type of svn and a dest type of hg (it sounds like you've already done this step)
create a collection of filemap files for use with hg convert's --filemap option
run hg convert with source type hg and dest type hg and the source being the mercurial repo created in step one -- and do it for each of the filemaps you created in step two.
The filemap syntax is shown in the hg help convert output, but here's the gist:
The filemap is a file that allows filtering and remapping of files and
directories. Comment lines start with '#'. Each line can contain one of
the following directives:
include path/to/file
exclude path/to/file
rename from/file to/file
So in your example your filemaps would look like this:
# this is Core.filemap
include Core
rename Core .
Note that if you have an include that the exclusion of everything else is implied. Also that rename line ends in a dot and moves everything up one level.
# this is Core.Tests
include Core.Tests
rename Core.Tests .
and so on.
Once you've created the broken-out repositories for each of the new repos, you can delete the has-everything initial repo created in step one and start setting up your subrepo configuration in .hgsub files.