I've got a Jenkins instance running on Windows and for one item, I need to check out from a CVS repository. As it is, the built-in CVS plugin for Jenkins is checking it out with Windows CRLF line endings. However, some of what needs to be done requires that the files have Unix style LF line endings.
Running a dos2unix like utility over the entire repository seems painful to do safely and reliably.
I could use Cygwin's cvs to do it by running a shell script build step, but I would prefer something better integrated with Jenkins.
So, short of writing up a script to call an external CVS, or fixing it afterwards with dos2unix, can Jenkins check out a CVS repository with Unix line endings on Windows?
Edit:
I've since come across this bug report indicating that this issue has been known since 2012 and not much has been done about it. I'll leave the question open in case someone does know of an alternative to what I've already mentioned.
I've added dos2unix steps pretty often in my build scripts for windows (automake chain is very sensitive to this stuff), it might seem a bit ugly/fragile, but it's ok I think for a Jenkins build : if it breaks because of new files you'll get build issues.
In my experience, there are always a couple of platform specific steps in the build, so a bit of custom code to support windows seems ok.
An alternative might be to tag your files as "binary" on the CVS server
(-kb flag) and make sure they are encoded correctly up there, this will avoid any transcoding at checkout time, but it seems more intrusive to me. I haven't actually used this for sources, I think it might mess the "diff" command.
A last idea might be to look more precisely at what tools are expecting the LF style endings on your win box, there might be mingW or windows versions you could deploy that support CRLF.
My vote still goes to the dos2unix solution overall.
If you don't want to make all your files binary in the repository, you can do each checkout or update as binary by adding "-kb" to the command.
Related
I have a bash script that works at the moment. It gets an image and JDK 8 from a link and then runs a installer for the JDK 8 to move on to setting up another piece of software.
As I was debugging the script, I kept finding myself having to delete directories and even the java installation because when I introduce a fix and rerun the script, I have to wait for everything to download again and I have to worry about duplicate files messing up my current logic -which can probably be improved, but I'll go to the StackExchange Code Review site later.
At the moment, I would like to know what approaches there are to prevent commands -like downloading the JDK and running the JDK installer script all over again and others- from running again.
What kind of general approaches are out there for cases such as these?
For the JDK download and running the installer, I did think of simply checking for the existing of java on the system and if there is then bash would not not to run those commands.
However, there are other commands I do not want run and I do want to simply check, for example, the existence of certain files to prevent wget-ing them all over again and moving them -causing duplicates. (Should I maybe suck it up and do that anyway as that might be best practice?)
I did also think of perhaps, at each successful command, outputting like a 1 to a text file and mapping each line in that text file to the commands run in the script (like using an if statement to see if that command had a 1 or not in the text file) and if it was a 0, then the script would know only to run that command and never the 1s.
That sounded clunky to me and I am pretty sure that is not a good approach.
I've been using Vim (MacVim) exclusively for months now, and I love it. Before using Vim though, I used Coda (I'm on OSX), and the one thing I miss about Coda is the way it marks my files for publishing via FTP whenever I edit them. I can then choose to upload the modified files single files individually, or to publish all of them in one go.
Is there anything that will do this or similar in Vim?
I'm aware netrw can edit directly over FTP, but I have all my sites running locally as mirrors of the online sites, so I need to edit locally and publish files remotely as and when I need. I've Googled for solutions but can't find anything.
I'm far more productive in Vim while in the editor, but having to open up an FTP program and hunt and peck to copy across files as I edit them seperately is a real pain, and makes me less productive overall when working on websites when compared to working in Coda.
Any suggestions welcome :)
Transmit, Cyberduck and YummyFTP (the ones I know) all have their own version of "automated folder syncing" where you work locally and any modified files are uploaded on change: you setup an "observer" and never have to hit a special button or shortcut again.
If you want to only use MacVim you could write a little command that uploads the current file on save: a script that would use the current file's path to construct an scp command. But this doesn't seem very portable/practical to me. It appears some people have already written something like that: here and there.
The right way is (not only in my opinion) to:
have everything under some kind of VCS like Git, Subversion or Mercurial or whatever floats your boat
write your code in a local clone/checkout and test the hell out of it on a local server
commit only working code
push milestones to a staging server used by you and your clients to test everything, this part can be automated via a post-commit hook or something like that
deploy only validated changes on your production server.
All the aforementioned Version Control Systems can be used directly from Vim's command line with :!git commit or :!svn update… If you need more abstraction, Fugitive (Git-only) or VCSCommand (multi-VCS, my choice) are here to help.
On your loss of productivity due to uploading files: I think it's very normal because you essentially perform very different tasks with very different neurological needs. This speed bump can also be experienced when previewing a layout change in your browser, looking up a color in Photoshop or any other similar task. I don't know of a way to pilot Photoshop, Chrome's dev tools or Outlook from within Vim and I don't think such a gizmo could reallistically exist so you will probably have to bear with it.
You can try git-ftp - a git based command line ftp client. Then you can manage your project as a Git repository, and git-ftp will only upload the files marked with Git - and only if they have changed.
The downside is that you are going to have to learn Git - and that's a bit overkill for what you need.
The upside is that you are going to learn and use Git.
I have since stumbled upon a Vim plugin which works with Transmit (which I happen to use as my main FTP client) and allows me to upload the current file to the server with a simple keymap (Ctrl+U) as I edit.
This strikes a nice balance between being a very simple solution, and one that does enough of what I need to improve my productivity significantly.
Any more suggestions are still welcome!
TortoiseHg is superior to TortoiseGit in many features.
Would be a matter of changing a few lines of code to make a new TortoiseGit based on TortoiseHg, or it requires weeks/months of development?
You can use Mercurial with Git repositories via the excellent hg-git extension, which is developed by the github folks. Any recent TortoiseHg version already has the dulwich library, you just need to clone hg-git and add a couple lines to your mercurial.ini file.
The instructions are in the TortoiseHg help under "Use with other VCS systems" .
Directions for configuring ssh can be found here. However, TortoiseHg will automatically use the plink it comes bundled with, so you don't need to worry about configuring PuTTY's plink.
Note that I've had some problems cloning large repos on XP due to file locking issues which don't seem to occur on Windows 7.
It would definitely not be just a matter of changing a few lines.
TortoiseHg is intimately written against Mercurial. It doesn't just wrap around the command line client, it integrates with the core python code.
The entire program, from bottom and up, would have to be rewritten.
It is much more constructive to give the TortoiseGit team good feedback on what you would like see improved.
Here's the scenario I'm currently running into:
Programmer A (Using a Mac Version of Dreamweaver) edits file client.php and commits that file to the production branch of Project Foo's repository
Programmer B (Using a Windows Version of Dreamweaver) edits file client.php to fix a bug in the that file. He then does a cp clientInfo.php ../prod-branch/clientInfo.php to take that bug-fix from his working copy to the production branch.
Programmer B then does an svn diff ../prod-branch/clientInfo.php to see what svn says his changes were only to discover that svn says he's changed every line in the file!
Now, this is what I believe is happening:
When the file gets edited by Mac, Dreamweaver on Mac replaces all the Windows newline characters with Mac newline characters so that it's readable in Dreamweaver. In short, Dreamweaver has altered every line in the file. Now, once the commit is done, svn sees that every line of the file has changed and marks this fact down. When the windows programmer makes a change and the newline characters get changed again, svn thinks that, again, every line has changed.
My question is this: How can we prevent this from happening? I know there's no way to undo the damage that's already been done, but I want to prevent this from happening in the future.
You need to use the "svn:eol-style" property on all text files. Usually setting it to "native" will suffice
Dreamweaver has an option to set which line break type it uses. Edit (on Mac: Dreamweaver ) -> Preferences, Code Format, Line break type.
Get your users to have the same setting, and things should play a little better together. It would be better, of course, if you can set your source control to ignore line break differences.
svn:eol-style is a property that can be set centrally in the repository and should sort out your problem.
Check out the chapter on New Line Sequences in the Subversion book.
The solution to this problem is the svn:eol-style property. When this property is set to a valid value, Subversion uses it to determine what special processing to perform on the file so that the file's line ending style isn't flip-flopping with every commit that comes from a different operating system. The valid values are:
I've been using Subversion for code control with TortoiseSVN to interface with the server for the past few months, and in general it's been going great! However, occasionally my FoxPro IDE will change the case of a file extension without warning where "program.prg" becomes "program.PRG") TortoiseSVN apparently takes this to mean the first file was removed, becoming flagged as "missing" and the second name comes up as "non-versioned", wreaking havoc on my ability to track changes to the file. I understand that Subversion has it origins in the case-sensitive world of *nix but, is there any way to control this behavior in either Subversion or TortoiseSVN to be file name case-insensitive when used with Windows?
Unfortunately, Subversion is case-sensitive. This is due to the fact that files from Subversion can be checked out on both case-sensitive file systems (e.g., *nix) and case-insensitive file systems (e.g., Windows, Mac).
This pre-commit hook script may help you avoid problems when you check in files. If it doesn't solve your problem, my best suggestion is to write a little script to make sure that all extensions are lowercase and run it every time before you check in/check out. It'll be a PITA, but maybe your best bet.
Windows does support case sensitivity, but you must send it the correct POSIX flags on CreateFile from the Windows API! A registry key may need changed (SFU/Tools for Unix and Ultimate Windows 7 has this registry entry already set so windows supports case sensitive file names).
Windows is designed off of Unix, but things such as Explorer.exe and other programs are designed to disallow case sensitivity for backwards compatibility and security (mostly when dealing with dos executing notepad.exe vs. NOTEPAD.EXE, where all caps is a virus or malware).
But Vista+ has security attributes which makes this obsolete.
TortiousSVN just doesn't support passing this posix flag while making and renaming files.
I use TortoiseSVN with VFP, and it mostly-seamlessly handles the case flipping. The only time it doesn't is if I have the file open in the IDE when I try to do the commit: the file lock VFP holds confuses it. Is this where your problem comes in, or are there other issues?
I did a presentation at FoxForward last year about using VFP with Subversion: most of the presentation dealt with the command line, but there are a couple of slides at the end that have links to tools that help you work with Subversion in VFP. http://docs.google.com/Presentation?id=dfxkh6x4_3ghnqc4
Kit, you comment above that VFP's binary-based source files are tough to work with in Subversion. The link I gave above mentions a couple of tools to make it easier, but the one I work with is Christof Wollenhaupt's TwoFox utility -- it converts a VFP project to text-only. You have to run it manually, but I don't have a problem with that.
http://www.foxpert.com/docs/cvs.en.htm
I believe the random upper and lower case on the extensions isn't random at all.
I remember testing on this. If you modify a program from the project manager.
By clicking on the modify button let's say. And then save the changes the extension is lower case. If you do a modify command from the command window and save the changes the extension is upper case. Apparently the coders at Microsoft didn't worry about the extension case being the same.
TortoiseSVN has a Repairing File Renames feature. It requires manual intervention and it actually issues a file rename operation to be committed but nonetheless addresses current use case by keeping file history.
Nope you sure can't. SVN is case-sensitive unless you were to rewrite the code somehow ... it is open-source.
We had a similar problem and I found a better solution than the ones exposed here, so I'm sharing it now:
For commits done manualy, now TortoiseSVN fixes the case of the file names automatically: it renames the local files to match the case of the versioned files (just by opening the commit window in that path), so there should be no problem with that.
For automated commits you cannot use TortoiseSVN, as it requires you to manually confirm the commit (it opens the commit window with a specific message, but you still have to click ok). But if you directly use Subversion (svn) to make an automated commit, then you will have the case-sensitive issue on that commit, as Subversion is still case-sensitive...
How to solve this for automated commits? Well, I tried a mixed approach: creating a batch file called FixCaseSensitiveFileNames.bat that you may call passing the path you want to fix before the commit, for example: call FixCaseSensitiveFileNames.bat C:\MyRepo. The batch file opens TortoiseSVN for a manual commit, and that automatically fixes the file names, but then it closes the commit window after a predefined pause, so you can continue with the automated commit with the case-sensitive file names already fixed. The pause is emulated with a local ping, and you can change the duration by changing the -n argument, which is the number of tries. If you don't make a long enough pause, it exist the risk to close the TortoiseSVN window before it makes its magic fix. Here it is the code of the batch file:
#echo off
REM *** This BAT uses TortoiseSVN to fix the case-sensitive names of the files in Subversion
REM *** Call it before an automated commit. The Tortoise commit fixes this issue for manual commits,
REM *** so the trick is opening the commit window and close it automatically after a pause (with ping).
REM *** %1 = path to be fixed
start TortoiseProc.exe /command:commit /path:"%1"
ping localhost -n 10 >nul
taskkill /im TortoiseProc.exe
This totally solved the issue for our automated daily build process. The only problem I see is a window will open for a few seconds, which was not a problem for our daily build, but if that is a problem for you there could be workarounds too...