I am working on a script that has become fairly convoluted. I suspect there are several sections that have nearly identical code. Can I (and how can I) open the file in vim, with two (or more) windows on the buffer, and diff the contents of the windows on the same file? vimdiff seems to work only on two files. If I make a copy of the file and try to vimdiff the two versions, the diff origin remains locked on the beginning of the file. Although I can unscroll-lock the windows, and move the windows to the parts of the file I want to compare the diffs do not show up. Any hints or tips? I could cut and paste the sections I want to compare to different files and then apply vimdiff but then I risk getting lost in what section came from where when I try to patch the separate files together, and I feel sure there must be a more straightforward, easier way.
What I usually do is diff to a copy
:%w %.alt
:vert diffsplit %.alt
And then happiliy rearrange the 'alt' version so that the pseudo-matching bits get aligned.
Note that (presumably) git contains spiffy merge/diff cow-powers that should be able to detect sub-file moved block changes.
Although I haven't (yet) actually put this into practice, I have a hunch that the very nice git plugin fugitive for vim might be able to leverage some of this horsepower to make this easier. Note: fully expect this to require scriptinh before being usable, but I still thought it would be nice to share this idea (perhaps you can share a script if you get to it first!)
As an alternative solution that I've been using occasionally and which works very nicely in my opinion is linediff.vim.
It allows you to use visual mode to select two bodies of text from arbitrary buffers (or the same for that matter) and run vimdiff on them. The beauty of it, is that when you edit and save the temporary diff buffers, you update the original buffers with the changes, without saving.
One of my use-cases is when I'm resolving merge issues related to script refactoring and reordering, where a function has been moved and perhaps also modified. In order to make sure you do not lose any of the modifications coming in from either ancestor, you diff the two versions of the function alone by visually selecting them and running the linediff command.
Related
I'm working with set of binary files that can be "decompiled" to or "compiled" from a set of INI files. Since both the binary and INI files are checked into my repository, I use a small script to (de)compile all of them.
Our workflow usually involves editing the binary files directly, and decompiling the modified binaries to INI format. However, occasionally we need to edit the INI files and compiling the changes to binaries.
The question: Can I make a single makefile that detects which set was modified more recently, and automatically issues (de)compile commands in either direction to keep both set of files up to sync? I prefer using common (GNU?) make features, but if there is a more specialized tool that works, I'm all ears.
(I could make two separate directives, "decompile-all" and "compile-all". I want to know if there's a single-command option.)
I don't see how that can work. Suppose it could be done in make; now you have two files foo.exe and foo.ini (you don't say what your actual filename patterns are). You run make and it sees that foo.exe is newer than foo.ini, so it decompiles the binary to build a new foo.ini. Now, you run make again and this time it sees that foo.ini is newer than foo.exe, because you just built the former, so it compiles foo.ini into foo.exe.
Etc. Every time you run make it will perform an operation on all the files because one or the other will always be out of date.
The only way this could work would be if you (a) tested to see if files did not have exactly identical time last modified times, and (b) had a way to reset the time on the compiled/decompiled file so that it was identical to the file it was built from, rather than "now" which is the default of course.
The answer is that make cannot be used for this situation. You could of course write yourself a small shell script that went through every file and tested that the last modified times were identical or not, and if not compiled them then used touch -m -r origin where origin is the file that had the newer modification time, so that both had the same modification time.
I'm having trouble monitoring a file for changes. I need to be able to know when a file changes, and when it does, I need the new line that was added. I intend to parse each line and find ones that match certain criteria, and act on information in those lines. I know the expected number of matching lines ahead of time, but I do not know how many lines in total will be added to the file, or where the matching lines will be.
I've tried 2 packages so far, with no avail.
fsnotify/fsnotify
As fas as I can tell, fsnotify can only tell me when a file is modified, not what the details of the modification was. Since I need to know what exactly was added to the file, this is no good for me.
(As a side-question, can this be run in a loop? The example that I tried exited after just one modification. I need to monitor for multiple modifications.)
hpcloud/tail
This package tries to mimic the Unix tail command, but it seems to have its own issues. The output that I get includes timestamps and other data - I just want the added line, nothing else. Also, it seems to think a file has been modified multiple times, even when it's just one edit. Further, the deal breaker here is that it does not output the last line if the line was not followed by a newline character.
Delegating to tail
I came across this answer, which suggests to delegate this work to the tail command itself, but I need this to work cross-platform (specifically, macOS, Linux and Windows). I don't believe that an equivalent command exists on Windows.
How do I go about tackling this?
#user2515526,
Usually changed diff is out of scope of file watchers' functionality, because, you know, you could change an image, and a watcher would need to keep a track several Mb of a diff in memory, and what if we have thousands of files?
However, as bad as it sounds, this may be exactly the way you want to implement this (sure, depends on your app, etc. - could be fine for text files), i.e. - keeping a map of diffs (1 diff per file) since last modification. Cannot say I like it, but sounds like fsnotify has no support for changes/diffs that you need.
Also, regarding your question about running in a loop, maybe you can get some hints here: https://github.com/kataras/iris/blob/8370d76910cdd8de043753ed81ae080eae8dc798/utils/file.go
Its a framework that allows to build a server that watches for TypeScript file changes. So sounds similar to your case/question.
Cheers,
-D
Want to upgrade my file management productivity by replacing 2 panel file manager with command line (bash or cygwin). Can commandline give same speed? Please advise a guru way of how to do e.g. copy of some file in directory A to the directory B. Is it heavy use of pushd/popd? Or creation of links to most often used directories? What are the best practices and a day-to-day routine to manage files of a command line master?
Can commandline give same speed?
My experience is that commandline copying is significantly faster (especially in the Windows environment). Of course the basic laws of physics still apply, a file that is 1000 times bigger than a file that copies in 1 second will still take 1000 seconds to copy.
..(howto) copy of some file in directory A to the directory B.
Because I often have 5-10 projects that use similar directory structures, I set up variables for each subdir using a naming convention :
project=NewMatch
NM_scripts=${project}/scripts
NM_data=${project}/data
NM_logs=${project}/logs
NM_cfg=${project}/cfg
proj2=AlternateMatch
altM_scripts=${proj2}/scripts
altM_data=${proj2}/data
altM_logs=${proj2}/logs
altM_cfg=${proj2}/cfg
You can make this sort of thing as spartan or baroque as needed to match your theory of living/programming.
Then you can easily copy the cfg from 1 project to another
cp -p $NM_cfg/*.cfg ${altM_cfg}
Is it heavy use of pushd/popd?
Some people seem to really like that. You can try it and see what you thing.
Or creation of links to most often used directories?
Links to dirs are, in my experience used more for software development where a source code is expecting a certain set of dir names, and your installation has different names. Then making links to supply the dir paths expected is helpful. For production data, is just one more thing that can get messed up, or blow up. That's not always true, maybe you'll have a really good reason to have links, but I wouldn't start out that way, just because it is possible to do.
What are the best practices and a day-to-day routine to manage files of a command line master?
( Per above, use standardized directory structure for all projects.
Have scripts save any small files to a directory your dept keeps in the /tmp dir, .
i.e /tmp/MyDeptsTmpFile (named to fit your local conventions) )
It depends. If you're talking about data and logfiles, dated fileNames can save you a lot of time. I recommend dateFmts like YYYYMMDD(_HHMMSS) if you need the extra resolution.
Dated logfiles are very handy, when a current process seems like it is taking a long time, you can look at the log file from a week ago and quantify exactly how long this process took, a week, month, 6 months (up to how much space you can afford). LogFiles should also capture all STDERR messages, so you never have to re-run a bombed program just to see what the error message was.
This is Linux/Unix you're using, right? Read the man page for the cp cmd installed on your machine. I recommend using an alias like alias CP='/bin/cp -pi' so you always copy a file with the same permissions and with the original files' time stamp. Then it is easy to use /bin/ls -ltr to see a sorted list of files with the most recent files showing up at the bottom of the list. (No need to scroll back to the top, when you sort by time,reverse). Also the '-i' option will warn you that you are going to overwrite a file, and this has saved me more than a couple of times.
I hope this helps.
P.S. as you appear to be a new user, if you get an answer that helps you please remember to mark it as accepted, and/or give it a + (or -) as a useful answer.
First off; I am not necessarily looking for Delphi code, spit it out any way you want.
I've been searching around (especially here) and found a bit about people looking for ways to compare to directories (inclusive subdirs) though they were using byte-by-byte methods. Second off, I am not looking for a difftool, I am "just" looking for a way to find files which do not match and, just as important, files which are in one directory but not the other and vice versa.
To be more specific: I have one directory (the backup folder) which I constantly update using FindFirstChangeNotification. Though the first time I need to copy all files and I also need to check the backup directory against the original when the applications starts (in case something happened when the application wasn't running or FindFirstChangeNotification didn't catch a file change). To solve this I am thinking of creating a CRC list for the backed up files and then run through the original directory computing the CRC for every file and finally compare the two CRCs. Then somehow look for files which are in one directory and not the other (again; vice versa).
Here's the question: Is this the fastest way? If so, how would one (roughly) get the job done?
You don't necessarily need CRCs for each file, you can just compare the "last modified" date for every file for most normal purposes. It's WAY faster. If you need additional safety, you can also compare the lengths. You get both of these metrics for free with the find functions.
And in your change notification, you should probably add the files to a queue and use a timer object to copy the new queued files every ~30sec or something, so you don't bog down the system with frequent updates/checks.
For additional speed, use the Win32 functions wherever possible, avoid any Delphi find/copy/getfileinfo functions. I'm not familiar with the Delphi framework but for example the C# stuff is WAY WAY WAY slower than the Win32 functions.
Regardless of you "not looking for a difftool", are you opposed to using Cygwin with it's "diff" command for the shell? If you are open to this its quite easy, particularly using diff with the -r "recursive" option.
The following generates the differences between 2 Rails installs on my machine, and greps out not only information about differences between files but also, specifically by grepping for 'Only', finds files in one directory, but not the other:
$ diff -r pgnindex pgnonrails | egrep '^Only|diff'
Only in pgnindex/app/controllers: openings_controller.rb
Only in pgnindex/app/helpers: openings_helper.rb
Only in pgnindex/app/views: openings
diff -r pgnindex/config/environment.rb pgnonrails/config/environment.rb
diff -r pgnindex/config/initializers/session_store.rb pgnonrails/config/initializers/session_store.rb
diff -r pgnindex/log/development.log pgnonrails/log/development.log
Only in pgnindex/test/functional: openings_controller_test.rb
Only in pgnindex/test/unit: helpers
The fastest way to compare one directory on the local machine to a directory on another machine thousands of miles away is exactly as you propose:
generate a CRC/checksum for every file
send the name, path, and CRC/checksum for each file over the internet to the other machine
compare
Perhaps the easiest way to do that is to use rsync with the "--dryrun" or "--list-only" option.
(Or use one of the many applications that use the rsync algorithm,
or compile the rsync algorithm into your application).
cd some_backup_directory
rsync --dryrun myname#remote_host:latest_version_directory .
For speed, the default rsync assumes, as Blindy suggested, that two files with the same name and the same path and the same length and the same modification time are the same.
For extra safety, you can give rsync the "--checksum" option to ignore the length and modification time and force it to compare (the checksum of) the actual contents of the file.
At a minimum, what I'd like is to have "hg merge" open up the conflicting versions in side by side instances of, say, MS Paint (ugh), so I can do the merging manually. Bonus points for tool suggestions other than MS Paint. At the moment I'm looking at Araxis Merge.
Please see the merge tool configuration page in the Mercurial wiki. You can make Mercurial launch a specific merge tool based on the file extension.
The Araxis tool look cool, but I have no experience with it (I use Linux). If that doesn't work for you, I would instead make a small script that will show all three images (the base version and the two conflicting versions) and let you choose the "winner".
You could uuencode or base64-encode the images (you can set up hooks for automatic conversion), so that you don't have to deal with binary files when merging. However, merge usually works on lines and not characters and I don't how effective this is, since the enocding usually don't contain new lines.
The Araxis Merge website states that it will integrate with Mercurial. The product is commercial so if you have a license you might want to hit them up for information on using their product with Mercurial.
If you don't have a license and want to use their product, they'll probably help you get the demo working. Then you can mock the problem using fake repositories and test it all out...
If you are trying to stay way from Commercial, then manually configuration the merge tools on a per file basis is the only way to go. I don't know of any open source or free paint programs that would support merging. As a sub for MS Paint, you could probably configure Paint.net to work with Mercurial (manually).
Here’s a basic way to configure Mercurial to enable the merging of images using MS Paint.
First create a file mspaintmergetool.cmd somewhere on the Windows System PATH (so that Mercurial may find it), containing:
START %WINDIR%\system32\mspaint.exe %1
START %WINDIR%\system32\mspaint.exe %2
START %WINDIR%\system32\mspaint.exe %3
START /WAIT %WINDIR%\system32\mspaint.exe %4
Then configure Mercurial to merge images using this script by adding the following to your .hgrc or mercurial.ini:
[merge-tools]
mspaintmergetool.args = $base $local $other $output
mspaintmergetool.binary = True
[merge-patterns]
**.bmp = mspaintmergetool
**.png = mspaintmergetool
**.jpg = mspaintmergetool
If during a merge an image named cat.png conflicts, four MS Paint instances will open up:
cat.png~base - this is what the image looked like just before development split into the two streams you are now trying to merge back together
cat.png~orig - this is what the image looks like in the branch you are merging into (a.k.a. local); it is the image you started out with in your working directory before starting this merge
cat.png~other - this is what the image looks like in the branch you are merging from (a.k.a. remote)
cat.png - when this Paint instance opens, it initially contains the same image as ~orig; your task is to edit the image in this instance into a combination of ~orig and ~other, save, and exit this Paint instance. Upon exit Mercurial will mark the conflict as resolved and resume with the merge. Note that the first three instances will not close automatically, so to avoid confusion with potential remaining conflicting images, you might want to close those Paint instances manually before closing this one.