I am writing a script which I would like to be able to determine the set of rendered files that changes to the Jekyll source would affect. For example, given the path to a .md file in my input to Jekyll, I'd like to find the .html file in the build output that the markdown was rendered to. Similarly, if I am looking at a file that is included as a template in other files (or a layout), I would like to take into account the fact that multiple other files use the content from the single input file and have my script return all of those files. How can I do this?
I am aware that I could diff the Jekyll output between two different builds, but I would rather not run an extra build if I don't have to. I expect that this will involve a Ruby script that calls into Jekyll.
My goal is to be able to diff a range of commits in git and find the output files that were changed, but the fact that the list of files is coming from git shouldn't matter; I can easily extract the list from a git diff.
Related
I am downloading multiple files using curl. The base URL for all the files is the same like
https://mydata.gov/daily/2017
The data in these directories are further grouped by date and file type. So the first data that I need has this directory
https://mydata.gov/daily/2017/001/17d/Roger001.gz
The second data being
https://mydata.gov/daily/2017/002/17d/Roger002.gz
I need to download up until the data for the last day of 2017 which is
https://mydata.gov/daily/2017/365/17d/Roger365.gz
How can I use curl or any other similar tool to download all the files to a single local folder, preferably adopting the original file names?
use for f in {001..365}; do curl https://mydata.gov/daily/2017/"$f"/17d/Roger"$f".gz -o /your-directory/Roger"$f".gz; done in bash terminal.
replace your-directory with your directory which you want to save files.
I am trying to code a script to automatically process some of our daily ftp files.
I have already coded the files to download from the source ftp using WinSCP and calling it in a .bat file, and would ideally like to call it within the same bat. Scripting Language does not matter, as long as I can run/call it from the original batch.
I need will extract the date from a filename, and unzip the contents into corresponding folders. The source file is delivered automatically daily via FTP, and the filename is:
SOFL_CLAIM_TC201702270720000075.zip
The bolded section is the date that I would like to extract.
The contents of the .zip include two types of content, multiple PDFs and a .dat file.
For the supplied date of 20170227, the pdfs need to get extracted to a folder following the format:
\%root%\FNOIs\2017\02-Feb\02-27-2017
At the same time, the .dat file needs to get extracted to multiple folders following the format:
\%root%\Claim Add\2017 Claim Add\02-2017
\%root2%\vendorFTP\VendorFolder
After extracting, I need to move the source zip to
\%root%\Claim Add\2017 Claim Add\02-2017
What is the best way off accomplishing all of this?
I am assuming it would be the for /f batch command, but I am new to batch coding and cannot figure out how to start it from scratch.
I also have 7zip installed, but do not understand how to use the command-line options.
You have asked for a lot in one question, and not shown any code or demonstrated effort on your part.
For the first part, once you have the filename in a variable:
set FILENAME=SOFL_CLAIM_TC201702270720000075.zip
You can get the date part with:
echo %FILENAME:~13,-14%
The syntax: :13,-14 means "Remove the first 13 letters and the last 14 letters." That should leave you with just the date.
When you integrate that into your script, Show Your Code
I've been using grepWin,
And I would like to somehow perform a series of queries for pdf links within .html files.
Thus far with the tool I have been using I just input each individual PDF name and copy the file paths of each reference.
This works fine but I have several hundred specific PDFs I need to find the references for,
And I was wondering if this was possible by using Cygwin or some other cmdline like Findstr to pipe a textfile of links to PDF's which I am searching.
I will give an example:
Spring-Summer.pdf
I would copy all of the paths to which the listed file is linked to within html files.
I then need that copied next to it, or in its own column within csv.
I'm not sure if it's at all probable anyone has asked this before. Currently I'm filling out a spreadsheet of links to these files for a website..
In Linux the following command will find all the html files which contain the specified string:
grep -Rl "Spring-Summer.pdf" <some root folder>
The -R option is to search recursively, and -l is to display just the file name without content.
The same should work on Cygwin.
I have a MacVim setup on my OSX machine so that the default Vim application (ie: /usr/bin/vim) is actually a symbolic link to the command-line version of Vim that ships with MacVim (ie: /Applications/MacVim.app/MacOS/vim), as it provides some key benefits over the stock Vim that ships with OSX10.6.
I periodically need to prepare a diff between a set of files, and export it into a colorful side-by-side-view HTML file. This is commonly achieved via:
vim -d file1 file2
(Within Vim): toHTML
The problem with this is that I have to manually check out the HEAD revision and a specific revision of the two sets of files, and do this operation for each pair of files. This is very time consuming.
Is it possible to have the results of svn diff command piped into Vim so I can have a colorful side-by-side-view diff for an entire directory (ie: the PWD), as opposed to just the unified diff view?
I have found several Vim scripts and bash scripts that attempt to achieve this, but there are two key problems:
I wish to explicitly call vim -d as the diff tool, and not vimdiff, as the MacVim application does not appear to ship with vimdiff, so I would be using the wrong version of Vim when launching the application
I wish to have multi-file diffs generated against an entire directory recursively, as opposed to just one or two files at a time.
If this is not feasible, I could likely create a bash script that more-or-less achieves this, but I'd like to avoid putting together a hacked/unreliable script if there is a more effective means of doing this.
Thank you.
vimdiff is merely an alternate name for vim. The binary checks to see how it's launched and determines its behavior accordingly. (That's why there isn't a different file for gvim and vim also.)
On my Mac OS and Linux machines I created a ~/bin directory, and then inside it created soft-links from the various names to my macvim binary. I put ~/bin very early in my path.
I'm not at work where I could check to make sure, but I think you can change the default SVN diff to point to vimdiff in ~/.subversion/config. Look for the diff-cmd section. https://stackoverflow.com/a/9604604/128421 might give you some useful information too.
You can try my aurum plugin, it ships with :AuVimDiff command which is as well capable of viewing all changes in a multiple tabs with vimdiff split:
AuVimDiff full HEAD 300
and get diffs between two revisions without you having to checkout them manually:
AuVimDiff file file1 HEAD 300
(file file1 part is optional, it will open diffsplit with current file if omitted. All revisions are also optional, for subversion specifying one revision is diffing it with file in the working directory and specifying no revisions is like svn diff: between BASE and working directory.)
300 here is just an example revision.
I have since resolved this fully in another, more recent post of mine. Posting a backtrack URL to it for future readers:
VIM - Passing colon-commands list via command-line
I have a project, where I'm forced to use ftp as a means of deploying the files to the live server.
I'm developing on linux, so I hacked together a bash script that makes a backup of the ftp server's contents,
deletes all the files on the ftp, and uploads all the fresh files from the mercurial repository.
(and taking care of user uploaded files and folders, and making post-deploy changes, etc)
It's working well, but the project is starting to get big enough to make the deployment process too long.
I'd like to modify the script to look up which files have changed, and only deploy the modified files. (the backup is fine atm as it is)
I'm using mercurial as a VCS, so my idea is to somehow request the changed files between two revisions from it, iterate over the changed files,
and upload each modified file, and delete each removed file.
I can use hg log -vr rev1:rev2, and from the output, I can carve out the changed files with grep/sed/etc.
Two problems:
I have heard the horror stories that parsing the output of ls leads to insanity, so my guess is that the same applies to here,
if I try to parse the output of hg log, the variables will undergo word-splitting, and all kinds of transformations.
hg log doesn't tell me a file is modified/added/deleted. Differentiating between modified and deleted files would be the least.
So, what would be the correct way to do this? I'm using yafc as an ftp client, in case it's needed, but willing to switch.
You could use a custom style that does the parsing for you.
hg log --rev rev1:rev2 --style mystyle
Then pipe it to sort -u to get a unique list of files. The file "mystyle" would look like this:
changeset = '{file_mods}{file_adds}\n'
file_mod = '{file_mod}\n'
file_add = '{file_add}\n'
The mods and adds templates are files modified or added. There is a similar file_dels and file_del template for deleted files.
Alternatively, you could use hg status -ma --rev rev1-1:rev2 which adds an M or an A before modified/added files. You need to pass a different revision range, one less than rev1, as it is the status since that "baseline". Deleted files are similar - you need the -d flag and a D is added before each deleted file.