I've been using grepWin,
And I would like to somehow perform a series of queries for pdf links within .html files.
Thus far with the tool I have been using I just input each individual PDF name and copy the file paths of each reference.
This works fine but I have several hundred specific PDFs I need to find the references for,
And I was wondering if this was possible by using Cygwin or some other cmdline like Findstr to pipe a textfile of links to PDF's which I am searching.
I will give an example:
Spring-Summer.pdf
I would copy all of the paths to which the listed file is linked to within html files.
I then need that copied next to it, or in its own column within csv.
I'm not sure if it's at all probable anyone has asked this before. Currently I'm filling out a spreadsheet of links to these files for a website..
In Linux the following command will find all the html files which contain the specified string:
grep -Rl "Spring-Summer.pdf" <some root folder>
The -R option is to search recursively, and -l is to display just the file name without content.
The same should work on Cygwin.
Related
I downloaded a big folder in Google Drive that was split into 5 parts:
Myfolder-20200911T192019Z-001.zip
Myfolder-20200911T192019Z-002.zip
Myfolder-20200911T192019Z-003.zip
Myfolder-20200911T192019Z-004.zip
Myfolder-20200911T192019Z-005.zip
I'm having some trouble to extract it into the single folder it originally is. Is there a straighforward way to unzip all of them together and recreate the original folder? Maybe some specific command in gzip? I didn't wish to install any program just to perform this task.
The above answer didn't work for me.
The files needed to be unzipped sequentially rather than just concatenated together.
For anyone coming here from a general search about combining zip files but specifically to combine multipart zips from Google Drive, I found this answer to be the one that worked:
https://superuser.com/questions/1255221/how-to-unzip-multiple-zip-files-into-a-single-directory-structure-e-g-google-d
i.e. for the above (creating an output directory to start with if necessary):
mkdir outputFolder
unzip "Myfolder-20200911T192019Z-00*" -d outputFolder
You can do cat Myfolder-20200911T192019Z* > total.zip to combine your zip files and then run unzip total.zip
I am downloading multiple files using curl. The base URL for all the files is the same like
https://mydata.gov/daily/2017
The data in these directories are further grouped by date and file type. So the first data that I need has this directory
https://mydata.gov/daily/2017/001/17d/Roger001.gz
The second data being
https://mydata.gov/daily/2017/002/17d/Roger002.gz
I need to download up until the data for the last day of 2017 which is
https://mydata.gov/daily/2017/365/17d/Roger365.gz
How can I use curl or any other similar tool to download all the files to a single local folder, preferably adopting the original file names?
use for f in {001..365}; do curl https://mydata.gov/daily/2017/"$f"/17d/Roger"$f".gz -o /your-directory/Roger"$f".gz; done in bash terminal.
replace your-directory with your directory which you want to save files.
Basically I have .bz2.gz.bz2 file which on extraction gives a .bz2.gz file and on again extraction gives .bz2 file. In this .bz2 file, is my txt file which I want to search on using grep command. I have searched for this but I got bzgrep command which will only search in bz2 file and not the corresponding .gz.bz2 file and give me no results.
Is there a command in unix system which will recursively search in a zipped archive for zipped archive and return results only when it finds the txt file inside it?
P.S: the txt file may be deep in the archive to level 10 max. I want the command to recursively find the txt file and search for the required string. And there will be no other than an archive inside the archive until the txt file level.
I'm not sure I fully understand but maybe this will help:
for i in /path/to/each/*.tar.bz2; do
tar -xvjf "$i" -C /path/to/save/in
rm $i
done
extract all `tar.bz2` and save them in directory then remove the `.bz2`
Thnx for sharing your question.
There are a couple of strange things with it though:
It makes no sense to have a .bz2.gz.bz2 file, so have you created this file yourself? If so, I'd advise you to reconsider doing so in that manner.
Also, you mention there is a .bz2 that would apparently contain different archives, but a .bz2 can only contain one single file by design. So if it contains archives it is probably a .tar.bz2 file in which the tar-file holds the actual archives.
In answer to your question, why can't you write a simple shell script that will unpack your .bz2.gz.bz2 into a .bz2.gz and then into a .bz2 file and then execute your bzgrep command on that file?
I do not understand where it is exactly that you seem to get stuck..
I am trying to code a script to automatically process some of our daily ftp files.
I have already coded the files to download from the source ftp using WinSCP and calling it in a .bat file, and would ideally like to call it within the same bat. Scripting Language does not matter, as long as I can run/call it from the original batch.
I need will extract the date from a filename, and unzip the contents into corresponding folders. The source file is delivered automatically daily via FTP, and the filename is:
SOFL_CLAIM_TC201702270720000075.zip
The bolded section is the date that I would like to extract.
The contents of the .zip include two types of content, multiple PDFs and a .dat file.
For the supplied date of 20170227, the pdfs need to get extracted to a folder following the format:
\%root%\FNOIs\2017\02-Feb\02-27-2017
At the same time, the .dat file needs to get extracted to multiple folders following the format:
\%root%\Claim Add\2017 Claim Add\02-2017
\%root2%\vendorFTP\VendorFolder
After extracting, I need to move the source zip to
\%root%\Claim Add\2017 Claim Add\02-2017
What is the best way off accomplishing all of this?
I am assuming it would be the for /f batch command, but I am new to batch coding and cannot figure out how to start it from scratch.
I also have 7zip installed, but do not understand how to use the command-line options.
You have asked for a lot in one question, and not shown any code or demonstrated effort on your part.
For the first part, once you have the filename in a variable:
set FILENAME=SOFL_CLAIM_TC201702270720000075.zip
You can get the date part with:
echo %FILENAME:~13,-14%
The syntax: :13,-14 means "Remove the first 13 letters and the last 14 letters." That should leave you with just the date.
When you integrate that into your script, Show Your Code
I need a way to select several files with check-boxes and drag them all to a batch-file icon. The first step in the script would be to be compressed them into a single zip file before proceeding to the next step. If possible, it would be useful to also end up with each in a separate zip of its own for storage.
I am not sure how to address the for:to commands to allow gathering all selected files into a single zip file in a script. The Windows 'sendto compressed zip' function works perfectly if I select all the files the copy and paste them to the Windows ZF filetype. But I don't know how to access it from within a batch.
Drag and drop multiple files
By default, if you drag a file onto a batch file it is the same as passing the filepath as a parameter. This can be accessed using %%1. Dragging multiple files would have a similar effect and be like calling test.bat file1.txt file2.txt file3.txt. Each "parameter" can be accessed using subsequent variables, %%1, %%2, %%3, etc, up to %%9.
Get more information here.
Zipping files
First you will need a utility that supports command line operations, such as 7-zip. Once you have the appropriate executable in your path, you will need to review the documentation on how to zip files from the command line. Instead of using the paths and filenames, you will use the variables mentioned above.
Here is some pseudo code:
zipfiles output.zip %%1 %%2 %%3 %%4 %%5
Notes
You will not be able to drop more than 9 files.
You may need to confirm that each variable exists. If you try to include %%9 in the zip command but you only dropped 8 files, you may get an error.