How do I loop through multiple files using shell script? - bash

I have multiple files whose format of filename like: qs_l2b_yearday.nc; for example: 1s_l2b_2008031.nc.
I'm trying to use shell script to download these files from remote server, but unfortunately I cannot find a way to do that. With my limited understanding the code should loop through years as outside loop and loop through days as inside loop. And the core code should use scp to download the file from server. I have figured out how to download one of those files using scp, but need some help to solve the "for loop" problem.
Year is from 1999 to 2009. Day is from 001 to 365


You can use a wildcard to scp multiple files like so
scp user#somehost:/some/path/to/files/qs_l2b_*.nc /some/target/directory/

Related

shell script to scp from one server to another by choice

How to copy files from one server to another server? I have 3 servers from which I have to copy files from one server to another server by choice. I know scp is the command to be used but I just wanted to know how to write a shell script which makes me copy files from one server to another service by choice. Any help is appreciated.

I would write a script that:
Has a static, defined list of three servers.
list_of_things=(one two three)
Takes as input/argument a file/file path on the local machine
first_argument=$0
second_argument=$1
Checks that the file exists
if [ -e first_argument ]
Loops through the list of servers, and uses scp to transfer the file to each server.
for item IN list_of_things; do
// do something here
then
Read through a bash scripting tutorial for more guidance: https://ryanstutorials.net/bash-scripting-tutorial/

How to extract date from filename in batch, and unzip to multiple directories using batch/7z

I am trying to code a script to automatically process some of our daily ftp files.
I have already coded the files to download from the source ftp using WinSCP and calling it in a .bat file, and would ideally like to call it within the same bat. Scripting Language does not matter, as long as I can run/call it from the original batch.
I need will extract the date from a filename, and unzip the contents into corresponding folders. The source file is delivered automatically daily via FTP, and the filename is:
SOFL_CLAIM_TC201702270720000075.zip
The bolded section is the date that I would like to extract.
The contents of the .zip include two types of content, multiple PDFs and a .dat file.
For the supplied date of 20170227, the pdfs need to get extracted to a folder following the format:
\%root%\FNOIs\2017\02-Feb\02-27-2017
At the same time, the .dat file needs to get extracted to multiple folders following the format:
\%root%\Claim Add\2017 Claim Add\02-2017
\%root2%\vendorFTP\VendorFolder
After extracting, I need to move the source zip to
\%root%\Claim Add\2017 Claim Add\02-2017
What is the best way off accomplishing all of this?
I am assuming it would be the for /f batch command, but I am new to batch coding and cannot figure out how to start it from scratch.
I also have 7zip installed, but do not understand how to use the command-line options.

You have asked for a lot in one question, and not shown any code or demonstrated effort on your part.
For the first part, once you have the filename in a variable:
set FILENAME=SOFL_CLAIM_TC201702270720000075.zip
You can get the date part with:
echo %FILENAME:~13,-14%
The syntax: :13,-14 means "Remove the first 13 letters and the last 14 letters." That should leave you with just the date.
When you integrate that into your script, Show Your Code

Loop through files in a FTP using MS DOS

Is there a way to loop through the files in an FTP from MS DOS.? Thanks.

The technique I'm using is to have two FTP scripts. One I use to capture the output of a DIR command. Having captured the DIR, I parse that for the names of files to be downloaded. Then I use the second script with information from the first to download the files.

PVRTexTool, is there a way to run it on multiple files at once?

I am using PVRTexTool to convert png files to pvr files but the tool seems to only be able to run on one file at a time(wont accept *.png as file name).
does anyone know how to run it on a group of files at once?
Its really a hassle to run it on all of my textures.

In a shell, run
for file in *.png ; do
PVRTexToll $file
done
(I don't know how to call PVRTeXTool from a command line, so please substitute the second line with a correct version)
This is a general way to feed each file to a command which only accepts one file at a time. See any introduction on shell scripting, e.g. this discussion of the for loop.

Bash script to upload first RAR part to ftp, before completion of second part (To Save Time and space)

i was making a bash script for my server which pack some directories with RAR and upload it to other ftp server, so some folders are big and i have to rar them in parts and have to wait for all parts to be rared before uploading them, which consumes lots of time and space
so i want to do it more fast like, upload every rared part on its completion and delete it afterwards automatically without waiting for another all parts, i know there are possibilities of getting data corrupted or else, but that's what i want to do and i confused about where to start the script.
Server OS: Ubuntu 9.10
thanks
Kevin

Start rar in the background
Check wheter rar is finished on the part by calling fuser foo.part2.rar.
If fuser does not return anything, you can transfer it to the remote FTP.

Resources