I have some -001.mkv files splitted in this way
the rings + of power-001.mkv
the rings + of power-002.mkv
..
nightmare.return-001.mkv
nightmare.return-002.mkv
..
I need to join -001.mkv files to obtain files like this
the rings + of power.mkv
nightmare.return.mkv
I thought of such a code, but doesn't work
for file in "./source/*001.mkv;" \
do \
echo mkvmerge --join \
--clusters-in-meta-seek -o "./joined/$(basename "$file")" "$file"; \
done
source is the source folder where -001.mkv files are located
joined is the target folder
Can you try this?
for f in ./source/*-001.mkv
do
p=${f%-001.mkv}
echo mkvmerge --clusters-in-meta-seek \
-o ./joined/"${p##*/}".mkv "$p"-*.mkv
done
Given your example, the for f in ./source/*-001.mkv should loop though nightmare.return-001.mkv and the rings + of power-001.mkv.
Let's assume that we're in the first iteration and $f expands to nightmare.return-001.mkv
${f%-001.mkv} right-side-strips -001.mkv from $f; that would give ./source/nightmare.return. We store this prefix in the variable p.
${p##*/} left-side-strips all characters up to the last / from $p; that would give nightmare.return.
So ... -o ./joined/"${p##*/}".mkv "$p"-*.mkv would be equivalent to ... -o ./joined/nightmare.return.mkv ./source/nightmare.return-*.mkv
PS: Once you checked that the displayed commands correspond to what you expect then you can remove the echo
Related
There are variables that I need to change and run sequentially within certain codes.For example, the ionex_list variable looks like this and i need to change this in a loop. I have 3 variable files like this and 9 pieces of data in a row (ionex_list,receiver_ids,data_record).
ionex_list data_record
jplg0910.19i.Z ISTA00TUR_R_20190910000_01D_30S_MO.crx.gz
jplg0920.19i.Z ISTA00TUR_R_20190920000_01D_30S_MO.crx.gz
... ...
jplg0980.19i.Z ISTA00TUR_R_20190980000_01D_30S_MO.crx.gz
jplg0990.19i.Z ISTA00TUR_R_20190990000_01D_30S_MO.crx.gz
For example, I need to show ISTA00TUR_R_20190910000_01D_30S_MO.crx.gz as -dataFile in rnxEditGde.py, but I get the error "File: "/y2019/d091" does not exist" and I cannot show the path. these .gz files are a few directories under from the directory I'm working in. Since I couldn't pass the rnxEditGde.py part, I can't see if the codes below work properly.
dir=`pwd`
function pl {
sed -n "$1p" $2
}
for j in {091..099}; do
ids=$(pl $j receiver_ids)
ionex=$(pl $j ionex_list)
dataRecordFile=$(pl $j data_record)
gd2e.py -mkTreeS Trees
sed -i "s/jplg.*/**$ionex**/g" $dir/Trees/ppp_0.tree
rnxEditGde.py -dataFile "/y2019/d${j}/**$dataRecordFile**" -o dataRecordFile.Orig.gz
gd2e.py \
-treeSequenceDir Trees \
-drEditedFile** $dataRecordFile** \
-antexFile goa-var/etc/antennaCalsGNSS/igs14_2038.atx \
-GNSS https://sideshow.jpl.nasa.gov/pub/JPL_GNSS_Products/MultiGNSS_Rapid_Test\
-runType PPP \
-recList **$ids** \
-staDb /goa-var/sta_info/sta_db_qlflinn \
>gd2e.log 2>gd2e.err
done
I would like to create a folder structure based on a brace expansion such as {a-z}. Each string generated by the brace-expansion should be a new folder. Furthermore, each of these folders should contain the same set of subfolders similarly generated. And this up to a given level.
An example for the range a-z and depth 16
a/a/a/a/a/a/a/a/a/a/a/a/a/a/a/a/
a/a/a/a/a/a/a/a/a/a/a/a/a/a/a/b/
a/a/a/a/a/a/a/a/a/a/a/a/a/a/a/c/
...
d/a/h/r/y/d/s/b/e/y/k/f/o/o/q/c/
...
z/z/z/z/z/z/z/z/z/z/z/z/z/z/z/y/
z/z/z/z/z/z/z/z/z/z/z/z/z/z/z/z/
The following code allows me to go upto depth 2:
for x in {a..z} ; do mkdir -p $x/{a..z} ; done
But how do I go further?
A recursive solution. Job is called with 2 params: max_depth and base_path
#!/bin/bash
function job()
{
local depth=$(($1-1))
local path=$2
local x
for x in a b c # reduced for test
do
mkdir -p "$path/$x"
((depth>0)) && job $depth "$path/$x"
done
}
job 3 ./test
Proof it with:
find test -type d
The simplest form would be to use any of the following lines:
mkdir -p {a..c}/{a..c} # depth 2
mkdir -p {a..c}/{a..c}/{a..c} # depth 3
mkdir -p {a..c}/{a..c}/{a..c}/{a..c} # depth 4
...
The brace-expansion will make all combinations and mkdir -p will take care of the rest.
Of course, you do not want to type this over and over for various sets. So you could generate the full brace-expansion-string with bash and use exec to process the brace-expansion-string before passing it to mkdir -p:
depth=3
set={a..c}
dirs=$(printf "/${set}%.0s" $(seq $depth))
mkdir -p $(eval echo .${dirs})
Be aware however that if your set has length m, and you want a depth of n, you are creating m^n directories. This number could conflict with the number of arguments you can pass on to a program.
Related information:
What is the maximum allowed depth of sub-folders?
https://www.in-ulm.de/~mascheck/various/argmax/
A recursive funcion may solve your problem. Take care with inodes generation when using high directory levels...
#!/bin/bash
function createDir {
mkdir -p $1 && cd $1;
for x in {a..z} ; do
local i=$(($2-1))
[ $i -lt 0 ] && continue;
createDir $x $i
done
cd ..
}
createDir $1 $2
Save into a file, like mkdir.sh, and call it: ./mkdir.sh <main_folder> <level>.
I have this code which scans folders and moves all files in each folder to a new one.
How do I make it so only every Nth file is moved?
#!/bin/bash
# Save this file in the directory containing the folders (bb in this case)
# Then to run it, type:
# ./rencp.sh
# The first output frame number
let "frame=1"
# this is where files will go. A new directory will be created if it doesn't exist
outFolder="collected"
# print info every so many files.
feedbackFreq=250
# prefix for new files
namePrefix="ben_timelapse"
#new extension (uppercase is so ugly)
ext="jpg"
# this will make sure we only get files from camera directories
srcPattern="ND850"
mkdir -p $outFolder
for f in *${srcPattern}/*
do
mv $f `printf "$outFolder/$namePrefix.%05d.$ext" $frame`
if ! ((frame % $feedbackFreq)); then
echo "moved and renamed $frame files to $outFolder"
fi
let "frame++"
done
Pretty sure I need to edit the line for f in *${srcPattern}/* but not sure of the correct syntax
If files in the ND850 folders are sequential when listed (i.e. padded frame numbers), and the folders themselves are in order, then the following code should work.
#!/bin/bash
# Maintain a counter, and the output frame number
let "frame=1"
let "outframe=1"
outFolder="collected"
# frequency
gap=5
namePrefix="ben_timelapse"
#new extension (uppercase is so ugly)
ext="jpg"
srcPattern="ND850"
echo "Copying and renaming 1 in every $gap files"
mkdir -p "$outFolder"
for f in *${srcPattern}/*
do
if ! ((frame % $gap)); then
outfile=`printf "$outFolder/$namePrefix.%05d.$ext" $outframe`
cp $f "$outfile"
echo "copied $f to $outfile"
let "outframe++"
fi
let "frame++"
done
Try this instead of your mv command after do:
if ! ((frame % 5)); then
a=$((frame / 5));
mv $f `printf "$outFolder/$namePrefix.%05d.$ext" $a`
fi
It will move frame=5,10, and so on, to $outFolder/$namePrefix.00001.$ext,$outFolder/$namePrefix.00002.$ext, and so on
I have a FTP folder receiving files from a remote camera. The camera stores the video file name always as ./rec_YYYY-MM-DD_HH-MM.mkv. The video files are stored all in the same folder, the root folder from the FTP server.
I need to move these files to another folder, with this new scheme:
Remove rec_ from the file name.
Change date format to DD-MM-YY.
Remove date from the file name and make it a folder instead, where that same file and all the others in the same date will be stored in.
Final file path would be: ./DD-MM-YYYY/HH-MM.mkv.
The process would continue to all the files, putting them in the folder corresponding to the day it was created.
Summing up: ./rec_YYYY-MM-DD_HH-MM.mkv >> ./DD-MM-YYYY/HH-MM.mkv. The same should apply to all files that are in the same folder.
As I can't make it happen directly from the camera, this needs to be done with Bash on the server that is receiving the files.
So far, what I got is script, which would get the file's creation date and use it to make a folder, and then get creation time to move the file with the new name, based on it's creation time.:
for f in *.mp4
do
mkdir "$f" "$(date -r "$f" +"%d-%m-%Y")"
mv -n "$f" "$(date -r "$f" +"%d-%m-%Y/%H-%M-%S").mp4"
done
I'm getting this output (with testfile 1.mp4):
It creates the folder based on the file's creation date;
it renames the file to it's creation time;
Then, it returns mkdir: cannot create directory ‘1.mp4’: File exists
If two or more files, only one gets renamed and moved as described. The others stay the same and terminal returns:
mkdir: cannot create directory ‘1.mp4’: File exists
mkdir: cannot create directory ‘2.mp4’: File exists
mkdir: cannot create directory ‘12-12-2018’: File exists
Could someone help me out? Better suggestions? Thanks!
Honestly I would just use Perl or Python for this. Here's how to embed either in a shell script.
Here's a perl script that doesn't use any libraries, even ones that ship with Perl (so it'll work without extra packages on distributions like CentOS that don't ship with the entire Perl library). The perl script launches one new process per file in order to perform the copy.
perl -e '
while (<"*.m{p4,kv}">) {
my $path = $_;
my ($prefix, $year, $month, $day, $hour, $minute, $ext) =
split /[.-_]/, $path;
my $sec = q[00];
die "unexpected prefix ($prefix) in $path"
unless $prefix eq q[rec];
die "unexpected extension ($ext) in $path"
unless $ext eq q[mp4] or $ext eq q[mkv];
my $dir = "$day-$month-$year";
my $name = "$hour-$min-$sec" . q[.] . $ext;
my $destpath = $dir . q[/] . $name;
die "$dir . $name is unexpectedly a directory" if (-d $dir);
system("cp", "--", $path, $destpath);
}
'
Here's a Python example, it's compatible with either Python 2 or Python 3 but does use the standard library. The Python script does not spawn any additional processes.
python3 -c '
import os.path as path
import re
from glob import iglob
from itertools import chain
from os import mkdir
from shutil import copyfile
for p in chain(iglob("*.mp4"), iglob("*.mkv")):
fields = re.split("[-]|[._]", p)
prefix = fields[0]
year = fields[1]
month = fields[2]
day = fields[3]
hour = fields[4]
minute = fields[5]
ext = fields[6]
sec = "00"
assert prefix == "rec"
assert ext in ["mp4", "mkv"]
directory = "".join([day, "-", month, "-", year])
name = "".join([hour, "-", minute, "-", sec, ".", ext])
destpath = "".join([directory, "/", name])
assert not path.isdir(destpath)
try:
mkdir(directory)
except FileExistsError:
pass
copyfile(src=p, dst=destpath)
'
Finally, here's a bash solution. It splits paths using -, ., and _ and then extracts various subfields by indexing into $# inside a function. The indexing trick is portable, although regex substitution on variables is a bash extension.
#!/bin/bash
# $1 $2 $3 $4 $5 $6 $7 $8
# path rec YY MM DD HH MM ext
process_file() {
mkdir "$5-$4-$3" &> /dev/null
cp -- "$1" "$5-$4-$3"/"$6-$7-00.$8"
}
for path in *.m{p4,kv}; do
[ -e "$path" ] || continue
# NOTE: two slashes are needed in the substitution to replace everything
# read -a ARRAYVAR <<< ... reads the words of a string into an array
IFS=' ' read -a f <<< "${path//[-_.]/ }"
process_file "$path" "${f[#]}"
done
If you cd /to/some/directory/containing_your_files then you could use the following script
#!/usr/bin/env bash
for f in rec_????-??-??_??-??.m{p4,kv} ; do
dir=${f:4:10} # skip 4 chars ('rec_') take 10 chars ('YYYY_MM_DD')
fnm=${f:15} # skip 15 chars, take the remainder
test -d "$dir" || mkdir "$dir"
mv "$f" "$dir"/"$fnm"
done
note ① that I have not exchanged the years and the days, if you absolutely need to do the swap you can extract the year like this, year=${dir::4} etc and ② that this method of parameter substitution is a Bash-ism, e.g., it doesn't work in dash.
your problem is: mkdir creates folder but you are giving filename for folder creation.
if you want to use fileName for folder creation then use it without extension.
the thing is you are trying to create folder with the already existing fileName
Sorry, I'm from Brazil and my english is not fluent.
I wanna concatenate 20 files using a shellscript through cat command. However when I run it from a file, all content of files are showed on the screen.
When I run it directly from terminal, works perfectly.
That's my code above:
#!/usr/bin/ksh
set -x -a
. /PROD/INCLUDE/include.prod
DATE=`date +'%Y%m%d%H%M%S'`
FINAL_NAME=$1
# check if all paremeters are passed
if [ -z $FINAL_NAME ]; then
echo "Please pass the final name as parameter"
exit 1
fi
# concatenate files
cat $DIRFILE/AI6LM760_AI6_CF2_SLOTP01* $DIRFILE/AI6LM761_AI6_CF2_SLOTP02* $DIRFILE/AI6LM763_AI6_CF2_SLOTP04* \
$DIRFILE/AI6LM764_AI6_CF2_SLOTP05* $DIRFILE/AI6LM765_AI6_CF2_SLOTP06* $DIRFILE/AI6LM766_AI6_CF2_SLOTP07* \
$DIRFILE/AI6LM767_AI6_CF2_SLOTP08* $DIRFILE/AI6LM768_AI6_CF2_SLOTP09* $DIRFILE/AI6LM769_AI6_CF2_SLOTP10* \
$DIRFILE/AI6LM770_AI6_CF2_SLOTP11* $DIRFILE/AI6LM771_AI6_CF2_SLOTP12* $DIRFILE/AI6LM772_AI6_CF2_SLOTP13* \
$DIRFILE/AI6LM773_AI6_CF2_SLOTP14* $DIRFILE/AI6LM774_AI6_CF2_SLOTP15* $DIRFILE/AI6LM775_AI6_CF2_SLOTP16* \
$DIRFILE/AI6LM776_AI6_CF2_SLOTP17* $DIRFILE/AI6LM777_AI6_CF2_SLOTP18* $DIRFILE/AI6LM778_AI6_CF2_SLOTP19* \
$DIRFILE/AI6LM779_AI6_CF2_SLOTP20* > CF2_FINAL_TEMP
mv $DIRFILE/CF2_FINAL_TEMP $DIRFILE/$FINAL_NAME
I solved the problem putting the cat block inside a function, and redirecting stdout to the final file.
Ex:
concatenate()