Using ffmpeg to split video files by size [closed] - bash

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 2 years ago.
The community reviewed whether to reopen this question 16 days ago and left it closed:
Original close reason(s) were not resolved
Improve this question
I'm trying to write a batch file using ffmpeg to automate the redundant daily task of taking footage from work that's recorded in 4gb blocks (which is standard in most DSLR cameras & GoPro's), and split the clips into 2gb files for streaming purposes. The idea is to have the script check external drive FOOTAGE's folder #import and split files after 2gb (since the max size is 4gb, this will alleviate the need for more than one split).
I'm also trying to amend the filenames of the split files, so FILE1 is 4gb, it splits into FILE1_1 and FILE1_2 which are 2gb each, respectively. Everything I've tried has just copied the original file into two new, identical files - no split or anything.
After doing some Googling and reading some of the answers here, I found this post, but it's based on duration, not size (recording video footage at varying levels of quality makes this pointless): Split into equal parts and convert many mp4 videos using ffmpeg
Can someone help me with this? I haven't come across any usable solutions utilizing what I understand to be the method, using -fs limit_size, and I really want to understand how this works.
UPDATE: Also found this, but it hasn't been updated in four years and I don't see anything in there regarding splitting that will prove helpful:
https://github.com/kcm1700/VideoSplitter/blob/master/

I just had the same problem, and after I didn't find a solution here, I wrote a quick bash script for that job.
Features:
Use ffmpeg's -fs-flag to limit filesize
Check length of resulting video part and calculate where to start next part
Enumerate video parts
proceed with as many parts as needed to contain the whole source file.
Allow for custom ffmpeg arguments to also reduce resolution and quality in one pass.
It takes only three arguments: The source filename, the desired filesize of each part and the desired arguments for ffmpeg.
Example call to split a file into 64MB parts:
./split-video.sh huge-video.mov 64000000 "-c:v libx264 -crf 23 -c:a copy -vf scale=960:-1"
Finally, the source code of the script:
#!/bin/bash
# Short script to split videos by filesize using ffmpeg by LukeLR
if [ $# -ne 3 ]; then
echo 'Illegal number of parameters. Needs 3 parameters:'
echo 'Usage:'
echo './split-video.sh FILE SIZELIMIT "FFMPEG_ARGS'
echo
echo 'Parameters:'
echo ' - FILE: Name of the video file to split'
echo ' - SIZELIMIT: Maximum file size of each part (in bytes)'
echo ' - FFMPEG_ARGS: Additional arguments to pass to each ffmpeg-call'
echo ' (video format and quality options etc.)'
exit 1
fi
FILE="$1"
SIZELIMIT="$2"
FFMPEG_ARGS="$3"
# Duration of the source video
DURATION=$(ffprobe -i "$FILE" -show_entries format=duration -v quiet -of default=noprint_wrappers=1:nokey=1|cut -d. -f1)
# Duration that has been encoded so far
CUR_DURATION=0
# Filename of the source video (without extension)
BASENAME="${FILE%.*}"
# Extension for the video parts
#EXTENSION="${FILE##*.}"
EXTENSION="mp4"
# Number of the current video part
i=1
# Filename of the next video part
NEXTFILENAME="$BASENAME-$i.$EXTENSION"
echo "Duration of source video: $DURATION"
# Until the duration of all partial videos has reached the duration of the source video
while [[ $CUR_DURATION -lt $DURATION ]]; do
# Encode next part
echo ffmpeg -i "$FILE" -ss "$CUR_DURATION" -fs "$SIZELIMIT" $FFMPEG_ARGS "$NEXTFILENAME"
ffmpeg -ss "$CUR_DURATION" -i "$FILE" -fs "$SIZELIMIT" $FFMPEG_ARGS "$NEXTFILENAME"
# Duration of the new part
NEW_DURATION=$(ffprobe -i "$NEXTFILENAME" -show_entries format=duration -v quiet -of default=noprint_wrappers=1:nokey=1|cut -d. -f1)
# Total duration encoded so far
CUR_DURATION=$((CUR_DURATION + NEW_DURATION))
i=$((i + 1))
echo "Duration of $NEXTFILENAME: $NEW_DURATION"
echo "Part No. $i starts at $CUR_DURATION"
NEXTFILENAME="$BASENAME-$i.$EXTENSION"
done
Hope, that helps :)

You can do this in one command using mp4box.
mp4box -splits 2000000 filename.mp4
where splits argument takes value in kilobytes. Due to presence of keyframes, segments won't be exactly 2GB.

One idea would be to use ffmpeg with the option -fs. This wil limit the file size. The size of the output file is slightly more than the requested file size.
This will only create one file. But you can build a loop until the whole file is split.
First create a part of the file, then check how long it is with:
ffprobe -i input.file -show_format -v quiet | sed -n 's/duration=//p'
Then start another file with the offset by using -ss
After that do another round of encoding until the whole file is split. You can also use a stream copy for audio and video

Related

Remove a section from the middle of a video without concat

How do I cut a section out of a video with ffmpeg?
Imagine I have a 60 second mp4 A.
I want to remove all the stuff from 0:15 to 0:45.
The result should be a 30-second mp4, which is composed of the first 15 seconds of A directly followed by the last 15 seconds of A.
How can I do this without using concat?
I know how I could do it by creating two intermediary files and then using ffmpeg to concat them. I don't want to have to perform so much manual work for this (simple?) operation.
I have also seen the trim filder used for removing multiple parts from a video. All the usages I've found show that it seems to be very verbose, and I haven't found an example for a case as simple as I would like (just a single section removed).
Do I have to use trim for this operation? Or are there other less verbose solutions?
The ideal would of course be something at least simple as -ss 0:15 -to 0:45 which removes the ends of a video (-cut 0:15-0:45 for example).
I started from
https://stackoverflow.com/a/54192662/3499840 (currently the only answer to "FFmpeg remove 2 sec from middle of video and concat the parts. Single line solution").
Working from that example, the following works for me:
# In order to keep <start-15s> and <45s-end>, you need to
# keep all the frames which are "not between 15s and 45s":
ffmpeg -i input.mp4 \
-vf "select='not(between(t,15,45))', setpts=N/FRAME_RATE/TB" \
-af "aselect='not(between(t,15,45))', asetpts=N/SR/TB" \
output.mp4
This is a one-line linux command, but I've used the bash line-continuation character ('\') so that I can vertically align the equals-signs as this helps me to understand what is going on.
I had never seen ffmpeg's not and between operators before, but I found their documentation here.
Regarding the usual ffmpeg "copy vs re-encode" dichotomy, I was hoping to be able to use ffmpeg's "copy" "codec" (yeah, I know that it's not really a codec) so that ffmpeg would not re-encode my video, but if I specify "copy", then ffmpeg starts and stops at the nearest keyframes which are not sufficiently close to my desired start and stop points. (I want to remove a piece of video that is approximately 20 seconds long, but my source video only has one keyframe every 45 seconds!). Hence I am obliged to re-encode. See https://trac.ffmpeg.org/wiki/Seeking#Seekingwhiledoingacodeccopy for more info.
The setpts/asetpts filters set the timestamps on each frame to the correct values so that your media player will play each frame at the correct time.
HTH.
If you want to use the copy "codec", consider the following approach:
ffmpeg -i input.mp4 -t "$start_cut_section" -c copy part1.mp4&
ffmpeg -i input.mp4 -ss "$end_cut_section" -c copy part2.mp4&
echo "file 'part1.mp4'" > filelist;
echo "file 'part2.mp4'" >> filelist;
wait;
ffmpeg -f concat -i filelist -c copy output.mp4;
rm filelist;
This creates two files from before and after the cut, then combines them into a new trimmed final video. Obviously, this can be used to create as many cuts as you like. It may seem like a longer approach than the accepted answer, but it likely will execute much faster because of the use of the copy codec.

How can I split an mp4 video with ffmpeg every time the volume is zero?

I need to split a video into many smaller videos.
I have tried PySceneDetect and its 2 scene detection methods don't fit my need.
The idea is to trigger a scene cut/break every time the volume is very low, every time audio level is less than a given parameter. I think overall RMS dB volume level is what I mean.
The purpose is to split an mp4 video into many short videos, each smaller video with short dialog phrases.
So far I have a command to get the overall RMS audio volume level.
ffprobe -f lavfi -i amovie=01x01TheStrongestMan.mp4,astats=metadata=1:reset=1 -show_entries frame=pkt_pts_time:frame_tags=lavfi.astats.Overall.RMS_level,lavfi.astats.1.RMS_level,lavfi.astats.2.RMS_level -of csv=p=0
How can I get only the minimum values for RMS level and its corresponding frame or time?
And then how can I use ffmpeg to split the video in many videos on every frame that corresponds to a minimum RMS?
Thanks.
Use silencedetect audio filter and feed its debugging output to segment output format parameter.
Here is a ready-made script:
#!/bin/bash
IN=$1
OUT=$2
true ${SD_PARAMS:="-55dB:d=0.3"};
true ${MIN_FRAGMENT_DURATION:="20"};
export MIN_FRAGMENT_DURATION
if [ -z "$OUT" ]; then
echo "Usage: split_by_silence.sh input_media.mp4 output_template_%03d.mkv"
echo "Depends on FFmpeg, Bash, Awk, Perl 5. Not tested on Mac or Windows."
echo ""
echo "Environment variables (with their current values):"
echo " SD_PARAMS=$SD_PARAMS Parameters for FFmpeg's silencedetect filter: noise tolerance and minimal silence duration"
echo " MIN_FRAGMENT_DURATION=$MIN_FRAGMENT_DURATION Minimal fragment duration"
exit 1
fi
echo "Determining split points..." >& 2
SPLITS=$(
ffmpeg -nostats -v repeat+info -i "${IN}" -af silencedetect="${SD_PARAMS}" -vn -sn -f s16le -y /dev/null \
|& grep '\[silencedetect.*silence_start:' \
| awk '{print $5}' \
| perl -ne '
our $prev;
INIT { $prev = 0.0; }
chomp;
if (($_ - $prev) >= $ENV{MIN_FRAGMENT_DURATION}) {
print "$_,";
$prev = $_;
}
' \
| sed 's!,$!!'
)
echo "Splitting points are $SPLITS"
ffmpeg -v warning -i "$IN" -c copy -map 0 -f segment -segment_times "$SPLITS" "$OUT"
You specify input file, output file template, silence detection parametres and minimum fragment size, it writes multiple files.
Silence detection parameters may need to be tuned:
SD_PARAMS environment variable contains two parameters: noise tolerance level and minimum silence duration. Default value is -55dB:d=0.3.
Decrease the -55dB to e.g. -70dB if some faint non-silent sounds trigger spitting when they should not. Increase it to e.g. -40dB if it does not split on silence because of there is some noise in it, making it not completely silent.
d=0.3 is a minimum silence duration to be considered as a splitting point. Increase it if only serious (e.g. whole 3 seconds) silence should be considered as real, split-worthy silence.
Another environment variable MIN_FRAGMENT_DURATION defines amount of time silence events are ignored after each split. This sets minimum fragment duration.
The script would fail if no silence is detected at all.
There is a refactored version on Github Gist, but there was a problem with it for one user.

Create m3u8 file from list of ts files

I want to create 'm3u8' file from the list of ts files. How can I do it?
I did search in google & read documentation of ffmpeg but I didn't find anything.
It's not clear which of the following cases you're asking about, so here's a quick answer for both:
If you're starting with a single file that contains your content
This is the most common case. In general, there are three steps to creating a playable HlS stream from source material.
for each desired output level (let’s say Bitrate for simplicity), you need to create a collection of segmented .ts files.
For each output level you need a playlist manifest (m3u8) that contains the list of segment files making up the content.
For the whole stream you need a single master manifest (another m3u8) that lists the playlists.
FFMpeg can do all three of these.
If you're starting with a collection of .ts files
If you really are starting with a collection of .ts files, you could either hand-build an m3u8 file as described in the previous answer, or you could write a script to do it.
In either case, there are some considerations for the .ts files:
If the segment files do not belong to an uninterrupted sequence (as they would if they were transcoded from a single source clip for use in HLS), you’ll need to insert EXT-X-DISCONTINUITY tags between segments that don’t have the same encoding characteristics or that don’t have monotonically increasing PTS (presentation timestamp) values.
While the segments don't need to all be the same length, the longest one must not exceed the (integer) number of seconds specified in the EXT-X-TARGETDURATION tag.
"For VOD content, the average segment bit rate MUST be within 10% of the AVERAGE-BANDWIDTH attribute"
When you've built your m3u8 file, it helps to run it through a validator to find any problems.This is a lot easier than scratching your head wondering why an HLS stream plays poorly or inconsistently across players/browsers.
mediaStreamValidator on macOS is very good https://developer.apple.com/documentation/http_live_streaming/about_apple_s_http_live_streaming_tools
Also consider the online tool at Theo: http://inspectstream.theoplayer.com/
You probably want a HLS structure. There's a lot of documentation at Apple (IIRC it was invented by Apple and then got adopted widely), e.g. a draft RFC and a page with example streams.
HLS consists of two levels: a master M3U8 which references other M3U8 which in turn reference the .ts files. You can omit the master M3U8 and just provide the "second level".
As a starting point, it may look something like this:
#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:10, no desc
media-000001.ts
#EXTINF:10, no desc
media-000002.ts
#EXTINF:10, no desc
media-000003.ts
The EXT-X-TARGETDURATION specifies how long each .ts file is (they must all be of the same length). It may either be a relative or absolute path.
Can be done with a bash script:
#!/usr/bin/env bash
file="hls.m3u8"
echo "#EXTM3U" > $file
echo "#EXT-X-VERSION:3" >> $file
echo "#EXT-X-MEDIA-SEQUENCE:24" >> $file
echo "#EXT-X-TARGETDURATION:10" >> $file
for i in `find *.ts -type f | sort -g`; do
l=$(ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 $i)
echo "#EXTINF:$l," >> $file
echo "$i" >> $file
done
echo "#EXT-X-ENDLIST" >> $file

FFMPEG Formating all videos in a folder to be the same for concating

Ok so I'm trying to make a programm that takes the mp4 files in a folder and formats them to be the same ratio and framerate to simplify concating.
This is what I have so far.
#!/bin/bash
NUMBER=0
MP=".mp4"
OUT="./Media/Rescaled/"
for filename in ./Media/*.mp4; do
echo "$filename"
NAME=$(basename "$filename" .mp4)
NEWNAME=$NAME$NUMBER$MP
let "NUMBER += 1"
ffmpeg -i "$filename" -y -r 24 -vf scale=640:640 $OUT$NEWNAME
done
I receive 2 errors that I can't really get my head around as of now:
The encoder 'aac' is experimental but experimental codecs are not enabled,
add '-strict -2' if you want to use it.
which I first of all am not sure where to put in and second of all have heard it messes with audio quality and
[mov,mp4,m4a,3gp,3g2,mj2 # 0x19434c0] Format mov,mp4,m4a,3gp,3g2,mj2
detected only with low score of 1, misdetection possible!
[mov,mp4,m4a,3gp,3g2,mj2 # 0x19434c0] moov atom not found
./Media/Whitenoise.mp4: Invalid data found when processing input
I am absolutely a novice at this, so I would appreciate answers with an explanation or pointing me to something that explains it well as I've been having trouble finding good ffmpeg material.
Thanks for reading through this!

Bash: sort find results using part of a filename [duplicate]

This question already has answers here:
Bash and sort files in order
(8 answers)
Closed 7 years ago.
I have 3 webcams set up in a building, uploading still images to a webserver. I'm using ffmpeg to encode the jpgs to mp4 video.
The directories are set up like this:
Cam1/201504
Cam1/201505
Cam2/201504
Cam2/201505
Cam3/201504
Cam3/201505
I'm using the following bash loop/ffmpeg parameters to make one video per camera, per year. This works well so far (well... except that my SSD is rapidly degrading in performance - too many simultaneous read/write operations?):
find Cam2/2013* -name "*.jpg" -print0 | xargs -0 cat | ffmpeg -f image2pipe -framerate 30 -vcodec mjpeg -i - -vcodec libx264 -profile:v baseline -level 3.0 -movflags +faststart -crf 19 -pix_fmt yuv420p -r 30 "Cam2-2013-30fps-19crf.mp4"
The individual files are named like this (confusing ffmpeg's built-in file sequencer):
Cam1_2015052413543201.jpg
Cam1_2015052413544601.jpg
Cam2_2015052413032601.jpg
Cam2_2015052413544901.jpg
I now need to create one video for an entire year across all 3 cameras, ordered by timestamp. To accomplish this, I need to sort the find results by the segment of the filename after the underscore.
What do I pipe the find output to to accomplish this? For example, the files above would be ordered like this:
Cam2_2015052413032601.jpg
Cam1_2015052413543201.jpg
Cam1_2015052413544601.jpg
Cam2_2015052413544901.jpg
Any help is very much appreciated!
sort
sort -t '_' -nk2
-t '_' # specifices that the field seperator should be an underscore
-nk2 # start sorting from the second field (after the underscore)..n sort according to numerical value/timestamp
output
Cam2_2015052413032601.jpg
Cam1_2015052413543201.jpg
Cam1_2015052413544601.jpg
Cam2_2015052413544901.jpg
pipe sort to find command like
sort -t '_' -nk2 --files0-from=-
Use sort with the --key option. See your man page of sort for details of the key format. Generally (for both coreutils and BSD sort) it should be F[.C][OPTS][,F[.C][OPTS]], where F is for field and C is for character position. Here you want to sort from the 5th character of the first field, so --key=1.5 will do:
> echo -e 'Cam1_2015052413543201.jpg\nCam1_2015052413544601.jpg\nCam2_2015052413032601.jpg\nCam2_2015052413544901.jpg' | sort --key=1.5
Cam2_2015052413032601.jpg
Cam1_2015052413543201.jpg
Cam1_2015052413544601.jpg
Cam2_2015052413544901.jpg
Here you seem to have not only basenames in the output of find, but relative paths with path segments like Cam1/201505/ prepended, but you can still count the number of characters and hence write the appropriate keydef. For instance, say the paths for the images in the example above are
Cam1/201505/Cam1_2015052413543201.jpg
Cam1/201505/Cam1_2015052413544601.jpg
Cam2/201505/Cam2_2015052413032601.jpg
Cam2/201505/Cam2_2015052413544901.jpg
Then
sort --key=1.17
will give you the correct order
Cam2/201505/Cam2_2015052413032601.jpg
Cam1/201505/Cam1_2015052413543201.jpg
Cam1/201505/Cam1_2015052413544601.jpg
Cam2/201505/Cam2_2015052413544901.jpg

Resources