How can I display lots of subtitles at arbitrary positions in a video using ffmpeg? - ffmpeg

Question

I am trying to write subtitles and display them at lots of positions throughout a video at arbitrary positions. I have a long list of times and positions to display the text in the video, but I don't know how to encode these times and positions into the subtitles file. A lot of the examples on the web only show how to display a single subtitle, or don't explain the syntax of the subtitles files.
I couldn't find a simple answer to how to do this, so I had to put together a lot of methods, and I'm going to answer my own question.

Answer 1

The main options for doing this are:
Using a GUI/standalone program (#llogan mentioned aegisub)
'drawtext' filter in ffmpeg
subtitles filter with an SRT file
subtitles filter with an SSA file
subtitles filter with an ASS file
I wanted to write my own subtitles in a text file (to make it scriptable), not use a standalone GUI. Some of methods don't allow arbitrary positioning (the subtitle will always be centered at the bottom). Others don't allow loading a long list of subtitles from a file, ie you have to encode them all into the ffmpeg command string, which is unwieldy.
Here's the documentation on the subtitles filter, but it doesn't actually explain the syntax of the accompanying files. In fact I found it very difficult to find clear documentation of ANY of the options for subtitles files. I went with ASS because it seemed to work, and I discovered enough templates on the internet that I was able to figure out the most important features by trial and error.
First, create a text file called desc.ass that looks like this:
[Script Info]
PlayResY: 600
WrapStyle: 1
[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, Alignment, MarginL, MarginV
Style: N,Arial,60,&H00FFFF,1,425,550
Style: NE,Arial,60,&H00FFFF,1,650,480
Style: E,Arial,60,&H00FFFF,1,700,250
Style: SE,Arial,60,&H00FFFF,1,600,50
Style: S,Arial,60,&H00FFFF,1,400,0
Style: SW,Arial,60,&H00FFFF,1,150,50
Style: W,Arial,60,&H00FFFF,1,100,250
Style: NW,Arial,60,&H00FFFF,1,150,500
[Events]
Format: Start, End, Style, Text
Dialogue: 00:00:01.00, 00:00:05.00, E, Text to display with the E style
Dialogue: 00:00:05.00, 00:00:05.50, N, Text to display with the N style
The actual file has many many more "Dialogue" lines after this, but I truncated for readability.
I don't really understand the Script Info section yet. :(
The V4+ Styles section allows you to specify an arbitrary number of "Styles", which define color, position, font, and a lot of other things. Each actual subtitle will use one of these styles.
If you have a small number of positions that each subtitle will be displayed in, I think it's easier to define a Style for each position. Because I wanted to display subtitles at one of 8 positions, I defined 8 styles. But you can override the position of any given subtitle (see below). So another option is to use a single Style for everything, and just define the position of each subtitle as necessary.
PrimaryColour is a hex color argument (although it seems to be backwards for some reason, so yellow gives you cyan and vice versa).
Alignment refers to the numerical keypad, so 1 means lower left and 5 means center.
MarginL and MarginV allow you to apply horizontal and vertical offsets to the default alignment.
You can specify lots more attributes here just by adding elements to the Format line, and then specifying them on each Style line. This page is the most comprehensive documentation I could find.
Everybody on the internet copy/pastes the same example text with lots of formats specified (e.g., here). This is another, simpler example. Would be great if someone could post clear documentation on what each of these attributes mean!
The [Events] section defines the individual subtitles. Here, I'm only specifying the basics: start time, end time, style name, and the text.
You can override the position of any given Dialogue element by adding the string {\pos(400,570)} to the beginning of the Text element on that line. (Obviously replace 400, 570 with your desired position.)
You can preview the results of applying these subtitles to a video named video.mkv with ffplay like this:
ffplay -i video.mkv -vf "subtitles=desc.ass"
Some people call this soft-subbing because the video itself isn't being changed, the subtitles are just displayed over it.
If you want to burn the subtitles permanently into the video (sometimes called hard-subbing), you can re-encode like this:
ffmpeg -i video.mkv -vf "subtitles=desc.ass" video_with_subtitles.mkv
Ref for burning subtitles