What makes a MP4 streamable?

Question

I'm digging into the Telegram bot API, and it show this option for sendVideo:

supports_streaming Boolean Optional

Pass True if the uploaded video is suitable for streaming

This suggests that some MPEG-4 video files are suitable for streaming, and some are not. What makes the difference? How can I make sure my video file is 'suitable for streaming'?

There's also [HLS](https://en.wikipedia.org/wiki/HTTP_Live_Streaming) and [MPEG-DASH](https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP) too: they both serve MPEG-4 containers but instead of using a long stream, it's sends lots of _ᴛɪɴʏ_ self-contained MPEG-4 files (each between 200ms to 6s in length) to the client. — Dai, Nov 17 '22 at 23:27
I suggest that wholly misses the point, even while it's a reasonable Question. How could any file, in or of itself, not be streamable? What is any file but a stream of data? There might be umpty questions about platforms or protocols and those are on a wholly different level. What makes a MP4 streamable depends not on the file but on the platforms and protocols and operating systems trying to stream it. Here, which platforms or protocols or OS are involved? — Robbie Goodwin, Nov 19 '22 at 19:52

score 36 · Accepted Answer · answered Nov 17 '22 at 10:55

As far as I know, MP4 container files may have their metadata (audio/video tracks, codec information) either at the beginning of the file before the actual data, or at the end. If the metadata is placed at the end, a player can't decode the video stream until it has downloaded the entire thing (unless it can seek through the file, which is e.g. possible using HTTP range requests).

For example, this file has mdat before moov, so it's not streamable as-is:

$ atomicparsley foo.mp4 -T
Atom ftyp @ 0 of size: 32, ends @ 32
Atom free @ 32 of size: 8, ends @ 40
Atom mdat @ 40 of size: 3280091, ends @ 3280131
Atom moov @ 3280131 of size: 139261, ends @ 3419392
     Atom mvhd @ 3280139 of size: 108, ends @ 3280247
     Atom trak @ 3280247 of size: 57400, ends @ 3337647
         Atom tkhd @ 3280255 of size: 92, ends @ 3280347
     ...
     Atom trak @ 3337647 of size: 81158, ends @ 3418805
         Atom tkhd @ 3337655 of size: 92, ends @ 3337747
     ...

See e.g. the FFMPEG "faststart" article.

Additionally (as I just found out), the audio track data can be either interleaved with video data, or not. If it's not interleaved for some reason, the player again needs to wait for the entire audio stream to be downloaded before it starts receiving video data (again unless it can seek back/forward).

See also Fragmentation, segmentation, splitting and interleaving.

And the reason the moov atom is conventionally placed at the end is the inverse of the streamed playback problem: where playback requires knowing the index metadata before the video, generating the index metadata requires having the video encoded/available. You can't really pre-generate the moov atom so making a video streamable requires moving the atom after the video is already encoded, whether by editing the output or by buffering the entire video during the encode. You can't write a streamable video directly to an output stream in one go. — Bob, Nov 17 '22 at 22:49
@Maarten: Programs that write MP4 files may leave room near the start for a `moov` atom to fill in later, so they don't have to copy the whole file (remux) to wrap it with different metadata. I think that's possible since MP4 allows padding, so you don't have to know the exact size the `moov` atom will be. Or you just didn't notice the copying when it's done writing the file the first time. — Peter Cordes, Nov 18 '22 at 12:41
@Bob: Note that other formats like MKV can write in order. If you play `ffmpeg` output while it's in the middle of encoding, it works, but there's no index so the player has to scan forward in the file. I think it puts the seek index at the end or something, or it leaves space for it, but even without that there's enough info to find and demux the streams (audio and video). That's one reason I almost always encode to MKV, not MP$, so I can see if I like the quality-per-bitrate tradeoff before it finishes. — Peter Cordes, Nov 19 '22 at 23:22

What makes a MP4 streamable?

1 Answers1