1

I want to extract the motion vector from the encoded h264 stream without performing a full decode of the frame.

I'm using the FFmpeg library and am aware of extracting the information using the av_frame_get_side_data. Unfortunately, it does a full decode of the frame before providing the AV_FRAME_DATA_MOTION_VECTORS.

Is it possible to parse the h264 to extract out the motion vectors only, ignoring all the other data.

DavidPostill
  • 153,128
  • 77
  • 353
  • 394
M.Akyuzlu
  • 13
  • 4

2 Answers2

1

[...] what I'm trying to do is extracting the motion vectors to calculate the motion percentage before decoding the package in order to save some CPU usage,[...]

There is an easier (and probably faster) way to calculate motion percentage from undecoded streams than computing motion vectors and post-processing them:

ffprobe -show_frames file.mp4 | grep pkt_size

The size of the P-frames is directly related to the amount of information contained in those vectors.

I hope this helps!

Further reference can be found here

Rabindranath Andujar
  • 11
  • 2
0

FFmpeg supplies the example file extract_mvs.c, which needs compiling to be used.

Once compiled, use this command:

./extract_mvs input.mp4 > output.txt

This extracts the motion vectors into a text file.

A ready-made tools is MV-Tractus.

This tool will give you motion vectors in the form of JSON for every frame.

A more recent tool is mv-extractor, described as :

This tool extracts frames, motion vectors, frame types and timestamps from H.264 and MPEG-4 Part 2 encoded videos.

harrymc
  • 455,459
  • 31
  • 526
  • 924
  • In both repo, they got the motion vector after decoding, in the MV-Tractus for example they used the avcodec_send_packet and avcodec_receive_frame functions My Idea is to got the motion vector from the packet not from the frame – M.Akyuzlu Aug 13 '22 at 13:53
  • If everybody does the same, this might be the way to do it. – harrymc Aug 13 '22 at 14:35
  • I know what it seems like but that was possible in the very old version, in the current way I decode the package and give the frame to the OpenCV and run background subtraction to get the motion percentage after doing some calculations, but what I'm trying to do is extracting the motion vectors to calculate the motion percentage before decoding the package in order to save some CPU usage, I could do that after the decoding but the gain will be bigger if I manage to do it before the decoding and I know that it was possible before – M.Akyuzlu Aug 13 '22 at 14:55
  • You'll need to patch ffmpeg yourself, I'm afraid. If it was possible before, you could try and find that old version. – harrymc Aug 13 '22 at 15:46