<p>It turns out it is impossible to live stream with the RPi hardware acceleration using ffmpeg.
For ffmpeg to configure the hardware acceleration to issue SPS/PPS frames during the stream and not only at the beginning of it, it needs to use a <a href="https://github.com/raspberrypi/firmware/issues/242" rel="nofollow noreferrer">specific API of the raspberry firmware</a> which it does not at the moment. </p>

<p><a href="http://frozen.ca/streaming-raw-h-264-from-a-raspberry-pi/" rel="nofollow noreferrer">Raspivid</a> and <a href="http://gstreamer-devel.966125.n4.nabble.com/omxh264enc-and-stream-format-string-avc-td4671543.html" rel="nofollow noreferrer">gstreamer</a> does that though. So turn to thoses solution and forget about ffmpeg.</p>

CC BY-SA 3.0
2017-04-21T17:45:43.920Z
If this is the issue, then you may be able to stream the omx encode to a pipe in mpegts format and then use a 2nd ffmpeg instance to pipe to rtp. The MPEG-TS muxer will insert xPS in-band.
Gyan
CC BY-SA 3.0
2017-04-21T16:01:00.940Z
2017-04-21T16:01:00.940Z
Luke Skywalker
<p>Here my FFmpeg patch to enable inline header on OMX hardware encoder.</p>

<p>libavcodec/omx.c
OMX_SetParameter(s->handle, OMX_IndexParamBrcmVideoAVCInlineHeaderEnable, &amp;vid_param_inline_header);
where vid_param_inline_header is a OMX_CONFIG_PORTBOOLEANTYPE struct.</p>

<p><a href="https://github.com/Serveurperso/FFmpeg/commit/6e410823e63103342e8fc3407ff6698808d4d1ef" rel="nofollow noreferrer">https://github.com/Serveurperso/FFmpeg/commit/6e410823e63103342e8fc3407ff6698808d4d1ef</a></p>

CC BY-SA 4.0
2019-12-10T18:35:07.633Z
Consider submitting a patch to ffmpeg-devel but make it an option instead of hardcoded.
Gyan
CC BY-SA 4.0
2019-12-10T21:26:46.533Z
Yes but there is a last problem:( only when I enable OMX InlineHeader the latency grow over the time.There is a strange time calculation problem inside the muxer. I get a lot of "application provided invalid, non monotonically increasing dts to muxer in stream xxxx/0" and it always freeze at 3335.
Serveurperso
CC BY-SA 4.0
2019-12-10T18:10:56.190Z
2019-12-10T18:10:56.190Z
Serveurperso
<p>Trying to stream a webcam through rtp with a raspberry pi.</p>

<p>Cating the SDP file on my desktop, and launching ffplay:</p>

<pre><code>ffplay sdp.out
</code></pre>

<p>then streaming out this way:</p>

<pre><code>ffmpeg -i /dev/video0 -c:v h264_omx -f rtp rtp://192.168.0.101:1234
</code></pre>

<p>works perfectly. But If I launch ffmpeg first THEN ffplay, ffplay stutters some error message about missing PPS and never displays any frame:</p>

<pre><code>[h264 @ 0x7fd97c0008c0] decode_slice_header error
[h264 @ 0x7fd97c0008c0] no frame!
[h264 @ 0x7fd97c0008c0] non-existing PPS 0 referenced
</code></pre>

<p>After looking for a while I understand some intermediary frame might be missing. However, using the <code>-g</code> option this way does not change anything:</p>

<pre><code>ffmpeg -i /dev/video0 -c:v h264_omx -g 30 -f rtp rtp://192.168.0.101:1234
</code></pre>

<p>It is a problem for me because I am using the raspberry as a securiy camera. My desktop can be restarted from time to time and I need ffplay to be able to catch after the streaming is started.</p>

<p>Any help is appreciated.</p>

CC BY-SA 3.0
2017-03-03T08:57:56.383Z
Rather than a intermediary frame being missing, it's about head frames. When ffplay starts, if the first frame it receives is not a keyframe, then that frame needs a previous (missing) frame to get decoded. That previous frame may be a I-frame not queued to be ever shown or a P frame referred to by a B-frame and would have been shown after the B-frame had it been received.
Gyan
CC BY-SA 3.0
2017-03-03T09:04:52.357Z
I'm confused. Does it mean there is no solution to this problem (it's the way rdp works, the listener has to be started and then the flow started) or there would be a way to regularly insert  a keyframe for the listener to be able to catch the stream in the middle.
Luke Skywalker
CC BY-SA 3.0
2017-03-03T09:08:57.920Z
Does the stream not become smooth after a few seconds?
Gyan
CC BY-SA 3.0
2017-03-03T09:09:54.287Z
No, ffplay keeps outputing error message for ever. It is never able to display any frame.
Luke Skywalker
CC BY-SA 3.0
2017-03-03T09:35:49.060Z
Does replacing `-g 30` with `-force_key_frames "expr:gte(t,n_forced)"` work?
Gyan
CC BY-SA 3.0
2017-03-03T10:00:39.363Z
Indeed, thanks to your suggestion I found another interesting question about this: http://superuser.com/questions/908280/what-is-the-correct-way-to-fix-keyframes-in-ffmpeg-for-dash. So I'll try `force_key_frames` and also check the result with the forensic method mentioned in the question and let you know.
Luke Skywalker
CC BY-SA 3.0
2017-03-03T18:00:26.113Z
I tried your solution to no avail. ffplay is still not able to display anything if not started first. If I dump the output into a file, I see that intermediate frames are generated:`./ffmpeg -i /dev/video0 -c:v h264_omx -force_key_frames 'expr:gte(t,n_forced)' -o file.avi`
and then `./ffprobe -i file.avi -select_streams v -show_frames -of csv -show_entries frame=pict_type`. O see some `frame,I` in the middle of the `frame,P`. So why would ffplay not able to decode those frame? Is this a limitation of the rdp protocol somehow?
Luke Skywalker
<ffmpeg><streaming>
h264_omx and non-existing PPS 0 referenced
Luke Skywalker
# h264_omx and non-existing PPS 0 referenced

(Asked by: username_0 on 2017-03-03)

Trying to stream a webcam through rtp with a raspberry pi.
Cating the SDP file on my desktop, and launching ffplay:
ffplay sdp.out

then streaming out this way:
ffmpeg -i /dev/video0 -c:v h264_omx -f rtp rtp://192.168.0.101:1234

works perfectly. But If I launch ffmpeg first THEN ffplay, ffplay stutters some error message about missing PPS and never displays any frame:
[h264 @ 0x7fd97c0008c0] decode_slice_header error
[h264 @ 0x7fd97c0008c0] no frame!
[h264 @ 0x7fd97c0008c0] non-existing PPS 0 referenced

After looking for a while I understand some intermediary frame might be missing. However, using the -g option this way does not change anything:
ffmpeg -i /dev/video0 -c:v h264_omx -g 30 -f rtp rtp://192.168.0.101:1234

It is a problem for me because I am using the raspberry as a securiy camera. My desktop can be restarted from time to time and I need ffplay to be able to catch after the streaming is started.
Any help is appreciated.

**Tags:** ffmpeg, streaming

> **Comments:**
> **username_1**: Rather than a intermediary frame being missing, it's about head frames. When ffplay starts, if the first frame it receives is not a keyframe, then that frame needs a previous (missing) frame to get decoded. That previous frame may be a I-frame not queued to be ever shown or a P frame referred to by a B-frame and would have been shown after the B-frame had it been received.  
> **username_0**: I'm confused. Does it mean there is no solution to this problem (it's the way rdp works, the listener has to be started and then the flow started) or there would be a way to regularly insert  a keyframe for the listener to be able to catch the stream in the middle.  
> **username_1**: Does the stream not become smooth after a few seconds?  
> **username_0**: No, ffplay keeps outputing error message for ever. It is never able to display any frame.  
> **username_1**: Does replacing `-g 30` with `-force_key_frames "expr:gte(t,n_forced)"` work?  
> **username_0**: Indeed, thanks to your suggestion I found another interesting question about this: http://superuser.com/questions/908280/what-is-the-correct-way-to-fix-keyframes-in-ffmpeg-for-dash. So I'll try `force_key_frames` and also check the result with the forensic method mentioned in the question and let you know.  
> **username_0**: I tried your solution to no avail. ffplay is still not able to display anything if not started first. If I dump the output into a file, I see that intermediate frames are generated:`./ffmpeg -i /dev/video0 -c:v h264_omx -force_key_frames 'expr:gte(t,n_forced)' -o file.avi` and then `./ffprobe -i file.avi -select_streams v -show_frames -of csv -show_entries frame=pict_type`. O see some `frame,I` in the middle of the `frame,P`. So why would ffplay not able to decode those frame? Is this a limitation of the rdp protocol somehow?  

---

## 2 answers

### Answer by username_0 on 2017-04-21. Score: 2 | Accepted answer

It turns out it is impossible to live stream with the RPi hardware acceleration using ffmpeg.
For ffmpeg to configure the hardware acceleration to issue SPS/PPS frames during the stream and not only at the beginning of it, it needs to use a [specific API of the raspberry firmware](https://github.com/raspberrypi/firmware/issues/242) which it does not at the moment.

[Raspivid](http://frozen.ca/streaming-raw-h-264-from-a-raspberry-pi/) and [gstreamer](http://gstreamer-devel.966125.n4.nabble.com/omxh264enc-and-stream-format-string-avc-td4671543.html) does that though. So turn to thoses solution and forget about ffmpeg.

> **Comments:**
> **username_1**: If this is the issue, then you may be able to stream the omx encode to a pipe in mpegts format and then use a 2nd ffmpeg instance to pipe to rtp. The MPEG-TS muxer will insert xPS in-band.  

---

### Answer by username_2 on 2019-12-10. Score: 0

Here my FFmpeg patch to enable inline header on OMX hardware encoder.

libavcodec/omx.c
OMX\_SetParameter(s->handle, OMX\_IndexParamBrcmVideoAVCInlineHeaderEnable, &vid\_param\_inline\_header);
where vid\_param\_inline\_header is a OMX\_CONFIG\_PORTBOOLEANTYPE struct.

<https://github.com/username_2/FFmpeg/commit/6e410823e63103342e8fc3407ff6698808d4d1ef>

> **Comments:**
> **username_1**: Consider submitting a patch to ffmpeg-devel but make it an option instead of hardcoded.  
> **username_2**: Yes but there is a last problem:( only when I enable OMX InlineHeader the latency grow over the time.There is a strange time calculation problem inside the muxer. I get a lot of "application provided invalid, non monotonically increasing dts to muxer in stream xxxx/0" and it always freeze at 3335.  

---

