<p>You can have timestamp gaps in an audio stream in Matroska-family or MOV/MP4 format files. That's how Google records and stores WebRTC recordings. There won't be zero signal audio data, rather there will be no packets corresponding to the gap and timestamps of the packets before and after the gap will indicate audio position.</p>
<p>However, it is not the encoder's job or feature in ffmpeg to drop those sections. You will need to use the <a href="http://www.ffmpeg.org/ffmpeg-filters.html#silenceremove" rel="nofollow noreferrer">silenceremove</a> filter with option <code>timestamp=copy</code> (6.1+ or git master &gt; May 2023 needed).</p>

CC BY-SA 4.0
2023-12-09T09:52:37.367Z
Would `silenceremove` produce such output with gaps? Or would it "collapse" the gaps and make the wall-clock-duration shorter? This was my misunderstanding while originally reading its documentation which led my to asking this question.
Vadim Kantorov
CC BY-SA 4.0
2023-12-09T10:45:39.597Z
silenceremove can only remove data; it doesn't consolidate gaps.
Gyan
CC BY-SA 4.0
2023-12-09T10:58:31.363Z
`silenceremove` does not affect any timestamps and they stay original (as if no gaps were introduced), right? My goal is to have the encoded file have the same duration as before encoding (and have silence imputed by zeros at decoding time). Do you know if gaps are supported by `webm`/`mka`? Am I correct that `ogg` does not support gaps? Would you have an advice of how to instruct `ffmpeg` to encode separately `.wav` streams / in a decoupled mode (for phone call recordings, there are long periods of silence if one considers every channel separately)?
Vadim Kantorov
CC BY-SA 4.0
2023-12-14T14:46:56.523Z
I tested the `silenceremove` solution. Unfortunately, it does not create `gaps` and it changes the duration of the audio - this is not what I need. I'd like to simply drop the silent packets from the stream, but preserve the timestamps and keep the audio duration and have the dropped packets be replaced by digital silence at decoding time.
Vadim Kantorov
CC BY-SA 4.0
2023-12-14T14:47:08.800Z
Here is a repro: `ffmpeg -y -f lavfi -i sine=f=440:d=5 -f lavfi -i anullsrc=d=3600:cl=mono -f lavfi -i sine=f=880:d=5 -filter_complex "[0][1][2]concat=n=3:v=0:a=1" 5sec_silent1h_5sec.wav;
ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus 5sec_silent1h_5sec.opus;
ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0 5sec_5sec.mkv`
Vadim Kantorov
CC BY-SA 4.0
2023-12-14T14:48:01.717Z
The good duration of `5sec_silent1h_5sec.opus` and bad duration of `5sec_5sec.mkv` are confirmed by `ffprobe` (1h10s and 10s respectively)
Vadim Kantorov
CC BY-SA 4.0
2023-12-14T16:14:39.537Z
Updated answer. See at the end.
Gyan
CC BY-SA 4.0
2023-12-14T16:19:59.643Z
Thanks for helping out with this :) I've just tried your updated proposal using ffmpeg 6.1 static bits from https://johnvansickle.com/ffmpeg/, the duration turns correct, but it has the same leakage/wrong pts problem as my attempts with `aselect` filter :(
Vadim Kantorov
CC BY-SA 4.0
2023-12-14T17:15:33.260Z
I converted the input `5sec_silent1h_5sec.wav` to be 48khz (in my original code above it was 44.1khz), but it made no difference - still leakage
Vadim Kantorov
CC BY-SA 4.0
2023-12-15T05:42:57.583Z
Probably to do with the attack/release used by the filter. Will have to debug when I have time.
Gyan
CC BY-SA 4.0
2023-12-15T12:00:11.173Z
Thank you! However, this leakage is quite long - a full second :( I hope my repro above will help!
Vadim Kantorov
CC BY-SA 4.0
2023-12-15T14:06:14.563Z
Also, maybe the leakage is related to some silenceremove filter options I used above?

But given the strangeness of such leakage, I suspect that somehow some packets of the 880hz-segment get incorrect timestamps assigned which forces them to be played before the 1h-long silence/gap
Vadim Kantorov
CC BY-SA 4.0
2023-12-09T04:31:38.507Z
2023-12-14T16:14:09.757Z
2023-12-14T16:14:09.757Z
Gyan
<p>Does <code>webm</code>/<code>mka</code>/<code>mkv</code>/<code>mp4</code> support discontinuous audio streams?</p>
<p>I'm wondering, can <code>ffmpeg</code> transcode <code>.wav</code>-audio into <code>opus</code>-encapsulated-into-<code>webm</code>, but filter out silent frames - especially full seconds of silence, completely drop them from the stream to save space? (maybe via <a href="https://ffmpeg.org/ffmpeg-bitstream-filters.html" rel="nofollow noreferrer">https://ffmpeg.org/ffmpeg-bitstream-filters.html</a> ?)</p>
<p>The goal of all this is to very efficiently represent near-silence in phone-call-recording files for archival purposes (i.e. by completely omitting it from the stream and having it replaced by digital zeros at audio decoding time).</p>
<p>As far as I understand <code>opus</code>-encapsulated-into-<code>ogg</code> does not support such timestamp discontinuities. <code>opus</code>-encapsulated-into-<code>rtp</code> assumingly does support this, but I'm wondering if more &quot;offline&quot; formats such as <code>webm</code> and whether it's possible to produce them directly without passing by <code>rtp</code> format first.</p>
<p>These frames could then be replaced by zeros at audio decoding time by some form of PLC or DTS gaps replacement by zero signal.</p>
<p>Thanks!</p>
<hr />
<p><strong>UPD1</strong>: Unfortunately, <code>silenceremove</code> filter does not work as I need it. The way I used it below, the audio duration after <code>silenceremove</code> is shortened by the amount of silence. This is not what I am looking for. I am looking to preserve the timestamps, have the silence imputed by digital silence at decoding time and preserve the original audio duration (as reported by <code>ffprobe</code> or audio players)</p>
<pre class="lang-shell prettyprint-override"><code>ffmpeg -y -f lavfi -i sine=f=440:d=5 -f lavfi -i anullsrc=d=3600:cl=mono -f lavfi -i sine=f=880:d=5 -filter_complex &quot;[0][1][2]concat=n=3:v=0:a=1&quot; 5sec_silent1h_5sec.wav

ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus 5sec_silent1h_5sec.opus

ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0 5sec_5sec.mkv
</code></pre>
<p>The good duration of 5sec_silent1h_5sec.opus and bad duration of 5sec_5sec.mkv are confirmed by ffprobe (1h10s and 10s respectively).</p>
<p>Is there a way to adjust the filters/command line arguments to preserve the audio duration?</p>
<hr />
<p><strong>UPD2</strong>
The following somewhat works, although ~1s of 880hz audio leaks into the first segment (before the long silence segment), no matter how I adjust the <code>between</code> bounds - very strange!</p>
<p><code>ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af &quot;aselect='between(t,0,4)+between(t,3606,3610)'&quot; 5sec_silent1h_5sec_with_leakage.mkv</code></p>
<p>If this leakage can be fixed, then this can be used <a href="https://superuser.com/a/1659916/813257">together</a> <a href="https://stackoverflow.com/questions/74418102/more-precision-from-ffmpeg-silencedetect">with</a> <code>silencedetect</code> filter.</p>
<p>But of course it would be nicer to get the output in a single invokation of ffmpeg.</p>
<p>The example output file with &quot;leakage&quot;: <a href="https://1drv.ms/u/s!Apx8USiTtrYmq9Q4QaETF2OJWdKGQA?e=ACTmMg" rel="nofollow noreferrer">https://1drv.ms/u/s!Apx8USiTtrYmq9Q4QaETF2OJWdKGQA?e=ACTmMg</a></p>
<hr />
<p><strong>UPD3</strong>: On a recent ffmpeg 6.1 adding <code>timestamp=copy</code> filter option also produces a file with a correct duration, but with leakage <code>ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0:timestamp=copy 5sec_1hsilence_5sec_with_leakage.mkv</code>. So it seems that timestamps are still not copied exactly.</p>

<ffmpeg><audio><matroska>
Possible to force FFmpeg to omit silence in audio files and produce discontinuous `webm`/`mka`?
Vadim Kantorov
# Possible to force FFmpeg to omit silence in audio files and produce discontinuous `webm`/`mka`?

(Asked by: username_0 on 2023-12-09)

Does webm/mka/mkv/mp4 support discontinuous audio streams?
I'm wondering, can ffmpeg transcode .wav-audio into opus-encapsulated-into-webm, but filter out silent frames - especially full seconds of silence, completely drop them from the stream to save space? (maybe via https://ffmpeg.org/ffmpeg-bitstream-filters.html ?)
The goal of all this is to very efficiently represent near-silence in phone-call-recording files for archival purposes (i.e. by completely omitting it from the stream and having it replaced by digital zeros at audio decoding time).
As far as I understand opus-encapsulated-into-ogg does not support such timestamp discontinuities. opus-encapsulated-into-rtp assumingly does support this, but I'm wondering if more "offline" formats such as webm and whether it's possible to produce them directly without passing by rtp format first.
These frames could then be replaced by zeros at audio decoding time by some form of PLC or DTS gaps replacement by zero signal.
Thanks!

UPD1: Unfortunately, silenceremove filter does not work as I need it. The way I used it below, the audio duration after silenceremove is shortened by the amount of silence. This is not what I am looking for. I am looking to preserve the timestamps, have the silence imputed by digital silence at decoding time and preserve the original audio duration (as reported by ffprobe or audio players)
ffmpeg -y -f lavfi -i sine=f=440:d=5 -f lavfi -i anullsrc=d=3600:cl=mono -f lavfi -i sine=f=880:d=5 -filter_complex "[0][1][2]concat=n=3:v=0:a=1" 5sec_silent1h_5sec.wav

ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus 5sec_silent1h_5sec.opus

ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0 5sec_5sec.mkv

The good duration of 5sec_silent1h_5sec.opus and bad duration of 5sec_5sec.mkv are confirmed by ffprobe (1h10s and 10s respectively).
Is there a way to adjust the filters/command line arguments to preserve the audio duration?

UPD2
The following somewhat works, although ~1s of 880hz audio leaks into the first segment (before the long silence segment), no matter how I adjust the between bounds - very strange!
ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af "aselect='between(t,0,4)+between(t,3606,3610)'" 5sec_silent1h_5sec_with_leakage.mkv
If this leakage can be fixed, then this can be used together with silencedetect filter.
But of course it would be nicer to get the output in a single invokation of ffmpeg.
The example output file with "leakage": https://1drv.ms/u/s!Apx8USiTtrYmq9Q4QaETF2OJWdKGQA?e=ACTmMg

UPD3: On a recent ffmpeg 6.1 adding timestamp=copy filter option also produces a file with a correct duration, but with leakage ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0:timestamp=copy 5sec_1hsilence_5sec_with_leakage.mkv. So it seems that timestamps are still not copied exactly.

**Tags:** ffmpeg, audio, matroska


---

## 1 answers

### Answer by username_1 on 2023-12-09. Score: 0

You can have timestamp gaps in an audio stream in Matroska-family or MOV/MP4 format files. That's how Google records and stores WebRTC recordings. There won't be zero signal audio data, rather there will be no packets corresponding to the gap and timestamps of the packets before and after the gap will indicate audio position.

However, it is not the encoder's job or feature in ffmpeg to drop those sections. You will need to use the [silenceremove](http://www.ffmpeg.org/ffmpeg-filters.html#silenceremove) filter with option `timestamp=copy` (6.1+ or git master > May 2023 needed).

> **Comments:**
> **username_0**: Would `silenceremove` produce such output with gaps? Or would it "collapse" the gaps and make the wall-clock-duration shorter? This was my misunderstanding while originally reading its documentation which led my to asking this question.  
> **username_1**: silenceremove can only remove data; it doesn't consolidate gaps.  
> **username_0**: `silenceremove` does not affect any timestamps and they stay original (as if no gaps were introduced), right? My goal is to have the encoded file have the same duration as before encoding (and have silence imputed by zeros at decoding time). Do you know if gaps are supported by `webm`/`mka`? Am I correct that `ogg` does not support gaps? Would you have an advice of how to instruct `ffmpeg` to encode separately `.wav` streams / in a decoupled mode (for phone call recordings, there are long periods of silence if one considers every channel separately)?  
> **username_0**: I tested the `silenceremove` solution. Unfortunately, it does not create `gaps` and it changes the duration of the audio - this is not what I need. I'd like to simply drop the silent packets from the stream, but preserve the timestamps and keep the audio duration and have the dropped packets be replaced by digital silence at decoding time.  
> **username_0**: Here is a repro: `ffmpeg -y -f lavfi -i sine=f=440:d=5 -f lavfi -i anullsrc=d=3600:cl=mono -f lavfi -i sine=f=880:d=5 -filter_complex "[0][1][2]concat=n=3:v=0:a=1" 5sec_silent1h_5sec.wav; ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus 5sec_silent1h_5sec.opus; ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0 5sec_5sec.mkv`  
> **username_0**: The good duration of `5sec_silent1h_5sec.opus` and bad duration of `5sec_5sec.mkv` are confirmed by `ffprobe` (1h10s and 10s respectively)  
> **username_1**: Updated answer. See at the end.  
> **username_0**: Thanks for helping out with this :) I've just tried your updated proposal using ffmpeg 6.1 static bits from https://johnvansickle.com/ffmpeg/, the duration turns correct, but it has the same leakage/wrong pts problem as my attempts with `aselect` filter :(  
> **username_0**: I converted the input `5sec_silent1h_5sec.wav` to be 48khz (in my original code above it was 44.1khz), but it made no difference - still leakage  
> **username_1**: Probably to do with the attack/release used by the filter. Will have to debug when I have time.  
> **username_0**: Thank you! However, this leakage is quite long - a full second :( I hope my repro above will help!  
> **username_0**: Also, maybe the leakage is related to some silenceremove filter options I used above?  But given the strangeness of such leakage, I suspect that somehow some packets of the 880hz-segment get incorrect timestamps assigned which forces them to be played before the 1h-long silence/gap  

---

