<blockquote>
<p><em>&quot;would the output data be in a specific format, such as JPEG (identified by the <code>FF D8</code> marker), or would it be raw bytes representing pixel data?&quot;</em></p>
</blockquote>
<p>An H.264 decoder should decode to raw YUV by default, although it's generally appreciated for standalone use to have a YUV4MPEG2 mode as well, for easier piping, and to have basic format metadata preserved.</p>
<p>Usually the encoding is in YU420P format. There are many examples of converting (via a <code>For-loop</code>) any YUV420p data into RGB data. You can finally encode the newly converted RGB into JPEG.</p>
<blockquote>
<p><em>&quot;Do you foresee any need for me to purchase a third documentation source?&quot;</em></p>
</blockquote>
<p>No, unless you want to also handle audio then you'll need the AAC documentation. <br>That is <code>iso/iec 13818-7</code>. You can also learn by analysing AAC bytes whilst following the source code of some open-source projects that are decoding AAC data. Articles online also explain structure of AAC. You can still get a result without the ISO document itself (<em>eg:</em> read articles from those who repeat its words as they teach about AAC audio's bytes structure).</p>

CC BY-SA 4.0
2023-06-30T18:08:54.557Z
Good, that made me confident.
Daniel
CC BY-SA 4.0
2023-06-29T17:48:45.113Z
2023-06-29T18:40:13.730Z
2023-06-29T18:40:13.730Z
damian101
<p>I am currently working on a personal programming challenge to develop an MP4 decoder using VB.NET and the .NET Framework 4.8.1 as a WinForms application. I have obtained the <code>ISO 14496-12</code> documentation, which serves as the basic structure for MP4 files.</p>
<p>At this stage, I have successfully prepared various lists containing frame sizes, chunk positions, and information from the stsc and stts boxes. Additionally, I have been able to identify keyframes within the MP4 files.</p>
<p>However, I have come to realize that the <code>ISO 14496-12</code> documentation is not sufficient for decoding the frame bytes, which are encoded in H.264/AVC. To proceed with decoding, I understand that I need to refer to the <code>ISO/IEC 14496-10</code> documentation specifically for H.264/AVC.</p>
<p>My question is twofold:</p>
<ul>
<li>When decoding the frame bytes according to the documentation, would the output data be in a specific format, such as JPEG (identified by the FF D8 marker), or would it be raw bytes representing pixel data? This clarification would help me. In the former case, I could simply save an image using .NET functions.</li>
<li>Considering my current progress and the information I have mentioned, do you foresee any need for me to purchase a third documentation source?</li>
</ul>
<p>Any guidance or suggestions would be appreciated.</p>

<mp4>
Decoding H.264/AVC Frames in MP4: What's the Output Format?
Daniel
# Decoding H.264/AVC Frames in MP4: What's the Output Format?

(Asked by: username_0 on 2023-06-28)

I am currently working on a personal programming challenge to develop an MP4 decoder using VB.NET and the .NET Framework 4.8.1 as a WinForms application. I have obtained the `ISO 14496-12` documentation, which serves as the basic structure for MP4 files.

At this stage, I have successfully prepared various lists containing frame sizes, chunk positions, and information from the stsc and stts boxes. Additionally, I have been able to identify keyframes within the MP4 files.

However, I have come to realize that the `ISO 14496-12` documentation is not sufficient for decoding the frame bytes, which are encoded in H.264/AVC. To proceed with decoding, I understand that I need to refer to the `ISO/IEC 14496-10` documentation specifically for H.264/AVC.

My question is twofold:

* When decoding the frame bytes according to the documentation, would the output data be in a specific format, such as JPEG (identified by the FF D8 marker), or would it be raw bytes representing pixel data? This clarification would help me. In the former case, I could simply save an image using .NET functions.
* Considering my current progress and the information I have mentioned, do you foresee any need for me to purchase a third documentation source?

Any guidance or suggestions would be appreciated.

**Tags:** mp4


---

## 1 answers

### Answer by username_1 on 2023-06-29. Score: 2 | Accepted answer

> *"would the output data be in a specific format, such as JPEG (identified by the `FF D8` marker), or would it be raw bytes representing pixel data?"*

An H.264 decoder should decode to raw YUV by default, although it's generally appreciated for standalone use to have a YUV4MPEG2 mode as well, for easier piping, and to have basic format metadata preserved.

Usually the encoding is in YU420P format. There are many examples of converting (via a `For-loop`) any YUV420p data into RGB data. You can finally encode the newly converted RGB into JPEG.

> *"Do you foresee any need for me to purchase a third documentation source?"*

No, unless you want to also handle audio then you'll need the AAC documentation.   
That is `iso/iec 13818-7`. You can also learn by analysing AAC bytes whilst following the source code of some open-source projects that are decoding AAC data. Articles online also explain structure of AAC. You can still get a result without the ISO document itself (*eg:* read articles from those who repeat its words as they teach about AAC audio's bytes structure).

> **Comments:**
> **username_0**: Good, that made me confident.  

---

