<p>At the very top of your $800 budget, you can get a Quadro M4000 that does NOT have the license limitation of 2 concurrent transcodes. We use these cards to transcode 5-10 incoming live streams to 2-3 output bitrates.</p>

CC BY-SA 3.0
2016-09-17T19:28:01.263Z
Can you comment on the time it takes to initiate a transcoding steam?  On my consumer card it was about a full second.  Since I have dozens of short clips coming in at the same time, a one second overhead per clip ends up costing me dearly.
user255406
CC BY-SA 3.0
2016-09-17T17:37:40.660Z
2016-09-17T17:37:40.660Z
Allan Jude
<p>There are two main features in NVIDIA NVENC encoding:</p>

<ol>
<li>Video memory if you need to transcode many streams.</li>
<li>NVENC SIP - than better this SIP than better encoding performance
is. It depends on GPU. You can find the best explanation related GPU
and NVENC SIP generation on <a href="https://en.wikipedia.org/wiki/Nvidia_NVENC" rel="nofollow noreferrer">Wikipedia website</a>. Maxwell Gen 2 is the best at the moment(for 2016).</li>
</ol>

<p>According to the license limitation: it's possible to run only 2 encoding threads simultaneously on the consumer level NVIDIA cards (any GTX card). It's regulated on the driver level, however it’s possible to remove this limitation. In this case, the value of maximum transcoding threads will depend on  video memory size and video engine utilization.
Video memory size which is necessary for one transcoding stream is various and depends on a video card model. For encoding SD stream on QUADRO K4200(4GB) is necessary 100 MB of video memory, but for encoding the same stream on GTX 980TI(6GB) we need 170MB.</p>

<p>My results in transcoding SD sources in real time are:<br></p>

<ul>
<li>QUADRO K4200(4GB): one transcoding thread costs 100 MB and we can run in parallel about 36 threads, but the bottleneck is the video engine utilization. I can run about 30 parallel threads with <em>"-preset hp -vcodec nvenc_264"</em> </li>
<li>GTX 980TI(6GB): the bottleneck is the video memory. I can run about 32
threads (32*170=5440) in parallel. I did it for my education, of
course. But the video engine results is better then on K4200 in 2.5
times. These 32 threads with <em>"-preset slow -vcodec nvenc_264 -vf
yadif=0"</em> and video engine utilization are only 80%.</li>
<li>I also tested it on GTX 660(2GB, Maxwell Gen 2). It was about 15
parallel threads due to video memory.</li>
</ul>

<p>My conclusion(for 2016): If you need to transcode no more than 2 threads in parallel, then  GTX 960 is a good variant. Also,  you can save some money to another hardware and look for another video card with Maxwell Gen1. If you go to the hack way, then GTX 960 is a good variant but only with 4GB of video memory.</p>

<p><strong>UPDATE FROM 2018:</strong>
Nowadays the situation in this area has changed in  better direction. The card with the best encoder is Tesla V100, but it's too expensive. The best work variant is the video cards based on Pascal CHIPs. If you prefer the hack way then GTX1050TI(4GB), it will be perfect for your budget in 800$. Otherwise, Quadro P2000 doesn’t have license limitations and also very comfortable for the same budget.</p>

CC BY-SA 3.0
2016-03-20T14:45:09.590Z
Thanks for your answer.  After some testing I realized that the overhead time it takes to initiate the transcoding becomes a huge issue with multiple short clips.  Have you experienced this?  I'm seeing maybe 1s/clip overhead.  When my clips take average 1s/clip to transcode using CPU only, the GPU benefit vanishes.  Thoughts?
user255406
CC BY-SA 3.0
2016-03-21T06:48:55.943Z
Unfortunately, I don't have experience with multiple short clips.
One idea: you need maintain maximum possible NVENC threads. It will be faster then on CPU. For this you need to use variant with many transcoding streams in parallel for this need to buy QUADRO video card or go to the hack way.
Ivan Kolesnikov
CC BY-SA 3.0
2016-03-21T06:49:00.363Z
Then you can to find out a maximum possible NVENC threads and  always maintain it or you can try to run new FFmpeg instance if the maximum number of streams will be exceeded then will be occurs the following error: "Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height".
Ivan Kolesnikov
CC BY-SA 3.0
2016-03-22T11:44:23.167Z
Unfortunately the clips will be uploaded sporadically, sometimes a bunch, sometimes a few.  I'm leaning towards a CPU only solution as this seems to be fool proof.
user255406
CC BY-SA 3.0
2016-03-22T12:08:49.083Z
You can try QuickSync in this case:
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/quicksync-video-ffmpeg-install-valid.pdf
Ivan Kolesnikov
CC BY-SA 3.0
2016-03-27T03:22:29.817Z
Have you any experience with QS?  I've been looking for overhead/delay examples with QS to no avail.
user255406
CC BY-SA 3.0
2016-03-28T07:17:42.093Z
Unfortunately, I haven't real work experience with QS, but some sources from Intel says what QT faster than NVENC in a four times.
Ivan Kolesnikov
CC BY-SA 3.0
2016-03-30T00:40:52.797Z
I have an i7 3770k in a Hackintosh.  Guess I'll boot from a live USB and try running QS on it to see how it fares.  I hope the Broadwell-E CPUs to be released in June will support QS.
user255406
CC BY-SA 4.0
2016-03-18T13:04:47.120Z
2018-08-14T11:34:11.923Z
2018-08-14T11:34:11.923Z
Ivan Kolesnikov
<p>I have a web project I am working on that requires transcoding of multiple short video clips (6-10 seconds) uploaded by users into a web friendly H.264 format on my linux server.  I have been able to enable NVIDIA NVENC hardware acceleration with ffmpeg and a 4 year old GeForce GTX 670, and I'm getting hardware encoding speeds twice that of my software encoding (Xeon E5-1620 v3).  With an $800 video card budget, I'd like to be able to transcode these short video clips as fast as possible because I will have multiple users uploading them simultaneously.  </p>

<p>The NVENC engine does have <a href="http://developer.download.nvidia.com/assets/cuda/files/NVENC_DA-06209-001_v07.pdf?autho=1452972945_d6791efa67031b38127384c5c437f683&amp;file=NVENC_DA-06209-001_v07.pdf" rel="nofollow">licensing limitations</a> when implemented on a consumer level NVIDIA card: only 2 video transcoding threads can be run simultaneously, even if you have multiple cards.  If I decide to go with one of the pricy quadro card line, then I am only limited by the other hardware elsewhere in my system in terms of how many threads I can run.  However, with my specific project I'm a lot better off transcoding these clips in a series rather than parallel because the clips will be viewed in the order they're uploaded. Clips later in the queue can be transcoded as earlier clips are viewed. If clips are transcoded in parallel on the same card, performance is inversely proportional to the number of simultaneous threads.</p>

<p>Having stated this, my plan is to set up two NVIDIA cards and run a single thread on each one to maximize throughput.  The <a href="https://developer.nvidia.com/nvidia-video-codec-sdk" rel="nofollow">NVIDIA codec SDK</a> is vague on NVENC performance difference between various cards, but it seems that there is a <a href="http://developer.download.nvidia.com/assets/cuda/files/NVENC_DA-06209-001_v07.pdf?autho=1452974230_cc98539d0a11948f1978940adf52093e&amp;file=NVENC_DA-06209-001_v07.pdf" rel="nofollow">major difference</a> between GPU generations Maxwell Gen 2> Maxwell Gen 1> Kepler.  I can find no reliable benchmarks for NVENC encoding (opposed to CUDA benchmarking, which is easy to find).</p>

<p>In the absence of hard benchmarking data comparing various cards, what features of the currently available cards would have to largest impact on single thread NVENC encoding speed?  Since the actual GPU is not utilized fully during transcoding, does GPU and memory clock speed affect this function much?  I have $400/card to spend, but if the entry level Maxwell Gen 2 GeForce GTX 960 is as good as the more recent cards, then I will put extra money into other aspects of the server (CPU/RAM/SSD etc).  I know this might seem like a subjective question (what's <em>best</em>), but I am trying to make a self-educated guess based on an understanding of the transcoding hardware.</p>

CC BY-SA 3.0
2016-01-19T10:58:37.963Z
I just wanted to say that I love this post and hope for an equally interesting answer.
I wonder why the GPU doesn't get utilized completely, is it lack of optimization? A bottle neck somewhere else in the pipeline?
kimgroth
CC BY-SA 3.0
2016-01-20T17:32:40.003Z
From what I've read there is a completely separate NVENC chip doing most of the work, so my guess would be that it is bottleneck.  NVIDIA is quite vague in explaining it for some reason.
user255406
CC BY-SA 3.0
2016-01-20T17:33:13.730Z
Will you 1+ this question to bump it up?
user255406
<ffmpeg><conversion><transcoding><nvenc>
what graphics card features effect NVIDIA NVENC hardware encoding speed?
user255406
# what graphics card features effect NVIDIA NVENC hardware encoding speed?

(Asked by: username_0 on 2016-01-16)

I have a web project I am working on that requires transcoding of multiple short video clips (6-10 seconds) uploaded by users into a web friendly H.264 format on my linux server. I have been able to enable NVIDIA NVENC hardware acceleration with ffmpeg and a 4 year old GeForce GTX 670, and I'm getting hardware encoding speeds twice that of my software encoding (Xeon E5-1620 v3). With an $800 video card budget, I'd like to be able to transcode these short video clips as fast as possible because I will have multiple users uploading them simultaneously.

The NVENC engine does have [licensing limitations](http://developer.download.nvidia.com/assets/cuda/files/NVENC_DA-06209-001_v07.pdf?autho=1452972945_d6791efa67031b38127384c5c437f683&file=NVENC_DA-06209-001_v07.pdf) when implemented on a consumer level NVIDIA card: only 2 video transcoding threads can be run simultaneously, even if you have multiple cards. If I decide to go with one of the pricy quadro card line, then I am only limited by the other hardware elsewhere in my system in terms of how many threads I can run. However, with my specific project I'm a lot better off transcoding these clips in a series rather than parallel because the clips will be viewed in the order they're uploaded. Clips later in the queue can be transcoded as earlier clips are viewed. If clips are transcoded in parallel on the same card, performance is inversely proportional to the number of simultaneous threads.

Having stated this, my plan is to set up two NVIDIA cards and run a single thread on each one to maximize throughput. The [NVIDIA codec SDK](https://developer.nvidia.com/nvidia-video-codec-sdk) is vague on NVENC performance difference between various cards, but it seems that there is a [major difference](http://developer.download.nvidia.com/assets/cuda/files/NVENC_DA-06209-001_v07.pdf?autho=1452974230_cc98539d0a11948f1978940adf52093e&file=NVENC_DA-06209-001_v07.pdf) between GPU generations Maxwell Gen 2> Maxwell Gen 1> Kepler. I can find no reliable benchmarks for NVENC encoding (opposed to CUDA benchmarking, which is easy to find).

In the absence of hard benchmarking data comparing various cards, what features of the currently available cards would have to largest impact on single thread NVENC encoding speed? Since the actual GPU is not utilized fully during transcoding, does GPU and memory clock speed affect this function much? I have $400/card to spend, but if the entry level Maxwell Gen 2 GeForce GTX 960 is as good as the more recent cards, then I will put extra money into other aspects of the server (CPU/RAM/SSD etc). I know this might seem like a subjective question (what's *best*), but I am trying to make a self-educated guess based on an understanding of the transcoding hardware.

**Tags:** ffmpeg, conversion, transcoding, nvenc

> **Comments:**
> **username_1**: I just wanted to say that I love this post and hope for an equally interesting answer. I wonder why the GPU doesn't get utilized completely, is it lack of optimization? A bottle neck somewhere else in the pipeline?  
> **username_0**: From what I've read there is a completely separate NVENC chip doing most of the work, so my guess would be that it is bottleneck.  NVIDIA is quite vague in explaining it for some reason.  
> **username_0**: Will you 1+ this question to bump it up?  

---

## 2 answers

### Answer by username_3 on 2016-03-18. Score: 4 | Accepted answer

There are two main features in NVIDIA NVENC encoding:

1. Video memory if you need to transcode many streams.
2. NVENC SIP - than better this SIP than better encoding performance
   is. It depends on GPU. You can find the best explanation related GPU
   and NVENC SIP generation on [Wikipedia website](https://en.wikipedia.org/wiki/Nvidia_NVENC). Maxwell Gen 2 is the best at the moment(for 2016).

According to the license limitation: it's possible to run only 2 encoding threads simultaneously on the consumer level NVIDIA cards (any GTX card). It's regulated on the driver level, however it’s possible to remove this limitation. In this case, the value of maximum transcoding threads will depend on video memory size and video engine utilization.
Video memory size which is necessary for one transcoding stream is various and depends on a video card model. For encoding SD stream on QUADRO K4200(4GB) is necessary 100 MB of video memory, but for encoding the same stream on GTX 980TI(6GB) we need 170MB.

My results in transcoding SD sources in real time are:

* QUADRO K4200(4GB): one transcoding thread costs 100 MB and we can run in parallel about 36 threads, but the bottleneck is the video engine utilization. I can run about 30 parallel threads with *"-preset hp -vcodec nvenc\_264"*
* GTX 980TI(6GB): the bottleneck is the video memory. I can run about 32
  threads (32\*170=5440) in parallel. I did it for my education, of
  course. But the video engine results is better then on K4200 in 2.5
  times. These 32 threads with *"-preset slow -vcodec nvenc\_264 -vf
  yadif=0"* and video engine utilization are only 80%.
* I also tested it on GTX 660(2GB, Maxwell Gen 2). It was about 15
  parallel threads due to video memory.

My conclusion(for 2016): If you need to transcode no more than 2 threads in parallel, then GTX 960 is a good variant. Also, you can save some money to another hardware and look for another video card with Maxwell Gen1. If you go to the hack way, then GTX 960 is a good variant but only with 4GB of video memory.

**UPDATE FROM 2018:**
Nowadays the situation in this area has changed in better direction. The card with the best encoder is Tesla V100, but it's too expensive. The best work variant is the video cards based on Pascal CHIPs. If you prefer the hack way then GTX1050TI(4GB), it will be perfect for your budget in 800$. Otherwise, Quadro P2000 doesn’t have license limitations and also very comfortable for the same budget.

> **Comments:**
> **username_0**: Thanks for your answer.  After some testing I realized that the overhead time it takes to initiate the transcoding becomes a huge issue with multiple short clips.  Have you experienced this?  I'm seeing maybe 1s/clip overhead.  When my clips take average 1s/clip to transcode using CPU only, the GPU benefit vanishes.  Thoughts?  
> **username_3**: Unfortunately, I don't have experience with multiple short clips. One idea: you need maintain maximum possible NVENC threads. It will be faster then on CPU. For this you need to use variant with many transcoding streams in parallel for this need to buy QUADRO video card or go to the hack way.  
> **username_3**: Then you can to find out a maximum possible NVENC threads and  always maintain it or you can try to run new FFmpeg instance if the maximum number of streams will be exceeded then will be occurs the following error: "Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height".  
> **username_0**: Unfortunately the clips will be uploaded sporadically, sometimes a bunch, sometimes a few.  I'm leaning towards a CPU only solution as this seems to be fool proof.  
> **username_3**: You can try QuickSync in this case: http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/quicksync-video-ffmpeg-install-valid.pdf  
> **username_0**: Have you any experience with QS?  I've been looking for overhead/delay examples with QS to no avail.  
> **username_3**: Unfortunately, I haven't real work experience with QS, but some sources from Intel says what QT faster than NVENC in a four times.  
> **username_0**: I have an i7 3770k in a Hackintosh.  Guess I'll boot from a live USB and try running QS on it to see how it fares.  I hope the Broadwell-E CPUs to be released in June will support QS.  

---

### Answer by username_2 on 2016-09-17. Score: 1

At the very top of your $800 budget, you can get a Quadro M4000 that does NOT have the license limitation of 2 concurrent transcodes. We use these cards to transcode 5-10 incoming live streams to 2-3 output bitrates.

> **Comments:**
> **username_0**: Can you comment on the time it takes to initiate a transcoding steam?  On my consumer card it was about a full second.  Since I have dozens of short clips coming in at the same time, a one second overhead per clip ends up costing me dearly.  

---

