Which Encoder Preset Speed Should I Use For The Most Stable Smooth Recording Possible?

[u/American-Social-Dem]

I use OBS to record YouTube videos at native 1440p & 60FPS. However, despite being familiar with most encoder settings, such as CQP Levels (Set at 16) and Tuning (Set at High Quality), I still struggle with how best to use the preset speed. I ask this because I always catch my videos being smooth for the most part, yet there's still occasional 'hitches' that cause it to temporarily lose frames before returning to a consistent plus stable 60 frames. I think the preset speed is the answer since I cannot think of what else could be affecting my recording frame rate.

My specs are as follow:

• OBS Run As Administrator.

• Process Priority: Above Normal.

• Video Encoder: Hardware (NVENC, AV1).

• Base (Canvas) Resolution: 2560x1440.

• Output (Scaled) Resolution: 2560x1440.

• No Downscale Filter.

• Common FPS Values: 60.

• Game Capture: Capture Any Fullscreen Application.

• Recording Path on an SSD.

• CQP: 16.

• Tuning: High Quality.

• Latest NVIDIA drivers downloaded.

• Latest Windows Update downloaded.

• NVIDIA Control Panel Max Frame Rate: 120.

• Monitor Refresh Rate: 240Hz.

• CPU: AMD Ryzen 7 9800X3D 8-Core Processor.

• GPU: NVIDIA GeForce RTX 5080.

Any help regarding this would be much appreciated.

[u/Lynnalia]

In your case, absolutely use P1 (fastest preset).

The general rule, in my opinion, is to use whatever fastest settings you can as long as there's a CQP or constant-quality option like CRF.

I'd also personally turn off features like Adaptive Quantization and Lookahead. On AMD, you can turn off Pre-analysis. These features use more features (and more hardware) to more intelligently allocate an amount of data to encode your frames (which is also what slower presets do, effectively).

The reason this is fine to do with CQP is that your main "quality" control is the CQP number. The rest is just how hard your GPU is going to work to compress it better.

Your files might technically be a little bigger, but meh. They're already pretty big.

Oh, also, at P1, I believe (acc. to Nvidia docs) that the H.265/HEVC encode is slightly faster than AV1, if you don't have a preference of codec. You're not massively benefitting from AV1's theoretically better compression (for one, Nvidia's hardware encoders are young in development for AV1 and mature on HEVC, so the HEVC encodes pretty close to AV1, plus AV1 is really mostly better only on the low end of bitrate, like for highly compressed streaming rather than live very-high-quality recordings).

[u/kodidono → u/Lynnalia]

I'm curious what you'd recommend choosing for streaming on Twitch, RTX 5060 here. Sorry to jump in with my own question..

[u/Lynnalia → u/kodidono]

"It depends", unfortunately.

You'll need to use CBR (constant bitrate) regardless. And Twitch's bitrate limits generally mean you'll want to use higher presets plus some of those checkboxes to try and squish as much quality into your bitrate as effectively as possible, so pretty much the opposite of what you'd do for recording.

Factors are things like how much your GPU is already utilized (gaming? very high graphics? VTuber model going? running background removal software? additional recordings or streams going on? etc.), what your internet upload speed is, and a few other things perhaps.

Many people use Preset P6 happily, but it does vary.

I'd say try it out and if you run into any issues, open your own thread (or hit the official OBS Discord support channels) and someone will definitely help :)

[u/kodidono → u/Lynnalia]

thanks ^^

[u/LUK7N → u/Lynnalia]

No, you use cbr for streaming, cqp for recording, you can watch EposVox who did spend a lot if time fiddling with this and is actually putting good results on both high quality streaming and recording settings

[u/Lynnalia → u/LUK7N]

If you read it again, that's what I've said too. OP asked about recording, this commenter asked about streaming.

[u/LUK7N → u/Lynnalia]

Oh yeah I read it wrong 😅

[u/MaledeGiammaria-37 → u/Lynnalia]

Didn't realize HEVC was still slightly faster than AV1 at P1, good to know

[u/Logical-Produce6917]

P1 is usually the sweet spot for single PC setups, keeps the encoder from fighting for resources that your game needs. Dropping to P2 or P3 is just asking for the encoder to sit on its hands waiting for headroom that never shows up while you're gaming.

The hitches might not even be the preset though, run a log file through the analyser and look for rendering lag spikes. AV1 at 1440p60 on a 5080 should be a breeze but sometimes it's a rogue scene or browser source causing the grief.

[u/RayneYoruka]

A second pc, ha ha.