AV1 has problem with 2 pass and the last percent

[Mister-XY]

Problem Description

Hello, I have a small but important problem with the AV1 codec or AV1 and 2-pass. Now I don't know if it's a handbrake problem or a codec problem. I noticed that in the last run from 99% onwards, the bps shot up when encoding and the image quality deteriorated dramatically, so that there were only blocks left. This is particularly noticeable in films that hardly have any credits. E.g. "All Quiet on the Western Front"

2nd movie and same problem

3th movie Ant_man with after Post-Credit szene.

Activity Log, Crash Log or any other details

HandBrake Nightly 20240506063248-fa6cf3cc8-master (2024050601)
OS: Microsoft Windows NT 10.0.22631.0
CPU: AMD Ryzen 9 7900 12-Core Processor             
Ram: 31967 MB, 
GPU Information:
  AMD Radeon(TM) Graphics - 31.0.24027.1012
Screen: 1920x1080
Temp Dir: C:\Users\Thomas\AppData\Local\Temp\
Install Dir: C:\Program Files\HandBrake Nightly\
Data Dir: C:\Users\Thomas\AppData\Roaming\HandBrake\Nightly

-------------------------------------------

 # Starting Encode ...

[05:42:26] base preset: AV1 VBR 2 Pass BluRay
[05:42:26] hb_init: starting libhb thread
[05:42:26] Starting work at: Fri May 10 05:42:26 2024
[05:42:26] 1 job(s) to process
[05:42:26] CPU: AMD Ryzen 9 7900 12-Core Processor
[05:42:26]  - logical processor count: 24
[05:42:26] Intel Quick Sync Video support: no
[05:42:26] hb_scan: path=I:\RIP\Im Westen nichts Neues_t01.mkv, title_index=1
Input #0, matroska,webm, from 'I:\RIP\Im Westen nichts Neues_t01.mkv':
  Metadata:
    title           : Im Westen nichts Neues
    encoder         : libmakemkv v1.17.6 (1.3.10/1.5.2) win(x64-release)
    creation_time   : 2024-05-08T04:54:07.000000Z
  Duration: 02:12:56.72, start: 0.000000, bitrate: 16820 kb/s
  Chapters:
    Chapter #0:0: start 0.000000, end 578.077500
      Metadata:
        title           : Chapter 01
    Chapter #0:1: start 578.077500, end 809.183375
      Metadata:
        title           : Chapter 02
    Chapter #0:2: start 809.183375, end 1377.125750
      Metadata:
        title           : Chapter 03
    Chapter #0:3: start 1377.125750, end 1653.360042
      Metadata:
        title           : Chapter 04
    Chapter #0:4: start 1653.360042, end 1926.007417
      Metadata:
        title           : Chapter 05
    Chapter #0:5: start 1926.007417, end 2073.071000
      Metadata:
        title           : Chapter 06
    Chapter #0:6: start 2073.071000, end 2305.469833
      Metadata:
        title           : Chapter 07
    Chapter #0:7: start 2305.469833, end 2658.739417
      Metadata:
        title           : Chapter 08
    Chapter #0:8: start 2658.739417, end 3010.465792
      Metadata:
        title           : Chapter 09
    Chapter #0:9: start 3010.465792, end 3204.159292
      Metadata:
        title           : Chapter 10
    Chapter #0:10: start 3204.159292, end 3509.923083
      Metadata:
        title           : Chapter 11
    Chapter #0:11: start 3509.923083, end 4110.397958
      Metadata:
        title           : Chapter 12
    Chapter #0:12: start 4110.397958, end 4475.512708
      Metadata:
        title           : Chapter 13
    Chapter #0:13: start 4475.512708, end 4683.720708
      Metadata:
        title           : Chapter 14
    Chapter #0:14: start 4683.720708, end 5193.897042
      Metadata:
        title           : Chapter 15
    Chapter #0:15: start 5193.897042, end 5532.610417
      Metadata:
        title           : Chapter 16
    Chapter #0:16: start 5532.610417, end 5935.721458
      Metadata:
        title           : Chapter 17
    Chapter #0:17: start 5935.721458, end 7030.064708
      Metadata:
        title           : Chapter 18
    Chapter #0:18: start 7030.064708, end 7887.045833
      Metadata:
        title           : Chapter 19
    Chapter #0:19: start 7887.045833, end 7976.718750
      Metadata:
        title           : Chapter 20
  Stream #0:0(eng): Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn
      Metadata:
        BPS-eng         : 15997370
        DURATION-eng    : 02:12:56.718750000
        NUMBER_OF_FRAMES-eng: 191250
        NUMBER_OF_BYTES-eng: 15950814418
        SOURCE_ID-eng   : 001011
        _STATISTICS_WRITING_APP-eng: MakeMKV v1.17.6 win(x64-release)
        _STATISTICS_WRITING_DATE_UTC-eng: 2024-05-08 04:54:07
        _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID
  Stream #0:1(deu): Audio: dts (dca) (DTS), 48000 Hz, mono, fltp, 768 kb/s (default)
      Metadata:
        title           : Mono
        BPS-eng         : 768000
        DURATION-eng    : 02:12:56.682666666
        NUMBER_OF_FRAMES-eng: 747814
        NUMBER_OF_BYTES-eng: 765761536
        SOURCE_ID-eng   : 001100
        _STATISTICS_WRITING_APP-eng: MakeMKV v1.17.6 win(x64-release)
        _STATISTICS_WRITING_DATE_UTC-eng: 2024-05-08 04:54:07
        _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID
  Stream #0:2(deu): Subtitle: hdmv_pgs_subtitle (pgssub)
      Metadata:
        BPS-eng         : 52600
        DURATION-eng    : 02:12:27.251812500
        NUMBER_OF_FRAMES-eng: 3010
        NUMBER_OF_BYTES-eng: 52253532
        SOURCE_ID-eng   : 001200
        _STATISTICS_WRITING_APP-eng: MakeMKV v1.17.6 win(x64-release)
        _STATISTICS_WRITING_DATE_UTC-eng: 2024-05-08 04:54:07
        _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID
[05:42:26] scan: decoding previews for title 1
[AVHWDeviceContext @ 000002279e8b5e80] Cannot load nvcuda.dll
[AVHWDeviceContext @ 000002279e8b5e80] Could not dynamically load CUDA
hwaccel: failed to create hwdevice
hwaccel: failed to create hwdevice
[05:42:26] scan: audio 0x1: dca, rate=48000Hz, bitrate=768000 Deutsch (DTS, 1.0 ch, 768 kbps)
[05:42:27] using container PAR 1:1
[05:42:27] scan: 10 previews, 1920x1080, 23.976 fps, autocrop = 0/0/0/0, aspect 16:9, PAR 1:1, color profile: 1-1-1, chroma location: left
[05:42:27] libhb: scan thread found 1 valid title(s)
[05:42:27] Starting Task: Subtitle Scan
[05:42:27] work: skipping crop/scale filter
[05:42:27] job configuration:
[05:42:27]  * source
[05:42:27]    + I:\RIP\Im Westen nichts Neues_t01.mkv
[05:42:27]    + title 1, chapter(s) 1 to 20
[05:42:27]    + container: matroska,webm
[05:42:27]    + data rate: 16820 kbps
[05:42:27]  * destination
[05:42:27]    + I:\Encode\Im Westen nichts Neues_t01.AV1.mkv
[05:42:27]    + container: Matroska (libavformat)
[05:42:27]      + chapter markers
[05:42:27]  * video track
[05:42:27]    + decoder: h264 8-bit (yuv420p)
[05:42:27]    + filters
[05:42:27]      + Framerate Shaper (mode=1)
[05:42:27]        + frame rate: 23.976 fps -> constant 23.976 fps
[05:42:27]      + Format (format=yuv420p10le)
[05:42:27]    + Output geometry
[05:42:27]      + storage dimensions: 1920 x 1080
[05:42:27]      + pixel aspect ratio: 1 : 1
[05:42:27]      + display dimensions: 1920 x 1080
[05:42:27]  * Foreign Audio Search: Passthrough, Forced Only, Default
[05:42:27]    + subtitle, Deutsch (PGS) (track 0, id 0x2, Picture)
[05:42:27] sync: expecting 191249 video frames
[05:42:34] reader: done. 1 scr changes
[05:42:34] work: average encoding speed for job is 0.000000 fps
[05:42:34] vfr: 0 frames output, 0 dropped and 0 duped for CFR/PFR
[05:42:34] vfr: lost time: 0 (0 frames)
[05:42:34] vfr: gained time: 0 (0 frames) (0 not accounted for)
[05:42:34] h264-decoder done: 0 frames, 0 decoder errors
[05:42:34] sync: got 0 frames, 191249 expected
[05:42:34] Subtitle track 0 (id 0x2) 'Deutsch (PGS)': 2066 hits (0 forced)
[05:42:34] No candidate detected during subtitle scan
[05:42:34] Starting Task: Analysis Pass
[05:42:34] work: skipping crop/scale filter
[05:42:34] work: track 1, dithering not supported by codec
[05:42:34] job configuration:
[05:42:34]  * source
[05:42:34]    + I:\RIP\Im Westen nichts Neues_t01.mkv
[05:42:34]    + title 1, chapter(s) 1 to 20
[05:42:34]    + container: matroska,webm
[05:42:34]    + data rate: 16820 kbps
[05:42:34]  * destination
[05:42:34]    + I:\Encode\Im Westen nichts Neues_t01.AV1.mkv
[05:42:34]    + container: Matroska (libavformat)
[05:42:34]      + chapter markers
[05:42:34]  * video track
[05:42:34]    + decoder: h264 8-bit (yuv420p)
[05:42:34]    + filters
[05:42:34]      + Framerate Shaper (mode=1)
[05:42:34]        + frame rate: 23.976 fps -> constant 23.976 fps
[05:42:34]      + Format (format=yuv420p10le)
[05:42:34]    + Output geometry
[05:42:34]      + storage dimensions: 1920 x 1080
[05:42:34]      + pixel aspect ratio: 1 : 1
[05:42:34]      + display dimensions: 1920 x 1080
[05:42:34]    + encoder: AV1 10-bit (SVT)
[05:42:34]      + preset:  5
[05:42:34]      + options: film-grain=8
[05:42:34]      + profile: auto
[05:42:34]      + level:   auto
[05:42:34]      + bitrate: 4500 kbps, pass: 1
[05:42:34]      + color profile: 1-1-1
[05:42:34]      + color range: tv
[05:42:34]      + chroma location: left
[05:42:34]  * audio track 1
[05:42:34]    + name: Mono
[05:42:34]    + decoder: Deutsch (DTS, 1.0 ch, 768 kbps) (track 1, id 0x1)
[05:42:34]      + bitrate: 768 kbps, samplerate: 48000 Hz
[05:42:34]    + mixdown: Mono
[05:42:34]    + encoder: AC3 (libavcodec)
[05:42:34]      + bitrate: 96 kbps, samplerate: 48000 Hz
[05:42:34] sync: expecting 191249 video frames
Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:	SVT-AV1 Encoder Lib v2.0.0
Svt[info]: SVT [build]  :	GCC 13.2.0	 64 bit
Svt[info]: -------------------------------------------
Svt[warn]: The input q value is ignored in vbr mode 43
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: preset 							: Pass 1
Svt[info]: -------------------------------------------
[05:42:34] sync: first pts video is 0
[05:42:34] sync: "Chapter 1" (1) at frame 1 time 0
[05:42:34] sync: first pts audio 0x1 is 0
[05:46:14] sync: "Chapter 2" (2) at frame 13861 time 52026975
[05:47:34] sync: "Chapter 3" (3) at frame 19405 time 72837765
[05:50:48] sync: "Chapter 4" (4) at frame 33019 time 123941317
[05:52:22] sync: "Chapter 5" (5) at frame 39645 time 148813665
[05:53:51] sync: "Chapter 6" (6) at frame 46182 time 173351928
[05:54:41] sync: "Chapter 7" (7) at frame 49705 time 186576390
[05:56:01] sync: "Chapter 8" (8) at frame 55277 time 207492285
[05:58:15] sync: "Chapter 9" (9) at frame 63749 time 239294055
[06:00:18] sync: "Chapter 10" (10) at frame 72180 time 270941921
[06:01:20] sync: "Chapter 11" (11) at frame 76827 time 288385597
[06:03:05] sync: "Chapter 12" (12) at frame 84158 time 315904338
[06:06:52] sync: "Chapter 13" (13) at frame 98552 time 369935816
[06:08:59] sync: "Chapter 14" (14) at frame 107306 time 402796143
[06:10:08] sync: "Chapter 15" (15) at frame 112298 time 421534863
[06:12:56] sync: "Chapter 16" (16) at frame 124533 time 467461995
[06:14:51] sync: "Chapter 17" (17) at frame 132653 time 497942445
[06:17:11] sync: "Chapter 18" (18) at frame 142319 time 534226192
[06:23:10] sync: "Chapter 19" (19) at frame 168554 time 632705823
[06:27:54] sync: "Chapter 20" (20) at frame 189101 time 709834125
[06:28:23] reader: done. 1 scr changes
[06:28:25] work: average encoding speed for job is 69.545113 fps
[06:28:25] vfr: 191250 frames output, 0 dropped and 0 duped for CFR/PFR
[06:28:25] vfr: lost time: 0 (0 frames)
[06:28:25] vfr: gained time: 0 (0 frames) (0 not accounted for)
[06:28:26] dca-decoder done: 747814 frames, 0 decoder errors
[06:28:26] h264-decoder done: 191250 frames, 0 decoder errors
[06:28:26] sync: got 191250 frames, 191249 expected
[06:28:26] sync: framerate min 23.976 fps, max 23.976 fps, avg 23.976 fps
[06:28:26] Starting Task: Encoding Pass
[06:28:26] work: skipping crop/scale filter
[06:28:26] work: track 1, dithering not supported by codec
[06:28:26] job configuration:
[06:28:26]  * source
[06:28:26]    + I:\RIP\Im Westen nichts Neues_t01.mkv
[06:28:26]    + title 1, chapter(s) 1 to 20
[06:28:26]    + container: matroska,webm
[06:28:26]    + data rate: 16820 kbps
[06:28:26]  * destination
[06:28:26]    + I:\Encode\Im Westen nichts Neues_t01.AV1.mkv
[06:28:26]    + container: Matroska (libavformat)
[06:28:26]      + chapter markers
[06:28:26]  * video track
[06:28:26]    + decoder: h264 8-bit (yuv420p)
[06:28:26]    + filters
[06:28:26]      + Framerate Shaper (mode=1)
[06:28:26]        + frame rate: 23.976 fps -> constant 23.976 fps
[06:28:26]      + Format (format=yuv420p10le)
[06:28:26]    + Output geometry
[06:28:26]      + storage dimensions: 1920 x 1080
[06:28:26]      + pixel aspect ratio: 1 : 1
[06:28:26]      + display dimensions: 1920 x 1080
[06:28:26]    + encoder: AV1 10-bit (SVT)
[06:28:26]      + preset:  5
[06:28:26]      + options: film-grain=8
[06:28:26]      + profile: auto
[06:28:26]      + level:   auto
[06:28:26]      + bitrate: 4500 kbps, pass: 2
[06:28:26]      + color profile: 1-1-1
[06:28:26]      + color range: tv
[06:28:26]      + chroma location: left
[06:28:26]  * audio track 1
[06:28:26]    + name: Mono
[06:28:26]    + decoder: Deutsch (DTS, 1.0 ch, 768 kbps) (track 1, id 0x1)
[06:28:26]      + bitrate: 768 kbps, samplerate: 48000 Hz
[06:28:26]    + mixdown: Mono
[06:28:26]    + encoder: AC3 (libavcodec)
[06:28:26]      + bitrate: 96 kbps, samplerate: 48000 Hz
[06:28:26] sync: expecting 191250 video frames
Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:	SVT-AV1 Encoder Lib v2.0.0
Svt[info]: SVT [build]  :	GCC 13.2.0	 64 bit
Svt[info]: -------------------------------------------
Svt[warn]: For CRF or 2PASS RC mode, the maximum needed Lookahead distance is 42. Force the look_ahead_distance to be 42
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile	tier (auto)	level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator 		: 1920 / 1080 / 24000 / 1001
Svt[info]: SVT [config]: bit-depth / color format 					: 10 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct 					: 5 / VQ / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type 			: 245 / 16 / key frame
Svt[info]: SVT [config]: BRC mode / target bitrate (kbps)				: VBR / 4500
Svt[info]: SVT [config]: film grain synth / denoising / level 				: 1 / 0 / 8
Svt[info]: -------------------------------------------
[06:28:26] sync: first pts video is 0
[06:28:26] sync: "Chapter 1" (1) at frame 1 time 0
[06:28:26] sync: first pts audio 0x1 is 0
[06:43:24] sync: "Chapter 2" (2) at frame 13861 time 52026975
[06:47:50] sync: "Chapter 3" (3) at frame 19405 time 72837765
[07:00:28] sync: "Chapter 4" (4) at frame 33019 time 123941317
[07:07:08] sync: "Chapter 5" (5) at frame 39645 time 148813665
[07:14:14] sync: "Chapter 6" (6) at frame 46182 time 173351928
[07:17:33] sync: "Chapter 7" (7) at frame 49705 time 186576390
[07:23:41] sync: "Chapter 8" (8) at frame 55277 time 207492285
[07:31:12] sync: "Chapter 9" (9) at frame 63749 time 239294055
[07:40:07] sync: "Chapter 10" (10) at frame 72180 time 270941921
[07:45:15] sync: "Chapter 11" (11) at frame 76827 time 288385597
[07:53:00] sync: "Chapter 12" (12) at frame 84158 time 315904338
[08:06:11] sync: "Chapter 13" (13) at frame 98552 time 369935816
[08:12:59] sync: "Chapter 14" (14) at frame 107306 time 402796143
[08:17:51] sync: "Chapter 15" (15) at frame 112298 time 421534863
[08:31:27] sync: "Chapter 16" (16) at frame 124533 time 467461995
[08:40:51] sync: "Chapter 17" (17) at frame 132653 time 497942445
[08:49:27] sync: "Chapter 18" (18) at frame 142319 time 534226192
[09:10:46] sync: "Chapter 19" (19) at frame 168554 time 632705823
[09:31:11] sync: "Chapter 20" (20) at frame 189101 time 709834125
[09:31:58] reader: done. 1 scr changes
[09:32:01] work: average encoding speed for job is 17.361473 fps
[09:32:02] vfr: 191250 frames output, 0 dropped and 0 duped for CFR/PFR
[09:32:02] vfr: lost time: 0 (0 frames)
[09:32:02] vfr: gained time: 0 (0 frames) (0 not accounted for)
[09:32:02] dca-decoder done: 747814 frames, 0 decoder errors
[09:32:02] h264-decoder done: 191250 frames, 0 decoder errors
[09:32:02] sync: got 191250 frames, 191250 expected
[09:32:02] sync: framerate min 23.976 fps, max 23.976 fps, avg 23.976 fps
[09:32:02] mux: track 0, 191250 frames, 4435656148 bytes, 4448.60 kbps, fifo 8192
[09:32:02] mux: track 1, 249271 frames, 95720064 bytes, 96.00 kbps, fifo 8192
[09:32:02] Finished work at: Fri May 10 09:32:02 2024
[09:32:02] libhb: work result = 0

 # Job Completed!

What Operating System are you running?

Windows 11

What version of HandBrake are you running?

Nightly 20240506063248-fa6cf3cc8-master

Where did you download HandBrake from?

snapshot

[galad87]

HandBrake only passes the frames to svt-av1, so probably something is going wrong in svt-av1. Their multi-pass implementation never got much work.

[Mister-XY]

That means I have to create a ticket on the Av1 Gitlab site.

[Mister-XY]

ok, i have make a ticket on gitlab
https://gitlab.com/AOMediaCodec/SVT-AV1/-/issues/2179

[Mister-XY]

i have also tested it under linux with Handbrake and have the same error. The Strange with Notenoughencoder and ffmpeg and 2 pass, i did not has this error. I have take Queue instead chunk. Tomorrow, i will make another test with Notenoughencoder.

[sr55]

There are many reasons you may not see this in upstream projects.

You may need to find a way to reproduce it in order for SVT to progress it. I'm not sure they'll accept a HB log as it adds a fair number of variables that may be triggering the issue.

[galad87]

How are you using 2 pass with FFmpeg? I don't think their svt-av1 implementation supports it yet.

[Mister-XY]

i have used it with the Tool Notenoughencoder

12.05.2024 06:58:44 : ==========================================================
12.05.2024 06:58:44 : INFO  - Started Async Task - UID: mjAnfyPOjgJVRBL
12.05.2024 06:58:44 : INFO  - Input: I:\RIP\Tischlein deck dich-A1_t00.mkv
12.05.2024 06:58:44 : INFO  - Output: I:\Encode\Tischlein deck dich-A1_t00.mkv
12.05.2024 06:58:44 : INFO  - Temp Folder: I:\Temp\NEAV1E\mjAnfyPOjgJVRBL
12.05.2024 06:58:44 : ==========================================================
12.05.2024 06:58:44 : DEBUG - GetFrameCount() 
12.05.2024 06:58:44 : INFO  - GetFrameCount() => 107234
12.05.2024 06:58:44 : DEBUG - ExtractSubtitles.Extract()
12.05.2024 06:58:44 : WARN  - ExtractSubtitles.Extract() => File already exist - Resuming?
12.05.2024 06:58:44 : WARN  - Queue is being processed in Parallel
12.05.2024 06:58:44 : DEBUG - EncodeAudio.Encode()
12.05.2024 06:58:44 : INFO  - EncodeAudio.Encode() => Command:  -map 0:a:0 -c:a:0 copy -ac:a:0 2 -metadata:s:a:0 language=deu
12.05.2024 06:58:44 : DEBUG - EncodeAudio.Encode() => ExitCode: 0
12.05.2024 06:58:44 : TRACE - GetVFRTimeStamps()
12.05.2024 06:58:44 : TRACE - GetVFRTimeStamps() => return
12.05.2024 06:58:44 : TRACE - VideoEncode.Encode()
12.05.2024 06:58:44 : INFO  - VideoEncode.Encode() => Chunk: I:\RIP\Tischlein deck dich-A1_t00.mkv
12.05.2024 06:58:44 : INFO  - VideoEncode.Encode() => Command: /C ffmpeg.exe -y -i "I:\RIP\Tischlein deck dich-A1_t00.mkv"  -an -sn -map_metadata -1 -pix_fmt yuv420p10le -strict -1 -c:v libsvtav1 -b:v 1200k -preset 7 -g 245 -svtav1-params aq-mode=1:tile-columns=0:tile-rows=0:lookahead=33:film-grain=8:film-grain-denoise=0  -pass 1 -passlogfile "I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\Video\000000_stats.log" -f webm NUL
12.05.2024 06:58:44 : TRACE - VideoEncode.Encode() => Added PID: 29504  Chunk: I:\RIP\Tischlein deck dich-A1_t00.mkv
12.05.2024 07:10:32 : TRACE - VideoEncode.Encode() => Removed PID: 29504  Chunk: I:\RIP\Tischlein deck dich-A1_t00.mkv
12.05.2024 07:10:32 : INFO  - VideoEncode.Encode() 2nd Pass => Command: /C ffmpeg.exe -y -i "I:\RIP\Tischlein deck dich-A1_t00.mkv"  -an -sn -map_metadata -1 -pix_fmt yuv420p10le -strict -1 -c:v libsvtav1 -b:v 1200k -preset 7 -g 245 -svtav1-params aq-mode=1:tile-columns=0:tile-rows=0:lookahead=33:film-grain=8:film-grain-denoise=0  -pass 2 -passlogfile "I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\Video\000000_stats.log" "I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\Video\000000.webm"
12.05.2024 07:10:32 : TRACE - VideoEncode.Encode() 2nd Pass => Added PID: 16876  Chunk: I:\RIP\Tischlein deck dich-A1_t00.mkv
12.05.2024 07:22:11 : INFO  - VideoEncode.Encode() => Exit Code: 0  Chunk: I:\RIP\Tischlein deck dich-A1_t00.mkv
12.05.2024 07:22:11 : TRACE - VideoEncode.Encode() 2nd Pass => Removed PID: 16876  Chunk: I:\RIP\Tischlein deck dich-A1_t00.mkv
12.05.2024 07:22:11 : INFO  - VideoEncode.Encode() => Exit Code: 0  Chunk: I:\RIP\Tischlein deck dich-A1_t00.mkv
12.05.2024 07:22:11 : DEBUG - VideoMuxer.Concat()
12.05.2024 07:22:11 : DEBUG - VideoMuxer.Concat() => Reading Chunk Directory by *.webm files
12.05.2024 07:22:11 : DEBUG - VideoMuxer.Concat() => Writing chunks.txt ...
12.05.2024 07:22:11 : TRACE - VideoMuxer.Concat() => Wrote 0 => I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\Video\000000.webm Chunk to chunks.txt
12.05.2024 07:22:11 : DEBUG - VideoMuxer.Concat() => Wrote 1 Chunk(s) to chunks.txt
12.05.2024 07:22:11 : DEBUG - VideoMuxer.Concat() => Command: ffmpeg.exe -y -f concat -safe 0 -i "I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\chunks.txt" -c copy "I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\temp_mux.mkv"
12.05.2024 07:22:12 : DEBUG - VideoMuxer.Concat() => MuxWithMKVMerge? : True
12.05.2024 07:22:12 : DEBUG - VideoMuxer.Concat() => MuxWithFFmpeg?   : False
12.05.2024 07:22:12 : DEBUG - VideoMuxer.Concat() => AudioCommand?    : --default-track 0:yes "I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\Audio\audio.mkv"
12.05.2024 07:22:12 : DEBUG - VideoMuxer.Concat() => SubsCommand?     : 
12.05.2024 07:22:12 : DEBUG - VideoMuxer.Concat() => VFRCommand?      : 
12.05.2024 07:22:13 : DEBUG - VideoMuxer.Concat() => Exit Code: 0
12.05.2024 07:22:13 : DEBUG - VideoMuxer.Concat() => STDOUT: mkvmerge v83.0 ('Circle Of Friends') 64-bit
»I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\temp_mux.mkv«: Der Demultiplexer für das folgende Format wird benutzt: »Matroska«.
»I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\Audio\audio.mkv«: Der Demultiplexer für das folgende Format wird benutzt: »Matroska«.
»I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\temp_mux.mkv« Spur 0: Das Ausgabemodul für das folgende Format wird benutzt: »AV1«.
»I:\Temp\NEAV1E\mjAnfyPOjgJVRBL\Audio\audio.mkv« Spur 0: Das Ausgabemodul für das folgende Format wird benutzt: »AC-3«.
Die Datei »I:\Encode\Tischlein deck dich-A1_t00.mkv« wurde zum Schreiben geöffnet.
Fortschritt: 0%
Fortschritt: 45%
Fortschritt: 95%
Fortschritt: 100%
Die Cue-Einträge (der Index) werden geschrieben …
Das Multiplexen dauerte 1 Sekunde.

[galad87]

Is anything written at all in that 000000_stats.log files?

[Mister-XY]

No, is a 0 bytes file

[galad87]

Then it's probably doing the same encoding twice, with no first pass.

[Mister-XY]

From what it looks like, the error has only been present since AV1 version 1.8, which is also reflected, but the multi-pass was changed in that version.
But I'm still testing quickly.
With HB 1.7.3 the error does not exist.

edit: test done. with Version 1.8 the 2pass error is there

[Mister-XY]

I've now got the 2 pass mode with Staxrip and don't have this problem. Here is the encoding line.

SvtAv1EncApp 2.0.0-A-0+4-e8223021_Mod_by_Patman_x64_msvc1939 [SVT-AV1-PSY]

"C:\Program Files\StaxRip\Apps\FrameServer\AviSynth\ffmpeg.exe" -threads 1 -hwaccel dxva2 -i "E:\BluRay\für 2 pass\Tischlein deck dich-A1_t00.mkv" -f yuv4mpegpipe -pix_fmt yuv420p -strict -1 -loglevel fatal -hide_banner - | "C:\Program Files\StaxRip\Apps\Encoders\SvtAv1EncApp\SvtAv1EncApp.exe" --input - --width 720 --height 576 --output "I:\Temp\Tischlein deck dich-A1_t00_temp\Tischlein deck dich-A1_t00_out.ivf" --frames 107234 --pass 2 --stats "I:\Temp\Tischlein deck dich-A1_t00_temp\Tischlein deck dich-A1_t00_out_2pass.log" --rc 1 --tbr 1200 --preset 13 --tune 0 --film-grain 8

and the --stats "I:\Temp\Tischlein deck dich-A1_t00_temp\Tischlein deck dich-A1_t00_out_2pass.log file has a size from 5 MB.

[Mister-XY]

I tested again withNotenoughencoder and this time selected SVT-AV1 as the encoder. What can I say, I can't reproduce the error here either. I can't recreate this error either with Staxrip or with Notenoughencoder (this time with the correct 2 pass). I only have it on Handbrake. But only with AV1. I can't reproduce it with X265 either.
I now also have a stats log file that is approximately 5 MB in size.
The first run goes through very quickly and the second run slowly, just like with Handbrake. So it seems to be a real 2nd pass this time.

17.05.2024 05:42:34 : INFO  - VideoEncode.Encode() => Chunk: E:\BluRay\für 2 pass\Dschafars Rückkehr-N1_t00.mkv
17.05.2024 05:42:34 : INFO  - VideoEncode.Encode() => Command: /C ffmpeg.exe -y -i "E:\BluRay\für 2 pass\Dschafars Rückkehr-N1_t00.mkv"  -an -sn -map_metadata -1 -pix_fmt yuv420p10le -strict -1 -nostdin -f yuv4mpegpipe - | "C:\Users\Thomas\AppData\Roaming\NotEnoughAV1Encodes\Apps\svt-av1\SvtAv1EncApp.exe" -i stdin --rc 1 --tbr 1200 --preset 13 --tile-columns 0 --tile-rows 0 --lookahead 33 --aq-mode 2 --film-grain 8 --film-grain-denoise 0  --pass 1 --stats "I:\Temp\NEAV1E\rmFMnzHukWafRZg\Video\000000_stats.log" --output NUL
17.05.2024 05:42:34 : TRACE - VideoEncode.Encode() => Added PID: 9360  Chunk: E:\BluRay\für 2 pass\Dschafars Rückkehr-N1_t00.mkv
17.05.2024 05:44:11 : TRACE - VideoEncode.Encode() => Removed PID: 9360  Chunk: E:\BluRay\für 2 pass\Dschafars Rückkehr-N1_t00.mkv
17.05.2024 05:44:11 : INFO  - VideoEncode.Encode() 2nd Pass => Command: /C ffmpeg.exe -y -i "E:\BluRay\für 2 pass\Dschafars Rückkehr-N1_t00.mkv"  -an -sn -map_metadata -1 -pix_fmt yuv420p10le -strict -1 -nostdin -f yuv4mpegpipe - | "C:\Users\Thomas\AppData\Roaming\NotEnoughAV1Encodes\Apps\svt-av1\SvtAv1EncApp.exe" -i stdin --rc 1 --tbr 1200 --preset 13 --tile-columns 0 --tile-rows 0 --lookahead 33 --aq-mode 2 --film-grain 8 --film-grain-denoise 0  --pass 2 --stats "I:\Temp\NEAV1E\rmFMnzHukWafRZg\Video\000000_stats.log" --output "I:\Temp\NEAV1E\rmFMnzHukWafRZg\Video\000000.ivf"
17.05.2024 05:44:11 : TRACE - VideoEncode.Encode() 2nd Pass => Added PID: 16928  Chunk: E:\BluRay\für 2 pass\Dschafars Rückkehr-N1_t00.mkv
17.05.2024 05:50:36 : INFO  - VideoEncode.Encode() => Exit Code: 0  Chunk: E:\BluRay\für 2 pass\Dschafars Rückkehr-N1_t00.mkv
17.05.2024 05:50:36 : TRACE - VideoEncode.Encode() 2nd Pass => Removed PID: 16928  Chunk: E:\BluRay\für 2 pass\Dschafars Rückkehr-N1_t00.mkv
17.05.2024 05:50:36 : INFO  - VideoEncode.Encode() => Exit Code: 0  Chunk: E:\BluRay\für 2 pass\Dschafars Rückkehr-N1_t00.mkv

[galad87]

Can you compare the bitrate distribution and see if it's similar?

[Mister-XY]

That is with Handbrake

Complete name               : I:\Encode\Tischlein deck dich-A1_t00.AV1.mkv
Format                      : Matroska
Format version              : Version 4
File size                   : 712 MiB
Duration                    : 1 h 11 min
Overall bit rate            : 1 392 kb/s
Frame rate                  : 25.000 FPS
Movie name                  : Tischlein deck dich
Encoded date                : 2024-05-17 04:16:38 UTC
Writing application         : HandBrake 20240514055317-40aaf8713-master 2024051701
Writing library             : Lavf61.1.100
ErrorDetectionType          : Per level 1 

and that with notenoughencoder

Complete name               : I:\Encode\Tischlein deck dich-A1_t00.mkv
Format                      : Matroska
Format version              : Version 4
File size                   : 713 MiB
Duration                    : 1 h 11 min
Overall bit rate            : 1 394 kb/s
Frame rate                  : 25.000 FPS
Encoded date                : 2024-05-17 15:53:51 UTC
Writing application         : mkvmerge v83.0 ('Circle Of Friends') 64-bit
Writing library             : libebml v1.4.5 + libmatroska v1.7.1 / Lavf61.1.100

i have use SVT-AV1 V2.0.0

[galad87]

I mean the size of each frame, with something like bitrate viewer, or something newer, that shows a nice graph of distribution of the bits.

[Mister-XY]

you mean like this?

Handbrake

NotEnoughEncoder

[Mister-XY]

a second movie with the same end.
both

HB

NEE

[Mister-XY]

i have tested V2.1.0 and in Handbrake the error is also there, i have tested V2.1.0 wit Notenoughencoder and there is no error.

[Mister-XY]

I can say that the randomly selected 24 FPS movie has exactly the same error.
And Tune=1 doesn't bring any advantages either, it ends in the same result.

Svt[info]: SVT [version]:	SVT-AV1 Encoder Lib v2.1.0
Svt[info]: SVT [build]  :	GCC 13.2.0	 64 bit
Svt[info]: -------------------------------------------
Svt[warn]: Preset M12 is mapped to M13.
Svt[warn]: Instance 1: It is recommended to not use Film Grain for presets greater than 6 as it produces a significant compute overhead. This combination should only be used for debug purposes.
Svt[warn]: For CRF or 2PASS RC mode, the maximum needed Lookahead distance is 42. Force the look_ahead_distance to be 42
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile	tier (auto)	level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator 		: 720 / 576 / 25 / 1
Svt[info]: SVT [config]: bit-depth / color format 					: 10 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct 					: 13 / PSNR / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type 			: 256 / 16 / key frame
Svt[info]: SVT [config]: BRC mode / target bitrate (kbps)				: VBR / 1200
Svt[info]: SVT [config]: AQ mode / variance boost 					: 2 / 0
Svt[info]: SVT [config]: film grain synth / denoising / level 				: 1 / 0 / 8

and that the output from Commandline from NEE

-------------------------------------
Svt[info]: SVStream mapping:
  Stream #0:0 -> #0:0T (mpeg2video (native) -> wrapped_avframe (native))
[version]:      SVT-AV1 Encoder Lib v2.1.0
Svt[info]: SVT [build]  :       GCC 10.2.0       64 bit
Svt[info]: LIB Build date: May 18 2024 11:47:30
Svt[info]: -------------------------------------------
[yuv4mpegpipe @ 0000021da63f9c00] Warning: generating non standard YUV stream. Mjpegtools will not work.
Output #0, yuv4mpegpipe, to 'pipe:':
  Metadata:
    encoder         : Lavf61.1.100
  Chapters:
    Chapter #0:0: start 0.080000, end 577.520000
    Chapter #0:1: start 577.520000, end 1289.360000
    Chapter #0:2: start 1289.360000, end 2066.000000
    Chapter #0:3: start 2066.000000, end 2768.240000
    Chapter #0:4: start 2768.240000, end 3746.960000
    Chapter #0:5: start 3746.960000, end 4289.360000
  Stream #0:0: Video: wrapped_avframe, yuv420p10le(tv, progressive), 720x576 [SAR 16:15 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn
      Metadata:
        encoder         : Lavc61.3.100 wrapped_avframe
Svt[warn]: Instance 1: It is recommended to not use Film Grain for presets greater than 6 as it produces a significant compute overhead. This combination should only be used for debug purposes.
Svt[warn]: Lookahead distance is not long enough to get best bdrate trade off. Force the look_ahead_distance to be 42
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile   tier (auto)     level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator               : 720 / 576 / 25 / 1
Svt[info]: SVT [config]: bit-depth / color format                                       : 10 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct                                    : 13 / PSNR / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type                      : 161 / 16 / key frame
Svt[info]: SVT [config]: BRC mode / target bitrate (kbps)                               : VBR / 1200
Svt[info]: SVT [config]: AQ mode / variance boost                                       : 2 / 0
Svt[info]: SVT [config]: film grain synth / denoising / level                           : 1 / 0 / 8
Svt[info]: -------------------------------------------

i have also changed to gop 161 in HB and the error is also there

Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:	SVT-AV1 Encoder Lib v2.1.0
Svt[info]: SVT [build]  :	GCC 13.2.0	 64 bit
Svt[info]: -------------------------------------------
Svt[warn]: Preset M12 is mapped to M13.
Svt[warn]: Instance 1: It is recommended to not use Film Grain for presets greater than 6 as it produces a significant compute overhead. This combination should only be used for debug purposes.
Svt[warn]: For CRF or 2PASS RC mode, the maximum needed Lookahead distance is 42. Force the look_ahead_distance to be 42
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile	tier (auto)	level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator 		: 720 / 576 / 25 / 1
Svt[info]: SVT [config]: bit-depth / color format 					: 10 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct 					: 13 / PSNR / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type 			: 161 / 16 / key frame
Svt[info]: SVT [config]: BRC mode / target bitrate (kbps)				: VBR / 1200
Svt[info]: SVT [config]: AQ mode / variance boost 					: 2 / 0
Svt[info]: SVT [config]: film grain synth / denoising / level 				: 1 / 0 / 8

i have also changed the gop-size in NEE to 256 and have no error.
I can use the same settings from NEE in HB and also have the 2pass error.

[Mister-XY]

To be honest, I don't know what else to test and how. If I take Handbrake 1.7.3 with Av1 V1.7.0 then the multipass works. If I now compile the hb.dll myself with a checkout to version 1.7.3 with AV1 1.7.0, then I don't have the multipass error. If I now change the module.defs by deleting the patch and only change the 3 lines for version 1.8.0 of AV1, then I have the error. That means. Handbrake does not have the multipass error with AV1 version 1.7.0 and as soon as you go to AV1 1.8.0 you have the error. Either there is an error somewhere in Handbrake that version 1.7.0 still hides or you control version 1.8.0 differently. This continues from version 1.8.

[galad87]

I tried a one hour encode but it didn't exhibit the issue. I'll try something longer tomorrow.

[Mister-XY]

Here the error occurs after just a few minutes. I quickly converted the film here.
Do you use something like quicksync as a decoder? I have an AMD Ryzen 9 7900 here.

[Mister-XY]

Next, you see, theres not the problem from one movie, thats a problem of many.

[galad87]

Fixed in 4705089, forcing key frames was causing the issue, even if force_key_frames is already disabled in vbr mode.

[Mister-XY]

i will test and give a feeback

[Mister-XY]

Perfect, thank you for your patience.