How Many Hidden Layers to Use in Leela Chess Zero? #1887

mooskagh · 2023-05-13T22:40:30Z

ginkgo20
May 13, 2023

I want to know about Leela architecture. Is there a huge difference between Alpha0 & Leela0 in architecture?

Answered by mooskagh

Initially, the topology of the Lc0 network was exactly the same as a0.

However, with time, some changes were introduced. Particularly,

Number of residual blocks changed over time, and the most recent resnet-based nets have 15 blocks (of 2 convolutional layers each).
Lc0 uses Squeeze-and-Excitation blocks instead of pure resnet.
Policy head is convolutional now rather than fully connected.
Value head returns W,D,L rather than a single V value.
Moves-left-head was not present in the a0 paper at all.

Moreover, the most recent and strongest networks are not resnet-based at all, they are transformer based.

ginkgo20 · 2023-08-30T23:32:12Z

What?!

mooskagh · 2023-08-31T06:53:40Z

Initially, the topology of the Lc0 network was exactly the same as a0.

However, with time, some changes were introduced. Particularly,

Number of residual blocks changed over time, and the most recent resnet-based nets have 15 blocks (of 2 convolutional layers each).
Lc0 uses Squeeze-and-Excitation blocks instead of pure resnet.
Policy head is convolutional now rather than fully connected.
Value head returns W,D,L rather than a single V value.
Moves-left-head was not present in the a0 paper at all.

Moreover, the most recent and strongest networks are not resnet-based at all, they are transformer based.

0 replies