Sure - I am on the Win11 beta channel. I was on build 22598.200 when writing my post, and Start11 v1.22. However when checking my Windows version, I noticed that I had a preview release pending download - I have installed it and am now on build 22610.100. I still have the same problem.
I did notice that if I close a couple of windows, the scroll bar will go away. It seems to happen if there are either 11 or 12 windows open. Once a certain number of windows are open, the scroll bar turns on, regardless of how much space is actually available.
I think I can pretty clearly see what's happening here - once the taskbar fills up with the windows at full size, each task shrinks to half size. When it does this, that is when the scroll bar appears, even though there is now plenty of space. But if the windows were still at full size, they would have had to spill onto a 3rd line. So whatever triggers the scroll bar isn't able to understand that the size allotted to each window shrank, and that I no longer need 3 lines to hold them all.