Maybe by 2040 Esports will have taken over, and actual real racing of physical bikes and cars will be seen as quaint and unbelievably dangerous and 'did people really do that'. In Esports, battery sizes and available energy are completely irrelevant as you can simply make up whatever physics you want.
Note that I'm not promoting this as an eventuality. Just saying that it may happen.
I'm not sure of the reasons - and see many different theories online - but it appears that the Japanese factories have been less agile and less creative than the European factories. If this is the root cause of the Japanese manufacturers falling behind, then even if certain technologies are banned, then it's likely to be the European factories that adapt to new regulations and opportunities quicker. So, while I feel that the current concessions will help Yamaha catch up (not yet sure about HRC), then unless the underlying problem is addressed then they'll fall behind again.
I've seen articles saying that the Japanese teams have two problems - most of the good engineers are in Europe, and are more likely to sign up to companies that allow them to work in Europe. And, with the teams being closer to the majority of the actual races, that it's easier for engineers to visit races and see the bikes in action. Note that I don't necessarily think that these are the correct descriptions of the reasons for falling behind, but put them forward for discussion. Speaking personally - I'm not sure that I know and/or understand the true reasons. But, unless the underlying causes are identified and addressed, any solution to the problem is likely to be temporary.
In hindsight, Ducati homologating their factory bikes as “Open class” that I mentioned previously, was the pivotal moment in the sport.
It was a Gigi decision, no doubt inspired by F1 designers who read the rulebook a thousand times and work their way around it. It caught the Japanese off guard. Sure, it was legal, but it’s not how they operated. Honor was the norm (recollect how the Japanese factories adhered to the minimum tyre pressure law, even when they knew their riders will have to fight in packs. Ducati ignored it with merry, because there was no penalty for it until last year).
Even poaching engineers from another Japanese brand was unheard of, because families worked generationally at these factories.
Fast forward to now, and all those things Ducati started; considered rude and unsportsmanlike back then, are the norm now. Even mighty HRC buckled a couple of years ago and hired a few Magneti Marelli engineers; a move that’s akin to Seppuku in the days gone by.
What you said about good engineers based in Europe is partially true. There are excellent engineers in Europe and Japan, if you’re talking skillset. However, the Japanese engineers are indoctrinated to follow the beaten path (Often guided by a mentor for several years at the start of their career) whereas European engineers are culturally rebellious and are open to exploring new avenues.
The reason Yamaha is recovering faster than HRC is also this. Yamaha’s MotoGP outfit has been based in Italy for some time now, which I believe was driven by Lin Jarvis and Davide Brivio. This team is now almost independent, co-led by Gigi’s former right hand man and the Japanese project leader (Unheard of a few years ago). Whereas the might of HRC’s talent base is still Tokyo based. Which worked in the old norm when you nail the spec of your motorcycle at the start of the season and introduce slight updates throughout the season (And usually one major update mid season). Not now, when the Europeans are bringing major parts every 3-4 races. Suzuki incidentally was a hybrid setup, and it worked perfectly because Davide Brivio in Europe and Sahara-San in Hiroshima were on the exact same wavelength. That quickly became undone when Brivio left and Sahara-san moved to Europe to start Suzuki’s disastrous “Decision by committee” era, which eventually led to their withdrawal.
The C*VID years only accelerated the decline when the Japanese race teams had to discuss their feedback over zoom with the factories (a lot getting lost in translation in the process) while the Europeans continued working as usual. The implosion of the Moto2 relationship of the RW racing team (Dutch) and the chassis manufacturer NTS (Japanese) happened for the same reason.
To HRC’s credit, they did recognize the importance of having. Europe based test team earlier than others. Rossi, in his final years at Yamaha also managed to force Iwata to change their ways, which started with the Jonas Folger experiment, and is now a full fledged test team built around Cal Crutchlow.
So maybe, leopards can change their spots. Fingers crossed.