The hardware will always be the weakest part of the Xbox business. Gamepass has made it totally irrelevant. Even the XSX, which has advantages over the PS5, hasn't really measured up that well. XSX versions of games are rarely better performing than their PS5 counterparts, and on the occasions that they are, it's still often only incrementally superior. And if you're the type of gamer who is after the highest resolutions, textures, framerates, effects, etc., then you are much better off mainlining PC than console.
The hard fact is that Gamepass has almost completely cannibalized Xbox's hardware business. By design, there is very little reason to invest in Xbox hardware. Their games come day and date to PC, and the Gamepass footprint with its "play anywhere" mentality just makes Xbox hardware more and more of a losing propositon.
If you can only afford to buy a single console, it will almost certainly be a Switch because there is no (legal) way to play their games anywhere else. If you can either afford two consoles, or just don't really enjoy Nintendo's first party output, then you'll pickup a PS5 because while Sony's first party games do make their way to PC, it's often a two to three year gap before those games are available there and many gamers aren't willing to wait that length of time for the hot new property.
And then there is the issue of the XSS. A pretty scrappy little console early this gen when most games were cross-gen, but now that we're slowly shifting into true current generation titles, it's weaknesses are showing. And there are not an insignificant amount of XSS owners out there. It will hold back the full experience for people that are only able to play on it.
With so many ways to play Xbox first party titles without needing Xbox hardware, it feels more and more that their hardware business will eventually become a niche rather than a pillar.