This is an issue with both servers. They both have seesawed back and forth over the past few years player count wise. It's nothing new and it's super random. There was a point Asylum only had 30 players max and now they're back at 135 daily..
There's really no answer for this other then Arma 3 is a 10yr old game and people switch up servers between Altis Life, KoTH, Wasteland, etc. It's natural. Sure patch notes matter and other small shit contributes but BARELY. The staff give you BS answers because there's really no direct answer.
Asylum and Oly will never go back to the point where we had 4+ servers running against each other at max pop unless Arma 4 happens