I played on Asylum for about 2 years before I came to Olympus, so I hope my opinion means a little bit more than most. I had a lot more fun playing on Asylum than on Olympus. The admins are generally more reasonable in Asylum. Admins took care of comp requests and ban appeals a lot more efficiently. I didn't feel that the admins favored one group of people over another.
Biggest difference I would say Olympus is more cop-oriented, where Asylum is more civ-oriented. I noticed on Asylum, you can RP your way out of a ticket depending on the cop. I could get a discounted ticket if my stories was good enough and unique. Where in Olympus, it feels like it is an US vs THEM. The majority of people playing as cops on Olympus hold grudges from civ-side and bring it over when they play as a cop. Instant tickets, robotic processing is a norm on Olympus.