The Warlords System was meant to establish a balance, forming the Three Powers concept. However, if you take a good look, they could've take down the Emperors with great operations.
For this demonstration, I'mma consider Whitebeard as the weakest Emperor.
If the Warlords and Admirals attack the Emperors without warning they could take them down easily. For example:
Whitebeard Pirates X Admirals & Seven Warlords
Whitebeard < Aokiji, Akainu & Kizaru.
Marco < Mihawk.
Ace < Crocodile.
Jozu < Kuma.
Commanders (12) < Hancock, Doffy & Moria.
Big Mom Pirates X Admirals & Seven Warlords
Big Mom < Aokiji, Akainu & Kizaru.
Katakuri < Mihawk.
Cracker < Crocodile.
Smoothie < Hancock.
Big Mom sons (many) < Kuma, Jinbe, Doffy & Moria.
Beasts Pirates X Admirals & Seven Warlords
Kaidou < Aokiji, Akainu & Kizaru.
King & Queen < Mihawk.
Jack < Jinbe.
Tobiroppo < Kuma, Hancock & Crocodile.
Hawkins & Apoo < Doflamingo.
I don't know if they'd be able to take down the Red Hair Pirates since Sengoku introduced Shanks' crew as inexpugnable, and Mihawk would probably refuse to fight Shanks' crew as Jinbe did against Whitebeard.