So at the moment it seems the AI never redirects airships once the orders are given. So you can tease the AI to attack you by starting to move your whole fleet away, then move them back and easily wipe out all of their fleets with the support of defences.
It's hard to know exactly how this should be fixed, as a stalemate of no-one moving isn't much fun either.
Yeah, that's definitely a problem that needs addressing. Interestingly, the "stalemate of no one moving" has a name: the fleet in being, which is an important if controversial part of naval doctrine.