Gear 2nd is just a general upgrade, with no real downside unless he uses it for too long, But Gear 3rd is much more Luffy putting everything he has into winning, and there is (or was) a massive penalty for using it. I prefer 3rd.
If you notice after the timeskip he seems to use Gear Third more than he used to (for obvious reasons). It is kinda hard to compare because one is meant for speed while the other is meant for power.They are used for different things. I prefer second but that doesn't mean its better.