No, without Invulnerables, you can end up in the situation where everyone deregisters* and the parachain cannot produce a block. Because the economic incentive is low, it’s a reasonable assumption that one actor could buy all the permissionless slots and perform such an attack. The economic incentives are low because we don’t want to charge users (either via inflation or via higher transaction fees) to pay for collation since it is not really a secure activity and it doesn’t scale (paying validators makes more sense because they secure all parachains, not just one).
I understand that you are probably right, most of the time, in that there is enough interest for slots that people will take them, but that is not the case we are designing for. We are designing to avoid the possibility of the chain completely stalling even once. Given the economic weakness of the permissionless slots (and assumption that a single entity could acquire them all), the Invulnerables are meant to guarantee a diverse group of known collators on the assumption that at least one of them will always be online and able to produce a block. Note that economically this is very different from validation, where slash size increases with the number of simultaneous offenders, so as one entity acquires enough validators to perform such an attack they are exposing themselves to a 100% slash.
As stated in the RFC, the goal is not to make collation profitable. There are other reasons to collate on system chains, like ensuring application liveness. While I recognize that costs like 4,000 USD/month are expensive for an individual, if you are e.g. Tether/Circle and want to make sure that USDT/C are working, this is not a big expense and there are much better reasons to pay it than hoping to recoup it in fees.
Rather than try to reduce the number of Invulnerables, I would recommend lobbying to increase the number of permissionless slots on the chains.
*Not technically true, the pallet won’t let the last candidate deregister, but the last candidate could just not run their node.