Church discipline has been a feature in every Reformed church from the beginning. It is a Biblical precept. What you are complaining about is not something that you can prevent either. If someone doesn't like your church's teaching, they will leave and start a new one. Does that mean your church is to blame for the apostasy? Likewise, the Bible is not to blame for any apostate, whether they be individuals or churches.
These days, yes someone who is Orthodox could leave the Orthodox church and start their own thing if they could find enough support for it, because that is the state of affairs since the Reformation, where anyone can just start their own denomination, or weird cult or whatever. Prior to that if you wanted to do that, you'd be unlikely to get as far. Arianism for example was stamped out, because the Church had the authority and power structure to ensure that heresy could be dealt with effectively. And now it's back, because that no longer exists.
The world where there are denominations and this is a perfectly acceptable state of affairs is a modern phenomena. If you lived in Anglo Saxon Britain and you didn't like the church, starting your own was not an option that would get you very far.