It is difficult to avoid priviledge especially when it comes to linguistics. Some of my students speak AAVE and this disadvantages them as many people see this as a sign of lack of intellect. By myself and other speaking a higher register dialect* of English (so called "proper English") we are taking advantage of this priviledge. I have myself spent years trying to learn the local white-colar educated register so that my scientific ideas will be taken seriously. The rule against split-infinitives likely has roots in classism. Unfortunately I don't see an easy solution without asking someone to give up their language. For my part I try promoting descriptivism instead of prescriptivism (belief in solecisms, barbarisms, and the idea that a language should be spoken/writen/etc a particular way).
*Note the difference between dialect and language is political not linguistic. It has been said a language is a dialect with an army of navy.