However, there's a little more at play in many cases. A lot of times, what people think of as their own rules aren't actually theirs. Many rules are imposed by other people, such as parents, or by society as a whole. These can be destructive or even lethal. If you can change it simply by deciding to do so ("I'm done eating meat; I'm a vegetarian now") then it's your rule. If you can't, then it probably came from somewhere else and will take a lot more work to fix if it's hurting you or just preventing you from pursuing happiness. The Com-Passionate Boundaries program brings this up in a flowchart of sexual hijinks safety.
Society does not have a right to hurt you or deprive you of happiness just because it doesn't like what you are, so long as your choices don't hurt people.