The issue with this is, as with the original version, who has these rights? (in your phrasing, who's "one"?)
What about those who seek to undermine others' rights under this framework? It's a "paradox of tolerance" situation.
Your point about societal enforcement is pertinent, but flawed, when considering the disproportionality of power and resources at play.
Encoding reciprocity and the mutual upholding of one another's rights is necessary for such a system to work adequately.