Improving AI Guard Rails with Moral Trust

Abstract

Given the complexity of the world, AI guard rails cannot accurately evaluate every possible situation or action. Thus, agents will often have to adopt what others believe is the right thing to do i.e., adopt their norms. However, agents should still do what they can to ensure these uncertified norms are correct. Here we tackle this problem by formalizing a notion of moral trust. That is, when an agent's guard rails fail, they should only adopt other agents' norms when the agents have proven to be morally trustworthy. In this paper we present a set of default moral trust values, analyze moral inconsistencies and how they should weigh on these values, define moral trust gradients and functions, and incorporate moral trust into a norm adoption formalism. We then demonstrate with examples that our formalism makes norm adoption safer, and thus AI guard rails more robust.