Trust-aware reinforcement learning

Project Title:

Trust-aware reinforcement learning


Assoc Prof Peter Vamplew, Dr Cameron Foale; Co-supervisor Assoc Prof Richard Dazeley (Deakin)

Contact person and email address:

Assoc Prof Peter Vamplew (

A brief description of the project:

A key area of application for AI systems is likely to be in the form of teams or partnerships between AI agents and humans, working together to achieve a shared objective. A critical factor in the success of such teams will be the extent to which the human team members trust the AI agent.

We hypothesise that a major determinant of human trust will be the extent to which the agent’s actions are in line with the expectations of the human. That is, while an agent may find it beneficial in terms of maximising its immediate objective to execute an action which may be unexpected for its team-mates, this should only be carried out when these immediate benefits outweigh any longer-term negative impact on the team-mates’ level of trust.

This project will explore this approach using a combination of multi-objective reinforcement learning and methods for building state-specific models of human expectations.