What someone wants may, or may not be morally relevant, and if relevant may be moral or immoral. So while the overall question "What ought one do?" might have a large answer (John Stuart Mills thought the theory of Utilitarianism was the answer), I have no reason to presume the answer depends on what I want or to presume any other hidden purpose.
This core of disagreement might not be resolvable. And that is okay.