Like in Differential, comments are drafts until the author submits them by submitting the form at the bottom of the page. They'll only be published to other users after that.

We don't do a very good job of cueing this in the UI right now.

@epriestley, okay just now I realised the button is actually 'save draft'. But I don't see the submit button? that's strange...

It's "Add Comment" at the very bottom of the page.

@epriestley, i see what you meant by cueing the UI now. I will not link 'Add comment' to 'submit my draft'... ever

I would suggest to show the inline comment together with existing comments, then a button next to the comment saying 'publish' or 'submit'