A lot of times now, I've clicked on an image to bring up the lightbox, then habitually hit the back button on my mouse to get out of it.
I think I'm used to how Facebook handles this -- if you hit the back button in the lightbox on Facebook, it simply closes the lightbox. On Phabricator, it takes me back an entire page.
I think Facebook implements this with the HTML 5 history API (but I could be wrong here - I'm not sure how else it would be done, but I'm sure there are other ways, and I haven't looked into it).
Would it be reasonable to explore using that here? I guess this sort of deps on T2086 if we do use the history API?