Of course it is very limiting to be labeled a lesbian or queer writer. We live in a homophobic culture, and even people who aren't hateful per se assume they won't get anything from a queer book.
There is a huge difference between writing a book, which is a private activity I engage in with myself, and wanting to engage in overly intimate personal conversations with strangers, which I pretty much never want to do.