Yes, there's a difference between quid pro quo sexual harassment and someone mooning over you in the workplace. The first is illegal and you have a lot of legal protection in dealing with it. The second is uncomfortable and, while it could eventually turn into a sexual harassment situation, does not necessarily involve the same legal protections while it's simply a possible romantic interest you don't return.
One expert on managing people in the workplace notes that simply being noncommittal all the time to avoid negative issues doesn't work. If someone indicates interest in you or asks you out, there are only so many excuses you can make. Certainly, you can start with this tactic and hope they get the hint, but not everyone does, and continuing to avoid direct communication can even hedge the situation into harassment territory.