With that statement you show your disregard of the others sexuality and desires.
Just as you are who you are in your sexual identity and preferences, they are who they are. Calling someone transphobic because they don’t physically desire you is a problem of its own.
If you want people to accept you as you are you must also accept them as they are.