Your friends won't think you are weird if you make a massive spastic spectacle of passing on a joint. They will just think you are a fucking asshole for not just saying "No thanks."
Also, when my friends pinned down my arms and legs and stuck the needle in my arm, it is nice to know that it was my fault because you can always say no.
My stars are for the painful realization that the only thing more annoying than Martin Short doing lame characters, is someone impersonating Martin Short doing lame characters.