I'm 6'6" and the large fits me well, but if I was any shorter I would say that it would be too long. If you're shorter than I am I would entertain the idea of a medium, especially if you don't have a long torso.
Hope this helps. My suggestion would be to order both so you can be sure about the fit if you're still on the fence.