Your bag and tent should keep you warm enough - even if you get hit by some nasty weather. Keep in mind, though, that you won't be in your bag in your tent the entire time the sun is down. You'll be setting up, taking down, eating, peeing, getting ready to head for the summit, etc.
Travelling light is awesome; but, in my opinion, being at Trail Camp overnight without warm clothes would be miserable.