Restriction, weight loss and binge

Hey,

I've gotten into restricting my intake. I still have 3 meals and two snacks, but restricted. I've seen weight loss and felt good about myself. Then, I every now and then, I binge on sweets, and it worries me that I'll gain, and I just feel awful afterwards, like I just undid all my progress.

Help!