You're probably losing muscle which will leave you with a skeleton-ish looking body. This is a good way to screw-up your metabolism. " I don't intend to stop " = sounds like a relapse to me.
Why not try to gain muscle? Lifting weights is fun! That way you look good, feel good, make your body happy, and stay at a healthy weight while having a healthy lifestyle!