I have a weird issue

I noticed it 2 times now :
1- First time : 7/8 days : 0% remaining : couldn't play, had to wait for charge...

2- second time (today) : 4/5 days, 15% remaining...
Is it normal ?
Should I remove the battery each I'm not using the Retrostone ?
