How to not slip into Nihilism from Vipassana?

I am sensing this disenchantment from letting go. Meditating on impermanence feels like nihilism to me. There is a fleeting moment of joy from the liberation and just watching emotions go by. And I also notice the joy of it but I let it be and not cling. Then suddenly comes this pointlessness feeling. Something like nihilism. I also don't cling to that but it worries me I may lose control and just do things carelessly. Any guidance on this?