Does buddhism have a concept of 'forgetting/deterioration' of spiritual 'ability'?

So, could attainments 'rot' / be forgotten, and is this a commonly observed thing? Does the Buddha warn about taking 'breaks', or does he talk about people leaving spiritual practice and as a result losing things? Is there a sort of 'wagging the finger' attitude from high attainers/authorities toward people sidelining spiritual practice for a while?