What is the opinion of Buddhism on the caste system and untouchability of Hinduism?

The Varnashrama dharma (caste system) is the backbone of Brahminism (Hinduism), untouchability is a part of that social order. We all know that Gautama Buddha was against Varnashrama dharma, and rejected the Vedic religion. But unfortunately, we see caste system and untouchability in many 'Buddhist' countries. For example, the Burakumin/Eta in Japan, Baekjeong in Korea, Ragyabpa in Tibet, etc. As we know as Buddhists that the Buddha was no hindu, then why this system is also in the societies of Buddhist countries?