Having varna based system and having polluted version of varna based system are two different state of society. Based on later, one must not disregard the ideal!

ततः कालात्मको योऽसौ स चांशः कथितो हरेः !
स पातयत्यघं धोरमल्पमल्पाल्पसारवत् ।। १४ ।।
अधर्म्मबीजसंभूतं तमोलोभसमुदूभवम् ।
प्रजासु तासु मैत्रेय रागादिकमसाधकम् ।। १५ ।। -श्रीविष्णुपुराणम्

Then time, described as the portion of Vishnu, infused direful sin, which affords petty happiness. This which is the seed of unrighteousness, the source of darkness and greed, sprang forth in men, giving birth to attachment, and making all efforts futile.

येषान्तु कालरूपोऽसौ पापविन्दुर्महामते ।
चेतःसुं ववृधे चक्रुस्ते न यज्ञेषु मानसम् ।। २९ ।।
वेदवादांस्तथा वेदान् यज्ञनिष्पादकं च यत् ।
तत्सर्व्व निन्दमानास्ते यज्ञव्यासेधकारिणः ।। ३० ।।
प्रवृत्तिमार्गव्युच्छित्तिकारिणो वेदनिन्दकाः ।
दुरात्मानो दुराचारा बहूवुः कुटिलाशयाः ।। ३१ ।।
संसिद्धायान्तु वार्त्तायां प्रजाः सृष्टूवा प्रजापतिः ।
मर्य्यादां स्थापयामास यथास्थानं यथागुणाम् ।। ३२ ।।
वर्णानामाश्रमाणाञ्च धर्म्मान् धर्म्मभृतां वर ।
लोकांश्च सर्व्ववर्णानां सम्यगू धर्म्मानुपालिनाम् ।। ३३ ।।

But that drop of sin which had been created by Time increased in men’s hearts and they disregarded **Yajna. Reviling the Vedas and the prescriptions of the Vedas, the Gods and all sacrificial rites, etc., obstructing Yajnas, and cutting the Path of Pravritti, they became malignant, vicious, and perverse in their designs. The means of subsistence being provided, Prajapati, having created people, established an orderly distinction among them, according to their status and their tendencies. He fixed the duties of the castes and orders and the worlds to be attained by all the castes who perfectly fulfilled their duties.

**Yajna = It indicated democracy at Nation-level. Group activity at individual level. Democratic living and group activity – two simple meanings. Separate note is require to understand Yajna in detail.
Dominance of one class of people over others is natural human tendency, hidden in everyone. I call it Abrahamic tendency. This tendency sees surface when there is decline in righteousness. Righteousness declines due to rise in greed and ego. Greed and ego stem out of ignorance. Ignorance stems from lack of education.

What did invaders do first? Demolished educational institutes (Temples – institutes of moral education and Universities – institutes of skill based education). Rest is history.