The fault begins from the rich ones /lords of the towns

And thus We have set up in every town great ones of its wicked people to plot therein. But they plot not except against their ownselves, and they perceive (it) not.

And when there comes to them a sign (from Allâh) they say: "We shall not believe until we receive the like of that which the Messengers of Allâh had received." Allâh knows best with whom to place His Message. Humiliation and disgrace from Allâh and a severe torment will overtake the criminals (polytheists, sinners) for that which they used to plot.

Al an'aam 123-124