James continues from chapter 4:1 speaking of infighting in the church, there is spoke generally about quarrels and fights. In v 11, James zones in on those who slander, speak evil against their own brethren and judge one another in a condescending way. He points to the just judge, the Lawgiver, who we know is Christ alone.
