Words as Walls

So why do some people still feel that derogatory language against disabled people is less of an issue than language used against other minority groups?