Carr is on tour with his Terribly Funny show which critics claim contained the gag.
Little People UK – co-founded and headed by Star Wars actor Davis and his wife Samantha – was incensed by Carr and contacted his agents to complain.
The charity, which supports people with dwarfism, said it took action after staff received messages from people with the condition who were offended by the jibe.
In a statement, it said: “People with dwarfism simply want to live their lives just like everyone else – going to school, shopping, walking down the street and going to work, sometimes in highly respected professions.
“To potentially hear a member of the public repeat such a comment, after seeing a stand-up show, can affect them for their entire life.
“This causes unnecessary distress and anxiety through no fault of the person with dwarfism, simply because they have a genetic difference.
Jimmy Carr made the joke during his Terribly Funny stand up tour (Image: SWNS)
