The letter from Farenthold, who stepped down from Congress last month following publicized allegations of sexual harassment, came in response to a request from Texas GOP Gov. Greg Abbott.
"Since I didn't call it and don't think it's necessary, I shouldn't be asked to pay for it," Farenthold wrote of the special election.
Ciara Matthews, deputy communications director for Abbott, said in a statement that Farenthold's decision is "disappointing."
"Mr. Farenthold's decision is disappointing, but it's not surprising that his last act would be to stick taxpayers with the bill at the worst possible time," the statement read. "While Mr. Farenthold may consider this resolved, we're not closing the case on this issue."
Abbott referenced the $84,000 figure in his letter to Farenthold last week, noting there is "no legal recourse requiring" Farenthold to repay the money.
"I am urging you to give those funds back to the counties in your district to cover the costs of the June