Updated at 6:14 p.m.

A week after he was arrested on sex charges, Sen. Norm McAllister (R-Franklin) left a message for Lt. Gov. Phil Scott Thursday morning saying he would not resign, according to the lieutenant governor.

In response, Scott and his fellow members of the Senate Committee on Committees voted Thursday afternoon to strip McAllister of his committee assignments.

The lieutenant governor announced the shakeup late Thursday afternoon from the Senate dais. He read a letter he wrote to McAllister saying he was “extremely disappointed” in the Franklin County Republican’s decision to remain in office.

“I truly hope you will reconsider,” Scott said in the letter.

The Committee on Committees includes Scott, Senate President Pro Tem John Campbell (D-Windsor) and Sen. Dick Mazza (D-Grand Isle). Campbell told the body the decision was unanimous.

“It was a long discussion and a difficult discussion, but it was one we felt was in the best interest of the state of Vermont,” Campbell said.

Scott announced that Mazza would replace McAllister on the Senate Committee on Agriculture, while Campbell would replace him on the Senate Committee on Insitutions.

Reached at his home Thursday afternoon, McAllister declined to comment. His attorney, Brooks McArthur, has not returned phone calls.

In an interview after the Senate recessed, Scott said he would consider acting to remove McAllister from the body. The Vermont Constitution empowers the Senate to expel its own members but does not detail a process for doing so.

“Expulsion could be a next step, but personally I feel as though we should await the proceedings, allow this to work out in court and then monitor the situation just to see what happens now,” Scott said. “But that could be the next step.”

McAllister pleaded not guilty last Friday in Franklin Superior Court to three felony counts of sexual assault and three misdemeanor counts of prohibited acts.

Authorities say McAllister sought sex for rent from two women, forced one of them to perform sexual acts and attempted to prostitute that woman to local farmworkers. The police say he sexually assaulted a third woman, who worked on his goat farm and later in Montpelier, for a period of years.

Seven Days reported Sunday that the third woman was a 20-year-old former Statehouse assistant who alleges she was 16 or younger when she was first allegedly assaulted.

McAllister, who was released on $20,000 bail Thursday night, has not appeared at the Statehouse since his arrest.