If the executor can tell, in advance, that the estate is going to involve an inordinate amount of work, or the executor fees listed in the Will cannot possibly compensate them for their time or they have concerns that the beneficiaries will not be willing to vary the fees, they have two choices:
Resign as executor
Hire a lawyer to apply to the Court to have their fees increased.