Gas Taxes, Subsidies

So we complain because we give oil companies tax subsidies and then high taxes on consumers at the pump. Let's think for a minute, why don't we just lower taxes at the pump and stop given subsidies? If we don't want to give money to oil companies (and since oil prices aren't driven by domestic producers) then we shouldn't. The only way to directly lower the cost at the pump is to lower taxes directly. Effectively this diverst from benefits from the companies to the consumers.