Car stumbling, stalling, and surging...?

96 Dodge Avenger 2.5L v6. No vacuum leak, catalytic converter has been gutted, and my mechanic and myself have been all over the air intake, and cannot find a problem with anything there.

It started as rough starts. The car would turn over fine, but its would stumble at startup, revving up to about 1000, instead of the usual 1600, and then stumble down to about 500, and then fluctuate until it sat at normal idle around 600rpm. Ideally it should be shooting up to 1600rpm and then sit around 600-800. Now, occasionally the car will completely stall, revving to about 800rpm before completely dying. Giving it a small shot of gas (or air, as some smart *** will tell me the accelerator gives the car air and not has) seems to help it level out.

Also, sometimes while accelerating at low speeds, such as backing out of a parking spot, or driving at town speed limits, the car will surge. At highway speeds there is a significant loss of power, but no surging.

I think it is fuel related. We tried adding some injector cleaner, which didn't seem to help much. My thoughts are that it is a plugged fuel filter. After driving the car for a while, a good hour or so, and then turning it off it seems to start completely fine. Anyone have any thoughts besides the fuel filter/fuel pump? No CEL either.