I further inspected my code. If I removed the bidirectional layer, the GPU computing speed returned to normal (keras 3) for the activation function, I saw posts that said only using 'tanh' will enable CuDNNLSTM, which is faster than lstm.