I have this problem too. "Fn" have delay and sometimes not switching languages. It seems like floating bug. I use "caps lock" like solution.