yeah, it seems that it’s my problem.
For a very long file, it doesn’t seem to try to scroll further, visually.
I have found that instead of sliding up my fingers and letting go of the fingers(on the trackpad), if I stop without letting go of them, it doesn’t change the font size, even if cmd is pressed later. It looks like the scrolling effect lingers. I will close this.