yep, it is like that, and that's the correct behaviour. speed is not set on per-pattern basis, but globally - once you the tracker gets to an F command, the speed is set up according to it, until it finds another F command overriding. the "song" bpm & spd values are just meant to set up the tracker to a certain state once, when a song is loaded.
this behaviour is also with many other effects: notes continue to play over pattern boarders, untill another note is set in that channel. portamento, vib, etc. parameters get stored for the channel, and remembered for these effects beyond pattern boarders if you use these effects with zero parameters. global volume, once changed, won't automagically be restored when you skip back in the tune, or restart from the beginning after stopping, unless it is set to the desired level on the row of the pattern you're starting playback with.
that means you have to keep track of the speeds, levels and parameters yourself, so if you enter another speed in pattern 2, the speed is applied until you change it again. just use an Fxx command on first row of pattern 1, too, so it's always playbacked at correct speed.
all those quirks are of course ultra-user/newbie unfriendly, but it is also part of the art of tracking to "abuse" them wherever possible to get unusual results from unusual pattern data. a bit like coding a microchip with assembler, and optimizing the code.
it would be possible to change the code of the tracker, so speed changes are remembered/set up per pattern, but that would break compatibility with lots of tunes. I think there's also lots of "insane" modules that rely on that behaviour. simplest example would be a song with a slow and a fast part, using the same fill patterns in between, and two "initial" patterns where the speed is set once.