Most incidents of "frisky cattle" have involved walkers with dogs crossing fields with calves in. Take away these two points and problems are very, very rare.
I confess that nowadays I sometimes look for which is the best escape route before I enter a field containing calves and I have occasionally detoured around a field to avoid them if I have sensed that they have young and may approach me. It is not something that I am unduly worried about but more something that I am aware of.
There are certainly rogue individual cows especially amongst highland cattle. If you are harassed by one it is important, if possible, to let the farmer know. Aggression has been bred out of them and any showing such tendencies are quickly despatched to Mr MacDonalds