Calculating the Weather

Since my time as a Senior Forecaster ending in the late 70s, improvements in NWP are quite astounding. However, forecasts are still far from perfect; small scale detail, such as we sailors would like to know about, remain a problem. The greatest improvements have taken place in periods up to a week ahead; most useful for planning and decision making.

However, I am jumping ahead; what is NWP? First and foremost, it is not a statistical process, nor is it pattern matching. Those have been tried and, quite simply, do not work. Nor do techniques based on planetary movements and sunspots.

It was in 1906 that a Norwegian physicist and meteorologist, Vilhelm Bjerknes, said that weather prediction could be a genuinely scientific process; the laws of physics, mathematically expressed, could be used to calculate the future state of the atmosphere from known initial conditions; like a physician making a medical prognosis, the forecaster has to start from as accurate a diagnosis as possible.

In 1906, computing meant hand operated adding machines, slide rules and log tables (does anyone use those these days?) It was before he and his colleagues in Bergen had developed the concept of weather fronts; meteorology was still in its infancy and this must be one of the best ever long range weather forecasts!

A simplified description of how the weather machine works is as follows:

• Heating and cooling of the earth varies with time of day, time of year, latitude and nature of the surface. Oceans will heat up and cool down very slowly. Deserts and other dry areas will heat up and cool down quickly. Conurbations, prairies forests and so on will all behave in different ways. Slopes facing the sun will heat up quicker than those facing away from the sun.
• Differential heating creates different air temperatures leading to pressure differences and, therefore, winds.
• Air movement causes a redistribution of pressure leading to changes in winds.
• Cooling of the air occurs when air is forced to rise by flowing over hills, by convection or by air streams converging. Cooling also can occur when the air is in contact with cold ground.
• As the air cools it is able to hold less water vapour; this leads to condensation and release of latent heat. If the air cools further, the drops may freeze and release more latent heat. These effects warm the air.

• If the ice melts then heat is needed. This comes from the air so cooling takes place. Again, if the water drops evaporate, then latent heat is required from the air and the air is cooled.
• These latent heat effects change the air temperatures and therefore the pressure patterns and, hence, the winds.
• Drops of water in the air form clouds which cut off the radiation from the sun and reflects it back out to space. But, cloud also absorbs radiation from the ground and re-radiates it back to earth.
• Topography affects weather on all scales; sea breezes, headlands, cliffs, valleys and hills on a local scale; mountain ranges, such as the Rockies and the Urals on the large scale. On even larger scale, continents and oceans play their part.

All these physical processes can be expressed mathematically and to form the basis of modern forecasting. Bjerknes was the first to realise this.

During World War 1, an outstanding British mathematician, Lewis Fry Richardson, a Quaker pacifist and a stretcher bearer in France, tried to test Bjerknes' ideas by calculating the pressure change 6 hours ahead at a single place in central Europe.

He got the answer wrong by a factor of 100 but he still published a book (in 1922) saying what he had done and why. Richardson had neither the data, the computing power nor the mathematical techniques necessary.

In his book, he envisaged forecast offices as spherical buildings, rather like the Albert Hall so as to simulate the earth, covered by people all doing sums at the same time and passing the results to near neighbours for the next round of calculation.

Conceptually, this was another remarkable piece of foresight, and is a description of the super-computer of today; parallel processing is not a new idea.

After WW2, Bjerknes' and Richardson's ideas were dusted off and, since then, meteorologists have been keeping ahead of the computer designers by specifying needs for greater and greater computing power. Massively parallel processors are fulfilling Bjerknes' dreams.

The real problem lies in putting numbers to the many equations. This is impossible to do precisely and many approximations or estimates have to be made and some very clever mathematical computing techniques used.

That Richardson's brave attempt failed is hardly surprising. Many of the advances in forecasting over the past 30 or more years have resulted from improvements in the estimates and approximations. Weather prediction, whether for hours, days or centuries remains the largest, most complex computational problem facing mankind.

Calculations are made from values of all the various parameters on a 3-D grid. The horizontal grid spacing defines the size of weather feature that can be represented. Roughly, models can only describe weather or topographic features about 4 or 5 times the grid length.

To cover the whole globe, as is necessary for prediction of large scale weather for the next few days models now (mid-2010) use grid lengths around 25 km or 1/4 of a degree latitude. To model smaller scale weather for the next few hours up to about 36 hours, needs grid spacings of 1.5 to 12 km.

It is necessary to calculate over the whole globe for two reasons. First, weather can travel a long way in a day; fast moving Atlantic lows may have speeds up to 60 knots.

Secondly, there are "tele-connections." Well known, large scale examples are the El Niño and La Niña which seem to have long range effects on weather in areas far removed. A simple analogue might be a half inflated air-bed. Push down in one place and it will pop up somewhere else.

The UK Met Office global grid, 787456 grid points, 70 levels at each from 20 m to 80 km above the surface of the earth with 25 km spacing.

Starting from the best analysis possible at, say, 00 UTC, the equations used to describe all the physical processes are used to provide estimate of the rate at which changes are taking place. This is done at nearly 800,000 points at each of 70 levels up to a height of 80 km. From the rates of change in each meteorological element, values are estimated 15 minutes ahead.

The new data are then used to predict for the next 15 minutes and so on up to, in the case of the Met Office, 5 days ahead. As a user, experience is that forecasts over this period are usually good in general terms. Errors become more obvious as time goes on. See my page on limitations to forecast accuracy.

Limited area models are used nested in the global model but with smaller grid lengths and shorter time steps. Thus limited area or meso-scale models have input on the large scale combined with data analysed on the same grid ie resolution of the models. As far as I am aware, no commercial company providing meso-scale forecasts makes such an analysis.

Another major problem that seems to be ignored by some providers of these fine scale forecasts is that small weather details have short lifetimes. Output should only be used as general guidance.

It is a characteristic of the atmosphere that small disturbances can grow into major systems. This effect is known as chaos and there have been suggestions that a butterfly flapping its wings in Tokyo could result in a storm in Washington.

The answer is NO, but hurricanes do start from small groups of thunderstorms. Atlantic lows often start as imperceptible waves on a front of the US Seaboard. Thunderstorms grow rapidly from local heating of the surface. It is all rather like trying to predict where the first bubble will rise from a pan of water - before you have lit the gas.

Despite all the effort put into data assimilation there are always uncertainties in detail. This has the result that, at some stage, any forecast calculation will go wrong when the effects of errors in the data analyses manifest themselves.

Major weather services, such as the UK Met Office, tackle this by starting with best analysis possible to produce what is called a deterministic forecast. They then run the forecast several times with small, random, variations in the data. If all the answers are similar then the forecaster is confident. If the answers diverge then there is clearly some doubt.

For large scale weather here lows and highs can have lifetimes lasting days, the deterministic forecast works pretty well up to about 3 - 4 days. After that the effects of chaos become increasingly noticeable and forecasts less reliable.

On smaller scales, weather features of about 100 - 130 km size should be predictable up to a day or so ahead. Smaller still, sea breezes and showers or thunderstorms may be only a few tens of km size. Their development will depend on equally small weather such as cloud cover. In effect they will not be predicted well by any deterministic forecast. These really require an ensemble approach, not yet available to end users. Consequently, any small (meso-) scale forecast has to be use as an indication and not taken too literally.

We see the use of ensembles on the large scale when the forecaster on TV may be highly confident about the next few days on some occasions and cautious on others. For short term use, meso-scale ensembles can give useful guidance on the likelihood of rain turning to snow. In my day, that was always a nightmare.

Reading the previous section, I hope that the reasons for what we all observe is becoming clearer. That is, that forecasts of general weather type for the next few days - perhaps up to five days, or more - are remarkably good. To my mind, as a user of forecasts this has been the major and most obvious improvement since 1977 when I last worked as a Met Office Senior Forecaster at the then Central Forecast Office at Bracknell (now the National Meteorological Centre, Exeter).

It also follows from the above section that very local, detailed forecasting can only be possible for very short periods ahead. How far ahead and what level of detail local forecasting will be possible is, as yet, uncertain. Watch this space!

For the racing yachtsman, whether in dinghies or round-the-cans yachts it will be necessary for years to come to use careful observation, coupled with observation and experience. Anyone who has heard my old friend and colleague David Houghton will be aware of what can be done along those lines.

For those with very deep pockets, such as America's Cup contenders there might well be some advantages in using very small scale models if there are the data to support them Such modelling is beyond my experience and not relevant to the majority of sailor users of forecasts.

However, fairly low cost meso-scale forecasts are now available from a number of sources, Racing sailors should try them and see what benefits they can get. In theory, they should help but that is a judgement that can only be based on experience.

For the coastal cruising yachtsman, the general forecasts give excellent advice for the next few days. Used properly, sailors should be able to avoid getting caught out in really bad weather. They should also be able to be in port or good sheltered anchorages when bad weather comes. Importantly, they should be able to avoid being in a position when they are in places that they do not want to be in, in weather that they do not want to go out in.

The blue water sailor making passages lasting weeks can only plan using climatology. His ability to avoid bad weather once out there is strictly limited. His only real tactic is to be aware of warnings and prepare for bad weather before it comes. He has to be fully equipped to do so.

Using computer generated forecasts, it is only sensible to keep an eye on what a human forecaster says. He or she will have seen output from ensembles and will convey the uncertainties in the broadcast texts. See also my page on limitations to accuracy of forecasts.

The amount of information that it is and will be possible for forecasters to provide is enormous. Serious thought will have to be given to how the user will access and use it. Onshore is one matter with broadband communications. Afloat is another. How many sailors will wish to be stuck down below, nose, to a laptop computer?

When will high speed communications become affordable for access over radio systems?

One thing is certain and that is that the written or spoken word will not and cannot cope with all the variations in the weather that exist, whether they are well forecast or not.