Calculating the Weather

Many sailors will have realized that Numerical Weather Prediction has produced great advances in weather prediction. Since my time as a Senior Forecaster ending in the late 70s, the improvements are quite astounding. However, forecasts are still far from perfect; small scale detail, such as we sailors would like to know about, can still be lacking. The greatest improvements have taken place in predictions over periods up to a week ahead. These are most useful for planning purposes and help greatly in decision making.

However, I am jumping ahead; what is NWP? First and foremost, it is not a statistical process, nor is it pattern matching. Those have been tried, especially for long range forecasting, and, quite simply, do not work. Nor do techniques based on planetary movements and sunspots.

It really all began in 1904 when a Norwegian physicist and meteorologist, Vilhelm Bjerknes, said that, in principle, weather prediction could be a genuinely scientific process; the laws of physics, mathematically expressed, could be used to calculate the future state of the atmosphere from known initial conditions; like a physician making a medical prognosis, the forecaster has to start from as accurate and complete a diagnosis or analysis as possible.

Bjerknes wrote at a time when computing involved hand operated adding machines, slide rules and log tables (does anyone use those these days?) It was before he, with his son and other colleagues in Bergen, had developed the concept of fronts; meteorology was still in its infancy and this must be one of the best ever long range weather forecasts!

In broad and very general terms, a simplified description of how the weather machine works is as follows:

• Heating and cooling of the earth varies with time of day, time of year, latitude and nature of the surface. Oceans will heat up and cool down very slowly. Deserts and other dry areas will heat up and cool down quickly. Conurbations, prairies forests and so on will all behave in different ways. Slopes facing the sun will heat up quicker than those facing away from the sun.
• The resultant surface heating differences create different air temperatures leading to pressure differences. Pressure differences lead to air movement - wind.
• Winds - movement of air - cause a redistribution of pressure leading to changes in the winds.
• Heating of the oceans, seas, rivers, lakes and vegetation results in the air containing water vapour. Cooling of the air occurs when air is forced to rise by flowing over hills, by convection or by air streams converging. Cooling also can occur when the air is in contact with cold ground.
• As the air cools it is able to hold less water vapour until, eventually, the water vapour starts to condense into droplets. This releases latent heat. If the air continues to cool then the drops may freeze to become ice or snow, again releasing latent heat.

• These effects warm the air. If the ice melts then heat is needed. This comes from the air so cooling takes place. Again, if the water drops evaporate, then latent heat is required from the air and, again, cooling tales place.
• These latent heat effects change the air temperatures and therefore the pressure patterns and, hence, the winds.
• The formation of drops of water in the air causes cloud to form. Cloud cuts off the radiation from the sun and reflects it back out to space - that is why cloud appears so bright when you fly over it. But, cloud also absorbs radiation from the ground and re-radiates it back to earth.
• Topography affects weather on all scales. We experience sea breezes and the effects of headlands, cliffs, valleys and hills on a local scale. On the larger scale, mountain ranges, such as the Rockies and the Urals have correspondingly large scale effects. On an even larger scale, continents are important - eg the Roaring Forties of the Southern Oceans result from the relatively small amount of land compared to the Northern Hemisphere. There is less land to slow the air down.

The atmospheric system, thus, has multiple feedbacks, both positive and negative. All these physical processes can be expressed mathematically and that concept is the basis of modern forecasting. Bjerknes was the first to realise this.

During World War 1, an outstanding British mathematician, Lewis Fry Richardson, a Quaker pacifist and a stretcher bearer in France, tried to calculate the pressure change 6 hours ahead at a single place in central Europe. He got the answer wrong by a factor of 100 but he still published a book (in 1922) saying what he had done and why. Basically, he had neither the data necessary nor the computing power. Even more importantly, the mathematical techniques had not been developed.

In his book, he envisaged the forecast room of the future as a spherical building, rather like the Albert Hall so as to simulate the earth, full of people with hand calculators and slide rules all doing sums at the same time. Conceptually, this was another remarkable piece of foresight, and is a description of the super-computer of today; parallel processing is not a new idea.

After WW2, when electronic computing became a reality, Bjerknes' and Richardson's ideas were dusted off. Ever since then, meteorologists have been keeping ahead of the computer designers by specifying needs for greater and greater computing power and speed. The latest, massively parallel processors used by weather services are fulfilling their dreams.

The real problem lies in putting numbers to the many equations. This is, in fact, impossible to do precisely and many approximations and estimates have to be made. There are also some very clever mathematical computing techniques used. Richardson failed in his very brave attempt on both counts.

In plain language all the physical processes mentioned above have to be estimated and many of the advances in forecasting over the past 30 or more years have been possible because of improvements in these "guesstimates". That is, at least, in principle. The practice is not as easy, as those words might imply.

There are several different kinds of data used in NWP. First there are observations made at the surface of the earth from fixed and moving platforms on land and sea. These observations may be entirely made by a human being, totally automatic or a combination of both. These can be provided by professional observers, by air traffic controllers, coast guards, officers-of-the- watch on board ship or, sometimes, willing volunteers. Observations are almost always made at the Main Synoptic Hours of 00, 06, 12 and 18 hours UTC. Observations may be made hourly, particularly from automatic stations on land and at sea.

At a selection of land stations and from a few ships there are measurements from free flying balloons, known as radio sondes. These are normally made at 00 and 12 UTC for temperature, humidity and wind. At 06 and 18 hours UTC a few stations may make full soundings but many more will simply measure wind alone. At a few locations there are measurements from a kind of radar that can detect movement of clear air and so measure winds throughout the atmosphere above the location of the equipment.

All these data are gathered and exchanged internationally at the Main Synoptic hours. In addition other types of observation can be available at any time of day or night. There are measurements of wind and temperature from aircraft on routine commercial flights. There are data from drifting buoys. There are satellites in low level, near polar orbit or in geostationary orbit.

Satellite data are often very difficult to use because satellites are not making in situ observations. Usually, they measure the effects of the atmosphere. An example is on the infra-red radiation from the earth. Another example is the deduction of surface wind from the scattering of a radar beam fired at the sea. Getting maximum benefit from such data is an ongoing problem.

All weather data are exchanged throughout the world in very quick time. The amount of data received at any major weather centre depends upon the nature of the data. In the UK, for example, data are received hourly from the near continent, three hourly from parts of N America and further parts of the continent and 6 hourly at the main synoptic hours from the rest of the world.

At the major NWP centres, eg the national weather services of the UK, US, France, Germany, Japan, Australia and, the ECMWF etc, the data have, first to be checked for errors. These can occur due to human error, malfunctioning of equipment or in communications. Sometimes this process is entirely automatic and sometimes with human input. Where possible, data are corrected and only rejected if necessary.

Data are assimilated into the NWP process in several ways by the various centres. One is the use of a 4-D analysis . In this, all observational data and all forecast data at each and every time of each observation are fitted in an optimal way. For those who understand the terminology, it is akin to making a line or a curve fit the data by "least squares"

In this way, all the data for the previous 6 hours will have been used to give the analysis for 06 UTC the best possible start. Data from the various differing sources are of varying quality and this is taken into account by giving variable weight to the way in which they are used. Similarly, when combing the predicted values with new data, the prediction is given a lower weight than new data.

The whole process is, as will be gathered, immensely complex and it is fair to say that as much effort goes into assessing and analysing the basic data as into the prediction process itself. Perhaps, even more, since the old adage of "garbage in garbage out" is very true.

One of the most important approximations insofar as we sailors are concerned is the Grid Length. NWP cannot calculate the atmosphere as the continuum which it is. Models have to represent the atmosphere at a 3 - dimensional grid of points. The nearer the points are together, the better will the model be able to represent the atmosphere.

There are two limitations to the grid length spacing. First is the computing power available. Secondly, there is the amount of meteorological and topographic information. For day to day forecasting with outlooks for the next few days a global model is needed. At the time of writing these have a grid length, typically, about 20 NM (0.5625° longitude x 0.375° latitude for the Met Office model in November 2008).

For short period, local forecasting then much smaller grid lengths can be used over smaller areas. The UK Met Office currently runs a meso-scale, limited area model for much of the North Atlantic from the east of the USA to Western Europe. This has a grid length of about 6 NM. For more detailed very short period forecasts of about an hour or so ahead then models can be run with shorter grid lengths. The UK Met Office is trialling a model with a 2 NM grid length model just for the area of the UK and is working on a 1 NM or less model.

The US Navy runs an operational Coupled Ocean/Atmosphere Meso-scale Prediction System (COAMPS) model with a 6 NM grid length. Meso-scale models are run by a number of research institutions both for research and on behalf of private sector firms.

There are two limitations on the value of such limited or local area models. First, of course, is the computing power that limits the area covered. Computing power requirements increase by a factor of about 16 every time that the grid length is halved. Secondly, there is the uncertainty of what is moving into an area and the effects of large scale weather features which are unlikely, in themselves, to be forecast precisely . The model may be very good at dealing with the local topographic effects on the sea breeze, say, but the results will be compromised if the larger scale model that is bringing cloud across gets the amount of cloud wrong!

An important consequence of the grid length is the size of weather and topographic features that can be described and modelled. As a rough guide, this is between four and five times the grid length. In other words, a 20 NM grid for global models cam only cope with weather and topography some 100 NM size. Such models do not even know that the Isle of Man exists!

As will be seen here, there can be at most one grid point over the IOM and, quite possibly no grid point at all. It would need a grid lngth of about 2 NM to give a general outline of the island and, even then, it would not represent the headlands and bays.

Now imagine a6 NM grid over Torbay or the Solent nd you will see that meso-scale models with hese grid lenghts will not deal with the topography of those areas.

The UK and several other Meso-scale models with 6 NM grid lengths can deal with features down to about 25 - 30 NM. They cannot cope with many of our favourite sailing waters such as the Solent or the Clyde. To model the Tor Bay sea breeze would need a grid length of about 0.5 NM or less, The Solent would need a model with grid lengths of around 300 metres or less. To model Start Point would need a grid length measures in 10s of metres, perhaps 100 metres at most.

Global model output can be obtained easily and freely from the US Global Forecast System. These are useful, particularly for planning purposes.

Output from meso-scale models can be obtained for several days ahead from a number of companies and Météo France. However, I suggest that Meso-scale forecasts should only be used up to around 24 to 36 hours ahead for a 6 NM model and for a few hours only, 24 at most, from models running with smaller grid lengths. But, users should experiment and see how useful they find the data. Accuracy and usefulness are not necessarily synonymous.

When National Met Services run meso-scale models they have the vast amount of raw data mentioned above. They can, also, use human judgement to ensure that the model is using all the data. This enables them to start with analyses that are on a scale comparable with the resolution of the models. However, even satellite and radar techniques can only observe weather on a scale of around 2-3 NM at best. We sailors know only too well how much variation there can be in wind over such short distances.

When a commercial company or university department is running such meso-scale models, they usually will have to rely upon a starting point provided by a National Met Services. For most, this will be the US GFS model which provides output at 1/2 degree intervals. These data have to be interpolated down to model grid lengths. Therefore, such organisations start without all the detail that exists in weather patterns. This will compromise attempts at detailed prediction.

What this means, in practice, is that improvements in meso-scale modelling by private services will not be as great as with the national services. Near land, it will be a few hours into the prediction before the effects of topography really taken into account by the mathematics. They will not handle existing small weather features at all.

There will be some benefits from such meso-scale models because they do model atmospheric physical processes better than global models. However, the limited areas for which these models can be run limits the time ahead for which the data are likely to be useful.

The time ahead at which they will be useful will depend upon the type of weather. In a mobile weather situation, a meso-scale model with a 6 NM grid length may only give useful additional information at 24 to 36 hours ahead. A model with a finer grid length may only be useful up to 24 hours ahead. NOTE - these are illustrative figures and not at all precise.

However, the proof of any pudding is in the eating. Users with enquiring minds might like to try using meso-scale model forecast and see for themselves how good they are. I would expect that the Météo France meso-scale model, with its better starting point will do better than the commercial services. Were the UK Met Office to release data from its experimental model running at 2 NM grid length, then it, also, would perform better.

Reading the previous section, I hope that the reasons for what we all observe is becoming clearer. That is, that forecasts of general weather type for the next few days - perhaps up to five days, or more - are remarkably good. To my mind, as a user of forecasts this has been the major and most obvious improvement since 1977 when I last worked as a Met Office Senior Forecaster at the then Central Forecast Office at Bracknell (now the National Meteorological Centre, Exeter).

It also follows from the above section that very local, detailed forecasting can only be possible for very short periods ahead. How far ahead and what level of detail local forecasting will be possible is, as yet, uncertain. Watch this space!

For the racing yachtsman, whether in dinghies or round-the-cans yachts it will be necessary for years to come to use careful observation, coupled with observation and experience. Anyone who has heard my old friend and colleague David Houghton will be aware of what can be done along those lines.

For those with very deep pockets, such as America's Cup contenders there might well be some advantages in using very small scale models if there are the data to support them Such modelling is beyond my experience and not relevant to the majority of sailor users of forecasts.

For the coastal cruising yachtsman, the general forecasts give excellent advice for the next few days. Used properly, sailors should be able to avoid getting caught out in really bad weather. They should also be able to be in port or good sheltered anchorages when bad weather comes. Importantly, they should be able to avoid being in a position when they are in places that they do not want to be in, in weather that they do not want to go out in.

The blue water sailor making passages lasting weeks can only plan using climatology. His ability to avoid bad weather once out there is strictly limited. His only real tactic is to be aware of warnings and prepare for bad weather before it comes. He has to be fully equipped to do so.

See also my page on limitations to accuracy of forecasts.

The amount of information that it is and will be possible for forecasters to provide is enormous. Serious thought will have to be given to how the user will access and use it. Onshore is one matter with broadband communications. Afloat is another. How many sailors will wish to be stuck down below, nose, to a laptop computer?

When will high speed communications become affordable for access over radio systems?

One thing is certain and that is that the written or spoken word will not and cannot cope with all the variations in the weather that exist, whether they are well forecast or not.

It is all too easily get carried away by the power of the models and lose sight of the realities of the atmosphere. I would like to think that sailors are able to make sensible use of model output and not be discouraged because GRIB or other objective forecasts do not describe what they observe. There are variations in wind that the best models cannot know exist, let alone predict. I have never worked as a modeller but as a senior forecaster, I have worked with modellers on data input, assimilation and interpretation of output. What follows is based on my discussions with some of the best scientists in the field.

First and foremost, NWP models cannot fully resolve features occurring on the scale of their grid lengths. Consequently there has to be some filtering to remove energy from very short wavelengths. For the same reason, topographic effects must also be smoothed. As a result, information on scales below about 5 grid lengths is damped. If this is not done, then computational noise will swamp the meteorological signal.

Thus a 0.1 degree (6 NM) grid length model provides an un-damped forecast only for features of scale 0.5 degree or more ie 30 NM or more..

I would expect such features to have a natural predictability of, perhaps, 6 to 12 hours at best.

A model with a grid length of 1 or 2 NM will only deal with features between 5 and 10 NM and only for a very few hours. (Just watch a large cumulus cloud grow and disappear within a matter of a couple of hours or so.) Where such features are forced, e.g. a synoptically forced trough or a topographically forced sea breeze, the predictability should be commensurate with that of the forcing scale. Synoptic scale features of 100-500 NM will have a predictability of more than one day. The time scale of a topographic heating is measured in hours and the effects of the sea breeze on a similar scale.

Note that "predictability" here means predictable with a "perfect" model and a "perfect" observing system. In reality, accurate forecasts will be achieved up to these limits only on favourable occasions. A small error in the large scale synoptic pattern might well create large (in relative terms) errors in small scale detail. A small error in cloud amount on the scale of a sea breeze coast will have a catastrophic effect on a sea breeze forecast.

Even if detail at, say, a 1 km (0.5 NM) scale could be observed (which it cannot), and we had a perfect prediction model (which we have not), it is unlikely that it (ie the detail) would still be there by the time a user obtained the analysis information, and certainly not by the time that a 6-hour forecast was produced.

The biggest benefits from a model with a very short grid length will thus be in the one day or so ahead prediction of topographically forced weather features, up to one day prediction of sub-synoptic troughs forced by the synoptic flow, and 6 - 12 hour prediction of organised convective systems.

The same arguments apply to the observed data. Where the features of interest are largely forced by topography or synoptic scale structures, the most important observations are those that define the synoptic scale.

For locally generated features, we have to rely on those sources of observations that are truly on the very small scale - eg satellite and radar imagery. These both give an effective observation spacing of around 5 km. Over the sea, of course radar data are only available to a fairly short distance from the coast.

In practice, there are, quite simply, not enough data to define the very small weather detail that sailors observe over the sea. Satellite (QuikScat) winds are not available often enough and have problems near coasts and in cloudy areas anyway. Over land, data from radar can be used to help predict small scale and short lived detail.

There is also the pragmatic observation that reducing the grid length has enabled improved performance in predicting the larger scale evolution. It is for this reason that global forecasting centres continue to move to finer grid lengths in global models. For example, if a hurricane can be predicted more accurately by resolving its circulation better, then a correct prediction becomes more likely of whether re-curvature will occur, where it will engage in the mid-latitude flow, and hence the synoptic evolution over the UK at 4-5 days ahead.

How much grid lengths can be reduced in global models is uncertain and will depend on observational data - as well as computer power. The future is likely to be one of finer resolution global models but with not a great improvement on short term, small detail over the sea.

The major benefits of fine scale models are likely to be for prediction of heavy showers over land, clearing skies leading for fog or ice on roads. To focus on the specific needs of sailors. a NWP model with very short grid lengths will offer improvements in the following areas:

• More realistic representation of the impact of topography on coastal winds and weather on such scales as the strong winds through the Dover Strait.

• More reliable forecasts of small scale thundery troughs and the like..
• More reliable forecasts of the development of organised convection systems, with associated severe weather.

Useful gains might be longest for the first area, less for the second, and perhaps only 12 hours for the third.