Workshop Ensemble Prediction and Uncertainties in Flood Forecasting

30 March 2006

09.00
Welcome + Introduction to the goals of the workshop - Prof. Dr. Manfred Spreafico – President of the CHR

09.10
Therese Bürgi - Federal Office for Environment: Swiss hydrological forecast system - Objectives and problem statement

09.25
Keynote
John Schaake - U.S. National Weather Service: Hydrologic ensemble prediction: Past, present and opportunities for the future

Theme block I -  Uncertainty in numerical weather predictions

Chairman: Ilmar Karro / Rapporteur: Michael Kunz

10.10
Per Undén - Swedish Meteorological and Hydrological Institute: Global EPS systems - principles, use and limitations

10.40
Coffee and Tea + Poster Session

11.10
André Walser / Mathias Rotach - MeteoSwiss: The benefit of a limited-area ensemble prediction system with respect to flood forecasting

11.45
Susanne Theis - German Weather Service: Plans for high-resolution forecasts and ensembles at the German Weather Service

12.15
Georg Pistotnik - Central Institute for Meteorology and Geodynamics Vienna: Combination of meteorological nowcasting and ensemble methods in operational flood forecasting

12.45
Discussion on Theme I

13.15
Lunch

Theme block II - Hydrological Ensemble Forecasts

Chairman: Günter Blöschl / Rapporteur: Daniel Viviroli

14.30
Roman Krzysztofowicz - Systems Engineering and Statistics - University of Virginia: Bayesian theory of ensemble forecasting

15.00
Florian Pappenberger - University of Lancaster, Department of Environmental Science: Cascading uncertainty in flood forecasting

15.30
Christian Reszler - Institute of Hydraulics and Water Resources Engineering - Technical University Vienna: Operational ensemble forecasts of floods

16.00
Coffee and Tea + Poster Session

16.30
Göran Lindström - Swedish Meteorological and Hydrological Institute: Evaluation of ensemble streamflow forecasting at SMHI

17.00
Mark Verbunt - Atmospheric and Climate Science, ETH: Ensemble Flood Forecasting in Switzerland: Selected case studies of extreme events 

17.30
Massimiliano Zappa - Swiss Federal Institute for Forest, Snow and Landscape Research: Towards (quasi-)operational demonstration of hydrometeorological ensemble prediction systems: The MAP D-PHASE and COST PROFIT projects

18.00
Discussion on Theme II

19.30
Workshop Dinner

31 March 2006

Theme block III - Communication of Uncertainties 

Chairman: Peter Krahe / Rapporteur: Clemens Mathis

09.00
Frank Lantsheer - Royal Dutch Meteorological Institute: Severe weather warnings and focus on awereness

09.30
Jutta Thielen - Joint Research Centre, Land management Unit: Added value of ensemble prediction system products for medium-range flood forecasting on European scale

10.00
Christoph Hegg - Swiss Federal Institute for Forest, Snow and Landscape Research: IFKIS-HYDRO an information system for flood risks at local & regional levels

10.30
Coffee and Tea + Poster Session

11.00
Gerd Tetzlaff - German Committee for Disaster Prevention: Uncertainties of forecasts of extreme precipitation events

11.30
Leonie Bolwidt / Ben Zweverink - Rijkswaterstaat RIZA and Directorate East Netherlands: Dealing with uncertainties during high discharge conditions in The Netherlands

12.00
Discussion on Theme III

12.45
Summary and Conclusion of the Workshop

13.15
Closing of the Workshop

13.30
Lunch

Günter Blöschl (Chairman)
Institute of Hydraulics and Water Resources Engineering - Technical University Vienna
Karlsplatz 13
A-1030 Vienna
Tel. +43-1-58801 22315
bloeschl@hydro.tuwien.ac.at

Ilmar Karro
Swedish Meteorological and Hydrological Institute
SE-60176 Norrköping
Tel.: +46-11-495 8000
ilmar.karro@smhi.se

Eric Sprokkereef
Secretariat CHR/KHR
Postbus 17
NL-8200 AA Lelystad
Tel: +31 26 3688367
e.sprokkereef@chr-khr.org

Stephan Vogt
Federal Office for the Environment
CH-3003 Bern
Tel.: +41-31-322 90 91
stephan.vogt@bafu.admin.ch

Peter Krahe
Federal Institute of Hydrology
Postfach 200253
D-56002 Koblenz
Tel. +49-261-1306 5234 
krahe@bafg.de

Brief report on the first presentation block
"Uncertainty in numerical weather prediction"

Reporter: Michael Kunz

As a result of many complex and highly nonlinear processes, precipitation is the atmospheric variable which is most important for hydrological purposes but also difficult to predict. In order to quantify the uncertainties of precipitation forecasts, various methods of ensemble prediction systems (EPS) have been developed over the past decades. However, there are many open questions and scientific challenges that have to be overcome.

By considering uncertainties arising from an inaccuracy in the initial conditions, several large forecast centres like the ECMWF or NCEP run global EPS operationally. Each member of the EPS represents one possible physically-based scenario so that the total ensemble allows estimating the probability function for several atmospheric parameters. However, the coarse resolution of the global models gives an underestimation of the precipitation amounts and blurs small-scale variations of the spatial distribution especially over mountainous terrain (Undén) [1]. Regional EPS from a Limited Area Model combines the advantage of a probabilistic forecast from the global model with the better representation of small-scale terrain variations of a high-resolution numerical weather forecast. Walser and Rotach [2] showed in a case study of the flooding event of August 2005 that COSMO-LEPS (clustering approach) yields substantially more information on regional and rather extreme weather events than the global EPS of ECMWF. Hence, the coupling of COSMO-LEPS with a hydrologic model seemed to be an attractive strategy to get probabilistic runoff forecasts which could enable risk-based flood warnings.

Another well established approach in ensemble weather prediction is to create an artificial forecast by combining different numerical weather forecasts (poor man ensemble) as reported by Trepte [3]. Pistotnik [4] introduced the model system INCA (Integrated Nowcasting through Comprehensive Analysis) that additionally includes a nowcasting technique which is based on the extrapolation of current observations. In this approach, where the different model forecasts are weighted according to the forecast lead-time, significantly improves the forecast skills especially during the first forecast hours.

By using an EPS for hydrological purposes, it is important to keep in mind several constraints of the forecasts (Undén) [1]:

• The spread of the members is around a model forecast and not around the nature-truth. All members may be wrong in the same way and/or contain a complex set of biases due to model deficiencies (approximations, numerical algorithms, parameterizations).
• By adding sizeable perturbations to the most likely initial state, each member is likely to be significantly worse then the control run.
• Many EPS are of limited use for short range forecasts when the optimisation time for the growth of the forecast errors lies in the same range.
• The forecast uncertainty increases mostly with forecast lead-time and with precipitation amount. The uncertainty also is dependent on the time and space scales over which the forecast is averaged.

According to the discussion between meteorologists and hydrologists, the central problem with precipitation forecasts is that they tend to be strongly biased compared to the observations. Hydrologists, however, need well-calibrated forecasts on a local scale. Additionally, the ensemble forecasts tend to underestimate the actual uncertainty since they do not account for important model inaccuracies. A crucial issue therefore is the model climatology or Model Output Statistics (MOS) that should be considered in an appropriate form of data pre-processing. Long-term information about model output and its verification is required to limit the effects of random sampling variability of the results. Besides the problem to hold a large sample size, the model climatology may change with changes of the model physics. The users identified a lack of information and communication between model developers and hydrologists that has to be improved in the future.

In general, the meteorological forecaster has a good knowledge about the characteristics of the weather forecast model, its reliabilities and well known limitations. Besides, he has great expertise in the interpretation of the forecasts. He should be able to modify ensemble forecasts, e.g. by systematically shifting precipitation fields. There is a great wish to improve the communication between the operational forecasters and the hydrologists in order to transfer this knowledge. To improve the forecast chain from precipitation to discharge in the future, it is important that both operators need to know the end-user and their demands as well as to understand each other problems.

Finally, it was discussed that the end-users should be actively involved in the development of forecast products to meet their requirements. In the case of extreme flood events, the end-user needs more information including probability estimates that may help in the decision to adopt protective measures like evacuations, and to justify these measures against the population.

[1] - Per Undén - Swedish Meteorological and Hydrological Institute: Global ensemble forecast systems - principles, some applications and limitations
[2] - André Walser / Mathias Rotach - MeteoSwiss: The benefit of a limited-area ensemble prediction system with respect to flood forecasting
[3] - Sebastian Trepte - German Weather Service: Plans for high-resolution forecasts and ensembles at the German Weather Service
[4] - Georg Pistotnik - Central Institute for Meteorology and Geodynamics Vienna: Combination of meteorological nowcasting and ensemble methods in operational flood forecasting

Brief report on the second presentation block
Hydrological ensemble forecasts

Reporter: Daniel Viviroli

Theme Block II consisted of presentations dealing with hydrological ensemble forecasts. Subjects dealt with were identification and decomposition of Ensemble Prediction System (EPS) uncertainties with Bayesian Theory [1], propagation of uncertainties in the chain of EPS constituents [2], operational EPS forecasts with updating in an Austrian meso-scale catchment [3], experiences from 30 months of operational EPS forecasts for 45 catchments in Sweden [4], EPS-based flood forecasting for the Swiss part of the Rhine River basin with examination of rare events [5], and a framework for demonstrating the ability of EPS for improved forecasts of heavy events in the Alps [6].

The subsequent discussion can be divided roughly in two main subject areas: On the one hand, uncertainties and errors of EPS were considered, showing that there are still significant issues to be investigated. On the other hand, the applicability of EPS results and their transfer to more practically oriented users was reviewed critically.

Uncertainties and errors

There was consensus that uncertainties in EPS have to be examined thoroughly, as was also suggested by Roman Krzysztofowicz in his presentation [1]. Especially the incorporation and further decomposition of hydrological uncertainties is a difficult task. Göran Lindström [4] proposed to start as simple as possible and then improve what needs to be improved. It also has to be borne in mind that EPS uncertainty is a function of basin scale and event characteristics and therefore is heavily space and time dependent. Furthermore, the lead time considered (cf. [3]) determines whether processes or data inputs are the major source of uncertainty. Such instationarities need to be stated clearly in any investigation of uncertainty. Furthermore, errors and uncertainties are propagated in the chain of observation, Numerical Weather Prediction (NWP) and hydrological model. Since hydrological variables are integrative in space and time, they are ideal for testing EPS results. There was agreement that the information content of hydrological data should be made more use of. Hydrological data should also be employed more frequently through updating procedures (e. g. Ensemble Kalman Filter). More generally speaking, data should be collected with view to the most crucial sources of uncertainty (see [2]).

A major concern is the identification of biases in EPS results; this needs to be investigated on basis of long-term series of observation data. With the same data, the probability of EPS results should be examined more thoroughly as well, as opposed to presenting only the number of EPS results that exceed a defined threshold. The estimated probability distributions should be conditioned on observations to increase consistency. Inconvenience for such investigations arises from the frequent change in EPS key features, e. g. in resolution, parameterisation and number of ensemble members. In order to increase the accuracy of forecasts, it was proposed to combine various EPS.

An important research question concerns the appropriate number of ensemble members to achieve reliable results. On the one hand, an ensemble should be large enough to allow a reasonable estimation of probabilities and to meet the requirements of the end-users as to reliability of the results. On the other hand, ensemble size is limited by the computational time available for a forecast.

Concerning the event size to be investigated more closely with EPS, it was proposed to increase the focus on big storm events, since their behaviour differs from the one of medium and small storms and therefore poses additional difficulties. The closer examination of interesting events and event types could also be used to improve the communication and co-operation between Hydrologists and Meteorologists involved with EPS techniques. However, an important question to be answered yet is how to deal with extreme events that are not represented in the atmospheric EPS.

A point that by general agreement seems important is how to combine the deterministic models with statistical methods. Since statistics fail to reproduce heavily non-linear processes (as they occur especially under extreme conditions) and cannot account for the dynamics in atmospheric and hydrological systems, a creative combination of statistical and deterministic approaches should be sought. For instance, statistical post-processing could serve to capture biases and reveal lack of representativity in EPS.

Applicability

Although there was a separate theme block dealing with communication of uncertainties, the applicability of EPS techniques and the transfer to more practically oriented end-users was discussed to quite some extent. This transfer is difficult in that the communication between a forecaster (hydrologist or meteorologist) and an end-user is highly individual, i. e. that forecaster A and end-user B will end up with different conclusions and results than forecaster C and end-user D. This is also due to the fact that the actual cost/loss-ratio depends on the individual end-user. In view of above considerations, the reaction of end-users to (uncertain) results itself is subject to uncertainty. Therefore, the role of the "human factor" in the decision-making process (forecaster or end-user) should be further examined.

The question arose what the added value of EPS forecasts would be at all as compared to deterministic forecasts. In reply, it was stressed that the major benefit is the increase in information given about what to expect within the forecast period. The forecast values are complemented with a probability which helps the end-user in making his decisions. It was suspected that the demand for EPS-type results will increase when this added value is recognised more widely and the end-users get more accustomed to the interpretation of such forecasts. Although it is essential to instruct the end-users in utilisation of EPS results, the interpretation still will demand the involvement of experts. To facilitate the decision process, it seems desirable to establish joint data interfaces and platforms for visualisation and warning (cf. [6]).

To increase flexibility of EPS applications, there should be more possibilities for interaction, e. g. in changing initial conditions or adjusting the river stage manually. In case of extraordinary conditions (e. g. an ice jam in a region where this is not expected), model results may become invalid but may be "fixed" by an experienced forecaster or end-user. Human judgement could also be crucial for error identification.

From the end-user's perspective, the value of EPS results diminishes heavily if their spread increases. Therefore, the "spaghetti plots" seem not suitable for communication with end-users, and more appropriate solutions have to be sought, such as probability maps (cf. theme block III). It was stated, however, that NWP tend to cluster since they are based upon physical processes and that our impression of large spreads in EPS results may be caused by frequent examination of difficult cases. On the whole, only understandable results should be distributed, with respect to the end-user's crucial questions: when, where, how much, and with what probability? But what are consequences of a probabilistic forecast? What action is to be taken? While an EPS prediction shows a range of possibilities, the end-user's decision often is of a "yes/no"-type and therefore less flexible. Already the decision whether to issue a warning or not is a critical one since false alarms diminish credibility (which should be guaranteed with greatest priority), while omitted warnings may lead to fatalities and legal consequences. How to deal therefore with a probabilistic forecast that announces the possibility of a rare event, although with low probability (see [5])?

To conclude, it seems that research concerning hydrological ensemble forecasts makes progress. However, stable and reliable applications are difficult to establish at present, not least because of frequent changes in the underlying EPS. Co-ordination of research and exchange of experience is therefore of great importance. In order to reduce scepticism of end-users, the transfer of probabilistic results into a "yes/no"-world should be improved

[1] Roman Krzysztofowicz, Bayesian theory of ensemble forecasting (presentation not available as pdf)
[2] Florian Pappenberger, Cascading uncertainty in flood forecasting
[3] Christian Reszler, Operational ensemble forecasts of floods
[4] Göran Lindström, Evaluation of ensemble streamflow forecasting at SMHI
[5] Mark Verbunt, Ensemble flood forecasting in Switzerland: Selected case studies of extreme events
[6] Massimiliano Zappa, Towards (quasi-)operational demonstration of hydrometeorological ensemble prediction systems: The MAP D-PHASE and COST PROFIT projects

Brief report on the third presentation block
Communication of uncertainty

Reporter: Clemens Mathis

The presentations and discussions within Theme Block III aim to give answers to the following questions:

• How should uncertainty be communicated to decision-makers and to the public?
• What is the role of a human forecaster?

Ensemble Prediction Systems (EPS) in flood forecasting can give an indication that an extreme flood may occur, but do not forecast how extreme the flood will actually be. Nevertheless, flood early warning systems based on ensemble technique help to reduce flood damage. Safety measures (e.g. evacuation) can cost a lot of money, therefore the person or group of persons who has to make a decision is under considerable pressure. The difficulty for a human forecaster is to translate the uncertainties, which are inherent in the forecasting systems, for the end-user (water resource manager, rescue service, general public), who only wants to know a yes or a no.

EPS show a wide range of possible discharge outcomes and the communication of this diversity has not yet been solved. It is less a question of flood forecasting than of communication sciences. The correct way depends on many factors: characteristics and size of the river catchment, people's experiences of previous floods, and costs of the safety precautions are but a few. Therefore, there are many different solutions. The respective characteristics require adapted solutions.

At the Royal Netherlands Meteorological Institute (KNMI) an expert team determines the Weather Warning [1]. Frank Lantsheer showed in his presentation that this ensemble approach leads to better results. Furthermore, he presented the European Multiservice Meteorological Awareness System (EMMA) project by highlighting the problem of harmonisation of warning levels.

The EPS of the European Commission's Joint Research Centre produce 51 discharge curves. With box representations combining numbers and colours, the uncertainty of the medium-range forecasts is transferred into uncertainty measures. It is an effective way of visualizing the deterministic and probabilistic results of the European Flood Alert System (EFAS) in one diagram [2]. The end-users addressed here are the national regional flood forecasting and warning centres.

Forecasting in small catchment areas (< 100 km²) is very difficult. The lead time is very short. Nevertheless, the Swiss Federal Institute for Forest, Snow and Landscape Research has developed the IFKIS Hydro project, which was explained by Christoph Hegg [3]. It has been tested in two case studies. He sees the advantages of the ensemble predictions but stressed that additional information in crisis situations is needed and no added value will be reached without training of decision makers/end-users or members of their staff.

Extreme events are rare. The return period is in the range of about 100 years or more. Therefore, there has been little experience with forecasts of extreme precipitation events. In his written abstract Gerd Tetzlaff [4] showed the physics of extreme precipitation and the problem of validation of the forecast quality. He argues that, in addition to the mathematical and statistical validation of precipitation which has occurred and been recorded, criteria for good forecasts related to economical values should also be defined.

In this connection, he gave a summary of the Third International Conference on Early Warning (EWC III) held in Bonn, Germany on 27-29 March 2006. He highlighted the symposium findings that early warning is a major element of disaster risk reduction. It helps to prevent loss of life and reduces the potential economic impact of disasters. To be effective, early warning systems need to actively involve the communities at risk, facilitate public education and awareness, communicate and disseminate warnings and messages, and ensure that there is a constant state of preparedness.

The flood forecasting practices in the Netherlands with 55 different water level forecasts were presented by Leonie Bolwidt [5]. The question to the crisis manager is: How can the best of all this information be used? Ben Zweverink has had years of experience as a crisis manager. His decision tree for evacuation shows the importance of well-founded decisions. The authorities in the Netherlands have well-prepared flood management plans.

The ensuing discussion could not solve all the problems of the uncertainties. The participants felt that the human forecaster is needed in order to communicate the uncertainties to the end-user. Due to the uncertainties inherent in meteorological forecasts, communication between meteorologists and hydrological forecasters is necessary, especially in the case of extreme events. One advantage of the medium-range EPS is getting earlier information about a flood. The authorities have thus more time to prepare. The major benefit of ensemble prediction systems is the high probability that the right discharge curve is contained. The solution to finding the right one in every case is still missing. More case studies have to be carried out and lessons must be learned from them. Also, our knowledge of user needs has to be improved.

As a general conclusion it can be stated that probabilistic forecast products have to be tailored to the user-dependent level of decision-making.

[1] Frank Lantsheer, Severe weather warnings and focus on awareness.
[2] Jutta Thielen, Added value of ensemble prediction systems products for medium-range flood forecasting on European scale.
[3] Christoph Hegg, IFKIS-Hydro, an information system for flood risks at local and regional levels.
[4] Gerd Tetzlaff, Uncertainties of forecasts of extreme precipitation events.
[5] Leonie Bolwidt/Ben Zweverink, Dealing with uncertainties during high discharge conditions in The Netherlands.