The following CAMASE electronic newsletter has some interesting items
related to modeling you might be interested in.
Gerrit
>Received: from HEARN.nic.SURFnet.nl (hearn.nic.surfnet.nl [192.87.5.132])
by tangine.bae.griffin.peachnet.edu (8.6.10/8.6.10) with SMTP id LAA11247
for <[log in to unmask]>; Thu, 30 Nov 1995 11:05:28 -0500
>Received: from HEARN.NIC.SURFNET.NL by HEARN.nic.SURFnet.nl (IBM VM SMTP V2R2)
> with BSMTP id 2568; Thu, 30 Nov 95 16:49:45 +0100
>Received: from NIC.SURFNET.NL (NJE origin LISTSERV@HEARN) by
HEARN.NIC.SURFNET.NL (LMail V1.2a/1.8a) with BSMTP id 6293; Thu, 30 Nov 1995
16:49:37 +0100
>Received: from NIC.SURFNET.NL by NIC.SURFNET.NL (LISTSERV release 1.8b) with
> NJE id 1501 for [log in to unmask]; Thu, 30 Nov 1995 16:49:23
> +0100
>Received: from HEARN (NJE origin SMTP@HEARN) by HEARN.NIC.SURFNET.NL (LMail
> V1.2a/1.8a) with BSMTP id 6285; Thu, 30 Nov 1995 16:49:21 +0100
>Received: from mgate.nic.agro.nl by HEARN.nic.SURFnet.nl (IBM VM SMTP V2R2)
> with TCP; Thu, 30 Nov 95 16:49:15 +0100
>Received: from ABW1 (ABW1) by AGRO.NL (PMDF V5.0-4 #12026) id
> <[log in to unmask]> for [log in to unmask]; Thu, 30 Nov
> 1995 16:49:02 +0000 (MED)
>Received: from AB.DLO.NL by AB.DLO.NL (PMDF V4.3-7 #7552) id
> <[log in to unmask]>; Thu, 30 Nov 1995 16:48:19 +0000 (GMT)
>X-VMS-To: CAMASE-L
>MIME-version: 1.0
>Content-transfer-encoding: 7BIT
>Approved-By: "Ing. M.C. Plentinger, AB-DLO" <[log in to unmask]>
>Message-ID: <[log in to unmask]>
>Date: Thu, 30 Nov 1995 16:48:19 +0000
>Reply-To: Quantitative Methods of Research on Agricultural Systems and the
> Environment <[log in to unmask]>
>Sender: Quantitative Methods of Research on Agricultural Systems and the
> Environment <[log in to unmask]>
>From: "Ing. M.C. Plentinger, AB-DLO" <[log in to unmask]>
>Subject: CAMASE_NEWS Extra edition, November 1995
>To: Multiple recipients of list CAMASE-L <[log in to unmask]>
>Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
>
> N E W S L E T T E R
>
> O F
>
> A G R O - E C O S Y S T E M S
>
> M O D E L L I N G
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> Published by AB-DLO November 1995, Extra edition
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
>
> \==================================================================\
> \ \
> \ CONTENTS \
> \ \
> \ GUIDELINES \
> \ \
> \ EVALUATION \
> \ \
> \ Definitions \
> \ Guidelines \
> \ References \
> \ \
> \ SENSITIVITY AND UNCERTAINTY ANALYSIS \
> \ \
> \ Definitions \
> \ Guidelines \
> \ References \
> \ \
> \ CALIBRATION \
> \ \
> \ Definitions \
> \ Guidelines \
> \ References \
> \==================================================================\
>
> ====================================================================
> EDITORIAL I
>
> CAMASE is a concerted action that would not have come into existence
> without the generous support by the European Commission's RTD
> programme. this is greatly appreciated by us and, we are sure, by
> our readers.
> Frits Penning de Vries,
> Marja Plentinger
> ====================================================================
>
> ====================================================================
> EDITORIAL II
>
> Systems analysis and simulation are commonly used tools of
> researchers. Yet, many of us learned to use them by ourselves, by
> trial and error. In the process, we tumbled in many pitfalls,
> sometimes even without realizing it. It was suggested to CAMASE to
> make an effort to produce guidelines for modelling and distribute
> widely. We have pooled our limited wisdom is this matter, and
> propose two short sets of guidelines on these pages. We hope they
> will make a difference.
> No doubt that our guidelines can be improved further! May we have
> your comments?
> ====================================================================
>
> * * *
>
> GUIDELINES
>
> The need for guidelines for modelling has been expressed several
> times, particularly by those out of the main stream of developments.
> In the CAMASE project, we have developed a first draft of guidelines
> for 'validation', 'sensitivity and uncertainty analysis' and
> 'calibration'. These are presented below, preceded by some relevant
> definitions. To provide readers with more details and access to
> examples, we added references to some of the most relevant
> scientific papers. We welcome very much all responses to further
> improve the guidelines and the set of most relevant papers. A next
> step in upgrading the quality of model building and model use should
> be a manual with more explicit guidelines, procedures, tools and
> examples. Ongoing projects on software quality aim (see next
> CAMASE_NEWS) at developing such manuals.
>
> We acknowledge the input of Dr.Ir. A.K. Bregt, Dr. Ph. Debaeke,
> Dr.ir. W.A.H. Rossing, Ir. M.J. van der Velden, Ir. G.W.J. van de
> Ven, and Dr.ir. A.L.M. van Wijk.
>
> Frits Penning de Vries,
> Michiel Jansen, and
> Klaas Metselaar.
>
>
> * * *
>
> EVALUATION
>
> * Definitions
>
> - Evaluation
> The broadest term to describe the action of judging the adequacy of
> a model. Evaluation includes checking internal consistency and
> units used in a computer program, comparison of model output with
> an independent data set of real world observations, uncertainty
> analysis, and judgement of utility. The term 'test' is sometimes
> used with the same meaning.
>
> - Validation
> The term will be used here in its most common utilitarian sense of
> establishing the usefulness and relevance of a model for a
> predefined purpose. It is a recurrent activity in a phase of model
> development. Models have always a limited range of validity, and it
> is necessary to specify clearly what it is. In case of predictive
> models, a major part of the validation consists of an assessment of
> prediction accuracy.
>
> - Verification
> This technical term designates the inspection of the internal
> consistency of the model and its software implementation. Some
> important elements are: analysis of dimensions and units, on-line
> checks on mass conservation, detection of violation of natural
> ranges of parameters and variables. Verification also comprises
> inspection of qualitative behaviour of the model and its
> implementation, for instance a check whether the response of one
> model output to changing values of one parameter conforms to
> theoretical insights.
>
> - Calibration and validation data
> Sets of data used to calibrate respectively validate a model.
>
> - Cross-validation
> A procedure for calibrating and validating a model with a limited
> number of representative data sets. It consists of repeated
> subdivision of all the data into calibration and validation data,
> followed by corresponding calibration and validation. The average
> of the observed prediction errors over the subdivisions provides an
> estimate of the prediction error in an entirely new situation.
> There are several variants of cross-validation. In the most popular
> one, called leave-one-out validation, each independent data set
> gets the role of validation data exactly once, at which occasion
> the complementary set gets the role of calibration set.
>
>
> * Guidelines
>
> # Make explicit for what purpose the model is being validated, and
> compare whether this is compatible with the objectives for which
> the model was developed.
>
> # Make explicit in the description of the model which processes or
> natural resources are limiting the behaviour of the model.
>
> # It is meaningless to simply state that a model is valid. After a
> successful validation a model is shown to be of practical use for a
> specific purpose over a specific range. A discussion of acceptable
> error size, with due regard to the specific purpose, should be
> included. Large errors might make the model of little practical
> value as a predictor though it might still have an instructive
> value. Validation of absolute values of key variables is best.
>
> # Model evaluation should start with verification of the model and
> its software implementation.
>
> # In model evaluation every model output should be subject to
> validation. If the model is to be used in predictions, such as
> scenario studies, the validation of the model is more efficiently
> focused on issues of interest, which could be differences between
> scenarios, or the resulting ranking of alternatives such as e.g.
> predicting the yield of different varieties.
>
> # The validation data should be representative for the situations
> in which the model is to be used: Swedish data, for instance, may
> be unsuitable to validate a model to be used in Spain. The
> validation set should -if possible- cover the range of situations
> encountered in predictions.
>
> # Although prediction accuracy will benefit from representative
> calibration data, representativity of the calibration data is not
> required for soundness of the validation.
>
> # The calibration data and the validation data should be different.
> In studies where a large number of validations are executed, there
> is a chance that calibration set and validation set are identical,
> when calibration and validation set are arbitrarily taken from the
> available sets.
>
> # Validation should be repeatable by colleague scientists. This
> means that all crucial validation data (in a broad sense,
> comprising input, output, model structure) should be well
> documented and accessible. Validation data set should be of high
> quality.
>
> # When dealing with complex models, divide and rule:
> a. If the subject of a model is too large for regular validation
> (e.g. an entire region), the model is to be subdivided into
> components that are validated separately. Provide a logical
> reasoning for which the aggregate model is consistent, and do
> not miss crucial interactions among the components.
> b. If the subject of the model is takes too long for regular
> validation (e.g. long term changes in soil structure and
> organic matter), validation should be undertaken for shorter
> periods, and indirect evidence (time series from different
> environments) collected.
>
>
> * References
>
> Addiscott, T., J. Smith & N. Bradbury, 1995. Critical evaluation
> of models and their parameters. Journal Environmental Quality 24:
> 803-807
> Colson, J., D. Wallach, A. Bouniols, J.B. Denis & J.W. Jones, 1995.
> Mean squared error of yield prediction by SOYGRO. Agronomy
> Journal 87: 397-402
> Debaeke, Ph., K. Loague & R.E. Green, 1991. Statistical and
> graphical methods for evaluating solute transport models:
> overview and application. J.Contaminant Hydrology 7: 51-73
> Hamilton, M.A., 1991. Model validation: an annotated bibliography.
> Commun. Statist. Theory Meth. 20(7): 2207-2266
> Koning, G.H.J. de, M.J.W. Jansen, C.A. van Diepen & F.W.T.
> Penning de Vries, 1993. Crop growth simulation and statistical
> validation for regional yield forecasting across the European
> Community. CABO-TT Simulation Reports 31.
> Penning de Vries, F.W.T., 1977. Evaluation of simulation models in
> agriculture and biology: conclusions of a workshop. Agricultural
> Systems 2 (1977): 99-107
> Power, M., 1993. The predictive validation of ecological and
> environmental models. Ecological modelling 68: 33-50
> Rosenberg, N.J., M.S. McKenney, W.E. Easterling & K.M. Lemon, 1992.
> Validation of EPIC model simulations of crop responses to current
> climate and CO2 conditions: comparisons with census, expert
> judgement and experimental plot data. Agric. Met. 59: 35-51
> Scholten, H., 1994. Blueprint of a supramodel for quality assurance
> of the simulation modelling process. Full paper submitted to
> European Simulation Symposium, Istanbul, Turkey, October 9-12, 1994
> Scholten, H. & M.W.M. van der Tol, 1994. Towards a metrics for
> simulation model validation. In: Grasman, J. & G. van Straten
> (Eds.). Predictability and nonlinear modelling in natural
> sciences and economics. Proceedings of the 75th Anniversary
> Conference of WAU, April 5-7, 1993, Wageningen, The Netherlands.
> Kluwer Publishers, Dordrecht. 398-410
>
>
> * * *
>
> SENSITIVITY AND UNCERTAINTY ANALYSIS
>
> * Definitions
>
> - Input
> All parameters, initial values, tabulated functions, and driving
> variables in the model. For some analyses, tabulated functions may
> have to be parameterized.
>
> - Uncertainty
> In this context: imperfect knowledge regarding aspects of a model.
> Uncertainty regarding model variables is usually specified by a
> probability distribution or by a sample of measured values (an
> empirical probability distribution); sometimes it is specified by a
> set of possible values. We adhere to the probabilistic concept of
> uncertainty, and we use variances as measure of uncertainty.
>
> - Sources of uncertainty
> Uncertainty exists at the level of inputs and output of the model.
> Uncertainty at the level of model formulation also exists. In these
> guidelines, however, we will assume that the model is deterministic,
> and that uncertainties are solely introduced via the inputs. Input
> uncertainty is caused by natural variation (e.g. weather, soil or
> genetic variation) as well as by imperfection of data. Although the
> causes of uncertainties may differ, their effect is the same, namely
> uncertainty about the model outputs. It is up to the modeller
> whether or not to incorporate natural variation in the model; the
> choice depends also on the spatial or temporal scale at which the
> model is used.
> The input uncertainty of different parameters may contain
> correlations caused by biological or physical mechanisms, e.g.
> correlation between development rate before and after flowering,
> or between weather at two consecutive days. Correlation can also
> be caused by the nature of the data analyzed to estimate parameters,
> e.g. correlation between estimates of intercept and slope of a
> regression line.
>
> - Sensitivity analysis
> Definitions vary. In most studies, sensitivity analysis is the study
> of model properties through - not necessarily realistically sized -
> changes in the input variables and the analysis of its effect on
> model outputs. The questions addressed are for instance:
> # whether or not some output is affected at all by some input
> # continuity, differentiability, monotonic increase or decrease of
> the model's response to input variation
> Most of the variation of outputs is generally caused by a small
> number of inputs.
>
> - Uncertainty analysis
> Definitions vary. In most studies, uncertainty analysis is the study
> of output uncertainty as a function of a careful inventory of the
> different sources of uncertainty present in the model. The questions
> addressed are for instance:
> # What is the prediction uncertainty due to all uncertainties in
> model inputs? (Total uncertainty, often expressed as variance)
> # How do inputs (singly or in groups) contribute to prediction
> uncertainty?
>
> - One-at-a-time sensitivity analysis
> An analysis of responses to variation of one input at a time,
> whereas the other inputs are kept at nominal values. One-at-a-time
> graphs can be informative and may reveal discontinuities; in these
> graphs a model response is plotted against the studied input, which
> latter varies in small steps over some range.
>
> - Factorial sensitivity analysis
> Analysis where inputs are varied according to a so-called factorial
> design. In the most common factorial design, called two-level
> design, each input has two levels: low and high. A full two-level
> factorial design for n inputs requires 2n model runs. If this number
> is prohibitive, one may apply a fractional factorial design, in
> which only a fraction of the input combinations is realized.
>
> - Local sensitivity analysis (or differential sensitivity analysis)
> An analysis of responses to very small variations around some
> setting of the input, e.g. nominal values. Logical sensitivity
> analysis is the effort to establish by theoretical study of the
> model, or by inspection of results of sensitivity or uncertainty
> analysis, whether the model is sensitive at all for changes in an
> input.
>
> - Elicitation
> A formal procedure to translate expert knowledge regarding input
> uncertainty into probability distributions.
>
>
> * Guidelines
>
> # All parameters should be accessible for uncertainty- and
> sensitivity analysis. The source code of a model should not
> contain unexplained numerical values.
>
> # Perform sensitivity analysis for verification of the model and
> its implementation. Repeated running of the software over a broad
> range of circumstances already constitutes a non-trivial test. Then
> check whether the qualitative behaviour of responses conforms to
> theoretical expectations.
>
> # A logical sensitivity analysis can help to detect inputs for which
> an output is entirely insensitive (factor screening). These sleeping
> inputs might be ignored in subsequent analyses. However, be aware of
> the fact that the sensitivity of an input may depend on the values
> of other inputs.
>
> # Apply factorial sensitivity analysis if you are interested in the
> interaction between inputs. This is important when the response to
> an input depends on the setting of other inputs.
>
> # Use one-at-a-time sensitivity analysis to detect irregularities,
> e.g. discontinuities, in the response such as may preclude specific
> model- calibration techniques.
>
> # For research papers on models and validation studies, an
> uncertainty analysis is highly recommended.
>
> # The establishment of input uncertainty constitutes the most
> elaborate and most critical stage of uncertainty analysis.
> Literature and experiments constitute the most natural source of
> information. Expert knowledge is another source. Be aware that
> experts in agro-ecology are not automatically experts in
> probability; formal elicitation procedures may be helpful.
>
> # Data providing information about input uncertainty pertain often
> to separate submodels. Information about correlation in uncertain
> inputs can be quite valuable since such information may greatly
> reduce output uncertainty.
>
> # Artificially generated weather data are often practical to use.
> Weather generators are also models and need to be validated.
>
> # If possible, perform uncertainty analysis for all variables
> simultaneously. For large models, the analysis may have to be
> performed for submodels separately.
>
> # Simple random sampling from the input uncertainty distribution is
> a good starting point, but latin hypercube sampling may be advisable
> for efficiency. Both methods can incorporate correlations; simple
> random sampling is conceptually simple and theoretically well
> developed.
>
> # When comparing alternative scenarios, calculate the relevant
> contrasts with the same values of the input sample. This provides
> the most efficient estimates of the scenario effects.
>
> # Uncertainty analysis may be used (and regarded) as partial
> validation: the total uncertainty about crucial model outputs
> should be acceptable for the current application. Validation
> through uncertainty analysis is only partial because structural
> uncertainty in the model, is hardly never described as 'input'
> uncertainty.
>
> # Large uncertainty contributions of individual inputs or groups of
> inputs to model output indicate that it is worthwhile to know more
> about these (groups of) inputs, whereas it is pointless to gain new
> information about other inputs. Thus, uncertainty analysis provides
> information to support decisions on research priorities.
>
> # By the same token, uncertainty analysis provides support in the
> selection of calibration parameters.
>
> # Compare estimated model-uncertainty with the current empirical
> uncertainty. Differences may be due to: structural errors in the
> model and errors in the presumed input uncertainty distribution,
> such as, absence of uncertain inputs, absence of correlations
> between inputs, erroneous specification of distributions etc.
>
>
> * References
>
> - General
> Beck, M.B., 1987. Water quality modeling: a review of the analysis
> of uncertainty. Water Resources Research 23, 1987: 1393-1442
> Bouman, B.A.M., 1994. A framework to deal with uncertainty in soil
> and management parameters in crop yield simulation; a case study
> for rice. Agricultural Systems 46: 1-17
> Hamby, D.M., 1994. A review of techniques for parameter sensitivity
> analysis of environmental models. Environmental monitoring and
> assessment 32: 135-154
> Janssen, P.H.M., 1994. Assessing sensitivities and uncertainties in
> models: a critical evaluation. In: Grasman, J. & G. van Straten
> (Eds.). Predictability and Nonlinear Modelling in Natural Sciences
> and Economics. Kluwer, Dordrecht. 344-361
> Kleijnen, J.P.C. & W. van Groenendaal, 1992. Simulation: a
> statistical perspective. Wiley.
> Kremer, J.N., 1983. Ecological implications of parameter uncertainty
> in stochastic simulation. Ecological modelling 18: 187-207
>
> - Technical aspects
> Bouman, B.A.M. & M.J.W. Jansen, 1993. RIGAUS, Random Input Generator
> for the Analysis of Uncertainty in Simulation. Simulation Report
> CABO-TT, no. 34. AB-DLO. 26 pp + appendices.
> Haness, S.J., L.A. Roberts, J.J. Warwick & W.G. Cale, 1991. Testing
> the utility of first order uncertainty analysis. Ecol. Modell. 58:
> 1-23
> Iman, R.L. & W.J. Conover, 1982. A distribution-free approach to
> inducing rank correlation among input variables. Commun. statist.
> -simual. computa. 11(3): 311-334
> Iman, R.L. & J.C. Helton, 1988. An investigation of uncertainty and
> sensitivity analysis techniques for computer models. Risk Analysis
> 8: 71-90
> Jansen, M.J.W., W.A.H. Rossing & R.A. Daamen, 1993. Monte Carlo
> estimation of uncertainty contributions from several independent
> multivariate sources. Conference predictability and nonlinear
> modeling, Wageningen, April 1993.
> Janssen, P.H.M., P.S.C. Heuberger & R. Sanders, 1993. UNCSAM 1.1: a
> software package for sensitivity and uncertainty analysis. RIVM.
> Lenthe, J. van, 1993. A blueprint of ELI: A new method for eliciting
> subjective probability distributions. Behavior Research Methods,
> Instruments & Computers 25(40): 425-433
> McKay, M.D., R.J. Beckman & W.J. Conover, 1979. A comparison of
> three methods for selecting values of input variables in the
> analysis of output from a computer code. Technometrics 21: 239-245
>
> - Weather generators
> Geng, S., F.W.T. Penning de Vries & I. Supit, 1985. Analysis and
> simulation of weather variables. Part II: Temperature and solar
> radiation. Simulation report CABO-TT 5.
> Racsko, P., L. Szeidl & M. Semenov, 1991. A serial approach to local
> stochastic weather models. Ecological Modelling 57: 27-41
>
> - Applications
> Aggarwal, P.K., 1995. Uncertainties in crop, soil and weather inputs
> used in growth models - implications for simulated outputs and
> their applications. Agricultural systems 48(3): 361-384
> Blower, S.M. & H. Dowlatabadi, 1994. Sensitivity and uncertainty
> analysis of complex models of disease transmission: an HIV model
> as an example. Internat. Statist. Review 62(2): 229-243
> Rossing, W.A.H., R.A. Daamen & M.J.W. Jansen, 1994. Uncertainty
> analysis applied to supervised control of aphids and brown rust
> in winter wheat. Part 1. Quantification of uncertainty in
> cost-benefit calculations. Agricultural Systems 44, 419-448
> Rossing, W.A.H., R.A. Daamen & M.J.W. Jansen, 1994. Uncertainty
> analysis applied to supervised control of aphids and brown rust
> in winter wheat. Part 2. Relative importance of different
> components of uncertainty. Agricultural Systems 44: 449-460
> Voet, H. van der & G.M.J. Mohren, 1994. An uncertainty analysis of
> the process-based growth model FORGRO. Forest ecology and
> management 69: 157-166
>
>
> * * *
>
> CALIBRATION
>
> * Definitions
>
> - Calibration
> The adjustment of some parameters such that model behaviour matches
> a set of real-world data; it is a restricted form of parametrization
> of models.
>
> - Calibration criterium
> A function of the parameter values and the calibration data, that
> provides a measure of the compatibility of the parameter values with
> the data.
>
> - Point calibration
> A calibration that results in a single optimal parameter vector.
> Many individual parameter vectors are often compatible with the
> available calibration data, so that the point calibration may be
> non-robust.
>
> - Set calibration
> A calibration that results in a set of parameter vectors compatible
> with the calibration data.
>
> - Distribution calibration
> A calibration that results in a probability distribution of
> parameter vectors compatible with the calibration data.
>
> - Robust calibration
> A calibration leading to results that are rather insensitive to
> minor changes in the calibration data.
>
>
> * Guidelines
>
> # Ensure that the calibration method will never result in physically
> impossible parameter vectors.
>
> # Non-sensitive parameters are a major cause of non-robustness.
> Sometimes such parameters are given a fixed typical value. Be aware
> that the calibration results are conditional on the values of these
> fixed parameters.
>
> # Many calibration methods yield local optima of the criterium:
> small changes from such an optimum give worse values of the
> criterium, but further away better values may be realized. It is
> advised to apply such methods repeatedly with different starting
> points.
>
> # Set calibration and distribution calibration may be advised in
> order to circumvent the problems with point calibration. These
> methods, however, are less well-developed, and are computer
> intensive.
>
> # Regarding the calibration method to be chosen: Use results from
> a one-at a time parameter sensitivity analysis to look whether the
> implicitly defined relations between state variables and parameters
> are continuous or discontinuous and linear or nonlinear. If the
> model response is smooth, the model can be linearized, and fast
> optimization procedures using locally linear approximation are
> possible. If discontinuous, more robust calibration procedures
> should be used.
>
> # In the proposed calibration procedures, parameter probability
> distributions, based on literature reviews or on well-documented
> expert knowledge, are assumed to be available.
>
> # Parameter choice is best based on a ranking of the model
> parameters as to their contribution to output uncertainty.
>
> # If the model is not embedded in a parameter estimating procedure,
> calibration can be executed as follows: Use sensitivity analysis to
> analyse relations between state variables. Determine independent
> subsystems, and calibrate the individual subsystems, taking care
> that once a subsystem is calibrated, that subsystem is not modified
> in following calibration steps. Calibrate a single parameter from
> each independent subsystem. This calibration method yields a point
> estimate.
>
> # If the model is embedded in an optimization procedure, calibration
> can be executed as follows: Choose parameters on the basis of their
> contribution to the output uncertainty.
>
> # Use a parameter estimation procedure in which parameter sets are
> generated according to the distributions and correlations between
> parameters established in the uncertainty analysis.
>
> # Estimate the parameters simultaneously.
>
> # The uncertainty of the parameters after calibration can be derived
> under the following conditions: The model is correct and the
> non-calibrated parameters have a negligible effect on the output
> uncertainty. To investigate the effect of non- calibrated parameters
> one should execute an uncertainty analysis.
>
> # If a model for the measurement errors is available, and the
> calibration criterium is based on it, one may execute a set- or
> distribution calibration. Both calibrations allow to quantify total
> uncertainty about crucial model outputs after calibration. This
> uncertainty should be acceptable for the application.
>
> # If the above methods are not possible, calibration becomes a work
> of art, which can yield good predictions, but provides no assessment
> of prediction uncertainty.
>
>
> * References
>
> Aldenberg, T., J.H. Janse & P.R.G. Kramer, 1995. Fitting the
> dynamical model PCLake to a multi-lake survey through Bayesian
> statistics. Ecological modelling 78: 83-99
> Beven, K.& A. Binley, 1992. The future of distributed models: model
> calibration and uncertainty prediction. Hydrological processes 6:
> 279-298
> Janssen, P.H.M. & P.S.C. Heuberger, 1995. Calibration of process
> orientated models. Ecological Modelling (to be published).
> Keesman, K. & G. van Straten, 1989. Identification and prediction
> propagation of uncertainty in models with bounded noise. Int. J.
> control 49: 2259-2269
> Klepper, O. & D.I. Rouse, 1991. A procedure to reduce parameter
> uncertainty for complex models by comparison with real system
> output illustrated on a potato growth model. Agricultural Systems
> 36 (1991) 375-395
> Molen, D.T. van der & J. Pintr, 1993. Environmental model
> calibration under different specifications: an application to the
> model SED. Ecological Modelling 68: 1-19
> Scholten, H. & M.W.M. van der Tol, 1994. SMOES: a Simulation Model
> for the Oosterschelde EcoSystem. Part II: calibration and
> validation. Hydrobiologica, 282/283: 453-474
> Straten, G. van & K.J. Keesman, 1991. Uncertainty propagation and
> speculation in projective forecasts of environmental change - a
> lake eutrophication example. J. of Forecasting 10: 163-190
>
> ====================================================================
> CAMASE: A CONCERTED ACTION FOR THE DEVELOPMENT AND TESTING OF
> QUANTITATIVE METHODS FOR RESEARCH ON AGRICULTURAL SYSTEMS AND THE
> ENVIRONMENT.
>
> CAMASE is financially supported by the European Community Specific
> Programme for Research, Technological Development and Demonstration
> in the Field of Agriculture and Agro-industry, including Fisheries.
>
> The objectives of CAMASE are to advance quantitative
> research on agricultural systems and their environment in the
> EU-countries, by improving systems research in participating
> institutes through exchange and standardization of concepts,
> approaches, knowledge, computer programs and data.
> CAMASE relates to a small network of research groups, and
> a broad group of scientists receiving information. The network
> consists of scientists from five groups in Europe: Denmark
> (Royal Veterinary and Agricultural University, Copenhagen),
> France (Institut National de la Recherche Agronomique,
> Toulouse), Spain (Cordoba University, Cordoba), Scotland
> (Institute of Ecology and Resource Management, Edinburgh) and
> The Netherlands (AB-DLO, TPE-WAU and SC-DLO, Wageningen).
>
> With CAMASE_NEWS, we aim to improve communication among
> scientists working in agro-ecosystem modelling and interested
> in better access to appropriate models, data, and related
> tools, instruction materials. CAMASE-core groups and others can
> contribute spontaneously or will be invited to contribute.
> Responsibility for the opinions expressed rests with the
> authors.
>
> CAMASE_NEWS will appear four times per year. Please submit
> news items for CAMASE_NEWS and requests for new subscriptions
> to:
>
> F.W.T. Penning de Vries/M.C. Plentinger
> DLO Research Institute for Agrobiology and Soil Fertility (AB-DLO)
> P.O.Box 14
> 6700 AA WAGENINGEN
> The Netherlands
> Telephone: +31.317.475961
> Telefax: +31.317.423110
> Internet: [log in to unmask]
>
> After an e-mail request for subscription, you will receive a
> form to give your address, which is necessary for postal
> mailings.
>
> ====================================================================
>
=======================================================================
Gerrit Hoogenboom
Associate Professor
Department of Biological and Agricultural Engineering
The University of Georgia
Griffin, Georgia 30223-1797, USA
Phone: 770-228-7216
FAX: 770-228-7218
E-Mail: [log in to unmask]
=======================================================================
|
