The default methodology adopted by Agrosmart for precipitation forecasting evaluation reports is described below.
Contingency Table
The methodology is based on a Contingency Table (Table 1).
This is an easy to understand and simple to build table. The objective is to define the number of times a given event was predicted or not according to what was observed. It has the following form:
TABLE 1: contingency table.
Thus, according to the table above: “a” corresponds to the number of events predicted and that have occurred, true positive; "b" is the number of events predicted, but did not occur, false positive; "c" is the number of events that occurred, but were not predicted, false negative; and "d" is the number of events that were not predicted and did not occur, true negative.
For each report, a forecast point is compared with the nearest weather station or rain gauge.
Agrosmart Metrics
From these four parameters (“a”, “b”, “c” and “d”), it is possible to calculate the following metrics:
Table 2: Metrics
Event Definition
The definition of a forecasted precipitation event can be considered in several ways, according to the objectives of the analysis. If there no criteria was defined for what is to be considered an “precipitation event”, the default one will be used. At Agrosmart, the default criteria for “precipitation event” is:
● Its probability is greater than 50%;
● If the accumulated for the day is equal to or greater than 2.5mm.
If any of these conditions are not met, the event will be considered as “not forecasted”.
Similarly, for an observed precipitation event to be considered, the daily accumulated precipitation should be greater than 2.5mm.
How to analyze the result
To explain how we should interpret the results, we present the following example:
"For a given forecast point, a study was carried out for 90 days. In this study, we evaluate the "forecast for tomorrow" (D + 1) in relation to the observed according to the standard methodology of Agrosmart."
Therefore, we will have 90 events that will be distributed according to the contingency table. The result is shown below:
TABLE 3: Example of Results presented in a Contingency Table
With this data it is possible to calculate Agrosmart metrics.
TABLE 4: Example of Result
Based on these figures, we can look to the future and estimate how the assertiveness of the forecast should be:
● Considering 100 next days, in approximately 87 of these days the Agrosmart forecast will be correct.
● 9 out of 10 rain events that occur will be correctly forecasted by Agrosmart.
Important Considerations
Atmospheric modeling is a highly complex procedure, it depends on numerous factors and a simple change in one of them is enough for the simulation result to be completely different from the expected. There are different meteorological systems operating in Brazil and in the world, at different times of the year and with different spatial and temporal characteristics, so it is natural that the assertiveness of the weather forecast varies throughout the year.
No model is considered to be a faithful representation of the atmosphere and no forecast will be correct at all times. Agrosmart works actively, with a team of highly qualified meteorologists and agronomists, to constantly improve the quality of forecasts always seeking to deliver maximum value to our customers.