GPP/RE discussion

• Who am I:– Ankur Desai– National Center for Atmospheric Research

• What we’re doing

• Initial results

• What we have

• What should we do

• Plan of action

What we’re doing

• Using same dataset from NEE gap filling (12 site-years, 51 scenarios/site) to comparing GPP/RE across methods

• 10 of 15 methods produce GPP/RE– Neural networks and tables do not?– 9 of these analyzed so far– With variants, currently at 19 analyzed

• Unlike NEE, no benchmark– However, BETHY model is part of group– Can use BETHY as benchmark– Because I’m lazy

What we’re doing

• Hypotheses:– Intramethod variability in GPP/RE <

Intermethod variability for any site• i.e., insensitive to most gaps

– Site GPP/RE estimates vary < 20% across methods

– Using BETHY as model benchmark, mean of other methods is similar to BETHY GPP/RE

• More sophisticated methods have less difference to BETHY GPP/RE than simpler methods

– Variability in GPP/RE < GPP or RE– Others?

What we’re doing

Name Abbrev. Author Type Color

BETHY_365D B365 Kattge process model blueBETHY_12D B12 Kattge process model blue

NLR_A NA Noormets regression (AQRTa) magentaNLR_EM NE Desai regression (Eyring) orange

NLR_FM_AD NFA Richardson regression (abs dev) brownNLR_FM_OLS NFO Richardson regression (least sq) brown

NLR_LM_INT_DC NLID Falge regression (int) redNLR_LM_INT_LU NLIL Falge regression (int) redNLR_LM_INT_RE NLIR Falge regression (int) redNLR_LM_TA_DC NLTD Falge regression (air temp) redNLR_LM_TA_LU NLTL Falge regression (air temp) redNLR_LM_TA_RE NLTR Falge regression (air temp) redNLR_LM_TS_DC NLSD Falge regression (soil temp) redNLR_LM_TS_LU NLSL Falge regression (soil temp) redNLR_LM_TS_RE NLSR Falge regression (soil temp) red

NLR_FCRN_1 NC1 Barr regression red brownNLR_FCRN_2 NC2 Barr regression (with intercept) red brown

SPM SPM Stauch semi-parametric greenUKF UKF Hollinger Kalman filter green

MDS MDS Reichstein Marginal samplingMIM MIM Hui Multiple imputation  

Initial results

• To date, all datasets have been processed and put in common binary format

• Daily and annual sums of GPP,RE computed

• Mean and variance across 51 replicates computed and across methods computed

• Diagnostic plots made– Box plots look at GPP/RE across methods

(letters) , replicates (gray bars), mean and st.dev (+) across methods and range (box)

– Colors delineate method type

Initial results

• Other plots– GPP/RE plots– Cumulative 2-week smoothed GPP (negative

values) and RE (positive values) to look at when methods diverge for a site

• Line is method mean, shadow is variance across replicates

– Similar plots made for growing season (mid-May to mid-Sept) and dormant season (all other months)

– Benchmark plots look at similar data as percent different from BETHY full run

Initial results (annual)

Initial results (annual)

Initial results (annual)

Initial results (cumulative)

Initial results (cumulative)

Initial results (cumulative)

Initial results (cumulative)

Initial results (summer)

Initial results (summer)

Initial results (dormant season)

Initial results (dormant season)

What do we have

• A lot of messy plots!– Data reduction is hard (5 GB of data!)– Tables might work better?

• Have not done hypothesis testing yet– Generally looks like intermethod var. is higher

than 20% in some cases and biases in some methods compared to BETHY

– Intramethod var. << intermethod var.– Will we be able to tease mechanistic reasons

for method differences from this analysis?– Can we make any recommendations?

GPP table  be1_2000 be1_2001 de3_2000 de3_2001 fi1_2001 fi1_2002 fr1_2001 fr1_2002 fr4_2002 il1_2002 it3_2001 it3_2002

BETHY_365D 1398 ± 0 1359 ± 0 1588 ± 0 1476 ± 0 984 ± 0 982 ± 0 1921 ± 0 2003 ± 0 1380 ± 0 1212 ± 0 1464 ± 0

BETHY_12D 1377 ± 0 1446 ± 0 1535 ± 0 1520 ± 0 1099 ± 0 1021 ± 0 1892 ± 0 2018 ± 0 1414 ± 0 1245 ± 0 1391 ± 0

NLR_A 1527 ± 7 1704 ± 9 1730 ± 12 1592 ± 11 1039 ± 7 1133 ± 7 1986 ± 15 2033 ± 15 1577 ± 9 783 ± 73 1199 ± 8 1485 ± 15

NLR_EM 1365 ± 9 1458 ± 9 1542 ± 8 1467 ± 7 915 ± 4 972 ± 6 1729 ± 12 1791 ± 10 1416 ± 9 618 ± 7 965 ± 10 1221 ± 11

NLR_FM_AD 1385 ± 10 1485 ± 10 1539 ± 13 1460 ± 13 955 ± 6 1023 ± 7 1768 ± 16 1806 ± 15 1344 ± 10 607 ± 10 1011 ± 10 1321 ± 16

NLR_FM_OLS 1424 ± 9 1525 ± 11 1588 ± 11 1491 ± 12 973 ± 5 1048 ± 6 1829 ± 15 1873 ± 13 1376 ± 10 626 ± 10 1057 ± 11 1378 ± 14

NLR_LM_INT_DC 1471 ± 23 1547 ± 29 1578 ± 23 1560 ± 16 936 ± 11 1000 ± 12 2028 ± 40 1993 ± 22 1297 ± 21 774 ± 59 1065 ± 11 1405 ± 18

NLR_LM_INT_LU 1468 ± 26 1575 ± 23 1483 ± 18 1514 ± 20 926 ± 19 1020 ± 21 1981 ± 33 2017 ± 26 1342 ± 24 812 ± 51 1126 ± 24 1444 ± 27

NLR_LM_INT_RE 1553 ± 28 1637 ± 25 1549 ± 19 1544 ± 22 952 ± 20 1049 ± 24 2048 ± 33 2108 ± 29 1382 ± 27 1761 ± 5 1141 ± 22 1484 ± 25

NLR_LM_TA_DC 1533 ± 9 1602 ± 16 1548 ± 9 1451 ± 9 1013 ± 9 1072 ± 8 1942 ± 18 1983 ± 15 1457 ± 8 492 ± 18 1068 ± 8 1347 ± 13

NLR_LM_TA_LU 1502 ± 21 1617 ± 21 1525 ± 17 1482 ± 18 983 ± 16 1037 ± 16 1935 ± 20 1991 ± 20 1467 ± 23 535 ± 21 1052 ± 19 1342 ± 23

NLR_LM_TA_RE 1508 ± 20 1619 ± 22 1522 ± 18 1491 ± 19 997 ± 16 1051 ± 18 1944 ± 21 1991 ± 18 1478 ± 24 584 ± 21 1037 ± 18 1364 ± 24

NLR_LM_TS_DC 1438 ± 8 1507 ± 14 1546 ± 10 1444 ± 9 952 ± 8 1011 ± 7 1835 ± 16 1880 ± 13 1457 ± 8 492 ± 18 1022 ± 8 1274 ± 12

NLR_LM_TS_LU 1413 ± 20 1529 ± 20 1522 ± 16 1476 ± 18 930 ± 18 984 ± 17 1825 ± 19 1884 ± 19 1467 ± 23 535 ± 21 1017 ± 18 1268 ± 22

NLR_LM_TS_RE 1415 ± 20 1530 ± 21 1518 ± 18 1484 ± 18 941 ± 18 993 ± 20 1831 ± 19 1883 ± 18 1478 ± 24 584 ± 21 1084 ± 210 1282 ± 22

SPM 1442 ± 10 1513 ± 10 1583 ± 9 1509 ± 17 959 ± 5 1022 ± 6 1818 ± 18 1891 ± 11 1419 ± 10 631 ± 7 918 ± 9 1327 ± 10

UKF 1560 ± 10 1649 ± 12 1767 ± 25 1666 ± 18 1076 ± 13 1168 ± 17 2101 ± 25 2143 ± 24 1700 ± 14 888 ± 13 1274 ± 38 1668 ± 23

NLR_FCRN_1 1417 ± 9 1512 ± 12 1580 ± 9 1502 ± 10 964 ± 5 1017 ± 8 1822 ± 18 1860 ± 15 1424 ± 7 573 ± 13 1038 ± 9 1314 ± 9

NLR_FCRN_2 1441 ± 9 1523 ± 12 1613 ± 10 1526 ± 8 972 ± 5 1023 ± 9 1852 ± 14 1886 ± 14 1422 ± 7 608 ± 7 1040 ± 8 1355 ± 11

 

mean 1462 ± 15 1561 ± 16 1572 ± 14 1509 ± 14 970 ± 11 1037 ± 12 1898 ± 21 1942 ± 17 1441 ± 15 700 ± 22 1066 ± 26 1369 ± 17

var 60 ± 7 67 ± 7 73 ± 5 57 ± 5 42 ± 6 51 ± 6 105 ± 8 99 ± 5 92 ± 8 289 ± 20 97 ± 48 105 ± 6

max 1593 1722 1885 1717 1127 1195 2170 2217 1748 1771 1868 1726

min 1285 1394 1409 1402 803 863 1694 1754 1197 405 893 1135

RE table be1_2000 be1_2001 de3_2000 de3_2001 fi1_2001 fi1_2002 fr1_2001 fr1_2002 fr4_2002 il1_2002 it3_2001 it3_2002

BETHY_365D 1049 ± 0 872 ± 0 1051 ± 0 953 ± 0 842 ± 0 788 ± 0 1329 ± 0 1437 ± 0 1061 ± 0 926 ± 0 1475 ± 0

BETHY_12D 1003 ± 0 908 ± 0 1236 ± 0 1045 ± 0 834 ± 0 849 ± 0 1313 ± 0 1547 ± 0 1051 ± 0 967 ± 0 1357 ± 0

NLR_A 1182 ± 10 1167 ± 14 1217 ± 12 1110 ± 9 855 ± 9 918 ± 7 1414 ± 12 1459 ± 10 1255 ± 11 651 ± 126 905 ± 8 1460 ± 16

NLR_EM 989 ± 10 896 ± 11 981 ± 10 924 ± 5 720 ± 5 725 ± 7 1119 ± 9 1190 ± 10 1076 ± 11 385 ± 6 658 ± 7 1134 ± 14

NLR_FM_AD 1027 ± 13 934 ± 8 975 ± 7 953 ± 6 772 ± 5 794 ± 7 1179 ± 7 1224 ± 8 1023 ± 8 431 ± 13 717 ± 12 1278 ± 16

NLR_FM_OLS 1074 ± 12 984 ± 11 1038 ± 8 995 ± 7 796 ± 5 826 ± 5 1251 ± 9 1300 ± 9 1059 ± 9 446 ± 12 779 ± 12 1353 ± 14

NLR_LM_INT_DC 1127 ± 20 1042 ± 31 1048 ± 30 1051 ± 15 768 ± 14 785 ± 14 1438 ± 38 1414 ± 23 963 ± 23 578 ± 55 789 ± 12 1379 ± 17

NLR_LM_INT_LU 1133 ± 30 1047 ± 28 955 ± 25 974 ± 24 751 ± 30 795 ± 31 1403 ± 37 1452 ± 32 1008 ± 28 615 ± 50 853 ± 24 1422 ± 37

NLR_LM_INT_RE 1217 ± 33 1110 ± 29 1025 ± 26 1006 ± 27 776 ± 30 829 ± 35 1469 ± 35 1539 ± 33 1052 ± 31 1559 ± 5 870 ± 22 1476 ± 35

NLR_LM_TA_DC 1189 ± 12 1097 ± 16 1018 ± 11 942 ± 8 845 ± 10 856 ± 8 1352 ± 16 1404 ± 15 1123 ± 10 296 ± 11 792 ± 10 1321 ± 16

NLR_LM_TA_LU 1168 ± 26 1089 ± 27 997 ± 26 941 ± 23 808 ± 27 811 ± 25 1357 ± 28 1425 ± 26 1133 ± 26 336 ± 27 779 ± 21 1320 ± 34

NLR_LM_TA_RE 1173 ± 26 1092 ± 27 998 ± 28 953 ± 24 821 ± 27 831 ± 29 1365 ± 28 1422 ± 24 1149 ± 29 372 ± 25 766 ± 19 1356 ± 32

NLR_LM_TS_DC 1093 ± 10 1002 ± 14 1016 ± 11 935 ± 8 784 ± 9 796 ± 7 1245 ± 13 1301 ± 13 1123 ± 10 296 ± 11 746 ± 9 1248 ± 15

NLR_LM_TS_LU 1078 ± 24 1001 ± 26 994 ± 26 936 ± 23 755 ± 29 759 ± 27 1247 ± 27 1318 ± 25 1133 ± 26 336 ± 27 744 ± 20 1246 ± 32

NLR_LM_TS_RE 1079 ± 25 1003 ± 26 994 ± 27 946 ± 24 764 ± 29 773 ± 31 1252 ± 27 1314 ± 24 1149 ± 29 372 ± 25 814 ± 210 1273 ± 31

SPM 1098 ± 10 980 ± 12 1045 ± 7 982 ± 22 808 ± 12 822 ± 8 1256 ± 13 1325 ± 12 1077 ± 9 465 ± 7 569 ± 5 1306 ± 13

UKF 1210 ± 16 1116 ± 19 1310 ± 26 1152 ± 20 905 ± 11 970 ± 17 1568 ± 25 1590 ± 23 1430 ± 16 875 ± 24 1064 ± 28 1699 ± 22

NLR_FCRN_1 1072 ± 11 978 ± 13 1059 ± 8 992 ± 6 789 ± 7 795 ± 10 1245 ± 11 1315 ± 13 1081 ± 8 397 ± 13 756 ± 7 1284 ± 11

NLR_FCRN_2 1083 ± 13 988 ± 15 1073 ± 11 1002 ± 7 796 ± 17 794 ± 10 1263 ± 12 1316 ± 11 1111 ± 6 427 ± 4 758 ± 7 1331 ± 16

 

mean 1117 ± 18 1031 ± 19 1044 ± 18 988 ± 15 795 ± 16 816 ± 16 1319 ± 20 1371 ± 18 1114 ± 17 520 ± 26 786 ± 25 1346 ± 22

var 66 ± 8 73 ± 8 89 ± 9 64 ± 8 47 ± 10 59 ± 11 114 ± 11 104 ± 8 104 ± 9 300 ± 30 115 ± 48 122 ± 9

max 1260 1204 1362 1235 932 1015 1617 1670 1477 1567 1590 1762

min 921 835 803 783 548 563 1065 1150 833 158 558 1036

What should we do

• Other hypotheses

• Other kinds of benchmarks/models

• Other kinds of comparisons– Artificial data?

• Technical approach– Different kinds of figures– Different analysis techniques / stats

• Philosophical questions

• Is this manuscript worthy?

Plan of action• Get all data!

– mixed gap runs - late oct– no il1, it3_2001– Jens to give Dave BETHY data (all sites), Dave to corrupt, Antje to

gapify (10 mixed scenarios 35% missing + 0% missing) - next 3-4 mos– 10 Mixed gaps only + r0– Fill and decompose as you would when publishing GPP/RE for your

sites - final sets Spring 07

• Run other benchmarks, tests if needed– Running corrupt data through methods– Seasonal diurnal plots– ANOVA of GPP or RE for site x method– Find independent data (chamber, inventory, etc…)

• Share data• Discuss - Desai to create wiki• Write a manuscript - Dec.• Delegate tasks