UPDATING TABLE DRIVEN CODE FORMS

18 April 2006(Joël Martellet,

WMO, World Weather Watch, Data Processing and Forecasting Systems)

UPDATING THE TABLE DRIVEN CODE FORMS

• General Procedures• All amendments to BUFR, CREX and GRIB must be proposed in

writing to the WMO Secretariat. The proposal must specify the needs, purposes and requirements and include information on a contact point for technical matters.

• An Expert Team on Data Representation and Codes (ET/DRC) under the Commission for Basic Systems (CBS) Open Programme Area Group on Information Systems and Services (OPAG/ISS), supported by the Secretariat, validates then the stated requirements and develops a draft recommendation to respond to the requirements as appropriate.

• What happens next depends on whether the draft recommendation involves changes to the code structure or new functions for the decoder/encoders, or simply additions to the supporting tables.

Structure change: example Section 1 BUFRSection 1 - Identification Section.

OctetNo.

Contents

1 – 3 Length of section, in octets

4 BUFR master table number – this provides for BUFR to be used torepresent data from other disciplines, and with their own versions of mastertables and local tables. For example, this octet is zero for standard WMOFM 94 BUFR tables, but ten for standard IOC FM 94 BUFR Tables whoseuse is focused on oceanographic data.

5 – 6 Originating centre: code table 0 01 033

7 Update sequence number (zero for original BUFR messages; incrementedfor updates)

8 Bit 1 = 0 No optional section

= 1 Optional section included

Bits 2 – 8 set to zero (reserved)

9 Data Category type (BUFR Table A)

10 Data Category sub-type (defined by local ADP centres)

11 Version number of master tables used (currently 9 for WMO FM 94 BUFRtables)

12 Version number of local tables used to augment the master table in use

13 Year of century

14 Month

15 Day

16 Hour

17 Minute

13-14 Year (4 digits) |15 Month |16 Day |17 Hour |18 Minute |19 Second |

Proposednew date format

Other important changes

• Addition of a new Table C operator in BUFR or CREX:– This implies a new function for the encoder and

decoder, therefore program modifications.

• Additions of new Template(s) in GRIB Edition 2:– This implies new “sub-structure(s)” for the

encoder and the decoder, therefore program modifications.

Code changes which do not require substantial modification of the encoding/decoding programs

• Type of concerned modifications– Additions to Tables A, B, D (BUFR/CREX) and Code or Flag

Tables (BUFR/CREX/GRIB2)

• Applications adjustments– The encoding software should have access to the new version of

Tables A, B, D (BUFR/CREX) and Code or Flag Tables (BUFR/CREX/GRIB2). It must be changed to insert the new value(s), if the new value(s) is(are)used.

– The decoding software should have access to the new version of Tables A, B (BUFR/CREX), D (BUFR/CREX) and Code or Flag Tables, but the program itself does not need to be changed.

– Applications further down beyond the decoder have to be updated to use or discard the new decoded value(s)

Code changes which do require substantial modifications of the encoding/decoding programs

• Type of concerned modifications:

– Modification of the code structure, or introduction of new BUFR/CREX Table C operators (it makes a new edition), or insertion of new GDT, PDT or DRT (GRIB 2 templates)

• Applications adjustments:

– The encoder software must be modified to generate the new code structure (BUFR/CREX/GRIB2) or/and to use the new BUFR/CREX Table C operators

– The GRIB2 encoder software must be modified if the building of new template(s) is needed

– The decoder software must be modified

– Applications further down beyond the decoder have to be updated to use or discard the new decoded value(s) or the new decoded template(s)

Updating the Structures

• When the recommended solution developed by the Expert Team on Data Representation and Codes requires changes to the BUFR, CREX or GRIB code structures, or new Table C operators, the recommendation must be approved by both the full CBS and the full WMO Executive Council.

• However, it must first be endorsed by the Chairperson of OPAG/ISS, who will ensure the validation (encoding-decoding of reports with new structure by two independent encoders and decoders (e.g. ECMWF and NCEP-USA) has been performed, prior to its consideration by CBS.

• This must be done early enough that the draft recommendation can be published as a CBS pre-session document at least three months prior to the CBS Session.

Additional entries to the Tables(1)• In principle, one should make only additions to the set of entries in a Table;

existing entries must be not modified. Why? Because of the archived data. One wants to keep always the facility of decoding old parameters (e.g. re-analysis projects).

• Table additions could follow, in principle, the same approval process as changes to the code structures, however, table additions are not only, far less disruptive than code structure changes, they are also required more often and with greater urgency.

• Therefore, a special approval process has been developed by the WMO Secretariat to ensure the necessary flexibility is available to respond to urgent requirements of users during inter-session periods (i.e., between Sessions of CBS).

• This approval process is referred to as the "Fast Track". Under this procedure, the recommendation does not need to be approved by the full CBS and the full EC. Rather, after approval by the Chairpersons of the ET/DR&C and of OPAG/ISS, the recommendation need only to be approved by the president of the CBS on behalf of CBS.

Additional entries to the Tables(2)• New Entries in Tables are allocated after consideration of the request

by the Secretariat, and approval by the Chairpersons of the ET/DRC and OPAG/ISS, and the CBS president.

• Then: these new entries are listed in the WMO web server in the corresponding file: “New entries awaiting validation”

• Then the validation of these new entries must be performed and demonstrated to the Chairpersons of the ET/DRC and OPAG/ISS, who will recommend to the President of CBS to declare these new entries pre-operationalValidation = encoding-decoding containing the new entries by two independent encoders and decoders (e.g. ECMWF and NCEP-USA)

• Then the entries validated will be stored in the file of “pre-operational entries”, which can be used for experimental pre-operational exchanges between centres.

• Finally, this will be approved every two year by CBS and EC for full operational implementation, the fist Wednesday following the first day of next November.

IMPLEMENTATION PROCEDURES FOR ADDITIONS OF DESCRIPTORS IN BUFR/CREX TABLES A, B AND D, GRIB TABLES AND NEW GRIB TEMPLATES -- TABLES LISTED IN

WMO WEB SERVER

Requirements to be justified for: ALLOCATED

ENTRIES (AWAITING

VALIDATION)

Approved by chairs of ET/DR&C, OPAG ISS

and CBS president PREOPERATIONAL IMPLEMENTATION

(VALIDATED ENTRIES)

Approved by CBS and EC

FOR FULL IMPLEMENTATION

(INCLUDED IN MANUAL ON CODES)

In all cases, WMO Members must be notified of amendments approved early enough to allow a period at least three months between the receipt of the notification and the date of full operational implementation.

IMPLEMENTATION PROCESS FOR ADDITIONS

Time-table for implementing additions to Codes Tables

• WMO Codes group experts (ET/DR&C) exchange emails and meet once per year

• Additions are validated and declared pre-operational, usually once a year (or more often if needed)

• CBS endorses every two year (e.g. Fall 2006)• Executive Council approves (Spring 2007), then:• Fully operational on first Wednesday after the 1st November

(2007) (so, there is always at least one year delay after CBS approval)

• Non table driven Codes: TAC= (SYNOP, SATOB, etc.) should not be changed anymore (except Aviation codes!)

• New requirements are met by migrating to BUFR (or CREX if necessary) and to GRIB 2

Thanks for your attention!

QUESTIONS?