ADVANCE Diagnostic Database (version 1.0):
Introduction
Model diagnostics aims to characterize, compare and classify the behavior of models for climate policy analysis. The experimental setup of diagnostic studies is dedicated to generate model output that can be used to estimate a set of diagnostic indicators of model response to carbon pricing policies. The ultimate goal is to better understand differences in model behavior, enable fingerprinting of model responses, and classify models along their fingerprints. The feasibility of this approach has been demonstrated in the AMPERE (Kriegler et al., TFSC, 2015) and PIAMDDI projects (Wilkerson et al., Energy Policy, 2015).
The ADVANCE project is continuing this work by issuing a community wide call to participate in a new diagnostics study, and by offering a platform to collect the results from individual energy-economy and integrated assessment modeling teams in a single database. This database offers easy access to the diagnostic indicators developed in AMPERE (Kriegler et al., TFSC, 2015), their comparison across participating models, and thus allows teams to assess how their model is situated in the space of available models. Information on how to participate in the study, and the type of diagnostic scenarios to be submitted can be found in the ADVANCE diagnostics study protocol.
In addition, the database offers a direct comparison of the climate response to emissions scenarios. Emissions from global models are fed into the simple climate model MAGICC to allow a comparison of climate indicators from a harmonized climate model across different energy-economy and integrated assessment models. Note that the implementation of the data processing with MAGICC is still in the development phase and will be improved over the coming weeks.
The ADVANCE diagnostic database is meant to make the process of data collection and comparison more efficient and relies on accurately completed data templates as distributed below. It is planned to add more diagnostic tests in the future, as a result of which many of the variables that are listed in the data template have been classified as recommended or optional. More information about the data template structure can be found in the IAMC data template and database documentation.
Registration
Access to the database requires prior registration of model and users. Further, the registration procedure demands information about the regional composition of the model to be registered and how the native model regions map to set of predefined regions. The following steps are necessary to submit scenario data to the ADVANCE Diagnostic Database:
- A registration form needs to be completed by each modeling team and sent via e-mail to ADVANCE Diagnostic Database Administrator. The registration form contains the model name, the model's regional composition, how the native model regions map to set of predefined regions and meta data about the main characteristics of the model. In addition, basic user-related information to set up user accounts for the modeling groups is asked for. Please consult the instructions tab in the registration form for detailed instructions and additional guidance on the region mapping.
- After successful registration of the model and its users, a confirmation e-mail will be sent to the users after which scenario results can be submitted to the web-database using a standardized IAMC data template (new version including techno-economic variables). In addition to time-series data for the scenarios, it is required to submit scenario-specific meta data which is important for classifying the scenarios along several dimensions.
- After uploading a template with scenario data, you will receive an e-mail confirmation about the status of the submission and eventually log file with a detailed description of potential issues that were encountered in the import procedure. The log file should be carefully inspected and you may also want to use the visulaization capabilities of the web-database to visually check the uploaded scenario results. Please consult the instructions tab in the data template as well as the Database tutorial on this page for further details on the procedure.
A short tutorial on the use of the web database can be found below. If you experience technical problems with this database, please contact the ADVANCE Diagnostic Database Administrator.
A short tutorial on the use of the database
The Navigation tabs
At the upper end of the browser window five navigation tabs can be found that provide different functionality of the web database. These five tabs are described in more detail in the following section.About tab
The About tab provides information about the database and gives instructions on how to use the database. With exception of the About tab which can be viewed without special permissions, all other tabs require a valid user account (login and password) for the database. After entering the credentials the four additional tabs to navigate the database can be accessed. Which scenarios can be viewed depends on the individual user permissions (see User account). The difference between these three tabs for viewing is the way how scenario data can be combined for viewing.
Common Features of the Sectors, Series, and Scatter tabs
In all three view tabs the following selections can be made in the navigation bars on the upper left-hand side of the browser window:(1.) Regions: In the upper left area of the screen is a field named Regions. Depending on the tab (see above) you may select one or multiple regions for which the data is shown on the screen. Generally the regions are organized in the two folders Compare and Model Name. While in the Compare folder you can find regions that are (approximately) defined across many models (World, five macro regions and seven countries commonly compared in modeling comparison exercises, see Region definitions), the Model name folder contains the native regions that you report to the database.
(2.) Scenarios: This field includes the list of scenarios from which one or more scenarios can be selected. In addition to scenarios, for a number of variables historical and base year data can be shown to compare with scenario results. Note that historical data is only available for the regions in the Compare folder (see also previous paragraph) and that currently only some emission and energy variables are covered. Further, displaying historical data is at the moment restricted to the Sectors tab.
(3.) Variables: In this field the variables can be selected for which the data is shown on the screen. Note that in the Sectors tab it is necessary to not only required to tick one or multiple variables for selection, but also to mark a variable name (highlighted in blue) for the graph on the right hand side to be updated. It is not important which variable or variable category is marked to initiate the graph update.
The Chart Preview on the upper right-hand side of the browser window shows the graph of the selected data (variable + scenarios + regions). In addition, the horizontally oriented Query Results area in the middle of the screen shows the data in tabular format.
It is possible to export the data either into Excel or two different graphical formats (PNG = portable network graphics, SVG = scalable vector graphics). In order to do so, select one of the options in the Output Options window at the bottom of the browser window. The field titled Notes shows additional information or explanatory text for the selected variables. The availability of notes is still under development and ultimate the contents will depend on input from modeling teams.
Manage tab
The Manage tab provides an overview of the scenarios that are currently in the database, shows logs of scenario operations (additions and deletes) and allows users with sufficient permissions to upload and delete scenarios from the database.
Uploads can be initiated in the upper part of the Manage tab by means of a file selector that shows up when clicking the Browse... button. In the file selector a valid template in Microsoft Excel 2003 (*.xls) or 2007 (*.xlsx) formats can be chosen. By clicking the Upload file button the upload procedure is started. After the successful upload, a short message is shown above the file selector that acknowledges whether the upload itself has been successful and which next steps have been taken. If the uploaded file is a valid template, the import procedure will be initiated and a message in a green frame is shown. This "green" acknowledgment does not imply that the whole procedure was successful, but just means that the file upload was successful and the file has a format readable by the database import algorithm. In case the upload itself did not work or if the file has a format that is not supported by the import algorithm (e.g. Excel 1997 format which also has the *.xls extension), an error message with more details on the problem surrounded by a red frame is shown. As the import procedure, depending on the size of the template and the number of regions and scenarios, can take several minutes, you will be acknowledged about the progress by e-mail. Therefore it is important that your account information (see details about User account below) is up-to-date. The e-mail will contain information on whether the import was in general successful and the most important warnings (e.g. unknown variable names, variables not provided) and errors in its main text. A more extensive log file is attached to the e-mail. It is important that you check the log file carefully to ensure that not just the overall import was successful, but also that all data you provided made it to the database. Please note that uploading a scenario with the name of an existing scenario results in overwriting the previously existing scenarios. Upon successful import of the scenario(s), you can view the data using the three view tabs described above. It is strongly recommended to have a look at the data, making use of the online visualization capabilities of the database.
Below the upload section of the Manage tab, you can find the list of scenarios that are currently in the database. In case you are a modeling group member, you will only see the scenarios of your own model. If you are a project coordinator scenarios from multiple models will be shown here. If you have upload permissions a Delete button will be shown behind the scenario name. Clicking this button will delete the scenario from the database which cannot be reversed.
Further below the scenario table, a log section is displayed. The log section includes the date and time of successful imports, the user who uploaded the scenario(s), both model and scenario name, a link to original template file that was uploaded and a link to the log file of the respective import procedure (the same log file that was sent by e-mail).
User account
There are three types of database users:
- Modeling group members who can view scenarios of their model.
- Modeling group members who can view and upload scenarios of their model.
- Project coordinators who can view all scenarios of all models that belong to a specific comparison exercise.
Log file and data quality control
When uploading a scenario data template, a log file is automatically generated and sent to the user who initiated the upload by e-mail. The log file contains important information about the upload procedure and should be checked carefully to evaluate whether the upload can be considered successful. The log file consists of 6 sections.
- Parse DATA section: general information on the import process of the "data" sheets included in the uploaded template
- Check REGIONs section: information on region names used in the template that are not known to the import algorithm, i.e. which are among the pre-defined regions or the regions registered for the specific model
- Check VARIABLEs section: information on variable names which are not known to the import algorithm or which are part of the original template, but have not been submitted in the upload
- Check HISTORY section: comparison of the submitted base year scenario data with available statistical data (see details below)
- Run DIAGNOSTICS section: run diagnostic indicators (see details below)
- IMPORT into database section:
- Parse COMMENTs section: summary information on the scenarios that were imported into the database
Comparison of historical and scenario data
When importing scenario data into the database, some quality data control routines are in place that compare submitted base year data with available statistics. The relevant information of the comparison can be found in the log file that is sent to you after uploading new scenarios to the database. Currently statistical data is only available for the following categories: Population, GDP (MER and PPP), Primary Energy, Secondary Energy|Electricity, Final Energy, and Fossil and Industrial CO2 emissions for the World. Historical data is from various sources, i.e. from UN (population), the World Bank (GDP), the IEA Energy Balances (energy) and CDIAC (CO2 emissions). The comparison will be performed for the first year submitted which is assumed to be the base year a model is calibrated to. The following rules are applied to global level data in the comparison at present:
- An information (INFO) or warning (WARN) is issued if the model's base year data is outside the range of a factor of 1/(1 + f) to 1 + f around the historical value where f is - depending on the variable - 10 to 25%.
- An error (ERROR) is issued if the model's base year data is outside the range of a factor of 1/(1 + f) to 1 + f around the historical value where f is - depending on the variable - 25 to 50% which in many cases will hint at a unit issue or similar. An error will lead to the rejection of the scenario submission.
Running Diagnostic Indicators
After checking a scenario submission (for REGION, VARIABLES, HISTORY; see above) a couple of diagnostic indicators is generated. In version 1.0 of the ADVANCE Diagnostic Database (2014-07-01) the diagnostic tests run are:
- 'MetaCheck' - Check whether required scenario meta data are submitted. In case they are missing file will be rejected
- 'magicc6' - Run magicc6 climate model on input data
- 'tiPE' - Transformation Index (Primary Energy)
- 'tiFE' - Transformation Index (Final Energy)
- 'rai' - Relative Abatement Index
- 'cioei' - Carbon Intensity over Energy Intensity
- 'cav' - Costs per Abatement Value
Region definitions
The consolitated results in the database are shown at regional aggregations of the World, five macro regions and seven individual countries/regions that are commonly used in scenario analysis. These regions are defined as follows:
Aggregation on the five region level
OECD90+EU = Includes the OECD 1990 countries as well as EU members and candidates.
Albania,
Australia,
Austria,
Belgium,
Bosnia and Herzegovina,
Bulgaria,
Canada,
Croatia,
Cyprus,
Czech Republic,
Denmark,
Estonia,
Fiji,
Finland,
France,
French Polynesia,
Germany,
Greece,
Guam,
Hungary,
Iceland,
Ireland,
Italy,
Japan,
Latvia,
Lithuania,
Luxembourg,
Malta,
Macedonia,
Montenegro,
Netherlands,
New Caledonia,
New Zealand,
Norway,
Poland,
Portugal,
Romania,
Samoa,
Serbia,
Slovakia,
Slovenia,
Solomon Islands,
Spain,
Sweden,
Switzerland,
Turkey,
United Kingdom,
United States of America,
Vanuatu
REF = Countries from the Reforming Ecomonies of the Former Soviet Union.
Armenia,
Azerbaijan,
Belarus,
Georgia,
Kazakhstan,
Kyrgyzstan,
Republic of Moldova,
Russian Federation,
Tajikistan,
Turkmenistan,
Ukraine,
Uzbekistan
ASIA = The region includes most Asian countries with the exception of the Middle East, Japan and Former Soviet Union states.
Afghanistan,
Bangladesh,
Bhutan,
Brunei Darussalam,
Cambodia,
China,
China Hong Kong SAR,
China Macao SAR,
Democratic People's Republic of Korea,
East Timor,
India,
Indonesia,
Lao People's Democratic Republic,
Malaysia,
Maldives,
Mongolia,
Myanmar,
Nepal,
Pakistan,
Papua New Guinea,
Philippines,
Republic of Korea,
Singapore,
Sri Lanka,
Taiwan,
Thailand,
Viet Nam
MAF = This region includes the countries of the Middle East and Africa.
Algeria,
Angola,
Bahrain,
Benin,
Botswana,
Burkina Faso,
Burundi,
Cameroon,
Cape Verde,
Central African Republic,
Chad,
Comoros,
Congo,
Cote d'Ivoire,
Democratic Republic of the Congo,
Djibouti,
Egypt,
Equatorial Guinea,
Eritrea,
Ethiopia,
Gabon,
Gambia,
Ghana,
Guinea,
Guinea-Bissau,
Iran (Islamic Republic of),
Iraq,
Israel,
Jordan,
Kenya,
Kuwait,
Lebanon,
Lesotho,
Liberia,
Libyan Arab Jamahiriya,
Madagascar,
Malawi,
Mali,
Mauritania,
Mauritius,
Morocco,
Mozambique,
Namibia,
Niger,
Nigeria,
Oman,
Qatar,
Reunion,
Rwanda,
Saudi Arabia,
Senegal,
Sierra Leone,
Somalia,
South Africa,
Sudan,
Swaziland,
Syrian Arab Republic,
Togo,
Tunisia,
Uganda,
United Arab Emirates,
United Republic of Tanzania,
Western Sahara,
Yemen,
Zambia,
Zimbabwe
LAM = This region includes the countries of Latin America and the Caribbean.
Argentina,
Bahamas,
Barbados,
Belize,
Bolivia,
Brazil,
Chile,
Colombia,
Costa Rica,
Cuba,
Dominican Republic,
Ecuador,
El Salvador,
Guadeloupe,
Guatemala,
Guyana,
Haiti,
Honduras,
Jamaica,
Martinique,
Mexico,
Netherlands Antilles,
Nicaragua,
Panama,
Paraguay,
Peru,
Puerto Rico,
Suriname,
Trinidad and Tobago,
Uruguay,
Venezuela
Seven individual countries/regions commonly used in scenario analysis
Brazil = Federative Republic of Brazil
China = People's Repulic of China
India = Republic of India
EU = European Union (27 member countries)
Japan = State of Japan
Russia = Russian Federation
USA = United States of America
Frequently Asked Questions (under development)
- Do I need to register different model version separately?
Yes, in case you want to upload scenarios that have been generated with different versions of your model, all model version need to be registered separately, because they will most likely have different features on the "meta_model" tab of the registration form (e.g., different number of regions, different technology coverage). Please make sure to add a version tag to the model name, which then needs to be in all "Model" columns in the registration form, including the "user_data", "meta_model", "model_regions" and the "region_aggregation" tabs and in the data templates. - What happens if I resubmit a previously submitted scenario?
The previously submitted version of the scenarios gets fully replaced by the new version, i.e. it is first deleted and a new import is started. - Can I submit a scenario batch in multiple files?
Yes, multiple scenarios can be submitted either in a single file (on a single "data" tab or in multiple tabs the names of which start with "data", such as "data2", "data3", etc.) or in several files (e.g. one scenario per file). Note, however, that it is not possible to submit data belonging to the same scenario in multiple files, because the last file with data belonging to the scenario will always overwrite existing data of that scenario. For diagnostic scenarios, it is important that a the corresponding baseline scenario has either been submitted prior to the other scenarios or that the baseline is part of the scenario batch. - Why do I get an error message that states that I am not allowed to submit scenario data for my model?
Several reasons exist for such an error message, (i) you may have been registered as a user without submission rights, i.e. you can view scenario data, but not submit, (ii) the model name may include a typo (e.g. different capitalization than in the registered version), (iii) sometimes rows that appear empty are left at the bottom of the data block on the "data" tab which trigger an error about model "." . - I am going to submit several scenarios in a single file. How should I provide accompanying information on the "meta_scenario" tab?
Please use multiple rows in the "meta_scenario" tab by copying the existing row 3 in the tab as often as needed. Please note that it is mandatory to submit meta data for all scenarios that are included in a template. - Does the order of the variables on the "data" tab matter?
No, the variable order is absolutely arbitrary, but it is important that each row is completed correctly. Model, scenario, region and variable name need to be provided in each row, since the information is processed automatically. Please note that no spelling information of the model, region and variable names are tolerated.
ADVANCE Diagnostic Database, 2015
Available at: https://tntcat.iiasa.ac.at/ADVANCEWP1DB
Responsible for this page: ADVANCE Diagnostic Database Administrator