Here are the contents of the trace data file: Number of plots available = 1 PLOT # 1 is a skill score plot composed of 4 trace(s). INDEPENDENT VARIABLE FOR THIS PLOT: THRSH STAT = FHO GENERAL LABELS # FLD NAM LABEL 1 FHOUR 48 2 LEVEL SFC 3 V_ANL ANOWPM 4 V_RGN G246 5 STATS FHO 6 PARAM PM25MX 7 VYMDH 201808010000-201808312359 STAT = PARM = PM25MX TRACE # 1 LABELS # FLD NAM LABEL 1 MODEL CMAQ5X/148 STAT = PARM = PM25MX TRACE # 2 LABELS # FLD NAM LABEL 1 MODEL FV3CMAQV502_9/148 STAT = PARM = PM25MX TRACE # 3 LABELS # FLD NAM LABEL 1 MODEL CMAQ5XBC/148 STAT = PARM = PM25MX TRACE # 4 LABELS # FLD NAM LABEL 1 MODEL FV3CMAQV502BC/148 Select number of the plot to display (0=exit): Here are the traces in the plot # 1: Number of plots available = 1 PLOT # 1 is a skill score plot composed of 4 trace(s). INDEPENDENT VARIABLE FOR THIS PLOT: THRSH STAT = FHO GENERAL LABELS # FLD NAM LABEL 1 FHOUR 48 2 LEVEL SFC 3 V_ANL ANOWPM 4 V_RGN G246 5 STATS FHO 6 PARAM PM25MX 7 VYMDH 201808010000-201808312359 STAT = PARM = PM25MX TRACE # 1 LABELS # FLD NAM LABEL 1 MODEL CMAQ5X/148 STAT = PARM = PM25MX TRACE # 2 LABELS # FLD NAM LABEL 1 MODEL FV3CMAQV502_9/148 STAT = PARM = PM25MX TRACE # 3 LABELS # FLD NAM LABEL 1 MODEL CMAQ5XBC/148 STAT = PARM = PM25MX TRACE # 4 LABELS # FLD NAM LABEL 1 MODEL FV3CMAQV502BC/148 Select plot display type by number: 0 - Print statistics for traces in plot defined above. 1 - Graphical display of plot as defined above. 2 - Scatter plot (Requires 2 series of computational results, either from the same trace or two different traces.) 3 - Old-style histogram (requires exactly one trace) 4 - Display scatter plot data as line plots 5 - Display ROC curves (select comp code 1032, 1362, or 1363 below) 6 - Display attribute diagram (select comp code 1360 below) 7 - Display quantile reliability diagram (select comp code 2307 below) Enter choice: Select the computational method by number from the list of items below: 1 - Enter one computational code to apply to all traces 2 - Explicitly enter trace numbers and corresponding computational codes 3 - Explicitly enter two trace numbers and one computational code to compute and display simple differences 4 - Explicitly enter two trace numbers and one computational code to compute and display % differences 5 - Explicitly enter two trace numbers and one computational code (if trace numbers are identical, enter two different computational codes) to compute and display skill score using the second trace as the reference--- ss=(t1-t2)/(prfct score - t2) 6 - Explicitly enter two trace numbers and one computational code (if trace numbers are identical, enter two different computational codes) to compute statistical significance of the differences of the paired values of the two traces Enter choice: ************************************************************************* * FHO -- Forecast, hit, observation rate statistics * * * * PM = column label for computational codes for performance measures * * CPR = column label for computational codes for critical performance * * ratios (CPR) for the performance measures * * * * Computation Codes Action * * * * PM CPR * * * * 9002 Trace of data field values averaged over all pts* * -n Return raw data for field n (undivided sum) * * -1 Contributing record count at all trace points * * 1 Return data from field 1 = N, the data count * * 2 Return data from field 2 = F, fraction forecast * * 3 Return data from field 3 = H, fraction correct * * 4 Return data from field 4 = O, fraction observed * * * * ----- Additional fields for computational prefix 104: * * * * 5 Return extension field 5 = H', bias removed * * fraction correct * * 6 Return optional ex fld 6 = F', bias removed * * fraction forecast * * Note: If field 5 is present without field 6, * * then F' = O = the observed fraction. * * * * 1001 Compute BIAS from FHO values * * 1002 1058 Compute Equitable Threat Score (ETS) * * 1003 1057 Compute Critical Success Index (=TS) * * 1004 1056 Compute False Alarm Ratio (FAR) * * 1005 1063 Compute Postagreement (PA) * * 1006 1051 Compute Prefigurance (=POD) * * 1007 1051 Compute Probability of Detection (POD) * * 1008 1057 Compute Threat Score (TS) * * * * 1009 Cumulative Forecast Prcp Vol % vs Thrshld * * 1010 Cumulative Correctly Frcst Prcp Vol % vs Thrshld* * 1011 Cumulative Observed Prcp Vol % vs Thrshld * * 1012 Forecast Prcp Vol % of Ttl vs Thrshld * * 1013 Correctly Forecast Prcp Vol % of Ttl vs Thrshld * * 1014 Observed Prcp Vol % of Ttl vs Thrshld * * 1015 Forecast Prcp # % of Ttl # vs Thrshld * * 1016 Correctly Forecast Prcp # % of Ttl # vs Thrshld * * 1017 Observed Prcp # % of Ttl # vs Thrshld * * 1018 Forecast # % of Ttl <= Thrshld vs Thrshld * * 1019 Correctly Frcst # % of Ttl <= Thrshld vs Thrshld* * 1020 Observed # % of Ttl <= Thrshld # vs Thrshld * * * * 1021 1055 False Alarm Rate (f-h)/(N-o) (POFD) * * 1022 1062 Conditional Miss Rate (o-h)/(N-f) (CMR) * * 1023 1059 Heidke Skill Score (HSS) * * 1024 1053 Peirce-Hanssen-Kuipers True Skill Score (TSS) * * 1025 Compute ODDS RATIO (OR) * * 1026 Compute log of ODDS RATIO * * 1027 Compute standard error of log of ODDS RATIO * * 1028 1061 ODDS RATIO Skill Score (ORSS) * * 1029 Compute ODDS RATIO Skill Score Cubed * * 1030 Compute log (ODDS RATIO) / STD ERROR of same * * 1031 Area between ODDS RATIO ROC & chance ROC curves * * 1032 ROC curve parametric in OR for time series * * 1033 Odds ratio Parameterized ROC area (OPRA) score * * 1034 1054 Frcst accuracy rate = proportion correct (PC) * * 1035 1052 Extreme Dependency Score (EDS) * * 1036 1060 Clayton Skill Score (CSS) * * 1037 1064 Explicit dH/dF Threat Score (TSDHDF) * * 1038 1065 Explicit dH/dF Equitable Threat Score (ETSDHDF) * * 1039 1066 Explicit dH/dA Threat Score (TSDHDA) * * 1040 1067 Explicit dH/dA Equitable Threat Score (ETSDHDA) * * 1041 Symmetric Extreme Dependency Score (SEDS) * * 1042 Extremal Dependence Index (EDI) * * 1043 Symmetric Extremal Dependence Index (SEDI) * * 1068 CPR reference midway between CPR benchmarks * * ^ ^ ^ * * PM Code CPR Code Performance Measure (PM) * * * * The critical performance ratio (CPR) is described in Brill (2009) * * published in W & F. * * * * Any of the preceding codes may be computed from bias adjusted FHO * * values. To do this, precede the 4-digit code number with one of * * the following bias adjustment specification prefixes: * * * * 101 Fedor Mesinger's dH/dF bias adjustment * * 102 Conservation of Odds Ratio bias adjustment * * 103 Fedor Mesinger's dH/dA bias adjustment * * 104 Bias removed by frequency matching (requires * * extended FHO record -- FHOH'[F'], where H' is * * bias removed hit fraction, and F' is bias * * removed forecast fraction, which is replaced by * * O=observed fraction if absent) * * * * Example 1: -1020003 displays the bias adjusted hit count * * Example 2: 1021008 displays the bias adjusted threat score * * * * Any of the preceding 10xx series that are N dependent may be * * computed using F+O-H from the Mth preceding threshold. The first M * * thresholds use the original value of N, the total number. Thres- * * hold must be the independent variable. To do this, precede the * * 10xx code with the following prefix: * * * * 20M The F+O-H value from i-M threshold is used, M>0 and M<10 * ************************************************************************* Select computational code from table above. Please enter one computational code(s). Enter the intra-trace data combination lag interval (The statistic value or score at a point is computed after combining data from the preceding (LAG - 1) points with data at the point. If lag < 0, the combination restarts at every lag+1 trace value.) Enter LAG (RETURN or 0 = no lag interval; -9999 = set for each trace): Enter # of trace points for running average smoother window (RETURN or 0 = no smoothing). ************************************************************************* * The information in the control file is key word tagged. An equal * * sign (=) separates the keyword from control information following * * it. Here are the keywords and their meanings: * * * * ADLINE = Line c/t/w for attr diag ref lines in clockwise order * * BORDER = Background lines color/dash/width * * BXWSKR = Box-whisker objects line color/width/alpha/whskr_flag * * B_TEXT = color/[#]x;y/size;font;width;HWSW/border_flag defining * * box-whisker annotation text box * * CLEAR = Clear screen flag (yes or no) * * COLORS = Color number assignments * * DEVICE = Device|name|x size;y size|color type * * FACTOR = Scaling factor(s) to apply to data (lft_sf/rht_sf) * * HSTCLR = Histogram bar colors for each trace separated by ; * * L_TEXT = Text size/font/width/hw flag for axis tic-mark labels * * LABLEV = label level * * 0 = No labels * * 1 = Mix of user-defined and default labels * * 2 = ONLY user defined title and trace key labels * * LINE = Line colors/types/widths/marker_frequency * * MARKER = Marker colors/types/sizes/widths * * NSMPLS = Number of Monte Carlo trials to use in resampling * * PANEL = Panel location (0=full screen) * * PTYPE = l.s. ; r.s. Y axis plot types/h:w ratio/lm;bm;rm;tm * * REFLIN = Reference line: x1,y1;x2,y2;x3,y3/color/type/width * * REVERS = Yes/no flag to switch X and Y axses * * RSTRCS = Right-side Y axis display trace #s separated by / * * RWTLIN = r.s. Line colors/types/widths/values for witness lines * * RYAXIS = Right side Ystrt/ystop/yinc/lbl;gln;tck * * SERCOR = YES/NO Serial correlation flag for significance test 1 * * T_TEXT = Text size/font/width/hw flag for title|trace|axis text * * TITLE = Title string * * TRACEn = User provided text label key for trace number n * * WITLIN = l.s. Line colors/types/widths/values for witness lines * * If VALUES are missing, 0 is assumed. * * XAXIS = Xstrt/xstop/xinc/lbl;gln;tck * * t_start:t_stop:increment/lbfq;gdfq;tkfq * * For a time axis X is in days, starting at 0. * * XLABEL = Label text for X axis/# of char size units from margin * * XUDLBL = list of labels for X axis tic marks separated by / * * YAXIS = Ystrt/ystop/yinc/lbl;gln;tck * * YLABEL = Label text for Y axis/# of char size units from margin * * YRLBEL = Label text for r.s. Y axis/# of char units from margin * ************************************************************************* The table above describes the graphics control file. Below are the default entries in the graphics control file: DEVICE = xw|VSDB CLEAR = yes Set a graphics control file entry: Keywords may be abbreviated to first 3 letters. Last entry overides previous ones for a keyword. Reasonable defaults are provided for unset keywords. SYNTAX: keyword=input OR enter one of the following commands r = run plot program with current keyword settings h = display keyword help list l = list current settings e = exit this program a followed by the name of a file containing keyword settings to append to the current file f followed by the name of a file containing all keyword settings h followed by a keyword gives help for that keyword h colors gives help information on colors s followed by a file name in to which to save the current keyword settings Set a graphics control file entry: Keywords may be abbreviated to first 3 letters. Last entry overides previous ones for a keyword. Reasonable defaults are provided for unset keywords. SYNTAX: keyword=input OR enter one of the following commands r = run plot program with current keyword settings h = display keyword help list l = list current settings e = exit this program a followed by the name of a file containing keyword settings to append to the current file f followed by the name of a file containing all keyword settings h followed by a keyword gives help for that keyword h colors gives help information on colors s followed by a file name in to which to save the current keyword settings Set a graphics control file entry: Keywords may be abbreviated to first 3 letters. Last entry overides previous ones for a keyword. Reasonable defaults are provided for unset keywords. SYNTAX: keyword=input OR enter one of the following commands r = run plot program with current keyword settings h = display keyword help list l = list current settings e = exit this program a followed by the name of a file containing keyword settings to append to the current file f followed by the name of a file containing all keyword settings h followed by a keyword gives help for that keyword h colors gives help information on colors s followed by a file name in to which to save the current keyword settings All entries are shown. Only the last for each keyword is used. REVERS=NO line=4;2;4;2;3 2; 3; 6;/1;1;2;2;1 5; 6; 10;/9/ marker=0 l_text=1.5/22/1.5/hw t_text=1.7/22/4.0/sw|1.4/22/1.8/sw|1.6/22/2.0/sw|1.4/22/1.6/sw yaxis= 0.7/1.02/.1/1;1 clear=yes trace1=OPERATIONAL trace2=FV3CMAQV502_9 trace3=OPERATIONAL-Bias-Corrected trace4=FV3CMAQV502BC trace5= trace6= trace7= trace8= trace9= trace10= DEVICE=psc WITLIN=1/1/8/0 hstclr=0 panel=0 xudlbl= xaxis= xlabel= 01h-Avg PM25MX Threshold 12 UTC Cycle ylabel=Hit-Rate title=DAY 2 01h-Avg PM25MX Hit-Rate avged by Threshold \ 20180801 to 20180831 \ West-US Set a graphics control file entry: Keywords may be abbreviated to first 3 letters. Last entry overides previous ones for a keyword. Reasonable defaults are provided for unset keywords. SYNTAX: keyword=input OR enter one of the following commands r = run plot program with current keyword settings h = display keyword help list l = list current settings e = exit this program a followed by the name of a file containing keyword settings to append to the current file f followed by the name of a file containing all keyword settings h followed by a keyword gives help for that keyword h colors gives help information on colors s followed by a file name in to which to save the current keyword settings Graph rel trc # = 1 Plot rel trc # = 1 Orig trc # = 1 Graph rel trc # = 2 Plot rel trc # = 2 Orig trc # = 2 PLT_DRIV +03 = Warning: counts for bins or thresholds differ Graph rel trc # = 3 Plot rel trc # = 3 Orig trc # = 3 TRACE #03 error from RDTD_TRC: +02 = ALL TRACE POINTS ARE MISSING PLT_DRIV +03 = Warning: counts for bins or thresholds differ Graph rel trc # = 4 Plot rel trc # = 4 Orig trc # = 4 PLT_DRIV +03 = Warning: counts for bins or thresholds differ Warning: GEMPLT subprocesses are still running. To save these graphics keyword settings, enter s and a space followed by a file name; OR To save the plot as a color postscript file, enter cp and a space followed by a file name; OR To save the plot as a black and white postscript file, enter bw and a space followed by a file name; OTHERWISE, press enter. Graph rel trc # = 1 Plot rel trc # = 1 Orig trc # = 1 Graph rel trc # = 2 Plot rel trc # = 2 Orig trc # = 2 PLT_DRIV +03 = Warning: counts for bins or thresholds differ Graph rel trc # = 3 Plot rel trc # = 3 Orig trc # = 3 TRACE #03 error from RDTD_TRC: +02 = ALL TRACE POINTS ARE MISSING PLT_DRIV +03 = Warning: counts for bins or thresholds differ Graph rel trc # = 4 Plot rel trc # = 4 Orig trc # = 4 PLT_DRIV +03 = Warning: counts for bins or thresholds differ Warning: GEMPLT subprocesses are still running. To resume with a different trace data file, enter tdf and a space followed by a file name; OTHERWISE, press enter. Invalid entry --- continuing. Here are the contents of the trace data file: Number of plots available = 1 PLOT # 1 is a skill score plot composed of 4 trace(s). INDEPENDENT VARIABLE FOR THIS PLOT: THRSH STAT = FHO GENERAL LABELS # FLD NAM LABEL 1 FHOUR 48 2 LEVEL SFC 3 V_ANL ANOWPM 4 V_RGN G246 5 STATS FHO 6 PARAM PM25MX 7 VYMDH 201808010000-201808312359 STAT = PARM = PM25MX TRACE # 1 LABELS # FLD NAM LABEL 1 MODEL CMAQ5X/148 STAT = PARM = PM25MX TRACE # 2 LABELS # FLD NAM LABEL 1 MODEL FV3CMAQV502_9/148 STAT = PARM = PM25MX TRACE # 3 LABELS # FLD NAM LABEL 1 MODEL CMAQ5XBC/148 STAT = PARM = PM25MX TRACE # 4 LABELS # FLD NAM LABEL 1 MODEL FV3CMAQV502BC/148 Select number of the plot to display (0=exit):