Differences

This shows you the differences between two versions of the page.

--- zsf [2024/01/11 14:17] – andrew
+++ zsf [2026/02/10 16:36] (current) – andrew
@@ Line 9: / Line 9: @@
 ==== Data Processing ====
-To begin making the new zenith sky coefficients you must first create a test file directory and then a unique file in that directory for each brewer being analyzed. You can host the test file wherever is most convenient for you, ie /Desktop or /Documents. Populate the new brewer file with the following;
+New zenith sky coefficients should be done with data after a transfer or absolute calibration has been done with up to 1 year of data but can be performed with a few months of processed, cleaned DS O3 with ZS data. It is important that the files must contain both the cleaned DS and ZS data for creating new constants as the calibration needs to fit the ZS data to the DS.
-  * ds2zs.exe
-  * ds2zs.cfg
-  * ICF file for the Brewer being analyzed
-  * Relevant B-files for the brewer being analyzed
-Open the ds2zs.cfg file in the test directory you created with notepad or another text editor. Edit the file with the following parameters:
-  * files: change the b file naming to reflect the brewer being analyzed as displayed above
-  * icf: enter the icf file to use in re-processing
-  * zsf: refers to one of the following 4 models based on location. Set to &II
-  * dtime: maximum time difference between ZS and DS (default 10)
-  * min mu: minimum air mass factor for DS/ZS pairs (default 1.2)
-  * max mu: maximum air mass factor for DS/ZS pairs (default 4.2)
-  * max std: max DS/ZS ozone standard deviation within the measurement (default 2.5)
-  * 03 height above station (km): ozone layer height above the station (default 22km)
-  * Rayleigh height (km): effective air height for Rayleigh calculations (default 5km)
-  * Difference between 03ETC and SLR6: the difference between O3ETC and SLR6 at calibration. This number can be found in the brewer’s icf file near the bottom of the text file. Refer to figure 3.
-Save the newly updated cfg file to the test directory and rename it ds2zsxxx.cfg.\\
+You will have to merge the .dat files using the merge function under ‘tools’ drop-down menu in the matlab software. Select the files you wish to combine and press okay. Upload the file to be used for the calibration in the 'clean o3 data file' section of the 'Sky Calibration tab'.
-\\
-Highlight and drag the newly created cfg file and drag it into the ds2zs.exe file.\\
-\\
-The ds2zs.exe program will execute its routine and create two files in the test directory.
-  * ds2zs.dat: file contains the ozone data that was used in the processing and reprocessed ZS data with the new calculated coefficients.
-  * ds2zs.zsf: file contains the newly calculated coefficients and statistics for the ZS processing. This file is used by the operation Brewer software once it has been renamed.
-If the program does not run or create the two files above run the program in a command line. To do this type ‘cmd’ into the navigation bar in the ‘Test’ directory you created or navigate with a new command terminal. Once in the correct directory write ‘ds2zsxxx.cfg ds2zs.exe’ and the program will run in your command terminal. From here you can see why the program failed. In cases where it does not provide an explanation it is possible that the program did not create the two new files because there is not enough data for the coefficients.\\
+{{ :zsf-sky_calibration_window_1.png?600 |}}
-Rename the ds2zs.zsf file to zsfJJJyy.xxx. Once this is done repeat for every brewer you wish to analyze in the respected folder.
-==== Testing ====
-Once a new zsf file is created it is important to make sure that the newly created coefficients and statistics for the ZS processing improves the results for ZS measurements. To do this the old zsf file and new zsf file must be compared in BFilePro or another data processing software. For this example we will be using BFilePro and Brewer #039.\\
-Create a new directory in the file that houses BFilePro and add the following files:
+Select which instrument you wish to calibrate and then select the appropriate region that the instrument operates in to use the correct albedo for creating the new coefficients.
-  * dcfJJJYY.xxx
-  * dcfJJJYY.xxx.dwl
-  * etcYYYY.xxx
-  * ICFJJJYY.xxx
-  * UVRJJJYY.xxx
-  * zsfJJJYY.xxx
-Make sure that your data processing software is set up. In BFilePro make sure that the following options are selected in the 'Options' menu under the 'Calculations' window.
+{{ :zsf-albeto.png?600 |}}
-  * Recalculate b-files
-  * Calculate aerosol
-  * Apply SL correction
-In the 'Instrument constants' window select ‘Add a new calibration’ and a new window will pop up. Set the calibration start date to the same day as the date in the ICF file. Eg ICF09623.xxx you would set the date to 096 Julian day (April 6th) 2023. Be aware that if it’s a leap year it will affect the Julian day/calendar date. \\
-Link the correct files to the ‘Calibration Details’ window, these will be the same files that you transferred over during the beginning of the ‘Testing’ process. All the files should be assigned except the ‘Neutral filters’ which can be left blank.
-Once the files to use are selected, select the ‘OK’ button and it will return you to the previous window. There are two boxes highlighted below that need to be calculated. To do this you must go into the instruments ICF file and navigate to the bottom of the text file and take note of two numbers at line 61 and 62.\\
-\\
-To calculate to value for the first box to the right of ‘SL’ take the number above the box being calculated, for this example it is 3180, and subtract it by the number found on line 61 of the ICF file, 1238 for this example, and fill in the box with the difference. For the second box to the right of ‘SL’ take the number above the box being calculated and subtract it by the number found on line 62 of the ICF file. Check the box ‘Save these options’ and press ‘OK’.
-Direct BFilePro to the directory that stores all the B-files for the instrument you are analyzing. \\
-\\
-{{:bfilepro_instrument_constants.jpg?400|}}{{:txt_file_instrument_constants.jpg?300|}} \\
-This will populate the window on the left-hand side of BFilePro with the available files for that instrument. Highlight b-files that will provide both ZS and DS measurements for your instrument and click ‘Process’ at the bottom of the panel. Take note of what date range you are analyzing, if there isn’t Ds and Zs data during this time, clear the graph and select a new time. \\
-\\
-Select ‘New’ in plot results and toggle ‘Zs’ on and off to find a good couple days to compare data. Use a marquee select dragging left to right to zoom in on desired period and once this is found note the difference between the ‘Ds’ and ‘Zs’ measurements, how well does the data agree? \\
-\\
-After taking note of the relationship, clear the graph by selecting the ‘Clear Graph’ button. You are now going to see how the Zs data changes in relation to the Ds data with the new zsf file you created in the ‘Data Processing’ section of this document. \\
-\\
-Open the ‘Options’ menu from the top left-hand side of BFilePro. In the ‘Instrument constants’ section select ‘Edit this Calibration’ and change the pathway to the ZS file to the new zsf file you created in the previous step.  Click ‘Ok’ to close the edit calibration window and ‘Ok’ and save changes in the ‘Options’ menu.\\
-\\
-Re process the graph and look how the new zsf coefficients affect the relationship between the Ds and Zs data. Do the new zsf coefficients improve how well the Zs measurements relate to the Ds? If so that means the new zsf coefficients are an improvement and should be used going forward.
+You then select the calculation parameters. Most of the preset options are standard and what you will use to create the new ZSF file, you will need to select the correct date range. Use a date that starts close to the most recent calibration to insure the most accurate DS data is used to create the new coefficients.
+{{ :zsf-options.png?600 |}}
+Select 'Run ZSF fitting' to begin the program. After you've selected to run the program you will jump to the 'Sky Calibration Plot' page of the software. Once the process is complete the graphs will populate with data to show how the new ZSF calibration will change the fitting of the ZS data.
+{{ :zsf-sky_calibration_plots.png?600 |}}
+If the new ZSF improve the agreement between ZS and DS data by ~5% apply the new calibration to the instrument.