Table of Contents
ECCC Brewer Spectrometer Network
There are three stages of the data transfer for the Brewer Spectrometer.
Station
At the station each computer computer typically runs one to two (1-2) Brewer Spectrometers. Brewer files are saved in the following location
- Windows¹: C:\Br#xxx
- Linux: /brews/Br#xxx
Data transfer schedule
At 25 minutes and 55 minutes past each hour.
- All data files up to 75 days old are pushed to exp-studies².
- Update AVG files at exp-studies².
At 04:00 UTC.
- Station directory is synchronized with the program master at exp-studies². This will create an exact copy and delete any items that do not exist
- All data is pushed to exp-studies².
At 16:00 UTC brewnet_year_mover³ is ran
Experimental Studies
Experimental Studies (exp-studies) is a server used by the Remote Sensing group for data transferring, data processing and data storage. The Exp-studies server is used as a conduit for data transfer between the station computers, through the firewall, onto our brewnet computer within the EcoNet.
Brewer data files are kept at ~/brewnet/Br#xxx
Data transfer schedule
- All data transfer from and to exp-studies is done on the station computers or brewnet
- At 16:00 UTC brewnet_year_mover³ is ran
- Bulletin updates every 30 minutes on the hour and 30 minute pass the hour.
Bulletin
Exp-studies also hosts and processes the data for the bulletin. The bulletin gets updated on the hour (xx:00) and half hour (xx:30). The bulletin is a versatile tool with different uses.
- Indicates data file age (data latency may be an indication of an issue with the brewer/computer)
- Provides plots of different instruments at the same site (an indication of data agreement or outliers)
Brewnet
Brewnet is the Brewer Spectrometer server that houses the archived data and other associated documents for all CBSN Brewer Spectrometers.
Brewer files are located: /home/brewer/brews/brewnet/Brewers/Br#xxx
Data transfer schedule
On the hour and 30 minutes of each hour.
- All data files up to 75 days old pulled from exp-studies.
- Update AVG files from exp-studies.
At 05:00 UTC.
- The program master directory on brewnet is copied with an exact copy to exp-studies.
- All data is pulled from exp-studies².
At 16:00 UTC brewnet_year_mover³ is ran
Notes:
¹ Window computers are to be discontinued and to be replaced Linux computers
² exp-studies: Experiential studies is the server used to process and host data between station and brewnet.
³ brewnet_year_mover is a python program that moves any data files older than 75 days into a archived year directory(I.E. 2019, 2020…)