====== ECCC Brewer Spectrometer Network ======

There are three stages of the data transfer for the Brewer Spectrometer. \\
{{:brewer_network_data_flowchart20200312.png?400|}}

===== Station =====

At the station each computer computer typically runs one to two (1-2) Brewer Spectrometers.
Brewer files are saved in the following location
  * Windows¹: C:\Br#xxx
  * Linux: /brews/Br#xxx

==== Data transfer schedule ====
At 25 minutes and 55 minutes past each hour.
  * All data files up to 75 days old are pushed to exp-studies².
  * Update AVG files at exp-studies².

At 04:00 UTC.
  * Station directory is synchronized with the program master at exp-studies². This will create an exact copy and delete any items that do not exist 
  * All data is pushed to exp-studies².
At 16:00 UTC brewnet_year_mover³ is ran


===== Experimental Studies =====
Experimental Studies (exp-studies) is a server used by the Remote Sensing group for data transferring, data processing and data storage. The Exp-studies server is used as a conduit for data transfer between the station computers, through the firewall, onto our brewnet computer within the EcoNet. 

Brewer data files are kept at ~/brewnet/Br#xxx

==== Data transfer schedule ====
  * All data transfer from and to exp-studies is done on the station computers or brewnet
  * At 16:00 UTC brewnet_year_mover³ is ran
  * Bulletin updates every 30 minutes on the hour and 30 minute pass the hour.

==== Bulletin====
Exp-studies also hosts and processes the data for the [[hhttps://exp-studies.tor.ec.gc.ca/~ogyua/bulletins.php|bulletin]]. The bulletin gets updated on the hour (xx:00) and half hour (xx:30). The bulletin is a versatile tool with different uses.
  * Indicates data file age (data latency may be an indication of an issue with the brewer/computer)
  * Provides plots of different instruments at the same site (an indication of data agreement or outliers)


===== Brewnet =====
Brewnet is the Brewer Spectrometer server that houses the archived data and other associated documents for all CBSN Brewer Spectrometers.

Brewer files are located: /home/brewer/brews/brewnet/Brewers/Br#xxx

==== Data transfer schedule ====
On the hour and 30 minutes of each hour.
  * All data files up to 75 days old pulled from exp-studies.
  * Update AVG files from exp-studies.

At 05:00 UTC.
  * The program master directory on brewnet is copied with an exact copy to exp-studies. 
  * All data is pulled from exp-studies².

At 16:00 UTC brewnet_year_mover³ is ran


===== Notes: =====

¹ Window computers are to be discontinued and to be replaced Linux computers \\
² exp-studies: Experiential studies is the server used to process and host data between station and brewnet.\\
³ brewnet_year_mover is a python program that moves any data files older than 75 days into a archived year directory(I.E. 2019, 2020...)