Canadian Brewer Spectrometer Network

User Tools

Site Tools

Trace: • cbsnet
cbsnet

Table of Contents

Everything to know about CBSNet

CBSNet is the database computer for the Brewer log form, Brewer Data, Straylight processing and etc.

crontab

The list below shows the different sections used in the crontab a crontab jobs are setup so that they runs a command at is specific time. The format for the crontab job are as follows. 

m h  dom mon dow   command
m - minutes
h- hours
dom - day of month
mon - month
dow - day of week

a * can be used as a wild card.

Below is a list of crontab jobs setup on cbsnet

Brewer Support

Brewer Support

USER
########Brewer Support######## 00,30 0-3,5-23 * * * /home/cbsnet/schedule_task/rsync.sh 00 4 * * * rsync -vhrltp –exclude-from /home/cbsnet/schedule_task/exclude-list-all.txt -e “ssh -i ”/home/cbsnet/schedule_task/id_depot“” remsens_ext@depot.cmc.ec.gc.ca:/incoming/brewer/ /home/cbsnet/brews/brewnet/Brewers 05,35 * * * * /home/cbsnet/schedule_task/rsync-all.sh 29 10 * * * python3 /home/cbsnet/schedule_task/stray_correction.py -a 0 16 * * * python3 /home/cbsnet/schedule_task/year_mover.py 03,33 * * * * bash /home/cbsnet/schedule_task/rsync_ios.sh 191 187 145 109 017 > ~/temp.log 0 4 * * * /home/cbsnet/schedule_task/sync_program.sh

  • rsync.sh is a script pulling data near real time data (Look at ~/schedule_task/exclude-list.txt to see what is excluded) from remsens_ext@depot.cmc.ec.gc.ca
  • rsync-all.sh is a script for pulling all data from remsens_ext@depot.cmc.ec.gc.ca
  • stray_correction.py is a script that processes straylight correction on the b-file. Files are copied into a stray_light corrected folder in each brewer data folder. The constant used for the straylight correction is pulled from the icf from each of the bfile
  • year_mover.py is a script that moves old bfile to its corresponding year directory for each brewer
  • rsync_ios.sh pushes triad and IOS instrument data to IOS' server
  • sync_program.sh pushes the master and test program directory to remsens_ext@depot.cmc.ec.gc.ca for program auto updates

Brewer Log Forms

Brewer Log Forms

USER
########Brewer Log forms######## 0,30 * * * * /home/cbsnet/brews/brewnet/imit/brewer-log-form/scripts/server/push-cache.sh 0 23 * * * /home/cbsnet/brews/brewnet/imit/brewer-log-form/scripts/server/push-update.sh 0 23 * * * rsync -rthv /home/cbsnet/brews/brewnet/imit/brewer-log-form-deploy remsens_ext@depot.cmc.ec.gc.ca:/incoming/brewer_log/update/ -e “ssh -i ”/home/cbsnet/schedule_task/id_depot“” 1 * * * * /home/cbsnet/brews/brewnet/imit/brewer_log_form/import_log_form.sh

Brewer Support Addon

Brewer Support Addon

USER
########Brewer Support Add-on######## #59,15,29,45 * * * * /home/cbsnet/eubrew_downloader.py rsit Spectrometer2016 183 157 185 03,33 * * * * bash /home/cbsnet/schedule_task/rsync_ios.sh 191 187 145 > ~/temp.log 15 10 * * * fossil sync -R ~/brews/cbsnet/Brewer_Programs/IOS_Program/brewer.fossil 0 8 * * 1-5 /home/cbsnet/schedule_task/station_ip_update.py 30 4 * * 2 /home/cbsnet/schedule_task/db_backup.sh

IDEAs Database

IDEAs Database

USER
########Ideas Database######## 10,40 0-3,5-23 * * * /var/www/html/dokuwiki/ideas/loadbfiles.sh 15 4 * * * /var/www/html/dokuwiki/ideas/loadbfiles_subs.sh 30 4 * * * curl http://localhost/dokuwiki/ideas/clean_data.php 40 7 * * * curl http://localhost/dokuwiki/ideas/load-bfiles.php?cscan=1 50 0 * * * curl http://localhost/dokuwiki/ideas/update_master_soft_db.php # 5,35 * * * * curl http://localhost/dokuwiki/ideas/load_pid.php #13 3 15 * * curl http://localhost/dokuwiki/ideas/load_uvsys.php 0 9 * * * curl http://localhost/dokuwiki/ideas/get_fx_rates.php?db=brewpart 33 5 * * * sh /home/cbsnet/schedule_task/wiki_sync.sh 10 * * * * mysql < ~/schedule_task/mysql_sloavg

Brewer Log Form

The Brewer log form's back end deals with hosting the admin section adding/removing/editing operators, Brewers and stations.
These can be access via the following links,
Manage Operators
Manage Stations
Manage Instruments

The log form backend is running off cbsnet via pm2.

To check the status

To check the status

USER
cbsnet$ pm2 ls ┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐ │ id │ name │ mode │ ↺ │ status │ cpu │ memory │ ├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤ │ 1 │ brewer-admin │ fork │ 1 │ online │ 0% │ 66.2mb │ │ 0 │ brewer-back │ fork │ 1 │ online │ 0% │ 56.5mb │ └────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘

How to reset pm2 the service

How to reset pm2 the service

USER
cbsnet$ pm2 restart 1 Use –update-env to update environment variables [PM2] Applying action restartProcessId on app [1](ids: [ '1' ]) [PM2] [brewer-admin](1) ✓ ┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐ │ id │ name │ mode │ ↺ │ status │ cpu │ memory │ ├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤ │ 1 │ brewer-admin │ fork │ 2 │ online │ 0% │ 16.3mb │ │ 0 │ brewer-back │ fork │ 1 │ online │ 0% │ 53.3mb │ └────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘
Where '1' is the id number

Error in log forms

Archived and error logs can be found in /var/cache/brewer+Log/logs.
Error logs are identified by the prefix of the file name ERROR_*.
Some common log issues

  1. Too many ,. This is usually found when right before the comment line or comments including commas
  2. The use of a ;. semicolon are not supported
  3. Comments too long. The max length for the comment is 500 characters.

Metabase

Metabase is an open source program that is able to read from a database and plot/display it in a friendly interface. The CBSN version can be found here.

To check the status of metabase

To check the status of metabase

USER
cbsnet@Thinkbrew:~$ su root Password: root@Thinkbrew:/home/cbsnet# systemctl status metabase ● metabase.service - Metabase server 

   Loaded: loaded (/etc/systemd/system/metabase.service; disabled; vendor pre>
   Active: active (running) since Sun 2025-12-07 20:22:44 GMT; 3 days ago
 Main PID: 32791 (java)
    Tasks: 160 (limit: 37964)
   Memory: 2.8G
      CPU: 45min 41.735s
   CGroup: /system.slice/metabase.service
           └─32791 /usr/bin/java -jar /home/cbsnet/metabase/metabase.jar

This indicates metabase is running properly.

How to restart metabase

How to restart metabase

USER
cbsnet@Thinkbrew:~$ su root Password: root@Thinkbrew:/home/cbsnet# systemctl restart metabase


Backups

  1. Drive Backups are managed by timeshift.
    1. Monthly Backup
      1. Backup are made on the 1st of each month at 00:00:00 UTC
      2. up to 3 copies are saved at one time
    2. Weekly Backup
      1. Backup are made on each Sunday at 21:00:00 UTC
      2. up to 2 copies are saved at one time
  2. Database backups are managed by the db_backup.sh
    1. Database base backup are done every Tuesday and at 04:00:00 UTC
    2. Up to 9 copies are saved at one time
    3. Database that are being backed up are
      1. brewerdata
      2. brewer_db
      3. brewer_security
      4. brewerpart
      5. metabase


Database

There are currently 5 database (as of Dec 17, 2025) running on cbsnet via mariadb (Version 10.5.29-MariaDB-0+deb11u1 Debian 11)

  1. brewerdata - this database is primary used to host the IDEAS database and includes all the brewer data. The best way to view this data is via metabase.
  2. brewer_db - this database is used to for hosting the brewer log form data.
  3. brewer_security - this database is used to host the user information needed for the brewer log form
  4. brewerpart - This database is used to host the brewer parts database. This is what brewparts.exe uses
  5. metabase - This database is used to host the config and different views used for metabase.
cbsnet.txt · Last modified: 2025/12/17 14:13 by reno