Canadian Brewer Spectrometer Network

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Everything to know about CBSNet ======

CBSNet is the database computer for the Brewer log form, Brewer Data, Straylight processing and etc.

===== crontab =====
The list below shows the different sections used in the crontab
a crontab jobs are setup so that they runs a command at is specific time. The format for the crontab job are as follows. <code>m h  dom mon dow   command
m - minutes
h- hours
dom - day of month
mon - month
dow - day of week
</code>
a * can be used as a wild card.

Below is a list of crontab jobs setup on cbsnet
<hidden Brewer Support>
<konsole>
########Brewer Support########
00,30 0-3,5-23 * * * /home/cbsnet/schedule_task/rsync.sh
00 4 * * * rsync -vhrltp --exclude-from /home/cbsnet/schedule_task/exclude-list-all.txt -e "ssh -i "/home/cbsnet/schedule_task/id_depot""  remsens_ext@depot.cmc.ec.gc.ca:/incoming/brewer/ /home/cbsnet/brews/brewnet/Brewers
05,35 * * * * /home/cbsnet/schedule_task/rsync-all.sh
29 10 * * * python3 /home/cbsnet/schedule_task/stray_correction.py -a
0 16 * * *  python3 /home/cbsnet/schedule_task/year_mover.py
03,33 * * * * bash /home/cbsnet/schedule_task/rsync_ios.sh 191 187 145 109 017  > ~/temp.log
0 4 * * * /home/cbsnet/schedule_task/sync_program.sh
</konsole>
   * __rsync.sh__ is a script pulling data near real time data (Look at ~/schedule_task/exclude-list.txt to see what is excluded) from remsens_ext@depot.cmc.ec.gc.ca
   * __rsync-all.sh__ is a script for pulling all data from remsens_ext@depot.cmc.ec.gc.ca
   * __stray_correction.py__ is a script that processes straylight correction on the b-file. Files are copied into a stray_light corrected folder in each brewer data folder. The constant used for the straylight correction is pulled from the icf from each of the bfile
   * __year_mover.py__ is a script that moves old bfile to its corresponding year directory for each brewer
   * __rsync_ios.sh__ pushes triad and IOS instrument data to IOS' server
   * __sync_program.sh__ pushes the master and test program directory to remsens_ext@depot.cmc.ec.gc.ca for program auto updates

</hidden>
<hidden Brewer Log Forms>
<konsole>
########Brewer Log forms########
 0,30 * * * * /home/cbsnet/brews/brewnet/imit/brewer-log-form/scripts/server/push-cache.sh
 0 23 * * * /home/cbsnet/brews/brewnet/imit/brewer-log-form/scripts/server/push-update.sh
 0 23 * * * rsync -rthv /home/cbsnet/brews/brewnet/imit/brewer-log-form-deploy remsens_ext@depot.cmc.ec.gc.ca:/incoming/brewer_log/update/ -e "ssh -i "/home/cbsnet/schedule_task/id_depot""
1 * * * * /home/cbsnet/brews/brewnet/imit/brewer_log_form/import_log_form.sh
</konsole>
</hidden>
<hidden Brewer Support Addon>
<konsole>
########Brewer Support Add-on########
#59,15,29,45 * * * * /home/cbsnet/eubrew_downloader.py rsit Spectrometer2016 183 157 185
03,33 * * * * bash /home/cbsnet/schedule_task/rsync_ios.sh 191 187 145  > ~/temp.log
15 10 * * * fossil sync -R ~/brews/cbsnet/Brewer_Programs/IOS_Program/brewer.fossil
0 8 * * 1-5 /home/cbsnet/schedule_task/station_ip_update.py
30 4 * * 2 /home/cbsnet/schedule_task/db_backup.sh
</konsole>
</hidden>
<hidden IDEAs Database>
<konsole>
########Ideas Database########
 10,40 0-3,5-23 * * * /var/www/html/dokuwiki/ideas/loadbfiles.sh
 15 4 * * * /var/www/html/dokuwiki/ideas/loadbfiles_subs.sh
 30 4 * * * curl http://localhost/dokuwiki/ideas/clean_data.php
 40 7 * * * curl http://localhost/dokuwiki/ideas/load-bfiles.php?cscan=1
 50 0 * * * curl http://localhost/dokuwiki/ideas/update_master_soft_db.php
# 5,35 * * * * curl http://localhost/dokuwiki/ideas/load_pid.php
#13 3 15 * * curl http://localhost/dokuwiki/ideas/load_uvsys.php
0 9 * * * curl http://localhost/dokuwiki/ideas/get_fx_rates.php?db=brewpart
 33 5 * * * sh /home/cbsnet/schedule_task/wiki_sync.sh
10 * * * * mysql < ~/schedule_task/mysql_sloavg
</konsole>
</hidden>

===== Brewer Log Form =====

The Brewer log form's back end deals with hosting the admin section adding/removing/editing operators, Brewers and stations.\\
These can be access via the following links,\\
[[http://cbsn.hiitravel.ca:8671/embed/operators|Manage Operators]]\\
[[http://cbsn.hiitravel.ca:8671/embed/stations|Manage Stations]]\\
[[http://cbsn.hiitravel.ca:8671/embed/instruments|Manage Instruments]]\\

The log form backend is running off cbsnet via pm2.\\
<hidden To check the status>
<konsole>
cbsnet$ pm2 ls
┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐
│ id │ name               │ mode     │ ↺    │ status    │ cpu      │ memory   │
├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤
│ 1  │ brewer-admin       │ fork     │ 1    │ online    │ 0%       │ 66.2mb   │
│ 0  │ brewer-back        │ fork     │ 1    │ online    │ 0%       │ 56.5mb   │
└────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘

</konsole>
</hidden>
<hidden How to reset pm2 the service>
<konsole>
cbsnet$ pm2 restart 1
Use --update-env to update environment variables
[PM2] Applying action restartProcessId on app [1](ids: [ '1' ])
[PM2] [brewer-admin](1) ✓
┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐
│ id │ name               │ mode     │ ↺    │ status    │ cpu      │ memory   │
├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤
│ 1  │ brewer-admin       │ fork     │ 2    │ online    │ 0%       │ 16.3mb   │
│ 0  │ brewer-back        │ fork     │ 1    │ online    │ 0%       │ 53.3mb   │
└────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘
</konsole>
Where '1' is the id number
</hidden>

==== Error in log forms ====
Archived and error logs can be found in <key>/var/cache/brewer-log/logs</key>.\\
Error logs are identified by the prefix of the file name <key>ERROR_*</key>.\\
Some common log issues
  - Too many <key>,</key>. This is usually found when right before the comment line or comments including commas
  - The use of a <key>;</key>. semicolon are not supported
  - Comments too long. The max length for the comment is 500 characters.

===== Metabase =====
Metabase is an open source program that is able to read from a database and plot/display it in a friendly interface. The CBSN version can be found [[http://metabase.cbsn.hiitravel.ca|here]].
<hidden To check the status of metabase>
<konsole>
cbsnet@Thinkbrew:~$ su root
Password:
root@Thinkbrew:/home/cbsnet# systemctl status metabase
● metabase.service - Metabase server
     Loaded: loaded (/etc/systemd/system/metabase.service; disabled; vendor pre>
     Active: active (running) since Sun 2025-12-07 20:22:44 GMT; 3 days ago
   Main PID: 32791 (java)
      Tasks: 160 (limit: 37964)
     Memory: 2.8G
        CPU: 45min 41.735s
     CGroup: /system.slice/metabase.service
             └─32791 /usr/bin/java -jar /home/cbsnet/metabase/metabase.jar
</konsole>
This indicates metabase is running properly. </hidden>
<hidden How to restart metabase>
<konsole>
cbsnet@Thinkbrew:~$ su root
Password:
root@Thinkbrew:/home/cbsnet# systemctl restart metabase
</konsole>
</hidden>
\\
===== Backups =====
  - Drive Backups are managed by [[https://github.com/linuxmint/timeshift|timeshift]]. 
    - Monthly Backup
      - Backup are made on the 1st of each month at 00:00:00 UTC
      - up to 3 copies are saved at one time
    - Weekly Backup
      - Backup are made on each Sunday at 21:00:00 UTC
      - up to 2 copies are saved at one time
  - Database backups are managed by the db_backup.sh
    - Database base backup are done every Tuesday and at 04:00:00 UTC
    - Up to 9 copies are saved at one time
    - Database that are being backed up are
      - brewerdata
      - brewer_db
      - brewer_security
      - brewerpart
      - metabase
\\
===== Database =====
There are currently 5 database (as of Dec 17, 2025) running on cbsnet via mariadb (Version 10.5.29-MariaDB-0+deb11u1 Debian 11)
  - brewerdata - this database is primary used to host the IDEAS database and includes all the brewer data. The best way to view this data is via [[https://metabase.cbsn.hiitravel.ca|metabase]].
  - brewer_db - this database is used to for hosting the brewer log form data.
  - brewer_security - this database is used to host the user information needed for the brewer log form
  - brewerpart - This database is used to host the brewer parts database. This is what brewparts.exe uses
  - metabase - This database is used to host the config and different views used for [[https://metabase.cbsn.hiitravel.ca|metabase]].