====== Everything to know about CBSNet ======
CBSNet is the database computer for the Brewer log form, Brewer Data, Straylight processing and etc.
===== crontab =====
The list below shows the different sections used in the crontab
a crontab jobs are setup so that they runs a command at is specific time. The format for the crontab job are as follows. m h dom mon dow command
m - minutes
h- hours
dom - day of month
mon - month
dow - day of week
a * can be used as a wild card.
Below is a list of crontab jobs setup on cbsnet
########Brewer Support########
00,30 0-3,5-23 * * * /home/cbsnet/schedule_task/rsync.sh
00 4 * * * rsync -vhrltp --exclude-from /home/cbsnet/schedule_task/exclude-list-all.txt -e "ssh -i "/home/cbsnet/schedule_task/id_depot"" remsens_ext@depot.cmc.ec.gc.ca:/incoming/brewer/ /home/cbsnet/brews/brewnet/Brewers
05,35 * * * * /home/cbsnet/schedule_task/rsync-all.sh
29 10 * * * python3 /home/cbsnet/schedule_task/stray_correction.py -a
0 16 * * * python3 /home/cbsnet/schedule_task/year_mover.py
03,33 * * * * bash /home/cbsnet/schedule_task/rsync_ios.sh 191 187 145 109 017 > ~/temp.log
0 4 * * * /home/cbsnet/schedule_task/sync_program.sh
* __rsync.sh__ is a script pulling data near real time data (Look at ~/schedule_task/exclude-list.txt to see what is excluded) from remsens_ext@depot.cmc.ec.gc.ca
* __rsync-all.sh__ is a script for pulling all data from remsens_ext@depot.cmc.ec.gc.ca
* __stray_correction.py__ is a script that processes straylight correction on the b-file. Files are copied into a stray_light corrected folder in each brewer data folder. The constant used for the straylight correction is pulled from the icf from each of the bfile
* __year_mover.py__ is a script that moves old bfile to its corresponding year directory for each brewer
* __rsync_ios.sh__ pushes triad and IOS instrument data to IOS' server
* __sync_program.sh__ pushes the master and test program directory to remsens_ext@depot.cmc.ec.gc.ca for program auto updates
########Brewer Log forms########
0,30 * * * * /home/cbsnet/brews/brewnet/imit/brewer-log-form/scripts/server/push-cache.sh
0 23 * * * /home/cbsnet/brews/brewnet/imit/brewer-log-form/scripts/server/push-update.sh
0 23 * * * rsync -rthv /home/cbsnet/brews/brewnet/imit/brewer-log-form-deploy remsens_ext@depot.cmc.ec.gc.ca:/incoming/brewer_log/update/ -e "ssh -i "/home/cbsnet/schedule_task/id_depot""
1 * * * * /home/cbsnet/brews/brewnet/imit/brewer_log_form/import_log_form.sh
########Brewer Support Add-on########
#59,15,29,45 * * * * /home/cbsnet/eubrew_downloader.py rsit Spectrometer2016 183 157 185
03,33 * * * * bash /home/cbsnet/schedule_task/rsync_ios.sh 191 187 145 > ~/temp.log
15 10 * * * fossil sync -R ~/brews/cbsnet/Brewer_Programs/IOS_Program/brewer.fossil
0 8 * * 1-5 /home/cbsnet/schedule_task/station_ip_update.py
30 4 * * 2 /home/cbsnet/schedule_task/db_backup.sh
########Ideas Database########
10,40 0-3,5-23 * * * /var/www/html/dokuwiki/ideas/loadbfiles.sh
15 4 * * * /var/www/html/dokuwiki/ideas/loadbfiles_subs.sh
30 4 * * * curl http://localhost/dokuwiki/ideas/clean_data.php
40 7 * * * curl http://localhost/dokuwiki/ideas/load-bfiles.php?cscan=1
50 0 * * * curl http://localhost/dokuwiki/ideas/update_master_soft_db.php
# 5,35 * * * * curl http://localhost/dokuwiki/ideas/load_pid.php
#13 3 15 * * curl http://localhost/dokuwiki/ideas/load_uvsys.php
0 9 * * * curl http://localhost/dokuwiki/ideas/get_fx_rates.php?db=brewpart
33 5 * * * sh /home/cbsnet/schedule_task/wiki_sync.sh
10 * * * * mysql < ~/schedule_task/mysql_sloavg
===== Brewer Log Form =====
The Brewer log form's back end deals with hosting the admin section adding/removing/editing operators, Brewers and stations.\\
These can be access via the following links,\\
[[http://cbsn.hiitravel.ca:8671/embed/operators|Manage Operators]]\\
[[http://cbsn.hiitravel.ca:8671/embed/stations|Manage Stations]]\\
[[http://cbsn.hiitravel.ca:8671/embed/instruments|Manage Instruments]]\\
The log form backend is running off cbsnet via pm2.\\
cbsnet$ pm2 ls
┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐
│ id │ name │ mode │ ↺ │ status │ cpu │ memory │
├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤
│ 1 │ brewer-admin │ fork │ 1 │ online │ 0% │ 66.2mb │
│ 0 │ brewer-back │ fork │ 1 │ online │ 0% │ 56.5mb │
└────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘
cbsnet$ pm2 restart 1
Use --update-env to update environment variables
[PM2] Applying action restartProcessId on app [1](ids: [ '1' ])
[PM2] [brewer-admin](1) ✓
┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐
│ id │ name │ mode │ ↺ │ status │ cpu │ memory │
├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤
│ 1 │ brewer-admin │ fork │ 2 │ online │ 0% │ 16.3mb │
│ 0 │ brewer-back │ fork │ 1 │ online │ 0% │ 53.3mb │
└────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘
Where '1' is the id number
==== Error in log forms ====
Archived and error logs can be found in /var/cache/brewer-log/logs.\\
Error logs are identified by the prefix of the file name ERROR_*.\\
Some common log issues
- Too many ,. This is usually found when right before the comment line or comments including commas
- The use of a ;. semicolon are not supported
- Comments too long. The max length for the comment is 500 characters.
===== Metabase =====
Metabase is an open source program that is able to read from a database and plot/display it in a friendly interface. The CBSN version can be found [[http://metabase.cbsn.hiitravel.ca|here]].
cbsnet@Thinkbrew:~$ su root
Password:
root@Thinkbrew:/home/cbsnet# systemctl status metabase
● metabase.service - Metabase server
Loaded: loaded (/etc/systemd/system/metabase.service; disabled; vendor pre>
Active: active (running) since Sun 2025-12-07 20:22:44 GMT; 3 days ago
Main PID: 32791 (java)
Tasks: 160 (limit: 37964)
Memory: 2.8G
CPU: 45min 41.735s
CGroup: /system.slice/metabase.service
└─32791 /usr/bin/java -jar /home/cbsnet/metabase/metabase.jar
This indicates metabase is running properly.
cbsnet@Thinkbrew:~$ su root
Password:
root@Thinkbrew:/home/cbsnet# systemctl restart metabase
\\
===== Backups =====
- Drive Backups are managed by [[https://github.com/linuxmint/timeshift|timeshift]].
- Monthly Backup
- Backup are made on the 1st of each month at 00:00:00 UTC
- up to 3 copies are saved at one time
- Weekly Backup
- Backup are made on each Sunday at 21:00:00 UTC
- up to 2 copies are saved at one time
- Database backups are managed by the db_backup.sh
- Database base backup are done every Tuesday and at 04:00:00 UTC
- Up to 9 copies are saved at one time
- Database that are being backed up are
- brewerdata
- brewer_db
- brewer_security
- brewerpart
- metabase
\\
===== Database =====
There are currently 5 database (as of Dec 17, 2025) running on cbsnet via mariadb (Version 10.5.29-MariaDB-0+deb11u1 Debian 11)
- brewerdata - this database is primary used to host the IDEAS database and includes all the brewer data. The best way to view this data is via [[https://metabase.cbsn.hiitravel.ca|metabase]].
- brewer_db - this database is used to for hosting the brewer log form data.
- brewer_security - this database is used to host the user information needed for the brewer log form
- brewerpart - This database is used to host the brewer parts database. This is what brewparts.exe uses
- metabase - This database is used to host the config and different views used for [[https://metabase.cbsn.hiitravel.ca|metabase]].