App4Cam Backend

Table of Contents

1. Development
   • Prerequisites
   • Setup
   • Running the app
   • Test
   • API documentation
   • License notice
2. Deployment
   • 1. Installing dependencies, creating user, folders and setting permissions
     • Hostname configuration: sudo host
   • 2. Installing Motion
     • Configuring Motion
     • Running Motion as service
     • "libcamerify" Motion
   • 3. Setting up network behavior
     • On Raspberry Pi
     • On Variscite
   • 4. Installing ExifTool
   • 5. Enabling user services and USB auto-mounting
     • Checking USB auto-mounting
   • 6. Make sure automatic time synchronisaton is disabled
   • 7. Installing Witty Pi 4 (Raspberry Pi only)
   • 8. Adding FTP access (Optional for Raspberry Pi only)
   • 9. Hardware Configuration & logging
     • DiMON temperature sensor
     • NewtCAM interface board
     • Setting log rotation
   • 10. Setting up the reverse proxy for the backend
   • 11. Deploying the application
   • 12. For continuous deployment (CD) only
3. Release procedure

Development

Prerequisites

• >= Node.js 18.x

Setup

1. Run: npm install
2. Copy config file config/sample.env to config/development.env.
3. Set DISABLE_ACCESS_CONTROL_ALLOW_ORIGIN to false.
4. Optionally edit further options as needed.

Running the app

# development
$ npm run start

# watch mode
$ npm run start:dev

# production mode
$ npm run start:prod

Test

# unit tests
$ npm run test

# e2e tests
$ npm run test:e2e

# test coverage
$ npm run test:cov

API documentation

An OpenAPI Specification (OAS) is available under /api.

License notice

The following license notice must be included as a comment at the beginning of every source code file:

Copyright (C) 2022-2024  Luxembourg Institute of Science and Technology

App4Cam is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

App4Cam is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with App4Cam.  If not, see <https://www.gnu.org/licenses/>.

Deployment

This software requires the following tools to be installed:

1. Installing dependencies, creating user, folders and setting permissions

Execute the setup script with root permissions:

sudo scripts/setup/set-up-environment.sh

- Hostname configuration: sudo host

If you type sudo in the terminal and get the error message sudo: unable to resolve host it means there is an issue with the system's hostname configuration. To fix it:

1. Verify the current hostname of your system (for NewtCAM is imx8mm-var-dart):

hostname

2. Check if it exists on the /etc/hosts file:

nano /etc/hosts

3. Update Hostname (if necessary):

(For NewtCam) Add imx8mm-var-dart after localhost

2. Installing Motion

Motion is a configurable software that monitors video signals from different types of cameras and create videos and/or saves pictures of the activity. Motion is installed from the release deb files which provides a more recent version than the one available via apt. The most recent versions can be downloaded here Motion releases.

Raspberry Pi DiMON (Architecture: arm64 / OS: bookworm)

bookworm_motion_x.x.x-x_arm64.deb

Current installed version: Motion 4.7.0

Variscite NewtCam (Architecture: arm64 / OS: bullseye)

bullseye_motion_x.x.x-x_arm64.deb

Current installed version: Motion 4.7.0

After determining the deb file name appropriate for our distribution and platform we open up a terminal window and type (example for the arm64 bookworm):

wget https://github.com/Motion-Project/motion/releases/download/release-x.x.x/bookworm_motion_x.x.x-x_arm64.deb

Next, install the retrieved deb package. The gdebi tool will automatically retrieve any dependency packages.

sudo apt-get install gdebi-core
sudo gdebi bookworm_motion_x.x.x-x_arm64.deb

Dependecies issues If you're having dependency issues and you're certain that all requirements are fulfilled, follow this tutorial to update the dependencies file and recompile the package Package Dependencies.

- Configuring Motion

1. Open Motion service file: sudo nano /lib/systemd/system/motion.service
2. Add the argument to the following line, with <device-type> being either DiMON or NewtCAM:

ExecStart=/usr/bin/motion -c /home/app4cam/app4cam-backend/motion/config/<device-type>/motion.conf

3. Reload systemd: sudo systemctl daemon-reload
4. Add motion user to app4cam user group so that they can change the motion config file: sudo usermod -a -G app4cam motion
5. If you use auto-deployment, add motion service start and stop scripts: sudo /home/app4cam/app4cam-backend/scripts/setup/create-motion-scripts.sh

The configuration are described in detail at https://motion-project.github.io/motion_config.html).

- Running Motion as service

Motion needs to be run as a service so that it automatically starts whenever the device is started.

1. Enable service: sudo systemctl enable motion
2. Start service: sudo systemctl start motion
3. Verify that the service is running: sudo systemctl status motion

During development, you may need to stop Motion: sudo systemctl stop motion

- "libcamerify" Motion

libcamerify is needed for rpicam support (used with the newer RPi cameras). By default rpicam-apps is already installed on the new bookworm os. If you get any issues libcamerify Motion as suggested here.

Note: Typicall issues can be solved by just doing the step Install dependencies.

1. Make sure the camera is well configured and is compatible with rpicam. Get an image preview with: rpicam-hello

2. Modify the motion service sudo nano /lib/systemd/system/motion.service changing the ExecStart line to
   ExecStart=libcamerify /usr/bin/motion

3. Save and close. Follow with:

   sudo systemctl daemon-reload
   sudo systemctl start motion.service

4. Disable the autofocus of the camera. For the 64MP Hawkeye, you need to remove the key "rpi.af" with its value in the file /usr/share/libcamera/ipa/rpi/vc4/arducam_64mp.json after backing it up:

   sudo cp /usr/share/libcamera/ipa/rpi/vc4/arducam_64mp.json /usr/share/libcamera/ipa/rpi/vc4/arducam_64mp.json.backup
   sudo nano /usr/share/libcamera/ipa/rpi/vc4/arducam_64mp.json

IMPORTANT: When using pivariety cameras (e.g. 64MP Hawkeye) do not update RPICAM.

3. Setting up network behavior

- On Raspberry Pi

We want to configure the Raspberry Pi in a way that it will connect to a previously configured Wifi network when the Pi is in range of the router (Laboratory conditions) or Automatically setup a Raspberry Pi access point when a known wifi network is not in range (Field conditions). For this purpose we will use the script Autohotspot developed by RaspberryConnect.com.
On a new terminal, run the setup script with root privileges:

wget https://github.com/RaspberryConnect/AutoHotspot-Installer/releases/download/v0.74/AutoHotspot-Setup.tar.xz
tar xf AutoHotspot-Setup.tar.xz
sudo Autohotspot/autohotspot-setup.sh

You will be presented with these options:

 1 = Install Autohotspot with eth0 access for Connected Devices
 2 = Install Autohotspot with No eth0 for connected devices
 3 = Install a Permanent Access Point with eth0 access for connected devices
 4 = Uninstall Autohotspot or permanent access point
 5 = Add a new wifi network to the Pi (SSID) or update the password for an existing one.
 6 = Autohotspot: Force to an access point or connect to WiFi network if a known SSID is in range
 7 = Change the access points SSID and password
 8 = Exit

We should go for Option 2: Install Autohotspot with No eth0 for connected devices.

Once installed and after a reboot the Raspberry Pi will connect to a router that has previously been connected to and is listed in /etc/wpa_supplicant/wpa_supplicant.conf. If no router is in range then it will generate a WiFi access point. The Pi can use the eth0 connection and also be accessed from a device on the etho network.
The default SSID of App4Cam and password of 0123456789 will be used. Once a connection to the access point has been made you can access the Raspberry Pi via ssh & VNC.

ssh pi@10.0.0.5
vnc: 10.0.0.5::5900

If no error messages was presented, just exit the script and reboot your device. The "network behavior" should be well configured.

You can then also remove the files:

rm -r Autohotspot
rm AutoHotspot-Setup.tar.xz

- On Variscite

1. Run with root privileges the script: /home/app4cam/app4cam-backend/scripts/setup/variscite/setup-access-point.sh
2. Verify that the connection is running: nmcli connection show
3. Check the Wi-Fi's broadcast IP address: ifconfig

4. Installing ExifTool

ExifTool is needed to add the device ID to the metadata of each shot file.

1. Download latest version from website: wget <download-url>
2. Unpack the distribution file: gzip -dc Image-ExifTool-<latest-number>.tar.gz | tar -xf -
3. Change into directory: cd Image-ExifTool-<latest-number>
4. Prepare make file: perl Makefile.PL
5. Optionally, run tests to verify system compatibility: make test
6. Install for all users: sudo make install

5. Enabling user services and USB auto-mounting

Execute the setup script with root permissions:

sudo scripts/setup/set-up-user-services.sh

- Checking USB auto-mounting

Only perform this checks if you have enabled this functionality in the setup script before.

1. Make sure that there is no udev rule configured for USB auto-mounting. The OS images of Variscite devices may come with a preconfigured rule in /etc/udev/rules.d/automount.rules that needs to be commented out.
2. Log in as user: su - app4cam
3. Verify that the service is running: systemctl --user status udiskie
4. Logout: exit

6. Make sure automatic time synchronisaton is disabled

Verify the result line NTP service of running: timedatectl

If it is still active, run: sudo timedatectl set-ntp 0

7. Installing Witty Pi 4 (Raspberry Pi only)

Witty Pi is a realtime clock (RTC) and power management board added to the Raspberry Pi. It also allows to define ON/OFF sequences. Currently we are suporting witty pi version 3 or 4.

For witty pi 4, run these two commands in the raspberry pi home directory:

wget http://www.uugear.com/repo/WittyPi4/install.sh
sudo sh install.sh

A more extensive tutorial can be found at https://www.uugear.com/product/witty-pi-4/.

If using the Witty pi 3, after the above commands, run this script to make the device compatible with the HW:
sudo /home/app4cam/app4cam-backend/scripts/setup/raspberry-pi/witty-pi/select-wittypi.sh

8. Adding FTP access (Optional for Raspberry Pi only)

The FTP access can be used as an alternative way to download multiple files without the need to archive files.

1. Install FTP server: sudo apt install vsftpd

2. Modify the configuration: sudo nano /etc/vsftpd.conf

   Make sure the following settings are present, i.e. uncommented or added:

   anonymous_enable=NO
   local_enable=YES
   local_umask=022
   chroot_local_user=YES
   user_sub_token=$USER
   local_root=/home/app4cam/data
   allow_writeable_chroot=YES
   

3. Restart the server: sudo systemctl restart vsftpd

Now, you can connect via an FTP client with the device's IP address, port 21, the username created and the corresponding password.

9. Hardware Configuration & logging

- DiMON temperature sensor

The DS18B20 is a digital temperature sensor that uses the 1-Wire communication protocol to communicate with the RPi. It provides accurate temperature readings with a resolution of up to 12 bits. Integrating the DS18B20 temperature sensor with the Witty Pi on a RPi involves connecting the sensor to another than the default GPIO pin of the RPi and configuring it accordingly. Assign 1-wire to a different GPIO pin by editing the interface in: /boot/firmware/config.txt file.

Replace the dtoverlay=w1-gpio if found or simply add: dtoverlay=w1-gpio,gpiopin=18

Note: If the 1-Wire interface is enabled on GPIO-4 and Witty Pi’s software is installed, it might be impossible to login to the RPi because it always shuts itself down before we get the chance to login.

- NewtCAM interface board

To get a complete overview of the hardware available and it's configuartion please read the wiki variscite-guide.

• WiFi Control - Follow the local guide available here WiFi Control.
• Battery Monitoring - Follow the local guide available here Battery Monitoring.
• RTC control - Follow the local guide available here RTC control.
• Hardware initialisation - Follow the local guide available here Hardware initialization.

- Setting log rotation

Part of the app4cam log file was being lost during system reboots or shutdowns. This was identified as a result of excessively long log rotation intervals. To prevent this, the rotation period was reduced from 4 weeks to 2 weeks by editing the logrotate configuration file:

nano /etc/logrotate.conf

And set rotation to 2 weeks:

# keep 2 weeks worth of backlogs
rotate 2

10. Setting up the reverse proxy for the backend

Execute the setup script with root permissions:

sudo scripts/setup/set-up-reverse-proxy.sh

11. Deploying the application

Before starting, install the following dependency: sudo apt install gpiod

Option 1: Download the artifact archive from Gitlab:

1. Download and extract the archive into the home folder.
2. Change into the directory: cd app4cam-backend
3. Install the production dependencies: npm ci --omit=dev --ignore-scripts

Option 2: Build it yourself:

1. Make sure Git is installed.
2. Clone this repository into the home folder: git clone --single-branch --branch main https://git.list.lu/host/mechatronics/app4cam-backend.git
3. Change into the directory: cd app4cam-backend
4. Install dependencies: npm ci
5. Build: npm run build
6. Set a configuration file. For instance, use the sample file: cp config/sample.env config/production.env

Final steps

1. Adapt the configuration file if needed: nano config/production.env
2. Start service: systemctl --user start app4cam-backend
3. Verify the service is running: systemctl --user status app4cam-backend

You can check the service logs with the following command: journalctl --user -u app4cam-backend -e

12. For continuous deployment (CD) only

If you have set up the frontend already, you can just follow its setup guide.

Release procedure

1. Run the following script with the real version number in a Git Bash:

   scripts/release/release-version.sh "<version-number>"

2. Push the pre-version commit to the remote repository once it should be shared:

   git push