BigBlueButton Infrastructure Migration, with Pod Support

Context

As part of the recovery plan, a virtual classroom solution from the Ministry of Higher Education and Research (ESR), based on the free and open-source software BigBlueButton (BBB), has been deployed nationwide.

More information can be found on the following websites:

For institutions that have never had a local BigBlueButton infrastructure, using this virtual classroom solution (BBB ESR) is straightforward.

However, for institutions that previously had a local BBB infrastructure, using the ESR’s BBB has impacts for users. This documentation can be useful for these institutions.

Impacts

Changing infrastructure is extremely simple, regardless of the platform:

By modifying these parameters, the platform will point to the new BBB architecture.

For users, the impacts concern recordings, which will no longer be visible. Sessions/meetings will still be accessible to users, as they are saved on the respective platform, but not the recordings. These recordings are saved directly on the BBB infrastructure (either on the BBB server or the Scalelite server). In the platforms, links to these recordings are displayed when viewing the session/meeting.

Ultimately, when changing infrastructure, old recordings will no longer be displayed to users; and when the old BBB architecture is shut down, old recordings will no longer be available if nothing has been done beforehand.

Constraints

Here is a reminder of the constraints to consider, which explain the proposed solution:

Provided Solution

The idea is based on:

This solution relies entirely on Pod and does not impact BigBlueButton in any way. No modifications are required on the BigBlueButton side.

This migration script is configurable and offers several possibilities:

  1. For those with few recordings to recover:
    • This script will convert presentations from the old BBB architecture into video files (via the bbb-recorder plugin) and place these files in the directory for Recording Claim.
    • Of course, if there are already video presentations, the video file will be directly copied.
    • Once all videos have been encoded, the local BBB architecture can be shut down. Users will need to retrieve their videos from the Recording Claim tab in Pod.
  2. For those with many recordings to recover:
    • The idea is to give users time to choose which recordings they want to keep (it is not possible or useful to convert everything).
    • For this, the old BBB/Scalelite server must remain open for at least a few months (just to access the recordings).
    • On the script side, if necessary, access to the Moodle database will be required to know who did what.
    • Thus, for each BBB recording, the script will create a line in “My External Videos,” of type BigBlueButton, for the moderators (who will be created if necessary in the Pod database). They can then import these recordings into Pod themselves.
    • In case some recordings are not identifiable, they will be associated with an administrator (to be configured in the script).
    • Additionally, if access to the Moodle database allows, an informational message will be placed directly in Moodle, at the level of the concerned BBB sessions.

BigBlueButton Infrastructure Migration

The script is designed to be configurable, with the ability to manage a certain number of recordings and to test it beforehand (using a dry mode).

Architecture of the Solution

Plugin bbb-recorder

To convert BBB playback presentations, I based it on the GitHub project bbb-recorder: a plugin, independent of BigBlueButton, that allows converting - via a script - a BigBlueButton web presentation into a video file.

If needed, this plugin also allows live streaming (RTMP stream) of a BigBlueButton course.

This plugin bbb-recorder had already been used for the old system, in Pod v2 (see ESUP-Portail Documentation) and has been used successfully many times.

Functioning of bbb-recorder

Running the bbb-recorder script performs the following steps:

  1. Launches a Chrome browser in the background.
  2. Chrome visits the provided link corresponding to the BigBlueButton web presentation.
  3. It performs screen recording in the form of a video file.

Installation of bbb-recorder on Encoding Servers

The reference documentation is accessible here.

For my part, on Debian 11 servers, here is what was done.

Installation of Chrome and Prerequisites
sudo -i
apt install xvfb
curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add
echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list
apt-get -y update
apt-get -y install google-chrome-stable

(info) Being an encoding server, I assume that ffmpeg is already installed. If necessary, ffmpeg must be installed.

Effective Installation

Here is the installation for a pod user.

cd ~
git clone https://github.com/jibon57/bbb-recorder
cd bbb-recorder
npm install --ignore-scripts
cp .env.example .env

Management of the directory containing the videos: in my case /data/www/pod/bbb-recorder and the log directory /data/www/pod/bbb-recorder/logs.

mkdir /data/www/pod/bbb-recorder
mkdir /data/www/pod/bbb-recorder/logs

If bbb-recorder was not installed with the correct user (pod), the generated video files may not be accessible by the Pod user and thus may not be encoded by the encoding servers.

In practice, this results in a 1° successful encoding: the BBB web presentation will be converted into a video file, but this video file will not be accessible to Pod and cannot be converted into a Pod video.

Configuration of bbb-recorder

{
  "rtmpUrl": "rtmp://xxxxxxxx:xxxxxxxxxx@xxxxx.univ.fr:1935/live/stream",
  "ffmpegServer": "ws://localhost",
  "ffmpegServerPort": 4000,
  "auth": "xxxx",
  "copyToPath": "/data/www/pod/bbb-recorder"
}

const BBBUrl = "https://xxxx.univ.fr/bigbluebutton/";
BBBSalt = "xxxxxxxxxxxxxxxxxxxx";
joinName = "recorder";

Note that bbb-recorder uses a temporary directory to generate a video before it is copied to the configured directory (see copyToPath). This temporary directory corresponds to ../Downloads.

Thus, in the case of an installation in the home directory of the pod user, the temporary directory created and used by bbb-recorder is /home/pod/Downloads.

It is necessary to ensure sufficient storage space is available.

Configuration

Configuration in Pod

Once bbb-recorder is installed on the various encoding servers, the bbb plugin must be configured directly in Pod, by editing the custom/settings_local.py file (on the encoders and on the frontend):

# Use import-video module
USE_IMPORT_VIDEO = True

# Use plugin bbb-recorder for import-video module
# Useful to convert presentation playback to video file
USE_IMPORT_VIDEO_BBB_RECORDER = True

# Directory of bbb-recorder plugin (see documentation https://github.com/jibon57/bbb-recorder)
# bbb-recorder must be installed in this directory, on all encoding servers
# bbb-recorder create a directory Downloads, at the same level, that needs disk space
IMPORT_VIDEO_BBB_RECORDER_PLUGIN = '/home/pod/bbb-recorder/'

# Directory that will contain the video files generated by bbb-recorder
IMPORT_VIDEO_BBB_RECORDER_PATH = '/data/www/pod/bbb-recorder/'

The configuration elements are as follows:

If you use the 1° option of the script, i.e., the recording claim system, you can choose not to give users the functionality to convert BBB presentations into video files. Thus, you can set USE_IMPORT_VIDEO_BBB_RECORDER = False in this case.

The other parameters are necessary, at a minimum, when executing the script.

Regarding the directory containing the video files generated by bbb-recorder (IMPORT_VIDEO_BBB_RECORDER_PATH), it must be created manually - along with its log subdirectory - with the following command lines; feel free to modify them according to your system architecture and permissions:

mkdir /data/www/pod/bbb-recorder/logs -p
chown pod:nginx /data/www/pod/bbb-recorder/logs

While automatic creation of these directories would have been possible, considering the potential issues related to architecture and permissions, it seemed preferable for the Pod administrator to create these two directories manually. They know what they are doing and can choose the location, permissions, or other aspects.

Script migrate_bbb_recordings

This script is accessible in Pod, in the pod/video/management/commands/migrate_bbb_recordings directory.

This script cannot be executed as is; it is essential to perform additional configuration, independent of the Pod configuration, directly in this file.

If you have a large number of recordings on an old version of BBB (tested in 2.2), you may encounter a 404 error when executing the script, related to the nginx timeout on the BBB server. In this case, you will need to increase its value in the /etc/bigbluebutton/nginx/web.nginx file of the BBB server.

See github.com/bigbluebutton/bigbluebutton/issues/10570

Script Functioning

As mentioned above, this script offers 2 possibilities:

  1. For those with few recordings to recover:
    • This script will convert presentations from the old BBB architecture into video files (via the bbb-recorder plugin) and place these files in the directory of a recorder for Recording Claim (see ESUP-Portail Documentation).
    • Of course, if there are already video presentations, the video file will be directly copied.
    • Once all videos have been encoded, the local BBB architecture can be shut down. Users will need to retrieve their videos from the Recording Claim tab in Pod.
    • This is possible using the --use-manual-claim parameter and the configuration directly in this file.
    • Please note that depending on your Pod architecture, encoding will be performed either via Celery tasks or directly, one after the other.
    • Feel free to test with a few recordings and run this script in the background (using &).
  2. For those with many recordings to recover:
    • The idea is to give users time to choose which recordings they want to keep (it is not possible or useful to convert everything).
    • For this, the old BBB/Scalelite server must remain open for at least a few months (just to access the recordings).
    • On the script side, if necessary, access to the Moodle database will be required to know who did what.
    • Thus, for each BBB recording, the script will create a line in “My External Videos,” of type BigBlueButton, for the moderators (who will be created if necessary in the Pod database). They can then import these recordings into Pod themselves. If some recordings are not identifiable (for example, from sources other than Pod or Moodle), they will be associated with an administrator (to be configured in the script).
    • Additionally, if access to the Moodle database allows, an informational message will be placed directly in Moodle, at the level of the concerned BBB sessions.
    • This is possible using the --use-import-video parameter, the optional --use-database-moodle parameter, and the configuration directly in this file.
  3. You can also perform a CSV export of the list of BBB recordings (allowing you to process them elsewhere).
    • This is possible using the --use-export-csv parameter, the optional --use-database-moodle parameter, and the configuration directly in this file.

This script also allows you to:

This script has been tested with Moodle 4.

Internal Script Configuration
Parameter Description Default Value / Format
SCRIPT_BBB_SERVER_URL Old URL of the BigBlueButton/Scalelite server 'https://bbb.univ.fr/'
SCRIPT_BBB_SECRET_KEY BigBlueButton or Scalelite LOADBALANCER_SECRET key 'xxxxxxxxxx'
SCRIPT_PLAYBACK_URL_23 Is the BBB version greater than 2.3, regarding playback URLs? Useful for reading presentations in 2.0 format (<=2.2) or 2.3 format (>=2.3) True
SCRIPT_RECORDER_ID Recorder used to retrieve BBB recordings (useful with --use-manual-claim) 1
SCRIPT_ADMIN_ID Administrator to whom the recordings will be added if the moderators are not identified (useful with --use-import-video) 1
DB_PARAMS Moodle database connection parameters (useful with --use-import-video and --use-database-moodle) See details below
SCRIPT_INFORM Informational message that will be set in Moodle (mdl_bigbluebuttonbn.intro) Preliminary message for the University of Montpellier
IGNORED_SERVERS List of servers to ignore ["not-a-moodle.univ.fr"]
USE_CACHE Use the last BBB response stored in an XML file instead of re-running the request False
Format of DB_PARAMS
{
    "default": {
        "host": "bddmoodle.univ.fr",
        "database": "moodle",
        "user": "moodle",
        "password": "",
        "port": null,
        "connect_timeout": 10
    },
    "server2": {
        "host": "bddmoodle.univ.fr",
        "database": "moodle2",
        "user": "moodle2",
        "password": "",
        "port": null,
        "connect_timeout": 10
    }
}
Script Arguments
Argument Description Default Value / Format
--use-manual-claim Use manual claim False
--use-import-video Use video import module False
--use-database-moodle Use Moodle database to search for moderators (requires --use-import-video or --use-export-csv) False
--min-value-record-process Minimum value of recordings to process 1
--max-value-record-process Maximum value of recordings to process 10000
--dry Simulate what will be done False
--use-export-csv Export recordings to CSV format (with --use-database-moodle to include moderators) False
Examples and Use Cases

⚠️ Before executing this script without simulation, remember to perform the necessary backups (Pod database, Moodle database if write access is required).

Initialization
cd /usr/local/django_projects/podv3
workon django_pod3
1. Claiming All Recordings (Simulation)
python -W ignore manage.py migrate_bbb_recordings --use-manual-claim --dry
2. Claiming the Last 2 Recordings (Simulation)
python -W ignore manage.py migrate_bbb_recordings --min-value-record-process=1 --max-value-record-process=2 --use-manual-claim --dry &
3. External Video Import with Moodle Database (Simulation)
python -W ignore manage.py migrate_bbb_recordings --use-import-video --use-database-moodle --dry
4. External Video Import without Moodle Database (Last 10, Simulation)
python -W ignore manage.py migrate_bbb_recordings --min-value-record-process=1 --max-value-record-process=10 --use-import-video --dry
5. CSV Export with Moodle Database (Last 10, Simulation)
python3 manage.py migrate_bbb_recordings --use_export_csv --use-database-moodle --max-value-record-process=10 --dry

Operation

Script Output

The script displays a set of information for each processed line; do not hesitate to run it in dry mode. Script

💡 It is possible to run the script multiple times; this will not create duplicates. However, it may re-encode recordings that were already encoded during the first pass.

Administration Interface

Recorder

Recorder

List of Recordings

List

Import of External Videos

Import

Logs of the Solution

Depending on your environment, Pod logs can be found in the /var/log/syslog file.

Asynchronous Tasks (CELERY_TO_ENCODE = True)

On encoding servers, Celery logs are located in /var/log/celery/worker1.log (depending on your configuration, if you use multiple workers, you may have multiple files).

RabbitMQ-Server

If you use RabbitMQ-Server, on this server, you can find information in /var/log/rabbitmq/rabbit@xxxxx.log.

bbb-recorder

The logs of bbb-recorder processes are accessible in the directory configured via IMPORT_VIDEO_BBB_RECORDER_PATH/logs.

Deletion of External Recordings from the Video Import Module

If you opted for the 2° possibility and the use of the video import module: on the day of the complete shutdown of your old infrastructure, the old links will no longer work. If you wish to delete the external recordings that concerned your old infrastructure, you can do so directly via an SQL query to be executed in the Pod database, namely:

# Replace SCRIPT_BBB_SERVER_URL with the correct value, according to your environment
DELETE FROM import_video_externalrecording WHERE source_url like 'SCRIPT_BBB_SERVER_URL%'

💡 I prefer not to include this query in the script; it is up to you to decide when you want to perform this deletion. Of course, don’t forget to make a backup first.