OnlyOffice Deployment

Overview

Deployment Status

Task	Status	Comments
Openstack project		Resources have been granted
Puppet Configuration		Minimal configuration ready
Cluster deployment		(Using malandes environment, not merged in qa yet) Document server + Redis + RabbitMQ + DBoD deployment OK. Missing: Load Balancing configuration (Agreed not needed now)
Integration with Piotr's Owncloud		Test OK
Integration in CERNBox		Test OK

OnlyOffice Architecture

The following diagram describes the OnlyOffice architecture:

Flavor	VCPUS	RAM	Total Disk
m2.large	4	8GB	40 GB
m2.medium	2	4GB	20 GB
m2.small	1	2GB	10 GB

Openstack, Puppet and Foreman details

Configuration

Commands to create the VMs in Openstack

eval $(ai-rc "OnlyOffice Cluster")

ai-bs --foreman-hostgroup onlyoffice/redis \
         --landb-responsible malandes \
         --cc7 \
         --nova-flavor m2.large \
         --foreman-environment malandes \
         --nova-sshkey malandes_key \
         redis-test

ai-bs --foreman-hostgroup onlyoffice/rabbitmq \
         --landb-responsible malandes \
         --cc7 \
         --nova-flavor m2.large \
         --foreman-environment malandes \
         --nova-sshkey malandes_key \
         rabbitmq-test

ai-bs --foreman-hostgroup onlyoffice/document_server \
        --landb-responsible malandes \
        --cc7 \
        --nova-flavor m2.xlarge \
        --foreman-environment malandes \
        --nova-sshkey malandes_key \
        document-server-test

DB on Demand Configuration

DB configuration details

• General Parameters:
  • Host: dbod-onlyoffice.cern.ch
  • port: 6605
  • user: admin
  • password: execute in aiadm tbag show --hg onlyoffice db-admin-password

• Modify the Document Server configuration scrip /usr/bin/documentserver-configure.sh. In the function execute_db_script(), change:
  • The postgres port to DB_PORT=6605 (Line 277)
  • Add the port as a parameter in CONNECTION_PARAMS="-h$DB_HOST$ -p DB_PORT -U$DB_USER -w" (Line 241)
• Connect to the database and create the following:
  • psql -q -hdbod-onlyoffice.cern.ch -p6605 -U admin
  • CREATE DATABASE onlyoffice;
  • CREATE USER onlyoffice WITH password 'onlyoffice';
  • GRANT ALL privileges ON DATABASE onlyoffice TO onlyoffice;
• Allow user onlyoffice to connect from any host by modifying the pg_hba.conf file adding the following line at the end (this can be done via DBoD GUI)
  • host    onlyoffice             onlyoffice      0.0.0.0/0       trust

Cephfs Configuration

The following instructions can be done via the Openstack GUI (Shares menu) or the CLI, as explained here. If you want to use the CLI, configure your shell setup before to be able to work with manila as explained here.

• Create a share: manila create --share-type "Meyrin CephFS" --name onlyoffice-test cephfs 1
• Define the access rules for the share: manila access-allow onlyoffice-test cephx onlyoffice-test01
• Get the access_key for the used authentication identifier: manila access-list onlyoffice-test
• Mount the share with Puppet as explained here
  • tbag set --hg onlyoffice/document_server flax.onlyoffice-test01.secret (provide the access_key when it asks for the secret)
  • add the cephfs puppet configuration in the document_server.pp manifest in gitlab

Nginx configuration

For version 1.18.0-1, due to this issue, run the following: setsebool -P httpd_setrlimit 1

The following workaround is also needed for version nginx-1.18.0-1: the nginx default.conf file should be renamed as shown below so the OnlyOffice configuration can be taken into account instead:

[root@document-server-2 ~]# cd /etc/nginx/conf.d/
[root@document-server-2 conf.d]# ll
total 4
-rw-r--r--. 1 root root 1093 Apr 21 17:05 default.conf
lrwxrwxrwx. 1 root root   45 Apr 10 07:03 ds.conf -> ../../onlyoffice/documentserver/nginx/ds.conf
[root@document-server-2 conf.d]# mv default.conf default.conf.old

Restart both nginx and supervisord services.

For version 1.16.1-1, in order to integrate with OwnCloud, the following changes were needed:

setsebool -P httpd_can_network_connect 1
setsebool -P httpd_use_fusefs 1

Source: Stackoverflow article. Not sure this is also needed if 1.18-1-1 is installed from scratch. Just in case, I documented it.

Redis configuration

• In /etc/redis.conf, comment bind 127.0.0.1 and add bind 0.0.0.0 to allow remote access on redis server.

RabbitMQ configuration

• As explained in this article guest user in RabbitMQ cannot be used from a remote host. For this reason, it is necessary to create a new user:

[root@rabbitmq-test ~]# rabbitmqctl add_user rabbitmq rabbitmq
Creating user "rabbitmq" ...
...done.
[root@rabbitmq-test ~]# rabbitmqctl set_user_tags rabbitmq administrator
Setting tags for user "rabbitmq" to [administrator] ...
...done.
[root@rabbitmq-test ~]# rabbitmqctl set_permissions rabbitmq ".*" ".*" ".*"
Setting permissions for user "rabbitmq" in vhost "/" ...
...done.

Document server configuration

• Create the file /etc/onlyoffice/documentserver/local.json with the necessary connection parameters for Redis, RabbitMQ and DB. (For the parameter names and syntax, check the example file here). It must be noted that default.json is overwritten during updates. That is why it is recommended to create the file local.json with the customised content of default.json.
• Or run the configuration script:

[root@document-server-standalone-test ~]# /usr/bin/documentserver-configure.sh
Configuring database access...
Host: dbod-onlyoffice.cern.ch
Database name: onlyoffice
User: onlyoffice
Password:
Trying to establish PostgreSQL connection... psql -q -hdbod-onlyoffice.cern.ch -p6605 -Uonlyoffice -w
....
OK
Installing PostgreSQL database... OK
Configuring redis access...
Host: redis-test

Trying to establish redis connection... OK
Configuring AMQP access...
Host: rabbitmq-test:5672
User: rabbitmq
Password:
Trying to establish AMQP connection... OK
Restarting services... OK

• The following change needs to be applied for the time being (until the CERN CA chain is deployed on the machine): In /etc/onlyoffice/documentserver/local.json change "rejectUnauthorized": true to "rejectUnauthorized": false.

Integration

OwnCloud Integration

In order to test OO with OwnCloud integration:

• Connect to oc.cern.ch (Take admin password from tbag show --hg onlyoffice oc.cern.ch-admin-password)
• In the right top corner, click on Admin -> Settings. On the left column, click on Additional Information. Then fill the OnlyOffice server settings by entering the Document Editing Service Address and save.
• If the are errors, after saving, an error message will be displayed. Otherwise, everything is OK.

Note: if you use a SSL certificate from a non-trusted CA (like CERN CA), on the OwnCloud container you need to modify /var/www/nextcloud/config/config.php and add:

'onlyoffice' => 
  array ( 
       'verify_peer_off' =>TRUE, 
  ), 

Although you should use a proper certificate or make CERN CA trusted.

CERNBox Integration

Issue 1

There are some changes in the callback URLs from document server version 5.4.1 to 5.5.1. This has been an issue when running tests opening files on both production and testing infrastructure at the same time. In order to avoid these issues, it is recommended to clean all folders in /var/lib/onlyoffice/documentserver/App_Data/cache/files and clean tables in the database:

onlyoffice=> delete from doc_changes;
onlyoffice=> delete from task_result;

Issue 2

The ST team has implemented a fix on the CERNBox side to avoid losing the contents of a file. This used to happen when CERNBox was getting 403 or similar (different from 200 in any case) when taking a file from OO. The contents of the file were replaced with:

<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>

The replacement is not happening any more after the fix. However, the error itself, happens sometimes: the changes after modifying a file are not written back to EOS. It seems to be a random issue that we are not able to reproduce. The file changes remain in the OO cache, so they are not lost. In CERNBox logs there is still a 403 error.

Issue 3

On the document server side (docservice log), the following error used to appear in the log every time a file was closed. A fix was implemented on the CERNBox side and there is no error now. For the record, it looked like this:

[2020-04-30T13:02:32.869] [ERROR] nodeJS - postData error: docId = eoshome-m._.97201250101100544;url = https://cbox-webng-03.cern.ch/index.php/apps/onlyoffice/storage/track/MALANDES%20Public/Tests/Doc-created-cbox-30Apr.docx?x-access-token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJhY2NvdW50X2lkIjoibWFsYW5kZXMiLCJncm91cHMiOltdLCJkaXNwbGF5X25hbWUiOiJNYXJpYSBBbGFuZGVzIFByYWRpbGxvIChtYWxhbmRlcykifQ.BSAg_l1N061mZBlcfNWpEoyhyXXH6LZ8TyDbivkgKK0;data = {"key":"eoshome-m._.97201250101100544","status":4,"actions":[{"type":0,"userid":"malandes"}]}
Error: Error response: statusCode:500 ;body:

    at Request._callback (/snapshot/server/build/server/Common/sources/utils.js:0:0)
    at Request.init.self.callback (/snapshot/server/build/server/Common/node_modules/request/request.js:185:22)
    at Request.emit (events.js:198:13)
    at Request.<anonymous> (/snapshot/server/build/server/Common/node_modules/request/request.js:1154:10)
    at Request.emit (events.js:198:13)
    at IncomingMessage.<anonymous> (/snapshot/server/build/server/Common/node_modules/request/request.js:1076:12)
    at Object.onceWrapper (events.js:286:20)
    at IncomingMessage.emit (events.js:203:15)
    at endReadableNT (_stream_readable.js:1143:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)

License

The License file needs to be placed in the following location: /var/www/onlyoffice/Data/license.lic

Tutorials and Documentation

Please, read the following documentation to learn about Openstack, Puppet and Foreman:

• Config Docs
• Config Training

Please, check the documentation about OnlyOffice installation in Linux:

• Installation steps
• List of open ports
• Onlyoffice API
• Changelog

Please, check these resources for OnlyOffice deployment and usage at CERN:

• OO Monitoring in Grafana
• CERN configuration
• REVA code

Debugging

• Check open connections : netstat -pant
  • Redis:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      26675/redis-server

• • RabbitMQ:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:5672           0.0.0.0:*               LISTEN      1138/beam.smp

• • Document Server:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      423/nginx: worker p
tcp6       0      0 :::8000                 :::*                    LISTEN      29366/docservice
tcp6       0      0 :::8080                 :::*                    LISTEN      29364/spellchecker
tcp6       0      0 :::80                   :::*                    LISTEN      423/nginx: worker p

• Restart document server processes: supervisorctl restart all
• Restart nginx server process: service nginx restart
• Log files are available under: /var/log/onlyoffice/documentserver
  • converter
  • docservice
  • nginx

• Enable DEBUG logs in /etc/onlyoffice/documentserver/log4js/production.json:

        "categories": {
                "default": { "appenders": [ "default" ], "level": "DEBUG" }

local.json

Tailored configuration file:

{
  "services": {
    "CoAuthoring": {
      "requestDefaults": {
        "rejectUnauthorized": false
      },
      "sql": {
        "dbHost": "dbod-onlyoffice.cern.ch",
        "dbName": "onlyoffice",
        "dbUser": "onlyoffice",
        "dbPass": "onlyoffice",
        "type": "postgres",
        "dbPort": "6605"
      },
      "redis": {
        "host": "redis-test"
      },
      "token": {
        "enable": {
          "request": {
            "inbox": false,
            "outbox": false
          },
          "browser": false
        },
        "inbox": {
          "header": "Authorization"
        },
        "outbox": {
          "header": "Authorization"
        }
      },
      "secret": {
        "inbox": {
          "string": "secret"
        },
        "outbox": {
          "string": "secret"
        },
        "session": {
          "string": "secret"
        }
      }
    }
  },
  "queue": {
    "type": "rabbitmq"
  },
  "rabbitmq": {
    "url": "amqp://rabbitmq:rabbitmq@rabbitmq-test:5672"
  }
}