Setting up a clean install of dpm on CENTOS-7

How we setup a testing DPM on the latest CENTOS7 @ Edinburgh.

This is a pedagogical set of steps that we took to get a new instance of dpm installed on centos-7 on a machine in Edinburgh. I'm not an expert, so this is just a description of what I've done.

DPMEdinburghManualInstallPoolNode

Get a machine running CENTOS7

Not going to cover how to do that here.

Some Additional Resources:

I found these useful. They may be updated/or-not but they were helpful for their contents:

* https://twiki.cern.ch/twiki/bin/view/DPM/DpmSetupManualInstallation

* https://www.gridpp.ac.uk/wiki/User_Interface_%28UI%29_to_support_approved_VOs

* http://hep.ph.liv.ac.uk/~sjones/RPMS.voms/

* https://twiki.grid.iu.edu/bin/view/Documentation/Release3/Edg-mkgridmap

* https://github.com/apel/ssm/releases

Setup the OS for grid based tasks

First enable epel:

yum install epel-release
yum update

Now install the wlcg repos:

cd /etc/yum.repos.d
wget http://linuxsoft.cern.ch/wlcg/wlcg-centos7.repo
yum update

Now you need the gridpp VOs:

cd /etc/yum.repos.d
wget http://repository.egi.eu/sw/production/cas/1/current/repo-files/EGI-trustanchors.repo
yum update
yum install lcg-CA

echo '[gridppVoms]
name=gridppVoms repo
baseurl=http://hep.ph.liv.ac.uk/~sjones/RPMS.voms/
enabled=1
gpgcheck=0
priority=100' > /etc/yum.repos.d/gridppVoms.repo
yum update
yum install gridpp-*

Now setup the EGI repo:

yum install yum-priorities
cd /etc/yum.repos.d
wget http://repository.egi.eu/community/software/preview.repository/2.0/releases/repofiles/centos-7-x86_64.repo
echo "priority=40" >> centos-7-x86_64.repo
yum update

Setup the host-key:

openssl pkcs12 -in inputCert.p12 -out hostcert.pem -nokeys
openssl pkcs12 -in inputCert.p12 -out hostkey.pem -nodes

/bin/cp hostkey.pem /etc/grid-security/hostkey.pem
/bin/cp hostcert.pem /etc/grid-security/hostcert.pem
chown root:root /etc/grid-security/host*.pem
chmod 444 /etc/grid-security/hostcert.pem
chmod 400 /etc/grid-security/hostkey.pem
ls -la /etc/grid-security/host*.pem
-r--r--r--. 1 root root 1869 Dec 13 10:59 hostcert.pem
-r--------. 1 root root 3720 Dec 13 10:59 hostkey.pem

/bin/cp hostkey.pem /etc/grid-security/dpmmgr/dpmkey.pem
/bin/cp hostcert.pem /etc/grid-security/dpmmgr/dpmcert.pem
chown dpmmgr:dpmmgr /etc/grid-security/dpmmgr/dpm*.pem
chmod 444 /etc/grid-security/dpmmgr/dpmcert.pem
chmod 400 /etc/grid-security/dpmmgr/dpmkey.pem
ls -la /etc/grid-security/dpmmgr/dpm*.pem
-r--r--r--. 1 dpmmgr dpmmgr 1869 Dec 13 13:45 /etc/grid-security/dpmmgr/dpmcert.pem
-r--------. 1 dpmmgr dpmmgr 3720 Dec 13 13:45 /etc/grid-security/dpmmgr/dpmkey.pem

You also need mkgridmap:

yum install edg-*
echo 'group vomss://lcg-voms2.cern.ch:8443/voms/atlas?/atlas/Role=lcgadmin atlas
group vomss://voms2.cern.ch:8443/voms/atlas?/atlas/Role=lcgadmin atlas
group vomss://lcg-voms2.cern.ch:8443/voms/atlas?/atlas/Role=production atlas
group vomss://voms2.cern.ch:8443/voms/atlas?/atlas/Role=production atlas
group vomss://lcg-voms2.cern.ch:8443/voms/atlas?/atlas atlas
group vomss://voms2.cern.ch:8443/voms/atlas?/atlas atlas
group vomss://voms.hellasgrid.gr:8443/voms/dteam?/dteam/Role=lcgadmin dteam
group vomss://voms.hellasgrid.gr:8443/voms/dteam?/dteam/Role=production dteam
group vomss://voms.hellasgrid.gr:8443/voms/dteam?/dteam dteam
group vomss://lcg-voms2.cern.ch:8443/voms/lhcb?/lhcb/Role=lcgadmin lhcb
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb/Role=lcgadmin lhcb
group vomss://lcg-voms2.cern.ch:8443/voms/lhcb?/lhcb/Role=production lhcb
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb/Role=production lhcb
group vomss://lcg-voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb
group vomss://lcg-voms2.cern.ch:8443/voms/cms?/cms/Role=lcgadmin cms
group vomss://voms2.cern.ch:8443/voms/cms?/cms/Role=lcgadmin cms
group vomss://lcg-voms2.cern.ch:8443/voms/cms?/cms/Role=production cms
group vomss://voms2.cern.ch:8443/voms/cms?/cms/Role=production cms
group vomss://lcg-voms2.cern.ch:8443/voms/cms?/cms cms
group vomss://voms2.cern.ch:8443/voms/cms?/cms cms
group vomss://grid-voms.desy.de:8443/voms/ildg?/ildg ildg
group vomss://lcg-voms2.cern.ch:8443/voms/ops?/ops ops
group vomss://voms2.cern.ch:8443/voms/ops?/ops ops
group vomss://voms.fnal.gov:8443/voms/lsst?/lsst lsst
gmf_local /etc/lcgdm-mapfile-local' > /etc/lcgdm-mkgridmap.conf
touch /etc/lcgdm-mapfile-local
/usr/libexec/edg-mkgridmap/edg-mkgridmap.pl --conf=/etc/lcgdm-mkgridmap.conf --output=/etc/lcgdm-mapfile --safe

Add the following cron job for the map file:

55 5,11,17,23 * * * (date; /usr/libexec/edg-mkgridmap/edg-mkgridmap.pl --conf=/etc/lcgdm-mkgridmap.conf --output=/etc/lcgdm-mapfile --safe) >> /var/log/lcgdm-mkgridmap.log 2>&1

(This output location works for dmlite, if you change it update your dmline config!)

Install mariadb:

yum install mariadb-server.x86_64

Before starting mariadb consider if you want to edit: /etc/my.cnf The exact config of this is probably well beyond the scope of a dpm install but it's probably good to at least have:

[mysqld]
user=mysql
datadir=/local/mysql
socket=/var/lib/mysql/mysql.sock
back_log=500
query_cache_size=256M
query_cache_limit=16M
innodb_buffer_pool_size=4096M
innodb_flush_method=O_DIRECT
innodb_flush_log_at_trx_commit=2
skip-innodb_doublewrite
innodb_support_xa=0
innodb_thread_concurrency=8
innodb_log_buffer_size = 8M
key_buffer_size = 16M
innodb_data_home_dir=/local/mysql
skip-external-locking
innodb_file_per_table=1
max_connections=2000
thread_cache_size=8
log-bin=mysqld-bin
expire_logs_days=2
log-slow-queries=/var/log/mysql/slowqlog

[mysqld_safe]
log-error=/var/log/mysql/mysqld.log
pid-file=/var/lib/mysql/mysqld.pid

Create the log and run folders:

mkdir -p /var/log/mysql /var/lib/mysql
chown -R mysql:mysql /var/log/mysql /var/lib/mysql

This config places the logs in the usual locations and places the mysql data in the folder /local.

Hence the following is also needed:

yum install policycoreutils-python
mkdir -p /local/mysql
chown -R mysql:mysql /local/mysql
semanage fcontext -a -t mysqld_db_t "/local/mysql(/.*)?"
restorecon -R -v /local/mysql

Enable and start mariadb:

systemctl enable mariadb.service
systemctl start mariadb.service

now change the root pass:

mysql -u root
MariaDB [(none)]> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('MyNewPass');

Setup the dpm tools

Install dpm:

yum install dpm*
# yum install dmlite-* ## This has conflicts on centos7 from dpm-*-dome
yum install dmlite-docs dmlite-dpmhead dmlite-plugins* dmlite-shell

Add the dpmmgr account for managing/running dpm and dpns

groupadd -g 151 dpmmgr
useradd -c "DPM manager" -g dpmmgr -u 151 -r -m dpmmgr

Configuring the head node:

Setup some folders and copy the host key/cert to the relevant places:

mkdir /etc/grid-security/dpmmgr
cp /etc/grid-security/hostcert.pem /etc/grid-security/dpmmgr/dpmcert.pem
cp /etc/grid-security/hostkey.pem /etc/grid-security/dpmmgr/dpmkey.pem
chown -R dpmmgr.dpmmgr /etc/grid-security/dpmmgr

Enable the dpm services of choice:

systemctl enable /usr/share/dpm-mysql/dpm.service
systemctl enable /usr/share/dpm-mysql/dpnsdaemon.service
systemctl enable /usr/share/dpm-mysql/srmv2.2.service

NB: There are a lot of warnings about this but it seems like the best way to install the service.

Now configure the mariadb service:

NB: The example below is for the srm-test.gridpp VM and has a different password than pass in production. For the sake of simplicity the single db account of dpmmgr is used for both dpm and dpns but this can be different if required.

First setup the cns_db and dpm_db databases using the examples in the dmlite package:

mysql -u root -p < /usr/share/dmlite/dbscripts/cns_mysql_db.sql
mysql -u root -p < /usr/share/dmlite/dbscripts/dpm_mysql_db.sql

Now lets fix the permissions:

mysql -u root -p
MariaDB [(none)]> use mysql
MariaDB [mysql]> GRANT ALL PRIVILEGES ON cns_db.* TO 'dpmmgr'@'srm-test.gridpp.ecdf.ed.ac.uk' IDENTIFIED BY 'pass' WITH GRANT OPTION;
MariaDB [mysql]> GRANT ALL PRIVILEGES ON cns_db.* TO 'dpmmgr'@'localhost' IDENTIFIED BY 'pass' WITH GRANT OPTION;
MariaDB [mysql]> GRANT ALL PRIVILEGES ON dpm_db.* TO 'dpmmgr'@'srm-test.gridpp.ecdf.ed.ac.uk' IDENTIFIED BY 'pass' WITH GRANT OPTION;
MariaDB [mysql]> GRANT ALL PRIVILEGES ON dpm_db.* TO 'dpmmgr'@'localhost' IDENTIFIED BY 'pass' WITH GRANT OPTION;

This has to be the fqdn and localhost of the machine. I think this can be more restrictive than all permissions but this worked for us.

Now setup the dmlite package so that it's correct:

/etc/dmlite.conf.d/mysql.conf:

MySqlHost srm-test.gridpp.ecdf.ed.ac.uk
MySqlUsername dpmmgr
MySqlPassword pass
MySqlPort 0
NsDatabase cns_db
DpmDatabase dpm_db
NsPoolSize 100
MapFile /etc/lcgdm-mapfile
HostDNIsRoot no

Configure services

Setup the host to be allowed /etc/shift.conf:

RFIOD TRUST srm-test.gridpp.ecdf.ed.ac.uk pool90-glite.rdf.ac.uk
RFIOD WTRUST srm-test.gridpp.ecdf.ed.ac.uk pool90-glite.rdf.ac.uk
RFIOD RTRUST srm-test.gridpp.ecdf.ed.ac.uk pool90-glite.rdf.ac.uk
RFIOD XTRUST srm-test.gridpp.ecdf.ed.ac.uk pool90-glite.rdf.ac.uk
RFIOD FTRUST srm-test.gridpp.ecdf.ed.ac.uk pool90-glite.rdf.ac.uk
DPM   TRUST srm-test.gridpp.ecdf.ed.ac.uk pool90-glite.rdf.ac.uk
DPNS  TRUST srm-test.gridpp.ecdf.ed.ac.uk pool90-glite.rdf.ac.uk
DPM PROTOCOLS rfio gsiftp xroot https
RFIO DAEMONV3_RDMT_BUFSIZE 524288
RFIO DAEMONV3_RDSIZE 524288
DPM REQCLEAN 3m

(I've added pool90-glite here as this is going to be a storage node attached to srm-test)

Configure the dpm, srm2.2 and dpnsdaemon services.

These services are symlinked to the /usr area and the config for these services are under /etc/sysconfig:

/etc/sysconfig/dpm:

RUN_DPMDAEMON="yes"
ALLOW_COREDUMP="no"
NB_FTHREADS=60
NB_STHREADS=20
DPM_USE_SYNCGET="yes"
GLOBUS_THREAD_MODEL="pthread"
DPNS_HOST="srm-test.gridpp.ecdf.ed.ac.uk"
DPM_HOST="srm-test.gridpp.ecdf.ed.ac.uk"
DPMCONFIGFILE="/etc/dpm-mysql/db.conf"

/etc/sysconfig/dpnsdaemon:

RUN_DPNSDAEMON="yes"
RUN_READONLY="no"
RUN_DISABLEAUTOVIDS="no"
ALLOW_COREDUMP="no"
NB_THREADS=80
DPNSDAEMONLOGFILE="/var/log/dpns/log"
GLOBUS_THREAD_MODEL="pthread"
ORACLE_HOME=''
DPNS_HOST=srm-test.gridpp.ecdf.ed.ac.uk
DPM_HOST=srm-test.gridpp.ecdf.ed.ac.uk
NSCONFIGFILE=/etc/dpm-mysql/db_ns.conf

/etc/sysconfig/srmv2.2:

RUN_SRMV2DAEMON="yes"
ULIMIT_N=4096
ALLOW_COREDUMP="no"
NB_THREADS=99
SRMV2DAEMONLOGFILE ="/var/log/srmv2.2/log"
GLOBUS_THREAD_MODEL="pthread"
ORACLE_HOME=''
DPNS_HOST=srm-test.gridpp.ecdf.ed.ac.uk
DPM_HOST=srm-test.gridpp.ecdf.ed.ac.uk
DPMCONFIGFILE=/etc/dpm-mysql/db.conf

Here the files /etc/dpm-mysql/db.conf and /etc/dpm-mysql/db_ns.conf need to be configured as follows:

/etc/dpm-mysql/db.conf:

dpmmgr/pass@localhost

/etc/dpm-mysql/db_ns.conf:

dpmmgr/pass@localhost

Make the files accessible to the user account running the services:

chown -R dpmmgr:dpmmgr /etc/dpm-mysql/db{,_ns}.conf

Manually make the log dirs with the correct permissions:

mkdir -p /var/log/srmv2.2 /var/log/dpns /var/log/dpm
chown -R dpmmgr:dpmmgr /var/log/srmv2.2 /var/log/dpns /var/log/dpm

Start the services:

systemctl start dpm
systemctl start dpnsdaemon
systemctl start srmv2.2

You should be able to check the following:

systemctl status dpm
● dpm.service - LCG Disk Pool Manager server daemon (MariaDB Backend)
   $ Loaded: loaded (/usr/share/dpm-mysql/dpm.service; enabled; vendor preset: disabled)
   $ Active: active (running) since Wed 2017-03-29 10:38:40 BST; 2s ago

...

systemctl status dpnsdaemon
● dpnsdaemon.service - LCG Disk Pool Manager nameserver daemon (MariaDB Backend)
   $ Loaded: loaded (/usr/share/dpm-mysql/dpnsdaemon.service; enabled; vendor preset: disabled)
   $ Active: active (running) since Wed 2017-03-29 10:38:40 BST; 1min 35s ago

...

systemctl status srmv2.2
● srmv2.2.service - LCG Disk Pool Manager SRM v2.2 server daemon (MariaDB Backend)
   $ Loaded: loaded (/usr/share/dpm-mysql/srmv2.2.service; enabled; vendor preset: disabled)
   $ Active: active (running) since Wed 2017-03-29 10:40:09 BST; 49s ago

...

Protocol Configuration:

rfiod

/etc/sysconfig/rfiod

RUN_RFIOD="yes"
RFIOLOGFILE=/var/log/rfio/log
OPTIONS="-sl"
GLOBUS_THREAD_MODEL="pthread"
RFIO_READOPT=16
DPNS_HOST=srm-test.gridpp.ecdf.ed.ac.uk
DPM_HOST=srm-test.gridpp.ecdf.ed.ac.uk
RFIO_PORT_RANGE="20000 25000"

systemctl enable /usr/lib/systemd/system/rfiod.service
systemctl start rfiod

xrootd

To setup xrootd we configured the following:

/etc/xrootd/xrootd-dpmdisk.cfg

ofs.trace all
xrd.trace all
cms.trace all
oss.trace all

all.adminpath /var/spool/xrootd
all.pidpath /var/run/xrootd
xrd.network nodnr


xrootd.monitor all flush 30s fstat 60 lfn ops xfr 5 window 5s dest fstat info user redir atlas-fax-eu-collector.cern.ch:9330
xrootd.async off

if exec xrootd
 xrootd.seclib libXrdSec.so
 sec.protocol /usr/lib64 gsi -crl:3 -key:/etc/grid-security/dpmmgr/dpmkey.pem -cert:/etc/grid-security/dpmmgr/dpmcert.pem -md:sha256:sha1 -ca:2 -gmapopt:10 -vomsfun:/usr/lib64/libXrdSecgsiVOMS.so -gridmap:/etc/lcgdm-mapfile
 sec.protocol /usr/lib64 unix
 xrootd.export /
 xrd.port 1095 
 xrd.report atl-prod05.slac.stanford.edu:9931 every 60s all -buff -poll sync
 ofs.authlib libXrdDPMDiskAcc.so.3
 ofs.osslib libXrdDPMOss.so.3
 ofs.authorize
 ofs.persist auto hold 0
 ofs.tpc pgm /usr/bin/xrdcp --server
 all.role server
 all.sitename UKI-SCOTGRID-ECDF
 xrootd.chksum max 3 adler32 /usr/bin/chksumadler32.sh
fi

if exec cmsd
 all.role server
fi

dpm.defaultprefix /dpm/srm-test/home
dpm.mmreqhost localhost

dpm.nohv1
if exec xrootd
fi
dpm.dmconf /etc/dmlite.conf

Edit the service to start as dpmmgr:

[Unit]
Description=XRootD xrootd deamon instance %I
Documentation=man:xrootd(8)
Documentation=http://xrootd.org/docs.html
Requires=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/bin/xrootd -l /var/log/xrootd/xrootd.log -c /etc/xrootd/xrootd-%i.cfg -k fifo -s /var/run/xrootd/xrootd-%i.pid -n %i
User=dpmmgr
Group=dpmmgr
Type=simple
Restart=on-abort
RestartSec=0
KillMode=control-group
LimitNOFILE=65536
WorkingDirectory=/var/spool/xrootd

[Install]
RequiredBy=multi-user.target

Start service:

systemctl enable xrootd@dpmdisk
systemctl start xrootd@dpmdisk

Add a redirector: /etc/xrootd/xrootd-dpmredir.cfg

ofs.trace all
xrd.trace all
cms.trace all
oss.trace all

all.adminpath /var/spool/xrootd
all.pidpath /var/run/xrootd
xrd.network nodnr


xrootd.seclib libXrdSec.so
sec.protocol /usr/lib64 gsi -crl:3 -key:/etc/grid-security/dpmmgr/dpmkey.pem -cert:/etc/grid-security/dpmmgr/dpmcert.pem -md:sha256:sha1 -ca:2 -gmapopt:10 -vomsfun:/usr/lib64/libXrdSecgsiVOMS.so -gridmap:/etc/lcgdm-mapfile
sec.protocol /usr/lib64 unix
xrootd.export /

ofs.cmslib libXrdDPMFinder.so.3
ofs.osslib libXrdDPMOss.so.3

ofs.authorize
ofs.forward all

dpm.xrdserverport 1095

all.role manager

ofs.authlib libXrdDPMRedirAcc.so.3

dpm.defaultprefix /dpm/srm-test/home
dpm.mmreqhost localhost

dpm.dmconf /etc/dmlite.conf

From the dpm instructions

 The /etc/xrootd/dpmxrd-sharedkey.dat file should be created to contain the xootd key, which should be the same for all the DPM cluster. Ensure that permissions are 600 and the owner is dpmmgr.

For a key, use a 64 byte string of random ascii characters, no newline. Other keys may be possible, consult the xroot docs for more info. 

/etc/xrootd/dpmxrd-sharedkey.dat

chown dpmmgr:dpmmgr /etc/xrootd/dpmxrd-sharedkey.dat
chmod 600 /etc/xrootd/dpmxrd-sharedkey.dat

Enable redirector:

systemctl enable xrootd@dpmredir.service
systemctl start xrootd@dpmredir.service

https (davs):

Remove externally created ssl configs and make an empty file:

rm /etc/httpd/conf.modules.d/00-ssl.conf
touch /etc/httpd/conf.modules.d/00-ssl.conf
rm /etc/httpd/conf.modules.d/00-dav.conf
touch /etc/httpd/conf.modules.d/00-dav.conf
rm /etc/httpd/conf.d/ssl.conf
touch /etc/httpd/conf.d/ssl.conf

Make sure only 1 file beginning with `z` and ending in `dav.conf` exists: /etc/httpd/conf.d/zlcgdm-dav.conf

Make sure the file `dav.conf` doesn't exist also.

Edit /etc/httpd/conf/httpd.conf

To include:

User dpmmgr
Group dpmmgr

Start services:

systemctl enable httpd
systemctl start httpd

gridftp (gsiftp):

/etc/sysconf/globus:

conf=/etc/gridftp.conf
confdir=/etc/gridftp.d
export GLOBUS_TCP_PORT_RANGE="20000,25000"
export GLOBUS_TCP_SOURCE_RANGE="20000,25000"
export GLOBUS_THREAD_MODEL="pthread"

/etc/gridftp.conf:

inetd 0
daemon 1
detach 1
chdir 1
fork 1
single 0

cas 1
secure_ipc 1
ipc_auth_mode host
allow_anonymous 0

log_level ERROR,WARN,INFO,TRANSFER
log_single /var/log/dpm-gsiftp/gridftp.log
log_transfer /var/log/dpm-gsiftp/dpm-gsiftp.log
disable_usage_stats 1
usage_stats_target usage-stats.globus.org:4810

data_node 0
stripe_blocksize 1048576
stripe_layout 2
stripe_blocksize_locked 0
stripe_layout_locked 0

blocksize 262144
sync_writes 0

port 2811

control_preauth_timeout 120
control_idle_timeout 600
ipc_idle_timeout 600
ipc_connect_timeout 600

banner_terse 0
login_msg "Disk Pool Manager (dmlite)"

load_dsi_module dmlite
use_home_dirs 1
debug 0

Start the services correctly:

systemctl disable globus-gridftp-server
systemctl enable dpm-gsiftp
systemctl start dpm-gsiftp

Testing:

Make a local directory and add it to the storage

mkdir -p /test_data
chown -R dpmmgr:dpmmgr /test_data
dpm-addpool --poolname test
dpm-addfs --poolname test --server srm-test.gridpp.ecdf.ed.ac.uk --fs /test_data --st 0

Check the storage is present:

dpm-qryconf

Make a folder within dpns and set the ability for the test proxy to access it:

dpns-mkdir /dpm/srm-test/home/dteam
dpns-chmod 755 /dpm
dpns-chmod 755 /dpm/srm-test
dpns-chmod 755 /dpm/srm-test/home
dpns-chmod 755 /dpm/srm-test/home/dteam
dpns-entergrpmap --group dteam
chmod root:dteam /dpm/srm-test/home/dteam
dpns-setacl -m d:u::7,d:g::7,d:o:5 /dpm/srm-test/home/dteam

For some reason I need to add my user to have r/w access as well as the above to run tests in this folder.

dpns-setacl -m u:'/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=rcurrie/CN=675917/CN=Robert Andrew Currie':rwx,m:rwx /dpm/srm-test/home/dteam

Create a proxy for testing:

voms-proxy-init --cert=/root/robs-cert/usercert.pem --key=/root/robs-cert/userkey.pem --voms=dteam

Perform all the tests:

dpm-tester.py --host srm-test.gridpp.ecdf.ed.ac.uk --path=/dpm/srm-test/home/dteam --verbose

To run a single test run:

dpm-tester.py --host srm-test.gridpp.ecdf.ed.ac.uk --path=/dpm/srm-test/home/dteam --verbose --tests root
dpm-tester.py --host srm-test.gridpp.ecdf.ed.ac.uk --path=/dpm/srm-test/home/dteam --verbose --tests davs
dpm-tester.py --host srm-test.gridpp.ecdf.ed.ac.uk --path=/dpm/srm-test/home/dteam --verbose --tests gsiftp

Cleanup:

dpm-rmpool --poolname test

IPTables rules:

Required ports to open within iptables are:

(from iptables-save)

-A INPUT -p tcp -m multiport --dports 5015 -m comment --comment "050 allow DPM" -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m multiport --dports 5015 -m comment --comment "050 allow DPM" -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m multiport --dports 5010 -m comment --comment "050 allow DPNS" -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m multiport --dports 1094 -m comment --comment "050 allow cmsd" -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m multiport --dports 2811 -m comment --comment "050 allow gridftp control" -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m multiport --dports 20000:25000 -m comment --comment "050 allow gridftp range" -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m multiport --dports 80,443 -m comment --comment "050 allow http and https" -j ACCEPT
-A INPUT -p tcp -m multiport --dports 5001 -m comment --comment "050 allow rfio" -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m multiport --dports 20000:25000 -m comment --comment "050 allow rfio range" -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m multiport --dports 1095 -m comment --comment "050 allow xrootd" -m state --state NEW -j ACCEPT

We chose to manage the firewall by iptables:

systemctl stop firewalld
systemctl disable firewalld
yum install iptables-services
systemctl enable iptables
systemctl start iptables

Extra Packages:

Install dpns-arguspoll for updating argus data:

yum install dpm-argus

Now add an appropriate crontab:

00 08 * * * /usr/bin/dpns-arguspoll srm_update https://auth2.glite.ecdf.ed.ac.uk:8154/authz > /dev/null

Install ssmsend for monitoring (latest at time of writing 2.1.7-1):

wget https://github.com/apel/ssm/releases/download/2.1.7-1/apel-ssm-2.1.7-1.el7.noarch.rpm
yum install apel-ssm-2.1.7-1.el7.noarch.rpm

Optional:

Add the following file /etc/profile.d/dpm_env.sh:

export DPNS_HOST=`hostname -f`
export DPM_HOST=`hostname -f`

This means that commands run on the head node just-work from even the root account when logging into the machine.

Strange problems I encountered during/after install

1

Problem: Commands like dpm-qryconf and dpns-ls hang with no comment in the log file (have to be killed with Ctrl+C).

Solution: You didn't configure the database correctly (one of the db (at least) is missing)). Go back and fix this.

2

Problem: A lot of host not trusted errors in the dpm log:

03/29 15:28:15.115  2528,2 dpm_serv: [129.215.213.94] (srm-test.gridpp.ecdf.ed.ac.uk): Host is not trusted, identity provided was (GSI,"/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=srm-test.gridpp.ecdf.ed.ac.uk")

Solution: Check the `/etc/shift.conf` This has to be set correctly

3

Problem: Problems talking to an added pool

Solution: Check the `/etc/shift.conf`

4

Problem: Machine is installed but can't list data: I see:

[root@srm-test ~]# dpm-listspaces
ERROR: unable to open DPM root (/dpm)

Solution: The machine possibly has the wrong certificate installed for the hostname. (The fact that local communication is authenticated is good, just surprising)

5

Problem: Host not known errors

[root@srm-test ~]# dpns-ls /dpm
send2nsd: NS009 - fatal configuration error: Host unknown: UNUSED /dpm: Host not known

Solution: This error was due to the host certificate not being registered and a host certificate from another (retired) machine was used in testing. (Moral of the story is always use the correct host certificate for the machine you wish to setup)

The host name of UNUSED was a bit confusing as this string couldn't be found in the config files anywhere

-- RobCurrie - 2017-07-17