Upgrading the TeamDrive Host Server =================================== General Upgrade Notes --------------------- There are two basic approaches to updating a TeamDrive Host Server: **in-place**, by replacing the software with a newer version on the live system, or starting a **new instance and migrating the configuration** and data (MySQL Database and Space Volumes) to the new instance. For older installations, performing a migration to a freshly installed instance might be the better approach, to get rid of accumulated "cruft" and to start from a clean slate. In case the current system is still running a 32-bit installation, moving to a 64-bit system is required, as newer versions of the Host Server **no longer support 32-bit environments**. Updating requires a service interruption, as the Host Server components (e.g. the Apache http Server) need to be stopped while the update is in progress. Short downtimes usually pass unnoticed by the TeamDrive Clients, they will simply try again after a short waiting period. Local Client operations can continue. The Host Server-specific MySQL Databases and Space Volumes are the two crucial pieces of data that need to be preserved during updates. Take backups prior to performing an update and *verify they worked correctly*. In case of an in-place upgrade, both the databases and Space Volumes can be taken over "as is". When performing a migration to a new instance, the databases and volumes need to be copied or moved to the new host. Updates between different Host Server versions (e.g. from 3.0.011 to 3.0.013) usually require changes to the MySQL table structures. Starting with version 3.0.013, these changes are applied automatically when starting the service after updating. Reversing these changes (e.g. reverting to the previous version) requires going back to the previous backup, there is **no automatic roll-back of changes to the database/table structures**. Starting with version 3.0.013, updates to a new build (e.g. from 3.0.013.0 to 3.0.013.1) can be performed using yum/RPM. Updating from older versions requires manual intervention, as the installations were performed without automatic package management. In-place Upgrading Version 3.0.013 to a Newer Build --------------------------------------------------- The use of RPM packages makes updating from one build to another (e.g. from 3.0.013.0 to 3.0.013.1) a fairly straightforward and automatic process. Usually, you can simply replace the existing packages while the service is running. The update performs an immediate restart of the services (``httpd`` and ``td-hostserver`` automatically): .. parsed-literal:: [root\@hostserver ~]# yum update td-hostserver Check the chapter :ref:`releasenotes-3.0.013` for the changes introduced in each build. In-place Upgrading from Older Versions to 3.0.013 ------------------------------------------------- These instructions assume a default installation of the TeamDrive Host Server (version 3.0.009, 3.0.010 or 3.0.011) on RHEL6 or a derivative distribution like CentOS 6 (64-bit) that was set up based on the Host Server installation instructions or using the TeamDrive Host Server Virtual Appliance for VMware. .. note:: The following approach does not work on 32-bit systems and it does not apply to custom installations on other Linux distributions. If you performed an installation to different locations/directories, the process might work, but has not been tested/verified. The overall procedure is similar in all cases |---| we'll remove the old software components while maintaining the MySQL databases and Space Volumes, install the current versions of the Host Server packages and and migrate a few configuration settings by performing the following steps: - Stop the Apache http Server and PrimeBase processes (PBAC, PBAS) - Perform a backup of the Host Server's MySQL Databases - Remove the PrimeBase Application Environment and related files - Remove old Apache modules - Remove the ``teamdrive`` user and its home directory ``/home/teamdrive`` - Install the new Host Server RPM packages ``yvva`` and ``td-hostserver`` - Update the configuration files, remove the old configuration files after merging the settings - Update the privileges for the ``teamdrive`` MySQL user, if required - Start the TeamDrive Hosting Service and Apache http Server, check the log files for any errors - Test the new setup with a local test client before allowing all user Clients to connect to the new instance again The following paragraphs explain these steps in more detail. Stop the TeamDrive Services ~~~~~~~~~~~~~~~~~~~~~~~~~~~ As a first step, the currently running TeamDrive Hosting Services need to be shut down. Start by stopping the Apache http Server:: [root@hostserver ~]# service httpd stop Next, stop the API, Admin Console and background tasks:: [root@hostserver ~]# pbctl stop Use ``pbctl status`` to check that the services have been stopped (their ``Status`` needs to be ``Stopped``) and ``ps`` or ``pstree`` to double check that there are no stray ``httpd``, ``pbeas``, ``ase``, ``pbas``, ``pbac`` or ``smm`` processes running. Use ``kill `` or ``pkill `` to terminate these, if they don't disappear shortly after you issued the stop commands. .. _createmysqlbackup: Create a MySQL Backup ~~~~~~~~~~~~~~~~~~~~~ After all TeamDrive Services have been stopped, you should now create a backup of the MySQL databases, e.g. using ``mysqldump``:: [root@hostserver ~]# mysqldump -u root -p --force \ --databases hostapilog pbpg pbur pspace td2apilog \ | gzip > td-hostserver-mysql-$(date +%Y-%m-%d_%H.%M).sql.gz .. note:: Older versions of of the Host Server (3.0.011.3 and older) used ``td2apilog`` instead of ``hostapilog`` and an additional ``pbpg`` database. The ``--force`` option in the example above ensures that the dump continues, even if some of the databases don't exist in your setup. Backup the old Installation and Configuration Files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Next, create a backup the old PrimeBase Application Environment, Apache Modules and config files, if you don't have a full system backup already (e.g. a VM snapshot) that you could revert to in case of issues. Note that some of these files might not exist on your local installation. The following sample shell script will skip these and add all existing ones to a backup tar archive named ``td-hostserver-backup-YYYY-MM-DD.tar.gz`` in the current directory:: #!/bin/sh BACKUP="td-hostserver-backup-$(date +%Y-%m-%d).tar" FILES=" /etc/httpd/conf.d/ssl.conf /etc/httpd/conf.d/pbas.conf /etc/httpd/conf.d/teamdrive.conf /etc/httpd/conf/httpd.conf /etc/httpd/modules/mod_pbas*.so /etc/httpd/modules/mod_pspace*.so /etc/httpd/myssl /etc/init.d/primebase.boot /etc/logrotate.d/teamdrive /etc/primebase /etc/profile.d/custom.csh /etc/profile.d/custom.sh /etc/profile.d/primebase.sh /etc/profile.d/teamdrive.sh /home/teamdrive /usr/local/lib /usr/local/lib64 /usr/local/primebase" for a in $FILES do if [ -e $a ] then tar rvf $BACKUP $a fi done gzip $BACKUP Remove Obsolete Files and Binaries ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Now, remove the files and binaries that are no longer required (except for some config files). Again, note that not all of these files might exist on your local installation:: [root@hostserver ~]# rm -rf \ /etc/httpd/conf.d/pbas.conf \ /etc/httpd/modules/mod_pbas*.so \ /etc/httpd/modules/mod_pbt.so \ /etc/httpd/modules/mod_pspace*.so \ /etc/init.d/primebase.boot \ /etc/logrotate.d/teamdrive \ /etc/primebase \ /etc/profile.d/custom.csh \ /etc/profile.d/custom.sh \ /etc/profile.d/primebase.sh \ /etc/profile.d/teamdrive.sh \ /home/teamdrive \ /usr/local/primebase Finally, remove the ``teamdrive`` user (Host Server version 3.0.013 runs under the user ``httpd`` instead):: [root@hostserver ~]# userdel teamdrive The system has now been prepared for installing the new version of the Host Server Software. Install the new Host Server Software ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Install the new Host Server components (``td-hostserver`` and ``yvva``) from the dedicated TeamDrive Host Server yum repository:: [root@hostserver ~]# wget -O /etc/yum.repos.d/td-hostserver.repo \ http://repo.teamdrive.net/td-hostserver.repo [root@hostserver ~]# yum install td-hostserver The new Host Server software version 3.0.013 uses a different configuration file layout. The required values need to be migrated manually from the previous configuration files. Update the following configuration files with a text editor before deleting the old ones: - Migrate the MySQL login credentials (especially ``username`` and ``password``) from the ``[p1db]`` configuration group in ``my.cnf`` to the new configuration file ``/etc/td-hostserver.my.cnf`` and delete the ``[p1db]`` group from ``my.cnf`` when done. If your Host Server version was installed before September 2013, your ``my.cnf`` file likely does not include a ``[p1db]`` group. In this case, fill out the correct values like ``username`` and ``password`` in ``/etc/td-hostserver.my.cnf`` based on your installation notes. Also ensure that the ``socket`` path used in ``td-hostserver.my.cnf`` matches the one defined in the ``[mysqld]`` group in``my.cnf`` and adjust the path name, if necessary. - Migrate the ``RewriteRule`` for the Storage Upgrade button from ``/etc/httpd/conf.d/teamdrive.conf`` to ``/etc/httpd/conf.d/td-hostserver.httpd.conf`` and remove ``teamdrive.conf`` afterwards. - Edit ``/etc/httpd/conf.d/ssl.conf``, ensure the the following rewrite rules exist (usually at the bottom of the file within the ``VirtualHost`` section and look as follows:: RewriteEngine on RewriteLogLevel 0 RewriteLog "/var/log/httpd/rewrite.log" RewriteRule ^/admin$ /admin/ [R] RewriteRule ^/admin(.*) /yvva/p1a$1 [PT] RewriteRule ^/pbas/p1_as/api/(.*)$ /yvva/api/$1 [PT] If the following rewrite rules already exist in this file, make sure to replace them with the rules above:: RewriteRule ^/admin(.*) /pbas/p1_as/p1a$1 [PT] RewriteRule ^/depot(.*) /pbas/p1_as/p1p$1 [PT] .. _checkmysqlprivileges: Check and Update the privileges for the ``teamdrive`` MySQL User ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting with version 3.0.013, the ``teamdrive`` MySQL user needs privileges on the following MySQL databases: ``hostapilog`` and ``pspace``. Make sure that these exist and add the missing ones, if required. The following example assumes a MySQL database running on ``localhost``. Amend the commands accordingly, if you are using a remote MySQL database. Log into the MySQL database as the ``root`` user and use the ``SHOW GRANTS`` statement to check the user's current privileges:: mysql> SHOW GRANTS FOR 'teamdrive'@'localhost'; +----------------------------------------------------------------------+ | Grants for teamdrive@localhost | +----------------------------------------------------------------------+ | GRANT USAGE ON *.* TO 'teamdrive'@'localhost' IDENTIFIED BY PASSWORD | | '*0132BD69C7F09A437AF424D3B94A6A56C3AC4AC1' | | GRANT ALL PRIVILEGES ON `pspace`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `pbpg`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `pbur`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `td2apilog`.* TO 'teamdrive'@'localhost' | +----------------------------------------------------------------------+ 5 rows in set (0.00 sec) In this case, the ``teamdrive`` user does not have privileges on the new ``hostapilog`` database. You can add these using the following command:: mysql> GRANT ALL PRIVILEGES ON `hostapilog`.* TO 'teamdrive'@'localhost'; Query OK, 0 rows affected (0.00 sec) mysql> SHOW GRANTS FOR 'teamdrive'@'localhost'; +----------------------------------------------------------------------+ | Grants for teamdrive@localhost | +----------------------------------------------------------------------+ | GRANT USAGE ON *.* TO 'teamdrive'@'localhost' IDENTIFIED BY PASSWORD | | '*0132BD69C7F09A437AF424D3B94A6A56C3AC4AC1' | | GRANT ALL PRIVILEGES ON `pspace`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `pbpg`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `pbur`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `td2apilog`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `hostapilog`.* TO 'teamdrive'@'localhost' | +----------------------------------------------------------------------+ 6 rows in set (0.00 sec) As a final step, the ``hostapilog`` database needs to be created:: mysql> CREATE DATABASE hostapilog; Query OK, 1 row affected (0.00 sec) We'll keep the other privileges for now, until we have started the new version of the Host Server and the table conversion has concluded. Once everything works, the privileges on the ``pbpg``, ``pbur`` and ``td2apilog`` databases can be revoked. Start the Host Server Components ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Now start the TeamDrive Hosting Service:: [root@hostserver ~]# service td-hostserver start Starting TeamDrive Hosting Services: [ OK ] Check the log file for any errors:: [root@hostserver ~]# less /var/log/td-hostserver.log Example:: 140613 15:03:31 [Note] yvvad startup 140613 15:03:31 [Note] Using config file: /etc/td-hosting.conf 140613 15:03:31 [Note] yvvad running in repeat 60 (seconds) mode 140613 15:03:31 [Note] *** Version 3.0.013: Adding VolGlobalID column to Volume table 140613 15:03:31 [Note] ALTER TABLE pspace.Volume ADD COLUMN VolGlobalID VARCHAR(40) CHARACTER SET ascii NULL AFTER Identifier; 140613 15:03:32 [Note] Setting VolGlobalID of Volume: vol01 140613 15:03:32 [Note] *** Version 3.0.013: Upgrading Session table 140613 15:03:32 [Note] DELETE FROM Session; 140613 15:03:32 [Note] ALTER TABLE pspace.Session ADD COLUMN VolGlobalID VARCHAR(40) CHARACTER SET ascii NULL AFTER VolumeName; 140613 15:03:32 [Note] *** Version 3.0.013: Creating table AdminSession 140613 15:03:32 [Note] CREATE TABLE pspace.AdminSession ( 140613 15:03:32 [Note] ID INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY, 140613 15:03:32 [Note] Cookie VARCHAR(100) CHARACTER SET ascii UNIQUE NOT NULL, 140613 15:03:32 [Note] CreationTime TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, 140613 15:03:32 [Note] AccessTime TIMESTAMP NULL, 140613 15:03:32 [Note] SessionData MEDIUMBLOB 140613 15:03:32 [Note] ) ENGINE=InnoDB; 140613 15:03:32 [Note] *** Version 3.0.013: Dropping Setting.IndNameSetting index 140613 15:03:32 [Note] ALTER TABLE pspace.Setting DROP INDEX IndNameSetting; 140613 15:03:32 [Note] *** Version 3.0.013: Dropping database pbur 140613 15:03:32 [Note] DROP DATABASE pbur; 140613 15:03:32 [Note] *** Version 3.0.013: Setting PathToSAKConverter to "/opt/teamdrive/sakh/" Note the automatic conversion of the MySQL databases to the 3.0.013 format and the changes to some configuration settings. The number of changes to the table structures depends on the release date of the previous version. We have successfully tested this process on Host Server versions back to 3.0.009 (August 2012). The amount of data stored in these tables affects how long the conversion will take. After the table transformation has finished, the now obsolete MySQL databases have also been removed:: [root@hostserver ~]# mysql -u teamdrive -p -e "SHOW DATABASES;" Enter password: +--------------------+ | Database | +--------------------+ | information_schema | | hostapilog | | pspace | +--------------------+ You can now revoke the no longer needed privileges on these databases. Log in as the MySQL ``root`` user to perform these changes:: mysql> SHOW GRANTS FOR teamdrive@localhost; +-----------------------------------------------------------------------------+ | Grants for teamdrive@localhost | +-----------------------------------------------------------------------------+ | GRANT USAGE ON *.* TO 'teamdrive'@'localhost' IDENTIFIED BY PASSWORD | | '*0132BD69C7F09A437AF424D3B94A6A56C3AC4AC1' | | GRANT ALL PRIVILEGES ON `pspace`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `pbpg`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `pbur`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `td2apilog`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `hostapilog`.* TO 'teamdrive'@'localhost' | +-----------------------------------------------------------------------------+ 6 rows in set (0.00 sec) mysql> REVOKE ALL PRIVILEGES ON pbpg.* FROM 'teamdrive'@'localhost'; Query OK, 0 rows affected (0.00 sec) mysql> REVOKE ALL PRIVILEGES ON pbur.* FROM 'teamdrive'@'localhost'; Query OK, 0 rows affected (0.00 sec) mysql> REVOKE ALL PRIVILEGES ON td2apilog.* FROM 'teamdrive'@'localhost'; Query OK, 0 rows affected (0.00 sec) mysql> show grants for teamdrive@localhost; +-----------------------------------------------------------------------------+ | Grants for teamdrive@localhost | +-----------------------------------------------------------------------------+ | GRANT USAGE ON *.* TO 'teamdrive'@'localhost' IDENTIFIED BY PASSWORD | | '*0132BD69C7F09A437AF424D3B94A6A56C3AC4AC1' | | GRANT ALL PRIVILEGES ON `pspace`.* TO 'teamdrive'@'localhost' | | GRANT ALL PRIVILEGES ON `hostapilog`.* TO 'teamdrive'@'localhost' | +-----------------------------------------------------------------------------+ 3 rows in set (0.00 sec) Next, start the Apache http Server:: [root@hostserver ~]# service httpd start Starting httpd: [ OK ] Check the log files for any errors:: [root@hostserver ~]# less /var/log/httpd/error_log [root@hostserver ~]# less /var/log/mod_pspace.log [root@hostserver ~]# less /var/log/mod_yvva.log In case of any errors, check the chapter :ref:`troubleshooting` for guidance. Log into the Administration Console ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ After the services have been started, try logging into the Administration Console and verify the settings. Note that logging into the Administration Console is only possible for the ``HostAdmin`` user account. This user's password has been migrated from the ``pbur`` database automatically. If you used a different user name or don't recall the password you used, see chapter :ref:`adminconsolepassword` for details on how to reset it. Enable the TeamDrive Hosting Service at System Boot ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the update was successful and the service is up and running, make sure it gets started automatically when the system reboots:: [root@hostserver ~]# chkconfig | grep td-hostserver td-hostserver 0:off 1:off 2:off 3:off 4:off 5:off 6:off [root@hostserver ~]# chkconfig td-hostserver on [root@hostserver ~]# chkconfig | grep td-hostserver td-hostserver 0:off 1:off 2:on 3:on 4:on 5:on 6:off [root@hostserver ~]# chkconfig | grep httpd httpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off Migrating an Older Host Server Version to a 3.0.013 Instance ------------------------------------------------------------ Perform the installation of the 3.0.013 Host Server instance from scratch as outlined in the Installation Guide or start off booting and setting up the Host Server Virtual Appliance. For the setup and initial installation, the new instance can be started with a different IP address and host name. However, before starting the services and making the new Host Server available to the TeamDrive clients again, it **must** be accessible under the original hostname, e.g. "hostserver.yourdomain.com". Make sure to update your DNS records accordingly (a change of IP address is fine). Once the Software has been installed and configured, don't perform the steps to register the new Host Server with the Registration Server. Instead, first stop the Apache Server and Services on the old Host Server and make sure they are not yet running on the new instance, either. Copy over the MySQL databases from the old system, e.g. by creating a ``mysqldump`` as outlined in chapter :ref:`createmysqlbackup` and importing these databases into the new instance's MySQL Server. .. note:: Please ensure that the MySQL import is performed into a "clean" database and there are no remains of a previous installation (e.g. when using the Host Server Virtual Appliance, which ships with pre-populated databases). If the Host Server Database is located on an external MySQL host, it's sufficient to simply point the new Host Server instance to this server by providing the appropriate login credentials as explained in chapter :ref:`changingmysqlconnection`. (You likely have to create a new MySQL user account and grant privileges to allow incoming connections to these databases from the new Host Server). Next, the existing Space Volumes need to be migrated to the new Host Server instance. In case you're using NFS, simply update ``/etc/fstab`` on the new Host Server and mount the file system. Double check that the ``apache`` user has write permissions on this mount point. If you are using other means of storage that can be migrated from one host to another (e.g. iSCSI targets, virtual hard disk devices), take the appropriate actions to make them available to the new Host Server instance. Alternatively, create a backup of the Space Volumes and restore them on the new instance or use a tool like ``rsync`` to transfer the Space data files. .. note:: Ensure that the Space Volumes are mounted on the same paths as they were on the previous Host Server instance! It is essential to preserve the directory structure and mount points, as previous Host Server versions included the Space Volume name in the Space URLs that are used by the TeamDrive Clients. At this point, proceed with the update process as outlined in chapter :ref:`checkmysqlprivileges` and following. For performing the table conversions, the ``teamdrive`` MySQL user needs to to have privileges on both the existing as well as the new MySQL databases required for the TeamDrive Hosting Service. As a final step, make sure to update your DNS so the new Host Server is reachable under the old host name.