Release Notes - Version 3.0.013

Host Server Version 3.0.013 is the next major release following after version 3.0.011 (Version 3.0.012 was an internal release that has not been published).

Version 3.0.013 contains the following features and notable differences to version 3.0.011:

  • The TeamDrive Host Server installation can now be performed via RPM on Red Hat Enterprise Linux 6 and derivative distributions, which significantly improves the installation procedure and the process of applying updates.
  • The initial setup and registration of a Host Server is now fully web-based. It’s no longer necessary to provide a hosting.txt or properties file. Instead, all the required information can be entered in a web form.
  • The entire Host Server configuration is now stored in the MySQL database. This includes configuration settings for S3 daemon and TSHS.
  • The web-based TeamDrive Hosting Service Administration Console has been improved significantly, by simplifying the work flows for common administration tasks and fixing several usability issues.
  • TSHS, the TeamDrive Scalable Hosting Storage and the TeamDrive S3 Daemon provide additional scalability options to expand the storage capabilities of a TeamDrive Hosting Service.
  • It’s now possible to generate a monthly report that contains detailed statistics about all existing Depots and Spaces within these depots, including the monthly traffic and disk usage.
  • The Host Server no longer depends on the PrimeBase Application Environment. Instead, it now uses the Yvva Runtime Environment, which replaces the following components:
    • mod_yvva replaces mod_pbas for providing the web-based Administration Console and API. The stand-alone pbas instance is no longer required. As a consequence, the pbur MySQL database which was used by PBAS to manage user accounts and privileges is no longer required and has been removed.
    • yvvad replaces pbac for running background tasks. The former p1_autotask background task PBAC instance is now provided by the service td-hostserver, which uses yvvad.
    • yvva replaces pbac for command line operations that involve executing PBT code on the shell.
  • The installation location of the TeamDrive PBT code has been changed from /home/teamdrive/pbas to /opt/teamdrive/hostserver/.
  • The sakgen binary that used to be installed in /home/teamdrive/sakh is no longer required. Instead, the functionality to encrypt Space Depot access keys is now provided by the tshs binary.
  • All TeamDrive Host Server processes now run under the user ID used by the Apache http Server (apache). A dedicated teamdrive user account is no longer required.
  • By default, the MySQL databases are now installed in the default location /var/lib/mysql instead of /spacedb, which made it difficult to enable SELinux on the MySQL instance.
  • For security reasons, the MySQL credentials required for accessing the MySQL Database are no longer stored in the default MySQL configuration file /etc/my.cnf. Instead, the [p1db] options group has now been moved into a dedicated configuration file /etc/td-hostserver.my.cnf, only readable by the apache user.
  • The Apache httpd Server configuration file has been renamed from teamdrive.conf to td-hostserver.httpd.conf.
  • The overall robustness of the TeamDrive Host Server has been improved by issuing more meaningful error messages and performing more safety and consistency checks.
  • Each Space Volume now contains a file teamdrive-volume-id that contains a unique global volume ID, to ensure that multiple volumes are mounted to the correct location.

Change Log - Version 3.0.013

3.0.013.16 (YYYY-MM-DD)

3.0.013.15 (2015-08-17)

  • S3: Fixed bug with high IO, upload could not proceed and other uploads will be blocked. (HOSTSERVER-529)

3.0.013.14 (2015-06-04)

  • S3: Fixed bug in parsing S3 access log entries for traffic calculation (resolves Error getting spaceid errors in td-hostserver.log). Additionally, the S3 log analyser script now only downloads and processes objects from the log bucket that contain the string access_log-. (HOSTSERVER-500)
  • mod_pspace: Added support for calculating traffic from S3-compatible object stores that do not support access logging via log buckets in the way that Amazon S3 does it. Now, if a redirect to S3 is performed and S3LogBucketName has not been specified, the request length will be logged as bytes sent. (HOSTSERVER-499)
  • s3d: The S3 daemon has now been split into two processes, a worker process and a watchdog process. If the worker process dies, the watchdog will restart it. Killing the watchdog process will also kill the worker process. The watchdog will always try to restart the worker, but depending on the frequency with which the worker is dying the watchdog will wait before trying to restart it. The minimum wait is 3 seconds, the maximum is 30 minutes. (HOSTSERVER-508)

3.0.013.13 (2015-05-11)

  • mod_pspace/s3d: Added workaround to handle a deviation in the Ceph 0.8 Object Store S3 API: the “list multipart upload parts” API request returns ListMultipartUploadResult instead of listpartsresult (see BUG#11494 in the Ceph bug tracker for details). (HOSTSERVER-484)
  • mod_pspace: Added missing call to s3d_delete() when an “Upload to file that has already been transfered to S3” is detected. Due to the missing call, Clients could end up in an endless loop, showing a “wrong md5” error in the log file. (TDCLIENT-2045)
  • mod_pspace: Added new module option watched_space_id that can be used to trace Client accesses to a specific Space for debugging purposes. See Tracing Client Accesses to a Single Space for details. (HOSTSERVER-486)

3.0.013.12 (2015-04-14)

  • s3d: Uploading the last.log file failed with a checksum error if the log was written to before the upload was complete. s3d now only transfers the data size used when calculating the checksum. This will allow the last.log file to grow while being uploaded to S3. (HOSTSERVER-474)
  • s3d: Fixed unsafe object references during multi-part uploads which may have lead to s3d crashes. (HOSTSERVER-454)
  • Installation: The td-hostserver RPM package will no longer reset the permissions and ownerships of the /spacedata and /spacedata/vol01 directories to 700 and apache:apache during an update, if they had been changed by the administrator after the initial installation. Depending on how the Space Volume is mounted, the RPM installation could fail with an error like error: unpacking of archive failed on file /spacedata. A new installation will still create the directories using these permissions/ownerships by default. (HOSTSERVER-401)
  • Host Server: Converted the type of the StatisticRest setting from INT to DATE, to avoid an error that could occur when updating from very old Host Server Versions (the resetTraffic() auto task failed with an Invalid integer literal error). This also fixes a potential issue that could result in the reset routine being run multiple times on the day the traffic is reset. (HOSTSERVER-478)
  • Documentation: Fixed link structure in the HTML documentation so that clicking Next and Previous within a document works as expected. (HOSTSERVER-471)

3.0.013.11 (2015-03-30)

  • Administration Console: Updated logo and favicon.
  • Host Server: Updated some error messages by replacing “Repository” with “Depot”. Ensure that a Space Depot that has been marked as “Deleted” no longer allows the creation of new Spaces. (HOSTSERVER-456)
  • mod_pspace: Reduced logging of errors by only logging Client accesses to deleted Spaces as an error if the Space status is zero. (HOSTSERVER-449)
  • mod_pspace: Fixed a crashing bug that could occur in rare situations. (HOSTSERVER-457)
  • s3d: Fix unsafe access to the thread pool that may have caused s3d to crash in certain situations. (HOSTSERVER-454)
  • s3d: Fixed a problem that caused a crash if a multipart upload was interrupted before completion and then restarted again. The parts list could have holes in it for the parts that were successfully uploaded in the first try.
  • Documentation: Added section that instructs the user to perform a yum update after installing the VM image. Reformatted the 3.0.013 release notes and replaced the table with regular sections for improved readability.
  • Documentation: Added Failover and Scalability chapter to the Administration Guide, added description of the startup sequence/dependencies to the Installation Guides. (HOSTSERVER-431)

3.0.013.10 (2015-01-26)

  • s3d: Fixed a problem that caused a crash from time to time. The crash would occur if a request for an object’s header timed out or was interrupted.
  • Host Server: Fixed bug in the calculation of DiskUsed for Space Volumes that did not contain any Spaces. (HOSTSERVER-452)
  • Administration Console: The Volume repair button now only appears if a repair is actually required (previously it appeared whenever there was an error on the volume).
  • Installation: added a new RPM package td-hostserver-doc-html that contains the Host Server documentation in HTML format, installed in the Host Server’s Apache document root /var/www/html/td-hostserver-doc/. Access to the documentation can be restricted by editing /etc/httpd/conf.d/td-hostserver-doc.httpd.conf. (HOSTSERVER-450)
  • Installation: fixed bug in upgrading from older versions and the hostapilog database did not get created. (HOSTSERVER-446)

3.0.013.9 (2015-01-14)

  • mod_pspace/s3d: fixed unexpected object "vol01/..." starting with 'vol' was found in the bucket... error , which prevented the Apache module from starting. This error could occur after updating from a previous version if S3 was already enabled, and the old object format (prefixed by volume name) was used on an S3 compatible object store. (HOSTSERVER-447)

3.0.013.8 (2015-01-13)

  • API: Added missing activatedepot API command and added new tag <changeinfo> to add a free form comment to the change history of the following API commands: activatedepot, assignusertodepot, createdepotwithoutuser, deactivatedepot, deletedepot. Updated API version to 3.0.004. (HOSTSERVER-337)
  • Installation: fixed typo in the installation script that adds the RewriteRules to ssl.conf. Added RewriteRule in preparation for accepting Client requests for Space data via SSL/TLS (not supported yet).
  • Installation: the binary tarball distribution now includes debug versions of the Host Server binaries (s3d-debug and tshs-debug) and Apache module (mod_pspace-debug.so, to better support analyzing possible crashing bugs. (HOSTSERVER-445)
  • Installation: fixed possible upgrade error from previous versions: moving the MySQL table pbpg.Keys to the pspace database failed if an empty pspace.Keys table already existed. (HOSTSERVER-441)

3.0.013.7 (2014-12-12)

  • Fixed error in creating an index during the initial MySQL table creation (HOSTSERVER-440)

3.0.013.6 (2014-12-09)

  • Installation: fixed possible upgrade error from 3.0.011 when the MySQL database pbpg still existed, but the Keys table was already moved to the pspace database (HOSTSERVER-427)
  • Fixed bug in which failed Auto Tasks were not executed anymore (HOSTSERVER-407)
  • mod_pspace: fixed possible crash when system settings are NULL (e.g. in an upgrade scenario from 3.0.011 to 3.0.013, when httpd was started before yvvad performed the required schema updates)
  • mod_pspace: Fixed possible “Admin API: AES decode error- curruption detected” error when updating from older versions (timing issues could result in the generation of duplicate private keys) (HOSTSERVER-420, HOSTSERVER-422)
  • Increased the size of the S3Options settings field from 200 to 2000 chars, to accommodate longer option strings required for certain OpenStack environments (HOSTSERVER-425)
  • Installation: updated RewriteRule sets in the httpd configuration files (removed obsolete /depot rule, HOSTSERVER-424)

3.0.013.5 (2014-09-26)

  • mod_pspace: fixed a Space corruption bug that could occur when updating from a previous Host Server version to version 3.0.013 and Space Volumes were using a non-standard naming scheme (not “volxxx”)
  • Admin Console: added “Repair” button that allows performing an automatic repair of Volumes affected by the corruption bug. Clients will be notified to perform a Space Restore operation on affected Spaces.

3.0.013.4 (2014-09-18)

  • Admin Console: fixed 404 errors when opening the Admin URL without a trailing slash (HOSTSERVER-398)
  • Admin Console: the input focus is now automatically set to the password field (HOSTSERVER-392)
  • s3d: Fixed bug in path deletion on S3: if the path ended with ‘/’ it wasn’t being deleted.
  • s3d: exceptions are now logged in /var/log/s3d.log

3.0.013.3 (2014-09-05)

  • mod_pspace: Replaced the previously used MD5 implementation with calls to the MD5 routines provided by OpenSSL (yielding a 70% performance improvement when calculating MD5 checksums on large files) (HOSTSERVER-355)
  • mod_pspace: consolidated brand-specific settings into one place and disabled multi-part uploads for OpenStack
  • mod_pspace: Fixed bug where failed uploads (resulting in MD5 checksum failures) would still be accounted for as bytes written in the Space usage statistics (HOSTSERVER-352)
  • Fixed autotask resetTraffic() to properly reset the traffic for Spaces that had the SPACE_TRAFFIC_FULL status flag enabled. (HOSTSERVER-353)
  • Installation: security enhancement: set ServerTokens to Prod and ServerSignature to Off in httpd.conf to disable displaying the Apache Server version and OS version in the HTTP headers and on error pages (HOSTSERVER-357)
  • mod_pspace: Disabled unnecessary buffering of files fetched from S3 object store and passed back to the client. (HOSTSERVER-356)
  • tshs: add-s3-host will ping the S3 service before actually adding the host details.
  • Admin Console: security enhancement: don’t display the version and build number on the login page and https redirection page (HOSTSERVER-359)
  • Security enhancement: disabled unneded HTTP methods in td-hostserver.httpd.conf (ony allow GET, POST, PUT, disable HEAD, OPTIONS, TRACE) (HOSTSERVER-361)
  • Virtual appliance security enhancement: set ServerTokens to Prod and ServerSignature to Off in httpd.conf to disable displaying the Apache Server version and OS version in the HTTP headers and on error pages (HOSTSERVER-357)

3.0.013.2 (2014-07-14)

  • To avoid confusion, the S3-related configuration option openStackAuthURL was renamed to openStackAuthPath

3.0.013.1 (2014-07-11)

  • Initial public release