Troubleshooting

List of relevant configuration files

/etc/httpd/conf.d/td-hostserver.httpd.conf:

The configuration file that loads and enables the TeamDrive Host Server-specific modules for the the Apache http Server:

  • mod_pspace.so: this Apache module provides the actual Host Server functionality by accepting incoming data from the TeamDrive clients as well as delivering data to other clients upon request.
  • mod_yvva.so: this Apache module is responsible for providing the web-based Host Server Administration Console as well as the Host Server API interface.
/etc/logrotate.d/td-hostserver:
This file configures how the log files belonging to the TeamDrive Host Service are being rotated. See the logrotate(8) manual page for details.
/etc/td-hosting.conf:
This file defines how the td-hostserver background service is started using the yvvad daemon.
/etc/td-hostserver.my.cnf:
This configuration file defines the MySQL credentials used to access the pspace MySQL database. It is read by the Apache modules mod_yvva and mod_pspace as well as the yvvad daemon that runs the td-hostserver background tasks and the yvva command line client.
/etc/yvva.conf:
This configuration file contains configuration settings specific to the Yvva Runtime Environment that are shared by all Yvva components, namely the mod_yyva Apache module, the yvvad daemon and the yvva command line shell.
/etc/tshs.conf:
This configuration file defines a number of maintenance tasks performed by the tshs background service.

List of relevant log files

In order to debug and analyse problems with the Host Server configuration, there are several log files that you should consult:

/var/log/mod_yvva.log:

The log file for the Yvva Application Server module which provides the web-based Host Server Administration Console and API. Consult this log file when you have issues with associating the Host Server with the Registration Server, errors when issuing API requests or problems with the Administration Console. You can increase the amount of logging by changing the Yvva setting log-level from error to trace or debug in /etc/httpd/conf.d/td-hostserver.httpd.conf:

<Location /yvva>
  SetHandler yvva-handler
  YvvaSet root-path=/opt/teamdrive/hostserver
  YvvaSet mysql-cnf-file=/etc/td-hostserver.my.cnf
  YvvaSet log-level=error
</Location>

After changing these values, you need to restart the Apache http Server service using service httpd restart.

/var/log/td-hostserver.log:
The log file for the td-hostserver background task. Check this one to verify that background tasks are being processed without errors. The log file location can be configured by changing the file name passed to the log-file option in the configuration file /etc/td-hosting.conf. The log level can be increased by changing the default value error for the log-level option to trace or debug. Changing these values requires a restart of the td-hostserver background process using service td-hostserver restart.
/var/log/mod_pspace.log:
This log file contains error messages related to the mod_pspace Apache module, particularly when using TSHS. It needs to be writable by the user that the Apache http Server runs under (apache by default). The log file location is configured by the server setting ModuleLogFile and the amount of logging can be changed by adjusting the server setting ModuleLogLevel via the Host Server Administration Console. The value defines the maximum level of logging of messages logged: 1 = Protocol, 2 = Error, 3 = Warning, 4 = Trace, 5 = Debug. Changing these value requires restarting the Apache http Server.
/var/log/httpd/:
The Apache httpd Server’s log files (e.g. error_log) might also contain additional relevant error messages (e.g. from mod_pspace) that should be checked. The amount of logging is affected by the ModuleLogLevel setting described above.
/var/log/tshs.log:
This log file contains errors and other messages generated by the tshs background service. The log file location and amount of output are defined in file /etc/tshs.conf, via the options log-file and log-level. Possible values in the order of verbosity are protocol, error, warning, trace, debug. The default is warning.
/var/log/s3d.log:
This log file is written by the TeamDrive S3 daemon s3d and provides log messages and errors specific to the s3d background service. The log file location is defined in the init script /etc/init.d/s3d.

Common errors

Errors When Registering the Host Server

If the Host Server Registration fails, check /var/log/mod_yvva.log on the Host Server as well as /var/log/pbt_mod.trace on the Registration Server for hints. See the Troubleshooting chapter in the Registration Server Installation Manual for details.

MySQL Errors When Upgrading From an Older Host Server Version

If you observe Access denied or Unknown database errors from the MySQL server like the following ones after starting the updated TeamDrive Host Server using an older MySQL table structure:

140618 10:56:37 [Note] DROP DATABASE pbpg;
140618 10:56:37 [Error] -12036 (1044): Access denied for user
'teamdrive'@'localhost' to database 'hostapilog'
140618 10:56:37 [Error] "p1setup.pbt" P1Setup:upgradeSettings(328)
140618 10:56:37 [Error] "p1setup.pbt" P1Setup:setupDatabase(14)
140618 10:56:37 [Error] "p1setup.pbt" (506)

Unknown database:

140618 10:24:31 [Error] -12036 (1049): Unknown database 'hostapilog'
140618 10:24:31 [Error] "p1setup.pbt" P1Setup:upgradeSettings(328)
140618 10:24:31 [Error] "p1setup.pbt" P1Setup:setupDatabase(14)
140618 10:24:31 [Error] "p1setup.pbt" (506)
140618 10:24:31 [Error] "p1_shared.pbt" (2)

Double check that the hostapilog database actually exists and that the teamdrive user has the required privileges to access it.

Create the database using CREATE DATABASE hostapilog; and grant the required privileges using GRANT ALL PRIVILEGES ON `hostapilog`.* TO 'teamdrive'@'localhost';. Restart the TeamDrive Service again using service td-hostserver restart, it should now conclude the schema conversion.

If you observe a Can't connect to local MySQL server error like the following one in /var/log/httpd/error_log:

[Thu Jun 19 16:20:50 2014] [notice] mod_pspace 1.5.04 Loaded; Build Jun 19
2014 13:24:58;Crash-Reporting-Disabled
[Thu Jun 19 16:20:50 2014] [error] Failed to boot Admin API: MySQL 2002:
Can't connect to local MySQL server through socket
'/var/lib/mysql/mysql.sock' (2)

or in /var/log/td-hostserver.log:

140619 16:42:06 [Error] -12036 (2002): Can't connect to local MySQL server
through socket '/var/lib/mysql/mysql.sock' (2)

Double check that the MySQL Server is up and running and that the socket configuration setting in the [mysqld] group in /etc/my.cnf matches the one in /etc/td-hostserver.my.cnf.

The default value is /var/lib/mysql/mysql.sock. If the value in my.cnf is different, e.g. /tmp/mysql.sock, we suggest to revert back to the default value there instead of changing it in td-hostserver.my.cnf (unless you have an explicit reason to change the default socket path, of course). Restart MySQL and the TeamDrive Hosting Services after changing this value.

Admin Console: Clicking on “Host” Results in a “500 Internal Server Error”

If you observe an error message like the following when clicking on Host in the Host Server Administration Console:

500 Internal Server Error
ERROR -1: TshsMain: void CSDBConn::connect(CSDB.cc:1116) MySQL 1044: Access
denied for user 'teamdrive'@'localhost' to database 'tshs_admin'

Or:

500 Internal Server Error
ERROR -1: TshsMain: void CSDBConn::connect(CSDB.cc:1116) MySQL 1049: Unknown
database 'tshs_admin'

You likely changed the setting TSHSEnabled to True, but did not configure the MySQL settings for accessing the tshs_admin database in /etc/td-hostserver.my.cnf.

If you changed the setting by accident, simply set TSHSEnabled back to False.

Otherwise, consult the chapter TeamDrive Scalable Hosting Storage in the Team Drive Host Server Administration Guide for details on how to enable and configure TSHS properly.

“Duplicate key” MySQL errors when updating the database

If you observe “Duplicate key” errors in the Traffic or Owner tables when upgrading these to the latest schema version, you first need to manually remove the duplicates via the MySQL client or another tool like MySQL Workbench. Older versions of the Host Server database schema did not have UNIQUE constraints on some columns, which caused the creation of duplicate entries. For the Traffic table, this usually only affects older traffic accounting information that can safely be removed.

Duplicates in the Owner table are likely caused by user names or email addresses that refer to the same user account, but using different capitalization. In this case it helps to cross-reference the affected users with their information in the Registration Server Database - likely one of these accounts has not been actively used and can be deleted. Please contact support@teamdrive.net if you need assistance in resolving these conflicts.

Admin API Error: MySQL 1040: Too many connections

On a busy server, you might observe one of the following error messages in the Apache http Server’s error log file from time to time:

[error] Failed to boot Admin API: MySQL 1040: Too many connections
[error] [client xxx.xxx.xxx.xxx] (500)Unknown error 500: Admin API Error:
MySQL 1040: Too many connections

In /var/log/mod_yvva.log you might observe a similar error:

141107 12:11:44 [Error] -12036 (1040): Too many connections
141107 12:11:44 [Error] "startup.yv" (80)

This error indicates that the number of child processes spawned by the Apache http Server (e.g. when many TeamDrive Clients attempt to connect to the Host Server concurrently), causes the MySQL Server to run out of threads for handling the incoming database connections.

By default, the MySQL Server is configured to accept 151 concurrent connections. Each Apache child process can establish up to two MySQL connections (one for mod_pspace and one for mod_yvva, depending on what kind of requests it needs to serve). Therefore, the maximum number of connections should be adjusted to be at least 1.5 times the maximum number of child processes spawned by the Apache http Server (defined by the MaxClients directive in the Apache http Server configuration file /etc/httpd/conf/httpd.conf).

The value can be changed by adding the system variable max_connections to the [mysqld] configuration group in the MySQL Server configuration file /etc/my.cnf, e.g.:

[mysqld]
datadir=/var/lib/mysql
max_allowed_packet=4M
max_connections=350
socket=/var/lib/mysql/mysql.sock
user=mysql

You need to either restart the MySQL server in order to apply this change, or change the value at run-time, by running the following SQL statement as the MySQL root user:

mysql> SET GLOBAL max_connections=350;

Keep in mind that increasing the maximum number of connections also increases the memory requirements of the MySQL Server. For more details, please consult the MySQL Server and Apache http Server documentation: