Importing User Accounts via CSV Files

Instead of manually creating individual user accounts via the Administration Console as described in chapter Adding Users Manually, it is possible to import multiple user accounts into the Registration Server’s database from a file containing the account information as a CSV (comma-separated values) list.

The CSV import can be enabled and configured via the Provider Settings located in the CSVIMPORT group. See the TeamDrive Registration Server Reference Guide for more details on these settings.

There are two options on how to upload the CSV file:

  • Upload the import file to a “hot folder”, which can be configured using the CSV_IMPORT_DIR setting. The upload can be performed by an external system, e.g. via rsync, scp or sftp, or via a local cron job (e.g. using wget or any other tool to pull the file from a remote location).
  • Upload the import file manually via the Registration Server Administration Console.

The data import via a hot folder is performed by an Auto Task which polls a directory for files containing CSV data at a defined interval (once every hour by default). See “CSV Import”-Task for details.

CSV File Structure

The CSV file must contain the following fields, separated by comma or semicolon:

username:Unique user name which will be used to create the user account. If set, this value must not change, as it is the unique refererence to identify an existing user account. Alternatively, it is possible to use “magic usernames” which are generated by the Registration Server automatically. In this case, the user uses his email address to log into his account. To use magic usernames, leave the username field blank and select another field instead of the user name in the provider setting CSVIMPORT/CSV_IDENTITY_COLUMN to clearly identify a user. You also need to set the provider setting CLIENT/USE_EMAIL_AS_REFERENCE to True, to enable using the email address to identify the account.
email:Email address of the user. This value may change, if the user’s email address has changed. The email address must be unique, if the Provider Setting CLIENT/USE_EMAIL_AS_REFERENCE is set to True or if the Registration Server Setting UserEmailUnique is set to True.
password:A password for the user. If empty, the user can define a password during the initial registration process as described in the Reference Guide. Changes to an existing user password will be ignored.
distributor:The Provider Code of the user’s Provider.
reference:A free text field which can be used to assign an external reference id (e.g. a cost center)
department:A free text field which can be used to set a department reference for the user. May be changed, if the provider setting CSVIMPORT/CSV_ALLOW_SET_DEPARTMENT is set to True (which is the default).
language:Language code of the user. This value may change.
authid:This field is optional. It can contain a unique ID that can be used as an alternative reference.

Example file structure (without an authid field):

username;email;password;distributor;reference;department;language
TeamDriveUser1;TD_User1@yourdomain.com;;password1;1234;Int1;EN
TeamDriveUser2;TD_User2@yourdomain.com;;password2;1342;Int2;DE
TeamDriveUser3;TD_User3@yourdomain.com;;password3;1452;Int2;DE

Note

Note that even though the CSV file contains a field to define a user’s provider code, this value is currently not used. Instead, the provider code is defined by the user that uploads the CSV file via the Administration Console or by the directory the file is located in. If you need to upload user accounts for multiple providers, create one file per provider account and upload them separately.

Enable CSV Upload via the Administration Console

You can enable the CSV import functionality via the Administration Console by adding the provider setting CSVIMPORT/CSV_IMPORT_ACTIVE and setting it to True via the Administration Console.

Additionally, the Auto Task “CSV Import” must be enabled, by setting its status to Enabled via the Administration Console (Server Management -> Manage Auto Tasks). Change the frequency to the desired time interval in which this Auto Task should be executed. For testing purposes, it might make sense to set it to a very short frequency (e.g. 1 minute).

Note

The import of a single user requires about 1 second. Make sure that the Auto Task’s Frequency allows enough time for the currently running job to finish before another task is started.

If your list of users does not change frequently, it might make sense to keep the Auto Task disabled and only activate it temporarily, after a new CSV file has been uploaded.

In this mode, the CSV files and result logs are stored in the Registration Server’s database and can be managed via the Administration Console.

To upload your CSV user data manually via the Administration Console, follow the instructions outlined in chapter Adding Users via CSV File Import.

Uploading CSV Files to a Directory

As an alternative to the manual upload via the Administration Console, you can define a directory on the Registration Server that will be scanned for new CSV files periodically.

This so called “hot folder” allows for an automated process to create or disable user accounts by uploading updated CSV files using tools like scp, sftp or rsync from another server. An example directory structure can be created in /var/tmp using the following command (replace XXXX with your provider code):

[root@regserver ~]# install -m 700 -d /var/tmp/csvimport/XXXX/error
[root@regserver ~]# install -m 700 -d /var/tmp/csvimport/XXXX/success
[root@regserver ~]# chown -R apache:apache /var/tmp/csvimport
[root@regserver ~]# tree /var/tmp/csvimport
csvimport/
`-- XXXX
    |-- error
    `-- success

3 directories, 0 files

In addition to activating CSV import via the CSV_IMPORT_ACTIVE setting as outlined above, you need to add and configure the following Provider Settings via the Administration Console:

CSVIMPORT/CSV_USE_FILESYSTEM:
Set this option to True to use a directory on the Registration Server for uploading user account information in a CSV file. You should only enable this setting after you created the required directory structure and updated the following settings accordingly. Changing this setting to True will automatically add the following settings to your Provider Settings.
CSVIMPORT/CSV_UPLOAD_DIR:
This directory is the location for uploading new CSV files that should be processed by the import script (e.g. /var/tmp/csvimport/XXXX/ in the example above). The name must end with a slash. Each provider must to use a different directory. It must be readable and writable for the Linux user that the CSV import job is running under (apache by default).
CSVIMPORT/CSV_SUCCESS_DIR:
This directory contains the log files for successful CSV imports (e.g. /var/tmp/csvimport/XXXX/success in the example above). The name must end with a slash. It must be readable and writable for the Linux user that the CSV import job is running under (apache by default).
CSVIMPORT/CSV_ERROR_DIR:
This directory contains the log files for failed CSV imports (e.g. /var/tmp/csvimport/XXXX/error in the example above). The name must end with a slash. It must be readable and writable for the Linux user that the CSV import job is running under (apache by default).

Now copy the CSV file containing your user accounts into the directory defined in CSV_UPLOAD_DIR (e.g. /var/tmp/csvupload/XXXX in the example above).

Note

Please ensure that the file’s ownership and permissions are set correctly, so that the Auto Task can delete the file after it has been processed.

After the Auto Task has been executed, the file will be imported into the database and processed. Afterwards, you can review the processing status via the Administration Console (Manage Servers -> CSV user imports).

Customizing the CSV Import

The CSV import can be further customized using the following Provider settings:

CSVIMPORT/CSV_ALLOW_SET_DEPARTMENT:
Set this to False, if the department may not be changed by the CSV import of an existing user account.
CSVIMPORT/CSV_IDENTITY_COLUMN:
This setting specifies which field in the CSV file will be used to identify users during the CSV import (options are: username, email, reference and authid)
CSVIMPORT/DISABLE_MISSING_CSV_USERS:
If set to True, any user account not present in the CSV file will be disabled on the Registration Server. In this mode, your CSV user file always needs to contain all active user accounts.
CLIENT/USE_EMAIL_AS_REFERENCE:

Set this setting to True if you wish to use the user’s email address to reference your users between your system and the Registration Server. A few things need to be changed:

  1. The username fields in the CSV file must be left empty
  2. The Registration Server will generate a unique username for new users. The generated usernames will continue to be used internally, because the system is based upon using usernames. You will still see them in the Administration Console of the Registration Server and the Host Server Administraion Console, however in the Client application (required version at least 3.0.7 (257) ) users will only see their email address. The CSV import will match the email address back to the usernames (for users that existed before the import). You have to make sure, that no duplicate email address will be created using the Admin Console.
  3. The Client Setting allow-email-login must be set to true, so that your users can login using their email addresses.