Importing User Accounts via CSV Files¶
Instead of manually creating individual user accounts via the Administration Console as described in chapter Adding Users Manually, it is possible to import multiple user accounts into the Registration Server’s database from a file containing the account information as a CSV (comma-separated values) list.
The CSV import can be enabled and configured via the Provider Settings located
in the CSVIMPORT
group. See the TeamDrive Registration Server Reference
Guide for more details on these settings.
There are two options on how to upload the CSV file:
- Upload the import file to a “hot folder”, which can be configured using the
CSV_IMPORT_DIR
setting. The upload can be performed by an external system, e.g. viarsync
,scp
orsftp
, or via a local cron job (e.g. usingwget
or any other tool to pull the file from a remote location). - Upload the import file manually via the Registration Server Administration Console.
The data import via a hot folder is performed by an Auto Task which polls a directory for files containing CSV data at a defined interval (once every hour by default). See “CSV Import” Task for details.
CSV File Structure¶
A CSV import can be used to create new users and update existing user accounts.
When a user is created, the setting CLIENT/USER_IDENTIFICATION_METHOD
(see USER_IDENTIFICATION_METHOD) specifies
how a account will be identified by the user. In other words, the name used
during login. This can either by a username or an email address.
Your import file must conform to this specification.
If a user already exists, then the import can recognise this fact and update
a user record, instead of creating a new one. In this case the setting
CSVIMPORT/CSV_IDENTITY_COLUMN
(see :ref:``) specifies the import column
that uniquely identifies the user.
Note that the value of this column may not change, in order for the update
to work. Only set CSV_IDENTITY_COLUMN
to a value other than username
,
if you know that this field is never updated.
The CSV file must contain the following fields, separated by comma or semicolon:
username: | This is the a globally unique name for a user account. Usernames are unique
over all TeamDrive Registration Servers. A username must be specified
if USER_IDENTIFICATION_METHOD or CSV_IDENTITY_COLUMN is set
to username . This field may be omitted if USER_IDENTIFICATION_METHOD
is set to default or email . In this case you will create a user
account which is only identified by an email address. In order to do this,
CSV_IDENTITY_COLUMN must be set to a value other than username .
Once set, the username may not change.
Registration Server version 3.6 will not allow users to be created with usernames
that look like email addresses (that contain the “@” character). Such
usernames are still allowed as reference to users created by previous
versions of the Registration Server.
By default, this is also the CSV_IDENTITY_COLUMN column. |
---|---|
email: | Registration email address of the user. This value is not optional.
The Registration Server ensures that the email is unique per Provider.
There may be additional uniqueness contraints imposed by the global
settings EmailGloballyUnique and UserEmailUnique . If the
Provider setting ISOLATED_EMAIL_SCOPE is set to True then the global
settings are ignored (see ISOLATED_EMAIL_SCOPE). |
password: | A password for the user. If empty, the user can define a password during the initial registration process as described in the Reference Guide. Changes to an existing user password will be ignored. |
distributor: | The 4 letter Provider Code of the user’s Provider (optional, see note below). |
reference: | A free text field which can be used to assign an external reference ID (e.g.
a cost center). If CSV_IDENTITY_COLUMN is set to this column, then
CLIENT/EXT_USER_REFERENCE_UNIQUE should be set to ``True` for the
Provider. |
department: | A free text field which can be used to set a department reference for the
user. May be changed, if the provider setting
CSVIMPORT/CSV_ALLOW_SET_DEPARTMENT is set to True (which is the
default). If CSVIMPORT/DISABLE_MISSING_CSV_USERS is set to True
then this field must be identical for all records in the import file
(i.e. you must use one import file per Department). |
language: | Language code of the user. This value may change. |
authid: | This field is optional. It can contain a unique ID that can be used as an
alternative reference. If CSV_IDENTITY_COLUMN is set to this column,
then the value in this column will be used to identify the user on update
In this case, the value in this column may not change. This is the same
ID that is used by an external authentication service such as AD or LDAP.
As a result, if the column is used you must be certain that it conforms
to the usage of any existing or future external authentication service
(this is the value $ldap_user_id_attr referred to in
Configuring External Authentication using Microsoft Active Directory / LDAP). |
Example file structure (without an authid field):
username;email;password;distributor;reference;department;language
TeamDriveUser1;TD_User1@yourdomain.com;password1;;1234;Int1;EN
TeamDriveUser2;TD_User2@yourdomain.com;password2;;1342;Int2;DE
TeamDriveUser3;TD_User3@yourdomain.com;password3;;1452;Int2;DE
Note
Note that even though the CSV file contains a field to define a user’s provider code, this value is currently not used. Instead, the provider code is defined by the user that uploads the CSV file via the Administration Console or by the directory the file is located in. If you need to upload user accounts for multiple providers, create one file per provider account and upload them separately.
Enable CSV Upload via the Administration Console¶
You can enable the CSV import functionality via the Administration Console by
adding the provider setting CSVIMPORT/CSV_IMPORT_ACTIVE
and setting it to
True
via the Administration Console.
Additionally, the Auto Task “CSV Import” must be enabled, by setting its
status to Enabled
via the Administration Console (Server Management ->
Manage Auto Tasks). Change the frequency to the desired time interval in
which this Auto Task should be executed. For testing purposes, it might make
sense to set it to a very short frequency (e.g. 1 minute).
Note
The import of a single user requires about 1 second. Make sure that the Auto Task’s Frequency allows enough time for the currently running job to finish before another task is started.
If your list of users does not change frequently, it might make sense to keep the Auto Task disabled and only activate it temporarily, after a new CSV file has been uploaded.
In this mode, the CSV files and result logs are stored in the Registration Server’s database and can be managed via the Administration Console.
To upload your CSV user data manually via the Administration Console, follow the instructions outlined in chapter Adding Users via CSV File Import.
Uploading CSV Files to a Directory¶
As an alternative to the manual upload via the Administration Console, you can define a directory on the Registration Server that will be scanned for new CSV files periodically.
This so called “hot folder” allows for an automated process to create or
disable user accounts by uploading updated CSV files using tools like scp
,
sftp
or rsync
from another server. An example directory structure can
be created in /var/tmp
using the following command (replace XXXX
with
your provider code):
[root@regserver ~]# install -m 700 -d /var/tmp/csvimport/XXXX/error
[root@regserver ~]# install -m 700 -d /var/tmp/csvimport/XXXX/success
[root@regserver ~]# chown -R apache:apache /var/tmp/csvimport
[root@regserver ~]# tree /var/tmp/csvimport
csvimport/
`-- XXXX
|-- error
`-- success
3 directories, 0 files
In addition to activating CSV import via the CSV_IMPORT_ACTIVE
setting as
outlined above, you need to add and configure the following Provider Settings
via the Administration Console:
- CSVIMPORT/CSV_USE_FILESYSTEM:
- Set this option to
True
to use a directory on the Registration Server for uploading user account information in a CSV file. You should only enable this setting after you created the required directory structure and updated the following settings accordingly. Changing this setting toTrue
will automatically add the following settings to your Provider Settings. - CSVIMPORT/CSV_UPLOAD_DIR:
- This directory is the location for uploading new CSV files that should be
processed by the import script (e.g.
/var/tmp/csvimport/XXXX/
in the example above). The name must end with a slash. Each provider must to use a different directory. It must be readable and writable for the Linux user that the CSV import job is running under (apache
by default). - CSVIMPORT/CSV_SUCCESS_DIR:
- This directory contains the log files for successful CSV imports (e.g.
/var/tmp/csvimport/XXXX/success
in the example above). The name must end with a slash. It must be readable and writable for the Linux user that the CSV import job is running under (apache
by default). - CSVIMPORT/CSV_ERROR_DIR:
- This directory contains the log files for failed CSV imports (e.g.
/var/tmp/csvimport/XXXX/error
in the example above). The name must end with a slash. It must be readable and writable for the Linux user that the CSV import job is running under (apache
by default).
Now copy the CSV file containing your user accounts into the directory defined
in CSV_UPLOAD_DIR
(e.g. /var/tmp/csvupload/XXXX
in the example above).
Note
Please ensure that the file’s ownership and permissions are set correctly, so that the Auto Task can delete the file after it has been processed.
After the Auto Task has been executed, the file will be imported into the database and processed. Afterwards, you can review the processing status via the Administration Console (Manage Servers -> CSV user imports).
Customizing the CSV Import¶
The CSV import can be further customized using the following Provider settings:
- CSVIMPORT/CSV_ALLOW_SET_DEPARTMENT:
- Set this to
False
, if the department may not be changed by the CSV import of an existing user account. - CSVIMPORT/CSV_IDENTITY_COLUMN:
- This setting specifies which field in the CSV file will be used to identify
users during the CSV import (options are:
username
,email
,reference
andauthid
) - CSVIMPORT/DISABLE_MISSING_CSV_USERS:
- If set to
True
, any user account not present in the CSV file will be disabled on the Registration Server. In this mode, your CSV user file always needs to contain all active user accounts. In addition, an import file may only contain the users of one Department, if the Department field is used. - CLIENT/USER_IDENTIFICATION_METHOD:
- This setting determines how a user account is identified by the user. In other words, what name is used on login to TeamDrive. See USER_IDENTIFICATION_METHOD for more details.
- CLIENT/EXT_USER_REFERENCE_UNIQUE:
- If you wish to use the
reference
columns to identify user when updating user accounts during import, then this value must be set toTrue
. See EXT_USER_REFERENCE_UNIQUE for more details).