Open Client Client-Library/C Reference Manual

Home

Open Client Client-Library/C Reference Manual

Chapter 2 Topics

Callbacks

Encryption callbacks

Adaptive Server and SQL Server versions 10.0 and later use an encrypted password handshake when the client requests it. Servers based on Open Server version 10.0 or later may also use this feature.

The client application must enable password encryption by calling ct_con_props and setting the CS_SEC_ENCRYPTION property.

Client-Library's default encryption handler performs the password encryption required by SQL Server and Adaptive Server. Simple client applications that connect to either of these servers do not need an encryption callback.

However, Client-Library applications that act as gateways to Adaptive Server or SQL Server need to handle password encryption explicitly. These applications must install an encryption callback routine that passes the server's encryption key to the client and returns the encrypted password back to the server. See "Password encryption in gateway applications".

In addition, Client-Library applications that connect to an Open Server using a customized password encryption technique must install an encryption callback routine to perform the required password encryption.

For an explanation of the handshaking process for password encryption, see "Security handshaking: encrypted password".

Do not confuse password encryption with data encryption. An encryption callback encrypts only passwords. Data encryption encrypts all commands and results sent over the connection and is performed by an external security service provider. See "Security features" for more information.

Defining an encryption callback

An encryption callback is defined as follows:

CS_RETCODE CS_PUBLIC 
 encrypt_cb(connection, pwd, pwdlen,
            key, keylen, buf, buflen, outlen)

CS_CONNECTION    *connection;
 CS_BYTE                  *pwd;
 CS_INT                      pwdlen;
 CS_BYTE                  *key;
 CS_INT                      keylen;
 CS_BYTE                  *buffer;
 CS_INT                      buflen;
 CS_INT                      *outlen;

where:

connection is a pointer to the CS_CONNECTION structure representing the connection that is logging in to the server.
pwd is a user password or a remote-server password to be encrypted. A user password matches the value of the CS_PASSWORD connection property. A remote-server password matches the string passed to ct_remote_pwd. The pwd string is not always null-terminated.
pwdlen is the length, in bytes, of the password.
key is the key that the encryption callback uses to encrypt the password. The encryption key is supplied by the remote server.
keylen is the length, in bytes, of the encryption key.
buffer is a pointer to a buffer. The encryption callback should place the encrypted password in this buffer. This buffer is allocated and freed by Client-Library. Its length is described by buflen.
buflen is the length, in bytes, of the *buffer data space.
outlen is a pointer to a CS_INT. The encryption callback must set *outlen to the length of the encrypted password in *buffer.

An encryption callback should return CS_SUCCEED to indicate that the password has been successfully encrypted. If the encryption callback returns a value other than CS_SUCCEED, Client-Library aborts the connection attempt, causing ct_connect to return CS_FAIL.

Password encryption in gateway applications

To handle encrypted passwords, a gateway application must:

Supply an encryption callback routine.
Call ct_callback to install the encryption callback either at the context level or for a specific connection.
Call ct_con_props to set the CS_SEC_ENCRYPTION property to CS_TRUE.

When the gateway calls ct_connect to connect to the remote server:

The remote server responds with an encryption key, causing Client-Library to trigger the encryption callback.
The encryption callback passes the key on to the gateway's client.
The gateway's client encrypts the password and returns it to the encryption callback.
The encryption callback places the encrypted password into *buffer, sets *outlen, and returns a status code to Client-Library.
- If the callback returns CS_SUCCEED, Client-Library sends the encrypted password to the remote server.
- If the callback returns CS_FAIL, Client-Library aborts the connection process, causing ct_connect to return CS_FAIL.

Client-Library calls the encryption once to encrypt the password defined by CS_PASSWORD, and one additional time for each remote server password defined by ct_remote_pwd.

A gateway to Adaptive Server or SQL Server must take special steps to make sure that encrypted remote passwords are handled correctly. The first time the encryption callback is called for a connect attempt, the gateway must perform the following actions:

Clear the default remote password with ct_remote_pwd(CS_CLEAR).
ct_connect creates a default remote password if the gateway has defined no remote passwords before calling ct_connect. The gateway must clear this default.
Challenge the gateway's client for encrypted local and remote passwords with srv_negotiate.
Call ct_remote_pwd once for each encrypted remote password.
Place the encrypted local password into *buffer and set *outlen to its length.
Return CS_SUCCEED if no error occurred.

Each subsequent invocation of the callback should return one of the encrypted remote passwords read from the gateway's client in response to the challenge.

A gateway forwards the encryption key and reads the client's response with Server-Library calls. See srv_negotiate in the Open Server Server-Library/C Reference Manual.

See "Choosing a network security mechanism" for more information.


	Expand Search