Home

	Expand Search

Installation and Administration Guide for Mainframe Connect for DB2 MVS-CICS

Chapter 9 Troubleshooting

Chapter 9

Troubleshooting

This chapter provides guidelines for what to do when a client application cannot access data on the mainframe. The guidelines include how to locate an error and cover the most frequent problems.

This chapter also includes operational and process flow overviews for Enterprise Connect. The information applies to both CICS LU 6.2 and TCP/IP environments, except where differentiated.

The sections in this chapter are:

• Verifying connectivity after installing MainframeConnect for DB2/MVS-CICS

• Where to start troubleshooting

• Common problems and suggested solutions

• Components

• Coordination

This chapter describes mainframe troubleshooting. For comparable DirectConnect information, see the Transaction Router Service User's Guide for DirectConnect and the Access Service User's Guide for DirectConnect for MVS.

Verifying connectivity after installing MainframeConnect for DB2/MVS-CICS

Be sure the AMD2 transaction processor and the client and server connections are configured correctly. You can use one of the standalone utilities (snaping or cicsping) shipped with Open ServerConnect to test the network connections between the workstation and mainframe. Use snaping for Open ServerConnect for LU 6.2 and cicsping for Open ServerConnect for TCP/IP.

These utilities do not require DirectConnect to be active. For details on using them, see the Installation Guide for DirectConnect for MVS.

If these utilities fail to operate correctly, the failure is caused by a configuration error or line outage problem. Based on the error message returned, you can narrow your search. Resolution may require coordination with the DirectConnect administrator to make changes to the configuration.

After snaping or cicsping runs successfully, be sure the DirectConnect administrator defined all the necessary remote procedure call (RPC) and connection information to DirectConnect. Be sure that DirectConnect is started. Use the guidelines in the following section for further problem determination.

Where to start troubleshooting

System administrators at the client, DirectConnect server and mainframe must check components systematically to locate the source of the problem. Check for problems in this sequence:

• Major outage, such as a CICS crash, or your personal computer being unplugged

• Client application

• Client LAN

• Client network setup

• DirectConnect server

• Connection from the client application to the DirectConnect server

• Connection from the DirectConnect server to the mainframe

• Configuration between the transaction processor and Open ServerConnect or MainframeConnect for DB2/MVS-CICS

For any of these problems, the appropriate system administrator should:

• Record specific information about the error message(s), including:

  • Error message number

  • Associated SNA sense codes, SNA Services error codes, or TCP/IP error codes

  • The time the error occurred

  • The client or user affected

• See the appropriate documentation, as needed. For example, if you need information about MainframeConnect for DB2/MVS-CICS errors, see Appendix A, "Error Messages"

• Perform the recommended action.

• Continue the process until the problem is resolved.

Common problems and suggested solutions

Problems often can be traced to configuration errors or network, line, modem, or adapter outages.

Configuration errors

Configuration errors are most often the cause of communications failure. To resolve these errors, you need the following information, which is created when the network is installed and successfully implemented:

• CICS table definitions for TCT, PCT, PPT, FCT, and RCT

• SNA/NCP definitions for the logical unit (LU) and associated Logmode table

• SDLC or Token Ring connection charts to the mainframe

• Named pipe and SPX charts to clients

• TCP/IP connection charts to clients

• For CICS LU 6.2, server SNA configuration

• For CICS TCP/IP, server TCP/IP configuration

• Sybase interface files for clients and DirectConnect servers

• Sybase security definitions, including client logins, connection groups, and transaction groups

Verify that this information is still the same as before the error occurred. If it is not the same, determine if a recent change is contributing to the problem. The following subsections explain the causes and actions for common configuration errors.

Cause

Any of the following:

• Mismatched LU definitions between SNA and server

• Mismatched modenames

• Incorrect SNA MODETAB and APPLID macros

• Incorrect CICS TCT definitions

Action

Correct the spelling.

Coordinate with the DirectConnect administrator to check connection and modename profiles using the snaping or cicsping utility shipped with the product.

Cause

Any of the following:

• CICS PCT or PPT entry error

• host security error

• Incorrect transaction ID in the DirectConnect RPC table

Action

Verify definitions.

Coordinate with the DirectConnect administrator for correct security and transaction ID setups.

Cause

Address incorrectly configured with NCP (assumes correct line or modem setup)

Action

Check both ends of the SDLC station or Token Ring address configuration.

Cause

Any of the following:

• SNA and DirectConnect LU definition errors

• SSCPID value in the local LU profile set incorrectly

Action

Use the SDLC trace and error log facilities to find the error.

Mainframe network operational failure

On the mainframe, there are two frequent causes of operational errors:

• The CICS or SNA operator varied the resource out of service.

• SNA placed the line, physical unit (PU), or LU into a non-operating (INOP) state because of a network outage.

In these cases, either:

• The DirectConnect administrator sees SNA software time-out and connection failure messages in trying to start up DirectConnect, or

• The requesting client sees a SNA software message indicating that the system could not start the RPC.

When you are contacted about such messages, reactivate the necessary mainframe resources.

Network session or line failures

Line, adapter, or modem outages result in error messages at both the SNA console and DirectConnect. DirectConnect records the message and, if possible, sends a similar error message to any affected clients.

Intermittent hardware errors and line degradation problems can disrupt processing, but can be difficult to find. You should check periodically for these types of problems. For example:

• To check for hardware errors, use the associated SNA error logs and report errors to IBM Service.

• To check for line degradation, use SNA to periodically report the SDLC line statistics. Examine the statistics for a significant number of re-transmissions or idle detect time-outs. Line degradation results in random SDLC line failures or very slow response to the client, even during a moderate processing load.

Components

MainframeConnect for DB2/MVS-CICS support consists of several components on the IBM mainframe and the DirectConnect platform, as Figure 9-1 shows. These components provide tracing and logging, which you can use to locate errors. This section includes examples of tracing and logging for CICS.

The following figure shows Open ServerConnect and DirectConnect support components in a CICS environment:

DirectConnect support

DirectConnect performs the following functions:

• Receives requests from client applications

• Converts the requests to the appropriate communications protocol call

• Sends the requests to the mainframe

Each TRS or access service includes a unique server name, which clients use to select a server for communication. Each TRS or access service has its own set of configuration information.

As shown in Figure 9-1 , DirectConnect uses four files:

• srv.log for logging TDS traffic between TRS and client workstations, and for recording errors

• ngtds for tracing Sybase TDS traffic between TRS and mainframe SNA

• servername.log for logging TDS traffic between the Access Service and client workstations, and for recording errors

• tds.trc for tracing Sybase TDS traffic between the Access Service and mainframe SNA

For more information, see the Transaction Router Service User's Guide for DirectConnect and the Access Service User's Guide for DirectConnect for MVS.

DirectConnect communications with MainframeConnect for DB2/MVS-CICS

DirectConnect depends on the communications support, supplied by the platform vendor, to communicate with MainframeConnect for DB2/MVS-CICS. Depending on the server-based communications software installed, DirectConnect uses SNA LU 6.2 or TCP/IP Uplink support.

SNA software uses the SNA trace file to record SDLC/SNA traffic between the server and mainframe. The vendor trace utility extracts this file.

For Windows NT platforms, the error log file records errors that SNA Server for Windows NT detects.

For CICS TCP/IP environments, third-party TCP/IP trace facilities provide a way of obtaining low-level TCP/IP level traces between the mainframe and DirectConnect. For AIX platforms, the error log file records errors that SNA Server/6000 detects. The IBM error log report utility extracts this information. Microsoft SNA Server for Windows NT also has a trace process.

Mainframe Communications with DirectConnect

Mainframe-based communications support provides the "transport" level of function. Depending on the mainframe communications software installed, Gateway-Library uses SNA/NCP or TCP/IP for MVS.

For CICS LU 6.2 environments, you can use the SNA General Trace Facility (GTF) files to trace SDLC/SNA traffic between Gateway-Library and the mainframe. The IBM TAP utility extracts this information.

For CICS TCP/IP environments, you can use the Netstat facility to check the status of TCP/IP connections, as well as to deactivate them if problems occur.

You can use the TCP/IP trace facility to trace traffic between DirectConnect and the mainframe.

The Sybase listener transaction provides the interface between CICS and MVS/TCP. The listener enables CICS applications to use the following TCP/IP communication facilities:

• The IBM-supplied TCP/IP Listener transaction to initiate transactions when operating with TCP/IP devices, such as DirectConnect

• The TCP/IP Sockets Interface provides CICS user exit traces, denoting each TCP call issued and the associated return call.

The Sybase listener TRACE parameter and Open ServerConnect DEBUGSW parameter settings provide CICS user exit traces, denoting each TCP call issued and the associated return call.

For CICS, Gateway-Library tracing stores information on the TDS traffic between the mainframe and server in the VSAM ESDS file, SYTDLOG1. This information includes any errors detected in the traffic.

Open Server's Trace flag will write Open Server TCP/IP Trace point to the CICS Message User log, as well as the CICS Aux trace facility.

Some TD calls fill up internal TDS buffers before sending them out to the network. For example, a TDSNDROW or TDSNDMSG call does not cause execution of a corresponding CICS EXEC SEND call, unless the TDS buffer becomes full.

Warning!

To avoid losing records, periodically archive or delete the trace records on SYTDLOG1. Trace records append to this file until it is full, then it rejects any further records.

MainframeConnect for DB2/MVS-CICS

MainframeConnect for DB2/MVS-CICS provides the AMD2 transaction, which automatically processes client SQL language requests using the DB2 dynamic SQL facilities.

Using this product, client applications can communicate directly with DirectConnect or with another server that communicates with DirectConnect, such as Adaptive Server or ASE/CIS (as shown in the following figure):

MainframeConnect for DB2/MVS-CICS has a trace flag that will write out MFC trace points to a CICS Temp Storage Queue name "CE + 'first 6 bytes of the user id'."

For MainframeConnect for DB2/MVS-CICS, you can also use the standard DirectConnect tracing support. See Figure 9-1 .

You can determine the errors logged. Enter changes in SOURCE library member AMD2MAMX, as detailed in MainframeConnect for DB2/MVS-CICS installation instructions. For example, to indicate that you want a message logged, set LOG=Y.

SYRTMAMX TYPE=ENTRY, MSGNR=32000, SQLCODE=0, LEVEL=11, STATEC=0, STATESC=0, LOG=Y, SQLFATAL=N

Warning!

If a connection fails, the error log is the only method of tracking an error. To prevent the log from running out of room, initialize the log file regularly.

In a CICS environment, MainframeConnect for DB2/MVS-CICS provides the AMD2LOG and AMD2LOGO files. The AMD2 transaction writes errors to AMD2LOG, including task ID, term ID, timestamp, and the unique message code for each error. AMD2 also returns an error message to the client.

Under CICS, if AMD2LOG is full or no longer accepts messages, AMD2 writes errors to the overflow file, AMD2LOGO. AMD2 then issues error message 33219:

MainframeConnect for DB2/MVS-CICS error (Write to log file AMD2LOG failed, switching to overflow log AMD2LOGO).

Under CICS, you can close, archive (if desired), and purge AMD2LOG when it becomes full. During the time AMD2LOG is closed, AMD2 writes messages to AMD2LOGO that you can close, archive, and purge once you reopen AMD2LOG.

In this example, AMD2 writes an internal error 32000 to AMD2LOG (or AMD2LOGO, if AMD2LOG is full).

For CICS environments, you must define the error log files to CICS in an FCT entry as follows:

• AMD2LOG

  DFHFCT    TYPE=DATASET,
       ACCMETH=VSAM,
       DATASET=AMD2LOG,
       RECFORM=(FIXED,BLOCKED),
       DSNAME=your.syrtlog.dataset.name,
       DISP=SHR,
       SERVREQ=(ADD,BROWSE)

• AMD2LOGO

  DFHFCT    TYPE=DATASET,
       ACCMETH=VSAM,
       DATASET=AMD2LOGO,
       RECFORM=(FIXED,BLOCKED),
       DSNAME=your.syrtlogo.dataset.name,
       DISP=SHR,
       SERVREQ=(ADD,BROWSE)

Coordination

System administrators at the mainframe, DirectConnect and client need to coordinate troubleshooting efforts. To aid in your analysis, this section describes the processing flow from the client through DirectConnect to the mainframe.

Processing flow and DirectConnect requirements

DirectConnect processing can involve CT-Library, DB-Library, and ODBC. The following figure shows DirectConnect processing with DB-Library (for more information, see the DirectConnect for MVS Access Service User's Guide for DirectConnect for MVS):

The following points describe the sequence shown in Figure 9-3 and highlight Enterprise Connect requirements:

The following points describe both RPC and dynamic SQL processing flows.

• Assuming DirectConnect is started, the client opens a LAN connection to a designated DirectConnect Service and logs in. If the message

  Server name not found in interface file

  occurs, make sure:

  • The client interfaces file is correctly set up.

  • The client Sybase path variable is correctly defined.

  • The DirectConnect server is specified in the DSQUERY variable.

• On receiving the client login information, DirectConnect checks security as follows (for TRS only):

  • If security is turned on, DirectConnect ensures the client is authorized. If the client is not authorized, this error occurs:

    Security Violation: Login denied (no login entry).

  • If the client is authorized or security is turned off, DirectConnect acknowledges the login.

• When the client application needs to invoke an RPC or language request on the mainframe, the client sends a request to DirectConnect over the LAN connection.

• DirectConnect receives the request and performs a table lookup to find the mainframe session and Open ServerConnect transaction ID to use. The RPC and connection must be in the table. If security is turned on, the client must be authorized to use the RPC and connection to the mainframe. Provided that the table lookup and security check are successful, the line is up, and the session is active, DirectConnect allocates a conversation with the named transaction.

  If a failure occurs during this process, one of the following error messages is written to both the DirectConnect log and client:

  • Security Violation: Access to RPC 'xxxx' denied.

    The client is not authorized or is not listed correctly.

  • Request Rejected: No host connections are available.

    Connections to the mainframe are unavailable.

  • Request Rejected: Remote procedure 'xxxx' not found.

    The RPC name was entered incorrectly or the name is not in the lookup table.

• DirectConnect sends the client's External Data Representation (XDR) information and RPC parameters to the mainframe, and then waits for a reply from the transaction.

• On the mainframe, the transaction processor initiates the named transaction, and the transaction issues Open ServerConnect Gateway-Library calls. These calls read the client's XDR information and RPC parameters. The transaction also performs associated processing, such as issuing static SQL DB2 requests or reading VSAM or other database data.

• The transaction issues Gateway-Library calls to send results back to the client. These calls perform any data conversions, generate the TDS reply datastream, and send out reply data.

• DirectConnect receives each TDS reply packet and forwards it to the client, which continues until the Open ServerConnect transaction issues a TDSNDDON call.

  If a failure occurs during this process, an error message is written to the DirectConnect log. An "Unexpected EOF from Adaptive Server" message is written to the client. (The mainframe is acting as an Adaptive Server.) Gateway-Library tracing functions, if in use, also record errors in this process. See Figure 9-2

• When completed with the request, the transaction exits; then the conversation terminates. For a long-running transaction (also called a user-defined transaction), the transaction can remain active through multiple requests before the conversation terminates. If a long-running transaction ends before it finishes processing, determine whether appropriate client support is set up. For example:

  • The client can be set up to disconnect after invoking the transaction and before the transaction ends.

  • Adaptive Server logs out after sending a client request and, therefore, does not support long-running transactions.

AMD2 supports user-defined transactions, which are comparable to long-running transactions based on Gateway-Library support.

Process flow during attention sequences

Any of the following actions results in an attention sequence:

• Database-Library issues a dbcancel( ) command.

• A SQL user cancels processing while the server is sending results.

• An APT program or form issues a closesql command.

When an attention sequence is issued, the process flow is as follows:

• Open Client issues an attention packet to DirectConnect, then discards anything else received until it receives a TDSDONE packet with the attention Ack bit on.

• DirectConnect converts the attention packet into a SNA SIGNAL command, issuing an LU 6.2 request-to-send verb. DirectConnect then discards any results received from the mainframe until receiving a TDS DONE packet with the attention Ack bit on.

• At the mainframe, CICS receives the SIGNAL command and informs Open ServerConnect.

• Gateway-Library passes back a return code, indicating TDS_CANCEL_RECEIVED, on all subsequent TDSNDROW, TDSNDMSG, and TDSETPRM calls from an application. Any data associated with TDSNDROW or TDSNDMSG calls is discarded until the application issues a TDSNDDON call.

  For details on these calls, see the Programmer's Reference for Open ServerConnect. COBOL and PL/1 versions of this guide are available.

• When the application issues a TDSNDDON call, Open ServerConnect support sends a TDS DONE packet with the attention Ack bit on. This ends the attention sequence.