| |
Home |
![[Table of Contents]](/dwicons/b_toc.gif)
|
Sybase dbQueue User's Guide |
![[-]](/dwicons/i_colpse.gif)
| Chapter 2: dbQueue Architecture |
This chapter contains the following topics:
A dbQueue system consists of product components and system entities.
Product components are those that you receive when you purchase dbQueue, and include the API libraries, Sybase Central, and the QTM.
System entities refer to elements of your system that you configure in order to implement dbQueue system functionality. Some system entities, such as the domain catalog, application databases, and queues, have a physical implementation. Others, such as message types and transfers, have a metadata implementation; that is, they define the function and relationships of other system entities.
Some product components are also system entities. For example, the QTM is distributed with the software, but you must also configure and administer it as a system entity.
Figure 2-1 provides an overview of the basic relationship between product components and system entities.
Figure 2-1: Sybase dbQueue system elements
dbQueue product components include:
dbQueue is distributed with several API libraries that offer a great deal of flexibility for application development. dbQueue APIs include C++ objects and ActiveX components. dbQueue also supports the CT-Lib, ODBC, and JDBC APIs. You can use any of these APIs to perform enqueue and dequeue operations. You can use dbQueue objects and ActiveX components to administer the system. Table 2-1 shows the capabilities of each dbQueue API.
dbQueue Operations supported | Languages supported | ||||||
|---|---|---|---|---|---|---|---|
API | administration | queuing | SQL | C | C++ | Visual Basic/ | Java |
SQL | X | X | |||||
CT-Lib | X | X | X | ||||
ODBC | X | X | X | ||||
JDBC | X | X | |||||
C++ Objects | X | X | X | ||||
ActiveX | X | X | X | X | |||
See Chapter 4, "Building dbQueue Applications," for more information about using the APIs to build applications.
Sybase Central is a graphical user interface (GUI) tool that is built on top of the dbQueue objects library. Use it to define, create, and administer dbQueue. It uses graphical representations of entities and dialog boxes to guide you through each step. It is also compatible with SQL Server and SQL Anywhere.
Use Sybase Central to set up the dbQueue system. Once the system is up and running, you can use it to add, delete, and reconfigure entities, and to monitor the status of the queues and QTMs.
Sybase Central includes a queue browser that you can use to view, enqueue, and dequeue messages.
See Chapter 3, "Using Sybase Central," for more information about Sybase Central.
A trial version of Sybase SQL Anywhere for Windows NT is distributed with this release of dbQueue. SQL Anywhere is a scalable dataserver designed to be deployed on desktop, workgroup, or mobile environments. dbQueue uses the trial version of SQL Anywhere as a demo database. If you already have a copy of SQL Anywhere installed, you can use that copy.
dbQueue includes a Java-based queuing tool that performs enqueue and dequeue operations using a Java-enabled Web browser. See Chapter 5, "Java Queuing Tool," for more information.
There are six types of system entities:
A domain is defined in the dbQueue system as a boundary that provides administrative focus. It may be physical (for example, all the databases in the New York office) or task-specific (for example, all the databases tied to accounting functions). A dbQueue system may comprise one or more domains.
A domain includes one or more application databases, the queues that reside in these databases, QTM programs that transfer messages between queues, and a catalog that describes all the entities within the domain.
Figure 2-2 illustrates a single-domain sample system with two application databases, two queues, one QTM, and one catalog.
Figure 2-2: Single-domain system
Figure 2-3 illustrates a two-domain system. The New York and Boston domains are connected by a Wide Area Network (WAN).
Figure 2-3: A two-domain system
The domain scope is the area of administrative responsibility of the domain. It is limited to the entities defined within the domain. Each domain has one catalog and at least one application database (which cannot belong to another domain). You can define the domain scope to be of any size or configuration.
The following list illustrates three typical domain configurations:
Queues communicate with queues in other domains by defining remote queues as residents in a foreign data source. Enqueue and dequeue operations between queues in different domains are identical.
The domain catalog consists of tables that define the entities in the domain. Each entity in the domain is defined by a set of attributes. For example, a queue might be defined by its name, location, size, and types of messages it can hold. This information is stored as an entry in the catalog tables. Each entity in the domain has one entry in the catalog.
If you add an entity to the system using Sybase Central, dbQueue automatically enters information about that entity into the catalog. If you delete an entity, dbQueue deletes its entry. And if you alter an entity, the dbQueue updates the information in the catalog.
Some entities, such as queues, have both metadata definition and physical implementation. The metadata entry for the queue reflects the physical implementation of the queue. For example, if you increase the queue size, dbQueue updates the metadata entry before dbQueue alters the physical queue.
Each domain has one catalog, which dbQueue automatically implements when you define the domain. The catalog tables reside in the catalog database.
WARNING! Only one administrator should access a catalog at a time. The catalog does not have explicit protection against simultaneous updates by multiple administrators, and may result in the loss of catalog data. Be particularly aware of this limitation when you are using Sybase Central.
Typically, application databases are the user databases from which client applications access data. dbQueue queues are implemented as tables within these databases. Applications perform queuing operations in addition to normal database operations.
Figure 2-4 illustrates an application database with a dbQueue queue.
Figure 2-4: Application databases with dbQueue queue
Application databases must be up and running before they can be defined and added to the domain catalog. You cannot create application databases with dbQueue. Instead, dbQueue uses the metadata definition to generate SQL and stored procedures that prepare the database for queue installation. Later, during queue installation, dbQueue generates additional SQL and stored procedures to implement the queue.
Application databases cannot belong to more than one domain, and each domain must have at least one application database. An application database can contain more than one queue.
dbQueue applications use messages to send information to other applications. A message has a header and a body. The header contains control information, and the body contains the data.
Each message belongs to a message type. A message type defines the structure of the message data, including the number of columns, the datatype, the length, and the scale of each column in the message body.
Message types must exist before applications can enqueue or dequeue messages. You can use Sybase Central or the dbQueue API to create message types.
When an application enqueues a message, it specifies the message type along with the data. When it dequeues a message, it reads the message type from the header in order to retrieve the data with the appropriate data structure.
For example, an order entry application needs to pass four pieces of data to another application by enqueuing into the manufacture queue. The four pieces of information are: the name of the client, the invoice number, the product name, and the price. The message type is named "takeorder" and has the following columns and datatypes:
name DBQ_VARCHAR
invoice DBQ_INT
product DBQ_VARCHAR
price DBQ_NUMERIC
If the order is for Jones, its invoice number is 1234, the product is a radio, and the price is $39.95, the enqueue command in the application may appear as:
enq_manufacture Jones, 1234,radio, 39.95
Figure 2-5: A message contains header information and data
The header contains the message type name, timestamp, and user-defined fields subject, message ID, and priority. You can perform operations such as dequeue by subject and dequeue by message ID. See the Sybase dbQueue Reference Guide for a complete list of dequeue methods and procedures. Table 2-2 lists the header fields.
Header Fields | Datatype | Description |
|---|---|---|
MessageTypeName | DBQ_VARCHAR | Message type name |
Subject | DBQ_VARCHAR | User-defined subject field |
MessageID | DBQ_INT | User-defined ID number |
Timestamp | DBQ_DATETIME | Date and time message was enqueued |
Sender | DBQ_VARCHAR | Sender login |
Priority | DBQ_INT | User-defined priority number |
Message types are defined globally for the domain and are stored as metadata in the domain catalog. An application can use as many message types as needed for its different data formats.
You must define all messages types for a particular queue in the domain before you can add them to the queue. Once you create a queue, you can add any supported message type to the queue.
dbQueue queues store messages for applications until they are needed. They are implemented as tables and stored procedures within application databases to utilize the transactional integrity, backup, and recovery mechanisms of data servers.
You can configure queues to store messages of different message types. Messages are stored in a queue following a first-in first-out (FIFO) order.
Applications perform queuing operations using the dbQueue API libraries. See Chapter 4, "Building dbQueue Applications," for information.
Queues are either visible or packaged. A visible queue is a database table that has been implemented to behave as a queue, and can accept only one message type. A message is stored as a row in the table. If the message type has integer, varchar, and datetime columns, a visible queue for this message type will include a table with integer, varchar, and datetime columns for its data fields. Control columns for the header are included in all queue implementations.
Because message data is stored in corresponding database columns, the data is accessible and can be viewed using standard database query commands. Visible queues can be queried using the native SQL query mechanisms of the database. dbQueue supplies the qar_Q special function to query the queue contents and to locate specific messages in a queue. Always use dbQueue-supplied functions to query queues unless directed otherwise by Sybase Technical Support.
Figure 2-6 illustrates how visible queues are implemented.
Figure 2-6: Visible queue implementation
The message data in packaged queues is placed into packets and stored in varchar or image columns. The advantage of packaged queues is that they can accept messages of multiple types. During the enqueue operation, messages are placed into packets before they are added to the queue. During the dequeue operation, the packets are unpacked and retrieved into columns according to the message type information in the header.
Figure 2-7 illustrates how packaged queues are implemented.
Figure 2-7: Packaged queue implementation
Each queue is defined either as a source queue or a target queue. Applications can enqueue to source queues but cannot dequeue from them. Unlike source queues, target queues can accept both enqueue and dequeue operations. Messages are transferred from source queues to target queues using the QTM.
You can use a single target queue if both the enqueuing and dequeuing applications have direct access to the queue. Typically, this situation occurs when both the applications are accessing the same application database at the same site. In this configuration, the target queue functions as both the source and the target. Because the message transfer between queues is not needed in this instance, a QTM is not required.
Figure 2-8 illustrates the relationship between source and target queues, their associated applications, and the QTM. In scenario A, the source and target queue are in separate application databases and the QTM is used to transfer the messages between them. In scenario B, a single target queue resides in an application database where both applications have direct access.
Figure 2-8: Source and target queues
Creating a queue has two phases: when you defined the queue, its entry is written into the catalog metadata; it is then implemented as a physical queue in the application database.
You can perform both phases at the same time or separately. Either way, dbQueue automatically generates the SQL and stored procedures to prepare the database and implement the queues. If you are creating many queues that are distributed over a WAN, it may be more efficient to define all the queues first and then implement them together as a batch process to minimize network traffic.
Once you have defined and implemented a queue, add a list of accepted message types to a packaged queue or a single message type to a visible queue.
Note: Before you can create a queue, you must have already defined the application database in the domain where they will reside. You must also have defined the message types that the queue will accept.
Queues within a domain are addressed using a combination of the data source name, schema name, and the queue name, which is the same as the table name. The syntax is:
domain_name:datasource:queuename
For example, the address of a queue named takeorder defined in the dbq_sa schema managed by the NY_server is:
Eastern_Sales:NY_server:takeorder
Using this scheme, all queues within the domain have a unique address.
Messages in a queue are ordered by their origin-commit order and follow the FIFO ordering principle. For example, if the same application sends message A and then message B to a single queue, message A will be forwarded before message B.
Figure 2-9 illustrates a queue with four messages. A new message, E, will enqueue to the end of the queue. The first message, A, is dequeued first from the head of the queue.
Figure 2-9: Applications enqueue and dequeue to a FIFO queue
There are no ordering rules if the messages are sent to different queues.
A source queue can be enqueued by many applications, including the QTM. A target queue can be both enqueued and dequeued by more than one application.
Figure 2-10: Queues support multiple accesses
A target queue can be written to by several QTMs. This is known as "fan in" configuration.
Figure 2-11: Target queue fan in from QTMs
You can "fan out" from a source queue to multiple target queues. To perform fan out, define a transfer from the source queue to each target queue. For example, to distribute messages from source queue S1 to three targets (T1, T2, and T3), create three transfers: S1 to T1, S1 to T2, and S1 to T3.
Figure 2-12: Multiple target queues (fan out)
QTMs transfer messages from source queues to target queues. Each QTM is a daemon process that runs at the source queue site. QTMs continuously monitor the source queue for undelivered messages and transfer them to the target queues.
The QTM ensures that the transfer to the target queues occurs in the same order in which messages were enqueued in the source queue.
When a QTM starts up, it reads the catalog for information on the number of source queues it needs to transfer, the source queue names and locations, the target queue names and locations, and any other transfer information. The QTM locates the source queues and starts one internal thread for each source queue. Each source queue can be associated with only one QTM.
QTMs use the sql.ini or the interfaces file to identify the servers and services on the network.
A transfer defines how messages are transferred. Transfers are defined for the scope of the domain and are stored as metadata in the domain catalog. Each transfer definition has a name and lists the source queue, the target queue, the transfer schedule, the retention period, and the QTM that will perform the transfer.
For example, transfer T1 might specify that source queue Q1 is to be transferred to target queue Q2 using QTM_NY.
There are several design issues you should consider when planning a dbQueue implementation.
You may want to put all this information into a worksheet format that you can fill in as you read the rest of this chapter.
Figure 2-13: Creating and administering a dbQueue system
You can use the following methods to administer your system.
For information about using Sybase Central, refer to Chapter 3, "Using Sybase Central." For information on embedding system information directly into dbQueue applications and using compiled scripts, please refer to Chapter 4, "Building Your Applications."
Note: Sybase Central runs on Windows NT only. However, you can use it to administer the system on both Windows NT and UNIX. For example, you can have your application database, queues, and catalog on a UNIX system, but administer them using Sybase Central running on Windows NT on the network. If you are on a UNIX-only system, then you must use compiled scripts to administer the system.
|
|