Table of Contents

1.  Introduction
2.  Glossary
3.  Concepts
4.  The Parts
5.  The Organization
6.  Design Decisions
6.1  The Core Library
6.1.1  Protocol Enforcement
6.1.2  Memory Management
6.2  The Wrapper Library
6.3  The Application Layer
§  Author's Address
A.  Copyright and License

1. Introduction

This memo documents the architecture of the C BEEPCORE libraries. It is intended to be read in conjunction with the Profile Tutorial and the doc++ documentation. Where particular implementation choices are mentioned, they refer to the implementation choices of the current implementation, with generalizations naturally possible.

2. Glossary

Profile

- A specification of the behavior of a particular BEEP channel.

Profile Implementation

- A piece of code implementing a profile's specification. Sometimes simply "profile" where this is not ambiguous. This is a "concrete class" implementing the profile.

Profile Registration

- A structure that specifies the entry points and other static information about one profile within a profile implementation. This is similar to an "abstract interface" to a class.

Profile Instance

- A data structure representing an open channel. The instance has a reference to its registration and serves as an "instance" of the profile "class."

Application

- The top-level code that runs in a process. It is responsible for managing profile registrations and sockets, and assigning sockets and registrations to wrappers.

Session Structure

- The low-level session instance as defined by the core libraries. The session structure represents a single session, from greeting to close. It tracks open channels and outstanding messages. However, it does no I/O itself, and it has no concept of "profile" as such. A new session structure must be created to generate a greeting, so a TLS tuning reset will create a new session structure.

Wrapper

- The wrapper is the code that provides I/O services for the session and interfaces to profile instances. It is responsible for determining which profiles shall run, creating threads for them, managing the timing requirements of the lower layer libraries, locking where necessary, logging of important events, and so on.

Wrapper Instance

- This represents an entire connection, from the opening of the socket to the closing of the socket. It is a structure managed by the wrapper code.

Wrapper Mode

- Each session within a wrapper is in a particular mode. A tuning reset can change modes. For example, "plaintext" and "encrypted" might be two modes. Profiles indicate the modes in which they are active when they register, and the wrapper ensures that active profiles are properly managed. Modes are arbitrary short strings.

Request, Indication, Response, Confirmation

- These four terms (from the OSI reference model) name the four messages which cross the API boundaries in a client-server model. The request is the initial message sent. When it arrives at the peer, it is an indication. The peer's answer is formed as a response. When the response is delivered, it is a confirmation. Note that confirmations can be negative or positive; both <error> and <ok> can be confirmations.

3. Concepts

There are a number of concepts that are used throughout this document. Understanding these concepts is vital to understanding the document and the libraries. Although the RFCs include many of these concepts, the libraries have implementation-specific meanings for them. These concepts are described herein.

Frame vs. Message

- The first distinction is between a frame and a message. While both are clearly defined in the RFCs, the libraries deal only with frames. A frame that happens to hold both the beginning of a message and the end of a message is a complete message as well as a complete frame. Many places in the API accept a "frame," such as the routine to send a frame over a channel. It should be understood that if one wants to send a complete message, the complete message is put into a frame and one sends the frame. There are also API calls to determine if enough frames have arrived to form a complete message and to accept a series of frames and turn them into a single complete frame.

Session vs. Wrapper Instance

- The RFCs are mildly confusing about what constitutes a session. In some places, a session is described as being established with a greeting. In other places, the description implies there is one session per transport connection. These two different interpretations come into conflict when a tuning reset is performed, which issues a new greeting over the same transport connection. For the purposes of this document, a new session starts just before a greeting is issued, and that session persists until channel zero is closed. Channel zero is closed when either the underlying transport connection is closed or before a non-BEEP negotiation (such as a TLS negotiation) starts, before another greeting is sent. The session can also be terminated without closing the underlying socket by a tuning reset understood by both peers but which does not result in a new BEEP greeting. (For example, a tuning reset after which some different protocol is used over the socket would fall into this category.) On the other hand, one wrapper instance corresponds to multiple BEEP sessions. A wrapper instance is finished when the last BEEP-formatted message has been exchanged. This occurs when a session is closed, a socket error occurs, a protocol error occurs, or a tuning reset explicitly destroys the wrapper without closing the socket (which is an example of the different-protocol tuning reset described previously).

Tuning Profile

- The RFCs describe a tuning profile as one which configures something about the entire session and then either goes idle or initiates a tuning reset. In this document, and for purposes of the library, a tuning profile is one that initiates a tuning reset. There is no practical reason to distinguish a profile which runs briefly then idles from one which continues to run. In the sense used in this document, a tuning profile monopolizes the connection, preventing other profiles from exchanging data while it runs.

Frame-mode vs. Message-mode

- A profile implementation has two ways in which it can deal with incoming messages. It has the choice of dealing with frames as they arrive, or waiting until the remote peer has finished sending an entire message. This is entirely a matter of implementation choice, not a matter of protocol. By dealing with individual frames rather than waiting for entire messages, one can handle arbitrarily large messages and one might achieve a greater throughput via pipelining of I/O and processing. By dealing only with complete messages, one might achieve a significant simplification in message parsing and management.

Mode

- Each wrapper instance associates a "mode" with each session. This mode is represented as an arbitrary string. It is simply a selection mechanism for deciding which of the configured profile implementations will be initialized and/or advertised in the greeting of that particular session. The initial mode is specified when the wrapper instance is created. A profile instance that initiates a tuning reset can specify the mode of the new session after the tuning reset completes and before the new greeting (if any) is sent.

Initiator vs. Listener Wrapper Instances

- In the RFCs, the initiating session is the one that issues an active open on the underlying transport connection, while the listening session is the one that issues the passive open. In socket terms, the initiator calls "connect" while the listener calls "accept." Since initializing a wrapper involves passing in an already-connected socket, one must also pass in whether this is a listener or initiator wrapper. The application is responsible for opening the socket, so the wrapper itself cannot deduce this. Indeed, it might be different for two different sockets (and thus wrapper instances) within the same application. At the protocol level, the only real difference is the set of channel numbers that may appear within the <start> element.

Initiator vs. Listener Profile Implementations

- On the other hand, the "initiator" and "listener" terms also apply to profile implementations. In this case, a "listener" profile is one whose URI is advertised in the greeting. An "initiator" profile is one that is initialized but is not advertised in the greeting. Either type of profile may accept <start> requests, and either type may initiate <start> requests. However, since a profile must be initialized before a channel can be started for that profile, a profile listed in a <start> request must be either an initiator or listener profile for the current mode.

Profile Registration

- In the RFCs, a profile registration is a mechanism for reserving and advertising profiles via the IANA. In the context of the wrappers, a profile registration is a structure listing the URI and entry points (along with a few other items), informing the wrapper of which functions to invoke upon the occurrence of certain events.

4. The Parts

The library is broken into a number of layers. There are four major parts of any C-BEEPCORE based application that intercommunicate to form a server or client. A brief (and imprecise) description of each follows:

Core Library

- The core library is written in pure portable C. It invokes no operating system services on its own. It does no memory allocation or I/O on its own. It is responsible for parsing incoming frames, formatting outgoing frames, tracking open channels, generating and responding to SEQ frames, and in general all the stuff that every RFC3080 and RFC3081 implementation will need to do. It is not expected that the authors of profiles will use the core library directly. Instead, they will always use a wrapper.

Wrapper

- The wrapper is a piece of code that "wraps around" the core library. It provides for synchronization, I/O, memory allocation, mapping of URIs in <start> requests to code in the application, thread management, and so on. All access by profile implementations to the BEEP functionality will be via a wrapper.

Profile Implementations

- The "meat and potatos" of the system is of course the profile implementations. Profile implementations are pieces of code that implement profiles. They are registered with a wrapper and invoked by the application and the wrapper. They are responsible for generating and responding to individual messages over channels.

Application

- Finally, the application that puts it all together. The application is responsible for parsing any configuration files, creating sockets, loading any dynamically-loaded profile implementations, registering profile implementations with wrappers, and starting the wrapper. It's also responsible for cleaning up after the wrapper has finished its work, closing sockets and so on.

It is important to note that profiles and applications access only the wrapper code, and the wrapper code regulates all access to the core code.

Of course, there is the difference between code and process, which is glossed over here. That is, while there is likely only one wrapper (i.e., one piece of code), there may be many wrapper instances active at once, each serving a particular socket. While there is likely only one implementation of each profile, there will be one instantiation of each profile for each channel running that profile. A single application might be running in multiple processes. And finally, the core library can create session structures to track various BEEP sessions in a single process.

5. The Organization

The system is broken up into four general pieces: the application layer, the wrapper, the core library, and profiles:

The overall structure:

The reason for splitting the profile implementations off as separate chunks of code should be pretty obvious. The design of the BEEP protocol is to allow exactly this sort of thing. Splitting the profile code from the rest of the code allows for new profiles to be developed after the library has been finished.

The reason for separating the wrapper from the application is to avoid making decisions in the wrapper about finding profile implementations, configuration, and so on. It also allows the wrapper to be used for both initiators and listeners without change. If Invisible Worlds supplied the main() routine, the configuration would be so complex as to be unmanagable. Instead, the developer of an application is responsible for configuration. The developer is also responsible when coding initiators for starting the appropriate channels with the appropriate start data. The application also decides whether to run one session per process or multiple sessions per process.

Finally, the reason for separating the core library from the wrapper is to allow the core library to be reused in places where the wrapper might not be portable. For example, the core library could be used as part of an Apache module, embedded in a BEEP-enabled PDA, or used in a test and performance harness. Each of these applications would require a new wrapper, but all could use the same core library. In addition, a new wrapper is likely necessary for various operating systems, while the same core library will serve all operating systems, including those without threads.

As can be seen above, the application is naturally the outermost structure. It is responsible for creating the socket, loading the profile implementations, instantiating a wrapper, and registering the profile implementations with the wrapper. It manages the creation and destruction of sockets and wrappers. In an initiating application, it is also responsible for starting the channels once greetings are exchanged.

6. Design Decisions

The most obvious design decision was the overall architecture of the system, as described previously: the separation of wrapper from core, and of application from wrapper.

6.1 The Core Library

The core library (CBEEP.c) was designed with a number of goals in mind. It does not rely on any operating system services except where absolutely necessary. In particular, it does not rely on threads, I/O, or memory allocation. All these services are provided by the wrapper. While it uses a few routines from string.h and such, in theory the core library could be written entirely in C with no other libraries or OS services linked in.

The core is not "active". The only time the library does anything is when the wrapper calls it. For example, all parsing of frame headers takes place as the wrapper copies the bytes into the library. All management of internal channel structures occurs when the wrapper asks to dequeue a frame from channel zero. The selection of which frame to send next and the generation of SEQ messages occurs when the wrapper asks if any bytes are available to be written to the socket. Notifications are issued as part of dealing with other calls. For example, notifying the wrapper that a full frame has arrived is handled during the call that gives the bytes from the socket to the library.

In addition, the core library does not deal with profiles at all. It deals with channels and URIs, but the actual concept of a "profile" as such is not necessary. Hence, it is the wrapper's responsibility to map channels and URIs to profiles.

The core library itself is only serially reentrant for any given session instance. That is, if one thread is running within one of the blu_* or bll_* routines, no other thread should enter any of those routines passing a reference to the same session. On the other hand, since all global data is stored in session structures, multiple sessions can be in progress at the same time in a reentrant way.

The core library is designed with efficiency in mind. Frames are expected to be read from the socket directly into the memory that is passed to the profile, and written directly from the memory the profile fills in.

Internally, the core library stores all state information in a session structure or a structure pointed to therefrom. There are a few input and output buffers allocatated to the session (head_buf, read_buf, seq_buf), and each session maintains a pointer to a frame being read (read_frame). It also holds the greeting from the other peer. Finally, each session structure points to a table of channel structures, one for each open channel (including channel zero), currently organized as a linked list sorted by priority. Of course, general session state is maintained also.

Each channel, in turn, holds an ordered list of incoming frames, an ordered list of outgoing frames, a "committed" frame which is the next to be sent or in the process of being sent on this channel, an ordered list of received messages that have not been fully answered, and an ordered list of sent messages that have not been fully answered. The channel structure also holds all the other state necessary to make things work.

6.1.1 Protocol Enforcement

The core library enforces many aspects of the BEEP protocol. It refuses to accept frames beyond the valid sequence range. It refuses to send frames out of order. It refuses to send a reply to a message number that has not been received. Etc. However, there is one condition that is not enforced by the core library. In particular, if a close request for a channel is queued on channel zero, it is possible that request will be sent over the socket before all the messages already queued for the channel being closed have been sent. It is up to the wrapper to prevent this by delaying the close request until the channel is quiescent.

6.1.2 Memory Management

As part of the design of the core library, memory usage can be limited. When instantiating a wrapper or a session, a pointer to a malloc-like function and a free-like function is provided by the application. However, it is not the intent that these be used to limit memory usage, particularly since there is no convenient way of telling which wrapper or session is allocating the memory. Instead, the per-channel and per-wrapper limits should be set.

The per-channel memory limit is the "receive window." Peers are not permitted to send more payload than will fit in the window at one time. After frames that have been received have been passed to bpc_frame_destroy, a SEQ is sent to allow the peer to send more. This limit keeps the peer from filling up local memory if the local profile stops reading the channel for any reason.

The per-wrapper limit is a single limit for all frame allocations made with bpc_buffer_allocate or bpc_frame_aggregate. It is an arbitrary limit, enforced only if set. It is essentially the sender side of the receive window limit. It prevents the local machine from filling up all of memory if the remote peer stops reading the channel for any reason.

6.2 The Wrapper Library

In contrast to the core library, the wrapper library is designed to be very active and asynchronous. It mediates between the core library and the rest of the application and profiles. It starts multiple threads when multiple channels are open. It invokes callbacks on profiles, and expects the profiles to answer these callbacks.

Other wrappers might take an entirely different approach to threading and timing and configuration. For example, a wrapper designed to be used as an Apache module might use the Apache interface to threading and configuration. We do not discuss these possiblities further here.

In general, a profile is described by a profile registration structure. That structure describes when the profile implementation should be active and how the wrapper should invoke the profile implementation. The functions that the wrapper calls all start with "pro_" for "profile." The functions that the profile invokes start with "bp_" or "bpc_", for "BEEP library wrapper" or "BEEP library wrapper, channel-specific", respectively.

The library uses some OSI reference model terminology. In particular, it uses the terms "request", "indication", "response", and "confirmation" in a precise way. Generally, either an application or a profile will invoke a request, such as a request to start a channel. The peer will receive this request as an indication, and reply with a response. The originator of the request will receive the answer as a confirmation. Hence, all b*_ functions that cause protocol actions cause requests or responses. Also, all pro_ functions that cause protocol actions are indications or confirmations.

Technically, the pro_ functions implemented by the profile are intended to return immediately without blocking. However, there is an extra threading layer provided by the wrapper, so with the appropriate set-up, profile implementations don't need to worry about blocking. They still must, however, ensure that every indication is answered with a response, for example.

The wrapper is designed to allow a profile to easily interact with its peer. Interacting with multiple peers requires multiple profile instances and more sophisticated programming technique to link them together. For example, a profile implementing a multi-user game would need to implement some sort of shared data structure to allow the profile instance associated with each player to affect the other profile instances on other wrappers. That is, each "player" could easily affect the "world," but the application would have to arrange for the changes to the "world" to be distributed back out to all "players."

6.3 The Application Layer

Each application will have its own application layer. A couple of simple application layers are provided, but more complex applications need to be coded to cooperate with the profiles they support.

The application is responsible for loading configuration, loading profiles, creating a connected socket (via connect or accept), instantiating a wrapper for that socket, and cleaning up when the wrapper is finished. An initiating application is also responsible for starting any required channels, while a listening application can usually rely on the initiator to trigger any actions it needs to take.

The applications that come with the wrapper and core libraries follow a simple operation:

• Typically a configuration file is read that specifies:
  • Profiles that are dynamically-loaded.
  • Transport information such as server name and port number.

• They call a library routine that loads the profiles and invokes the _init entry point of each profile so loaded, building a list of all registrations returned thereby.
• The listener mode sets up a listening socket and accepts connections; in contrast, the initiator mode attempts to connect to the server at the port provided.
• Once the connection is established, the listener forks and instantiates a wrapper in the "plaintext" mode; in contrast, the initiator does not fork, but simply instantiates a wrapper in the "plaintext" mode.
• Within the application layer that comes with the library, individual initiating profiles are responsible for establishing connections. Typically, each profile will expose additional entry points (other than _init) for this purpose.

The code for a very simple forking server (listener) might look like this:

The code for a very simple initiator might look like this:

Author's Address

Appendix A. Copyright and License

Blocks Public License
Copyright (c) 2000, Invisible Worlds, Inc.
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
• Neither the name, trademarks, or tradenames of Invisible Worlds, Inc., nor the names, trademarks, or tradenames of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL INVISIBLE WORLDS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.