Msg*Log: Reliable Messaging for Cougaar Object Services and - - PowerPoint PPT Presentation

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 1

Msg*Log: Reliable Messaging for Cougaar

Object Services and Consulting, Inc. Steve Ford, Craig Thompson, David Wells, Tom Bannon {ford, thompson, wells, bannon}@objs.com http://www.objs.com/ultralog

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 2

Objective/Approach/Impact

• Objective

– develop a scalable, reliable, survivable communication infrastructure for the DARPA Cougaar architecture of loosely connected agents

• Approach

– extend Cougaar message transport with inter-node message transport mechanisms based on the robust, ubiquitous email (SMTP) and NetNews (NNTP). – provide Cougaar with a higher level adaptive Message Transport capability that is policy-based and extensible and can automatically switch among transports to cope with degradation

• f the communications infrastructure.
• Impact

– Logistics organizations will be able to reliably construct, monitor, and revise logistics plans under chaotic conditions

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 3

The Problem

• The Cougaar planning process is highly distributed. Reliable and

timely message delivery to multiple planning agents is mandatory.

• Cougaar’s current communications mechanism is based on Java

RMI extended with message queues and retry policies.

– A strength of RMI is its tight integration with Java and its speed. – A weakness is its dependence on point-to-point direct connectivity between sender and receiver and an accurate knowledge of the recipient’s IP address and port ID. In the worst case, end-to-end connectivity between critical pairs of nodes may never exist, at least not often enough to produce plans in an acceptable time.

• Thus, even though an RMI-based approach can tolerate chaotic

conditions, it may not be able to accomplish much during them, which will prevent Ultra*Log from reaching its goal of “no more than 20% capability degradation and 30% performance degradation under conditions of 45% information infrastructure loss in an environment of 90% of maximal real-world chaos”.

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 4

Adding Survivable Communications

• RMI, SMTP, and NNTP have different operational

characteristics

– RMI's direct delivery property makes it the fastest, but requires the greatest network and recipient stability. – SMTP and NNTP implement a store-and-forward model, so direct end-to-end connectivity is not needed. This makes them more attractive when networks begin to partition, but also makes them slower. Furthermore, in both SMTP and NNTP, recipients “pick up” messages rather than the messages being “delivered” as in RMI. – NNTP in particular is efficient at distributing messages with high fanout through multiple layers and large numbers of servers, making it particularly useful under highly chaotic conditions.

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 5

Other Benefits

• Protocol diversity makes the system less vulnerable to cyber-

attack, which could focus on protocol weaknesses, implementation flaws, or rigid and predictable message routing patterns.

• Both SMTP and NNTP provide significant support for remote

message archiving, which can be a basis for remote persistence and message logging facilities.

• Traffic analysis will be somewhat more difficult, thus increasing

security.

• With SMTP and NNTP, you don’t need control of intermediate

nodes.

• Both SMTP and NNTP can send messages through firewalls.
• Both SMTP and NNTP are scalable and industrial strength.
• NNTP adds a reliability/replication infrastructure.

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 6

Adding Email and NNTP Based Message Transport

• Convert Cougaar messages to email messages or news postings.
• Transmit them via the existing email or news infrastructure.
• Convert them back to Cougaar messages on the receiving end.
• Route them to the appropriate cluster(s) using the existing

Cougaar infrastructure.

• The same encoding will be used by the SMTP and NNTP Message

Transport.

• Will include the capability to pick up email or news messages on

an appropriate schedule.

• Will have the ability to try alternate sites.
• Several possibilities for mapping Cougaar Cluster addressing to

email and news addressing

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 7

Adaptive Message Transport

• Provide the ability to explicitly route messages via alternate

Message Transports improves the flexibility of inter-Node communications.

• Dynamically choose the “best” message transport from the

implemented set.

– Based on current sensed system state, policies and perceived threats – Currently undetermined connection to QoS, policy and detection parts of Cougaar.

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 8

Integrating the New Transport Mechanisms into Cougaar

• The design goal is to use RMI where it can be successful and

to seamlessly switch to the other protocols when needed

• New transport mechanisms can be integrated into the Cougaar

architecture mechanisms by subclassing the Cougaar MessageTransport class (via service plugins?)

• Adaptive Message Transport might be done the same way.
• We expect the eventual design to be an integration of

Cougaar’s current Message Transport implementation, of QuO, the new Policy Management capability, and our new transports and policies.

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 9

Challenges

• Spamming can be used to mount denial-of-service attacks
• It may be difficult to determine when replicated messages or

news postings are no longer needed and can be purged.

• When multiple protocols and delivery paths are used, it might

be difficult to ensure complete ordering of message delivery.

• In practice, it will be difficult to determine the actual “level of

chaos” in network conditions.

• Will performance be “good enough”?

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 10

Our Starting Point

• eGents

– leverage our work on agents that communicate by email. eGents uses an agent communication language (FIPA ACL) encoded in XML. eGents on the Palm => agents in the field (ubiquitous agents).

• Use the Cougaar architecture and implementation

– as an integration target to receive the Msg*Log technology – as a potential way to download improved Msg*Log code to Cougaar Clusters

• Use existing protocols

– use the SMTP, POP3, and NNTP protocols and existing implementations of email and news servers and clients

• Relevant technology context

– Components, Beans, SOAP and OMG, Software architectures, Domains and Policies, Survivability and Threat-based resource allocation – Direct control over network routing & caching, Messaging middleware, Low level multi-transport communications, ISIS, Ensemble, and related groupware, Quorum, CoABS Grid, HTTP-based Data Channels, Intelligent Swarms

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 11

Basis for confidence

• Recent work on software architectures has led to the conjecture

that system-wide non-functional properties including scalability, reliability, security, and survivability can be controlled by influencing the communication paths (the connectors) between components

• Msg*Log design is based on mature protocols (SMTP and

NNTP) supported by a robust, ubiquitous server infrastructure.

• Identified potential weaknesses will not preclude the system

from working; at worst, they degrade performance. Furthermore, all appear to have straightforward remedies.

• Prior OBJS implementation of email-based agent

communication (eGents).

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 12

Evaluation Criteria

• Our hypotheses:

– Msg*Log will allow Cougaar plan quality to degrade more slowly as a function of increasing chaos than would be the case in the existing Cougaar baseline system employing only RMI-based transport – Msg*Log will allow Cougaar planning to continue at levels of chaos that would prevent the baseline system from performing any planning at all.

• Success will be measured by comparing the behavior of the

augmented and baseline Cougaar at the transport (delivery times, lost messages, resends) and Cougaar planning levels.

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 13

Schedule, Tasks, Deliverables

Steve Ford, Craig Thompson, David Wells, Tom Bannon – www.objs.com/ultralog – Ultra*Log Kickoff Workshop – 6-8 Feb 2001 – p. 14

OMG Agent SIG

Meeting #13 – Irvine, CA, February 26-27, 2001 Ultra*Log speakers wanted: architecture, mobility, ontology, policy management, … http://www.objs.com/agent/index.html