| Abstract
This dissertation describes the work and results of the project done by Jerome Tassel at British Telecommunication (BT) research labs for the M.Sc in distributed systems degree at the University of Kent at Canterbury. The objectives of the project were to evaluate Reflective Java, QoS management and charging as well as the emerging Internet Integrated Services Architecture. The main achievement of this project is a Java QoS control component which is a prototype version of a QoS management model (QoteS) based on Reflective Java that we have designed The main objective of this model is to allow easy, flexible addition of Quality of Service (QoS) control to a real-time Java Internet application. Flexibility is provided by a product from the ANSA research consortium: Reflective Java, which uses the Meta-object Protocol (MOP) to separate functional requirements (what the application does) from non-functional requirements (how it does it). This scheme permits the design and implementation of a generic QoS control element, which can then be added to an application for which QoS control was not originally required but now is. Such applications might be collaborative tools where requirements have moved from a text-based interface to a multimedia interface using audio and video streaming facilities. Alternatively, an existing application with rudimentary QoS control can be modified to use more professionally produced QoS control classes intended to reconcile competition for QoS between applications. The QoS control element that we designed also has scope for QoS adaptation, including the dynamic introduction of QoS control at run-time if best effort service degrades below a useful point. We believe this model provides a flexible way to control the usage of the resources available to the users down to a very finely grained level. A paper covering this project [46] has been submitted for publication at the COST237 conference entitled "multimedia services networks" taking place in December 1997 in Portugal. The call for papers for this conference and the paper submitted are in the appendices. |
Acknowledgements
I would like to thank Bob Briscoe (BT Labs) for his supervision, precious time and his meticulous reviewing work. Thank you very much as well for making my time at BT very informative and for involving me in extra project activities (BT IETF and ANSA work, COST237 conference). This was a great experience.
Many thanks as well to Andy King (University of Kent) for supervising this project and the researchers both at the university of Kent and BT research laboratories for their help and time during this project: Alan Smith., Steve Rudkin, Peter Bagnall and Gill Waters and all the ones I don't mention.
NOTE: Header numbers don't match in the body as Word did not do a very good job at "htmlising" the original .doc file. Use the table of content to travel through the document and don't pay attention to the numbering. In the same way, the Java code examples in the body are not very easy to read.
1. Introduction
2. Related work and results from literature survey
LIST OF FIGURES
Figure 1, RSVP software in routers & hosts
Figure 2, RSVP message processing
Figure 3, The MOP implementation in Reflective JAVA
Figure 4, Meta-object processing in Reflective Java
Figure 5, Web2Talk GUI
Figure 6. Original application model
Figure 7, Reflective model for (adaptive) QoS control, on a per host
basis
Figure 8, Budget control processing rules
Figure 9, Run-Time information
Figure 10, Per-host QoteS run time environment
Figure 11, QoS manager data structure
Figure 12, QoS manager state/debug panel
Figure 13, Elements of decision in the QoS manager
Figure 14, JavaShell GUI
Figure 15, The user QoS control interface
Figure 16, Sending RSVP PATH messages
Figure 17, Requesting bandwidth reservations
Figure 18, Java RSVP API
Figure 19, QoteS prototype environment
LIST OF ABBREVIATIONS
| ANSA | Advanced Network Systems Architecture |
| API | Application Programming Interface |
| APM | Architecture Project Management |
| ATM | Asynchronous Transfer Mode |
| BT | British Telecommunications plc. |
| COST | European Co-operation in the field of Scientific and Technical Research |
| CTOR | Constructor |
| DIMMA | Distributed Interactive MultiMedia Architecture |
| DTOR | Destructor |
| GSM | Global System for Mobile communications |
| GUI | Graphical User Interface |
| ICMP | Internet Control Message Protocol |
| IETF | Internet Engineering Task Force |
| IP | Internet Protocol |
| ISA | Integrated Services Architecture |
| JNI | Java Native Interface |
| JVM | Java Virtual Machine |
| MOP | Meta-object Protocol |
| ODP | Open distributed Processing |
| ORB | Object Request Broker |
| OSI | Open System Interconnection |
| OSI-RM | Open System Interconnection- Reference Model |
| PATH | RSVP PATH messages |
| Q.2931 | ATM signalling protocol |
| QoS | Quality of Service |
| QoteS | Quality of Service around the edge |
| RAPI | RSVP API |
| RESV | RSVP Reservation messages |
| RFC | Request For Comment |
| RM-ODP | Reference Model - Object Distributed Processing |
| RMI | Remote Method Invocation |
| RSVP | Resource reSerVation Protocol |
| TCP | Transmission Control Protocol |
| TINA | Telecommunication Information Networking Architecture |
| UKC | University of Kent at Canterbury |
| URL | Uniform Resource Locator |
| Appendix 1 | COST237 Call for papers |
| Appendix 2 | Paper covering this project's work submitted to the COST237 committee for acceptance |
| Appendix 3 | Original project specification |
The objective of this project is the evaluation of a product from the ANSA (Advanced Network System Architecture) research consortium (of which BT and UKC are part): Reflective Java [16, 17] as a solution for easy and efficient addition of QoS (Quality of Service) management into real-time multimedia Internet applications. This evaluation is done through the design and implementation of a prototype QoS architecture based on this ANSA product. The evaluation of charging as a basis for QoS control and investigating the use of the new Internet bandwidth reservation protocol (RSVP), in a Java programming environment, are also two goals of this project. This last goal arose incidentally to the original project aims because up to now RSVP was only provided in a C/C++ environment and RSVP is still in its experimental phase.
The traditional approach to add QoS control to an application would be to use a QoS management API (Application Programming Interface). Using an API means having to understand the application code and modifying it in order to use the API functionality, it also means adding complexity to the application and reduces the flexibility and complicates later upgrades of the application. This project investigates Reflective Java as solution to those issues. The commercial and strategic background to this project for BT is therefore the need to educate software engineers from supplier companies, now, so that the QoS controllers they produce are easy to integrate with existing and forthcoming applications, are easier to debug and their time to market is reduced. It is the best time for this work as QoS control for the Internet is about to be delivered and therefore we can educate software engineers as they are starting to think about adding QoS control in their applications. For example, a joint venture between CISCO/MCI/Intel has produced a set of RSVP enabled applications [15]. One of the problems they encountered during this work was the amount of time required to modify the existing applications (one person a month per application) and they also had problems regarding the presentation of the QoS control to the user.
QoS control in distributed systems is a current research area due to the emerging new types of multimedia streaming applications [68] (e.g. conferencing) requiring real time delivery of information and an Internet integrated services architectures enabling the sharing of the network between data streams with different QoS requirements (real-time or best effort). QoS management allows the real time delivery of information as it encompasses both the control and delivery of QoS in an integrated services architecture because it enables to differentiation of the requirements of each data stream. Our work focuses on real time delivery of information and charging issues in QoS management.
Reflective Java allows us to separate the functional requirements of an application (what it does) from the non functional ones (how it does it). It is based on the Meta-object Protocol (MOP) defined in 1987 by Maes Peti in [12]. The Java implementation of Reflection from ANSA puts particular emphasis on the use of Reflection in distributed systems by taking advantage of the network-centric capabilities of Java. Reflection is the base of our model and is evaluated in this project.
In this project QoS control on the Internet is provided by using the emerging QoS control protocol being defined by the Internet Engineering Task Force (IETF): the Resource reSerVation Protocol (RSVP). RSVP is being defined as part of a new effort for enabling integrated service on the Internet: the ISA (Integrated Service Architecture). RSVP was introduced in 1993 by Lixia Zhang et al in [2]. RSVP is based on flow control and receiver initiated reservation by setting priorities in modified routers along the data path on the Internet. It uses a set of messages to establish, maintain and release reservations, it is still in an experimental phase at the time of this project. Other areas of QoS control: user, application and host resources are provided at the software level by our solutions (this is we have implemented some pseudo code to simulate host reservations). A possible commercial impact of RSVP is a more efficient use of the network by the introduction of a market mechanism over the user of the resources, charging issues related to this remark are discussed in this project.
The second section of this document is dedicated to describing the research areas related to this project, resulting from the reading phase: RSVP, Java for distributed systems, Reflective Java, QoS control and the Web2Talk one-to-one telephony application which is used in this work for testing purposes. The third section introduces the design work and the choices done in the project. The fourth section describes the experiences of the implementation of a prototype of the previously described design and eventual changes to be made to the design as well as evaluating reflection and RSVP for QoS management and finally the work is summarised and further possible research is introduced in the concluding part.
The objective of the first stage of this project was to get up to date with the current advances in the work areas this project covers. We therefore focused our reading on the following subjects : In the following sections we present a summary of our reading for the above subjects. We start with the IETF Integrated Services Architecture (ISA).Internet applications are changing from simple RPC type text based point to point applications (e-mail, World-Wide-Web) to real time, multi-users, multimedia applications. The original design of the TCP/IP suite only provided support for a best effort delivery schemes, ideal for applications such as ftp, World-Wide-Web, Telnet, database access, e-mail but this is inadequate for real-time delivery of data [67]. The efforts of the Internet Engineering Task Force (IETF) are now focused on developing a new Internet architecture which can provide Integrated Services on the Internet (the ISA). The objective of the ISA is to move away from the current model of the Internet providing a single best effort QoS, to develop a model which can support a wide range of QoS, some with soft real time delivery constraints and some with more elastic timing delivery constraints. Integrated service networks allow streams of data with different QoS requirements to co-exist gracefully on the same network. The ISA objectives are described in [1, 5]. The full work of the IETF on ISA can be found at the URL cited in [1] as well as the current implementation of the diverse protocols making the ISA and example applications using them.
The main weakness of the Internet protocol for guaranteed delivery of data is the variable latency of packets within the routers along the data path [2, 4], due to queuing delays, which increase the jitter of the data stream, thus jeopardising the real-time delivery of data. The objective of RSVP, as part of the ISA, is to provide some control over the routing queuing delays and therefore the QoS of data streams as they pass through the routers.
RSVP attempts overcome the weakness of the Internet protocol by reserving bandwidth for real time streams across the routers by setting packet priorities within the routers[2, 4], RSVP does not replace IP, it is a control protocol like ICMP (Internet Control Message Protocol)). In the case of guaranteed delivery of streams (the requested QoS is opaque to RSVP and at the time of writing there are two QoS types available: guaranteed delivery and controlled load), RSVP allows to provide a guaranteed set of resources for a data stream based on an IP/port number, this reservation is not physical like in CBR traffic on the telephony network but priorities are set within the routers for packet scheduling, thus streams can use reserved bandwidth which is not actually being used.
The main attributes of RSVP are [4]:
In both cases, application and router, a RSVP process is in charge of bandwidth reservation control. To simplify this diagram the policy and admission control modules do not appear, those modules interact with the RSVP process only. The Classifier and Scheduler elements implement the bandwidth reservation dictated by RSVP.
We now describe the major features of the RSVP protocol which are of interest for our work[2].
Admission and policy control [32] is the subject of current research work to provide an administrative control of bandwidth sharing and making sure that streams keep to their agreed traffic properties.
The protocol uses a set of messages (signalling) to establish the reservation, the following diagram illustrates the processing of those messages [5]. PATH messages are sent from the sending applications to the receiving applications first and then reservation messages are sent from the receiving applications to the sending applications.
There are two types of messages[4], PATH and RESV messages. PATH messages are used by the sending applications to disseminate information about the data stream their are producing (1 on the figure above) and RESV messages are used to establish the reservations (6 & 7). The TSPEC object describes the traffic specification of the sending applications, the ADSPEC object deals with the requirements of the sending application itself and the properties of the path. ADSPEC objects are updated along the path but TSPEC are delivered unchanged. The FLOWSPEC object is the QoS asked by the receiving application, the attributes within this object are described bellow. TSPEC and FLOWSPEC arriving at a network element (router, node) are then used to set the appropriate reservation.
Those message processing rules and the fact that RSVP is receiver oriented is of major importance for this project, particularly for the design of the QoS control model as we must integrate in our model the fact that the sending applications disseminate information about their expected generated traffic and that the QoS decision is then made from this information at the receivers.
As we stated earlier two objects are present in the PATH messages [5]:
In the context of our project it is crucial to understand what are the QoS properties of RSVP as we are using them and we need to be able to define what QoS properties RSVP deals with and which ones it does not. The full descriptions of the fields in the RSVP QoS objects can be found in [5], those details are required for the implementation purposes.
An other important aspect of RSVP is that of the call backs. Errors, modifications might occur along the data path and messages are sent back to the RAPI' s in order to notify the applications using RSVP for them to take appropriate actions. Those upcalls are associated with an application level method which is called on receipt of such calls. The possible messages are :
In this first section we have described the major issues of RSVP related to this project, In section 2.3 we describe the important features of Reflective Java, an other main player in our architecture.
At the time of this project, the research areas in QoS control in distributed systems [6] are mainly focused on :The following paragraphs present major aspects about QoS control in distributed systems that have input into the work of this project.
Separation of QoS control and application [35]
Explicit separation of QoS policies from QoS mechanisms, application only specify the QoS required [37]. Different layers of QoS: application knows about frames/sec and network knows about bits/sec, there is a need for some mapping between layers. In [11] QoS profiles unknown to the user are used to specify the QoS requirements of the network, application, media used and the user can specify its own profile such as quality required …etc. This is typical example of separation between QoS control and the user.
Adaptation
QoS adaptation is mainly about increasing or reducing the QoS properties if more or less resources are available. This of course is a major requirement in environment where resource reservation is not available and only a best effort scheme is available, for example in case where current Internet applications must adapt to the network congestion and reduce the data rate in case of congestion. (or it might be the other way around: the network reduces the data rate and notifies the application). A example of quality of service control for adaptive distributed multimedia applications is given in [9].
In the context of our work we assume that resource reservation is available (using RSVP for network resources and pseudo-code for host resources, this is explained in a later section) and therefore the scheme described above does not apply anymore and the adaptation notion changes. In an environment where resource reservation is available adaptation deals with adapting to user requirements changing, more resources being available in which case a previously turned down reservation because the resources were not available can now be accepted and the lower QoS given to this reservation can be upgraded and application changing their requirements: addition of video/audio streams on the fly.
End-to-End QoS [35]
Application only knows about a reliable file transfer and can only monitor, react upon that, lower layers deal with the QoS properties of bytes of the file being transferred properly. This again introduces the idea of QoS layers and mapping between layers and a distributed QoS control 35].
The players in QoS specifications
What elements of a distributed systems have inputs in the QoS properties ? It is not only the application which has some input on the QoS specification but also the user, the network, operating systems. This is because the user can establish some media quality that he requires in the same terms as the application does, the network can also offer a set of QoS and the application/user can chose from them and finally the operating system control its resources.
QoS Specification
QoS specification includes the formal specification of QoS and its "transmission" [22]. There are arguments in the literature for formal specification of QoS. An other problem is that there are different contenders in the specification of QoS (as stated below) and they have different visions of the QoS properties. Negotiation is required to provide the most efficient QoS this means some negotiation entity must understand the QoS syntax of all other players. The idea of formal specification is to standardise this notation. In [11], the notion of profiles is introduced, profiles are used to specify the QoS requirements, they are based on a database notation.
Different levels of QoS awareness
The different elements of the distributed system have different knowledge about QoS properties: some users know only about a very high level QoS of the type good/bad/average, this translates the human perception of some multimedia data, and those users should not know more about the QoS. Some other users, maybe more aware of technical details, might know more such as the frame rate of a video, the sampling size of a audio stream etc…Again here there is a need for some mapping between those different levels of awareness and the GUI aspect of QoS control has here an important place. The notion of profiles described in [11] could be used for this purpose.
Choice of QoS properties = decision & negotiation
As we stated earlier there are four players in the input process for QoS decision: the user, the network, the applications and the host. There is therefore a need to establish a negotiation protocol between all those involved in input to ensure that the best QoS is delivered and that the resources, on the global distributed system, available to all the processes are used to their best. An example of such a protocol and the required architecture to support it are described in [45].
QoS architectures [26]
As stated in [10], the role of the QoS architecture is to :
"…define a set of quality of service configurable interfaces that formalise quality of service in the end-system and network, providing a framework for the integration of quality of service control and management mechanisms."
We believe a QoS architectures permits all aspects of QoS control to be integrated and therefore provide an efficient QoS control. It also reduces the duplication of QoS control elements across layers in the communication systems [25].
Existing QoS architectures are based on OSI-RM, RM-ODP and ATM networks as well as the new Internet IS Architecture. For a presentation and list of such architectures please see [10, 26].
Also each of those architectures focus on a specific environment ATM, ORB, TINA, OSI, Internet they also provide some generic software engineering solutions to the control of QoS in distributed systems. In our work we propose such an architecture making use of Reflection and focusing on the QoS control of multimedia applications [27] in any environment also we focus our implementation on Internet applications using RSVP as the QoS control element for network QoS provision.
The model we have designed in this project is based on a technique called Reflection and we use an implementation of Reflection done in Java. It is a prototype product delivered by APM under the ANSA research program contract. In the next sections we describe Java and Reflective Java. (The word Reflection should not be associated with the Reflection API [55, 56]] provided by Sun ™) In this section we define the important aspects of Java [19, 20, 64] regarding distributed systems which need to be mastered in order to understand the design of Reflective Java [48].Object orientation
Java is object oriented, a major requirement for a distributed system programming language allowing the elements of a working system to be designed independently of each other and distribute them at different locations within the system [53], pass messages between them as well as to define accessible interfaces.
Architecture neutral and portable: mobile code
The Java code is compiled into an hardware, operating system, window system independent binary code format: the byte-code. The byte code is then executed (interpreted) by an other element of the Java architecture: the Java Virtual Machine (JVM) which understands byte code and execute it or translates it into the host specific binary format. This scheme permits to run the Java byte code on any machine hosting a virtual machine without any needs to update the application. The application can be written once and run anywhere. This means that the same code can be run on any hosts of the distributed systems without being modified.
At the time of this project the JVM is not integrated into existing operating systems and therefore a JVM must be loaded on the top of the operating system for each new Java application.
An other main attraction of Java is that the JVM is integrated with most existing World Wide Web (the Web) browsers (Netscape, Microsoft and Sun browsers for example) enabling Java applications, called Java applets in this case, to run directly inside the browser environment. Applets can also be downloaded over the network making the Java code mobile and therefore even more attractive for a distributed system usage. The applets also add interactive and multimedia content to the information available on the Web.
Multithreading
Objects can be multi-threaded to increase performance (through parallelism) and enable multi-access capabilities.
Dynamic loading of classes
Classes are not all loaded when the code start to be executed but loaded as they are required and can be downloaded from the network. This improves performance, as it utilises memory more efficiently, and increases the degree off flexibility and extensibility of applications as well as the adaptation to evolving environments.
Security [21]
As Java code can be downloaded over the network and potentially over the Internet, systems must be protected from running subversive code. Incoming code into the JVM is verified before it is passed to the interpreter (JVM) for execution. This is done by an other element of the Java environment: the Byte Code Verifier. The byte code verifier checks that the code does not forge pointers (change the address of memory they point at), does not violate access restrictions. Java code which does not obey the rules of the byte verifier is not executed by the interpreter. There is also a domain separation between local and remotely loaded classes by the class loader so that imported classes can not spoof built-in classes. Different levels of network access restrictions can also be set: disallow network access, allow network access only from the host where code came from [54]. …etc. Moreover memory pointers (in C++ terms) do not exist so that the memory can not be manipulated randomly.
Java Remote Method Invocation (RMI) [65]
The Java RMI gives more distributed capabilities to Java by enabling classes on one JVM to call methods from classes on a different JVM which might be on a different host and potentially anywhere on the Internet. This provides some lightweight ORB facilities, a detailed discussion of the Java RMI and CORBA is presented in [57] this is out of the scope of this project.
All the above features make Java distributed system's facilities secure over a wide public network like the Internet and enable Java code to be written once and run anywhere. Reflective Java extends the mobile code facilities of Java by enabling the code to be also run in any environment, the next section present the features of Reflective Java.
Reflective Java is a product from the ANSA research consortium [16, 17, 50]. It allows the separation of the functional concerns of an application from the non functional ones. This is done using the Meta-object Protocol (MOP) [13]. The MOP separates the roles of application developers, one focuses on what the application does and an other on how it does it. This scheme means that some requirements can be added late in the life cycle of an application, even if they had not been thought about originally. And, as is explained later in this section, requirements can be added into the application dynamically meaning that Java code can be written once, run anywhere, even in an evolving environment. One can imagine some requirements necessitating a particular implementation in a fixed network environment and an other in a mobile network environment, Reflective Java permits the dynamic update of the implementation of these requirements as the environment changes. A task of this project was to identify the differences between using Reflective Java, the inheritance mechanism in object oriented language and Application Programming Interfaces, section 4.2.1 compare these three approaches.Background on the origins of Reflection can be found in [12, 14, 51] which are the seminal documents that explains how reflection can be introduced into object oriented languages.
The following diagram illustrates how Reflective Java implements the Meta-object Protocol :
In an object oriented world, an application is made up of classes. On the right side of the model above we can see that an application uses the services of some other class. Reflective Java allows to upgrade the service of classes and still retain their original behaviour. Methods in the original class are said to be made reflective by binding them to some extra code to be executed before and/or after their invocation. The reflection class on the diagram above acts as a third party binding entity between the original class and the newly defined extra behaviour in the meta-object class.
We now define the different elements which make-up reflection. The role of the developer of meta-objects as we stated earlier is to design and implement the meta-object which defines some non functional behaviour. Examples are meta-objects implementing the well known transparencies of the distributed systems world: concurrency control [17], replication/location [49, 18, 66] persistency [17], failure [66].. etc. A main feature of the meta-objects is that they are designed and implemented independently of the application context in which they will be later used. Meta-objects are written once and can be used by many applications later. Moreover the meta-object is bound dynamically to the reflection class so that it can be changed at run time, a new version of a service can be installed at run time without the need for modifying the application code.
The meta-object defines a couple of methods, "metaBefore" and "metaAfter". The code which implements the required extra behaviour is implemented in those two methods. The extra behaviour is, in fact, what should be done before and/or after the "normal" application method execution.
The following diagram illustrates this in the context of controlling QoS over a network socket connection establishment :
The calling code invokes the create_socket method, but with Reflection, the metaBefore method in the meta-object is invoked first (2) and then the create_socket method is invoked (3) through inheritance. In this example, the metaBefore method in the meta-object makes the reservation of bandwidth and then the socket is created. metaBefore is called before the invocation of the application class original method and metaAfter after the invocation. These two methods have access to the parameters of the original method calls. The code in these two methods is independent of the class which will use them later so that the meta-object can be used for any application.
Reflection also provides some more flexibility by allowing
the programmer to make only some of the application class methods reflective.
A script language is used to specify statically which meta-object is associated
with which application class and which methods of the original class are
to be made reflective. As we stated earlier the meta-object associated
with an application class can be dynamically modified. This script language
is as follow:
refl_class original_class : meta_qosrsvp{
public original_class():
1;
public void create_Socket(int
ip, int port): 101;
public void release_Socket(int
handle): 102;
}
The refl_class keyword indicates that the original_class is to be made reflective and that it will use the extra behaviour defined in the meta_qosrsvp meta-object. The numbers appearing on the right side of the method signature are category identifiers (these are explained later). It is the role of the system integrator, who has knowledge about the application and the meta-objects to decide what extra behaviour is required for which of the original methods and to write the appropriate script file.
The identifier numbers are used to create different sections of code in the meta-object. The identifiers are used in the script file to specify which section of the code is to be run for which application method. An example is given bellow
:metaBefore(param,
category_id){ if(category_id==101) reserve_bandwidth(…); … }
metaAfter(param, category_id){
if(category_id==102) release_reserved_bandwidth(…); …}
With the example meta-object and the script file example above, the original create_Socket() method will cause the establishment of some bandwidth reservation using RSVP before the socket is created and the communication handle returned. The release of a socket will cause the reservation to be released after the socket is freed. The code in the application only deals with socket programming and the meta-object only with QoS control. The script file is used to bind the two together.
Finally, the Reflective Java pre-processor is run over the script file and produces the reflection class (cf. figure 3), which implements the same methods as the original class wrapped with the extra code provided by the meta-object. We recall that the reflection class inherits the behaviour of the original application class. The application then needs to be updated manually by the system integrator to instantiate socket objects which require QoS control from the reflection class and not the original socket class. This pre-processor is written in C++ and only runs on Solaris and HP-UX. It produces Java code which can then be used on any operating system, so the operating system limitation is minor.
This implementation of the reflection technique in Java is made possible by one of the feature of Java that we described earlier: dynamic class loading. Other features of Java such as network download of classes enable this scheme to be very flexible and place Reflective Java ahead of some other reflection implementation described in [12, 13, 14]. Most of those implementations do not support the dynamic facilities that Reflective Java offer.
Readers interested in the internals of Reflective Java can find a definition of the reflection class in [16]. The previous cited documents also present the reflection techniques in depth and the use of meta -data in object oriented languages. Other implementations of "reflection" and meta-data programming in Java can be found in [18, 49, 55, 56].
In this work we attempt to use the flexibility of Reflective Java to separate QoS control of an application from the functional requirements of the application: sending multimedia data over the Internet. To conclude this introduction of concepts section we describe the application we use in our work for testing purposes.
Web2Talk [29] is used in this project as a test application for the implementation work we have produced using Reflective Java and RSVP. It was chosen completely at random (in fact it was the first real-time Java application we found) on purpose, to illustrate how the techniques we planned to use were application independent. Web2Talk is an Internet one to one telephony application written completely in Java by three M.Sc students as part of their final year course at Oregon state university. The application only runs on Solaris workstations as it accesses the audio device directly as a file (/dev/audio). It uses the Java Socket class to communicate data over the Internet using TCP and a Synchronisation server is used to keep track of the port number used by the participants to allow any participant to communicate with any other. The application provides the facility to register any participant. Then one can invite an-other (each participant is presented with a list of all participants from which he can invite any to communicate, see figure 5). If the invitation is accepted then the required communication channels are set-up for audio exchange between the two participants. The following is a snapshot of the Web2Talk graphical user interface :
The synchronisation server was originally running on a host at Oregon university meaning that the application could not be used by participants located within a firewall. To solve this problem, we obtained a copy of the server which is now running inside our firewall. Web2Talk clients have been modified to access this new server and not the one at Oregon university anymore. This of course means that, without some trivial further work to make the server variable, we can only test the application and our work within our firewall. This concludes the presentation of the pre-requisites to our work which the next sections describes.
The objectives of this project are the evaluation of Reflective Java in a QoS management context and the investigation of Internet bandwidth reservations using RSVP. To achieve those objectives we needed a prototype environment using Reflective Java and RSVP. The first part of this project was therefore dedicated to designing a QoS management architecture [41, 43, 45] for Java applications. In this section we report and discuss the architecture we designed and its implementation is described in the next section. The rational for the architecture is achieved by describing alternative approaches that were considered but rejected.The architecture we designed provides a generic (adaptive) QoS control component [59, 60, 61] which enables the best use of the host and network resources. We have named our architecture QoteS - Quality of Service around the edge. QoteS can be added to any existing Internet communication application written in Java without having to know the detailed internal mechanisms of the application and can be used to replace a deficient QoS element as well as provide reservation when it becomes necessary. Charging issues are also considered in QoteS. QoteS controls the network and host resources and provides a QoS control interface to the user for him to express his preferences.
In this section we first describe a typical Java Internet application then we present the limitations in the current ways of introducing QoS control into applications and finally QoteS is described and discussed.
The following model describes a typical Internet application written in Java.
The application through a communication class (Socket, ServerSocket, MulticastSocket, URL…etc.) interacts with the network. The application sends and/or receives data through the communication channel that this communication class manages.
Today's Internet communication applications, written in Java or any other language, provide functionality for sending the data and may provide some facilities to control the quality of service of the data sent. Typical examples are the vic and vat video/audio conference tools which provide a control panel for the properties of the multimedia streams. But those applications do not make use of any network or host resource reservation protocols (although some of them such as Vosaic or QuickTime Conferencing provide some form of QoS adaptation to network congestion in a best effort delivery environment). Therefore as the network or host becomes congested the quality of the communication decreases due to the unreliable nature of the Internet best-effort data transfer protocol and the lack of resource control in current operating systems. Also some work is currently being done in this latter area [43].
Moreover as QoS control is provided on a per-application basis the sharing of the resources available to the user and the sharing of network resources between users might not be optimal and QoS adaptation can only be done on a per application basis, which again might not be optimal. Another downside of mixing the code of the application task and the stream control is that it is then more difficult to understand the code of the application and to modify only the stream control part of application.
Our model helps towards removing the limitations of the current ways of integrating QoS control with real-time multimedia streaming applications by using Reflective Java. It achieves QoS control by providing guaranteed host and network resources and giving the user flexible control over his own resources. It enables efficient sharing of network and host resources among users by using the cost of QoS to create the market to temper demand. Our model provides a clear cut between the application and the QoS control. We also believe that adaptation can be more efficient as information across all real-time applications is available to the adaptation protocol. The QoS control element can be developed by a professional and as a result be more efficient and the use of reflection in the model reduces the time-to-market.
We now describe our architecture. The diagram on the following page describes our model using the BOOCH object oriented method [52]. Run time properties regarding the model are presented on figure 10.
This architecture is for the receiving hosts of the real time streaming client-server paradigm. QoteS "lives" on a single host and controls the resources for this single host. The architecture is able to distribute and control the resources of all the applications which are registered with it. The user has access to a QoS control interface which he can use to prioritise applications and streams by specifying the amount of money he is ready to spend on each of the streams and applications. Charging [33] is used throughout the model as a common denominator to make decisions on resource control. If interaction between copies of QoteS running on different host is required (maybe for control control of the resources used by a distributed receiver) then the Java RMI facilities would be used. We can divide the diagram on figure 7 into the following three parts :
In our design we have distributed the elements controlling different aspects of the QoS control so that they can be easily individually modified as the individual protocols they use get more mature and developed. We identified required control over network and host resources so we have designed separated objects for each of those. The design of the reflective architecture is based on the rules of Reflective Java and the application model is purely imaginative but illustrates correctly the traditional model of Java Internet applications. A set of separate objects is defined for the budget control for security and trust raisons and to allow us to use the results of further work to be produced by the group. Finally the QoS manager and user control objects are separated to support the need for many user QoS GUI controls and a single QoS manager to share resources between them all. The QoS manager uses the services of the budget control objects and resource control for QoS decision purposes and to make resource reservations.
We now describe the model by describing the classes involved in each of these three parts.
The original application (that is without QoS control) would only use the communication class (cf. Figure 6) and not the reflection class (as it does on figure 7). The reflection class is the reflective element that allows us to include QoS control in the application with minimal modifications to the code.The aim of our model is to minimise the need for code modification when QoS control is to be modified, upgraded or added to an application. During the design of our architecture, we found diverse ways of meeting our objectives by the use of reflection at the application level. We try to describe some of them in the following few paragraphs.
The result of the reflective architecture is a new communication class: the reflection class which meta-object provides QoS control through reservation both at the host and network level. As we stated earlier in section 2.3 where we described Reflective Java, the application programmer must modify the code of the application so that when a reliable controlled stream is required the communication object is instantiated not from the original unreliable communication class but from the new reflection class which implements QoS control over the communication channel.
This is constraining as the application code needs to be updated manually, but very flexible having done this once, as the meta-object can be dynamically modified at run-time if QoS control needs to be upgraded or replaced. Moreover the developer only needs to know where the communication channel is instantiated to modify its type to the reflective class. If reflection was to be added to the application when it is first designed, so that a reflection class already exists but it is bound to a dummy meta- (object which does not process anything) then dynamic modification of the meta-object could be use to set the appropriate QoS control meta-object. The application would then not require to be modified and the system integrator would need to know very little about the application.
Still, the application needs to be modified, which is even more constraining as it is done manually (however, a modified class loader could be used to avoid this). The main issue during this process is to decide which communication objects require QoS control as not all streams need timeliness; some are signalling streams for which a best effort delivery scheme is very well suited. We identified the following options:
Another important issue is how the QoS attributes of a stream can be defined on the behalf of the application. We use the RSVP PATH messages to discover what is the expected generated traffic from the source. Then the QoS manager ,with inputs the host resource controllers and possibly the user, decides what QoS to request for a given application data stream. The RSVP control is not done by the application (RSVP messages are in fact hidden from the application) but by the QoS manager. This solve one of the issues described in [15].
Thanks to Reflection and Java mobile code, the application could adapt itself to a changing underlying environment. An application originally might not require QoS control through reservation as proposed in our model because a best effort delivery service on a non congested network/host might be sufficient. But as the network/host becomes congested then reservation and policy control becomes required to provide real time services. The appropriate QoS control set of objects for a given environment could be downloaded through the network as well as the appropriate meta-object and be added to the application by using reflection, at run-time, without having to update the application code. When the environment changes again, e.g. from a fixed to a mobile network this time, a new set of QoS control objects and the appropriate meta-object could again be downloaded and the application updated.The reflective architecture is made up of the Reflection class and the QoS meta-object class. The role of the reflection class is to implement the binding, defined by the system integrator, between the communication class operations used by the application and the QoS control related calls in the QoS meta-object. Recall that the reflection class is produced automatically by the Reflective Java pre-processor. We therefore only discuss the design of the meta-object.
The following application network calls should be made reflective to use the services of our meta-object :
One of the main issue regarding the design of the meta-object is the delay between the time when the application requests a network connection or joins a multicast group, and the time when the connection handle is returned to the application. In a best effort environment this would simply mean joining the multicast group. But with QoS control added, and especially in the case of bandwidth reservation on the Internet using RSVP, this call means:
The reflective part of the QoS architecture allows us to evaluate the suitability of Reflective Java for abstracting the QoS management functions from the applications. The second main part of QoteS is the set of objects which control the resources available to the host and users (host plus network resources).
Through our study we have identified that the following elements should take part in the QoS decisions process :The main objective of this control is to provide a single GUI to the user, using a single unit of sharing that he can use to express priorities between his processes. This unit of sharing being money.
Adaptation here is made by the user who monitors (through human perception) the applications he is using and reacts by making decision about what to modify to get the appropriate QoS for what he requires using the interface to express those decisions. Therefore adaptation deals with all the attributes the user can control , which in our case is money.
There are some implementation issues regarding how this object can communicate with the QoS manager or the user budget agent as they are located on different Java Virtual Machines. Issues regarding communication between Java virtual machines [57, 58, 62] are described in section 6.
The next section describes more the budget control issues of the model.
The elements involved in budget control are the user, the user budget-agent, the network quoting engine and the host quoting engine.
The following diagram illustrates the message processing between all those elements. It also highlights what interfaces are required on each of the elements involved in budget control.
First the user defines an amount of money he is prepared to spend on a per period basis (1): £n a month, this value is then recorded by the user budget-agent. When the QoS manager receives a request for acquiring or adapting some QoS, it sends messages, which include the requested QoS properties, to the quoting engines which reply with a quote on how much the required QoS would cost (3, 5). The QoS manager role is then to merge those quotes to produce a final quote for the user budget-agent (6). This final quote is then accepted (7) or refused (8) by the user budget-agent according to some rules defined by the user and the available budget. In the case of a positive response a token is given to the QoS manager (7) to allow him to get the specified QoS with the specified quote. The budget is decremented to an interim value. A QoS monitor then needs to monitor what QoS is really delivered and, using the quotes already received tell a payment object to accommodate the expenditure accordingly and inform the user budget-agent (9) for it to modify the interim budget balance. In the case of a negative response from the user budget-agent, the QoS manager should find an other QoS specification, which has a lower price.
This design work on charging is a "simple" prototype of some more detailed work being done in the wider BT project We now describe each element of the budget control model of QoteS. The user budget-agent is responsible for the control of the user's funds. It should include at least the fund available for a given period for each user. It must provide operations for the user to set and check his available budget and a operation for the QoS manager to authorise (or not authorise) a quote. QoS Adaptation occurs when the rate of usage of funds is greater than the budgeted allowance and his resource reservation must be updated in consequence. A management interface should also be provided to the user budget agent for statistics collection. We therefore identified the need for the following services from the user budget agent :
The role of this object is to provide, from a server behaviour point of view :
It must be able to take input from the elements we defined as having a role in defining the QoS requirements, this is :
The following table shows the envisaged number of objects
running at one time in various scopes using QoteS.
| Receiver (application) | 1 per host unless a distributed receiver (e.g. a video wall for network management) |
| Host QoS manager | 1 per receiver (usually = host) |
| Network quoting engine. | 1 per network ? |
| Host resource manager | 1 per host |
| Host quoting engine | 1 per host |
| Users | N per host |
| Budget user agents | N as there is 1 per user |
| User QoS control interface | N as there is 1 per user |
| Real-time Communication objects | M as there is 1 for each real-time communication channel |
| Reflection objects | M as there is 1 for each real-time communication object |
| QoS Meta-object | M as there is 1 for each reflection object |
The main aspect to understand is that there is a single QoS manager per host which controls many users and their many applications. The following diagram illustrates the communication paths between the objects of our model at run time, this is on a per host basis in a distributed system. This diagram also presents some implementation issues regarding JVM instances. For more details about each of the objects on the diagram, refer to section 4.3.
So far, different virtual machines (VM) have to be loaded for each application and we have identified the need for many virtual machines One virtual machine for each user application and one for QoteS as shown of the above figure. There is a requirement to communicate between those virtual machines (shown by the "??" on the figure above) and this with the current Java technology would be done using the RMI API. But RMI is not optimised for communication between JVMs on the same host. We use a the JavaShell utility written at BT research, which permit the loading of multiple applications on the same virtual machine. We also want to be able to communicate between virtual machines owned by different users. The QoS manager VM is owned by the system and the user QoS interface VM is owned by each individual user. The JavaShell would work fine for a single user host but not for a multi-user host where all users contact the same QoS manager. Nevertheless it could be used by simulating the different users. Our decision so far is to use the JavaShell as it seems that current research work in Java operating systems is aiming at having a single VM per host [20]. We decided to provide a system based on a single user (a typical PC type host) in the short term [21]., as the added complexity of a multi-user VM was outside the scope of the project. Similarly, we decided to ignore the possible need for one logical QoS manager across multiple hosts acting as one logical receiver.
One other implementation issue is that if we decide to make the communication class reflective we have to choose which one(s) we want to make reflective, as Java defines many communication classes: Socket. MulticastSocket, Datagram or URL.
In addition, current implementation of the RSVP API are only available in C or C++ and QoteS is based on the use of Java so we need to find a way to bridge the gap between the two languages.
All these issues are discussed in the following section, which describes our implementation work.
In this section we describe the implementation work we have done on the model we have designed. Priority was given to implementing the overall architecture in order to test our design and evaluate Reflective Java. The result is a working prototype of the architecture which we have used with a modified version of an Internet telephony application: Web2Talk and a custom client/server application we have implemented for testing purposes to overcome some limitations of Web2Talk. We have implemented thirty-five classes in three thousand lines of code (Java and C++) which cover the following :
This data structure holds the list of the applications a user is currently using that have, at least, one object of type reflective socket. (Reflective socket is the application class that we have decided to make reflective, cf. 4.1.3.). For each application this data structure also holds the list of all the streams that are of type reflective socket (not the streams instantiated from the socket class as they do not require QoS control, cf. Section 4.1.3). The priority field in the application and stream class holds the current defined priority (by the user) of the application or stream. For example if a user is using two applications which have QoS control, one might have a priority of 30 and the other 70 out of 100, this is governed by the user QoS control interface described in a later section. The "maxNetQoS" field in the Stream class holds the QoS defined in the RSVP PATH messages for this stream. It is the maximum QoS as reserving more bandwidth that what the RSVP PATH messages advertise would mean wasting bandwith. The currentNetQoS is the currently reserved bandwidth for the stream.
The registration of users, applications and streams is done by calling the following function of the QoS manager :public static void doqosctrl(String appid, String userid, String hostid, int portid)
This function registers the specified user, application and stream only if they have not have yet been registered. This means the stream always gets registered but not the application or the user. This function also launches a QoS control interface for the user and a user budget agent when a new user is registered. If the user is already registered then the interface is only updated with a new application or reflective stream instance.
The QoS manager is also responsible for launching the different objects involved in the QoS control: the host resources control object and the quoting engines. To visualise this, we have implemented a QoS manager state "panel". The following figure is a snapshot of this panel.
The QoS manager is also responsible for issuing the network reservation calls using the JAVA RSVP API we have implemented.
The QoS manager also provides services for the user QoS control graphical interface. The methods ,implementing these services, essentially enable the user, through his graphical control interface, to modify the priorities of streams. At this stage we have not implemented the sharing of resources between applications but between streams only.
Another main function of the QoS manager is to decide on the host and network reservation attributes for the streams of the applications it controls. It is also the QoS manager which issues the reservation calls. The QoS manager interacts directly with the RSVP API we have implemented to make network bandwidth reservations and it interacts with the host controller to make host resources reservations.
The QoS manager gives resources to the stream with the highest priority first and then distribute the resources between the streams with lower priorities. The implementation of this QoS decision process revealed the complexity of the subject and we identified many alternatives to the one we implemented. The following diagram illustrates the element involved in this decision process.
Interaction of QoS manager with the budget control elements as defined on figure 7 has also been implemented, but without the last element of the interaction : actual expenditure. We have not provided a monitoring object but have provided scope for integration of QoteS with the monitoring object which is currently being developed in the group.
We have implemented some more classes which implement the QoS attributes. We have defined a class for the network QoS and a class for the host QoS. We have also implemented a class for the quote attributes.
Recall that the Java shells allows us to run more than one Java application on the same JVM therefore simplifying the communication between them. The above picture is a capture of the JavaShell graphical user interface. The class name field is used to specify the name of the class to run and the command line field to specify any parameters which the class main method accepts. The bottom half of the window will display the list of currently running classes and it will be possible to stop the execution of single classes, but at this stage this is not implemented.
We use the JavaShell to enable the applications on a host to interact with QoteS. The QoS manager in QoteS is a static object which can be referenced by any other object within the JavaShell by simply using the name of the QoS manager class as the object identifier. This a simple and efficient way for the meta-objects bound to the reflection class used by the user applications to interact with the QoS manager.
Using this GUI, the user specifies the importance of each stream (application) relative to each other. Also control is given to set priorities both for applications and streams, the QoS manager only deals with sharing resources over streams.
The lock/unlock button disable the use of the currently selected slider. When a slider is locked, the user cannot modify the priority of a stream anymore. The objective of the "proceed" button (not yet implemented ) is to reduce the amount of messaging between the user graphical interface and the QoS manager is QoteS. With the current implementation, on every move of one of the slider bars the new defined priorities are transmitted to the QoS manager which then modifies the reservations accordingly. This control is the subject for further work, we have already identified the following issues and different alternatives regarding the facilities it would provide :
To be in line with the wider BT research project within which this project took place and because of Reflective Java and the advantages of Java for distributed systems we described in section2.3, we decided to implement QoteS in Java and use Java as much as possible throughout the project. In order to communicate with the RSVP daemon on the host, the QoS manager must use one of the RSVP APs available. We decided to use the RAPI implemented by the IETF as it is widely and freely available but like all the RSVP API available it is a C implementation of the protocol interface. As a result we had to port this code to Java to be able to use it in our project. Before we describe this porting work, we explain how the RAPI is used to control bandwidth reservations on the Internet. The following two diagrams illustrate the order with which the RAPI functions must be called to establish bandwidth reservations on the Internet.
First a rapi_session call must be issued (t0) both at the receiver and sending end(s), this call creates a new RSVP session. A RSVP session is identified in a unicast environment by the destination IP address and port number pair and in a multicast environment by the multicast IP address and port number pair. The session call returns a session identifier (specific to an API instance) which is then used in later calls to identify a session. A single API instance can be used to control many RSVP sessions. The sender application(s) then issue a rapi_sender call (t1) which includes the QoS attributes of the sender stream, those attributes are then encapsulated in a RSVP PATH , by the RSVP daemon, message which is send to the receiver(s).
The receivers on reception of the PATH messages can then decide on what bandwidth to request for and issue reservation calls using the rapi_reserve() function (t3) specifying the requested QoS .This call produces (by the RSVP daemon) RSVP RESV messages which are sent along the same path as the RSVP PATH messages and establish bandwidth reservation in the routers and hosts along this path. The sender(s) receive notifications of those RESV messages.
To overcome the programming language difference between the RAPI written in C and the other parts of the implementation written in Java, we have written a Java subset of the calls available in the IETF RAPI using the Java Native Interface (JNI) [24] which permits to call C/C++ code from Java. The result is therefore a Java RSVP API which is not an implementation of RSVP but calls functions of the IETF C implementation. When native code was required we used C++ and not C to provide a more object oriented solution. The following diagram represents the implementation details of the Java RSVP API we wrote.
As we stated earlier it is still the RAPI which interacts with the RSVP daemon but the application (QoteS) uses Java calls to interact with the RAPI. Nevertheless, the Java RSVP API does not call directly the RAPI because of the way the JNI works. With the JNI, Java methods are written first and then a special version of the Java compiler is run over the code and produces a C header file (.h) which prototypes a C version of the previously declared Java methods, type translation is provided by some JNI types (int becomes jint etc…). The functions declared in this header file can then be implemented in the native language. But in our case we had the native code (the RAPI) before the Java methods were written, this is why we had to write a third level of abstraction in C++ (as illustrated on figure 16). We first wrote a Java class with the RSVP methods we needed, the following line is the declaration of the session creation method we defined in this class :public native int session(String hostid, int portid, String protocol, int err, int end);
We then used the JNI to produce a header file in C with the native declaration of the Java methods. The following is the automatically produced C prototype of the above Java method :JNIEXPORT jint JNICALL Java_rapi_session (JNIEnv *env, jobject obj, jstring hostid, jint portid, jstring protocol, jint jerr, jint rcv)
We then implemented the body of the function in C++ and this is where the functions from the RAPI are called. The following lines represent a subset of the implementation of the above function
.…//the following translate the call parameters in C++ var
const char *str = env->GetStringUTFChars(hostid,
0);
const char *str2 = env->GetStringUTFChars(protocol,
0);
…//Call the RAPI session call
_sid=rapi_session(&dest,
ipproto, 0 , async, NULL, &err);
…_fd= rapi_getfd(_sid);
//get the fd for daemon control
pipe
//now we need to listen for
events on this file descriptor
if(rcv){//following only for receivers
fd_set fds;
//set of file descriptorr we want to monitor
FD_ZERO(&fds);
//empty set FD_SET(_fd, &fds);
//add the rsvpd control pipe to the set
if(_DEBUG) printf("\nnative session call: do select call");
tm t; t.tm_sec=1;
//time out, if no PATH msg available after 1
//seconds then give up
//wait for a read possible on _fd, time-out
int nb = select(32, &fds, NULL, NULL, (timeval*)&t
);
if(nb>0) //else it mean interuption occured or time-out,
nb = rapi_dispatch(); //dispatch to handling function…
One other aspect of RSVP programming using the RAPI is the call-back method which is associated with a session. In the session call one parameter is the signature of an application level method. This method gets called when RSVP messages, as we described them in section 2.1, arrive at the API. The purpose of the call-back method is to identify the type of the message and act in consequence. In the case of PATH messages received for example, reservations should be issued using the attributes of the PATH message. The monitoring of messages arrival at the API is done by monitoring a socket using the select() unix system call. The socket identifier is found by passing the API RSVP session identifier to the RAPI get_fd() call. The select call blocks until a read event is detected in which case the RAPI dispatch() call is issued which calls the call-back method with appropriate parameters (RSVP message, environment etc…).
We also wrote in Java a pair of applications using our API for testing purposes. One acts as a sender and the other as a receiver. The following is the code of the sender application which creates a session and then use the send method to distribute RSVP PATH messages.
//A simple test class for the
java rapi
import rapi;class sender {
static boolean _debug = true;
public static void main(String
args[]){ rapi r = new rapi(); int errorno=0; int sid = r.session(args[0],
5678, "IPPROTO_UDP", errorno, 0); //0 for sender if(sid>0){ System.out.println("session
created"); int sdres = r.sender(sid, 7000, 6, 1500, 3000, 2000, 2000, 1500);
} else System.out.println("Problem on session establishment");
while(true);
}
}
This RSVP Java API is used by QoteS to control the bandwidth required by the application streams. Tests were made difficult by the immature nature of the RSVP implementations and the network environment available for this project. Nevertheless, we manage to create sessions, send and receive PATH messages in a multicast environment (but not unicast) and therefore the full logic of QoteS is implemented although it was impossible to establish reservations. This problem will be solved in the coming weeks by moving to a controlled environment where RSVP is fully operational but difficulties will probably be encountered because of RSVP message version incompatibilities between implementers (Intel, CISCO, IETF etc.…) on the hosts, the routers and between demons. The essence of the project was therefore not harned as the aim was to evaluate reflective techniques for QoS control and not evaluate RSVP, furthermore our aim is to provide a QoS control element which is adaptive and therefore independent of any network layer protocol. We decided not to spend too much time on trying to resolve an implementation problem which does not affect very much the evaluation of Reflective techniques for QoS control, furthermore receiving the RSVP PATH messages was more crucial and has been achieved.
In this section we describe the use of Reflective Java in our implementation work. One important aspect was to decide how to make use of Reflective Java in order to define the implementation aspects of the required meta-objects. We gathered the following set of requirements from the design stage and used them to make our decisions :In the following paragraphs we describe our implementation of the meta-object and the binding specification file which lead to some unexpected implementation work due to some limitations of Reflective Java. The result is a meta-object which interacts with the QoS manager in QoteS and a reflection class for the Java.net.Socket class [30].
Implementation of the Meta-object
As we wrote earlier, the aim of the reflective call is to call the QoS manager in order to register a stream. We therefore must assume that the Internet addressing properties of the stream are available in the parameters of the method being reflected upon. This of course reduces the portability of the meta-object as it can only be bound to application methods containing those parameters and in the same order as we define they are in the meta-object. This is particularly well suited to application methods which create a new stream as we can assume that the addressing properties of the stream are the minimum amount of information available as parameters due to the nature of socket programming on the Internet. This is where we argue that we felt our implementation was driven in part by our knowledge of the applications methods which could use it. The following is a subset of the code of the meta-object, this is the code used to contact the QoS manager and register a stream.
c_id==101) { //for reflective
selector 101 : creation //Call QoS manager
to deal with this new stream //The following accesses parameters and therefore
the meta //object is bound to be related to application methods with the
same //parameters
//this section extracts the parameters from the reflected
method call InetAddress par1 = (InetAddress) args.extractArg(0).extractObject();
int par2 = (int) args.extractArg(1).extractInt(); String hostid = par1.getHostName();
//call qos_manager to register the stream //names are
hard-coded for now to avoid distraction from primary task (see //later)
qos_manager.doqosctrl("Web2Talk","jtassel",hostid, par2);
//for test purposes we just make many of those calls
with different //application name and user names }//if
We decided to use the metaAfter meta method, reasons for this choice our detailed in the following section on the reflection class. Multithreading could be considered in the meta-object also it is not directly supported by Reflective Java. A separate thread could be launched to call the meta-object so that the Socket call at the application level could return immediately and RSVP control would be done on the side.
As we use the JavaShell to have a single Java Virtual Machine (instead of using the Java RMI as it is more lightweight) to communicate between the applications and QoteS, the QoS manager in QoteS is a static variable which can be called from any class running in the JavaShell environment simply by using the name of the QoS manager class.
In the call to the QoS manager, we could use system calls to discover the application name and the user name but then the code would not be portable anymore, this is one of the weakness of Java, not many system calls are available. We wrote the application name and user name in the call to the QoS manager for simplicity and not to have to make more JNI programming, for test purposes we simply make many calls to the QoS Manager with different application and user names. Nevertheless we do realise that for a final release of our prototype we would need to include some system calls (we can use the unix getuid() and the /etc/passwd system file to get the username and a call to getpid() and a redirected ps unix command result to find the name of the application)
We have used the selector facility of Reflective Java to differentiate the code required to register a stream with the QoS manager (selector 101) for QoS control and the calls to remove a stream from the QoS manager (selector 102).
In the model we designed the role of the sending application to send data and the appropriate RSVP PATH messages. The Java RSVP API could be directly used in the sending application to distribute RSVP PATH messages but the modifications required to the application would be of a same level of complexity as implementing a meta-object which provides the same service. Furthermore, having a meta-object to deal with RSVP on behalf of the application would mean that this meta-object could be replaced in a different environment with a different meta-object which deals with some other reservation protocol, ATM signalling (Q.2931) for example, without having to modify the application. A similar infrastructure as in QoteS could be used at the sender.
The next section describes the binding specification file and the modifications we had to make to the automatically generated reflection class due to some limitations of the current version of Reflective Java.
Coding of the Binding specifications
We decided to use the Socket class as the host class for reflection as it fits with the implementation of the meta-object (it has the correct list of parameters) and it is the standard class used in Java Internet applications. Using the Socket class also means that any application written in Java using the Socket class can make use of QoteS with very minor modifications as a new reflection class does not need to be created for each new application which requires to use QoteS. We decided to use the Socket constructor to register the new streams with the QoS manager, which then does QoS control on behalf of the application, and to use the destructor of the Socket class to release the reservations for one stream. These decisions lead to the following binding specification file. (we only show the constructor case)
import java.net.*; refl_class Socket : meta_qos { public Socket(InetAddress address, int port) throws IOException, Throwable : 101;… }
The refl_class keyword is followed by the name of the application class which is to be made reflective (Socket) and will therefore be the base class of the reflective class. The name of the application class (Socket) is followed by the name of the meta-object to use originally in the reflective class but this meta-object can later be replaced for another one at run time.
At this stage we experienced some of the limitations of the Reflective Java pre-processor. The pre-processor normally creates a new file for the reflection class and writes the implementation of the new version of the methods listed in the binding specification file. What we expected was therefore a new implementation of the Socket constructor which would keep to the following rules :
Implementation of the Reflection class
We had to modify the code so that not only would the meta-data be initialised and the original Socket constructor be called, but the meta methods from the aggregated meta-object would be called as well. However due to some limitations of Java, it is impossible to fully achieve this objective. Java does not allow the addition of any code before the call to the base constructor in a specialised class, the first line must be a call to super(…) which calls the base constructor. This means that when a constructor is made reflective the metaBefore method cannot be called and therefore only the metaAfter functionality of the aggregated meta-object can be used. This is the reason why the call to the QoteS QoS manager is located in the metaAfter method of the QoS meta-object we have implemented and not in metaBefore. See the table in section 4.2.3 for a suggested solution to this problem.
The following is a commented subset of the code of the reflection class that we had to modify manually:
public refl_Socket(InetAddress
address,int port) throws IOException, Throwable {
//Call original ctor super(
address, port);
//Grab argumentsArgPack args
= new ArgPack(); try{ Any _arg_1 = new Any(); _arg_1.insertObject((Object)address,
"InetAddress"); args.insertArg(_arg_1); Any _arg_2 = new Any(); _arg_2.insertInt(port);
args.insertArg(_arg_2); } catch (Throwable tex) { System.out.println("Exception:
insertArg failed"); System.out.println(tex.toString()); }
//set meta-object for this meta_object
= new meta_qos(this, "Socket"); active_metaclass = new String("meta_qos");
init_metadata();
//call metaAfter ArgPack _rslt
= new ArgPack(); String exception1 = meta_object.metaAfter(2, 101, args,
_rslt);
//Deal with Xceptions if(exception1.compareTo("noException")!=0)
{ if (exception1.compareTo("Throwable")==0) { throw new Throwable(); };
if (exception1.compareTo("IOException")==0) { throw new IOException();
}; }
};
The result is therefore a Java.net.Socket Reflective class, which has only the metaBefore functionality of the meta-object it is aggregated to. This class can be used in any application which needs QoS control for streams with timeliness requirements instead of the Java.net.Socket class.
We also experienced some naming and version control problems. The Reflective Java pre-processor does not allow the specification of a class name from the Java APIs as the base application class for the reflective class in the binding specification file. As a result we had to name the class Socket and modify the automatically generated import statements in the Reflective class code to import java.net.Socket; instead of import Socket;. In addition, the Reflective Java pre-processor always creates the implementation file of the reflection class under the same directory and file name. This is something we had to watch for each re-run of the pre-processor in order not to loose the manual modifications that we made to the reflection class. See the table in section 4.2.3 for a suggested solution to this problem.
In order to test QoteS, we needed to have a real-time Internet application written in Java., for which the code was available. By real-time we mean an application using some data having timeliness constraints such as video or audio. This proved to be a task more complicated than it originally seemed as few applications existed, at the time of this project, which fulfilled these requirements. We found only one application: Web2Talk [29], a one-to-one telephony application for the Internet for which the full code is available. Web2Talk was written by Roger Klein, Carsten Schulz-Key and Stephane Chatre as part of a final year M.Sc. project at Oregon State University in march 1997 and we already described its functionality in section 2.3.The modifications to Web2Talk were very minor. In the code we had to find where the socket objects were instantiated. Then we decided whether or not this socket was used for receiving audio data (not sending, as we have only implemented a meta-object dealing with the receiving end) rather than just signalling between the client application and the synchronisation server. We decided to provide QoS control to the audio streams only as there is no timeliness requirement for the signalling streams. This also means that the QoS manager in QoteS only knows about streams which do require QoS control and not any other. It also means that some streams can be left intact and do not suffer side performance effects from the run-time reflective path described in section 2.3.2 We only had to modify two lines of the code for Web2Talk: we had to modify the type of the socket object for audio reception from Socket to refl_Socket and the line where it is actually instantiated (new Socket(…) is now new refl_Socket(….)).
These changes were made easy because of the clarity of the code and the available comments. It was therefore as expected a trivial effort to modify the application in order to provide QoS control using Reflective Java and QoteS. Moreover the fact that Web2Talk was written entirely without the knowledge of our work also proves that Reflection allows the separation of concerns between the application mechanisms and policies. This is definitely a major outcome of our work regarding the evaluation of Reflective Java.
Nevertheless we have to point out the fact that this task would be made much more complicated by unstructured code and dynamic creation of objects rather than static as we explained in section 3.2. We also think that some automated process should be available across all source files (cf. find and replace as in word processors) in the case where all objects are to be made reflective, instead of having to manually update the code.
Those modifications and the implementation of QoteS lead to the prototype environment which is presented on the following page.
The environment is based on two hosts. Host 1 acts as the sending host and host 2 as the receiving host. QoteS is therefore used on host 2. The Web2Talk application is present on both hosts and has been modified to use QoteS. Our custom sender/receiver application is used to send RSVP PATH messages on the behalf of Web2Talk and receive them as well for debugging purposes. . The IETF rtap application can also be used to send PATH messages for debugging purposes. The IETF RSTAT application is also used (also not present on the diagram) to visualise the messages as they go through the routers and hosts.
In this section we present our evaluation of Reflective Java based on the design and implementation work we have done. We also evaluate Reflective Java in the context of QoS management. Reflective Java 1.0 is an innovative and promising product, however due to its relative immaturity it has deficiencies in some vital areas. We first present some general comments about Reflective Java, then we present the advantages, drawbacks and possible solutions of the main current problems with Reflective Java and finally we discuss the suitability of Reflective Java for QoS management. One of the first questions we tried to answer regarding Reflective Java was: What is the difference between using inheritance and Reflective Java ? One could imagine that inheritance could be used to implement a new application class, based on an original application class, which implements some extra non functional code required by the application. We came up with the following answers which favour Reflective Java over the use of inheritance:The main advantage of using Reflective Java is that it allows the programmers and designer of a networking application to separate the concerns about the transmission mechanisms of a networking application from the policies and mechanisms to control its QoS. This makes it very flexible in terms of re-usability, later upgrades, configurability and adaptability. The following points outline the advantages of using Reflective Java in QoteS :
| Pre-processor treats constructors (ctor) and destructor (dtor) methods differently (the "meta-methods" of the meta-object are not called) from modifier or selector methods. | It is complex to add reflection to ctor or dtor methods. | No performance overload on construction or destruction of objects. | Modify the pre-processor so that it treats the ctor and dtor method differently and call the metaAfter method for the ctor method and metaBefore for the dtor methods | |
| Reflective class code is available and it is sometimes required to manually modify it |
|
The reflective class can include more functionality and it makes the reflection scheme more flexible. Permits the ctor and dtor to be made reflective. | If the above modifications are made then the reflection class code can be hidden from the programmer and the class file (.class) released without the implementation (.java) file for it | |
| Due to limitations of Java itself, the ctor, even if the reflection class is modified manually, can only use the functionality of the metaAfter method and not metaBefore | Limited use of the meta-object aggregated with the reflection class. | Reduces performance overload of ctors and dtors | No code can be inserted before the super call as no actions can be performed on an object before it is constructed. This is an object oriented programming principle and is not specific to Java. Modifying this would therefore require modifying the OO principle. | |
| When specifying the application class in the binding specification file, the pre-processor does not recognise Java API's classes and produces naming errors | Requires manual modification of the reflection class code | Modify the pre-processor so that it recognises Java API classes. The pre-processor should recognise standard Java API class names of the from java.net.ClassName. | ||
| Reflection is at the class level and not object level |
|
Simple static modification of the application code | Two alternatives :
|
|
| No built-in multithreading, single meta path |
|
Simple | Use the selector functionality to chose a "meta path" and make use of multithreading in the meta-object to implement different "meta-paths". | |
| The meta-object has access to the parameters of the calling application method |
|
Useful access to the application method parameters | Give read only access to the reflected application method ? | |
| No versioning support for the files generated by the pre-processor | Easy to loose the implementation file of the reflection class | Add versioning facilities to the pre-processor (use a counter ?) | ||
We identified some limitations of RSVP:
In our model, the presentation of QoS control to the user proved to be difficult. Our work has proved to us that relative priorities between cost rates are the most useful concept in this respect. We have implemented an interim prototype of such a control and further work is going to be undertaken at BT to identify the best solution.
Reflective Java proved to be relatively immature but with the help of some manual code modifications we were able to use it positively and the success with which we added QoS control to Web2Talk proves that the concepts it supports are correct. This is definitely a major outcome of our work regarding the evaluation of Reflective Java and its suitability to the QoS management area. Current experimentation using RSVP [15] outlines the problems of QoS control integration with existing applications, which our work solves with the use of Reflective Java.
We have provided some ideas for enhancing the current implementation of Reflective Java which would make it very attractive. During our work we came across some other implementations of meta-programming and reflection in Java [18, 56] which seem more complete and mature but are also more complicated to master and require modifications to the Java virtual machine.
The implementation of the model is still continuing as at this first stage we concentrated on merely getting the overall architecture working. We aim to provide better control in a multicast communication environment, provide more event-based reactions by the QoS manager and finally we would like to develop a reusable RSVP API in Java because the one we implemented for this work is very specific to our QoS manager. Another area of possible extension is in the run-time heuristic decision over whether to make a socket reflective (and thus give it timeliness control) and finally the QoS manager could be further separated to avoid over-centralisation.
This work is having a constructive input in the QoS research community as shown by the interest to publish a paper covering this work at the COST97 conference on multimedia network services [34]. Moreover the outcome of this project will be used in a new project (which is in a proposal stage at the moment) between BT and Lancaster university entitled "reflective middleware for distributed multimedia applications" [47]. This work will investigate the suitability of reflective middleware to support, amongst other things, QoS management, adaptability, configurability and monitoring of distributed multimedia applications.
[1] RFC 1633, Integrated Services in the Internet architecture: an overview. R. Braden, D.Clark, S.Shenker. June 1994. (33 pages) http://www.isi.edu/div7/rsvp/pub.html[2] RSVP: A new resource reservation protocol. Lixia Zhang, Stephen Deering, Deborah Estrin, Scott Shenker and Daniel Zappala. IEEE Network. September 1993. (9 pages) http://www.isi.edu/div7/rsvp/pub.html
[3] RAPI -- RSVP Application Programming interface -- version 5. IETF RSVP Working group. Draft RFC, November 1997. R. Braden, D. Hoffman. http://www.isi.edu/div7/rsvp/pub.html
[4] Resource ReSerVation Protocol (RSVP) - Version 1 functional specifications. Internet draft available at IETF. R. Braden, L.Zhang, S.Berson, S. Herzog, S.Janin. November 1996. (82 pages) http://www.isi.edu/div7/rsvp/pub.html
[5] The use of RSVP with IETF Integrated Services. J. Wroclawski MIT LCS. Internet draft. Expires 3/97 http://www.isi.edu/div7/rsvp/pub.html
[6] Quality of Service Management in Distributed Systems. David Hutchison, Geoff Coulson, Andrew Campbell and Gordon S. Blair. Lancaster university. Published in "Network and Distributed Systems Management" Edited by Morris Sloman, ICS London. Addison-Wesley publishing, 1994. Chap. 11. (29 pages)
[7] Specifying QoS Multimedia Communications within Distributed Programming Environments. Daniel G. Waddington, Geoff Coulson and David Hutchison. Computing Department Lancaster university. (20 pages)
[8] Supporting adaptive services in a heterogeneous mobile environment. Nigel Davies, Gordon S. Blair, Keith Cheverst and Adrian Friday. Distributed Multimedia Research Group at Lancaster University, ADAPT project. In proceedings of the 1st workshop on mobile computing systems and applications, Santa Cruz, CA, December 8-9, 1994. (5 pages)
[9] Quality of Service control for adaptive distributed multimedia applications using Esterel. Andrew Grace, Alan Smith. BT Laboratories. (8 pages) Second International Workshop on High Performance Protocol Architectures, Sydney, 11-12 December 1995
[10] Flexinet - QoS Investigation. Oyvind Hanssen. ANSA research group. APM.1977.00.03
[11] A QoS Configuration System for Distributed Applications. Alan Smith, Andrew Grace, BT Laboratories. 5th IFIP International Workshop on Quality of Service (IWQOS'97) (4 pages) http://comet.ctr.columbia.edu/iwqos97/
[12] Concepts and experiments in computational reflection. Pattie Maes. AI-Lab, Vrije Universiteit Brussels. Object-Oriented Programming, Systems, Languages and Applications (OOPSLA) Proceedings, 4-8 October 1987. (8 pages)
[13] Computational Reflection in Class based Object Oriented Languages. Jacques Ferber, LAFORIA. Object-Oriented Programming, Systems, Languages and Applications (OOPSLA) Proceedings, 1-6 October 1989. (8 pages)
[14] Workshop: Reflection and Metalevel Architectures in Object-Oriented Programming. Organiser: Mamdouh H. Ibrahim. Object-Oriented Programming, Systems, Languages and Applications (OOPSLA/ECOOP) Report, 21-25 October 1990. (5 pages)
[15] Commercial Internet Multimedia Trials - Preliminary results - Fred Baker, cisco; Mark Baugher, Intel; Saib Jarrar, MCI; Henry Sinnreich, MCI. Munich IETF, RSVP working group. Set of slides. ftp://ftp.isi.edu/rsvp/ietf.meetings/munich.ietf/
[16] Design of Reflective Java. ANSA draft, Zhixue Wue and Scarlet Schwiderski, 23rd December 1996. APM.1818.00.06 (22 pages) http://www.ansa.co.uk/ Research/ReflectiveJava.htm. Public copy at ftp://ftp.ansa.co.uk/phase3-doc-root/APM.1936.02.ps.gz
[17] Design and Implementation of a persistence service for Java. ANSA, Scarlet Schwiderski, 27th, January 1997. http://www.ansa.co.uk/Research/ReflectiveJava.htm
[18] MetaJava: An efficient run-time meta architecture for Java Friedrich-Alexander-University,Erlangen-Nurnberg, Computer Science Department, Operating Systems-IMMD IV, Germany. Jurgen Kleinoder, Michael Golm. June 1996. TR-I4-96-03.
[19] The Java language Environment, A white paper. James Gosling, Henry McGilton. Sun microsystems computer company. October 1995. http://java.sun.com:80 /docs/index.html
[20] The Java Platform, A white paper. Douglas Kramer. Sun microsystems computer company. May 1996. http://java.sun.com:80/docs/index.html
[21] Secure computing with Java: Now and future. 4th June 1997. http://java.sun.com:80/docs/index. html
[22] QoS Mapping home page and set of supporting slides. Daniel G Waddington, BT-URI at Lancaster university, End-system QoS in Multi-service Networks project. 6 December 1995 http://www.comp.lancs.ac.uk/computing/users/dan/uri/mapper/index.html
[23] Java Network programming. Merlin and Conrad Hughes, Michael Shoffner, Maria Winslow.Manning 1997. ISBN 0-13-841206-5. (519 pages + CD-ROM)
[24] The Java native programming interface. http://java.sun.com:80/docs/books/tutorial/ native1.1/implementing/index.html
[25] Campbell, A.T., "QOS-Aware Middleware for Mobile Multimedia Networking" , Multimedia Tools and Applications, Special Issue on Multimedia Information Systems, 1997, (to appear)
[26] Aurrecoechea, C., Campbell, A.T. and L. Hauw, "A Survey of QoS Architectures", Multimedia Systems Journal , Special Issue on QoS Architecture, 1997, (to appear)
[27] Campbell, A.T., Lazar, A.A., Schulzinne, H. and R. Stadler "Building Open Programmable Multimedia Networks", Computer Communications Journal, 1997, (to appear)
[28] Campbell, A.T., Coulson G., and D. Hutchison, "Supporting Adaptive Flows in Quality of Service Architecture", Multimedia Systems Journal, Special Issue on QoS Architecture, 1997, (to appear)
[29] Web2Talk, Internet telephony application. Stephane Chatre, Roger Klein, Carsten Schulz-Key. Oregon state university. http://www.CS.ORST.EDU/~kleinro/Web2Talk/
[30] Extending Sockets in JDK 1.1, David Brownell, David Brown, 21 August 1996. Sun MicroSystems. http://java.sun.com:80/docs/books/tutorial/native1.1/implementing/index.html
[31] Specification of Guaranteed Quality of Service, S. Shenker, C. Partridge, R. Guerin, Xerox/BBN/IBM. IETF draft, 3 February 1997, expires 8/3/98. Ftp://ftp.ietf.org/internet-drafts/draft-ietf-intserv-guaranteed-svc-07.txt
[32] Policy control for RSVP: Architectural Overview, Shai Herzog, IBM. 11/22/96. IETF draft, ftp://ftp.isi.edu/internet-drafts/draft-ietf-rsvp-policy-arch-01.txt
[33] Cost-quality trade-offs in the Internet. Jean-Chrysostome Bolot, INRIA, France. Computer and ISDN Systems 28 (1996) 645-651.
[34] COST 237 Workshop '97. Http://www.alettiiscte.pt/COST237/Workshop97
[35] Quality of Service and Performance in Distributed Systems. Andrew Grace, Distributed Systems Group, BT Labs. 27/05/97, Internal report.
[36] A Model of Real-Time QoS, Guangxing Li. APM.1151.00.04, 16 March 1994.
[37] WinSock 2.0a, Quality of Service. Ftp://ftp.microsoft.com/esf/winsock/doc/wsapi.doc. Jean Alibu May 1997.
[38] Simple Integrated Media Access (SIMA), K. Kilkki Nokia Research Center. IETF Network Working Group, June 1997 expires 19 December 1997. Http://www-ncr.nokia.com/sima/sima.txt
[39] Real time applications on the Internet. Steve Rudkin, Andrew Grace, Mike Whybray. BT Labs UK. BT Journal vol 15 no. 2 April 1997.
[40] An analysis of the applicability of RSVP. Ursula Schwantag. Diploma Thesis at the institure of Telematics. Universitat Karlsruhe (TH). July 15, 1997.
[41] A quality of service architecture, Andrew Campbell, Geoff Coulson and David Hutchison, Department of computing, Lancaster university, June 1995, Http://www.lancaster.ac.uk/adapt/publications
[42] An introduction to DIMMA, Ian Macmillan. APM. APM.1995.00.01. 20th May 1997. ANSA phase III document, restricted to sponsors.
[43] An overview of the Rialto Real-Time architecture. Michael B. Jones, Joseph S. Barrera III, Alessandro Forin, Paul J. Leach, Daniela Rosu, Marcle-Catalin Rosu. July 1996. Technical report MSR-TR-96-13. Microsoft research advanced technology division. Seventh ACM SIGPOS European Workshop, September 1996.
[44] Random thoughts on delivering end-to-end Quality of Service on the Internet. Paul fergudon. On the Internet, March/April 1997.
[45] The QOS Broker, K. Nahrstedt and Jonathan M. Smith University of pennsylvania. IEEE multimedia Spring 1995.
[46] An adaptive QoS architecture for Internet multimedia Applications using Reflective Java. Jérôme Tassel, Bob Briscoe, Alan Smith. BT research labs, advanced research and technology. Paper given in appendices 2. (maybe published at COST237 97)
[47] Notes on reflective middleware, Gordon S. Blair. Department of Computing, Lancaster University. (to be published)
[48] Reflective Java, Zhixue, Scarlet Schwiderski. APM. APM.1931.01. ANSA Phase III. Restricted to Sponsors. Set of slides.
[49] A reflective model for mobile software objects. Ophir Holder, Israel Ben-Shaul. Israel Institute of technology. IEEE 1997. Page 339-346.
[50] Atomic data types, R.J. Stroud and Z. Wu. Department of computing science, university of Newcastle upon Tyne, UK.
[51] Reflective Facilities in smalltalk-80. Brian Foote, Ralph E. Johnson. Department of computer science. University of Illinois. October 1989, OOPSLA proceedings.
[52] Java, Grady Booch. Rational software corporation. 1997. Http://www.rational.com
[53] Distributed Objects on the Web, R.J. Briscoe, BT research labs, advanced research and technology. BT Journal vol 15 no. 2 April 1997.
[54] Java security and the class loader, Bill Venners. JavaWorld September 1997. Http://www.javaworld.com/javaworld/jw-09-1997/jw-09-hood.htm
[55] The Java Core Reflection API. Sun's online documentation of the Java Reflection API. Http://java.sun.com/products/jdk/1.1/docs/guide/reflection/ index.html
[56] Take an in-depth look at the Java Reflection API. Chuck McManis. September 1997. Http://www.javaworld.com/javaworld/jw-09-1997/jw-09-indepth.htm
[57] Java, RMI and CORBA. A white paper prepared by David Curtis OMG. 1997. Http://www.omg.org/news/wpjava.htm
[58] JavaOS: A standalone Java environment. Peter Madany. 22 April 1997. Http://java.sun.com/products/javaos/javaos.white.htm
[59] Integration of reliability into off-the-shelf distributed systems (work in progress). Werner Vogels. Department of computer science, Cornell university. Http://www.cs.oswego.edu/~rkr/coots97atw/ vogels.txt
[60] JavaBeans specification v1.0. Graham Hammilton. From the sun documentation site. http://java.sun.com/beans/spec.html
[61] The promise of distributed business components. Dave K. Kythe. AT&T Technical Journal. March/April 1996.
[62] Advanced Java techniques, professional Java fundamentals. Shy Cohen, Tom Mitchell, Andres Gonzales, Larry Rodrigues and Kerry Hammil. WROX Press 1996. ISBN 1-861000-38-3. (487 pages)
[63] On-line Java tutorial from JavaSoft. Sun Microsystems, March 1997. Http://ww.javasoft.com
[64] Client/Server programming with Java and Corba. Robert Orfali, Dan Harkley, Wiley. January 1997. (635 pages + CDROM)
[66] Transparent and Adaptable object replication using a reflective Java. Friedrich-Alexander-University, Erlangen-Nurnberg, Computer Science Department, Operating Systems-IMMD IV, Germany. Jurgen Kleinoder, Michael Golm. September 1996. TR-I4-96-07.
[67] Limitations of Internet Protocol Suite for Distributed Simulation in the Large Multicast Environment. Internet Draft. J. M. Pullen, M. Myjak,C.Bouwens. 25 November 1996. (4 pages)
[68] Scenarios and Appropriate Protocols for Distributed Interactive Simulation. Internet Draft. Steve Seidensticker, W. Garth Smth, M. Myjak. 13 June 1996 (7 pages)
[69] Resource reservation protocol (RSVP) Version 1 Applicability Statement Some Guidelines on Deployment. Fred Baker, Bob Braden, Scott Bradner, Lixia Zhang. Internet draft, ftp://ftp.ietf.org