Date: Sun, 19 Dec 1993 01:21:30 GMT The Angel Project - A Brief Overview ==================================== Authors (in order of sentence): ------------------------------- Tim Wilkinson (tim@cs.city.ac.uk) Ashley Saulsbury (ans@sics.se) Tom Stiemerling Kevin Murray (kam@cs.city.ac.uk) Paul Kelley (phjk@doc.ic.ac.uk) Peter Osmon Introduction ------------ This project was prompted by earlier work at City University on a message passing microkernel and its Unix server (called Meshix [1]). This work developed a distributed microkernel to utilise a message passing, shared nothing processor platform. While this goal was successfully achieved, there were a number of issues regarding Meshix which the group were not satisfied with [2]: * The performance of the message passing system, * The structure of a kernel relying on message passing, * The complex and abstract naming of server interfaces (yet another ad-hoc naming scheme), and * The difficulty of using a message passing paradigm for "real" operating system sevices. These problems led to a number of key ideas, which in combination provided the momentum to start work on the design of a new operating system which was to become Angel. The intention was to abandon the increasingly complex UNIX model, and examine what could be achieved with a simple kernel model and current technology. In doing this, a number of issues were addressed. Naming and the single address space ----------------------------------- The complexity of naming and communicating with servers and the unnecessary overhead of message processing in the Meshix kernel (even when optimised) led to the desire for a simpler naming scheme. Furthermore a naming scheme which was not *yet another* abstract scheme. The perhaps not so obvious choice was to make use of the one naming scheme a processor explicitly supports --- the address space. This decission inevitably leads to a single address space, and such was the kernel design born. However, such an approach is of little merit with only a limited 4 Gigabytes of space available. Fortunately, the decission to adopt a single address space coincided conveniently with the announcement of a very high performance RISC processor in 1991 --- the MIPS R4000. This processor provided an ample (debatably so) 64 bits of naming space. Unification of shared memory and distributed memory --------------------------------------------------- The group at the time was also working on an implementation of Kai Li's DVSM system for the Meshix kernel [3]. We quickly came to the conclusion that a microkernel message passing system like Meshix had some inherent deficiencies. While intellectually appealing, a message passing paradigm is not especially simple to control inside the kernel, and certainly not suitable for many types of operation --- for example, when moving large amounts of data from one process to another. Therefore, a DVSM system implemented *on top* of such a message passing system was not the correct approach. However, the basic concept of DVSM remained very appealing. Looking at other contemporary microkernel technology and forthcoming work, there was a distinct trend toward using a message passing paradigm, but with gratuitous use of shared memory and page-remapping in order to achieve performance. It seemed crazy to force people to use message passing, when the message passing implementation leveraged off shared memory for performance. Surely it was better to provide shared memory as the basic paradigm of the machine, and allow users the choice of whether to build a message passing system on top of that. With shared memory as our basic programming model, the job of handling communication between machines then fell neatly to a DVSM implementation. This tied in logically with our thoughts on a single address space operating system, and the use of the address space as the fundamental naming scheme of the operating system [4]. By adopting this general approach, a number of features came for free. For example, process migration between machines becomes trivial, the work being dealt with transparently by the DVSM. Another example is the ability to provide a cache coherent file system --- a trick which required only the detachment of a memory object from the life of a process. Current Status -------------- For ease of development, the primary kernel construction has been done as an emulation under SunOS [5]. This phase of the work is now complete with versions being used as a basis for other related areas of research (eg. the support of a UNIX environment). The most recent port of the kernel is targeted for a Sparc. However, an earlier native port was completed on a machine with Motorola 88K processors which proved invaluable in keeping the kernel targeted at real architectures. A number of tools have been developed. The most important of these was a C/C++ compiler based on GNU C [6]. This supports the two compilation models required; "spawn" to handle the unknown placement of the data segment (UNIX-like exec semantics), and "fork", which allow creation of duplicate data segments at different address (UNIX-like fork semantics). Work in progress ---------------- A number of sub-projects have been spawned within the group based on the initial kernel version. These include: * Design and construction of a windowing system to be used as the primary user interface. By providing such facilities early on in the project, we hope to avoid being trapped in an old fashioned "80x24" terminal based environment. * Shared library support to allow efficient sharing of code. Single address spaces make sharing code simple but mechanism for coordinating the finer aspects of this system are not complete. * Multiprocessor simulation platform to enable both verification of existing multiprocessor support as well as allowing experimentation with the requirements of a "real" platform. * Fault tolerance additions to the kernel which, although already researched [7], have yet to be added to the system. References ---------- [1] "Topsy: An Extensible UNIX Multicomputer", P. Winterbottom and P. Osmon, City Univesity, ENGLAND. [2] "Evaluating MESHIX---A UNIX compatible Micro-Kernel Operating System", P. Osmon, T. Stiemerling, T. Wilkinson, N. Williams, A. Whitcroft, City University, ENGLAND. EurOpen Autumn '92 Conference Proceedings. [3] "Implementing DVSM on the TOPSY multicomputer", A. Saulsbury, Imperial College, ENGLAND, T. Stiemerling, T. Wilkinson, City University, ENGLAND, Symposium on Experience with Distributed Multicomputer Systems III Newport Beach, March 1992. [4] "Angel: A Proposed Multiprocessor Operating System Kernel", T. Wilkinson, T. Stiemerling, P. Osmon; City University, ENGLAND, A. Saulsbury, P. Kelly; Imperial College, ENGLAND, European Workshop on Parallel Computing March 1992. [5] "Design and Implementation of an Object-Orientated 64-bit Single Address Space Microkernel", K. Murray, T. Wilkinson, P. Osmon, City University, ENGLAND, A. Saulsbury, Swedish Inst. of Comp. Sci., SWEDEN, P. Kelly, Imperial College, ENGLAND, 2nd USENIX Symposium on Microkernels and other Kernel Architectures, San Diego, August 1993. [6] "Compiling for a 64-Bit Single Address Space Architecture", T. Wilkinson, A. Saulsbury, K. Murray, T. Stiemerling, City University, ENGLAND. Technical Report. [7] "Implementing Fault Tolerance in a 64-bit Distributed Operating System" T. Wilkinson, City University, ENGLAND. PhD Thesis.