Lecture 13, supposed to be "spooky", but in fact permeated with
military terms and historical allusions. 

==================== Kernel's Memory Space ====================

The kernel is loaded into a set of fixed, platform-specific virtual address
ranges. To re-iterate, all addresses are virtual, and so is any address that
is encoded as a part of an instruction.

Kernel (virtual) address layouts:
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/i86pc/os/startup.c#358
lines 358--508

Global variables/symbols reflecting that layout:
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/vm/seg_kmem.h#48

(actual definitions in 
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/vm/seg_kmem.c#99)

Within these kernel address ranges, address mappings are created and managed 
by several kernel "segment drivers":

Kernel's segments (Ch. 11.1):

seg_kmem     -- normal non-pageable kernel memory management
 
seg_kp	     -- kernel pageable memory management  

seg_nf 	     -- ???

seg_map        -- file cache pages mapped into kernel space  (::addr2smap DCMD)

seg_kpm        -- all physical memory pages mapped into kernel space (only in x64)

Read more about these in 11.1.5-6 .

The Smap layer establishes a mapping from a virtual address to 
a "page identity" <sm_vp, sm_off>: 
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/vm/seg_map.h#75
In MDB, it is reported by the ::addr2smap  command.

An important observation about "large pages" is found on p. 530, 11.1.2 regarding
the effect of the large pages on the kernel's efficiency. 10% improvement is a lot.

==================== KMem Caches ====================

The two most interesting features of the Kmem caches design are

1. Avoidance of global structures that would need to be locked for every
    allocation from the slab (like a global freelist). Instead, per-CPU
    "magazines" are used.

2. Object-oriented style of a cache: constructor, destructor and reclaim
    method pointers are arguments to the kmem_cache_create() and
    will be called automatically when objects are allocated and freed.

(see 

In "design patterns" term this makes a named cache a "factory" of objects
of that type.

Ch 11.2.3.2 explains the theory of "caching", 11.2.3.1 gives the background
on the design.

I found the explanations of 11.2.3.4--7  somewhat confusing. Keep in mind
that despite all the military terminology, we are merely dealing with a list 
of finite-depth stacks.

The "magazines", "rounds", etc. are best understood from the logic of
kmem_cache_allocate():

http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/os/kmem.c#kmem_depot_alloc#2152

Note the *per-PCU shift* on line  2158, by the KMEM_CPU_CACHE macro from:
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/sys/kmem_impl.h#175

The "magazines" themselves are just arrays (stacks) of pointers into the main
slab structure, linked in a single list. Note that their depth is not statically
set in the definition (it is dynamically adjusted based on contention metrics):
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/sys/kmem_impl.h#216   --230

If the "magazine" and "depot" logic falls through (to line 2222) , the "slab" layer 
will take the global cache lock (and cause threads on all other CPUs to block 
in this critical section). This lock is taken on line 1696 for the whole cache,
not just for a per-CPU part. 

Finally, note the constructor call (if defined for the "class" of managed object)
on line 2248. This is code typical of an OO-language interpreter such as Ruby.

==================== Debugging Help ====================

Described in the textbook in Ch. 11.4  (we will revisit it).

Since heap block-management data is stored "in-band", accidental
or intentional overwrites of that data by an overflowing write
into a struct instance allocated within the slab will cause errors.

The precise mechanism of such deliberate exploitation is described in
http://doc.bughunter.net/buffer-overflow/free.html

To catch such conditions early, before they result in an arbitrary
write and possibly a kernel panic, Kmem includes built-in debugging.
Freed slab slots are filled with a special pattern of hex data,
known as "DEADBEEF" (after a valid hex integer that one can
spell with hex digits), etc.:

http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/sys/kmem_impl.h#79   (also 61 and the uses of KMF_DEADBEEF throughout kmem.c).

Theory: 
The huge "Big Theory Statement" comment starting at
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/os/kmem.c#41 ,
 around line 238.

==================== Non sequitur ====================

The "Brown Bess" Musket:
http://www.youtube.com/watch?v=SJMbxZ1k9NQ  -- in action
http://www.youtube.com/watch?v=Ho-QCmnNMl8  -- theory
http://www.youtube.com/watch?v=d47-51HhmxQ  -- more theory with details
 
Grenadiers:
http://en.wikipedia.org/wiki/Grenadier_(soldier)