Design and implementation of OS

• not solvable / no optimal solution
• characteristics
  • internal structure: varies widely
  • even at high level: design affected by choice of hardware / system type, etc.
    • desktop / laptop / mobile / distributed / real- tie
  • design: starts with specifying goals & specifications
    • user goals: OS being convenient to use
      • and ease to learn & use
      • reliable, safe, fast
    • system goals: easy to design, implement, and maintain
      • and flexible, reliable (hopefully) error-free & efficient
  • highly creative task of SWE

Important principle ⭐

• separation of policy & mechanism
  • policy: what will be done
  • mechanism: how to do it?
    • e.g. time construct, mechanism for ensuring CPU protection
      • deciding how long the timer will be: policy decision
• 👨‍🏫 policy: often changes
• separation of policy from mechanism: allows flexibility even policies change
  • w/ proper separation: mechanism can be used to policy decision for
    • prioritizing IO-intensive program
    • as well as vice versa
• OS: designed w/ specific goals in mind
  • which: determine OS policies
  • OS: implements policies through specific mechanisms

Implementation

• early OSes: written in assembly (~= 50 yrs a go)
• modern OS: written in higher level language
  • for some special device/ function: assembly might still be used
  • but general system: written in C/C++
  • frameworks: in Java, mostly
• w/ higher level language: it can be easily ported
  • also, code can be write faster, compact, and easier to maintain
• speed / increased storage requirement: not a concern now
  • technology advances!

OS structure

• general purpose OS: very large program
• various ways to structure one
  • monolithic structure
  • layered - specific type of modular approach
  • microkernel - mach OS
  • loadable kernel modules (LKMs) - modular approach

Simple structure

• the simplest possible
• structure not well-defined; started as a small / simple / limited system
• MS-DOS: written for maximum functionalist in minimum space
  • not carefully divided into modules
• doesn't separate between user & os problem
  • 👨‍🏫 NOT a dual system
  • can cause many trouble nowadays (all user: can do sudo and all)

Monolithic structure: original linux

• kernel: consists of everything below the syscall interface
  • and above the physical hardware
• employees multiprogramming
• 👨‍🏫 quite simple considering having to allocate stacks, etc.
• user: can't directly hardware
  • kernel is underneath
• 
• UNIX: initially limited by hardware functionality => limited structuring
  • 👨‍🏫 still, it is a single program!
    • a single, static binary file running in a single address space
  • monolithic structure!
• UNIX: made of two separable parts
  • kernel: further separated into interfaces & device drivers
    • expanded as UNIX evolves
  • system programs
• drawback: enormous functionality: combined to one level
  • difficult to implement, debug & maintain
• advantage: distinct performance!
  • no need to pass parameter, etc.
  • very little overhead in syscall interface and communication, etc.

Linux system structure

• i.e. modular design
  • allows kernel to be modified during run time
• based on UNIT
• still, monolithic:runs entirely in kernel mode w/ single address space

Modular design

• monolithic approach: too compact and tight interaction
• why not make the tie looser?
  • so that change in one module / functionality does not affect the others parts!
  • 👨‍🏫 ideally, and hopefully!
• one method: layered approach
  • e.g. file system: located above hardware

Layered approach

• OS: divided into no. of layers
  • each built on top of lower layers
  • layer $0$: hardware
  • layer $N$: user interface
• each layer: made of data structure & set of functions
  • that can be invoked by higher level layers
• each layer: utilized the services from a lower layer, provides a set of functions
  • and offers certain services for a higher layer
• 
• 👨‍🎓 linux: all in 1 layer
• information higher: layer not needing to know lower-layer's implementation
  • each layers: hides existence of its own data structures / operations to higher levels
• main advantage: simplification of construction & debugging
  • layered system: successfully used in other areas too
  • like TCP/IP vs. hardware (networking system)
• few systems tue pure layered approach though
  • overall performance will be affected
• trend: fewer layers w/ more functionality & modularization
  • while avoiding complex layer definition & interactions

Mixed kernel system structure

• kernel: became large & difficult to manage
• some people: wanted to make OS a core
  • and store all other nonessential parts outside
  • mainstream (OS): expansion
  • result: much smaller kernel
• Mach: developed at CMU in mid-1980s
  • modularized the kernel w/ microkernel approach
  • best-known microkernel OS: Darwin, used in Mac OS X, iOS
    • consisting of 2 kernels, one being Mach microkernel
  • UNIX / IBM, etc. thought it's desirable
• 👨‍🏫 but no general consensus on what's nonessential
  • but generally: minimal process, memory management and communication facility: must be in kernel
• some of the essential functions
  • provide communications between programs / services in user address space
    • through message passing
  • problem: if app wants to access file
    • it must interact w/ file server through kernel, not directly to file / file manager
  • 
• advantage
  • easier to extend microkernel system (without modifying the kernel)
  • minimal change to kernel
  • easier to post OS to ner architectures
  • more secure
• drawbacks: performance
  • increased system overhead
  • copying & storing messages during communication => makes it slow!
• not done by the mainstream
• Window NT: w/ layered microkernel
  • much worse performance thantWindow 95 / Window XP

Modular approach: current methodology

• best current practice: involving loadable kernel modules (LKMs)
  • i.e. programs can be added to kernel without rebooting
• once added, become a part of kernel, not user program!
• kernel: supports link to additional services
  • services: can be implements & added dynamically while kernel is running
  • dynamic linking: preferred to adding new features directly to kernel
    • as it doesn't require recompiling kernel per every change
• resembles layer : as it has well-defined & protected interfaced
  • yet, it's much more flexible
• also resembles microkernel: as primary module is only core functions and linkers
  • more efficient that pure microkernel though
• linux: w/ LKMs, primarily for supporting device drivers & file systems
• LKMs: can be inserted into kernel during booting & runtime
  • also can be removed during runtime!
• dynamic & modular kernel, while maintaining performance of monolithic system

Case study: Solaris

• 👨‍🏫 in my days... Solaris was a dominant workstation provider
  • PC wasn't a things... and we CS people used Linux
  • now, the company is not computer system program though
• it worked with modular system
• 

Hybrid system

• most OS: combines benefit of different structures
• Linux: monolithic (in single address space), but also modular
  • new functionality can be dynamically added
• windows: largely monolithic, with some microkernel behavior
  • also provide support for dynamically loadable kernel modules

macOS & iOS structure

• in architecture: mcOS & iOS: have much in common
  • UI, programming support, graphics, media, and even kernel environment
• Darwin: includes Mach microkernel and BSD UNIX kernel
  • with dual base programs
• 
• higher level structure
• Max OS X: hybrid, layered, Aqua UI (mouse / trackpad)
  • w/ Cocoa programming environment: API for Objective-C
  • core frameworks: support graphics and media
    • including Quicktime & OpenGL
  • kernel environment (Darwin) includes Mach microkernel & BSD
  • 
• iOS
  • structured on Mac OS; added C functionality
  • does not run applications natively; runs on different CPU architectures too (ARM & Intel)
    • core OS: based on Mac OS X kernel, but with optimized battery
  • 
  • Springboard UI: designed for touch devices
  • Cocoa Touch: Object-C API for mobile app development (touch screen)
  • Media services: layer for graphics, audio, video, quicktime, OpenGL
  • Core services: providing cloud computing & data bases
  • Core OS: based on Max OS X kernel
• Darwin
  • layered system consisting of Mach microkernel & BSD UNIX kernel
    • hybrid system
  • two syscall interfaces: Mach syscalls (aka traps) and BSD syscalls (POSIX functionality)
  • interface: rich set of libraries - standard C, networking, security, programming languages..
  • Mach: providing fundamental OS services
    • memory management, CPU scheduling, IPC facilities
  • kernel: provide IO kit for device drivers
    • dynamically loadable modules: aka kernel extensions (kexts)
  • BSD: THE first implementation supporting TCP/IP, etc.
  • 

Android

• developed by Open Handset Alliance (led by GOogle)
• runs on a variety of mobile platforms
  • open-source, unlike Apple's closed-sources
• similar to OS: layered system providing rich set of frameworks
• for graphics, audio, and hardware
• using a separate Android API for Java development (not standard Java)
  • Java app: execute on Android Run Time (ART)
  • VM: optimized for mobile devices w/ limited memory & CPU
• Java native interface (JNI): allows developers to bypass CM
  • and access: specific hardware features
  • programs written in JNI: generally not portable in terms of hardware
• libraries: include framework for:
  • Webkit: web browser dev
  • SQLite: DB support
  • SSLs: network support (secure sockets)
• hardware abstract layer (HAL): abstracts all hardware
  • providing applications w/ consistent view
    • independent of hardware
• uses Bionic standard C library by google, instead of GNU C library for Linux systems
• 

Introduction to processes

• 👨‍🏫 a real thing! no more high level!
• objectives:
  • identify components of process, representation, and scheduling
  • describe creation & termination using appropriate syscalls
  • describe and contrast inter-process communication w/ shared memory
    • and message passing methods

Four fundamental OS concepts

• process
  • instance of an executing program
  • made of address space & more than 1 thread of control
• thread (continues in Ch. 4)
  • single unique execution context
  • full description of program stated
    • captured by program counter, registers, execution flags, and stack
  • for parallel operations..?
• address space (w/ address translation)
  • programs: execute in address space
    • != memory space of physical machine
  • each program: starts w/ address 0
• dual mode operation (= Basic protection)
  • user program / kernel: runs in different modes
  • different permissions per programs
  • OS & hardware: protected from user programs
  • user programs: protected & isolated from one another

Process concept

• OS: executes a variety of programs: each runs as process
• process: captures a program execution, in a sequential manner
  • von Neumann architecture
• process: contains multiple parts
  • program code: aka text section
  • current activities or state: program counter / processor registers
  • stack: temporary data storage when invoking functions
    • e.g. for function parameters, return addresses, etc.
  • data section: containing global variables
  • heap: w/ dynamically allocated memory during run time
  • 👨‍🏫 stack & heap can grow dynamically: during execution
• program: passively stored on disk
  • process: active entity, w/ PC specifying next instruction to fetch & execute
• process: with life cycle: from execution to termination
• program: becomes a process
  • when executable file is loaded into main memory & get ready to run
• executes in address space, different from actual main memory location
• one program: can be executed by multiple processes
• a program: can be an execution environment for others (e.g. JVM)

Loading program into memory

• program: must be loaded to memory before execution

1. load its code & static data into memory / address space
2. some memory: allocated for program's runtime stack
   • for C: stack used for local variables, function parameters, return addresses
   • fill in: parameters for main(): e.g. argc,argv
3. OS: may allocate memory for program's heap
   • programs: request & returns it explicitly
4. dO: also d some initialization tasks
   • esp. relating to IO
     • e.g. UNIX: each process w/ 3 open file descriptors
     • standard input / output / error

Memory layout of a program

• global data: divided into initialized data and uninitialized data
• separate section for argc,argv parameters
•