Table of Content

image-20240216164139584

Intro

CPU frenquesncy

image-20231009140121426

RAM random access memeory

image-20231009140451363

image-20231009140912040

image-20231009141129480

image-20231009141317562

image-20231009142149572

image-20231009142626266

image-20231009142840018

Summary

我只能说, 百度百科讲的很好

大框架

image-20231012142113694

CPU

CPU主要由CPU核心(芯片)cache组成

**CPU核心(芯片)**一共有三个部分

  • control unit: 指挥中心, 内存中获取指令, 程序按正确的顺序执行

  • Arithmetic Unit: 数学和逻辑处理部分, 执行各种算术运算,如加法、减法、乘法和除法,以及逻辑运算,如与、或、非等

  • Registers: 用于存储临时数据和指令,非常快速的数据访问,因为它们位于CPU内部,不需要从主存中获取数据

Cache

A type of RAM set aside as a specialized buffer storage, used to optimize data transfers

我们可以理解为Cache就是Static RAM, 而Dynamic RAM就是我们说的RAM(也就是内存条)

  • L1 Cache:L1缓存通常是位于CPU内部的,与CPU核心紧密集成在一起。因此,L1 Cache是物理上与CPU核心直接相关的一部分

  • L2 Cache:L2 Cache的位置可以有所不同。在某些处理器中,L2 Cache可能位于CPU芯片上,但与CPU核心不同。在其他处理器中,L2 Cache可能是一个离CPU芯片较近的存储模块。

  • L3 Cache:L3 Cache通常是多个CPU核心之间的共享缓存,它可能位于多个CPU核心之间,通常在主板上的特殊位置。这种情况下,L3 Cache物理上不与任何特定的CPU核心直接关联。

多核心CPU和Cache的关系

高速缓存(Cache)可以根据 CPU 架构和多核处理器的配置而有所不同。一般来说,多核处理器的高速缓存结构可以分为两种主要类型:共享缓存和非共享缓存。

每个核心通常拥有自己的 L1 Cache 和 L2 Cache,这些缓存是私有的,只供该核心使用。

L3 Cache 通常是多个核心共享的,以确保更大的缓存容量和更好的数据共享,以支持多核 CPU 的协同工作。

  1. 共享缓存:在某些多核处理器中,多个核心可以共享一个或多个高速缓存层级,通常是 L3 Cache。这意味着所有核心都可以访问相同的高速缓存,从而在核心之间共享缓存中存储的数据和指令。这有助于减少数据复制,提高多核处理器的性能,特别是对于多线程应用程序。
  2. 非共享缓存:在其他多核处理器中,每个核心拥有自己的独立高速缓存层级,例如 L1 Cache 和 L2 Cache,这些缓存是私有的,不共享。这意味着每个核心的高速缓存不会被其他核心访问。这种配置有助于确保高速缓存的低延迟,但也可能导致一些数据复制。

ROM

在主板上, 通常包括计算机的启动程序(如BIOS)或其他重要的固定数据,用于引导计算机和设备。

ROM chips are located on the motherboard

–Basic instructions for booting the computer and loading the operating system are stored in ROM.–ROM chips retain their contents even when the computer is powered down.

RAM

就是内存条, 但Cache也是一种RAM

Cache

A type of RAM set aside as a specialized buffer storage, used to optimize data transfers,

内嵌在 CPU 芯片中或与 CPU 芯片相邻,

–SRAM (Static RAM) is used as cache memory to store the most frequently used data.

–SRAM provides the processor with faster access to the data than retrieving it from the DRAM

Main memory:

分为ROM 和 Voliatile memory(这部分通常是RAM)

Basic instructions for booting the computer and loading the operating system are stored in ROM.

RAM is temporary storage for data and programs that are being accessed by the CPU.

Mass storage

通常指硬盘驱动器(HDD)、固态硬盘(SSD)、光盘、磁带、网络存储等

Flash

Flash存储技术是实际物理存在的存储介质, 芯片或模块的形式存在。Flash 存储基于闪存存储技术,这是一种半导体存储技术

  • Flash 存储是一种基于闪存技术的非易失性存储器。它通常用于固态硬盘(Solid State Drive,SSD)、USB 闪存驱动器、内置存储(如手机内存)等。
  • Flash 存储具有高速读取和写入速度,低能耗,且没有机械部件。它是一种静态存储技术,不需要电源来维持存储的数据。
  • Flash 存储适合用于快速启动操作系统、应用程序和数据存储,特别是在便携设备和固态硬盘中使用。
  • 由于 Flash 存储没有移动部件,因此具有更高的耐用性和更少的机械故障。

image-20231012200001199

RAM里有些什么

右边那个是RAM, 灰色的是Address, 白色的是data, data可能为instructions,可能为number, text, 甚至为address

image-20231013134523249

Part 1 Computer Systems & Architecture

1.1 Computer Organization and Architecture

Computer Components

Three parts

  1. Memory and Disk Storage store both data and programs
  2. Processor
  3. Mechanisms for transferring data to and from the outside world, input the data iinto the computer, the computer tell you the result of computing

Measure of a Computer System -- Speed & Capacity, Time & Space

Speed: clock cycles per second

–Hertz = clock cycles per second (a unit of frequency and speed)

• 1MHz = 106=1,000,000Hz

• Processor speeds are measured in MHz or GHz.

Capacity: bytes

–Byte = a unit of storage • 1KB = 210= 1024 Bytes

• 1MB = 2^20= 1,048,576 Bytes

• 1GB = 2^30= 1,073,741,824 Bytes

•Main memory (RAM) is normally measured in GB

• Disk storage is normally measured in GB for small systems, TB (240) for large systems.

Time: second

–Millisecond = 1 thousandth (10‐3) of a second

•Hard disk drive access times are often 10 to 20 milliseconds (i.e., 10‐20 ms).

–Microsecond = 1 millionth (10‐6) of a second (μs)

–Nanosecond = 1 billionth (10‐9) of a second

•Main memory access times are often 50 to 70 nanoseconds (i.e., 50‐70 ns).

Space: meter

–Micron (micrometer) = 1 millionth (10^‐6) of a meter

• Circuits on computer chips are measured in microns (μm).

计算机芯片上的电子元件和电路的尺寸通常使用微米(microns)来衡量

image-20231009135542417

reciprocal adj.互惠的 n.倒数

下面的内容涉及到计算计算机系统中的周期时间(Cycle time)以及时钟频率(Clock frequency)之间的关系,以及如何将时钟频率转换为周期时间。

  1. Cycle Time(周期时间):Cycle time 是指一个计算机系统中执行一个基本操作(例如,执行一条指令)所需的时间。它通常以纳秒(nanoseconds,缩写为ns)为单位表示。较短的周期时间通常表示计算机系统的性能更高,因为它可以更快地执行操作。
  2. Clock Frequency(时钟频率):时钟频率是计算机系统中时钟信号的频率,通常以赫兹(Hertz,缩写为Hz)为单位表示。时钟信号用于同步计算机内部的各个组件和操作,例如 CPU 的执行和数据传输等。时钟频率越高,计算机系统的性能越高,因为它可以在单位时间内执行更多的操作。

下面的公式解释了这两个概念之间的关系:

  • 周期时间(Cycle time)等于时钟频率(Clock frequency)的倒数。

这意味着,如果你知道时钟频率,可以通过将其倒数来计算周期时间。公式如下:

周期时间(ns) = 1 / 时钟频率(Hz)

例如,如果一个总线(bus)的时钟频率是133兆赫兹(133,000,000 Hz),你可以使用这个公式来计算其周期时间:

周期时间(ns) = 1 / 133,000,000 Hz ≈ 7.52 纳秒/周期

这表示每个总线周期大约需要7.52纳秒的时间。这种信息对于计算和优化计算机系统的性能非常重要,特别是在需要考虑时序和时钟周期的应用中。

1.2 Basic Personal Computer System

主要讲hardware和software

Hardware

the physical equipment such as the case, storage drives, keyboards, monitors, cables, speakers, cameras, printers….

image-20231010103827368

Software means the operating system and programs.

• The operating system instructs the computer how to operate.

• Programs or applications perform different functions.

主板(motherBoard)

里面的元素需要沟通, exchange data, 就是通过bus来实现的

"bus"(总线)是一组电子连接线路或数据传输通道的集合,它们用于连接主板上的各种硬件组件,使它们能够相互通信和交换数据。总线在计算机体系结构中起着重要的作用,它允许不同的硬件组件(例如,中央处理器、内存、扩展插槽、存储设备、图形卡等)之间进行数据传输和协作。

bus可以是一组导线、电缆或信号线,它们携带不同类型的信息,如数据、地址、控制信号等。不同的总线用于不同的目的,例如连接CPU和内存、连接扩展卡、连接I/O设备等。

bus的速度、带宽和协议取决于计算机体系结构和硬件标准,对计算机性能和功能有重要影响。

CPU(central Processing Unit)

● The CPU is known as the brain of acomputer. It is also referred to as the processor.

● The CPU executes a program, which is a sequence of stored instructions.

–processes each piece of data as directed by the program and the instruction set

–executes one step of the program, the remaining instructions and the data are stored nearby in a special memory called cache

● Two major CPU architectures related to instruction sets:

•Reduced Instruction Set Computer (RISC) •Relatively small set of instructions ‐> very rapidly

•Complex Instruction Set Computer(CISC) •A broad set of instructions ‐>fewer steps per operation

将RISC和CISC特性结合在一起的处理器通常被设计成可以根据指令的复杂性和需求来选择执行方式。

●Some CPUs incorporate hyperthreading or hypertransport to enhance the performance of the CPU.

–Multiple pieces of code can execute in CPU simultaneously on each pipeline.

Hyperthreading是一种处理器内部的技术,它使单个核心模拟为多个逻辑核心。这样,一个物理处理器可以处理多个线程,从而提高多任务性能。

HyperTransport是一种用于系统总线和互联连接的技术, 提供高带宽、低延迟的点对点连接,允许各种组件之间进行快速数据传输。

●The amount of data that a CPU can process at one time depends on the size of the processor data bus (32‐bit or 64‐bit).

●The speed of the CPU is measured in cycles per second megahertz (MHz) or gigahertz (GHz).

●Overclocking is a technique used to make a processor work at a faster speed than its original specification

may result in damaging the CPU

缓存(cache)通常位于CPU内部, 是临时存储数据和指令的高速存储器, 主要目的是减少CPU访问主内存的延迟,(主内存Main Memory, 也称为RAM )

因为主内存相对于CPU的速度较慢。通过将最常用的数据和指令存储在高速缓存中,CPU可以更快地获取所需的信息,从而提高计算机的性能。

Cooling Systems

●Electronic components generate heat. Too much heat can damage components.

●A case fan makes the cooling process more efficient.

●A heat sink draws heat away from the core of the CPU. A fan on top of the heat sink moves the heat away from the CPU.

● Fans can be dedicated to cool the Graphics processing unit (GPU).

ROM and RAM

● Read‐only memory (ROM): ROM chips are located on the motherboard–Basic instructions for booting the computer and loading the operating system are stored in ROM.–ROM chips retain their contents even when the computer is powered down.

ROM是一种非易失性存储器,用于存储计算机的固件或不易更改的信息,例如启动程序、BIOS(基本输入/输出系统)固件和操作系统的引导加载程序,用于引导计算机和设备。ROM通常不在内存条中,而是存在于计算机的主板或其他存储介质中,如硬盘驱动器、固态硬盘或闪存芯片中。

● Random‐access memory (RAM):–RAM is temporary storage for data and programs that are being accessed by the CPU.–More RAM means more capacity to hold and process large programs and files, as well as enhance system performance.–RAM is volatile memory, which means that the contents are erased when the computer is powered off.

RAM是易失性存储器,这意味着数据在断电时会丢失。它用于计算机的临时存储,允许CPU快速读取和写入数据。RAM的容量越大,计算机可以同时运行的程序和处理的数据量就越多。

Cache

●A type of RAM set aside as a specialized buffer storage, used to optimize data transfers(i.e., matching the data transfer speed with CPU processing speed)

–SRAM (Static RAM) is used as cache memory to store the most frequently used data.

–SRAM provides the processor with faster access to the data than retrieving it from the DRAM (Dynamic RAM), or main memory.

●Three levels of cache

–L1 cache is internal cache and is integrated into the CPU.

–L2 cache is external cache and was originally mounted on the motherboard near the CPU. L2 cache is now integrated into the CPU.

–L3 cache is used on some high‐end workstations and server CPUs.

Error Checking

●Memory errors occur when the data is not stored correctly in the RAM chips.

●The computer uses different methods to detect and correct data errors in memory

–Parity: contains 8 bits for data and 1 bit for error checking (called parity bit)

–ECC: Error Correction Code, can detect multiple bit errors and correct single bit errors in memory

–Nonparity: does not check for errors in memory

Storage Drives

The hard disk drive (HDD)is a magnetic storage device.

The storage capacity is measured in gigabytes (GB) or terabytes (TB)–Magnetic hard drives have drive motors designed to spin magnetic platters and move the drive heads.

image-20231010120837544

image-20231010121048000

Adapter Cards

Examples of adapter cards: •Sound adapter and video adapter •Network Interface Card (NIC), wireless NIC, modem adapter •USB port

●Adapter cards increase the functionality of a computer by adding controllers for specific devices or by replacing malfunctioning ports.

●Types of expansion slots:

•Peripheral Component Interconnect (PCI) •Advanced Graphics Port (AGP) •PCI‐Express (PCIe) •Mini PCI (laptops)

RAID

Redundant Array of Independent Disks,独立磁盘冗余阵列, 是一种用于提高数据存储系统性能、可靠性和/或容错能力的技术。RAID将多个独立的硬盘驱动器(Hard Disk Drive, 也就是硬盘) 组合在一起,以创建一个更大、更可靠或性能更高的存储单元。

  1. 数据冗余:通过将数据复制到多个硬盘驱动器上,RAID可以提供数据冗余,即使一块硬盘驱动器发生故障,数据仍然可用。这有助于保护数据免受硬件故障的影响。
  2. 性能提升:RAID可以通过将数据分布到多个硬盘驱动器上来提高读写性能。不同的RAID级别提供不同程度的性能提升,从而满足不同的需求。
  3. 容错能力:RAID可以提高系统的容错能力,允许在硬盘驱动器发生故障时继续运行。这是关键的服务器和数据中心应用,以确保持续性能和可用性。

image-20231010144205194

image-20231010122345517

image-20231010143015653

1.3 Computer Level Hierarchy

image-20231010144639158

●Level 6: The User Level

–Program execution and user interface level.

–The level with which we are most familiar.

●Level 5: High‐Level Language Level

–The level with which we interact when we write programs in languages such as C, Pascal, Lisp, Go, and Java.

● Level 4: Assembly Language Level

level 5 的 High‐Level Language 会被 complied or interpreted into the assembly language before it can be translated into the machine language(which can be understand by the computer)

–Acts upon assembly language produced from Level 5, as well as instructions programmed directly at this level.

● Level 3: System Software Level

–Controls executing processes on the system.

–Protects system resources.

–Assembly language instructions often pass through Level 3 without modification.(直接到Level 2 machine)

● Level 2: Machine Level

–Also known as the Instruction Set Architecture (ISA) Level.

–Consists of instructions that are particular to the architecture of the machine.

–Programs written in machine language need no compilers, interpreters, or assemblers.

● Level 1: Control Level

–A control unit decodes and executes instructions and moves data through the system.

–Control units can be micro programmed or hardwired.

•A micro program is a program written in a low‐level language that is implemented by the hardware.

•Hardwired control units consist of hardware that executes directly machine instructions.

●Level 0: Digital Logic Level

–This level is where we find digital circuits (the chips).

–Digital circuits consist of gates and wires.

–These components implement the mathematical logic of all other levels.

Part 2 Operating Systems

2.1 What Is OS

●A software that works as an interface between a user and the computer hardware.

–Software required in order to run applications programs and utilities.

–The operating system (OS) controls almost all functions on a computer: software & hardware (input & output devices attached to the computer)

Why we need OS

●To make computer system convenient to use

–E.g., receiving input from keyboard; sending processing results to screen

●Resource competition: various applications (processes) compete to each other for getting the CPU time and memory space to complete the task.

–To utilize computer hardware in an efficient manner.

Two common CPU architectures to process data:

• x86 (32-bit architecture):

–uses a Complex Instruction Set Computer (CISC)

–has few registers than x64

• x64 (64-bit architecture)

–backward compatible with x86

–Adds additional registers specifically for instructions that use a 64-bit address space: to process much more complex instructions at a much higher rate

• 32-bit and 64-bit Windows OS

• 32-bit OS: Windows XP professional

Capable of addressing 4 GB of RAM (4GB=232B)

• 64-bit OS: Windows Vista, some of Windows 7, Windows 10

Capable of addressing 128+ GB of RAM • Enhanced performance for memory management and additional security features

image-20231010200807505

2.2 Functions and Components of OS

All computers rely on an operating system (OS) to provide the interface for interaction among users, applications, and hardware.

The operating system boots the computer and manages the file system.

The operating system has four main roles:

−Provide user interface : communicates with users

Text based (Shell)

Graphical user interface (GUI)

−Control hardware access (device management)

−Manage files and folders (storage management)

−Manage applications (memory/processor/job management)

BootStrapping

Bootloader(引导加载程序)是系统启动过程中的第一个程序,它的主要责任是启动操作系统。

选择、加载和启动操作系统。它充当了计算机硬件和操作系统之间的桥梁,确保操作系统正确加载到主内存中,并开始执行。一旦操作系统内核启动,它将掌管系统的运行,包括用户应用程序的执行和资源管理。

image-20231010203019794

  1. ROM(只读存储器):这是 "main memory" 的一个组成部分,通常包含计算机的固件或只读数据,如BIOS。ROM中的数据通常无法被普通用户修改,它们在计算机启动时提供了基本的引导和初始化信息。
  2. Volatile Memory(易失性存储器):这部分通常是RAM(随机存取存储器),它用于存储正在运行的程序和数据。RAM是易失性存储,意味着其中的数据在断电后会丢失。

image-20231010204103806

image-20231010204112176

Kernel

performs basic required functions

–File manager: directories (folders) & directory paths

–Memory manager

–Device manager

–Processor manager: event handling, scheduler and dispatcher

–Job manager: input&output, scheduling&control of jobs

file system management, physical storage device management, virtual memory management, device drivers for storage devices, and error handling related to storage and file systems.

image-20231010204821522

File Management, Device Manager,

image-20231010215441680

Storage Manager

image-20231010215436187

image-20231010220524572

Drive Mapping(驱动器映射)

  • 在许多操作系统中,硬盘驱动器、可移动存储设备、网络共享等存储介质通常被分配一个字母,如C:、D:、E:等,来标识它们。这些字母称为驱动器号码(Drive Letters)。

Memory Manager

image-20231011102338649

没有足够的RAM space, 就会用virtual memory, 因为virtual memory是用mass storage, 所以就会很慢, 也就是说, 你开了很多程序, RAM没空间了, 就用virtual memory, 电脑就会很慢

image-20231016105458964

Porcess Management(进程管理)

image-20231011104327971

  • 内核负责创建、终止和管理进程(程序的执行实例)。它分配和释放进程所需的资源,确保它们能够协调共享计算机的CPU时间。

这个process state就是记录当前activity的状态, 因为cpu是一个个来执行的, 等他执行你了, 他去执行别的, 这个时候这个activity的state就会记录下来, 等下次CPU再光顾的时候, 就会开始继续执行

Process Administration

  • Scheduler: maintains a process table

–Adds new processes to the process table and removes completed processes from the process table

  • Dispatcher: ensures the scheduled processes are actually executed

–Controls the allocation of time slices to the processes in the process table

–The end of a time slice is signaled by an interrupt -> process switch.

Note:

● Schedule: plan the time and place (order) for events (processes, or tasks)

● Dispatch: complete or carry out the processes or tasks

image-20231011211824351

image-20231011212021529

image-20231011212457570

Deadlock

死锁是一种情况,其中两个或多个进程或线程互相等待对方释放它们所需的资源,导致它们无法继续执行。

●Processes block each other,neither can continue

●Conditions required for deadlock

  1. Competition for non-sharable resources

  2. Resources requested on a partial basis

=如果资源只被请求了一部分,进程或线程可能不会释放资源,因为它们仍需要更多的资源来完成任务。比图

  1. An allocated resource cannot be forcibly retrieved

2.3 Characteristics of OS

●Multi-user(users)

–Two or more users have individual accounts that allow them to work with programs and peripheral devices at the same time.

●Multi-tasking(users)

The computer is capable of operating multiple applications/programs/tasks at the same time.

●Multi-processing(CPU of OS)

The operating system can support two or more CPUs.

●Multi-threading(CPU of OS)

A program can be broken into smaller parts that are loaded as needed by the OS. Multithreading allows individual programs to run simultaneously.

OS的分类

image-20231012104617186

Network Operating System is for multiple purpose

网络操作系统需要 robust 来确保其稳定运行,即使在压力和异常情况下也要能够保持可用性。而冗余性则是为了提供备份和容错能力,以应对设备和数据故障,从而保障网络的可靠性和持久性。

冗余性:冗余性在网络操作系统中是非常重要的,因为它提供了备份和容错能力。网络中的各种组件和设备,包括服务器、路由器、交换机等,都可能会出现故障。通过使用冗余配置,即多个备份设备,可以确保在主设备发生故障时,系统可以自动切换到备用设备,从而保持网络的可用性。

数据保护:网络操作系统通常管理着组织的重要数据,包括用户信息、配置文件、日志等。如果这些数据丢失或损坏,可能会对业务造成严重损害。因此,冗余性也可以涵盖数据冗余,即数据备份和复制,以确保数据的安全性和完整性。

  1. Desktop OS

    • General Functions: File management, device management, storage & process management, app/user interfaces

    • Common desktop OSs

      −Microsoft Windows: Windows 7 −Macintosh: Mac OS X −Linux −UNIX

    • Major difference of operating systems: availability and how much can be accomplished using the GUI −

      −Open source or proprietary (e.g., Linux vs. Windows)

      −GUI (Graphical User Interface) or CLI (Command Line Interface) : Windows & MacOS vs. Linux& Unix

  2. Network OS

    • Share networked resources conveniently and efficiently.

      • General OS functions: file management, device management, storage and processor management

      • Networking-specialized functions

        −Efficient and reliable communication & networking functions

        −Providing networking services, remote jobs inputting & processing; file transmission; email; remote printing etc.

    • Common NOS include:

      −Microsoft Windows Server

      −Linux

      −UNIX

      −Mac OS X Server

2.4 Virtualization and its advantages

image-20231012111528695

each virtual machine level is an abstraction of the level below it

每个较高级别的虚拟机器都是基于底层较低级别的虚拟机器或硬件的抽象。它们使用下一级别提供的服务和功能,并通过提供更高级别的接口来隐藏底层复杂性。

"提供了抽象" 意味着操作系统在底层硬件(如处理器、内存和磁盘)之上创建了一个更高级别的接口和服务层。这个接口和服务层隐藏了底层硬件的复杂性,以简化上层软件的开发和运行。

在虚拟化技术中,虚拟机器级别表示虚拟机(Virtual Machine)的概念。虚拟机器级别不是指计算机系统的层次结构级别,而是表示虚拟机(VM)的抽象层次。虚拟机可以在同一物理硬件上模拟多个独立的计算机系统,每个虚拟机在虚拟机监控程序(Hypervisor)的管理下运行。每个虚拟机可以视为一个虚拟机器级别,因为它们提供了一个独立的计算环境,与其他虚拟机相互隔离。这种虚拟化技术常用于服务器虚拟化、云计算和测试/开发环境等领域。

Virtual Machine

当您在Windows操作系统上安装一个Linux系统时,您创建了一个虚拟机(Virtual Machine)。这个虚拟机是一个独立的计算机环境,运行在您的物理硬件和Windows操作系统之上,同时模拟了一个完整的Linux系统。

您的Linux系统将运行在一个虚拟机中,这个虚拟机可以视为一个虚拟机器级别,它提供了独立的Linux环境,并与您的Windows系统相互隔离。

  • Are protected environment

    –The image of a totally different architecture

    –Processes can run within the environment.

  • An imaginary computer.

    –The underlying real machine (host machine) uses its system resources to host a virtual machine.

    •Host OS (host machine)

    •Guest OS (virtual machine)

    –The real machine is under the control of the kernel.

    The kernel receives and manages all resource requests that emit from processes running in the virtual environment.

优点

  • Hosting a virtual machine allows users to access the functionalities and resources provided by an OS that is not on the local PC (guest OS).

–Provides access to applications, file-sharing services, and other productivity tools.

  • Users can further increases the functionality of their system by running multiple virtual machines.

  • For example, on a Windows machine, you can

–Run software that requires an older versions of Windows or nonWindows OS (Linux, MacOSetc.).

–Experiment with other operating systems.

–Test software on multiple operating systems.

虚拟工具(Virtualization Tools - Hypervisor)

  • The software that creates and manages a virtual machine on a host machine is called the hypervisor, or Virtual Machine Manager (VMM).

    • Allocates the physical system resources, such as CPU, RAM, and hard drive, to each virtual machine as needed.

  • Windows 10 Hyper-V is a built-in virtualization tool of Windows 10

    •Allows you to partition system resources for a Window OS among virtual machines .

例如: Hyper-V, VirtualBox、VMware Workstation、

image-20231012113916903

image-20231012113939787

image-20231012113951748

Part 3 Computer Networks

##3.1 Essence of Computer Network

●Essence of a computer network

  • An open system: any device can join under certain regulations and rules

•Rules or agreements need to be established first

  • Share transmitting media(传输介质, 光纤,电缆,卫星....): optical fiber cable or wireless frequencies (i.e., share resources!)

    share the transmition media, share the cable

image-20231015104845099

Communications

image-20231015105616919

  1. Asynchronous and Synchronous

    Time relationship between a sender/requestor and a receiver/responder

    • Synchronous:

      get a response before next request can be sent: interactive, e.g., skype talk

      (request is dependent on response, 等待你的response, 然后才发request)

    • Asynchronous

      many requests can be sent before getting a response: delayed, e.g., email, file transmission, instant messages….

      可以发很多requests

  2. Unicast, Multicast, Broadcast

    Number relationship between sender(s) and receiver(s)

    • Unicast: one-to-one(我只发给一个人, 其他人收不到)

    • Broadcast: one‐to‐all

    • Multicast: one‐to‐many or many‐to‐many

    • (Anycast: multiple devices may have the same address, e.g., mirror servers, any server can be forwarded data according to “load balancing”)

      Anycast 允许多个具有相同 IP 地址的目标节点存在于网络中,但这些节点位于不同的位置。当数据包传送到具有相同 Anycast IP 地址的多个节点时,路由器会选择距离数据包源最近的节点来处理请求,通常是根据路由器配置的度量标准。

      load balancing: 目的是确保请求到达 Anycast IP 地址时被路由到最合适的目标节点,以降低负载,减少延迟,并提高整体性能。

      Anycast 结合load balancing可用于分布式网络服务的构建

  3. PAN, LAN, WAN

    • PAN(Personal Area Network): enable devices communicate over the range of a person. E.g.,

      比如键盘鼠标... 连接手机的智能手表

    • LAN (Local Area Network): privately owned network that operates within and nearby a single building like a home, office or factory. E.g., 比如 A campus network ● A company network

    • MAN (Metropolitan Area Network): covers a city, E.g., 比如 Cable television networks ● Mobile network in a city

    • WAN (Wide Area Network):spans a large geographical area, often a country or continent. E.g., a network connect offices in Perth, Melbourne and Brisbane.

      May usedifferent techniques, such as Mobile Telecommunication Networks, Satellite Network etc.

  4. Internet

    A collection of interconnected networks is called an internetwork or internet–a global mesh of interconnected networks

image-20231015114955471

  1. Wireless, wired. mobile, fixed networks

    image-20231015115825335

    fixed: 有线网络, 提供固定位置的通信和互联网接入服务

    wired: provide big high band with, high data rate

    注意区分mobile和wireless

    wireless: 蓝牙, wifi

    mobile: 移动设备

    • Mobile Network 更专注于移动通信,为手机用户提供语音通话和数据服务,通常由移动运营商提供。它们允许用户在移动时保持连接,但通常需要与特定的运营商合同相关。
    • Wireless Network 通常是指无线局域网(Wi-Fi)等网络,它们用于提供无线互联网接入和局域网连接。这些网络适用于移动设备,但不依赖于特定的运营商,因此更广泛用于家庭、企业和公共场所。

3.2 Components and Structures of Computer Networks

  1. Components of Computer Networks
  • Hardware

    • End devices :interface between the human and communications network (end of the network, 包括笔记本, pc, 甚至服务器)

      image-20231015142132848

    • Intermediary devices (Gateway, router)

      image-20231015142228875

  • Software

    •Protocols: e.g., for forwarding/routing Data, TCP, SMTP, Http

    •Application software: browsers, Apps

    •Network management and control: DNS, ICMP,..(Internet Control Message Protocol)

  1. Structure of Networks

    ● The way that the devices are arranged, called topology.

    • Physical topology: the layout of computers and where the computers are positioned.

    • Logical topology: how the information flows through the network.

      TCP是end to end

      mesh 网格

      image-20231015142758669

      image-20231015142832072

3.3 Mechanisms and Protocols of Computer Networks

1.Layered Architecture

(1) Communication principles

  • Communication:

    –Transmission data or information from one place (device) to another (device)

    –Information: organized data

  • Three elements for communication

    • Source device
    • Channel(transmission medium)
    • Destination device

encoder 编码器, translate data into signal, sent to a transmitter

image-20231015144033135

(2) end-to-end Message Transmission

没太懂

image-20231015144634763

end-to-end Message Transmission原则强调了将关键功能(如错误检测和处理)留给通信的端点,以便在需要时进行处理,而不依赖于网络中的每个中间节点。

(3) Mechanisms and protocols

image-20231015144812486

(4) Hierarchical Layers

网络通信通常采用 OSI 模型(Open Systems Interconnection Model)或 TCP/IP 模型,这些模型将网络通信分成不同的层级,以便更好地组织和管理通信协议。一般来说,OSI 模型定义了七个层级,而 TCP/IP 模型定义了四个层级。

OSI 模型(七层):

  1. 物理层(Physical Layer)
  2. 数据链路层(Data Link Layer)
  3. 网络层(Network Layer)
  4. 传输层(Transport Layer)
  5. 会话层(Session Layer)
  6. 表示层(Presentation Layer)
  7. 应用层(Application Layer)

TCP/IP 模型(四层):

  1. 网络接口层(Network Interface Layer)
  2. 网络层(Network Layer)
  3. 传输层(Transport Layer)
  4. 应用层(Application Layer)

image-20231015154001993

image-20231015151946295

purpose of organizing the functions in different layout

peers : 同一网络通信层级的两个entities(实体),

如何判断两个entities是否在同一层?

  • 两个设备都使用TCP协议进行通信,则它们位于传输层

  • 了解每个协议的功能和责任可以帮助确定其所在的层级。例如,IP(Internet Protocol)位于网络层,负责路由数据包;而HTTP(Hypertext Transfer Protocol)位于应用层(使不同应用相互沟通),用于传输Web页面和数据

  • 如果两个实体在同一物理网络或数据链路上进行通信,它们通常位于数据链路层。

image-20231015153211152

image-20231016085234988

OSI reference Model

  1. 物理层(Physical Layer)
  2. 数据链路层(Data Link Layer)
  3. 网络层(Network Layer)
  4. 传输层(Transport Layer)
  5. 会话层(Session Layer)
  6. 表示层(Presentation Layer)
  7. 应用层(Application Layer)

image-20231015153340287

intermediary device are not involved in the converstion of the protocol

1Physical Layer

用不同的媒介传输bit(0和1), 可以是电,光, 或者其他形式的电磁波, 数据从网路接口出去后, 会经过不同的网络topology

2data link

bit 被封装为帧 Frame (表示数据的特殊名字),会加上mac地址(物理地址), (为了通过mac地址对不同设备进行数据的传输, 就出现了交换机),

交换机(Switch): 里面维护了一张表格, 为每个连接到SW的机器的mac地址, 以及对应的物理端口(物理端口是在交换机上可以看到的那个口), 因此这种方式比起HUB(集线器), 不再需要对数据进行广播, 也可以多台设备同时发送,

交换机满足了小规模局域网的互联互通, 那如果要一个很大的网络呢?

我们知道在每一层, 数据都会被加上东西, 这一层它加的Head 和原有的数据被封装为Frame, 那么这个head里有些什么呢?有源地址和目的地址(mac地址)

同时这一层还会进行差错纠正(物理层传输的时候传错的), 以及还会有流控制(一边传的多快,一边是小水管)

交换机 二层交换机 多接口网桥 是一个东西

路由器 三层交换机 网关 是一个东西

3Network

用mac的寻找地址是不可靠的, 因此需要ip(ip就是实现end-to end的), Responsible for routing and forwarding, i.e., how packets are forwarded from source to destination

Routing protocols: SDPF, RIP, BGP

数据被封装成packet, 下图是packet Header的内容, 含有20bits

Service type: 看看紧不紧急

flag: 是否有被分成很多块

Checksum: error control

options: timestamp, security 以及其它的 parameters

image-20231016154054613

IP: 用于标识网络, 以及标识设备(在哪个网络下)

该层的核心是路由器router( 也叫做网关), 路由器用于连接不同的网络,

1和3之间的通信, 走Switch, 1和2之间的通信, 走router

image-20231016093247295

它有自己的一个routing table, 有网络一的IP,也有网络2的IP

routing table中包含了路由器可以到达的目的网络。目的网络在路由表中不存在的数据包会被丢弃。

使用网络号来表示destination,

image-20231016154105976

数据如何发送? 默认网关是在路由器上(例如192.168.0.1, 属于网络1默认网关 ; 192.168.1.51, 属于网络2的默认网关), ping通的原理就是属于网络1的机器通过自己的路由表查到默认网关是192.168.0.1, 然后把消息发到默认网关(也就是路由器), 路由器通过查询自己的路由表, 查到receiver(192.168.1.254)的默认网关是 192.168.1.51, 然后通过这个网关把消息发出去

image-20231016100959144

但IP地址是不能直接用于通信的, 它是一个抽象地址, 要通信还是要使用mac地址(真实地址), 所以从1.51到1.254这个数据的过程是什么样的?

通过ip地址去查mac地址, 进行广播, 问谁的地址是1.254(ARP协议)

1.254 收到这条消息进行回复, 说: 是我, 并且发送了自己的mac地址

然后在mac层(Datalink层)进行传输

image-20231016101623521

整个数据的传输为 0.102 -> 1.254

传输数据头ip地址(源和目的地)数据头Mac地址(源和目的地)
0.102 -> 0.10.102, 1.254mac102, mac0.1
0.1 -> 1.510.102, 1.254(不变)mac0.1, mac1.51
1.51 -> 1.2540.102, 1.254(不变)mac 1.51, mac1.254

Packet就是网络层里数据的名字, 在封装为二层Frame之前就是Packet

(发的时候由上到下包装,接受时又下到上拆封)

image-20231015155121375

这一层的问题:

  1. 数据丢失

  2. 不知道有没有被好好地接收到

image-20231016154141205

4Transport

这一层负责TCP(Transmission Control Protocol)和UDP(User Datagram Protocol)传输

一般叫segment

Spit the data from the higher layers into pieces (segment or PDU),to ensure all the pieces arrive at the other end: flow control and error control

flow control

Flow control: a faster sender cannot swamp a slow receiver with more messages it can handle; no congestions at the end nodes (sender and receiver) and routers (stop‐and‐wait,).

image-20231016161227095

Methods: Handshake mechanism, Sliding window, stop‐and‐wait

Error control

To ensure all the pieces arrive at the other end: flow control and error control.

● Error control: all the segments should be received: numbering and checking the “sequence” number, if a gap, then ask to retransmit

–Automatic Repeat Request(ARQ) 重传请求

–Error‐detecting code (CRC, Cyclic Redundancy Check,循环冗余校验, CRC 是一种校验和算法,用于检测数据是否在传输过程中发生了变化或损坏。但它不能纠正错误,只能检测到错误的存在。在纠错能力更强的情况下,通常会使用更复杂的技术,如重传请求(ARQ)或前向纠错(FEC)来修复错误

image-20231016161437970

Methods: Handshake, Acknowledgement, Retransmission

Connection Setup and Release

three way handshake

DR: disconnect request

CR: connect request

ACK: acknowledgement

  1. 第一步 - 客户端请求连接
    • 客户端(通常是一个计算机或设备)向服务器发送一个特殊的TCP数据包,称为SYN(同步)包。
    • SYN包包含客户端选择的初始序列号(ISN),以及一些其他控制信息。
    • 客户端进入"SYN_SENT"状态,等待服务器的响应。
  2. 第二步 - 服务器确认连接请求
    • 服务器接收到客户端的SYN包后,向客户端发送一个SYN-ACK包。
    • SYN-ACK包包含确认号(ACK),确认客户端的SYN请求,同时也包含服务器选择的初始序列号(ISN)。
    • 服务器进入"SYN_RCVD"状态,等待最后的确认。
  3. 第三步 - 客户端确认连接
    • 客户端接收到服务器的SYN-ACK包后,向服务器发送一个ACK包。
    • ACK包确认了服务器的SYN-ACK,同时也确认了服务器的ISN。
    • 服务器收到这个ACK包后,进入"ESTABLISHED"状态,而客户端也进入"ESTABLISHED"状态。
    • 此时,连接已经建立,双方可以开始进行数据传输。

image-20231016163001631

用mac和ip可以到达对方电脑, 如何让数据区指定的软件服务上? -> port(端口)

image-20231016164118363

TCP Header

image-20231016164413441

image-20231016164255654

sequence number: flow control, error control

They tell the router and end-devices how to interpret the meanings of the packet, they are the rules or a agreements among the devices in the network

实现了end to end, 段就是数据的名称

image-20231015160013622

5Session

登陆服务, 不需要每次重新登陆

同步服务, 比如接着上次看

6Presentation

数据的编码和解码, 文件压缩

7application

为应用程序提供交互服务, 比如文件传输服务(FTP), 远程登录服务, 打印服务, 电子邮件服务, 数据库服务

报文: 指的就是session, presentation, application的数据, 也就是报文

image-20231015160618793

image-20231015160703486

Services and Protocols

Protocols : between the entities at same layers of different devices

Service: between two adjacent layers

image-20231016084330209

(5) 信息如何流动

image-20231016084556864

(6) General Functions of Protocols

image-20231016084939537

image-20231016085059989

RTP: interactive , like skype

SMTP: email

(7) Advantages of Layered Model

image-20231016085746681

一层的改变不会影响另一层, 也就是说可以随便升级某一层, 可以支持complex system

2. Internet& Transport(TCP/IP)

​ 写在上面的transport layer中了

3. Addressing & Naming Schemes (IP Address, DNS)

image-20231016190646154

image-20231016195327403

image-20231016202137879

ISP: Internet Service Provider, map and translate public address with the private address

观察下图, 看看有什么问题

image-20231016203026620

因为现在这个ip是B类网络, B类IP地址的网络号占用前两个字节(16位),而主机号占用后两个字节(16位)。这意味着在一个B类网络中,有2^16 - 2 个可用的主机地址。减去2是因为其中一个地址保留为网络地址,另一个地址保留为广播地址,不能用于分配给具体的主机。

2^16: 2 to the power of 16

R1这个router 需要非常强大, 因为要负责 2^16的host 的manage, 如果下属没有那么多host, 那么ip地址会被浪费,同时, 这么多host都 organized in a flat way, 没有分类, 很混乱, 需要 subnetting

image-20231016212041199

borrow some bits from host id

Subnet Mask

  • 32‐bit number, e.g., 255.255.192.0

  • Identify which bits in the IP address are for the network ID (of a subnet) and which for the host ID

  • Function

    –Distinguish network ID and host ID

    –Segment a big network into small subnetworks

在子网掩码中,连续的1表示网络号的位,连续的0表示主机号的位。这意味着前18位用于网络号,后面的位用于主机号。这种子网掩码位的表示方法称为CIDR(无类别域间路由)表示法,它允许更灵活地划分IP地址,而不再受到固定的A、B、C类地址限制。

image-20231016214204287

给定的主机IP地址:141.14.72.24 子网掩码:255.255.192.0 (这个子网掩码对应于/18子网掩码位)

首先,将主机IP地址和子网掩码都转换为二进制形式,以便进行逻辑与操作:

主机IP地址(二进制):10001101.00001110.01001000.00011000 子网掩码(二进制):11111111.11111111.11000000.00000000

接下来,执行逻辑按位与操作。只有在两个相应的位都为1时,结果位才为1。否则,结果位为0。


10001101.00001110.01001000.00011000 (主机IP地址)
11111111.11111111.11000000.00000000 (子网掩码)
-----------------------------------
10001101.00001110.01000000.00000000

最终结果是: 网络ID(二进制):10001101.00001110.01000000.00000000

将这个二进制值转换回十进制,就得到了网络ID: 网络ID(十进制):141.14.64.0

那么, 该例子有多少台主机呢?

2^14, 为什么是14, 因为255.255.192.0中的192, 换为二进制为11000000, 然后后面是0, 换成二进制就是00000000, 一共有14个0

image-20231016214426537

image-20231016215626082

CIDR 允许将多个连续的 IP 地址块聚合成一个更大的地址块。这减少了路由表中的条目数,提高了路由表的效率。CIDR 地址块通常以前缀长度的形式表示,例如,/24 表示包括 24 位网络号在内的地址块。

CIDR 允许网络管理员将一个大的 IP 地址块划分为多个子网,以满足不同网络的需求。每个子网可以有自己的子网掩码和子网规模。

image-20231016221036008

DHCP 是 Dynamic Host Configuration Protocol(动态主机配置协议)的缩写,是一种网络协议,用于自动为计算机和其他网络设备分配 IP 地址和其他网络配置信息。DHCP 的主要功能是简化网络管理,减少手动 IP 地址配置的需求,以及有效地重用和分配 IP 地址。

image-20231016221654158

image-20231016222236474

IPv6 Addresses

image-20231016223252621

8 组 16 位的十六进制数表示

DNS

Application Service for resolving IP Address from a Name

  • Three Components

    –Name Space

    –Servers making the space available

    –Clients (resolvers) make queries to the servers

  • Features

    –Data retrieved globally, maintained locally

    –DNS lookup can be performed by any device

    –Remote DNS data are locally cacheable to improve performance

image-20231016224348246

image-20231016224447329

DNS(Domain Name System,域名系统)是互联网中用于将人类可读的域名(例如 www.example.com)转换为计算机可理解的-yh5fi61a781a70gp64akhffsp7y5bszelnp/) IP 地址(例如 192.0.2.1)的分布式命名系统。DNS的工作原理涉及以下步骤:

  1. 域名查询请求: 当您在浏览器中键入一个域名,例如 www.example.com,或点击一个链接时,您的计算机首先需要确定该域名对应的 IP 地址。它会发送一个域名查询请求到本地 DNS 解析器,通常由您的互联网服务提供商(ISP)提供。
  2. 本地DNS解析器: 本地 DNS 解析器是您的计算机或路由器上运行的程序,它负责处理域名解析请求。如果本地解析器已经在之前解析过相同的域名,它会从本地缓存中获取答案,从而避免向远程DNS服务器发出请求。
  3. 递归查询: 如果本地解析器没有该域名的解析记录,它将发起一个递归查询。在这个查询过程中,本地解析器会向根DNS服务器发出查询请求。
  4. 根DNS服务器: 根DNS服务器是DNS层次结构的最顶层,全球只有一组根DNS服务器。根DNS服务器不直接提供域名到IP地址的映射,而是指向下一级的顶级域(TLD Top Level)DNS服务器,例如.com、.org、.net等。
  5. 顶级域(TLD)DNS服务器: 一旦根DNS服务器将请求指向正确的TLD,TLD服务器会告诉本地解析器,哪个权威DNS服务器负责该特定域的解析。例如,对于.com域名,TLD DNS服务器将指向.com顶级域的权威DNS服务器(authoritative server, 比如.google)。
  6. 权威DNS服务器authoritative server: 权威DNS服务器是负责特定域名解析的最终实体。本地解析器将查询发送到权威DNS服务器,请求该域名的IP地址。
  7. 响应返回: 权威DNS服务器返回包含域名到IP地址映射的响应给本地解析器。本地解析器将这个响应存储在本地缓存中,以备将来的查询。然后,它将IP地址返回给请求的应用程序,如浏览器。
  8. 应用程序连接: 应用程序(例如浏览器)现在知道了目标服务器的IP地址,它可以发起连接请求到该IP地址,从而访问相应的网站或服务。

这个过程是分布式的,因为DNS系统是全球性的,包括多个级别的DNS服务器,每个级别负责处理特定范围的域名。这种层次结构和缓存机制使得DNS查询能够快速响应,并减轻了全局互联网上DNS服务器的负载。

image-20231016224859005

TTL: time to live, 存在这里十天

有两种searching

一个是直接返回来(cache)里面有, 这个叫recursive

一个是到处去问问问, 这个叫iterative

4. Hyper Text Transfer Protocol(HTTP)

image-20231017105338579

TCP比较靠谱, 东西不容易丢失

Stateless: 不保存之前请求的信息, 那怎么办? 用cookie

image-20231017105330081

image-20231017105546541

HTTP Request

image-20231017110152824

image-20231017111013802

Actions

HTTP/1.0

● GET: request the specified pages, and respond with the objects

● POST: request to process the data (which is included in the request message). This may result the establishment of new resources or modification to the existing resources.

● HEAD: asks server to leave requested object out of response (only ask for the head)

HTTP/1.1

● GET, POST, HEAD

● PUT: uploads file in body to the path specified in URL field

● DELETE: deletes file specified in the URL field

● TRACE: echo what has been requested (for test or diagnosis)

● OPTIONS: allow the clients to see the performance of server or what request methods are supported for resources.

image-20231017110932447

image-20231017110957006

Etag:和modified绑定, 有改, 就给一个Etag

Status Code

200 OK : request succeeded, the requested objects included in this message.

301 Moved Permanently: requested object moved, new location specified in this message (in the Data fields)

400 Bad Request: the request message is not understood by server

404 Not Found: the requested document not found on this server

505 HTTP Version Not Supported 

image-20231017111525176

image-20231017111532329

Cookies

image-20231017111615961

image-20231017112903544

image-20231017113042470