Protection: Access Control and Sandboxing

Protection is the mechanism in an operating system that controls how processes and users access resources. While security defends the system from external threats, protection manages access rights internally.

1. Protection Goals

┌────────────────────────────────────────┐
│        Protection Principles           │
│                                        │
│  Principle of Least Privilege          │
│                                        │
│  → Grant processes/users only the      │
│    minimum privileges needed for       │
│    their tasks                         │
│                                        │
│  Examples:                             │
│  - Web server: only read web files     │
│  - DB server: only access data dir     │
│  - Regular user: no kernel access      │
│                                        │
│  Need-to-Know (Separation):           │
│  → Each module can only access         │
│    information needed for its role     │
└────────────────────────────────────────┘

2. Protection Rings

A hardware-level structure that layers privilege levels.

┌─────────────────────────────────────┐
│       Protection Rings (x86)        │
│                                     │
│    ┌─────────────────────────┐      │
│    │ Ring 3: User apps       │      │
│    │  ┌─────────────────┐    │      │
│    │  │ Ring 2: (unused) │    │      │
│    │  │  ┌───────────┐  │    │      │
│    │  │  │ Ring 1:   │  │    │      │
│    │  │  │ (unused)  │  │    │      │
│    │  │  │ ┌───────┐ │  │    │      │
│    │  │  │ │Ring 0:│ │  │    │      │
│    │  │  │ │Kernel │ │  │    │      │
│    │  │  │ └───────┘ │  │    │      │
│    │  │  └───────────┘  │    │      │
│    │  └─────────────────┘    │      │
│    └─────────────────────────┘      │
│                                     │
│  Ring 0: Full hardware access       │
│  Ring 3: Limited instruction set    │
│  System calls transition rings      │
└─────────────────────────────────────┘

Modern System Privilege Structure

┌───────────────────────────────────────┐
│ Hypervisor mode (Ring -1, VMX root)   │
├───────────────────────────────────────┤
│ Kernel mode (Ring 0)                  │
│ - Memory management, process mgmt    │
│ - Device drivers, file systems       │
├───────────────────────────────────────┤
│ User mode (Ring 3)                    │
│ - Applications, libraries            │
│ - Request kernel services via syscall│
└───────────────────────────────────────┘

3. Protection Domains

A protection domain is the set of resources a process can access.

Domain D1 (User A):
  file1.txt → read, write
  file2.txt → read
  printer  → print

Domain D2 (User B):
  file2.txt → read, write
  file3.txt → read, write, execute

Domain D3 (Daemon process):
  file1.txt → read
  network  → send, receive

Domain Switching

A process transitioning from one domain to another.

// setuid 프로그램에 의한 도메인 전환 예시
// passwd 명령어: 일반 사용자 → root 권한으로 전환

// 실행 전: uid = 1000 (일반 사용자)
// /usr/bin/passwd 는 setuid root

// 실행 중: euid = 0 (root), ruid = 1000
// → /etc/shadow 파일 수정 가능

// 실행 후: uid = 1000 (원래 사용자로 복귀)

4. Access Matrix

A model that systematically represents access rights to resources.

           │  file1   │  file2   │  file3   │ printer │ Domain 2│
───────────┼──────────┼──────────┼──────────┼─────────┼─────────┤
Domain 1   │ read     │ read     │          │ print   │ switch  │
(User A)   │ write    │          │          │         │         │
───────────┼──────────┼──────────┼──────────┼─────────┼─────────┤
Domain 2   │          │ read     │ read     │         │         │
(User B)   │          │ write    │ write    │         │         │
───────────┼──────────┼──────────┼──────────┼─────────┼─────────┤
Domain 3   │ read     │          │          │         │         │
(Daemon)   │          │          │ execute  │         │         │

Access Matrix Implementation

1. Access Control Lists (ACL) - Column-based

file1's ACL:
┌────────────┬──────────────┐
│ Domain 1   │ read, write  │
│ Domain 3   │ read         │
└────────────┴──────────────┘

file2's ACL:
┌────────────┬──────────────┐
│ Domain 1   │ read         │
│ Domain 2   │ read, write  │
└────────────┴──────────────┘

// POSIX ACL 설정 예시 (Linux)
// 명령어로 ACL 설정
// setfacl -m u:user1:rw file1.txt
// setfacl -m g:developers:r file1.txt

// ACL 확인
// getfacl file1.txt
// # file: file1.txt
// # owner: root
// # group: root
// user::rw-
// user:user1:rw-
// group::r--
// group:developers:r--
// mask::rw-
// other::---

2. Capability Lists - Row-based

Domain 1's capabilities:
┌──────────┬──────────────┐
│ file1    │ read, write  │  ← capability
│ file2    │ read         │     = object + rights
│ printer  │ print        │
└──────────┴──────────────┘

Domain 2's capabilities:
┌──────────┬──────────────┐
│ file2    │ read, write  │
│ file3    │ read, write  │
└──────────┴──────────────┘

ACL vs Capability Comparison

Property	ACL	Capability
Perspective	Object (resource) centered	Subject (process) centered
Checking	Search ACL on file access	Check process capability list
Revocation	Easy (remove from ACL)	Hard (must track all copies)
Examples	UNIX permissions, Windows	Mach, seL4 microkernels

5. Role-Based Access Control (RBAC)

Instead of assigning permissions directly to users, permissions are assigned to roles and users are assigned to roles.

Users              Roles              Permissions
┌────────┐     ┌──────────┐     ┌──────────────────┐
│ Alice  │────→│ Admin    │────→│ User management  │
│ Bob    │     │          │     │ System config    │
└────────┘     └──────────┘     │ Log access       │
                                └──────────────────┘
┌────────┐     ┌──────────┐     ┌──────────────────┐
│ Carol  │────→│ Developer│────→│ Code repo access │
│ Dave   │     │          │     │ Build system use │
│ Eve    │────→│          │     │ Test env access  │
└────────┘     └──────────┘     └──────────────────┘

┌────────┐     ┌──────────┐     ┌──────────────────┐
│ Eve    │────→│ Auditor  │────→│ Read logs        │
└────────┘     └──────────┘     │ Generate reports │
                                └──────────────────┘
(Eve can hold both Developer and Auditor roles)

RBAC Advantages

Easy Management: Just add/remove roles on user join/leave
Least Privilege: Assign only necessary permissions to roles
Separation of Duties: Can restrict conflicting roles from being held simultaneously

6. Mandatory Access Control (MAC) - SELinux

DAC vs MAC

DAC (Discretionary):               MAC (Mandatory):
- File owner decides permissions   - System policy decides
- Users freely change permissions  - Only admin can change policy
- UNIX default file permissions    - SELinux, AppArmor

DAC problem:
If user gains root → all files accessible

MAC advantage:
Even root is restricted by SELinux policy

SELinux Concepts

SELinux security context:
user : role : type : level

Example:
system_u:system_r:httpd_t:s0    (Apache web server)
system_u:object_r:httpd_sys_content_t:s0  (Web content)

Policy rule:
allow httpd_t httpd_sys_content_t : file { read open getattr };
→ Processes of type httpd_t can only read, open, and
  get attributes of files of type httpd_sys_content_t

# SELinux 상태 확인
getenforce
# Enforcing, Permissive, Disabled

# 파일의 보안 컨텍스트 확인
ls -Z /var/www/html/
# -rw-r--r--. root root system_u:object_r:httpd_sys_content_t:s0 index.html

# 프로세스의 보안 컨텍스트 확인
ps -eZ | head -5
# system_u:system_r:init_t:s0     1  ?  00:00:03 systemd
# system_u:system_r:sshd_t:s0   850  ?  00:00:00 sshd

# 보안 컨텍스트 변경
chcon -t httpd_sys_content_t /var/www/html/newfile.html

7. Sandboxing

Isolates a program's execution environment to restrict access to the rest of the system.

Sandboxing Techniques

┌────────────────────────────────────────────┐
│           Sandboxing Layers                │
│                                            │
│  System Call Filtering (seccomp-bpf):     │
│  ┌──────────────────────────────────┐      │
│  │ Process                         │      │
│  │ → open(), read() → Allow        │      │
│  │ → execve()       → Block        │      │
│  │ → socket()       → Block        │      │
│  └──────────────────────────────────┘      │
│                                            │
│  Namespace Isolation (Linux):             │
│  ┌──────────────────────────────────┐      │
│  │ PID namespace: Process isolation│      │
│  │ NET namespace: Network isolation│      │
│  │ MNT namespace: FS isolation     │      │
│  │ USER namespace: User isolation  │      │
│  └──────────────────────────────────┘      │
│                                            │
│  chroot / pivot_root:                      │
│  ┌──────────────────────────────────┐      │
│  │ Restricts root directory visible│      │
│  │ to the process for FS isolation │      │
│  └──────────────────────────────────┘      │
└────────────────────────────────────────────┘

seccomp-bpf Example

#include <seccomp.h>
#include <stdio.h>
#include <unistd.h>

int main() {
    // seccomp 필터 초기화 (기본: 모든 시스템 콜 차단)
    scmp_filter_ctx ctx = seccomp_init(SCMP_ACT_KILL);

    // 허용할 시스템 콜만 추가
    seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(read), 0);
    seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(write), 0);
    seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(exit), 0);
    seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(exit_group), 0);

    // 필터 적용
    seccomp_load(ctx);

    // 이후 read, write, exit만 사용 가능
    // 다른 시스템 콜 호출 시 프로세스 종료
    printf("Sandboxed process running\n");

    seccomp_release(ctx);
    return 0;
}

Browser Sandboxing

┌──────────────────────────────────────────┐
│              Browser                      │
│                                          │
│  ┌──────────┐  ┌──────────┐              │
│  │ Browser  │  │ Browser  │              │
│  │ Process  │  │ Process  │ ← High priv  │
│  │ (main)   │  │ (GPU)    │              │
│  └────┬─────┘  └──────────┘              │
│       │                                  │
│  ┌────┴─────┬──────────┬──────────┐      │
│  │ Renderer│ Renderer  │ Renderer │      │
│  │ (tab 1) │ (tab 2)  │ (tab 3)  │      │
│  │ Sandbox │ Sandbox  │ Sandbox  │ ← Low│
│  │         │          │          │  priv │
│  │ File    │ Network  │ Syscall  │      │
│  │ limited │ limited  │ limited  │      │
│  └─────────┴──────────┴──────────┘      │
└──────────────────────────────────────────┘

8. Code Signing

A mechanism that verifies the origin and integrity of executables.

Developer:
1. Write code
2. Generate hash: hash(code) = H
3. Create signature: sign(H, private_key) = S
4. Distribute code + signature S + certificate

User (OS):
1. Download code
2. Extract public key from certificate
3. Verify signature: verify(S, public_key) = H'
4. Compute code hash: hash(code) = H
5. H == H' ? Allow execution : Block

┌────────────────────────────────────────────┐
│      OS Code Signing Policies              │
│                                            │
│  macOS:                                    │
│  - Gatekeeper: Only signed apps allowed   │
│  - Notarization: Apple server verification│
│                                            │
│  Windows:                                  │
│  - Authenticode: Executable signing       │
│  - SmartScreen: Reputation-based filter   │
│                                            │
│  Linux:                                    │
│  - Kernel module signature verification   │
│  - Secure Boot: UEFI boot chain verify   │
│                                            │
│  Mobile (iOS/Android):                     │
│  - App store signing required             │
│  - iOS: All code must be signed           │
└────────────────────────────────────────────┘

9. Summary

Protection Principles: Least privilege, separation of privilege, separation of duties
Protection Rings: Hardware-level privilege hierarchy (Ring 0 - Ring 3)
Access Matrix: Implemented as ACLs (object-centered) and capability lists (subject-centered)
RBAC: Role-based access control for easier management
MAC/SELinux: Mandatory access control restricts even root by policy
Sandboxing: Process isolation via seccomp, namespaces, etc.
Code Signing: Verifying origin and integrity of executable code

Quiz: Protection

Q1. What is the difference between DAC and MAC?

A1. DAC (Discretionary Access Control) allows file owners to freely set and change permissions. MAC (Mandatory Access Control) determines access based on security policies defined by the system administrator, and individual users or even root cannot bypass the policy.

Q2. Compare the advantages and disadvantages of ACL and capability list approaches.

A2. ACLs are object-centered, making it easy to see who can access a specific file and to revoke permissions. Capability lists are subject-centered, allowing quick identification of what resources a specific process can access, but it is difficult to find and revoke all capabilities for a specific resource.

Q3. How does sandboxing contribute to security?

A3. Sandboxing strictly limits the system resources a program can access. For example, even if a browser's renderer process is compromised through a vulnerability, the sandbox restricts file system access and system calls, making it difficult for the attacker to take over the entire system.