Networks and Distributed Systems

Modern computing environments operate through multiple networked computers working together. This article examines networking fundamentals, distributed system concepts and challenges, distributed file systems, MapReduce, and distributed coordination mechanisms.

1. Network Types

┌───────────────────────────────────────────────┐
│         Network Classification by Scale        │
│                                               │
│  PAN (Personal Area Network):                 │
│  ┌───────────────────┐                        │
│  │ Bluetooth, NFC     │  ~ meters             │
│  └───────────────────┘                        │
│                                               │
│  LAN (Local Area Network):                    │
│  ┌───────────────────┐                        │
│  │ Ethernet, Wi-Fi   │  Building/campus       │
│  │ 1~10 Gbps         │                        │
│  └───────────────────┘                        │
│                                               │
│  MAN (Metropolitan Area Network):             │
│  ┌───────────────────┐                        │
│  │ City-scale network│                        │
│  └───────────────────┘                        │
│                                               │
│  WAN (Wide Area Network):                     │
│  ┌───────────────────┐                        │
│  │ Cross-country/     │  Internet             │
│  │ continent          │                        │
│  └───────────────────┘                        │
└───────────────────────────────────────────────┘

2. TCP/IP Model

┌─────────────────────────────────────────────┐
│           TCP/IP 4-Layer Model              │
│                                             │
│  ┌─────────────────┐                        │
│  │ Application     │  HTTP, FTP, DNS, SSH   │
│  │                 │  Socket API            │
│  ├─────────────────┤                        │
│  │ Transport       │  TCP (reliable, ordered)│
│  │                 │  UDP (fast, connless)   │
│  ├─────────────────┤                        │
│  │ Internet        │  IP (addressing,routing)│
│  │                 │  ICMP, ARP             │
│  ├─────────────────┤                        │
│  │ Link            │  Ethernet, Wi-Fi       │
│  │                 │  MAC addresses         │
│  └─────────────────┘                        │
└─────────────────────────────────────────────┘

TCP vs UDP

Property	TCP	UDP
Connection	Connection-oriented (3-way)	Connectionless
Reliability	Guaranteed (retransmit)	Not guaranteed
Speed	Relatively slower	Fast
Use cases	Web, email, file transfer	Streaming, DNS, games

Socket Programming Example

// TCP 서버 예시 (C)
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>

int main() {
    int server_fd, client_fd;
    struct sockaddr_in server_addr, client_addr;
    socklen_t client_len = sizeof(client_addr);
    char buffer[1024];

    // 1. 소켓 생성
    server_fd = socket(AF_INET, SOCK_STREAM, 0);

    // 2. 주소 바인딩
    server_addr.sin_family = AF_INET;
    server_addr.sin_addr.s_addr = INADDR_ANY;
    server_addr.sin_port = htons(8080);
    bind(server_fd, (struct sockaddr *)&server_addr,
         sizeof(server_addr));

    // 3. 수신 대기
    listen(server_fd, 5);
    printf("서버 대기 중 (포트 8080)...\n");

    // 4. 연결 수락
    client_fd = accept(server_fd,
                       (struct sockaddr *)&client_addr,
                       &client_len);

    // 5. 데이터 수신
    int bytes = recv(client_fd, buffer, sizeof(buffer) - 1, 0);
    buffer[bytes] = '\0';
    printf("수신: %s\n", buffer);

    // 6. 응답 전송
    const char *response = "Hello from server!";
    send(client_fd, response, strlen(response), 0);

    // 7. 연결 종료
    close(client_fd);
    close(server_fd);
    return 0;
}

// TCP 클라이언트 예시 (C)
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>

int main() {
    int sock;
    struct sockaddr_in server_addr;
    char buffer[1024];

    sock = socket(AF_INET, SOCK_STREAM, 0);

    server_addr.sin_family = AF_INET;
    server_addr.sin_port = htons(8080);
    inet_pton(AF_INET, "127.0.0.1", &server_addr.sin_addr);
    connect(sock, (struct sockaddr *)&server_addr,
            sizeof(server_addr));

    const char *message = "Hello from client!";
    send(sock, message, strlen(message), 0);

    int bytes = recv(sock, buffer, sizeof(buffer) - 1, 0);
    buffer[bytes] = '\0';
    printf("서버 응답: %s\n", buffer);

    close(sock);
    return 0;
}

3. Distributed Systems

Multiple independent computers cooperate over a network to function as a single system.

Advantages and Challenges

Advantages:                        Challenges:
┌──────────────────────────┐      ┌──────────────────────────┐
│ Resource sharing          │      │ Network failures          │
│ - Share storage, compute │      │ - Partial failure handling│
│                          │      │                          │
│ Scalability              │      │ Consistency               │
│ - Horizontal scaling     │      │ - Data replica consistency│
│                          │      │                          │
│ Availability             │      │ Security                  │
│ - Auto-recovery on       │      │ - Network communication  │
│   node failure           │      │   security               │
│                          │      │                          │
│ Performance              │      │ Clock synchronization    │
│ - Parallel processing    │      │ - Event ordering in      │
│   throughput             │      │   distributed env        │
└──────────────────────────┘      └──────────────────────────┘

CAP Theorem

A distributed system cannot simultaneously satisfy Consistency, Availability, and Partition Tolerance.

        Consistency (C)
        ╱       ╲
       ╱   CA    ╲
      ╱ (single   ╲
     ╱   server)    ╲
    ╱                 ╲
Availability (A) ── Partition Tolerance (P)
         AP / CP

Since network partitions are inevitable:
- CP system: Consistency first (e.g., ZooKeeper, HBase)
  → Rejects some requests during partition
- AP system: Availability first (e.g., Cassandra, DynamoDB)
  → Allows inconsistency during partition, converges later

4. Distributed File Systems

NFS (Network File System)

Client A ──→ ┐
Client B ──→ ├→ NFS Server ──→ Local Disk
Client C ──→ ┘      │
                    └→ /exports/shared/
                        ├── project/
                        ├── data/
                        └── config/

Features:
- Access remote files as local (transparency)
- POSIX-compatible interface
- Client-side caching for performance

GFS (Google File System)

A distributed file system designed for large files.

┌──────────────────────────────────────────┐
│            GFS Architecture              │
│                                          │
│  ┌───────────────┐                       │
│  │  GFS Master   │  ← Metadata mgmt     │
│  │  (Name Node)  │  (file → chunk map)  │
│  └───────┬───────┘                       │
│          │                               │
│  ┌───────┼───────┬───────┐               │
│  ▼       ▼       ▼       ▼               │
│ ┌────┐ ┌────┐ ┌────┐ ┌────┐             │
│ │Chunk│ │Chunk│ │Chunk│ │Chunk│           │
│ │Srv 1│ │Srv 2│ │Srv 3│ │Srv 4│           │
│ └────┘ └────┘ └────┘ └────┘             │
│                                          │
│ Files → split into 64MB chunks           │
│ Each chunk replicated to 3 servers       │
└──────────────────────────────────────────┘

HDFS (Hadoop Distributed File System)

Open-source implementation of GFS, optimized for large-scale data processing.

┌─────────────────────────────────────────┐
│            HDFS Architecture            │
│                                         │
│  ┌────────────┐                         │
│  │ NameNode   │ ← Metadata (in memory) │
│  │            │   Filename→block map    │
│  │            │   Block→DataNode map    │
│  └─────┬──────┘                         │
│        │ Heartbeat + Block Report       │
│  ┌─────┼──────┬──────────┐              │
│  ▼     ▼      ▼          ▼              │
│ ┌────┐┌────┐┌────┐    ┌────┐            │
│ │DN 1││DN 2││DN 3│    │DN 4│            │
│ │Blk1││Blk1││Blk2│    │Blk2│            │
│ │Blk3││Blk2││Blk3│    │Blk1│            │
│ └────┘└────┘└────┘    └────┘            │
│                                         │
│ Default block size: 128MB               │
│ Replication factor: 3 (default)         │
└─────────────────────────────────────────┘

// HDFS Java API 예시
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FSDataInputStream;
import java.io.BufferedReader;
import java.io.InputStreamReader;

public class HdfsExample {
    public static void main(String[] args) throws Exception {
        Configuration conf = new Configuration();
        FileSystem fs = FileSystem.get(conf);

        // 파일 쓰기
        Path writePath = new Path("/user/data/output.txt");
        FSDataOutputStream out = fs.create(writePath);
        out.writeUTF("Hello, HDFS!");
        out.close();

        // 파일 읽기
        Path readPath = new Path("/user/data/output.txt");
        FSDataInputStream in = fs.open(readPath);
        BufferedReader reader = new BufferedReader(
            new InputStreamReader(in)
        );
        String line;
        while ((line = reader.readLine()) != null) {
            System.out.println(line);
        }
        reader.close();

        // 파일 삭제
        fs.delete(new Path("/user/data/output.txt"), false);
        fs.close();
    }
}

5. MapReduce

A programming model for processing large-scale data in parallel.

Input data (3 splits):

Split 1: "hello world hello"
Split 2: "world foo hello"
Split 3: "bar foo hello"

     │           │           │
     ▼           ▼           ▼
  ┌──────┐   ┌──────┐   ┌──────┐
  │Map 1 │   │Map 2 │   │Map 3 │   ← Map phase
  └──┬───┘   └──┬───┘   └──┬───┘
     │           │           │
  (hello,1)   (world,1)   (bar,1)
  (world,1)   (foo,1)     (foo,1)
  (hello,1)   (hello,1)   (hello,1)
     │           │           │
     └─────┬─────┘───────────┘
           │
     Shuffle & Sort (group by key)
           │
  ┌────────┴────────────────────────┐
  │ bar:    [1]                     │
  │ foo:    [1, 1]                  │
  │ hello:  [1, 1, 1, 1]           │
  │ world:  [1, 1]                 │
  └────────┬────────────────────────┘
           │
     ┌─────┴──────┐
     ▼            ▼
  ┌──────┐   ┌──────┐
  │Red. 1│   │Red. 2│   ← Reduce phase
  └──┬───┘   └──┬───┘
     │           │
  bar:1       hello:4
  foo:2       world:2

// WordCount MapReduce 예시 (Java)
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import java.io.IOException;

public class WordCountMapper
    extends Mapper<Object, Text, Text, IntWritable> {

    private final static IntWritable one = new IntWritable(1);
    private Text word = new Text();

    public void map(Object key, Text value, Context context)
        throws IOException, InterruptedException {
        String[] words = value.toString().split("\\s+");
        for (String w : words) {
            word.set(w);
            context.write(word, one);
        }
    }
}

public class WordCountReducer
    extends Reducer<Text, IntWritable, Text, IntWritable> {

    public void reduce(Text key, Iterable<IntWritable> values,
                       Context context)
        throws IOException, InterruptedException {
        int sum = 0;
        for (IntWritable val : values) {
            sum += val.get();
        }
        context.write(key, new IntWritable(sum));
    }
}

6. Cloud Storage

Services for storing and managing large-scale data in cloud environments.

┌────────────────────────────────────────────────┐
│       Cloud Storage Service Comparison          │
│                                                │
│  AWS:                                          │
│  ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐          │
│  │ S3   │ │ EBS  │ │ EFS  │ │Glacier│         │
│  │Object│ │Block │ │File  │ │Archive│         │
│  └──────┘ └──────┘ └──────┘ └──────┘          │
│                                                │
│  GCP:                                          │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐       │
│  │ Cloud    │ │Persistent│ │ Filestore│       │
│  │ Storage  │ │ Disk     │ │          │       │
│  └──────────┘ └──────────┘ └──────────┘       │
│                                                │
│  Storage Classes (S3):                         │
│  Standard → IA → Glacier → Deep Archive       │
│  (Hot)     (Warm) (Cold)   (Archive)          │
│  High access ───────────→ Low access          │
│  High cost ──────────────→ Low cost           │
└────────────────────────────────────────────────┘

7. Distributed Coordination

Consensus Algorithm - Raft

An algorithm for nodes in a distributed system to agree on the same state.

Raft Leader Election:

Node A (Follower)  Node B (Leader)  Node C (Follower)
    │                   │                │
    │  ←── Heartbeat ── │ ── Heartbeat → │
    │                   │                │
    │                   X (failure)      │
    │                   │                │
    │  (timeout)        │                │
    │  → Candidate!     │                │
    │                   │                │
    │ ── RequestVote ────────────────→   │
    │ ←──────────────── Vote ────────   │
    │                                    │
    │  → New Leader!                     │
    │                                    │
    │ ── AppendEntries (Heartbeat) ──→   │

ZooKeeper

A coordination service for distributed applications.

┌──────────────────────────────────────┐
│        ZooKeeper Ensemble            │
│                                      │
│  ┌────────┐ ┌────────┐ ┌────────┐   │
│  │Server 1│ │Server 2│ │Server 3│   │
│  │(Leader)│ │(Follow.)│ │(Follow.)│  │
│  └────────┘ └────────┘ └────────┘   │
│                                      │
│  ZNode tree structure:              │
│  /                                   │
│  ├── /config                         │
│  │   ├── /config/database            │
│  │   └── /config/cache               │
│  ├── /locks                          │
│  │   └── /locks/resource1            │
│  └── /members                        │
│      ├── /members/node1              │
│      └── /members/node2              │
└──────────────────────────────────────┘

// ZooKeeper 분산 잠금 예시 (Java, 의사 코드)
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.ZooDefs;

public class DistributedLock {
    private ZooKeeper zk;
    private String lockPath;

    public DistributedLock(ZooKeeper zk, String resource) {
        this.zk = zk;
        this.lockPath = "/locks/" + resource;
    }

    public void lock() throws Exception {
        String myNode = zk.create(
            lockPath + "/lock-",
            new byte[0],
            ZooDefs.Ids.OPEN_ACL_UNSAFE,
            CreateMode.EPHEMERAL_SEQUENTIAL
        );

        while (true) {
            var children = zk.getChildren(lockPath, false);
            String smallest = children.stream()
                .sorted()
                .findFirst()
                .orElse(null);

            if (myNode.endsWith(smallest)) {
                return; // 잠금 획득
            }
            Thread.sleep(100);
        }
    }

    public void unlock() throws Exception {
        zk.delete(lockPath, -1);
    }
}

8. Summary

Networks: TCP/IP 4-layer model, TCP (reliability) vs UDP (speed)
Distributed Systems: Benefits of resource sharing and scalability, CAP theorem trade-offs
Distributed File Systems: NFS (traditional), GFS/HDFS (large-scale data)
MapReduce: Map (distributed processing) + Reduce (aggregation) for parallel data processing
Cloud Storage: Object/block/file storage, cost optimization through tiering
Distributed Coordination: Raft consensus algorithm, ZooKeeper coordination service

Quiz: Networks and Distributed Systems

Q1. What is the difference between CP and AP systems in the CAP theorem?

A1. CP systems (e.g., ZooKeeper) maintain consistency during network partitions by rejecting some requests, sacrificing availability. AP systems (e.g., Cassandra) respond to all requests even during partitions, but data across nodes may be temporarily inconsistent, eventually converging (eventual consistency).

Q2. Why does HDFS replicate data blocks to 3 copies?

A2. Triple replication provides a balance between fault tolerance and performance. Even if one DataNode fails, data can be read from the remaining 2 replicas, and read requests can be distributed across replicas to increase throughput. HDFS also places replicas on different racks to guard against rack-level failures.

Q3. What is the role of the Shuffle phase in MapReduce?

A3. The Shuffle phase sorts and groups Map output by key, delivering all values with the same key to a single Reducer. This involves data transfer over the network, making it the most expensive phase in MapReduce and a key target for performance optimization.