A distributed system is a collection of independent computers that appear to users as a single coherent system. The main characteristics include:

• Multiple nodes: Components run on separate machines connected by a network
• Concurrency: Multiple processes execute simultaneously across different nodes
• Lack of global clock: No single point of time reference across all nodes
• Independent failures: Components can fail independently without affecting the entire system
• Message passing: Communication happens through network messages rather than shared memory

Examples include web services like Google Search, Netflix streaming, or banking systems where multiple servers work together to serve user requests.

The CAP theorem states that in a distributed system, you can only guarantee two out of three properties:

• Consistency (C): All nodes see the same data simultaneously
• Availability (A): System remains operational and responds to requests
• Partition Tolerance (P): System continues operating despite network failures

Implications: Since network partitions are inevitable in distributed systems, you must choose between consistency and availability:

• CP systems (like MongoDB): Sacrifice availability during partitions to maintain consistency
• AP systems (like Cassandra): Sacrifice consistency to remain available during partitions

Real-world systems often implement "eventual consistency" to balance these trade-offs.

Vertical Scaling (Scale Up):

• Adding more power (CPU, RAM, storage) to existing machines
• Simpler to implement and manage
• Limited by hardware constraints
• Single point of failure
• Example: Upgrading server from 8GB to 32GB RAM

Horizontal Scaling (Scale Out):

• Adding more machines to the system
• Better fault tolerance and theoretically unlimited scaling
• More complex to implement and coordinate
• Requires load distribution mechanisms
• Example: Adding more web servers behind a load balancer

Most modern distributed systems prefer horizontal scaling for better resilience and cost-effectiveness.

Synchronous Communication:

• Sender waits for receiver's response before continuing
• Blocking operation with immediate feedback
• Simpler error handling and debugging
• Can create cascading failures and reduce availability
• Example: HTTP request-response, RPC calls

Asynchronous Communication:

• Sender doesn't wait for response and continues processing
• Non-blocking operation with eventual notification
• Better performance and fault isolation
• More complex error handling and state management
• Example: Message queues, event-driven architectures

# Synchronous
response = api_call()  # Blocks until response
process(response)

# Asynchronous  
api_call_async(callback=process)  # Non-blocking
continue_other_work()

Load balancing distributes incoming requests across multiple servers to prevent overload and ensure high availability.

Main Types:

1. Round Robin: Requests distributed sequentially to each server
2. Least Connections: Routes to server with fewest active connections
3. Weighted Round Robin: Assigns weights based on server capacity
4. IP Hash: Routes based on client IP hash for session persistence
5. Least Response Time: Routes to server with fastest response

Layers:

• Layer 4 (Transport): Routes based on IP and port
• Layer 7 (Application): Routes based on content (HTTP headers, URLs)

Load balancers can be hardware-based, software-based, or cloud-managed services like AWS ALB.