Centralized, Decentralized and Distributed Systems

 

1. CENTRALIZED SYSTEMS:

We start with centralized systems because they are the most intuitive and easy to understand and define. 

Centralized systems are systems that use client/server architecture where one or more client nodes are directly connected to a central server. This is the most commonly used type of system in many organizations where a client sends a request to a company server and receives the response. 

 

Figure – Centralized system visualization 

Example – 
Wikipedia. Consider a massive server to which we send our requests and the server responds with the article that we requested. Suppose we enter the search term ‘junk food’ in the Wikipedia search bar. This search term is sent as a request to the Wikipedia servers (mostly located in Virginia, U.S.A) which then responds back with the articles based on relevance. In this situation, we are the client node, Wikipedia servers are the central server. 

Characteristics of Centralized System – 

• Presence of a global clock: As the entire system consists of a central node(a server/ a master) and many client nodes(a computer/ a slave), all client nodes sync up with the global clock(the clock of the central node). 
• One single central unit: One single central unit which serves/coordinates all the other nodes in the system. 
• Dependent failure of components: Central node failure causes the entire system to fail. This makes sense because when the server is down, no other entity is there to send/receive responses/requests. 
   

Scaling – 
Only vertical scaling on the central server is possible. Horizontal scaling will contradict the single central unit characteristic of this system of a single central entity. 

Components of Centralized System – 
Components of Centralized System are, 

• Node (Computer, Mobile, etc.). 
• Server. 
• Communication link (Cables, Wi-Fi, etc.). 
   

Architecture of Centralized System – 
Client-Server architecture. The central node that serves the other nodes in the system is the server node and all the other nodes are the client nodes. 

Limitations of Centralized System – 

• Can’t scale up vertically after a certain limit – After a limit, even if you increase the hardware and software capabilities of the server node, the performance will not increase appreciably leading to a cost/benefit ratio < 1. 
   
• Bottlenecks can appear when the traffic spikes – as the server can only have a finite number of open ports to which can listen to connections from client nodes. So, when high traffic occurs like a shopping sale, the server can essentially suffer a Denial-of-Service attack or Distributed Denial-of-Service attack. 
   

Advantages of Centralized System – 

• Easy to physically secure. It is easy to secure and service the server and client nodes by virtue of their location
• Smooth and elegant personal experience – A client has a dedicated system which he uses(for example, a personal computer) and the company has a similar system which can be modified to suit custom needs
• Dedicated resources (memory, CPU cores, etc)
• More cost-efficient for small systems up to a certain limit – As the central systems take fewer funds to set up, they have an edge when small systems have to be built
• Quick updates are possible – Only one machine to update.
• Easy detachment of a node from the system. Just remove the connection of the client node from the server and voila! Node detached.

Disadvantages of Centralized System – 

• Highly dependent on the network connectivity – The system can fail if the nodes lose connectivity as there is only one central node.
• No graceful degradation of the system – abrupt failure of the entire system
• Less possibility of data backup. If the server node fails and there is no backup, you lose the data straight away
• Difficult server maintenance – There is only one server node and due to availability reasons, it is inefficient and unprofessional to take the server down for maintenance. So, updates have to be done on-the-fly(hot updates) which is difficult and the system could break.

Applications of Centralized System – 

• Application development – Very easy to set up a central server and send client requests. Modern technology these days do come with default test servers which can be launched with a couple of commands. For example, Express server, Django server.
• Data analysis – Easy to do data analysis when all the data is in one place and available for analysis
• Personal computing

Use Cases – 

• Centralized databases – all the data in one server for use.
• Single-player games like Need For Speed, GTA Vice City – an entire game in one system(commonly, a Personal Computer)
• Application development by deploying test servers leading to easy debugging, easy deployment, easy simulation
• Personal Computers

Organizations Using – 
National Informatics Center (India), IBM 
 

2. DECENTRALIZED SYSTEMS:

These are other types of systems that have been gaining a lot of popularity, primarily because of the massive hype of Bitcoin. Now many organizations are trying to find the application of such systems. 

In decentralized systems, every node makes its own decision. The final behavior of the system is the aggregate of the decisions of the individual nodes. Note that there is no single entity that receives and responds to the request. 

 

Figure – Decentralized system visualization 

Example – 
Bitcoin. Let’s take Bitcoin for example because it is the most popular use case of decentralized systems. No single entity/organization owns the bitcoin network. The network is a sum of all the nodes who talk to each other for maintaining the amount of bitcoin every account holder has. 

Characteristics of Decentralized System – 

• Lack of a global clock: Every node is independent of each other and hence, has different clocks that they run and follow.
• Multiple central units (Computers/Nodes/Servers): More than one central unit which can listen for connections from other nodes
• Dependent failure of components: one central node failure causes a part of the system to fail; not the whole system

Scaling – 
Vertical scaling is possible. Each node can add resources(hardware, software) to itself to increase the performance leading to an increase in the performance of the entire system. 

Components – 
Components of Decentralized System are, 

• Node (Computer, Mobile, etc.)
• Communication link (Cables, Wi-Fi, etc.)

Architecture of Decentralized System – 

• peer-to-peer architecture – all nodes are peers of each other. No one node has supremacy over other nodes
• master-slave architecture – One node can become a master by voting and help in coordinating of a part of the system but this does not mean the node has supremacy over the other node which it is coordinating

Limitations of Decentralized System – 

• May lead to the problem of coordination at the enterprise level – When every node is the owner of its own behavior, its difficult to achieve collective tasks
• Not suitable for small systems – Not beneficial to build and operate small decentralized systems because of the low cost/benefit ratio
• No way to regulate a node on the system – no superior node overseeing the behavior of subordinate nodes

Advantages of Decentralized System – 

• Minimal problem of performance bottlenecks occurring – The entire load gets balanced on all the nodes; leading to minimal to no bottleneck situations
• High availability – Some nodes(computers, mobiles, servers) are always available/online for work, leading to high availability
• More autonomy and control over resources – As each node controls its own behavior, it has better autonomy leading to more control over resources

Disadvantages of Decentralized System – 

• Difficult to achieve global big tasks – No chain of command to command others to perform certain tasks
• No regulatory oversight
• Difficult to know which node failed – Each node must be pinged for availability checking and partitioning of work has to be done to actually find out which node failed by checking the expected output with what the node generated
• Difficult to know which node responded – When a request is served by a decentralized system, the request is actually served by one of the nodes in the system but it is actually difficult to find out which node indeed served the request.

Applications of Decentralized System – 

• Private networks – peer nodes joined with each other to make a private network.
• Cryptocurrency – Nodes joined to become a part of a system in which digital currency is exchanged without any trace and location of who sent what to whom. However, in bitcoin, we can see the public address and amount of bitcoin transferred, but those public addresses are mutable and hence difficult to trace.

Use Cases – 

• Blockchain
• Decentralized databases – Entire databases split into parts and distributed to different nodes for storage and use. For example, records with names starting from ‘A’ to ‘K’ in one node, ‘L’ to ‘N’ in the second node, and ‘O’ to ‘Z’ in the third node
• Cryptocurrency

Organizations Using – 
Bitcoin, Tor network 

3. DISTRIBUTED SYSTEMS:

This is the last type of system that we are going to discuss. Let’s head right into it! 

In decentralized systems, every node makes its own decision. The final behaviour of the system is the aggregate of the decisions of the individual nodes. Note that there is no single entity that receives and responds to the request. 

 

Figure – Distributed system visualization 

Example – 
Google search system. Each request is worked upon by hundreds of computers which crawl the web and return the relevant results. To the user, Google appears to be one system, but it actually is multiple computers working together to accomplish one single task (return the results to the search query). 

Characteristics of Distributed System – :

• Concurrency of components: Nodes apply consensus protocols to agree on the same values/transactions/commands/logs.
• Lack of a global clock: All nodes maintain their own clock.
• Independent failure of components: In a distributed system, nodes fail independently without having a significant effect on the entire system. If one node fails, the entire system sans the failed node continues to work.

Scaling – 
Horizontal and vertical scaling is possible. 

Components of Distributed System – 
Components of Distributed System are, 

• Node (Computer, Mobile, etc.)
• Communication link (Cables, Wi-Fi, etc.)

Architecture of Distributed System – 

• peer-to-peer – all nodes are peers of each other and work towards a common goal
• client-server – some nodes become server nodes for the role of coordinator, arbiter, etc.
• n-tier architecture – different parts of an application are distributed in different nodes of the systems and these nodes work together to function as an application for the user/client

Limitations of Distributed System – 

• Difficult to design and debug algorithms for the system. These algorithms are difficult because of the absence of a common clock; so no temporal ordering of commands/logs can take place. Nodes can have different latencies which have to be kept in mind while designing such algorithms. The complexity increases with the increase in the number of nodes. Visit this link for more information
• No common clock causes difficulty in the temporal ordering of events/transactions
• Difficult for a node to get the global view of the system and hence take informed decisions based on the state of other nodes in the system

Advantages of Distributed System – 

• Low latency than a centralized system – Distributed systems have low latency because of high geographical spread, hence leading to less time to get a response

Disadvantages of Distributed System – 

• Difficult to achieve consensus
• The conventional way of logging events by absolute time they occur is not possible here

Applications of Distributed System –  

• Cluster computing – a technique in which many computers are coupled together to work so that they achieve global goals. The computer cluster acts as if they were a single computer
• Grid computing – All the resources are pooled together for sharing in this kind of computing turning the systems into a powerful supercomputer; essentially.

Use Cases – 

• SOA-based systems
• Multiplayer online games

Organizations Using – 
Apple, Google, Facebook.

💪 Fault tolerance:

• Low: Centralized systems
• Moderate: Decentralized systems
• High: Distributed systems

🔧 Maintenance:

• Low: Centralized systems
• Moderate: Decentralized systems
• High: Distributed systems

🚀 Scalability:

• Low: Centralized systems
• Moderate: Decentralized systems
• High: Distributed systems

💰 Development:

• Moderate: Decentralized systems, Distributed systems
• High: Centralized systems

📈 Evolution:

• Low: Centralized systems
• High: Decentralized systems, Distributed systems

A centralized system is a system in which an individual, a group of people or a corporate entity holds the entire control over the functionality of the system.

Online social applications like Facebook, Twitter, Quora are examples of centralized systems.

We, end users don’t have a say in the architectural design, feature availability or the functionality of the applications. We don’t decide how the systems should operate. We have no control over the data.
Corporate entities hold the rights to modify or delete our data without any permission.

A centralized system has its risks, for instance, single point of failure. If the company goes out of business all our data is gone, for ever. And this has happened in the past.

Decentralized simply means not centralized. The control, as opposed to with a single entity, lies with the end users. BitTorrent a peer to peer network is an ideal example of this.
Where the peers are responsible for the availability of content on the network. More peers seed the data, higher is the availability & download speed.

Decentralized systems just like the distributed architecture have no single points of failure. Even if several nodes go down, the network as a whole is still up.

There is no single entity control, so there is zero possibility of the network going down anytime unless all the nodes go down simultaneously which is a rare possibility when we have systems connected all over the globe.

This kind of architecture is almost infinitely scalable, unlike a centralized system in which scalability depends upon the resources of the organization in charge.