gRPC & Protocol Buffers

High-performance communication for microservices

Home
Hld Concepts
Communication
Grpc Protobuf

What is gRPC?

gRPC (gRPC Remote Procedure Calls) is a high-performance RPC framework that lets you call remote functions as if they were local.

Simple Analogy

Think of REST like sending a letter (JSON text) through regular mail. gRPC is like sending a compressed package (binary) through express delivery (HTTP/2). It’s faster, more efficient, and can handle multiple packages at once.

Why gRPC?

Section titled “Why gRPC?”

Feature	REST/JSON	gRPC
Speed	Slower (text parsing)	Faster (binary)
Payload Size	Larger (text)	Smaller (binary)
Streaming	Limited	Full support
Type Safety	Runtime	Compile-time
Code Generation	Manual	Automatic
Browser Support	Excellent	Limited

Protocol Buffers: The Foundation

Section titled “Protocol Buffers: The Foundation”

Protocol Buffers (protobuf) is a binary serialization format. Think of it as a more efficient alternative to JSON.

JSON vs Protocol Buffers

Section titled “JSON vs Protocol Buffers”

JSON (text-based):

1
{
2
  "id": 12345,
3
  "name": "John Doe",
4
  "email": "john@example.com",
5
  "age": 30
6
}

Size: ~80 bytes

Protocol Buffers (binary):

1
[encoded binary data]

Size: ~25 bytes (3x smaller!)

Why Binary?

Section titled “Why Binary?”

Smaller size - No field names, just values
Faster parsing - No string parsing
Type safety - Schema enforces types
Backward compatible - Can evolve schemas

Defining a Service: .proto Files

Section titled “Defining a Service: .proto Files”

.proto files define your service contract. They’re like API documentation + code generator input.

Basic .proto Example

Section titled “Basic .proto Example”

1
syntax = "proto3";
2

3
package user_service;
4

5
// Message definition (like a struct/class)
6
message User {
7
  int32 id = 1;
8
  string name = 2;
9
  string email = 3;
10
  int32 age = 4;
11
}
12

13
message GetUserRequest {
14
  int32 user_id = 1;
15
}
16

17
message GetUserResponse {
18
  User user = 1;
19
}
20

21
// Service definition
22
service UserService {
23
  // Unary RPC: one request, one response
24
  rpc GetUser(GetUserRequest) returns (GetUserResponse);
25

26
  // Server streaming: one request, multiple responses
27
  rpc ListUsers(GetUserRequest) returns (stream User);
28

29
  // Client streaming: multiple requests, one response
30
  rpc CreateUsers(stream User) returns (CreateUsersResponse);
31

32
  // Bidirectional streaming: multiple requests, multiple responses
33
  rpc ChatUsers(stream ChatMessage) returns (stream ChatMessage);
34
}

Field Numbers

Section titled “Field Numbers”

Field numbers (1, 2, 3…) are important:

Used for binary encoding (not field names!)
Must be unique within message
Once used, never change (backward compatibility)
Use 1-15 for frequently used fields (1 byte encoding)

Message Types

Section titled “Message Types”

Common types:

int32, int64 - Integers
float, double - Floating point
bool - Boolean
string - UTF-8 string
bytes - Raw bytes
repeated - Arrays/lists
map - Key-value pairs

gRPC Service Types

Section titled “gRPC Service Types”

1. Unary RPC (Request-Response)

Section titled “1. Unary RPC (Request-Response)”

One request, one response. Like a function call.

1
service UserService {
2
  rpc GetUser(GetUserRequest) returns (GetUserResponse);
3
}

Flow:

1
Client → Request → Server
2
Client ← Response ← Server

2. Server Streaming

Section titled “2. Server Streaming”

One request, multiple responses. Server sends stream of data.

1
service UserService {
2
  rpc ListUsers(GetUserRequest) returns (stream User);
3
}

Flow:

1
Client → Request → Server
2
Client ← User 1 ← Server
3
Client ← User 2 ← Server
4
Client ← User 3 ← Server
5
...

Use cases:

Streaming search results
Real-time updates
Large dataset transfer

3. Client Streaming

Section titled “3. Client Streaming”

Multiple requests, one response. Client sends stream of data.

1
service UserService {
2
  rpc CreateUsers(stream User) returns (CreateUsersResponse);
3
}

Flow:

1
Client → User 1 → Server
2
Client → User 2 → Server
3
Client → User 3 → Server
4
Client ← Response ← Server

Use cases:

Batch uploads
Collecting metrics
Aggregating data

4. Bidirectional Streaming

Section titled “4. Bidirectional Streaming”

Multiple requests, multiple responses. Both sides stream.

1
service ChatService {
2
  rpc Chat(stream ChatMessage) returns (stream ChatMessage);
3
}

Flow:

1
Client → Message 1 → Server
2
Client ← Message 2 ← Server
3
Client → Message 3 → Server
4
Client ← Message 4 ← Server
5
...

Use cases:

Chat applications
Real-time collaboration
Interactive games

Implementing gRPC Services

Section titled “Implementing gRPC Services”

Server Implementation

Section titled “Server Implementation”

Python
Java

"user_service.py

1
import grpc
2
from concurrent import futures
3
import user_pb2
4
import user_pb2_grpc
5

6
class UserService(user_pb2_grpc.UserServiceServicer):
7
    def GetUser(self, request, context):
8
        """Unary RPC implementation"""
9
        user_id = request.user_id
10

11
        # Fetch user from database
12
        user = user_repository.find_by_id(user_id)
13

14
        if not user:
15
            context.set_code(grpc.StatusCode.NOT_FOUND)
16
            context.set_details("User not found")
17
            return user_pb2.GetUserResponse()
18

19
        # Return response
20
        return user_pb2.GetUserResponse(
21
            user=user_pb2.User(
22
                id=user.id,
23
                name=user.name,
24
                email=user.email,
25
                age=user.age
26
            )
27
        )
28

29
    def ListUsers(self, request, context):
30
        """Server streaming implementation"""
31
        users = user_repository.find_all()
32

33
        for user in users:
34
            yield user_pb2.User(
35
                id=user.id,
36
                name=user.name,
37
                email=user.email,
38
                age=user.age
39
            )
40

41
    def CreateUsers(self, request_iterator, context):
42
        """Client streaming implementation"""
43
        created_count = 0
44

45
        for user_request in request_iterator:
46
            # Create user
47
            user_repository.create(
48
                name=user_request.name,
49
                email=user_request.email,
50
                age=user_request.age
51
            )
52
            created_count += 1
53

54
        return user_pb2.CreateUsersResponse(
55
            created_count=created_count
56
        )
57

58
    def ChatUsers(self, request_iterator, context):
59
        """Bidirectional streaming implementation"""
60
        for message in request_iterator:
61
            # Process message
62
            response = process_message(message)
63

64
            # Send response
65
            yield response
66

67
def serve():
68
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
69
    user_pb2_grpc.add_UserServiceServicer_to_server(
70
        UserService(), server
71
    )
72
    server.add_insecure_port('[::]:50051')
73
    server.start()
74
    server.wait_for_termination()
75

76
if __name__ == '__main__':
77
    serve()

"UserService.java

1
import io.grpc.Server;
2
import io.grpc.ServerBuilder;
3
import io.grpc.stub.StreamObserver;
4
import user.UserServiceGrpc;
5
import user.UserOuterClass;
6

7
public class UserService extends UserServiceGrpc.UserServiceImplBase {
8

9
    @Override
10
    public void getUser(
11
            UserOuterClass.GetUserRequest request,
12
            StreamObserver<UserOuterClass.GetUserResponse> responseObserver) {
13
        // Unary RPC implementation
14
        int userId = request.getUserId();
15

16
        // Fetch user from database
17
        User user = userRepository.findById(userId)
18
            .orElseThrow(() -> Status.NOT_FOUND
19
                .withDescription("User not found")
20
                .asRuntimeException());
21

22
        // Build response
23
        UserOuterClass.GetUserResponse response = UserOuterClass.GetUserResponse.newBuilder()
24
            .setUser(UserOuterClass.User.newBuilder()
25
                .setId(user.getId())
26
                .setName(user.getName())
27
                .setEmail(user.getEmail())
28
                .setAge(user.getAge())
29
                .build())
30
            .build();
31

32
        responseObserver.onNext(response);
33
        responseObserver.onCompleted();
34
    }
35

36
    @Override
37
    public void listUsers(
38
            UserOuterClass.GetUserRequest request,
39
            StreamObserver<UserOuterClass.User> responseObserver) {
40
        // Server streaming implementation
41
        List<User> users = userRepository.findAll();
42

43
        for (User user : users) {
44
            UserOuterClass.User protoUser = UserOuterClass.User.newBuilder()
45
                .setId(user.getId())
46
                .setName(user.getName())
47
                .setEmail(user.getEmail())
48
                .setAge(user.getAge())
49
                .build();
50

51
            responseObserver.onNext(protoUser);
52
        }
53

54
        responseObserver.onCompleted();
55
    }
56

57
    @Override
58
    public StreamObserver<UserOuterClass.User> createUsers(
59
            StreamObserver<UserOuterClass.CreateUsersResponse> responseObserver) {
60
        // Client streaming implementation
61
        return new StreamObserver<UserOuterClass.User>() {
62
            private int createdCount = 0;
63

64
            @Override
65
            public void onNext(UserOuterClass.User user) {
66
                // Create user
67
                userRepository.create(
68
                    user.getName(),
69
                    user.getEmail(),
70
                    user.getAge()
71
                );
72
                createdCount++;
73
            }
74

75
            @Override
76
            public void onError(Throwable t) {
77
                responseObserver.onError(t);
78
            }
79

80
            @Override
81
            public void onCompleted() {
82
                UserOuterClass.CreateUsersResponse response =
83
                    UserOuterClass.CreateUsersResponse.newBuilder()
84
                        .setCreatedCount(createdCount)
85
                        .build();
86

87
                responseObserver.onNext(response);
88
                responseObserver.onCompleted();
89
            }
90
        };
91
    }
92

93
    public static void main(String[] args) throws Exception {
94
        Server server = ServerBuilder.forPort(50051)
95
            .addService(new UserService())
96
            .build()
97
            .start();
98

99
        server.awaitTermination();
100
    }
101
}

Client Implementation

Section titled “Client Implementation”

Python
Java

"grpc_client.py

1
import grpc
2
import user_pb2
3
import user_pb2_grpc
4

5
def get_user(user_id):
6
    """Unary RPC client"""
7
    with grpc.insecure_channel('localhost:50051') as channel:
8
        stub = user_pb2_grpc.UserServiceStub(channel)
9

10
        request = user_pb2.GetUserRequest(user_id=user_id)
11
        response = stub.GetUser(request)
12

13
        return response.user
14

15
def list_users():
16
    """Server streaming client"""
17
    with grpc.insecure_channel('localhost:50051') as channel:
18
        stub = user_pb2_grpc.UserServiceStub(channel)
19

20
        request = user_pb2.GetUserRequest()
21

22
        for user in stub.ListUsers(request):
23
            print(f"User: {user.name}")
24

25
def create_users(users_data):
26
    """Client streaming client"""
27
    with grpc.insecure_channel('localhost:50051') as channel:
28
        stub = user_pb2_grpc.UserServiceStub(channel)
29

30
        def user_generator():
31
            for user_data in users_data:
32
                yield user_pb2.User(
33
                    name=user_data['name'],
34
                    email=user_data['email'],
35
                    age=user_data['age']
36
                )
37

38
        response = stub.CreateUsers(user_generator())
39
        print(f"Created {response.created_count} users")

"GrpcClient.java

1
import io.grpc.ManagedChannel;
2
import io.grpc.ManagedChannelBuilder;
3
import user.UserServiceGrpc;
4
import user.UserOuterClass;
5

6
public class GrpcClient {
7
    private final ManagedChannel channel;
8
    private final UserServiceGrpc.UserServiceBlockingStub blockingStub;
9

10
    public GrpcClient(String host, int port) {
11
        this.channel = ManagedChannelBuilder.forAddress(host, port)
12
            .usePlaintext()
13
            .build();
14
        this.blockingStub = UserServiceGrpc.newBlockingStub(channel);
15
    }
16

17
    public UserOuterClass.User getUser(int userId) {
18
        // Unary RPC client
19
        UserOuterClass.GetUserRequest request = UserOuterClass.GetUserRequest.newBuilder()
20
            .setUserId(userId)
21
            .build();
22

23
        UserOuterClass.GetUserResponse response = blockingStub.getUser(request);
24
        return response.getUser();
25
    }
26

27
    public void listUsers() {
28
        // Server streaming client
29
        UserOuterClass.GetUserRequest request = UserOuterClass.GetUserRequest.newBuilder().build();
30

31
        blockingStub.listUsers(request).forEachRemaining(user -> {
32
            System.out.println("User: " + user.getName());
33
        });
34
    }
35

36
    public void createUsers(List<UserData> usersData) {
37
        // Client streaming client
38
        UserServiceGrpc.UserServiceStub asyncStub = UserServiceGrpc.newStub(channel);
39

40
        StreamObserver<UserOuterClass.User> requestObserver = asyncStub.createUsers(
41
            new StreamObserver<UserOuterClass.CreateUsersResponse>() {
42
                @Override
43
                public void onNext(UserOuterClass.CreateUsersResponse response) {
44
                    System.out.println("Created " + response.getCreatedCount() + " users");
45
                }
46

47
                @Override
48
                public void onError(Throwable t) {
49
                    t.printStackTrace();
50
                }
51

52
                @Override
53
                public void onCompleted() {
54
                    // Done
55
                }
56
            }
57
        );
58

59
        for (UserData userData : usersData) {
60
            UserOuterClass.User user = UserOuterClass.User.newBuilder()
61
                .setName(userData.getName())
62
                .setEmail(userData.getEmail())
63
                .setAge(userData.getAge())
64
                .build();
65

66
            requestObserver.onNext(user);
67
        }
68

69
        requestObserver.onCompleted();
70
    }
71
}

Error Handling in gRPC

Section titled “Error Handling in gRPC”

gRPC uses status codes (similar to HTTP but different):

Code	Meaning	HTTP Equivalent
`OK`	Success	200
`INVALID_ARGUMENT`	Bad request	400
`NOT_FOUND`	Resource not found	404
`ALREADY_EXISTS`	Conflict	409
`PERMISSION_DENIED`	Forbidden	403
`UNAUTHENTICATED`	Unauthorized	401
`RESOURCE_EXHAUSTED`	Rate limited	429
`INTERNAL`	Server error	500

Error Handling Example

Section titled “Error Handling Example”

Python
Java

"error_handling.py

1
import grpc
2
from grpc import StatusCode
3

4
def GetUser(self, request, context):
5
    try:
6
        user = user_repository.find_by_id(request.user_id)
7

8
        if not user:
9
            context.set_code(StatusCode.NOT_FOUND)
10
            context.set_details("User not found")
11
            return user_pb2.GetUserResponse()
12

13
        return user_pb2.GetUserResponse(user=user)
14

15
    except ValueError as e:
16
        context.set_code(StatusCode.INVALID_ARGUMENT)
17
        context.set_details(str(e))
18
        return user_pb2.GetUserResponse()
19

20
    except Exception as e:
21
        context.set_code(StatusCode.INTERNAL)
22
        context.set_details("Internal server error")
23
        return user_pb2.GetUserResponse()

"ErrorHandling.java

1
@Override
2
public void getUser(
3
        UserOuterClass.GetUserRequest request,
4
        StreamObserver<UserOuterClass.GetUserResponse> responseObserver) {
5
    try {
6
        User user = userRepository.findById(request.getUserId())
7
            .orElseThrow(() -> Status.NOT_FOUND
8
                .withDescription("User not found")
9
                .asRuntimeException());
10

11
        UserOuterClass.GetUserResponse response = buildResponse(user);
12
        responseObserver.onNext(response);
13
        responseObserver.onCompleted();
14

15
    } catch (IllegalArgumentException e) {
16
        responseObserver.onError(Status.INVALID_ARGUMENT
17
            .withDescription(e.getMessage())
18
            .asRuntimeException());
19

20
    } catch (Exception e) {
21
        responseObserver.onError(Status.INTERNAL
22
            .withDescription("Internal server error")
23
            .asRuntimeException());
24
    }
25
}

When to Use gRPC

Section titled “When to Use gRPC”

Use gRPC When:

Section titled “Use gRPC When:”

✅ Microservices communication (internal APIs)
✅ High-performance needs (low latency, high throughput)
✅ Streaming required (real-time data)
✅ Strong typing needed (compile-time safety)
✅ Polyglot environments (multiple languages)

Don’t Use gRPC When:

Section titled “Don’t Use gRPC When:”

❌ Public APIs (browser support limited)
❌ Simple CRUD (REST is simpler)
❌ Human-readable needed (JSON is better)
❌ Firewall restrictions (HTTP/2 may be blocked)

gRPC-Web

gRPC-Web allows gRPC in browsers, but it’s more complex than REST. Use REST for public web APIs, gRPC for internal services.

Performance Comparison

Section titled “Performance Comparison”

REST vs gRPC Benchmarks

Section titled “REST vs gRPC Benchmarks”

Typical performance differences:

Metric	REST/JSON	gRPC
Latency	10-50ms	2-10ms
Throughput	1K-10K req/s	10K-100K req/s
Payload Size	100%	30-50%
CPU Usage	Higher	Lower

Why gRPC is faster:

Binary encoding (no JSON parsing)
HTTP/2 (multiplexing, header compression)
Connection reuse (less overhead)
Efficient serialization (protobuf)

LLD Connection: Designing gRPC Services

Section titled “LLD Connection: Designing gRPC Services”

At the code level, gRPC translates to service interfaces, message types, and streaming handlers.

Service Interface Design

Section titled “Service Interface Design”

Python
Java

"service_design.py

1
# Design service interface
2
class UserServiceInterface:
3
    """Service interface defining contract"""
4

5
    def get_user(self, user_id: int) -> User:
6
        """Unary: Get single user"""
7
        pass
8

9
    def list_users(self, filters: dict) -> Iterator[User]:
10
        """Server streaming: List users"""
11
        pass
12

13
    def create_users(self, users: Iterator[User]) -> int:
14
        """Client streaming: Create multiple users"""
15
        pass
16

17
# Implementation
18
class UserService(UserServiceInterface):
19
    def __init__(self, repository: UserRepository):
20
        self.repository = repository
21

22
    def get_user(self, user_id: int) -> User:
23
        return self.repository.find_by_id(user_id)
24

25
    def list_users(self, filters: dict) -> Iterator[User]:
26
        return self.repository.find_all(filters)
27

28
    def create_users(self, users: Iterator[User]) -> int:
29
        count = 0
30
        for user in users:
31
            self.repository.create(user)
32
            count += 1
33
        return count

"ServiceDesign.java

1
// Service interface
2
interface UserService {
3
    User getUser(int userId);
4
    Stream<User> listUsers(Map<String, String> filters);
5
    int createUsers(Stream<User> users);
6
}
7

8
// Implementation
9
class UserServiceImpl implements UserService {
10
    private final UserRepository repository;
11

12
    public UserServiceImpl(UserRepository repository) {
13
        this.repository = repository;
14
    }
15

16
    @Override
17
    public User getUser(int userId) {
18
        return repository.findById(userId).orElseThrow();
19
    }
20

21
    @Override
22
    public Stream<User> listUsers(Map<String, String> filters) {
23
        return repository.findAll(filters).stream();
24
    }
25

26
    @Override
27
    public int createUsers(Stream<User> users) {
28
        AtomicInteger count = new AtomicInteger();
29
        users.forEach(user -> {
30
            repository.create(user);
31
            count.incrementAndGet();
32
        });
33
        return count.get();
34
    }
35
}

Key Takeaways

Section titled “Key Takeaways”

⚡ Performance First

gRPC is 5-10x faster than REST due to binary encoding and HTTP/2. Perfect for microservices.

🔄 Streaming Support

gRPC supports four streaming types: unary, server, client, and bidirectional. REST can’t do this.

📊 Strong Typing

Protocol Buffers provide compile-time type safety. Schema defines contract, code is generated.

🏗️ Service Contracts

.proto files define service contracts. Generate client/server code in any language automatically.

Next Steps

Section titled “Next Steps”

Learn API Gateway Pattern - single entry point for APIs
Understand Backend for Frontend - tailored APIs for clients
Master Rate Limiting - controlling API usage

📚 Related Articles

Continue learning with these related topics

Design Music Streaming Service like Spotify

Design a music streaming service similar to Spotify that allows users to search, play, and manage songs, create and share playlists, stream music in real-time, get personalized recommendations, and download songs for offline playback. Master this hard Low Level Design interview problem with step-by-step guidance, interactive UML class diagram builder, and complete solutions in Python, Java, C++, TypeScript, JavaScript, and C#.

→

Design a Notification Service

Design a notification service that can send messages to users through multiple channels (email, SMS, push notifications). The service should support rate limiting to prevent overwhelming users, batching to optimize resource usage, template management for consistent messaging, user preferences for channel selection, and retry mechanisms for failed notifications. Master this hard Low Level Design interview problem with step-by-step guidance, interactive UML class diagram builder, and complete solutions in Python, Java, C++, TypeScript, JavaScript, and C#.

→

Design a Library Management System

Design a Library Management System that allows members to search, borrow, return, and reserve books. The system should manage multiple copies of books, track due dates, calculate fines for overdue books, and notify members when reserved books become available. Master this easy Low Level Design interview problem with step-by-step guidance, interactive UML class diagram builder, and complete solutions in Python, Java, C++, TypeScript, JavaScript, and C#.

→

Designing a Task Management System

Design a comprehensive task management system that allows users to create, edit, delete, assign, and track tasks with various states, priorities, reminders, and history tracking. Master this easy Low Level Design interview problem with step-by-step guidance, interactive UML class diagram builder, and complete solutions in Python, Java, C++, TypeScript, JavaScript, and C#.

→

Designing a Tic Tac Toe Game

Design a Tic Tac Toe game that allows two players to play on a 3x3 grid. Players take turns placing their symbols (X or O) in empty cells. The game should detect wins (three in a row horizontally, vertically, or diagonally) and handle draws when all cells are filled. Master this easy Low Level Design interview problem with step-by-step guidance, interactive UML class diagram builder, and complete solutions in Python, Java, C++, TypeScript, JavaScript, and C#.

→

Design a Feed Generator

Design a feed generator system that aggregates content from multiple sources, ranks posts based on various algorithms, and delivers personalized feeds to users. The system should support different ranking strategies, caching mechanisms, real-time feed updates, and content filtering. Master this hard Low Level Design interview problem with step-by-step guidance, interactive UML class diagram builder, and complete solutions in Python, Java, C++, TypeScript, JavaScript, and C#.

→

Last updated: Jan 6, 2026

Previous
GraphQL Fundamentals Next
API Gateway Pattern