Skip to content
LowLevelDesign Mastery

gRPC & Protocol Buffers

High-performance communication for microservices

gRPC (gRPC Remote Procedure Calls) is a high-performance, open-source RPC framework developed by Google. It enables you to call remote functions as if they were local functions, abstracting away the complexity of network communication.

gRPC uses Protocol Buffers (protobuf) for serialization and HTTP/2 for transport. This combination provides significant performance advantages over traditional REST APIs using JSON over HTTP/1.1. The framework generates client and server code from .proto files, ensuring type safety and reducing boilerplate code.

Key Insight: gRPC is designed for inter-service communication in microservices architectures. It’s optimized for performance, with binary serialization reducing payload sizes by 50-70% compared to JSON, and HTTP/2 providing multiplexing and header compression.

REST with HTTP/1.1 and JSON text compared with gRPC using HTTP/2 and binary protobuf for faster calls and smaller payloads
FeatureREST/JSONgRPC
SpeedSlower (text parsing)Faster (binary)
Payload SizeLarger (text)Smaller (binary)
StreamingLimitedFull support
Type SafetyRuntimeCompile-time
Code GenerationManualAutomatic
Browser SupportExcellentLimited

Protocol Buffers (protobuf) is a binary serialization format. Think of it as a more efficient alternative to JSON.

JSON (text-based):

{
"id": 12345,
"name": "John Doe",
"email": "[email protected]",
"age": 30
}

Size: ~80 bytes

Protocol Buffers (binary):

[encoded binary data]

Size: ~25 bytes (3x smaller!)

  1. Smaller size - No field names, just values
  2. Faster parsing - No string parsing
  3. Type safety - Schema enforces types
  4. Backward compatible - Can evolve schemas

.proto files define your service contract. They’re like API documentation + code generator input.

syntax = "proto3";
package user_service;
// Message definition (like a struct/class)
message User {
int32 id = 1;
string name = 2;
string email = 3;
int32 age = 4;
}
message GetUserRequest {
int32 user_id = 1;
}
message GetUserResponse {
User user = 1;
}
// Service definition
service UserService {
// Unary RPC: one request, one response
rpc GetUser(GetUserRequest) returns (GetUserResponse);
// Server streaming: one request, multiple responses
rpc ListUsers(GetUserRequest) returns (stream User);
// Client streaming: multiple requests, one response
rpc CreateUsers(stream User) returns (CreateUsersResponse);
// Bidirectional streaming: multiple requests, multiple responses
rpc ChatUsers(stream ChatMessage) returns (stream ChatMessage);
}

Field numbers (1, 2, 3…) are important:

  • Used for binary encoding (not field names!)
  • Must be unique within message
  • Once used, never change (backward compatibility)
  • Use 1-15 for frequently used fields (1 byte encoding)

Common types:

  • int32, int64 - Integers
  • float, double - Floating point
  • bool - Boolean
  • string - UTF-8 string
  • bytes - Raw bytes
  • repeated - Arrays/lists
  • map - Key-value pairs

One request, one response. Like a function call.

service UserService {
rpc GetUser(GetUserRequest) returns (GetUserResponse);
}

Flow:

Client → Request → Server
Client ← Response ← Server

One request, multiple responses. Server sends stream of data.

service UserService {
rpc ListUsers(GetUserRequest) returns (stream User);
}

Flow:

Client → Request → Server
Client ← User 1 ← Server
Client ← User 2 ← Server
Client ← User 3 ← Server
...

Use cases:

  • Streaming search results
  • Real-time updates
  • Large dataset transfer

Multiple requests, one response. Client sends stream of data.

service UserService {
rpc CreateUsers(stream User) returns (CreateUsersResponse);
}

Flow:

Client → User 1 → Server
Client → User 2 → Server
Client → User 3 → Server
Client ← Response ← Server

Use cases:

  • Batch uploads
  • Collecting metrics
  • Aggregating data

Multiple requests, multiple responses. Both sides stream.

service ChatService {
rpc Chat(stream ChatMessage) returns (stream ChatMessage);
}

Flow:

Client → Message 1 → Server
Client ← Message 2 ← Server
Client → Message 3 → Server
Client ← Message 4 ← Server
...

Use cases:

  • Chat applications
  • Real-time collaboration
  • Interactive games

gRPC call flow from client through generated stub and HTTP/2 binary transport to server and service handler, and back
💡 Tip: Click dropdown to switch between languages
"user_service.py
import grpc
from concurrent import futures
import user_pb2
import user_pb2_grpc
class UserService(user_pb2_grpc.UserServiceServicer):
def GetUser(self, request, context):
"""Unary RPC implementation"""
user_id = request.user_id
# Fetch user from database
user = user_repository.find_by_id(user_id)
if not user:
context.set_code(grpc.StatusCode.NOT_FOUND)
context.set_details("User not found")
return user_pb2.GetUserResponse()
# Return response
return user_pb2.GetUserResponse(
user=user_pb2.User(
id=user.id,
name=user.name,
email=user.email,
age=user.age
)
)
def ListUsers(self, request, context):
"""Server streaming implementation"""
users = user_repository.find_all()
for user in users:
yield user_pb2.User(
id=user.id,
name=user.name,
email=user.email,
age=user.age
)
def CreateUsers(self, request_iterator, context):
"""Client streaming implementation"""
created_count = 0
for user_request in request_iterator:
# Create user
user_repository.create(
name=user_request.name,
email=user_request.email,
age=user_request.age
)
created_count += 1
return user_pb2.CreateUsersResponse(
created_count=created_count
)
def ChatUsers(self, request_iterator, context):
"""Bidirectional streaming implementation"""
for message in request_iterator:
# Process message
response = process_message(message)
# Send response
yield response
def serve():
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
user_pb2_grpc.add_UserServiceServicer_to_server(
UserService(), server
)
server.add_insecure_port('[::]:50051')
server.start()
server.wait_for_termination()
if __name__ == '__main__':
serve()
💡 Tip: Click dropdown to switch between languages
"grpc_client.py
import grpc
import user_pb2
import user_pb2_grpc
def get_user(user_id):
"""Unary RPC client"""
with grpc.insecure_channel('localhost:50051') as channel:
stub = user_pb2_grpc.UserServiceStub(channel)
request = user_pb2.GetUserRequest(user_id=user_id)
response = stub.GetUser(request)
return response.user
def list_users():
"""Server streaming client"""
with grpc.insecure_channel('localhost:50051') as channel:
stub = user_pb2_grpc.UserServiceStub(channel)
request = user_pb2.GetUserRequest()
for user in stub.ListUsers(request):
print(f"User: {user.name}")
def create_users(users_data):
"""Client streaming client"""
with grpc.insecure_channel('localhost:50051') as channel:
stub = user_pb2_grpc.UserServiceStub(channel)
def user_generator():
for user_data in users_data:
yield user_pb2.User(
name=user_data['name'],
email=user_data['email'],
age=user_data['age']
)
response = stub.CreateUsers(user_generator())
print(f"Created {response.created_count} users")

gRPC uses status codes (similar to HTTP but different):

CodeMeaningHTTP Equivalent
OKSuccess200
INVALID_ARGUMENTBad request400
NOT_FOUNDResource not found404
ALREADY_EXISTSConflict409
PERMISSION_DENIEDForbidden403
UNAUTHENTICATEDUnauthorized401
RESOURCE_EXHAUSTEDRate limited429
INTERNALServer error500
💡 Tip: Click dropdown to switch between languages
"error_handling.py
import grpc
from grpc import StatusCode
def GetUser(self, request, context):
try:
user = user_repository.find_by_id(request.user_id)
if not user:
context.set_code(StatusCode.NOT_FOUND)
context.set_details("User not found")
return user_pb2.GetUserResponse()
return user_pb2.GetUserResponse(user=user)
except ValueError as e:
context.set_code(StatusCode.INVALID_ARGUMENT)
context.set_details(str(e))
return user_pb2.GetUserResponse()
except Exception as e:
context.set_code(StatusCode.INTERNAL)
context.set_details("Internal server error")
return user_pb2.GetUserResponse()

Decision factors for choosing gRPC for performance-critical microservices and streaming versus REST for web and browser clients
  • Microservices communication (internal APIs) - Perfect for service-to-service calls where performance matters
  • High-performance needs (low latency, high throughput) - 5-10x faster than REST/JSON
  • Streaming required (real-time data) - Built-in support for server, client, and bidirectional streaming
  • Strong typing needed (compile-time safety) - Protocol Buffers provide type safety at compile time
  • Polyglot environments (multiple languages) - Generate client/server code in any language from .proto files
  • Public APIs (browser support limited) - Browsers have limited gRPC support (gRPC-Web exists but adds complexity)
  • Simple CRUD (REST is simpler) - REST is easier to understand and debug for simple operations
  • Human-readable needed (JSON is better) - JSON is easier to read and debug than binary protobuf
  • Firewall restrictions (HTTP/2 may be blocked) - Some firewalls/proxies don’t support HTTP/2 properly

REST versus gRPC comparison of text JSON over HTTP/1.1 against binary protobuf over HTTP/2, with gRPC winning on latency

Typical performance differences:

MetricREST/JSONgRPC
Latency10-50ms2-10ms
Throughput1K-10K req/s10K-100K req/s
Payload Size100%30-50%
CPU UsageHigherLower

Why gRPC is faster:

  1. Binary encoding (no JSON parsing)
  2. HTTP/2 (multiplexing, header compression)
  3. Connection reuse (less overhead)
  4. Efficient serialization (protobuf)

gRPC service design from .proto contract to generated stub, service handler implementing business logic, and repository data access

At the code level, gRPC translates to service interfaces, message types, and streaming handlers.

💡 Tip: Click dropdown to switch between languages
"service_design.py
# Design service interface
class UserServiceInterface:
"""Service interface defining contract"""
def get_user(self, user_id: int) -> User:
"""Unary: Get single user"""
pass
def list_users(self, filters: dict) -> Iterator[User]:
"""Server streaming: List users"""
pass
def create_users(self, users: Iterator[User]) -> int:
"""Client streaming: Create multiple users"""
pass
# Implementation
class UserService(UserServiceInterface):
def __init__(self, repository: UserRepository):
self.repository = repository
def get_user(self, user_id: int) -> User:
return self.repository.find_by_id(user_id)
def list_users(self, filters: dict) -> Iterator[User]:
return self.repository.find_all(filters)
def create_users(self, users: Iterator[User]) -> int:
count = 0
for user in users:
self.repository.create(user)
count += 1
return count

Performance First

gRPC is 5-10x faster than REST due to binary encoding and HTTP/2. Perfect for microservices.

Streaming Support

gRPC supports four streaming types: unary, server, client, and bidirectional. REST can’t do this.

Strong Typing

Protocol Buffers provide compile-time type safety. Schema defines contract, code is generated.

Service Contracts

.proto files define service contracts. Generate client/server code in any language automatically.