Image streaming with gRPC

Posted on Nov 17, 2022

Introduction

Discover the basics of using gRPC for image streaming with Python. This post offers a simple overview, highlighting the advantages and ease of setup.

Getting started

You'll need to install the gRPC Python package (grpcio) and the gRPC tools package (grpcio-tools) for Python.

You can install these packages using pip:

$ pip install grpcio grpcio-tools

First, let's define the .proto file for gRPC communication, where we define the service and its methods. Save this in a file named image_service.proto.

syntax = "proto3";

service ImageService {
  rpc SendImage (stream ImageRequest) returns (stream ImageReply);
}

message ImageRequest {
  bytes image_data = 1;
}

message ImageReply {
  bytes image_data = 1;
}

Next, compile the .proto file to generate Python code for gRPC. Run the following command:

$ python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. image_service.proto

This will generate image_service_pb2.py and image_service_pb2_grpc.py files.

Server Code

The server will receive the image from one client and forward it to the other, and vice versa. Save this in a file like image_service_server.py.

import grpc
import image_service_pb2
import image_service_pb2_grpc
from concurrent import futures
import queue

class ImageServiceServicer(image_service_pb2_grpc.ImageServiceServicer):

    def __init__(self):
        self.client_queues = []

    def SendImage(self, request_iterator, context):
        client_queue = queue.Queue()
        self.client_queues.append(client_queue)

        def reply_messages():
            while True:
                try:
                    reply = client_queue.get(timeout=10)
                    yield reply
                except queue.Empty:
                    return

        for request in request_iterator:
            for q in self.client_queues:
                if q is not client_queue:  # Don't echo back to the sender
                    q.put(image_service_pb2.ImageReply(image_data=request.image_data))

        return reply_messages()

def serve():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    image_service_pb2_grpc.add_ImageServiceServicer_to_server(ImageServiceServicer(), server)
    server.add_insecure_port('[::]:50051')
    server.start()
    server.wait_for_termination()

if __name__ == '__main__':
    serve()

Client Code

Finally, we can modify the original script to act as a gRPC client. Save this in a file like image_service_client.py.

import cv2
import grpc
import image_service_pb2
import image_service_pb2_grpc
import numpy as np
import threading

def generate_requests(cap):
    while True:
        ret, frame = cap.read()
        _, img_encoded = cv2.imencode('.jpg', frame)
        image_bytes = img_encoded.tobytes()
        yield image_service_pb2.ImageRequest(image_data=image_bytes)

def display_received_images(responses):
    for response in responses:
        nparr = np.frombuffer(response.image_data, np.uint8)
        img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
        cv2.imshow('Received Stream', img)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

def main():
    channel = grpc.insecure_channel('localhost:50051')
    stub = image_service_pb2_grpc.ImageServiceStub(channel)
    
    cap = cv2.VideoCapture(0)
    if not cap.isOpened():
        print("Error: Could not open webcam.")
        return

    requests = generate_requests(cap)
    responses = stub.SendImage(requests)

    display_thread = threading.Thread(target=display_received_images, args=(responses,))
    display_thread.start()

    while True:
        if cv2.waitKey(1) & 0xFF == ord('q'):
            cap.release()
            cv2.destroyAllWindows()
            break

if __name__ == '__main__':
    main()

Run the server first by executing python image_service_server.py. Then start two instances of the client by running python image_service_client.py twice.

Each client will capture frames from its webcam and send them to the server. The server will then forward these frames to the other client. The frames received from the other client will be displayed in a window labeled "Received Stream".

To exit, you can press the q key in any of the client windows.