Building Python Microservices: From 0 to 1 Creating a Highly Available E-commerce System-Blue Lotus Scripts

Origin

Have you ever been troubled by monolithic applications? As the codebase grows, modules become increasingly coupled, and a small change can trigger a chain reaction. As a Python developer, I deeply relate to this. Today, let me share how to build a truly decoupled, scalable microservice architecture for an e-commerce system using Python.

Breaking Down the Problem

Remember that e-commerce project I took over last year? The system was a massive Django application where any modification had to be made with extreme caution to avoid affecting other functionalities. The testing team frequently complained that regression testing took days. This made me think: why not try a microservice architecture?

Microservices isn't a new concept, but implementing it effectively is no easy task. Through this project's practice, I've developed a complete development methodology. Now, let's analyze this transformation process step by step.

Design

Before starting, we need to understand the core characteristics of microservices. Imagine a large e-commerce system as a shopping mall, where each microservice is like a specialty store, each performing its own function while working together.

Our system was ultimately divided into these core services: - Product Catalog Service: Manages product information, categories, and inventory - Order Service: Handles order creation, payment, and delivery - User Service: Responsible for user registration, authentication, and profile management - Payment Service: Integrates with payment gateways, processes payment flows - Search Service: Provides product search functionality

This separation is based on business capabilities rather than technical layers, which is an important principle in microservice design. Each service is a complete business function unit, developed and maintained by a dedicated small team.

Implementation

Let's start with the core product service. I chose the Flask framework because it's lightweight and flexible, perfect for building microservices. Let's look at the specific implementation:

from flask import Flask, jsonify
from flask_restful import Api, Resource
import asyncio
import aiohttp

app = Flask(__name__)
api = Api(app)

class ProductService(Resource):
    async def get_product_details(self, product_id):
        # Simulate fetching product information from database
        product = {
            'id': product_id,
            'name': 'iPhone 13',
            'price': 6999,
            'stock': 100
        }
        return product

    def get(self, product_id):
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        product = loop.run_until_complete(self.get_product_details(product_id))
        return jsonify(product)

api.add_resource(ProductService, '/products/<int:product_id>')

if __name__ == '__main__':
    app.run(port=5000)

Advanced Topics

After implementing basic functionality, we need to consider inter-service communication. I used a combination of REST API and message queues. Synchronous operations use REST API, while asynchronous operations use message queues.

For example, when a user places an order, the order service needs to query the product service in real-time for price information using REST API; while notifying the inventory service to reduce stock after order creation can be handled asynchronously using message queues.

I chose RabbitMQ for message queuing because of its high reliability and good Python support. Here's the implementation:

import pika
import json

class MessageQueue:
    def __init__(self):
        self.connection = pika.BlockingConnection(
            pika.ConnectionParameters('localhost')
        )
        self.channel = self.connection.channel()

    def publish_order_created(self, order_data):
        self.channel.queue_declare(queue='order_created')
        self.channel.basic_publish(
            exchange='',
            routing_key='order_created',
            body=json.dumps(order_data)
        )

    def consume_order_created(self, callback):
        self.channel.queue_declare(queue='order_created')
        self.channel.basic_consume(
            queue='order_created',
            on_message_callback=callback,
            auto_ack=True
        )
        self.channel.start_consuming()

Challenges

During implementation, we encountered several challenges. First was the complexity of distributed systems, where service call chains can be long, making problem localization difficult when issues arise.

To solve this, we introduced distributed tracing. Using OpenTelemetry to track request flow between services:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.jaeger.thrift import JaegerExporter

trace.set_tracer_provider(TracerProvider())
tracer = trace.get_tracer(__name__)

jaeger_exporter = JaegerExporter(
    agent_host_name="localhost",
    agent_port=6831,
)
trace.get_tracer_provider().add_span_processor(
    BatchSpanProcessor(jaeger_exporter)
)

@app.route('/products/<int:product_id>')
def get_product(product_id):
    with tracer.start_as_current_span("get_product") as span:
        span.set_attribute("product.id", product_id)
        # Business logic
        return jsonify({"status": "success"})

Results

After three months of refactoring, the new system went live. The results exceeded expectations:

Deployment frequency increased from 2 times per month to 3 times per week
Average fault recovery time reduced from 4 hours to 30 minutes
System scalability significantly improved, easily handling traffic peaks during Singles' Day
Development efficiency across teams improved by about 40%

Reflections

Looking back on this project, I have several insights to share:

Microservices isn't a silver bullet. It solves certain problems of monolithic applications but brings new challenges. The decision to adopt microservices should be based on actual circumstances.
Python performs excellently in microservice development, especially in rapid development and prototype validation. However, performance optimization is important, such as using asynchronous programming to improve concurrent processing capability.
The toolchain is important. Tools like containerization (Docker), service orchestration (Kubernetes), service discovery (Consul), monitoring and alerting (Prometheus + Grafana) can greatly reduce microservice maintenance costs.
Team culture needs to keep up. Microservice architecture requires teams to have full-stack thinking, demanding developers to understand operations and monitoring beyond coding.

Finally, I want to say that technology selection should always be based on solving real problems. Whether it's microservice architecture or Python, they're just tools. The key is understanding business requirements and finding the most suitable solution.

What do you think? Feel free to share your views and experiences in the comments.