# Go Cart Actor A distributed cart management system using the actor model pattern. ## Prerequisites - Go 1.24.2+ - Protocol Buffers compiler (`protoc`) - protoc-gen-go and protoc-gen-go-grpc plugins ### Installing Protocol Buffers On Windows: ```powershell winget install protobuf ``` On macOS: ```bash brew install protobuf ``` On Linux: ```bash # Ubuntu/Debian sudo apt install protobuf-compiler # Or download from: https://github.com/protocolbuffers/protobuf/releases ``` ### Installing Go protobuf plugin ```bash go install google.golang.org/protobuf/cmd/protoc-gen-go@latest go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest ``` ## Working with Protocol Buffers ### Generating Go code from proto files After modifying any proto (`proto/messages.proto`, `proto/cart_actor.proto`, `proto/control_plane.proto`), regenerate the Go code (all three share the unified `messages` package): ```bash cd proto protoc --go_out=. --go_opt=paths=source_relative \ --go-grpc_out=. --go-grpc_opt=paths=source_relative \ messages.proto cart_actor.proto control_plane.proto ``` ### Protocol Buffer Messages The `proto/messages.proto` file defines the following message types: - `AddRequest` - Add items to cart (includes quantity, sku, country, optional storeId) - `SetCartRequest` - Set entire cart contents - `AddItem` - Complete item information for cart - `RemoveItem` - Remove item from cart - `ChangeQuantity` - Update item quantity - `SetDelivery` - Configure delivery options - `SetPickupPoint` - Set pickup location - `PickupPoint` - Pickup point details - `RemoveDelivery` - Remove delivery option - `CreateCheckoutOrder` - Initiate checkout - `OrderCreated` - Order creation response ### Building the project ```bash go build . ``` ### Running tests ```bash go test ./... ``` ## HTTP API Quick Start (curl Examples) Assuming the service is reachable at http://localhost:8080 and the cart API is mounted at /cart. Most endpoints use an HTTP cookie named `cartid` to track the cart. The first request will set it. ### 1. Get (or create) a cart ```bash curl -i http://localhost:8080/cart/ ``` Response sets a `cartid` cookie and returns the current (possibly empty) cart JSON. ### 2. Add an item by SKU (implicit quantity = 1) ```bash curl -i --cookie-jar cookies.txt http://localhost:8080/cart/add/TEST-SKU-123 ``` Stores cookie in `cookies.txt` for subsequent calls. ### 3. Add an item with explicit payload (country, quantity) ```bash curl -i --cookie cookies.txt \ -H "Content-Type: application/json" \ -d '{"sku":"TEST-SKU-456","quantity":2,"country":"se"}' \ http://localhost:8080/cart/ ``` ### 4. Change quantity of an existing line (First list the cart to find `id` of the line; here we use id=1 as an example) ```bash curl -i --cookie cookies.txt \ -X PUT -H "Content-Type: application/json" \ -d '{"id":1,"quantity":3}' \ http://localhost:8080/cart/ ``` ### 5. Remove an item ```bash curl -i --cookie cookies.txt -X DELETE http://localhost:8080/cart/1 ``` ### 6. Set entire cart contents (overwrites items) ```bash curl -i --cookie cookies.txt \ -X POST -H "Content-Type: application/json" \ -d '{"items":[{"sku":"TEST-SKU-AAA","quantity":1,"country":"se"},{"sku":"TEST-SKU-BBB","quantity":2,"country":"se"}]}' \ http://localhost:8080/cart/set ``` ### 7. Add a delivery (provider + optional items) If `items` is empty or omitted, all items without a delivery get this one. ```bash curl -i --cookie cookies.txt \ -X POST -H "Content-Type: application/json" \ -d '{"provider":"standard","items":[1,2]}' \ http://localhost:8080/cart/delivery ``` ### 8. Remove a delivery by deliveryId ```bash curl -i --cookie cookies.txt -X DELETE http://localhost:8080/cart/delivery/1 ``` ### 9. Set a pickup point for a delivery ```bash curl -i --cookie cookies.txt \ -X PUT -H "Content-Type: application/json" \ -d '{"id":"PUP123","name":"Locker 5","address":"Main St 1","city":"Stockholm","zip":"11122","country":"SE"}' \ http://localhost:8080/cart/delivery/1/pickupPoint ``` ### 10. Checkout (returns HTML snippet from Klarna) ```bash curl -i --cookie cookies.txt http://localhost:8080/cart/checkout ``` ### 11. Using a known cart id directly (bypassing cookie) If you already have a cart id (e.g. 1720000000000000): ```bash CART_ID=1720000000000000 curl -i http://localhost:8080/cart/byid/$CART_ID curl -i -X POST -H "Content-Type: application/json" \ -d '{"sku":"TEST-SKU-XYZ","quantity":1,"country":"se"}' \ http://localhost:8080/cart/byid/$CART_ID ``` ### 12. Clear cart cookie (forces a new cart on next request) ```bash curl -i --cookie cookies.txt -X DELETE http://localhost:8080/cart/ ``` Tip: Use `--cookie-jar` and `--cookie` to persist the session across multiple commands: ```bash curl --cookie-jar cookies.txt http://localhost:8080/cart/ curl --cookie cookies.txt http://localhost:8080/cart/add/TEST-SKU-123 ``` ## Important Notes - Always regenerate protobuf Go code after modifying any `.proto` files (messages/cart_actor/control_plane) - The generated `messages.pb.go` file should not be edited manually - Make sure your PATH includes the protoc-gen-go binary location (usually `$GOPATH/bin`) --- ## Architecture Overview The system is a distributed, sharded (by cart id) actor model implementation: - Each cart is a grain (an in‑memory struct `*CartGrain`) that owns and mutates its own state. - A **local grain pool** holds grains owned by the node. - A **synced (cluster) pool** (`SyncedPool`) coordinates multiple nodes and exposes local or remote grains through a uniform interface (`GrainPool`). - All inter‑node communication is gRPC: - Cart mutation & state RPCs (CartActor service). - Control plane RPCs (ControlPlane service) for membership, ownership negotiation, liveness, and graceful shutdown. ### Key Processes 1. Client HTTP request (or gRPC client) arrives with a cart identifier (cookie or path). 2. The pool resolves ownership: - If local grain exists → use it. - If a remote host is known owner → a remote grain proxy (`RemoteGrainGRPC`) is used; it performs gRPC calls to the owning node. - If ownership is unknown → node attempts to claim ownership (quorum negotiation) and spawns a local grain. 3. Mutation is executed via the **mutation registry** (registry wraps domain logic + optional totals recomputation). 4. Updated state returned to caller; ownership preserved unless relinquished later (not yet implemented to shed load). --- ## Grain & Mutation Model - `CartGrain` holds items, deliveries, pricing aggregates, and checkout/order metadata. - All mutations are registered via `RegisterMutation[T]` with signature: ``` func(*CartGrain, *T) error ``` - `WithTotals()` flag triggers automatic recalculation of totals after successful handlers. - The old giant `switch` in `CartGrain.Apply` has been replaced by registry dispatch; unregistered mutations fail fast. - Adding a mutation: 1. Define proto message. 2. Generate code. 3. Register handler (optionally WithTotals). 4. Add gRPC RPC + request wrapper if the mutation must be remotely invokable. 5. (Optional) Add HTTP endpoint mapping to the mutation. --- ## Local Grain Pool - Manages an in‑memory map `map[CartId]*CartGrain`. - Lazy spawn: first mutation or explicit access triggers `spawn(id)`. - TTL / purge loop periodically removes expired grains unless they changed recently (basic memory pressure management). - Capacity limit (`PoolSize`); oldest expired grain evicted first when full. --- ## Synced (Cluster) Pool `SyncedPool` wraps a local pool and tracks: - `remoteHosts`: known peer nodes (gRPC connections). - `remoteIndex`: mapping of cart id → remote grain proxy (`RemoteGrainGRPC`) for carts owned elsewhere. Responsibilities: 1. Discovery integration (via a `Discovery` interface) adds/removes hosts. 2. Periodic ping health checks (ControlPlane.Ping). 3. Ownership negotiation: - On first contention / unknown owner, node calls `ConfirmOwner` on peers to achieve quorum before making a local grain authoritative. 4. Remote spawning: - When a remote host reports its cart ids (`GetCartIds`), the pool creates remote proxies for fast routing. --- ## Remote Grain Proxies A `RemoteGrainGRPC` implements the `Grain` interface but delegates: - `Apply` → Specific CartActor per‑mutation RPC (e.g., `AddItem`, `RemoveItem`) constructed from the mutation type. (Legacy envelope removed.) - `GetCurrentState` → `CartActor.GetState`. Return path: 1. gRPC reply (CartMutationReply / StateReply) → proto `CartState`. 2. `ToCartState` / mapping reconstructs a local `CartGrain` snapshot for callers expecting grain semantics. --- ## Control Plane (Inter‑Node Coordination) Defined in `proto/control_plane.proto`: | RPC | Purpose | |-----|---------| | `Ping` | Liveness; increments missed ping counter if failing. | | `Negotiate` | Merges membership views; used after discovery events. | | `GetCartIds` | Enumerate locally owned carts for remote index seeding. | | `ConfirmOwner` | Quorum acknowledgment for ownership claim. | | `Closing` | Graceful shutdown notice; peers remove host & associated remote grains. | ### Ownership / Quorum Rules - If total participating hosts < 3 → all must accept. - Otherwise majority acceptance (`ok >= total/2`). - On failure → local tentative grain is removed (rollback to avoid split‑brain). --- ## Request / Mutation Flow Examples ### Local Mutation 1. HTTP handler parses request → determines cart id. 2. `SyncedPool.Apply`: - Finds local grain (or spawns new after quorum). - Executes registry mutation. 3. Totals updated if flagged. 4. HTTP response returns updated JSON (via `ToCartState`). ### Remote Mutation 1. `SyncedPool.Apply` sees cart mapped to a remote host. 2. Routes to `RemoteGrainGRPC.Apply`. 3. Remote node executes mutation locally and returns updated state over gRPC. 4. Proxy materializes snapshot locally (not authoritative, read‑only view). ### Checkout (Side‑Effecting, Non-Pure) - HTTP `/checkout` uses current grain snapshot to build payload (pure function). - Calls Klarna externally (not a mutation). - Applies `InitializeCheckout` mutation to persist reference + status. - Returns Klarna order JSON to client. --- ## Scaling & Deployment - **Horizontal scaling**: Add more nodes; discovery layer (Kubernetes / service registry) feeds hosts to `SyncedPool`. - **Sharding**: Implicit by cart id hash. Ownership is first-claim with quorum acceptance. - **Hot spots**: A single popular cart remains on one node; for heavy multi-client concurrency, future work could add read replicas or partitioning (not implemented). - **Capacity tuning**: Increase `PoolSize` & memory limits; adjust TTL for stale cart eviction. ### Adding Nodes 1. Node starts gRPC server (CartActor + ControlPlane). 2. After brief delay, begins discovery watch; on event: - New host → dial + negotiate → seed remote cart ids. 3. Pings maintain health; failed hosts removed (proxies invalidated). --- ## Failure Handling | Scenario | Behavior | |----------|----------| | Remote host unreachable | Pings increment `MissedPings`; after threshold host removed. | | Ownership negotiation fails | Tentative local grain discarded. | | gRPC call error on remote mutation | Error bubbled to caller; no local fallback. | | Missing mutation registration | Fast failure with explicit error message. | | Partial checkout (Klarna fails) | No local state mutation for checkout; client sees error; cart remains unchanged. | --- ## Mutation Registry Summary - Central, type-safe registry prevents silent omission. - Each handler: - Validates input. - Mutates `*CartGrain`. - Returns error for rejection. - Automatic totals recomputation reduces boilerplate and consistency risk. - Coverage test (add separately) can enforce all proto mutations are registered. --- ## gRPC Interfaces - **CartActor**: Per-mutation unary RPCs + `GetState`. (Checkout logic intentionally excluded; handled at HTTP layer.) - **ControlPlane**: Cluster coordination (Ping, Negotiate, ConfirmOwner, etc.). **Ports** (default / implied): - CartActor & ControlPlane share the same gRPC server/listener (single port, e.g. `:1337`). - Legacy frame/TCP code has been removed. --- ## Security & Future Enhancements | Area | Potential Improvement | |------|------------------------| | Transport Security | Add TLS / mTLS to gRPC servers & clients. | | Auth / RBAC | Intercept CartActor RPCs with auth metadata. | | Backpressure | Rate-limit remote mutation calls per host. | | Observability | Add per-mutation Prometheus metrics & tracing spans. | | Ownership | Add lease timeouts / fencing tokens for stricter guarantees. | | Batch Ops | Introduce batch mutation RPC or streaming updates (WatchState). | | Persistence | Reintroduce event log or snapshot persistence layer if durability required. | --- ## Adding a New Node (Operational Checklist) 1. Deploy binary/container with same proto + registry. 2. Expose gRPC port. 3. Ensure discovery lists the new host. 4. Node dials peers, negotiates membership. 5. Remote cart proxies seeded. 6. Traffic routed automatically based on ownership. --- ## Adding a New Mutation (Checklist Recap) 1. Define proto message (+ request wrapper & RPC if remote invocation needed). 2. Regenerate protobuf code. 3. Implement & register handler (`RegisterMutation`). 4. Add client (HTTP/gRPC) endpoint. 5. Write unit + integration tests. 6. (Optional) Add to coverage test list and docs. --- ## High-Level Data Flow Diagram (Text) ``` Client -> HTTP Handler -> SyncedPool -> (local?) -> Registry -> Grain State \-> (remote?) -> RemoteGrainGRPC -> gRPC -> Remote CartActor -> Registry -> Grain ControlPlane: Discovery Events <-> Negotiation/Ping/ConfirmOwner <-> SyncedPool state ``` --- ## Troubleshooting | Symptom | Likely Cause | Action | |---------|--------------|--------| | New cart every request | Secure cookie over plain HTTP or not sending cookie jar | Disable Secure locally or use HTTPS & proper curl `-b` | | Unsupported mutation error | Missing registry handler | Add `RegisterMutation` for that proto | | Ownership flapping | Quorum failing due to intermittent peers | Investigate `ConfirmOwner` errors / network | | Remote mutation latency | Network / serialization overhead | Consider batching or colocating hot carts | | Checkout returns 500 | Klarna call failed | Inspect logs; no grain state mutated | ---