MAS-DUO: Reference Implementation
Repository: github.com/pgayos/MAS-DUO
Overview
MAS-DUO (Multi-Agent System for Dynamic Use and Optimisation) is an open-source Python re-implementation of the architecture described in the doctoral thesis “Improving the Decision Support in Shop Floor Operations by Using Agent-based Systems and Visibility Frameworks” (UCLM, 2012). The original prototype was written in Java on JADE; this version translates the concepts of Chapters 3 and 4 into a configurable simulation environment built on the PettingZoo AEC standard, with the goal of making the thesis propositions testable with modern multi-agent reinforcement learning tools.
Current Status
The project is work in progress and does not yet reproduce the quantitative results of the thesis. The environment runs, is deterministic for a given seed and passes the PettingZoo api_test and seed_test, but several parts of the architecture are modelled and not yet connected to the simulation loop:
- Products are not yet picked up and transported along their routes, so order completion metrics are not meaningful.
- The IS Platform negotiation is implemented but not yet triggered by product agents.
- The global policy weights
(A, B, C, D)are computed but do not yet drive the rewards returned by the environment. - The MDP Q-values are not yet updated after execution.
The next milestones are to fix the simulation core, add end-to-end integration tests, define KPIs (resources used, delays, utilisation) and compare against FIFO, operations research and MARL baselines over multiple seeds.
Thesis Concepts vs. Implementation
1. Three-Layer Architecture
The thesis separates cognitive agent reasoning (BDI), optimisation over state transitions (MDP) and enterprise-level policy arbitration (IS Platform). MAS-DUO maps them to these modules:
| Thesis concept | Implementation module |
|---|---|
| BDI cognitive loop (beliefs, desires, intentions) | logistics_env/agents/base_agent.py — BDIContext, GeneratedBelief, PhysicalBDIState |
| MDP state-transition engine | logistics_env/mdp/mdp_engine.py — MDPEngine (select_action, update_q, detect_visibility_gaps) |
| IS Platform (ERP + CRM + Expert System) | logistics_env/is_platform/is_platform.py — ISPlatform |
The intended execution cycle per simulation step is:
Observation → BDI loop (beliefs → intention)
→ MDPEngine → R(s, s')
→ NegotiationProposal → ISPlatform
→ GlobalPolicy update → propagated to all agents2. What/Where/When/Why State and RFID/EPCIS Visibility
The thesis builds agent beliefs from the What? Where? When? Why? dimensions of EPCIS events, and defines the state of a product as read point (Where) × business step (Why) (Eq. 4). In MAS-DUO every agent keeps a state with these four dimensions (called “4W” in the code):
| Dimension | EPCIS field | Implementation |
|---|---|---|
| What — object identity | epcList (EPC URI) | agent.state.what → urn:epc:id:sgtin:... |
| Where — physical location | readPoint / bizLocation | agent.state.where → Position(x, y) + zone_id |
| When — event timestamp | eventTime | agent.state.when → simulation step integer |
| Why — business context | bizStep / disposition | agent.state.why → BusinessStep enum |
EPC codes are modelled in logistics_env/objects/epc.py as Pure Identity URI strings. Binary encodings (e.g. SGTIN-96) and EPCIS events are not implemented yet.
3. Agent Society and Role Specialisation
MAS-DUO implements four concrete agent types:
| Thesis agent role | Class | Action space | Airport mapping |
|---|---|---|---|
| Physical product / job | ProductAgent | WAIT, REQUEST_MOVE, REQUEST_PROCESS, SIGNAL_READY | Flights (B737, A320, LCC, CARGO) |
| Mobile resource / equipment | RobotAgent | IDLE, MOVE (4 dirs), LIFT, DROP, CHARGE, NAVIGATE | Pushback tractors, stairs, fuel trucks, cargo loaders, passenger buses |
| Human resource / worker | WorkerAgent | IDLE, MOVE (4 dirs), PICK, PLACE, PROCESS, SCAN, REST | Ramp agents, supervisors, crew chiefs |
| Automated conveyor / line | ConveyorAgent | STOP, RUN, RUN_FAST, REVERSE | Baggage belts (gate → terminal) |
ProductAgent is the agent with the BDI loop; the other types are currently simpler reactive state machines that maintain the same four-dimensional state.
4. BDI–MDP Integration
In the thesis, BDI provides the intentional structure while the MDP provides the optimisation over state transitions. In MAS-DUO the product agent generates a GeneratedBelief from its observation, selects the next state of its route with the MDP engine and records it as its intention. Wiring that intention into the executed action, and updating the Q-values after each transition, is part of the ongoing work.
In the thesis airport case, flights have 10 business steps: the 8 IATA ground handling functions (Organisation & Management, Station Management, Load Control, Passenger Handling, Baggage Handling, Aircraft Handling & Loading, Aircraft Ground Movement, Cargo & Mail Handling) plus airport-specific steps. The current code uses a generic logistics vocabulary of business steps (UNPROCESSED, IN_TRANSIT, PROCESSING, DISPATCHED, …).
5. Reward Function and Global Policy
The thesis evaluates each transition with a weighted sum of operational criteria (Eq. 6 and 8):
$$R(s, s’) = A \cdot \text{Time} + B \cdot \text{Cost} + C \cdot \text{QoS} + D \cdot \text{Energy}$$
For the Ciudad Real Central Airport, the airport management staff set Equation 12: R(s,s') = 0.5 Time + 0.4 Cost + 0.1 Energy, with no QoS term so that no airline is given preference. MAS-DUO models these weights with the GlobalPolicy class and PolicyParameters(A, B, C, D):
| Parameter | Symbol | Thesis value (CRC) |
|---|---|---|
| Time / delay weight | A | 0.5 |
| Cost weight | B | 0.4 |
| QoS weight | C | 0.0 |
| Energy weight | D | 0.1 |
The weighted sum is implemented, and policies can change during a run via scheduled_changes. Making these weights drive the rewards returned by the environment is pending.
6. IS Platform — Negotiation
In the thesis, the ERP, CRM and expert-system agents validate the proposals of the physical agents and, if they reject them, send new policy parameters. MAS-DUO models the three sub-components:
| Sub-component | Thesis function | Implementation parameters |
|---|---|---|
| ERP | Cost and capacity constraints | max_cost, max_delay, production_capacity |
| CRM | QoS and client priority | min_qos_threshold, client_priority |
| Expert System | Minimum acceptable reward | min_reward_threshold |
Each sub-component evaluates a NegotiationProposal and the platform returns approval, rejection or a counter-proposal with updated PolicyParameters. The current evaluation is threshold-based (not yet a Contract Net protocol), and product agents do not raise proposals yet.
7. State Space — Read Points and Business Steps (Equation 11)
The airport configuration follows the state definition of the thesis:
Resource read points (RP): { HANGAR, PARK1, PARK2, PARK3, PARK4 } -> 5 RPs
Resource business steps (BS): { Free, Busy, InTransit, NotAvailable } -> 4 BS
|S_resource| = 5 × 4 = 20 states (Equation 11)
Flights: 4 parking read points × 10 business steps = 40 statesThe grid world (20 × 15 cells at 5 m/cell, a 100 × 75 m apron) represents the read points as zones, with A* pathfinding for resource navigation.
8. Airport Scenario — Ciudad Real Central Airport
Chapter 4.1 of the thesis evaluates MAS-DUO with a demonstrator of the Ciudad Real Central Airport ramp operations against a FIFO allocation. MAS-DUO ships two configurations for this scenario:
airport_gh_check.py— validation script with 4 flights (B737, A320, LCC, CARGO) and 21 agents. It runs a few random-policy steps to check configuration loading, state consistency and order tracking.airport_gh_demo.py— demonstration with 8 flights and a greedy Earliest Deadline First policy, rendered with Pygame. Fleet: 12 ground handling robots (pushback tractors, stairs, fuel trucks, ULD loaders, buses), 7 workers and 4 baggage belts.
These scripts are demonstrators of the environment, not a reproduction of the thesis results. The FIFO baseline of the thesis and the resource-reduction metric of its Table 4 are not implemented yet.
9. Observation Space
| Agent type | Vector shape | Components |
|---|---|---|
ProductAgent | (6,) float32 [0,1] | Normalised position (x,y), route progress, deadline proximity, energy, processing flag |
WorkerAgent | (6,) float32 [0,1] | Position, energy, fatigue level, zone index, previous action |
RobotAgent | (5,) float32 [0,1] | Position, battery level, load status, speed |
ConveyorAgent | (5,) float32 [0,1] | Operational state, occupancy, speed setting, energy consumption |
The compact vectors and the PettingZoo AEC interface are meant to make the environment usable with standard multi-agent RL algorithms (e.g. PPO variants, QMIX) once the simulation core is complete.
10. Framework and Technology Choices
| Thesis prototype | Implementation choice | Rationale |
|---|---|---|
| JADE (Java, FIPA-ACL) with a custom BDI module | PettingZoo AEC + Python | Modern RL ecosystem; direct connection to PyTorch-based libraries |
| EPCIS repository (Fosstrak) | epc.py (pure Python EPC model) | Self-contained, no JVM dependency |
| Demonstrator and laboratory test bench | Pygame renderer + headless mode | Visual inspection and scripted runs |
| — | JSON scenarios + config_loader.py (dataclasses) | Declarative scenario authoring, separate from the model |