MQTT Protocol Basics for IoT Applications

What is MQTT?

MQTT (Message Queuing Telemetry Transport) is a lightweight publish-subscribe messaging protocol designed for constrained devices and low-bandwidth networks. It’s perfect for IoT applications where devices need to communicate efficiently.

Why MQTT for IoT?

• Lightweight: Minimal overhead, ideal for embedded devices
• Reliable: Built-in quality of service levels
• Scalable: Supports thousands of concurrent connections
• Bi-directional: Devices can both send and receive messages
• Low power: Efficient for battery-powered devices

Core Concepts

1. Publish-Subscribe Pattern

Unlike traditional request-response, MQTT uses pub-sub:

Publisher → Topic → Broker → Subscribers

• Publisher: Sends messages to topics
• Broker: Routes messages between publishers and subscribers
• Subscriber: Receives messages from topics
• Topic: Named channel for messages

2. Topics

Topics are hierarchical strings separated by slashes:

home/living-room/temperature
home/bedroom/light/state
office/sensor/humidity

Wildcards:

• + matches one level: home/+/temperature
• # matches multiple levels: home/#

Setting Up an MQTT Broker

Using Mosquitto (Popular Open-Source Broker)

Install on Linux:

sudo apt-get update
sudo apt-get install mosquitto mosquitto-clients

Start the broker:

sudo systemctl start mosquitto
sudo systemctl enable mosquitto

Test with command-line tools:

# Subscribe to a topic
mosquitto_sub -h localhost -t "test/topic"

# Publish a message (in another terminal)
mosquitto_pub -h localhost -t "test/topic" -m "Hello MQTT!"

Configuration

Edit /etc/mosquitto/mosquitto.conf:

# Allow anonymous connections (for testing only!)
allow_anonymous true

# Listen on all interfaces
listener 1883 0.0.0.0

# WebSocket support
listener 9001
protocol websockets

# Persistence
persistence true
persistence_location /var/lib/mosquitto/

# Logging
log_dest file /var/log/mosquitto/mosquitto.log
log_type all

Quality of Service (QoS) Levels

MQTT provides three QoS levels:

QoS 0: At Most Once

• Fire and forget
• No acknowledgment
• Fastest, but messages may be lost

client.publish("sensor/data", "25.5", qos=0)

QoS 1: At Least Once

• Acknowledged delivery
• Messages guaranteed to arrive
• May receive duplicates

client.publish("sensor/data", "25.5", qos=1)

QoS 2: Exactly Once

• Four-way handshake
• Guaranteed single delivery
• Slowest, highest overhead

client.publish("sensor/data", "25.5", qos=2)

When to use each:

• QoS 0: Frequent sensor readings where occasional loss is acceptable
• QoS 1: Important data where duplicates can be handled
• QoS 2: Critical commands where duplicates would cause problems

Python MQTT Client

Install the Paho MQTT library:

pip install paho-mqtt

Basic Publisher

import paho.mqtt.client as mqtt
import time
import random

# Create client
client = mqtt.Client("temperature_sensor")

# Connect to broker
client.connect("localhost", 1883, 60)

# Publish messages
while True:
    temperature = random.uniform(20.0, 30.0)
    client.publish("home/living-room/temperature", f"{temperature:.1f}")
    print(f"Published: {temperature:.1f}°C")
    time.sleep(5)

Basic Subscriber

import paho.mqtt.client as mqtt

def on_connect(client, userdata, flags, rc):
    print(f"Connected with result code {rc}")
    # Subscribe on connect so subscription persists after reconnection
    client.subscribe("home/+/temperature")

def on_message(client, userdata, msg):
    print(f"{msg.topic}: {msg.payload.decode()}")

client = mqtt.Client("temperature_monitor")
client.on_connect = on_connect
client.on_message = on_message

client.connect("localhost", 1883, 60)

# Blocking call that processes network traffic and dispatches callbacks
client.loop_forever()

Advanced Features

Retained Messages

The broker stores the last message on a topic:

# Publish with retain flag
client.publish("home/status", "online", retain=True)

# New subscribers immediately receive the retained message

Use cases:

• Device status (online/offline)
• Last known sensor value
• Configuration settings

Last Will and Testament (LWT)

Automatically notify subscribers if a client disconnects unexpectedly:

client = mqtt.Client("sensor_device")

# Set LWT before connecting
client.will_set(
    "home/sensor/status",
    payload="offline",
    qos=1,
    retain=True
)

client.connect("localhost", 1883, 60)

# Publish online status
client.publish("home/sensor/status", "online", retain=True)

Keep-Alive

Maintain connection with periodic pings:

# Keep-alive interval in seconds (default 60)
client.connect("localhost", 1883, keepalive=30)

ESP32/Arduino MQTT Client

For embedded devices using Arduino framework:

#include <WiFi.h>
#include <PubSubClient.h>

const char* ssid = "YOUR_WIFI";
const char* password = "YOUR_PASSWORD";
const char* mqtt_server = "192.168.1.100";

WiFiClient espClient;
PubSubClient client(espClient);

void setup_wifi() {
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("\nWiFi connected");
}

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  
  String message;
  for (int i = 0; i < length; i++) {
    message += (char)payload[i];
  }
  Serial.println(message);
  
  // Handle message
  if (String(topic) == "home/led/set") {
    if (message == "ON") {
      digitalWrite(LED_PIN, HIGH);
    } else {
      digitalWrite(LED_PIN, LOW);
    }
  }
}

void reconnect() {
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    
    if (client.connect("ESP32Client")) {
      Serial.println("connected");
      client.subscribe("home/led/set");
      client.publish("home/status", "online");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(115200);
  pinMode(LED_PIN, OUTPUT);
  
  setup_wifi();
  client.setServer(mqtt_server, 1883);
  client.setCallback(callback);
}

void loop() {
  if (!client.connected()) {
    reconnect();
  }
  client.loop();
  
  // Publish sensor data every 10 seconds
  static unsigned long lastMsg = 0;
  unsigned long now = millis();
  
  if (now - lastMsg > 10000) {
    lastMsg = now;
    
    float temperature = readTemperature();
    char msg[50];
    snprintf(msg, 50, "%.2f", temperature);
    
    client.publish("home/temperature", msg);
  }
}

Security Best Practices

1. Use TLS/SSL

import ssl

client = mqtt.Client()

# Configure TLS
client.tls_set(
    ca_certs="/path/to/ca.crt",
    certfile="/path/to/client.crt",
    keyfile="/path/to/client.key",
    tls_version=ssl.PROTOCOL_TLSv1_2
)

client.connect("mqtt.example.com", 8883, 60)

2. Authentication

client.username_pw_set("username", "password")
client.connect("localhost", 1883, 60)

3. Network Segmentation

• Isolate IoT devices on separate VLAN
• Use firewall rules to restrict access
• Only allow necessary ports (1883, 8883)

Troubleshooting

Connection Issues

def on_connect(client, userdata, flags, rc):
    if rc == 0:
        print("Connected successfully")
    elif rc == 1:
        print("Connection refused - incorrect protocol version")
    elif rc == 2:
        print("Connection refused - invalid client identifier")
    elif rc == 3:
        print("Connection refused - server unavailable")
    elif rc == 4:
        print("Connection refused - bad username or password")
    elif rc == 5:
        print("Connection refused - not authorized")

Debug Logging

import logging

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)

client.enable_logger(logger)

Performance Tips

1. Use appropriate QoS: Don’t use QoS 2 unless necessary
2. Batch messages: Combine multiple readings when possible
3. Optimize topic structure: Keep topics short and logical
4. Clean sessions: Use clean_session=False for persistent subscriptions
5. Connection pooling: Reuse connections instead of reconnecting

Real-World Example: Temperature Monitoring

Complete system with publisher and subscriber:

# publisher.py - Simulates temperature sensor
import paho.mqtt.client as mqtt
import time
import random
import json

client = mqtt.Client("temp_sensor_01")
client.connect("localhost", 1883, 60)

while True:
    data = {
        "sensor_id": "temp_01",
        "temperature": round(random.uniform(18.0, 28.0), 2),
        "humidity": round(random.uniform(30.0, 70.0), 2),
        "timestamp": time.time()
    }
    
    client.publish(
        "sensors/temperature/living-room",
        json.dumps(data),
        qos=1
    )
    
    print(f"Published: {data}")
    time.sleep(10)

# subscriber.py - Monitors and logs data
import paho.mqtt.client as mqtt
import json
from datetime import datetime

def on_connect(client, userdata, flags, rc):
    print("Connected to broker")
    client.subscribe("sensors/temperature/#")

def on_message(client, userdata, msg):
    data = json.loads(msg.payload.decode())
    
    timestamp = datetime.fromtimestamp(data['timestamp'])
    print(f"[{timestamp}] {data['sensor_id']}: "
          f"{data['temperature']}°C, {data['humidity']}%")
    
    # Store in database, trigger alerts, etc.
    if data['temperature'] > 25:
        print("⚠️  High temperature alert!")

client = mqtt.Client("temp_monitor")
client.on_connect = on_connect
client.on_message = on_message

client.connect("localhost", 1883, 60)
client.loop_forever()

Conclusion

MQTT is the backbone of modern IoT communication. You’ve learned:

• Core MQTT concepts and architecture
• QoS levels and when to use them
• Python and Arduino implementations
• Security best practices
• Troubleshooting techniques

Next Steps

• Check out the IoT Home Automation project for a complete implementation
• Experiment with different QoS levels
• Set up TLS encryption
• Build your own IoT device

Resources

• MQTT.org Official Specification
• Mosquitto Documentation
• Paho MQTT Python Client
• HiveMQ MQTT Essentials