— Anju Chamantha — 4 min read
Website : http://e-monitor.herokuapp.com
GitHub : https://github.com/anjuchamantha/EMonitor
YouTube : https://youtu.be/ITUB2IDSrXQ
EMonitor is an ESP32 Microcontroller based device which gets environmental data such as Temperature, Humidity, Pressure, Light Intensity, and sends those data to a Back-end database server using CAP(Common Alert Protocol). Also, those data and other connection status information can be viewed from the OLED-display locally. The data that was sent to the server can be visualized using a simple web-page. The device also features self fault recovery mechanisms to add more reliability to the system. EMonitor also sends a warning e-mail to pre-defined e-mail addresses in case of any sensor data exceeds a given threshold value.
Prototype Device
Scope of the Project
EMonitor is an ESP32 microcontroller-based device which has Wi-Fi inbuilt. With its Xtensa LX6 microprocessor operating at 240 MHz and memory of 520 KiB SRAM, EMonitor is more than capable of capturing the environment and sending data to the server after doing the necessary calculations seamlessly. It runs on FreeRTOS, a leading operating system supported by Arduino which can be working with the relevant peripherals smoothly.
Temperature (°C) — DHT11 Sensor module
Humidity (%) — DHT11 Sensor module
Pressure (Pa) — BMP180 Sensor module
Light Level (%) — LDR module
The device can be powered using a Micro USB cable which is a very reliable mode of powering the ESP32 microcontroller. Since almost every standard mobile phone charging adapter outputs 5V, this device can be powered using the same way which means there’s nothing to worry about power. Also, EMonitor can be powered using any mobile phone charging power-bank which makes the device portable.
The ESP32 is a low power consuming microcontroller relative to other microcontrollers in the market and has adaptive power controlling mechanisms which can handle some basic power-related matters that make the EMonitor very reliable when powering with a Micro USB cable.
Data is transmitted from the EMonitor device to the cloud-hosted EMonitor server using CAP (Common Alerting Protocol) over the internet. CAP is an XML-based data format for exchanging public warnings and emergencies between alerting technologies.
The sensor data (temperature, humidity, pressure, light level) are read at T time intervals and after N number of sensor readings, the Mean & Standard Deviation of those N number of data are sent to the server using CAP protocol. So the data is sent to the server at TN time intervals. (There can be an average 2–3s delay other than TN due to the other logics and calculations)
Example: Take N=15 & T=60s => The sensor readings are taken every 1 minute and after 15 rounds/readings (after 1min x 15 = 15 minutes) the mean and standard deviation of those 15 samples are sent to the server.
Following are the basic data sent to the server as a message which is referred to as MSG
hereafter.
msg_id, timestamp, temperature, humidity, pressure, light, temperature_sd, humidity_sd, pressure_sd, light_sd
Other than these data, some constant data that are mandatory for CAP, like sender
, msg type
, category
, urgency
etc are also sent to the server.
The EMonitor device is capable of withstanding the following scenarios in which the connection is lost.
Wi-Fi connection Lost
Loss of connection to the EMonitor server
The device can self re-connect to Wi-Fi or self re-connect to the server in any of the above cases. But the data is secure even during such connection issues.
In both the above cases, EMonitor detects the connection issue and cash the raw data which were not sent to the server due to connection loss, to a buffer. After each TN time interval, the device checks the connection and if the connection is stable, it sends all the data which were buffered, to the server and clears the buffer. (During this process also if the connection is lost, it keeps the data safe in the buffer without removing)
Deployed EMonitor website: http://e-monitor.herokuapp.com/
The backend server is developed using Flask which is a Python-based web framework. It supports all the existing python libraries and tools and hence it is very much scalable and extensible.
The database used is PostgreSQL and all the data that comes to the server from the EMonitor device are saved in a database table of values, msg_id, timestamp, temperature, humidity, pressure, light, temperature_sd, humidity_sd, pressure_sd, light_sd
The backend supports the following end-points to communicate with it.
[POST] : /data
[GET] : /
The EMonitor device uses /data
end-point to send data to the server using CAP protocol.
/
endpoint is used to access the main dashboard of the EMonitor website, which shows the current(last received) readings as well as charts and the data of the last 20 readings received.
OLED Display shows the temperature, humidity, pressure, and light level values which are sent to the server with the msg_id
after each TN time interval. Also, it shows the Wi-Fi and Server connection statuses.(0 - Not connected, 1 - Connected)
It also shows if a MSG
is not sent to the server and buffered (using a *
symbol) and sends buffered MSG
s to the server
The EMonitor device sends a WARNING e-mail to a pre-defined e-mail address if the value of a MSG
exceeds a given threshold limit.
Currently, if the temperature mean exceeds 40°C it sends a warning e-mail to my personal email.
Breadboard Circuit Schematic
Schematic Diagram
PCB Design
As mentioned in 2.5) Self-Recovery in a Connection Loss, EMonitor is capable of self connecting to Wi-Fi & Server after a connection retrieval. During the period of connection loss, the MSGs are buffered and when the connection is established back, it sends the buffered MSGs to the server.
1void loop{2 //calculate readings and create CAP MSG3 if (WiFi.status() != WL_CONNECTED)4 {5 //if not connected, try to connect to WiFi in each loop iteration6 bool connected = connect_to_wifi();7 if (connected){8 //send MSG to server9 }10 else{11 //buffer MSG12 }13 }14 else{15 //send MSG to server16 }17 }1819 bool connect_to_wifi(){20 WiFi.begin(SSID,PW) //begin the wifi connection21 if (WiFi.status() == WL_CONNECTED)22 {23 return truue;24 }25 else{26 return false;27 }28 }
There are a total of 11 buffers as follows
1queue<String> buffer_identifier;2 queue<String> buffer_datetime;3 queue<int> buffer_msg_ids;45 //mean buffers6 queue<double> buffer_t;7 queue<double> buffer_h;8 queue<double> buffer_p;9 queue<double> buffer_l;1011 //sd buffers12 queue<double> buffer_t_;13 queue<double> buffer_h_;14 queue<double> buffer_p_;15 queue<double> buffer_l_;1617 void loop{18 //BUFFER LOGIC1920 //loop buffer if buffer is not empty21 // if connected22 // POST buffer data and pop from buffer23 // else break24 while (!buffer_msg_ids.empty()){25 if (WiFi.status() == WL_CONNECTED){26 //create CAP MSG from raw data in buffers27 if (sendPostRequest(xmlchar_buf, msg_id_buf)){28 popBuffers();29 }30 }31 }32 33 //MAIN LOGIC34 35 //calculate readings and create CAP MSG36 if (WiFi.status() != WL_CONNECTED)37 {38 bool connected = connect_to_wifi();39 if (connected){40 //send MSG to server41 if (!sendPostRequest(xmlchar, msg)){42 //buffer MSG43 pushToBuffers();44 }45 }46 else{47 //buffer MSG48 pushToBuffers();49 }50 }51 else{52 //send MSG to server53 if (!sendPostRequest(xmlchar, msg)){54 //buffer MSG55 pushToBuffers();56 }57 }58 }5960 void pushToBuffers(){61 //push all the raw data needed for MSG to queues62 }6364 void popBuffers(){65 //pop the front element from the queues66 }
When after a power failure and when the power is retrieved back, the device is automatically started and continued the data reading and data transmitting process as usual.
NodeMCU ESP-32S Microcontroller (LKR 1,050.00)
BMP180 Digital Barometric Pressure Sensor (LKR 185.00)
DHT11 Temperature & Relative Humidity Sensor (LKR 250.00)
LDR (LKR 10)
OLED Display 0.96" (LKR 600.00)
Resistor 220Ohm (LKR 1.00)
Capacitor 10 micro F (LKR 20) — Used to back out the code uploading issue of ESP32
Vero Board & Circuit Wires (LKR 150)
main.cpp
1void beginOLED(){2 //initial setup and starting of OLED Display3 }4 void displayText(){5 //display the content in the OLED display6 //in various places, this function is called to print sensor and other information7 }8 void getTimeStamp(){9 //get the current time10 }11 void popBuffers(){12 //pop the front element from the queues13 }14 void pushToBuffers(){15 //push all the raw data needed for MSG to queues16 }1718 void setup(){19 beginOLED();20 wait_and_connect_to_wifi();21 begin_sensors();22 configTime();23 }2425 void loop(){26 27 //BUFFER LOGIC2829 //loop buffer if buffer is not empty30 // if connected31 // POST buffer data and pop from buffer32 // else break33 while (!buffer_msg_ids.empty()){34 if (WiFi.status() == WL_CONNECTED){35 //generate CAP XML message from the data in buffers36 generateXMLStr();37 if (sendPostRequest(xmlchar_buf, msg_id_buf)){38 popBuffers();39 }40 }41 }42 43 //MAIN LOGIC44 45 //calculate readings and create CAP MSG46 int x = 0;47 int rounds = 15;48 int round_time = 2000;49 while (x < rounds){50 readTemperature();51 readHumidity();52 readPressure();53 readLightIntensity();54 55 delay(round_time);56 x++;57 }58 temperature = calculate_mean(t_, rounds);59 humidity = calculate_mean(h_, rounds);60 pressure = calculate_mean(p_, rounds);61 light = calculate_mean(l_, rounds);62 63 temperature_sd = calculate_sd(t_, rounds, temperature);64 humidity_sd = calculate_sd(h_, rounds, humidity);65 pressure_sd = calculate_sd(p_, rounds, pressure);66 light_sd = calculate_sd(l_, rounds, light);67 68 getTimeStamp(datetime_);69 70 //send WARNING e-mail71 if ((temperature > 40) && (WiFi.status() == WL_CONNECTED)){72 //if a warning e-mail has not sent in last 5minutes73 // send mail74 }75 76 //generate CAP XML message from the calculated data77 generateXMLStr();78 79 if (WiFi.status() != WL_CONNECTED)80 {81 bool connected = connect_to_wifi();82 if (connected){83 //send MSG to server84 if (!sendPostRequest(xmlchar, msg)){85 //buffer MSG86 pushToBuffers();87 }88 }89 else{90 //buffer MSG91 pushToBuffers();92 }93 }94 else{95 //send MSG to server96 if (!sendPostRequest(xmlchar, msg)){97 //buffer MSG98 pushToBuffers();99 }100 }101 }
1#database table model2 class EMonitor(db.Model):3 __tablename__ = 'sensor_data'4 id = db.Column(db.Integer, primary_key=True)5 msg_id = db.Column(db.String(128))6 timestamp = db.Column(db.String(128))78 temperature = db.Column(db.Float)9 humidity = db.Column(db.Float)10 pressure = db.Column(db.Float)11 light = db.Column(db.Float)1213 temperature_sd = db.Column(db.Float)14 humidity_sd = db.Column(db.Float)15 pressure_sd = db.Column(db.Float)16 light_sd = db.Column(db.Float)17 18 19 def extract_data_from_xml(xml_str):20 """21 Given a XML string(CAP) this extracts the 'parameter' values22 :param xml_str: XML as a string23 :return: data as dictionary {name:value}24 """25 26 def put_to_db(xml_data):27 """28 given a data dictionary this method put the data to the database29 :param xml_data: data as dictionary {name:value}30 :return: True if database write successful, otherwise False31 """32 33 @app.route('/data', methods=['POST'])34 def post():35 xml_str = request.data36 xml_data = extract_data_from_xml(xml_str)37 print("[POST] /data : ", xml_data)38 if put_to_db(xml_data):39 return "DATABASE updated"40 else:41 return "DATABASE modification failed !"42 43 @app.route('/')44 def index():45 #query the database table and get the list of data46 table = EMonitor.query.order_by(desc(EMonitor.timestamp)).limit(20).all()4748 return render_template("index.html", table)
The full source code(Both EMonitor device & Server) can be viewed in https://github.com/anjuchamantha/EMonitor