Arduino UNO
The Arduino UNO is used to collect with sensor directly. It is easy to get the value of temperature and humidity by collecting the sensor directly to one of the analog input on Arduino.[3] For round force sensor, since it is a force-sensitive resistor so we need to convert the reading into the weight. We first connect the round force sensor in series with a 10k ohm resistor and add a 5V DC voltage as the input across the circuit, By voltage division, we get the voltage across sensor, where the reading is a number between (0, 1023). The reading is linear to the voltage so we can easily convert the reading to the voltage (0, 5) across the sensor.
The next step is to find the relationship between voltage and force, we referenced the data sheet [2] and we found the log(voltage) is proportional to the log(weight). Since there are 4 sensor in the system and they are all located at the corner of the plate. Hence, the change of sensor may not sensitively reflect the weight change of the central of the plate. Given 4 coordinates, we use bilinear interpolation to estimate the weight change at the center of the plate. Until this step, all data conversion is done.
The next step is to find the relationship between voltage and force, we referenced the data sheet [2] and we found the log(voltage) is proportional to the log(weight). Since there are 4 sensor in the system and they are all located at the corner of the plate. Hence, the change of sensor may not sensitively reflect the weight change of the central of the plate. Given 4 coordinates, we use bilinear interpolation to estimate the weight change at the center of the plate. Until this step, all data conversion is done.
Raspberry Pi
The Arduino UNO collect all of the data from the sensor, the next step is to send the data from Arduino to Raspberry Pi. Here we use serial connection by a USB cable. Also, there is a embedded camera connected with Raspberry Pi that it monitors the plate surroundings.[4] We use a face-detection algorithm (OpenCV [1]) running in Raspberry Pi to judge whether a person(customer) pass the food and how long he/she may stop to get food. Combined with sensors’ data, the system is able to judge whether the customer like the food (person detected, weight decreases) or the customer don’t like the food (person detected, weight changes without change over threshold). After Raspberry Pi get enough data, it will send the data to cloud with regular time for storage and future process.
Whole Embedded System
Algorithms for main thread & customer tracking
For the main thread, we utilize the customer list maintained by the face tracking thread and the data list maintained by the sensor thread. We will generate a data packet for every 10s or when the customer list is not empty.In the first condition, when no customer is detected, we pack the weight and temperature, then send them to dynamoDB.When customer is detected, ie, the customer list is not empty, we traverse the customer list and find the timestamps of the customer come and leave, then we traverse the data list, calculate the weight change during this time and the standard deviation of weight center changes, we mark them as the food taken by the customer and index of customer fiddle with the food. Also, we calculate the frequency that customers come. Then we pack all the data and send it to dynamoDB. Fig.3 illustrate the details.
The flow chart of this process:
The flow chart of this process:
We continuously capture photo from the camera, and use a Haar feature match method to detect the people in this photo. Then, due to the limited computing resources of raspberry pi, we use the location and the diagonal of the detected faces for customer tracking, which means, if the newly captured face has similar location and diagonal with the existed customers, we consider them as the same person. The criteria of the judge is depend on the previous data and the processing speed (FPS).
For every coming customer, we build a dictionary recording timestamp, customer duration, and last captured time. The list is maintained continuously. We add new customers, update existing customers and delete invalid customer, invalid customer means the person whose “last captured time” is more than 3s ago.
The flow chart of this process:
For every coming customer, we build a dictionary recording timestamp, customer duration, and last captured time. The list is maintained continuously. We add new customers, update existing customers and delete invalid customer, invalid customer means the person whose “last captured time” is more than 3s ago.
The flow chart of this process:
To sum up
- From sensor to Arduino:
- We use Arduino UNO connected with temperature and humidity AND round force sensor to collect real-time temperature, humidity and food weight. Round force sensor is a force-sensitive resistor. The effective resistor range is between 200 ohm - 30k ohm with weight range between 20g - 10kg.
- We use Arduino UNO connected with temperature and humidity AND round force sensor to collect real-time temperature, humidity and food weight. Round force sensor is a force-sensitive resistor. The effective resistor range is between 200 ohm - 30k ohm with weight range between 20g - 10kg.
- From Arduino to Raspberry Pi:
- All the collected raw data then send to Raspberry Pi for second convert through USB cable.
- The temperature and humidity can be read directly, no second modification needed.
- The weight is a value between (0, 1023) where 0 refers no force and 1023 refers maximum force. The value is proportional to the voltage so that voltage division is needed to get voltage across the round force sensor. For this sensor: log(voltage) is proportional to log(resistance).
- Given 4 coordinates, we use bilinear interpolation to estimate the weight change in central of the plate.
- The temperature and humidity can be read directly, no second modification needed.
- A embedded camera is collected with Raspberry Pi to monitor users’ behaviour, which includes:
- Detect whether a user(customer) pass the food or the amount of time the user stay in front of the food.
- The detection works combined with round force sensor to detect whether how long the user taking the food
- Detect whether a user(customer) pass the food or the amount of time the user stay in front of the food.
- All the collected raw data then send to Raspberry Pi for second convert through USB cable.