Audio beehive monitoring based on IoT-Ai techniques: A survey and perspective

owski et al., 2018; Kulyukin et al., 2018). performances in speech recognition. A problem 
Moreover, the devices and sensors recording the that needs to be considered in using these 
bee sounds are reasonably priced and suitable for algorithms is how to build a bee sound dataset 
the conditions of production in Vietnam. large enough to train the classifying model. 
Therefore, these results can be applied to develop 
AI techniques-based systems for classifying Automatic identification of other important 
beehive states from bee sounds in bee farms in beehive states 
Vietnam. This will provide beekeepers with a The recognition of Varroa mites is just one 
continuous and autonomous analysis of their of several problems related to the recognition of 
beehives. The architecture of our proposed pest infestations based on audio data. After the 
approach is illustrated in Figure 2. study of Qandour et al. (2014), we could not find 
 any other studies related to this problem in the 
Exploring new AI algorithms for bee sound literature. That is to say, more research is needed 
analysis in this area. One could use other AI algorithms 
 We have found from reviewing the (as discussed in the previous subsection) or one 
 could combine sound data analyses with other 
references related to methods for monitoring 
 types of data like humidity and temperature 
beehives based on acoustic analyses in the 
 within a beehive to improve the performance of 
literature that most of these studies focused on 
 the monitoring process (Ferrari et al., 2008). The 
the problem of feature extraction from raw data. 
 system to collect these data would need to have a 
Although several AI algorithms, which are listed 
 simple design and low cost, consistent with the 
in Table 1, have been proposed to process the 
 production conditions in Vietnam. 
features from the extraction step, these 
algorithms were just some standard machine 
learning algorithms and some popular deep Conclusions 
learning algorithms. Many other AI algorithms An audio-based beehive monitoring system 
are well documented to have very good includes many components, ranging from 
performance in recognizing and classifying designing an IoT-based system to the use of 
audio data such as the Boltzmann machine feature extraction and AI algorithms. Among 
(Salakhutdinov & Hinton, 2009), deep recurrent them, the AI algorithms contribute significantly to 
neural networks (Phan et al., 2017), long short the performance and accuracy of the monitoring
 Figure 1. The architecture of audio-based beehive monitoring system 
https://vjas.vnua.edu.vn/ 537 
Audio beehive monitoring based on IoT-AI techniques: a survey and perspective 
 Table 1. A list of AI algorithms with applications for audio-based beehive monitoring 
 AI 
 Application Scenarios Feature Extraction References 
 algorithms 
 MFCCs 
 Robles-Guerrero et al. 
 MFFCs, Chroma sfft, Mel spectrograms, and 
 Recognition of the bee queen’s (2017) 
 LR Tonnetz 
 absence Amlathe (2018) 
 MFFCs, temporal features, and DFT 
 Kulyukin et al. (2018) 
 magnitudes 
 MFFCs, Chroma sfft, Mel spectrograms, and 
 Tonnetz Amlathe (2018) 
 KNN Recognition of bee sounds 
 MFFCs, temporal features, and DFT Kulyukin et al. (2018) 
 magnitudes 
 MFFCs, Chroma sfft, Mel spectrograms, and 
 Tonnetz Amlathe (2018) 
 RF Recognition of bee sounds 
 MFFCs, temporal features, and DFT Kulyukin et al. (2018) 
 magnitudes 
 MFCCs Zganks (2018) 
 HMM Recognition of swarming 
 MFCCs and LPC Zganks (2019) 
 Recognition of swarming Short Frequency Spectra Bencsik (2007) 
 PCA 
 Recognition of pest infestations PF and STFT Qandour et al. (2014) 
 LDA Recognition of pest infestations PF and STFT Qandour et al. (2014) 
 Nolasco et al. (20 
 MFCCs, Mel spectrograms, and HHT Nolasco (2018) 
 Recognition of the bee queen’s LPC Nolasco & Benetos 
 absence MFFCs, Chroma sfft, Mel spectrograms, and (2018) 
 Recognition of the bee queen’s Tonnetz Cejrowski et al. (2018) 
 SVM absence MFFCs, temporal features, and DFT Amlathe (2018) 
 magnitudes Kulyukin et al. (2018) 
 Recognition of swarming MFCC and Mel spectrograms Nolasco & Benetos 
 Recognition of pest infestations LPC (2018) 
 PF and STFT Cejrowski et al. (2018) 
 Qandour et al. (2014) 
 MFCCs, Mel spectrograms, and HHT Nolasco et al. (2019) 
 Recognition of the bee queen’s 
 MFFCs, temporal features, and DFT Kulyukin et al. (2018) 
 CNN absence 
 magnitudes 
 Recognition of bee sounds Nolasco & Benetos 
 MFCCs, Mel spectrograms, and HHT (2018) 
 Recognition of the bee queen’s 
 SOM StockWell Transform Howard (2013) 
 absence 
system. This paper has presented a survey on the The authors thank the anonymous referees 
AI techniques used in an audio-based beehive for their insightful and valuable suggestions 
monitoring system. Several opportunities for the which helped to improve the quality of the final 
applications of these advanced techniques to manuscript. 
monitoring beehives in Vietnam have been 
discussed. This is a promising topic worthy of 
 References 
being considered to contribute to improving the 
quality of monitoring honey bee colonies in Amlathe P. (2018). Standard machine learning techniques 
Vietnam. in audio beehive monitoring: Classification of audio 
 samples with logistic regression, K-nearest neighbor, 
 random forest and support vector machine. Master of 
Acknowledgements Science, Utah State University, 57 pages. 
538 Vietnam Journal of Agricultural Sciences 
 Nguyen Huu Du et al. (2020) 
Antonio R. G., Tonatiuh S. A., Efr´en, G. R. & Carlos E. G. Hochreiter S. & Schmidhuber J. (1997). Long short-term 
 (2017). Frequency Analysis of Honey Bee Buzz for memory. Neural Computation. 9(8): 1735-1780. 
 Automatic Recognition of Health Status: A Howard D., Duran O., Hunter G. & Stebel K. (2013). Signal 
 Preliminary Study. Research in Computing Science. Processing the acoustics of honeybees (APIS 
 142: 89-98. MELLIFERA) to identify the” queenless” state in 
Bencsik M., Bencsik J., Baxter M., Lucian A., Romieu J. & Hives. Proceedings of the Institute of Acoustics. 35(1): 
 Millet M. (2011). Identification of the honey bee 290-297.. 
 swarming process by analysing the time course of hive Huang N. E. & Schen S. (2005). An Introduction to Hilbert-
 vibrations. Computers and Electronics in Agriculture. Huang Transform and Its Applications. In: Huang N. 
 76(1): 44-50. E. & Schen S. (Eds.). Interdisciplinary Mathematical 
Bromenshenk J. J., Henderson C. B., Seccomb R. A., Rice Sciences, volume 5: Hilbert-Huang Transform and Its 
 S. D. & Etter R. T. (2009). Honey bee acoustic Applications. ISBN: 978-981-256-376-7. 
 recording and analysis system for monitoring hive Klein A.-M., Vaissiere B. E., Cane J. H., Steffan-Dewenter 
 health. U.S. Patent. No: 7,549,907 B2. I., Cunningham S. A., Kremen C. & Tscharntke T. 
Brundage T. J. (2012). Acoustic sensor for beehive (2007). Importance of pollinators in changing 
 monitoring. U.S. Patent 8: 152-590. landscapes for world crops. Proceedings of the Royal 
Cecchi S., Terenzi A., Orcioni S., Spinsante S., Primiani V. Society B: Biological Sciences. 274(1608): 303-313. 
 M., Moglie F., Ruschioni S., Mattei C., Riolo P. & Kulyukin V., Mukherjee S. & Amlathe P. (2018). Toward 
 Isidoro N. (2019). Multi-sensor platform for real time audio beehive monitoring: Deep learning vs. standard 
 measurements of honey bee hive parameters. IOP machine learning in classifying beehive audio samples. 
 Conference Series: Earth and Environmental Science. Applied Sciences. 8(9): 1573. 
 275: 012016. DOI:10.1088/1755-1315/275/1/012016. 
 Kulyukin V. A., Putnam M. & Reka S. K. (2016). 
Cejrowski T., Szymański J., Mora H. & Gil D. (2018). Digitizing buzzing signals into A440 piano note 
 Detection of the bee queen presence using sound sequences and estimating forager traffic levels from 
 analysis. Asian Conference on Intelligent Information images in solar-powered, electronic beehive 
 and Database Systems. Springer. 297-306. monitoring. Proceedings of the International 
Chollet F. (2017). Deep Learning with Python. Manning MultiConference of Engineers and Computer 
 Publications. 384 pages. Scientists, March 16-18, 2016, Hong Kong. 
Crawford M. (2017). Automated collection of honey bee Kulyukin V. A. (2019). BeePi: A Multisensor Electronic 
 hive data using the Raspberry Pi, Master of science, Beehive Monitor [Online]. Retrieved from 
 Appalachian State University, 69 pages. https://www.kickstarter.com/projects/beepihoneybees
Davis S. & Mermelstein P. Comparison of parametric meetai/beepi-a-multisensor-electronic-beehive-
 representations for monosyllabic word recognition in monitor on May 14, 2020. 
 continuously spoken sentences. IEEE Transactions on Li T. & Cavusgil S. T. (1995). A classification and 
 Acoustics, Speech, and Signal Processing. 28: 357- assessment of research streams in international 
 366. marketing. International Business Review. 4(3): 251-
Dietlein D. G. (1985). A method for remote monitoring of 277. 
 activity of honeybee colonies by sound analysis. Marchi E., Vesperini F., Squartini S. & Schuller B. (2017). 
 Journal of Apicultural Research. 24(3): 176-183. Deep recurrent neural network-based autoencoders for 
Kridi D. S., Carvalho C. G. N. d. & Gomes D. G. (2014). A acoustic novelty detection. Computational Intelligence 
 predictive algorithm for mitigate swarming bees and Neuroscience. DOI: 10.1155/2017/4694860. 
 through proactive monitoring via wireless sensor Martin S. J. (2001). The role of Varroa and viral pathogens 
 networks. Proceedings of the 11th ACM symposium in the collapse of honeybee colonies: a modelling 
 on Performance evaluation of wireless ad hoc, sensor, approach. Journal of Applied Ecology. 38(5): 1082-
 & ubiquitous networks. 41-47. 1093. 
Ferrari S., Silva M., Guarino M. & Berckmans D. (2008). Mezquida D. A. & Martínez J. L. (2009). Platform for bee-
 Monitoring of swarming sounds in bee hives for early hives monitoring based on sound analysis. A perpetual 
 detection of the swarming period. Computers and warehouse for swarm s daily activity. Spanish journal 
 electronics in agriculture. 64(1): 72-77. of agricultural research. 7(4): 824-828. 
Gil-Lebrero S., Quiles-Latorre F. J., Ortiz-López M., Michelsen A., Kirchner W. H. & Lindauer M. (1986). 
 Sánchez-Ruiz V., Gámiz-López V. & Luna-Rodríguez Sound and vibrational signals in the dance language of 
 J. J. (2017). Honey bee colonies remote monitoring the honeybee, Apis mellifera. Behavioral ecology and 
 system. Sensors. 17(1): 55-76. sociobiology. 18(3): 207-212. 
Haste T., Tibshirani R. & Friedman J. (2009). The elements Nolasco I. & Benetos E. (2018). To bee or not to bee: 
 of statistical learning: Data Mining, Inference, and Investigating machine learning approaches for beehive 
 Prediction. Springer-Verlag New York. 745 pages. sound recognition. Workshop on Detection and 
 DOI: 10.1007/978-0-387-84858-7. Classification of Acoustic Scenes and Events 
https://vjas.vnua.edu.vn/ 539 
Audio beehive monitoring based on IoT-AI techniques: a survey and perspective 
 (DCASE), New York, USA, October 25-26, 2018: mechanism of flight guidance in honeybee swarms: 
 133-137. subtle guides or streaker bees? The Journal of 
Nolasco I. (2018). Audio-based beehive state recognition, Experimental Biology. 211(20): 3287-3295. 
 Master thesis, Queen Mary University of London. Schurischuster S., Zambanini S., Kampel M. & Lamp B. 
Nolasco I., Terenzi A., Cecchi S., Orcioni S., Bear H. L. & (2016). Sensor study for monitoring varroa mites on 
 Benetos E. (2019). Audio-based identification of honey bees (apis mellifera). Visual Observation and 
 beehive states. IEEE International Conference on Analysis of Vertebrate and Insect Behavior Workshop, 
 Acoustics, Speech and Signal Processing (ICASSP), Cancun Mexico, December 4, 2016. 
 Brighton, UK, May 12 – 17, 2019: 8256-8260. Seeley T. D. & Tautz J. (2001). Worker piping in honey bee 
OSBeehives platform (2018). OSBeehives | BuzzBox Hive swarms and its role in preparing for liftoff. Journal of 
 Health Monitor & Beekeeping App. Retrieved from Comparative Physiology. 187: 667-676. 
 https://www.osbeehives.com on May 14, 2020. Tautz J. (2008). The Buzz about Bees: Biology of a 
Phan H., Koch P., Katzberg F., Maass M., Mazur R. & Superorganism. Springer, Berlin, Heidelberg. 
 Mertins A. (2017). Audio scene classification with Terenzi A., Cecchi S., Orcioni S. & Piazza F. (2019). 
 deep recurrent neural networks. arXiv preprint Features Extraction Applied to the Analysis of the 
 arXiv:1703.04770. Sounds Emitted by Honey Bees in a Beehive. 11th 
Qandour A., Ahmad I., Habibi D. & Leppard M. (2014). International Symposium on Image and Signal 
 Remote beehive monitoring using acoustic signals. Processing and Analysis (ISPA). 3-8. 
 Acoustics Australia. 42(3): 204-209. Vancata I. (1995). Using acoustic technology to monitor 
Ramírez M., Prendas J. P., Travieso C. M., Calderón R. & your hives. American Bee Journal. 135(9): 615-618. 
 Salas O. (2012). Detection of the mite Varroa Visscher P. K. & Seeley T. D. (2007). Coordinating a group 
 destructor in honey bee cells by video sequence departure: who produces the piping signals on 
 processing. 2012 IEEE 16th International Conference honeybee swarms? Behavioral Ecology and 
 on Intelligent Engineering Systems (INES). IEEE. Sociobiology. 61(10): 1615-1621. 
 103-108. 
 Zacepins A., Kviesis A., Stalidzans E., Liepniece M. & 
Ramsey M., Bencsik M. & Newton M. I. (2017). Long-term Jurijs Meitalovs (2016). Remote detection of the 
 trends in the honeybee ‘whooping signal’ revealed by swarming of honey bee colonies by single-point 
 automated detection. PLoS One. 12(7): e0181736. temperature monitoring. Biosystems Engineering. 
 DOI: 10.1371/journal.pone.0171162. 148: 76-80. 
Raspberry Pi Foundation (2016). Raspberry Pi Products". Zacepins A., Kviesis A., Ahrendt P., Richter U., Tekin S. 
 Retrieved from https://www.raspberrypi.org/products/ & Durgun M. (2016). Beekeeping in the future - Smart 
 on May 14, 2020. apiary managemen. 17th International Carpathian 
Rittschof C. C. & Seeley T. D. (2008). The buzz-run: how Control Conference (ICCC). 808-812. 
 honeybees signal ‘Time to go!’. Animal Behaviour. Zganks A. (2018). Acoustic monitoring and classification 
 75(1): 189-197. of bee swarm activity using MFCC feature extraction 
Salakhutdinov R. & Hinton G. (2009). Deep boltzmann and HMM acoustic modeling. ELEKTRO. 1-4. 
 machines. Artificial intelligence and statistics. 448- Zganks A. (2019). Bee Swarm Activity Acoustic 
 455. Classification for an IoT-Based Farm Service. Sensors. 
Schultz K. M., Passino K. M. & Seeley T. D. (2008). The 20(1).: 21-35. DOI: 10.3390/s20010021. 
540 Vietnam Journal of Agricultural Sciences