Novel Biological Based Method for Robot Navigation and Localization

1.Introduction

Navigation and localization are two closely linked processes with great importance in everyday life. Navigation is a field of study that deals with moving objects from one place to another. In a broader sense, it can refer to any skill or study that involves the trajectory determination and direction. One of the most important cognitive processes during navigation is decision-making, which requires integrating many neural activities across different brain regions[1]. On the other hand,localization involves always the question: Where is the object now? Try to find out the position relative to some landmark,usually the point of origin or destination[2].

大雪纷飞的冬天，呼朋唤友，围坐着吃火锅，可是一件相当幸福的事情。火锅不仅是当今社会人们最喜欢的美味之一，也是古人热爱的菜式之一。那么火锅起源于何时，又有着怎样的发展历程呢？

These two processes are of particular importance in robotics, and especially for mobile robots. The current location of a robot and the navigation strategies can be determined by using different sensors, depending on the characteristics of the mobile robot and the environment. Many researchers use different methods, such as laser range finder and vision based navigation, joysticks and haptic devices, odometers, and global positioning system (GPS), for robot navigation[3]-[6].

农产品科技含量较低，品牌潜力有待挖掘。农业品牌建设必须有过硬的质量和足够的资金作为基础，而过硬的质量又需要由强大的科技作为后盾。为确保品牌农产品的高质量和高效益，农业企业应当加强对农产品的深加工。然而，现阶段卧龙区已认证的无公害品牌除了青华镇永兴农贸公司粗加工玉米面和玉米糁外，其他企业都没有对其产品内在价值进行加工，最终严重影响了企业效益。

However, one method that works well for one robot in one environment may not work well or at all in another environment.For example, localization which works well in an outdoor environment may be useless indoors. In order to enhance these problems, scientists started to involve brain machine interface(BMI) for robot navigation and localization[7],[8].

根据活菌生长繁殖速率的不同，可将生长曲线大致分为迟缓期、对数期、稳定期和衰亡期，所以活菌在不同时间之下进行培养都会对活菌的数量造成影响，其中，细菌在对数期生长迅速，活菌数以恒定的几何级数增长，生长曲线图上细菌数的对数呈直线上升，达到顶峰状态，期间细菌的形态、染色性、生物活性等都较典型，对外界环境因素的作用敏感，所以常会选取对数期的活菌作为研究对象进行试验。

BMI technologies are a class of neuro-technologies originally developed for medical assistive applications[9]. They have been largely focused on improving the quality of life for particular clinical populations and include, for example,applications for advanced communications with paralyzed patients as well as the direct control of prostheses and wheelchairs[10]-[12]. A common problem of all BMI applications is the acquired signal quality. The signal quality depends on the electrode quality, electrode location, signal power, etc., but generally the recorded signal is noisy. Nevertheless, the tremendous growth of researches in the field of neuroscience over the past decades offers an approach to address these limitations. Invasive-BMI technology is a method where the electrodes are implanted intracranially, directly to the desired brain location, in the animal’s brain. This methodology provides neural signals of the best quality[13].

Firstly, we pre-processed all the recorded LFPs data and removed the bad data. As described above, we focused on the decision-making process and analyzed the one second time window LFPs data in each Y-junction, 0.5 s before and after the event. In this paper, we considered the cases when the rat made a correct and/or a wrong choice. The average value of each feature for all sessions and then for all animals was calculated.

Furthermore, it is well known that animals can accurately and rapidly navigate in complex spaces. This ability to be organized in time and space stands on determining a heading direction[14] and localizing themselves in reference to the surrounding environmental cues. Therefore, animals utilize internal and external cues in navigating and constructing spatial representation[15]-[17]. However, few studies simultaneously integrate real rat’s neuroscience data with robot navigation.

In this paper we are focused on the investigation of the rat’s navigation and localization strategies, especially on the decision-making process, during a procedural maze task.Based on the previous behavior results we suggest that rats might use two different navigation strategies to solve the complex Y-maze task[18],[19]. Our theories are consistent with many other studies so far, which have demonstrated that hippocampus and striatum play an important role in navigation and decision-making. In order to verify our hypothesis, we record local field potentials (LFPs) with bipolar electrodes implanted in these brain regions (two electrodes in hippocampus (left and right sides) and one in dorsolateral striatum). The recorded LFPs are then analyzed and processed by using different Matlab based toolboxes, in order to extract different features. These features are used as the input in the artificial neural network which is trained and used to control the robot. The results indicate that the robot learns to integrate the internal states as well as external sensors in order to localize and navigate in the complex environment.

近年来，大数据技术已经成为社会的一个热点话题，很多行业中都能够看到大数据的身影。所谓大数据技术，就是数据量比较大的数据集合。大数据中一个最基础的特点是数据量大，但是对于大数据来说还不够。大数据的核心是各种数据的处理以及应用。大数据技术中的数据类型不仅有数字类数据，同时还有一些非结构化的数据，比如图片等，这些不同类型的数据组成为一个数据集。对大数据进行挖掘和分析的时候，其能够使用到很多中分析方式，并且构建数据模型来找到不同的数据之间存在的关系，以便后续进行更高的分析和预测。大数据的使用目标就是能够应用到很多场景中，通过对数据隐形关系之间的分析，能够极大的提高工作的效率，并且更好的实现工作目标。

2.Materials and Methods

All procedures were approved by the University of Toyama Committee on the Animal Care and were in accordance with the National Institutes of Health Guidelines for the Care and Use of Laboratory Animals.

2.1　Subjects

Two adult male Wistar/ST rats, purchased from Japan SLC, Inc. (Hamamatsu, Japan), were served as subjects.Rats weighed between 290 g and 310 g at the time of arrival and were maintained individually in a controlled conditions room: Room temperature (23±2) °C with a 12 hours lightdark cycle (lights on from 8:00 AM). Water was available ad libitum in the home cage. Food was restricted to motivate food seeking behavior and to maintain the rats at 85% of their free-feeding weights. A different cage was utilized to transport each rat from the housing room to the experiment area. In order to avoid the rats’ confusion, the transportation cage was covered with a black curtain.

2.2　Apparatus

Also, we used a video recording system (movement tracker) composed by a low cost camera (KBRC-M05VU) and a Panasonic CF-N10 i5 notebook. Camera, fixed in the center of the experimental area and 170 cm above the maze, had the following properties: 30 frames per second, infrared camera with self-adjustment of exposure and image adjustment functions. We built an easy Matlab based graphical user interface in order to track the rat movement, to collect the camera data, and to calculate the time spent by the animal to reach the reward location. Fig. 5 shows the Rat 2 movements tracked in three different sessions.

Fig. 1. Schematic representation of the experiment apparatus.

Fig. 2. Real experiment environment.

In the end of each route was installed a metallic cup of 2 cm depth, which contained food (cereal pellets). The food odor was equal for each goal location. Four of them were covered with a metallic net, covered food, and one was left uncovered,the reward location. The experiment room is aspirated and kept at the constant temperature of (24±1) °C. The environment was surrounded by heavy black curtains to visually isolate the Y-maze area and to avoid any light leaking from the outer environment as shown in Fig. 2. Illumination was provided by two fluorescent bulbs. In each side of the experiment area were hanged four different visually distinct cues:Square—at home side, triangle—at the goal location side,X—in the left, and circle—in the right side of the maze.

As shown in Fig. 2, cues were printed in black, in 42 cm×42 cm white cardboards. The same maze was used to perform the experiments with the mobile robot.

2.3　Training Procedure

Firstly, the animals were trained to find the uncovered food location (the reward), to learn and memorize the correct route.During training, each rat was placed on the starting position and allowed to navigate freely on the maze environment until it reached the reward location, which was fixed throughout the experiments. The training sessions took place between 6:00 PM to 8:00 PM for all rats. They received 1 session (20 trials) of training per day. For each training session, a correct trial was recorded only when the animal reached the reward with the first attempt, without making any turns in incorrect routes in the Y-junctions. A trial was terminated once the rat reached the reward location. After the animal ate the food, the experimenter relocated the rat to the bucket and cleaned the maze. The rats were trained to learn the maze in a lighted environment, until they reached an asymptotic level of learning and a criterion of at least 85% correct.

2.4　Data Acquisition

Training and other experiments sessions performed under different changes of the environment settings were followed by the surgery and electrode implantation process.

尘世中的好男人，应该就是这个模样了吧。生而为人，谁还没几个缺点呢？毕竟你我，也只是许姣容那般的普通女子，求的是平安顺遂，要的是白头偕老，盼的是子孙满堂。完美太难得，幸福就很好。

Fig. 3 shows the data acquisition system used in this work.We recorded LFPs from hippocampus (left and right sides) and dorsolateral striatum (left side), simultaneously. Three bipolar electrodes were implanted in the rat brain. The electrode positions are shown in Table 1. After performing the perfusion and Nissl stain process, we verified the real electrodes locations, as shown in Fig. 4. There presented a 30 μm coronal section around each electrode location, taken from Rat 1 brain.Electrodes pins on the animal head were connected with a preamplifier in order to eliminate the effect of noise on the recorded brain signals. LFPs were recorded by using a Nihon Kohden multichannel amplifier. The recording frequency was at 1 kHz and then LFPs were sub-sampled at 1 kHz for further analysis. All the data was stored as .csv file, in a personal computer.

Fig. 3. Data acquisition system.

Table 1: Electrode positions

Note that AP represents antero-posterior axis, ML represents mediolateralaxis,and DV represents dorsoventral axis.

Animal　Coordinates Brain region　AP　ML　DV Rat 1 & Rat 2　Hippocampus　　–4.92　　±2.5　　–2.3 Sriatum　+0.96　　–3.6　　–3.5

Fig. 4. Electrodes locations in: (a) dorsolateral striatum (left side),(b) hippocampus (right side), and (c) hippocampus (left side).

In each Y-junction of the maze we installed a distance sensor (GP2Y0A21YK0F). Signals from these sensors were recorded together with LFPs in one channel, and were used to mark the rat position on the LFPs data.

Animals were trained in a multiple Y-maze (170 cm×210 cm) and elevated 35 cm above the floor, and consisted of one starting spot and five goal locations, as shown in Fig. 1 and Fig. 2. The goal arms (routes) were numbered 1 to 5,clockwise. The maze was built of 10 cm width light blue polystyrene materials and covered with a transparent PVC sheet. All the routes were surrounded by 2 cm bands in order to protect the animals from sliding and falling down.

2.5　Feature Extraction

The value of the output neurons is not exactly “0” or“1”, but varies in the range [0, 1]. In our implementation,the output is 1 if the value is above the threshold 0.5, and 0 for other values. In order to generate the best solution for the decision-making task, we used a floating point genetic algorithm (GA).

The recorded data was represented by matrix where C is the number of channels and N is the number of sample time points. Channel 1 recorded the signal from the sensors S1, S2, and S3, which were used as markers of the rat position on the recorded LFPs. Channels 2 and 3 recorded the LFPs from hippocampus left and right sides, respectively.Channel 4 recorded the LFPs from the left side of striatum.

In these LFPs traces we focused on the decision-making events, represented by sensors S1, S2, and S3. These markers represent the moments in which the animal started turning left or right in each Y-junction, so the animal has already decided which route to follow. We analyzed the one second time epoch in the Y-junction, 0.5 s before and after the event, as shown in Fig. 7. Every epoch was then divided into N time windows for each LFPs trace, and performed the fast Fourier transformation(FFT) by using

SPF级健康成年SD大鼠20只，雌雄各半，体质量220～280 g。随机分为模型组和注射用雷贝拉唑钠1 mg/kg组，每组5雄5雌。动物禁食不禁水约24h后，在异氟烷轻度麻醉状态下，仰卧位固定于固定板上，腹部手术区常规消毒。于腹部剑突下正中作切口，暴露胃及幽门，用细线缝扎幽门，同时于食管‐胃交界处纵行切开贲门肌约1 cm，分离至黏膜层完全暴露于视野中，以加强胃反流［7］。将胃轻轻送回大鼠体内，间断缝合腹壁肌肉和皮肤。

We obtained the frequency representation and the power spectra of the processed LFPs. For every window and channel,the sample data mean values were calculated as where. The data was stored in a feature vector, where (fi are all the features extracted from the hippocampal and striatal LFPs). These features were used to train a feed forward neural network (FFNN).

Fig. 5. Rat 2 movements tracked from the camera in three different sessions: (a) initial phase of training, (b) rat almost learned the task, and (c) rat passed the 85% correct learning criteria.

Fig. 6. One trial representation of raw LFPs traces.

Fig. 7. Time window around decision-making events.

3.Robot and Artificial Neural Network

In this section we define the architecture of the neural network used to control the robot and the robot characteristics.Based on the input data, the e-puck robot motion was determined and sent by a Bluetooth connection with a Matlab based personal computer.

3.1　E-Puck Robot

In all the conducted experiments in this work, we utilized an e-puck robot, as shown in Fig. 8. We used this robot because it has a simple mechanical structure and fine electronics design, offering many possibilities with its sensors,processing power, and extensions. E-puck is small, flexible,and user friendly, and has good robustness. It does not need any cables, providing optimal working comfort[20],[21].

where wi,j are weights connections between nodes of different layers.

Fig. 8. E-puck robot.

Fig. 9. E-puck robot proximity sensors.

Moreover, encoder sensors were used by the robot during the navigation in other parts of the maze, especially to define the moment when the robot reached the reward location.

3.2　Neural Network

The decision-making process, based on the animal’s LFPs, was done by applying a FFNN, as shown in Fig. 10.We employed three neural networks, one per each Y-junction,with 6 input, 8 hidden, and 2 output units. Six features, two from each LFP, were used as the input to the neural network.

In the implemented neural network, the sigmoid function was used as an activation function for the hidden and output units, as shown in (2):

Activation ai of a neuron i is the sum of all weighted inputs from neuron j of previous layer:

E-puck robot has several sensors, such as sound sensors,encoders, odometry sensors, camera, and proximity sensors. In our experiments, the robot utilized the data from proximity sensors to perceive the decision-making moment in the Y-junctions, as shown in Fig. 9. We set a threshold value (around 2.3 cm) for the sensors IR 2 and IR 5, so when this threshold was passed then the robot stop moving.

Fig. 10. Artificial neural network architecture, where NN W_i_h means neural network weights connections between the input and hidden layers and NN W_h_o means neural network weights connections between the hidden and output layers.

Two output units of the neural controllers are the decision-making values “0” or “1”, which represents

i) Output [0 1]: Robot will turn right;

ii) Output [1 0]: Robot will turn left.

LFPs are generated by neuronal ensembles and contain information about the underlying cellular activity. They have extensively been used to investigate central circuit functions.The recorded data was processed offline by using Matlab signal processing toolbox and Chronux toolbox, in order to extract different features. Fig. 6 represents raw hippocampal and striatal LFPs traces recorded simultaneously during a single representative trial (Rat 1, Session 3, Trial 5).

4.Results

4.1　Selected Features from Hippocampal and Striatal LFPs

六是成功组织了北京市第八届商业服务业技能大赛（中式烹调师）比赛。来自全聚德、便宜坊、东来顺、华天等近60家餐饮企业的551名选手参加了比拼，通过初赛、复赛、决赛层层选拔，最终产生金牌奖9名、银牌奖14名、铜牌奖23名、优秀组织奖11家企业。

As shown in Figs. 11, 12, and 13, we found a dominant theta (7 Hz to 11 Hz) and beta (15 Hz to 20 Hz) ranges oscillation in hippocampal LFPs, in all Y-junctions, and for correct and wrong choices. In the other hand, in dorsolateral striatum we did not find any dominance of theta or beta oscillation, except for the case when the animal performed a wrong choice, we found a boost in the theta oscillation.

为解决电加热启停带来的温湿度惯性，SCR电加热初始以30%的热量输出。当室内环境制热量需求变化时，根据PID控制算法，SCR可控硅电加热无级调节输出精确的加热量。

There were marked contrasts between the left and right hippocampal LFPs power in all the Y-junctions, for correct and wrong choices. Along with this, during the decision-making epoch in the 1st and 2nd Y-junctions of the maze, the peak frequency of the hippocampal beta oscillation significantly changed from 17.58 Hz to 18.55 Hz for the correct and wrong choices. In all the experiment sessions, animals performed always a correct choice on the 3rd Y-junction, as shown in Fig. 13.

Fig. 11. Hippocampal and striatal power representations for correct and wrong choices in the 1st Y-junction.

Fig. 12. Hippocampal and striatal power representations for correct and wrong choices in the 2nd Y-junction.

All the dorsolateral striatum power spectra are shown as logarithmic representations because of the very small amplitude of the LFP oscillation founded in striatum. In our implementation, for each recording site (HPCL—hippocampus left side, HPCR—hippocampus right side, and STRL—striatum left side), we selected the peak frequency values and the mean power values of the theta oscillation for correct choices, as the input for the neural network.

Fig. 13. Hippocampal and striatal power representations for correct choices in the 3rd Y-junction.

4.2　Robot Navigation

The offline trained FFNN with the extracted LFPs features was used to control the robot during the decision-making process.

Fig. 14 shows the robot motion during an experiment trial where the robot reached the reward location after three accurate decisions made by the neural controller in the Y-junctions.Fig. 14 (a) shows the robot in start position, ready to move forward. The robot detects the 1st Y-junction by its sensors and stops moving, as shown in Fig. 14 (b). Neural network 1 takes the decision to turn right, the robot turns right and moves forward. In Fig. 14 (c) is shown the robot position in the maze,navigating toward the reward location. The robot detects the 2nd Y-junction by its proximity sensors and stops moving forward, as shown in Fig. 14 (d). Neural network 2 takes the decision to turn left. The robot turns left and moves forward toward the next Y-junction, as in Figs. 14 (e) and (f). When the robot reaches the 3rd Y-junction, neural network 3 takes the decision to turn left. It turns left and moves toward the reward location, as in Fig. 14 (g). Fig. 14 (h) shows the completed task by the robot. It reaches the reward location by using the animal LFPs features and its own external sensors.

5.Conclusions

In this paper, we introduced a novel method for robot navigation based on the animal’s brain activity. Our method is based on the distinctive features of the hippocampal and striatal LFPs, noticed during the animal’s decision-making process. In addition, we showed how these features can be used to control the robot in order to perform the same decisions as animals do.

Fig. 14. Robot decision-making and motion during one experiment trial: (a) in start position, (b) detecting the 1st Y-junction, (c) in the maze, (d) detecting the 2nd Y-junction, (e) turning left, (f) moving forward, (g) reaching the 3rd Y-junction, and (h) completing task.

The presented method used the real neural network, animal brain, and artificial intelligence. In addition, the robot learned to integrate the internal states with the external sensors, in order to perform reliable navigation in the multiple Y-maze. In the future, we plan to develop an algorithm that can be easily modified for different types of robots and environments.

References

[1]B. Hofmann-Wellenhof, K. Legat, and M. Wieser,Navigation: Principles of Positioning and Guidance, New York: Springer-Verlag Wien, 2003, pp. 5-6.

[2]Wikibooks contributors, Robotics/Navigation/Localization,The Free Textbook Project, 2016.

[3]G. Capi, M. Kitani, and K. Ueki, “Guide robot intelligent navigation in urban environments,” Advanced Robotics, vol.28, no. 15, pp. 1043-1053, 2014.

[4]G. Capi, “A vision based approach for intelligent robot navigation,” Intl. Journal of Intelligent Systems Technologies and Applications, vol. 9, no. 2, pp. 97-107, 2010.

[5]B. Hua, E. Rama, G. Capi, and M. Jindai, “A human-like robot intelligent navigation in narrow indoor environments,”Intl. Journal of Information and Electronics Engineering,vol. 6, no. 5, pp. 308-312, 2016.

[6]T. Oshima, M. Li, E. Rama, and G. Capi, “Intelligent robot navigation for surveillance behavior—A remote based approach,” in Proc. of IEEJ Intl. Workshop on Sensing,Actuation, Motion Control, and Optimization, 2016, pp. 1-4.

[7]M. Mano and G. Capi, “An adaptive BMI based method for real time wheelchair navigation,” Intl. Journal of Innovative Computing, Information and Control, vol. 9, no. 12, pp.4963-4972, 2013.

[8]G. Capi, “Real robots controlled by brain signals—A BMI approach,” Intl. Journal of Advance Intelligence, vol. 2, no.1, pp. 25-35, 2010.

[9]N. Birbaumer, N. Ghanayim, T. Hinterberger, et al., “A spelling device for the paralyzed,” Nature, vol. 398, no.6725, pp. 297-298, 1999.

[10]F. Piccioneet, F. Giorgi, P. Tonin, et al., “P300-based brain computer interface: Reliability and performance in healthy and paralyzed participants,” Clinical Neurophysiology, vol.117, no. 3, pp. 531-537, 2006.

[11]B. J. Lance, S. E. Kerick, A. J. Ries, K. S. Oie, and K.McDowell, “Brain-computer interface technologies in the coming decades,” Proc. of the IEEE, vol. 100, pp. 1585-1599, May 2012.

[12]S. Halder, D. Agorastos, R. Veit, et al., “Neural mechanisms of brain-computer interface control,” Neuro Image, vol. 55,no. 4, pp. 1779-1790, 2011.

[13]M. A. Lebedev and M. A. L. Nicolelis, “Brain-machine interfaces: Past, present and future,” TRENDS　 in Neurosciences, vol. 29, no. 9, pp. 536-546, 2006.

[14]C. R. Gallistel, “The organization of learning,” Journal of Cognitive Neuroscience, vol. 3, no. 4, pp. 382-384, 1991.

[15]A. S. Etienne, R. Maurer, J. Berlie, et al., “Navigation through vector addition,” Nature, vol. 396, no. 6707, pp.161-164, 1998.

[16]A. S. Etienne, R. Maurer, and V. Seguinot, “Path integration in mammals and its interactions with visual landmarks,” The Journal of Experimental Biology, vol. 199, no. 1, pp. 201-209, 1996.

[17]S. J. Shettleworth, Cognition, Evolution, and Behavior, New York: Oxford University Press, 1998.

[18]E. Rama, G. Capi, S. Mizusaki, N. Tanaka, and S. Kawahara,“Effects of environment changes on rat’s learned behavior in an elevated Y-maze,” Journal　 of　 Medical　 and Bioengineering, vol. 5, no. 2, pp. 113-118, 2016.

[19]E. Rama, G. Capi, N. Tanaka, and S. Kawahara, “Rat’s spatial learning in an novel elevated Y-Maze,” in Proc. of the5th Annual Intl. Conf. on Cognitive Behavior Psychology,2016, pp. 140-145.

[20]F. Mondada, M. Bonani, X. Raemy, et al., “The e-puck, a robot designed for education in engineering,” in Proc. of the 9th Conf. on Autonomous Robot Systems and Competitions,2009, pp. 59-65.

[21]J. Hubert and Y. Weibel. (November 2008). ePic2.Documentation.　 [Online].　 Available: http://www.epuck.org/index.php?option=com_phocadownload&viewcate gory&id12:programs&Itemid38