Multisensory-motor integration in olfactory navigation of silkmoth, Bombyx mori, using virtual reality system

  1. Mayu Yamada
  2. Hirono Ohashi
  3. Koh Hosoda
  4. Daisuke Kurabayashi
  5. Shunsuke Shigaki

  • Curated by eLife

    eLife logo

    Evaluation Summary:

    This paper uses a multi-model virtual reality system to assess which combinations of visual, wind, and olfactory information male silk moths rely on to find a female. The overall conclusion is that for the moths to search effectively, wind direction information is an important input. Vision, on the other hand, while it is used to control angular velocity, does not appear to be important for the moths to search effectively. This paper is of interest to neuroscientists and engineers interested in how multimodal sensory input controls navigational behavior. The experiments and modeling effort provide an advance in our understanding of how odor and wind information are combined in male silkmoths as they search for females.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. The reviewers remained anonymous to the authors.)

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Most animals survive and thrive due to navigational behavior to reach their destinations. In order to navigate, it is important for animals to integrate information obtained from multisensory inputs and use that information to modulate their behavior. In this study, by using a virtual reality (VR) system for an insect, we investigated how the adult silkmoth integrates visual and wind direction information during female search behavior (olfactory behavior). According to the behavioral experiments using a VR system, the silkmoth had the highest navigational success rate when odor, vision, and wind information were correctly provided. However, the success rate of the search was reduced if the wind direction information provided was different from the direction actually detected. This indicates that it is important to acquire not only odor information but also wind direction information correctly. When the wind is received from the same direction as the odor, the silkmoth takes positive behavior; if the odor is detected but the wind direction is not in the same direction as the odor, the silkmoth behaves more carefully. This corresponds to a modulation of behavior according to the degree of complexity (turbulence) of the environment. We mathematically modeled the modulation of behavior using multisensory information and evaluated it using simulations. The mathematical model not only succeeded in reproducing the actual silkmoth search behavior but also improved the search success relative to the conventional odor-source search algorithm.

Article activity feed

  1. Version published to 10.7554/elife.72001 on eLife
  2. eLife

    Author Response:

    Reviewer #1 (Public Review):

    This paper seeks to address how animals use sensory feedback from multiple sensory modalities during the navigation of complex environments. In particular, the authors are concerned with the use of odor-based signals for search behaviors, as odors are sparse and do not provide reliable directional information. An additional goal of this paper is to model the general principles they extract from their experiments so that they can be applied to engineered systems. Through the use of a virtual reality (VR) system that combines visual, wind, and odor input in a controlled manner, the authors investigate the search behavior of the silkworm moth, Bombyx mori. The females of the species emit a pheromone that triggers both a search behavior as well as mating behavior in males. Whereas previous work has taken advantage of the highly stereotyped nature of this behavior, it is unclear how these animals integrate other sensory cues such as wind and vision to locate an odor source. By presenting these stimuli in different combinations where the odor is always present, the authors find that the presence of a wind source strongly influences the animal's behavior-specifically its ability to locate the target-although the direction from which it is presented is crucial when all three are together. Multimodal input also affects the moths' walking speed and turning behavior, and an updated model that incorporates wind direction outperforms other models of this behavior.

    The conclusions of the paper are generally supported by the presented data, although there are some issues with the framing of the paper and the model that I feel should be addressed.

    1. The introduction and discussion go to great lengths to emphasize the technical advance of the constructed VR apparatus. Although it is certainly an impressive achievement, the use of VR for exploring insect behaviors like search and navigation is hardly a new development at this point in time. Indeed, work on flies, Drosophila and otherwise, going back decades has used VR to extract principles regarding issues such as visually mediated flight control or walking. More pertinent to the ideas of multimodal sensory integration explored here, over the past decade numerous researchers have combined visual input with odor and other cues to discern the relative importance of each of these modalities during search behavior. For example, see Duistermars and Frye, 2008. Generally, I feel that the paper overemphasizes the technical advance without providing sufficient biological context. So much work has been done on Bombyx that a paper using these methods has the ability to address, but much of that literature is absent from the paper. I think focusing more on the behavior will broaden the appeal of the paper by putting it in conversation with a well-established phenomenon

    Thank you for providing these insights. As you pointed out, we improved the quality and discussion of the paper by adding literature to the introduction because there were few descriptions of the multisensory integration of other insects. Regarding VR research, the recent definition of VR is that "virtual reality creates a physical and a mental space for people." In other words, we need to connect a device that can provide multisensory stimuli to organisms (physical space) and a mental space where avatars reflect their actions. Based on this definition, we claim that most biological experiments with insects have been conducted in physical space (just utilizing multisensory stimulators) and have not measured the purposeful behaviors that result from connection to the mental space. Behavioral experiments using a multi-sensory stimulator have revealed how each modality is utilized, but it has not been directly investigated how other modalities affect actual navigation predominantly utilizing an odor, such as in the current study. In fact, the previously proposed insect-inspired algorithm for navigation was modeled based on experimental data using a multisensory stimulus device, but as a result of simulation, it behaves differently to the actual insect. On the other hand, the model derived from the experimental results using the VR system can reproduce the behavior of the insect (silkmoth). In other words, we claim that in order to model the superior functions of an organism, it is necessary to do more than provide multisensory stimuli, and it is important to obtain the relationship between multisensory stimuli and behavioral output when the organism performs the actual purposeful behavior (e.g., navigation behavior). However, as you pointed out, we necessarily revised the manuscript including those comments because it is a fact that wonderful biological findings have been clarified by multisensory stimulation experiments.

    1. I think that the model is well-done and fits with the goals of the paper. Asking about the role of wind direction in this behavior is an important step given the behavioral data presented. However, I am not convinced based on the data that the new model developed by the authors is much better than the surge-zigzag model. The success rates are slightly different (is the statistical difference a function of the number of runs or timesteps?) and both models search about the same amount of time before finding the source. Finally, the migration probability maps are rather similar, so it is hard for me to conclude that factoring in wind direction is necessary to get good performance out of the model.

    Thank you for your valuable comments. Because a surge-zigzagging algorithm utilizes only odor stimuli to determine actions, an agent moves to the edge where the change in odor information occurs most. The behavior of MiM2 is modulated by information on wind direction and it moves into the areas with a high probability of odor reach. Hence, we thought that the effectiveness of the model could be shown by starting the search for the agent near the edge. Consequently, we set the initial position (x, y) = (300 ± 100) in the area where the odor is hard to reach and carried out the simulation. As a result, the MiM2 algorithm, which actively moves in the area where the probability of odor reach is high, was significantly better than other conventional algorithms. As for the search time, because MiM2 is based on the surge-zigzagging, the movement speeds were almost the same; therefore, both algorithms (MiM2 and surge-zigzagging) require almost the same search time. However, we found that there is a difference in the search success rate because there is a difference in the selected route.

    Revised part: Lines 282–311 (red letters) at “Modeling and validation of behavioral modulation mechanisms”.

    Reviewer #2 (Public Review):

    This paper uses a multi-model virtual reality system to assess which combinations of visual, wind, and olfactory information male silk moths rely on to find a female. The overall conclusion is that for the moths to search effectively, wind direction information is an important input. Vision, on the other hand, while it is used to control angular velocity, does not appear to be important for the moths to search effectively. Given what is known about other walking and flying insects these results are not surprising. Although the virtual reality system is advertised as being able to provide naturalistic and complex stimuli, the visual stimuli are limited to a traditional LED array, and the olfactory stimulus is created by projecting a 3-dimensional plume into two dimensions. The analyses of the data are rather simplistic and do not provide a mechanistic description of what the moths are actually doing on a moment-by-moment basis. The authors then proceed to construct a model for search behavior that uses frequency and relative timing information for the odor stimulus in conjunction with the wind direction information. This search algorithm is ever so slightly better than the prior surge-zigzagging algorithm. The role of relative timing information in the olfactory signal supplied to the moths in the virtual reality experiments, however, was not investigated.

    Thank you for your valuable comments. We can respond to the following points:

    Effects of vision: Our biological experiments demonstrated that the visual stimulus maintains the balance of the angular velocities of the left and right rotational movements. When the effect of this balance on the function of an odor search was confirmed by simulation, the path became longer due to the bias in the search trajectory, and the search time became significantly longer. In addition, the success rate decreased as in the biological experiments. This suggests that an efficient odor-source search can be performed by maintaining the balance between the left and right angular velocities.

    Limitation of visual stimulus: The main purpose of our study was the integration of other sensory information during navigation using odor. Therefore, the problem becomes complicated when cognition, i.e., visual object identification, is included. For that reason, we adopted optical flow as a visual stimulus, which does not involve cognition but affects behavior. Because there were several studies using LED arrays to provide optical flow to an insect with compound eyes, we conducted experiments using LED arrays this time as well. We had already tested whether the optical flow by the LED array could be provided correctly (see Supplementary Material).

    The plume model: We utilized plume diffusion, which was observed in two dimensions as a virtual odor field. We captured the movement of particles on a two-dimensional plane using particle image velocimetry technology. Please refer to the Supplementary Material for details.

    The behavior analysis: By plotting each trajectory in the behavior analysis and calculating a histogram of the change in heading angle, we investigated what kind of feature difference occurs during the odor search depending on the conditions of sensory stimulation. Regarding the odor, which is a cue/stimulus, we analyzed the behavioral change with respect to the "frequency", instead of in response to a single stimulus. The reason for this is that recent studies have shown that while searching the environment (1) the frequency of the odor changes with distance from the odor source, and (2) the behavior of Drosophila in walking also changes with frequency. By analyzing the behavioral changes with respect to the odor detection frequency, we modeled the behavior during the search based on the odor frequency information. As a result, we found that not only the behavior close to the movement of a silkmoth could be reproduced compared to the conventional model, but the search success rate could also be increased.

    Simulation: As you pointed out, we increased the conditions for verification of simulation because the simulation conditions were limited. As a result, we found that the proposed model has a higher odor tracking performance than the other conventional models even if the initial position is different. However, as you pointed out, this is a relatively simple condition of simulation; therefore, in the future, we plan to implement the algorithm in an autonomous robot system and verify the performance of the algorithm in more complicated spaces, such as obstacle environments or an outdoor environment.

  3. eLife

    Evaluation Summary:

    This paper uses a multi-model virtual reality system to assess which combinations of visual, wind, and olfactory information male silk moths rely on to find a female. The overall conclusion is that for the moths to search effectively, wind direction information is an important input. Vision, on the other hand, while it is used to control angular velocity, does not appear to be important for the moths to search effectively. This paper is of interest to neuroscientists and engineers interested in how multimodal sensory input controls navigational behavior. The experiments and modeling effort provide an advance in our understanding of how odor and wind information are combined in male silkmoths as they search for females.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. The reviewers remained anonymous to the authors.)

  4. eLife

    Reviewer #1 (Public Review):

    This paper seeks to address how animals use sensory feedback from multiple sensory modalities during the navigation of complex environments. In particular, the authors are concerned with the use of odor-based signals for search behaviors, as odors are sparse and do not provide reliable directional information. An additional goal of this paper is to model the general principles they extract from their experiments so that they can be applied to engineered systems. Through the use of a virtual reality (VR) system that combines visual, wind, and odor input in a controlled manner, the authors investigate the search behavior of the silkworm moth, Bombyx mori. The females of the species emit a pheromone that triggers both a search behavior as well as mating behavior in males. Whereas previous work has taken advantage of the highly stereotyped nature of this behavior, it is unclear how these animals integrate other sensory cues such as wind and vision to locate an odor source. By presenting these stimuli in different combinations where the odor is always present, the authors find that the presence of a wind source strongly influences the animal's behavior-specifically its ability to locate the target-although the direction from which it is presented is crucial when all three are together. Multimodal input also affects the moths' walking speed and turning behavior, and an updated model that incorporates wind direction outperforms other models of this behavior.

    The conclusions of the paper are generally supported by the presented data, although there are some issues with the framing of the paper and the model that I feel should be addressed.

    1. The introduction and discussion go to great lengths to emphasize the technical advance of the constructed VR apparatus. Although it is certainly an impressive achievement, the use of VR for exploring insect behaviors like search and navigation is hardly a new development at this point in time. Indeed, work on flies, Drosophila and otherwise, going back decades has used VR to extract principles regarding issues such as visually mediated flight control or walking. More pertinent to the ideas of multimodal sensory integration explored here, over the past decade numerous researchers have combined visual input with odor and other cues to discern the relative importance of each of these modalities during search behavior. For example, see Duistermars and Frye, 2008. Generally, I feel that the paper overemphasizes the technical advance without providing sufficient biological context. So much work has been done on Bombyx that a paper using these methods has the ability to address, but much of that literature is absent from the paper. I think focusing more on the behavior will broaden the appeal of the paper by putting it in conversation with a well-established phenomenon

    2. I think that the model is well-done and fits with the goals of the paper. Asking about the role of wind direction in this behavior is an important step given the behavioral data presented. However, I am not convinced based on the data that the new model developed by the authors is much better than the surge-zigzag model. The success rates are slightly different (is the statistical difference a function of the number of runs or timesteps?) and both models search about the same amount of time before finding the source. Finally, the migration probability maps are rather similar, so it is hard for me to conclude that factoring in wind direction is necessary to get good performance out of the model.

  5. eLife

    Reviewer #2 (Public Review):

    This paper uses a multi-model virtual reality system to assess which combinations of visual, wind, and olfactory information male silk moths rely on to find a female. The overall conclusion is that for the moths to search effectively, wind direction information is an important input. Vision, on the other hand, while it is used to control angular velocity, does not appear to be important for the moths to search effectively. Given what is known about other walking and flying insects these results are not surprising. Although the virtual reality system is advertised as being able to provide naturalistic and complex stimuli, the visual stimuli are limited to a traditional LED array, and the olfactory stimulus is created by projecting a 3-dimensional plume into two dimensions. The analyses of the data are rather simplistic and do not provide a mechanistic description of what the moths are actually doing on a moment-by-moment basis. The authors then proceed to construct a model for search behavior that uses frequency and relative timing information for the odor stimulus in conjunction with the wind direction information. This search algorithm is ever so slightly better than the prior surge-zigzagging algorithm. The role of relative timing information in the olfactory signal supplied to the moths in the virtual reality experiments, however, was not investigated.

  6. Version published to 10.1101/2021.09.14.460318v1 on bioRxiv