1. Author Response:

    Reviewer #1:

    Heinze and colleagues present InsectBrainDatabase (IBdb), a resource that collects, displays and shares mostly neuroanatomical data from several insect species. Users are able to search and visualise neuronal morphologies in several ways, brain regions and species' information. On the whole, the site is very well built, with a clear intent on providing a good user experience, both for experienced insect researchers and naïve users. The authors have designed the site in a way that it could be used as a general data hub, pre-publication, with a clear versioning system, and this will be appealing for some researchers. The manuscript describes the site accurately, focusing on how a user would interact with it.

    Although the authors intend for IBdb to become a major resource for their community, it is not made clear how they will increase data deposition beyond simplifying this process. With a distributed curation approach, and the expectation that researchers will be submitting their own data, it is also not clear how they will ensure consistency, completeness and accuracy of data curation, an essential aspect to broaden the usage of their platform, guarantee transparency to users, while, at the same time, enriching the amount of searchable data.

    There are three major incentives for data deposition that go beyond providing a simple and intuitive deposition procedure. First, our database serves not only as an interactive data repository, but as a data management tool at the same time. The possibility to securely store, manage and visualize unpublished data in an online platform is, according to our knowledge, unique and, once routinely used, will make data deposition a natural part of the general workflow in a research laboratory. With increasingly mandatory data management plans for researchers, we provide an intuitive, integrated solution for data management and deposition for all researchers in the field of insect neuroscience, a scope that to date is not met by any other repository. Second, the fully automatic access granted by our API for both upload and download of data enables users to exploit the database structure to easily integrate their own data (as well as accessible data of other researchers) into automatic data analysis workflows. This automatic interface also enables third party applications to access deposited data and to reuse it for meta-analysis, computational modeling, etc. Thus, deposited data increases the visibility of the data owner via increased chance of reuse. Finally, the visualization tools available for data (especially anatomical data) on the IBdb go beyond what is easily achievable by single users and common free software. Freely combining own (published and unpublished) neuronal morphologies with data from other researchers is currently time consuming and difficult at best. With the provided tools, we have created a tool that significantly aids state of the art data visualization without requiring any previous training.

    We have emphasized these aspects more in the abstract as well as the main paper.

    1. The authors use the term 'cell type' often in the manuscript, and a few times in the user guide. The concept of 'cell type' is an essential one for neuroscience researchers. However, I was unable to find any reference to it on the site, particularly in the information pages for neurons. This led to ambiguity about what data was displayed, what constitutes a cell type and how I would search for it. For example, a pair of neurons that look of the same type are am-AMMC-1 (NIN-0000159) and am-AMMC-1' (NIN-0000171), although they have distinct, yet related, full names. A similar case for am-AMMC-2' (NIN-0000172) and am-AMMC-2 (NIN-0000160) although it looks like this could be in fact the same cell - the neuron image caption of the former notes that it is a mirror image of the actual cell though this information is not given elsewhere. Lastly, the authors mention that they do not require neuron morphologies to be registered to a particular template. This point only makes it more essential for neuron types to be transparently labelled.

    The reviewer addresses a very important point and we have made several additions to the manuscript and the database to address them. We have added a cell type definition in the paper and at several locations on the database site, as well as in the user guide (“All neurons of one brain hemisphere that exhibit identical projection patterns are defined as belonging to the same cell type. In many cases a cell type will consist of one individual neuron, while in other cases, many identical neurons will comprise a cell type. In those cases, the first level of similarity beyond the individual neuron will be defined as the cell type.”). We have added a short section to the results, addressing this issue. We also added a new supplemental figure illustrating our cell type definitions with examples.

    Regarding the specific examples mentioned by the reviewer: The mirrored bee neurons were a leftover from early stages of the database before consistent standards were in place. These datasets have been archived (see novel function below), so that they are no longer searchable, but remain accessible for anyone with the exact handle.

    With respect to cell types, we essentially pursue a connectivity based cell type definition in the long run, i.e. neurons that have the same up- and downstream connections belong to the same cell type (as in Hulse et al., 2020). Yet, while connectivity data will increasingly become available, it does not exist for most species. Therefore, a first approximation is an identity defined by neural projection fields. As two neurons with identical projections can still have different connections, these projection based definitions will likely have to be subdivided into several cell types once connectomics data becomes available. We have additionally added a discussion point to the database forum to enable a public discussion about these definitions, their limits and practicalities.

    Several issues were considered in the process:

    First, neuron types on the right and left hemisphere. While we assume that the insect brain is largely symmetric with respect to the midline, asymmetries are present and one cannot simply assume that all neurons from one side of the brain are completely identical to the other side. To not obstruct discovering potential systematic differences between hemispheres (and for practical data handling issues), we define analogous neuron types from the right and left hemisphere as separate cell types. This is also in line with the fact that they originate from different neuroblasts, hence providing a larger degree of compatibility with lineage based cell type definitions.

    Second, modular brain regions. In regions like the central complex and the optic lobe, highly similar neurons occur in isomorphic sets of repeating individuals. In the case of the central complex columnar cells, we classify those as different cell types (according to column identity), as it has become clear recently that the CX columns are not identical copies of each other, and they additionally derive from different neuroblasts. For the thousands of repeating modules in the optic lobe, this approach is not practical, as it would require that each columnar neuron (e.g. of the medulla) has a firmly assigned ommatidium. Additionally, while not all optic lobe cartridges can be expected to be identical, there will be a limited number of types (e.g. corresponding to ommatidial types). If those can be reliably identified, cell types can be assigned based on this information. Practically, the columnar processing systems in the optic lobe will be collapsed to a single column in the database, or to as many required to capture the ommatidial diversity in a species.

    Third, definitions of cell types across species: We define cell types only within single species. This is because a neuron could have the same function, but different developmental origin, the same origin but different connections, etc. While neurons with identical projection patterns will emerge when comparing across species, the thereby suggested homology will remain hypothetical unless developmental studies are conducted on these cells. This in fact highlights a major value of the insect brain database, i.e. the possibility to identify shared features of neural circuits across species and the development of novel hypotheses based on this information.

    Highly organized neuropils: In neuropils like the mushroom body, antennal lobe, or the central complex, cell types can be embedded in a hierarchy, including supertypes, neuron families, and cell classes. As for species phylogeny, the same terms are likely not always representing the same level in the hierarchy (i.e. a supertype in one brain region might correspond to a class in another), but such terms nevertheless provide helpful guidance for classifying neurons in complex and organized brain regions. For example, in the CX, there are many types of columnar neurons, each defined by their specific projection patterns (e.g. all individuals of the PFNv-R3 type). Yet they can be grouped into several supertypes, defined by the projection patterns without referring to specific columns (e.g. PFNv), into neuron families, defined by the overall projection patterns (e.g. PFN), and into a neuron class (columnar CX neuron; as opposed to tangential neurons, or pontine neurons). We have added the possibility to add these higher level classifiers as tags to cell types, allowing more complex search queries and easier handling of increasing numbers of database entries.

    We hope that these additions clarify what we mean by the term cell type.

    1. I found the curation of data was often incomplete or inconsistent. This might be a consequence of the distributed strategy for curation together with the significant direct input from users. A few illustrative examples: a. Completeness: the curation of neuroanatomical information for neurons is often missing, although there is enough information to do this for 'soma location', 'fiber bundle', 'morphology description'.

    We intentionally kept the mandatory requirements for neuron deposition low, to not increase the burden for data deposition. How rich(how complete) a data set is depends on the user who deposits it. Our philosophy is to provide a framework for data deposition rather than mandating content. Nevertheless, we realize that rudimentary datasets are of limited use to the community and undermine the credibility of the database. We previously only required arborization regions, an image, and the schematic neuron path to be specified before approval (to make the data findable). We have added soma location and morphology description to the mandatory fields to be filled in before approval, so that each entry has to include a text that describes the uniqueness of the cell type. Additionally, and more importantly, we now automatically block approval requests if mandatory fields are not populated/valid. We have made substantial effort to update existing datasets to a comparable standard, and will continue to do so until this standard has been reached for each entry in the database.

    b. Consistency: (1) I found 3 different forms to designate AMMC, used in 'keyword'. Furthermore, I found functional terms used for 'keyword' ('mechanosensory'), not captured by a 'Modality' descriptor. (2) There is significant inconsistency also in the species's pages. Some have descriptions taken from the web, others cite academic literature while others are missing this section entirely. Often, no linked publications are available.

    We thank the reviewers for highlighting these inconsistencies. We have now merged similar keywords into single, consistent keywords. We additionally added a new interface to encourage consistent keyword use moving forward. We agree that functional terms must indeed be reflected in function entries. To avoid these issues in the future, we have developed a checklist for curators and data contributors (found under the help menu on the site) that should be used to ensure a correct submission and approval process. We have explicitly included the point referring to function entries in both the curator and contributor checklists, that need to be followed when submitting or approving a dataset. This issue should thus no longer emerge for future entries and was fixed for existing entries. The issue of misspelled keywords will never be fully avoidable (albeit will occur much more rarely with the new interface), but the database administrators are now aware of the issue and will regularly review and, if needed, clean up the list of used keywords.

    Regarding the species descriptions; we have left this to the species owner and believe that the mentioned variety is not necessarily bad. As the datasets can evolve, revised texts can be added by the species owner. In some cases, original placeholder text has been carried over from early stages of development. These texts have now been replaced by up to date information. We have additionally ensured that publication lists are available on all species pages.

    As a note, many of the mentioned inconsistencies had been carried over from very early stages of the database, before the currently used curation and approval methods were developed. Some of these issues had actually been the incentive to develop these standards. We have eliminated inconsistencies from old datasets as much as possible by both updating and archiving.

    In general, we would like to point out that inconsistencies between neurons of different species often reflect different traditions and conventions between research groups working on those species. We see it as one of the advantages of the multi-species approach of the IBdb that these differences/inconsistencies become obvious. While resolving these issues will often take time and effort (and will involve researchers from all groups involved), the main function of the IBdb in this context is not to impose a strict set of rules, but to provide the framework in which inconsistencies can be exposed and resolved.

    1. The authors present the platform as a data hub for not only neuroanatomical but also functional data. There is of course potential, but currently there is very little functional data on the site. Thus I find the authors' claim on the abstract "by intimately linking data on structure and function" unproven at this point.

    Indeed that is true. To address this issue and to validate that our database can indeed handle functional data beyond a few examples, we have added more data. We have started to move substantial amounts of available data from the locust, Megalopta, the Monarch butterfly, and the remaining species the co-authors are curating, into the database. While this is straightforward for new data, adding functional data from several years back to complement morphologies from publications covering the last two decades, will require some time. This is because raw data has to be located, verified, and uploaded, which becomes increasingly difficult the older the data are (both because of data formats and storage choices). Intriguingly, this highlights a main advantage of using the IBdb as a data management tool. When using the database for managing ongoing research, functional data can be uploaded alongside morphological data, before it is made public, thus keeping data permanently findable and reusable. Generating this possibility was one of the main incentives for creating the database (rather than generating large amounts of content), but we agree with the reviewers that we need to show more proof of concept data that this possibility is indeed functioning satisfactorily. We hope that this was achieved by our additions of both experiment entries (now over 100) and function entries for numerous neuron datasets.

    Reviewer #2:

    Heinze et al. created a public online database for morphology and function of neurons in many insect species (IBdb). This database is a platform for shared and searchable data repository. It can visualize the morphology of multiple identified neurons in the context of neuropils with options to control various visualization features (i.e. color, transparency, etc.). The uniqueness of IBdb is to index brains and neuron data of many insect species, allowing comparative approaches. The structure and functions of the IBdb are uncomplicated and intuitive.

    All these features were described clearly, and the paper can also serve as an instruction of the database. My comments and concerns are therefore mainly about usability and future development.

    1. This comparative database does not include some species, most critically Drosophila melanogaster. This exclusion is a drawback, as searching homologous neurons of the Drosophila neurons in other insects, or vice versa, would be inspiring and promote further comparative approach. As the same neuropil nomenclature was used in the largest and probably most elaborate database with similar functions for the Drosophila brain, VirtualFlyBrain, and IBdb, it would be helpful to implement cross-species neuron search based on arbor areas (as mentioned in Line 508).

    We view the current state of the database as a starting point. The lead author’s group alone has 14 more species located in the private section of the database and other groups are similarly preparing species for publication in the near future (see new supplemental figure for total number of current species). Importantly, the manuscript is intended not primarily to describe the content of the database, but its outline and functional principles, with data integrated into the database to illustrate its usefulness.

    The omission of Drosophila was decided after consulting with the virtual flybrain (VFB) platform hosts, to prevent that we duplicate their efforts and in turn potentially inspire VFB to duplicate the content of our database. However, we completely agree that cross links between the two databases are essential. Thus, we have now integrated Drosophila by cross linking our database to VFB as suggested by both reviewers. With the support and expertise of VFB staff we have mapped all brain regions in the IBDB to their corresponding neuropils in VFB, so that search results in IBDB (graphical search) have a direct corresponding search result in VFB. While searching in the IBdb, an API mediated query is sent to VFB and results are displayed in a newly introduced panel in the bottom half of the screen as a list of single neuron entries. Each list contains information obtained from VFB and is linked to the corresponding entry at VFB. This feature is available only for the graphic brain region search, but includes complex multi-neuropil searches as well. We have added the description of this function into the section of the paper that describes the link between VFB and IBdb.

    1. More comprehensive 'preset' depository of published data would make this database more attractive, as users naturally tend to first go to the largest and most comprehensive one. VFB also made a big success in this respect by actively indexing massive data taken in different labs.

    We agree and have added more content from our own research. We are currently making an effort to scan the literature and contact authors of older papers to encourage the deposition of their data (similar to what NeuroMorpho.org hosts are doing). This naturally takes some time, but we have now mentioned this effort in the manuscript. To facilitate this process and the visibility of neurons in the database that belong to specific publications, we have added a new list item (publication-based datasets for neurons and experiments), for which each entry contains a list of all items that can be linked to a specific publication (automatically generated), thus increasing visibility for the authors of these publications.

    Additionally, to attract large datasets, we have introduced a new type of experiment (interactive experiment) that is aimed at depositing connectomics data from EM studies and similar large datasets that are based on skeleton data combined with neuropil surfaces. The first two datasets already contain more than 1500 neurons (including partial neurons) from the bumblebee central complex. Complete cells from these datasets will be included as cell types in the main part of the database, once a bridging registration is in place.

    1. There is a concern on sustainability, as administration/management (e.g. species, curation, approval) continuously need expertise. It would be powerful to come up with a mechanism to encourage participation of more active users.

    Encourage participation: We are planning to actively advertise the database via presentations at (virtual and in person) conferences at the Arthropod Neuroscience Network and the International Society for Neuroethology, to encourage more users to contribute and identify users interested in active curation. As an incentive for curators, curation will increase visibility of the curator (as each curator is credited on each dataset they approved), which should be particularly attractive for early to mid-stage career researchers. To increase this visibility, we have created an explicit page that highlights the individual curators, their affiliation and their responsibilities. This feature will be expanded to full curator profiles in the near future.

    We see three points related to sustainability that are relevant for the IBdb long term perspective:

    Technical maintenance: This issue is essential to ensure persistence of deposited data. As discussed in the manuscript, we have planned the technical maintenance and the associated costs for the next 10 years and are confident that this plan is sustainable and suited to manage the site, keep existing data accessible and keep the code up to date regarding changing web- standards and data formats. In practical terms, this is facilitated by the fact that the lead author, as well as several co-authors, are using the site as primary data management and deposition tool for all ongoing and future research activities. The mandatory nature of data management and public deposition and the lack of suitable alternatives will ensure that third party funding dedicated to this purpose will continuously be available for technical maintenance, server fees etc.

    Oversight to ensure quality of content: This issue requires continuous expertise, relevant for ensuring that new datasets meet all required standards and that old datasets are kept up to date, if new information becomes available. We believe that, at the level of cell types and experiments, the strongest incentive for keeping data up to date lies with the data owners, who also possess the highest expertise for these data. Overall, updating data is optional, and even without updates, any data deposited will adhere to the standards initially applied upon approval. The species curators, in coordination with the scientific administrator, have an oversight function to reinforce these standards (see point 1 above). As curators actively perform research on the species they curate, there is substantial self-interest to maintain high standards to facilitate their research. On the level of species, the scientific administrator has the main responsibility. Given the limited number of species that will be included (realistically not more than several hundred for the near future), the associated workload is limited and manageable by a single person. Additionally, we are actively pursuing dedicated funding for one to two full time assistant positions for database curation. These positions will be devoted to performing routine maintenance, assisting new and existing users with data upload, identifying and resolving issues with existing datasets by regularly checking all entries, as well as actively attracting new users by identifying research papers with data appropriate for deposition in the IBdb. These assistants will also have the role to identify issues that regularly occur on the user end, with the aim of developing solutions to help streamlining and improving the site.

    Finally, continued relevance of the IBdb: The incentives for users to deposit data are given by the visualization possibilities and exposure of the data. Additionally, the possibility to programmatically access own and other researchers’ data via API allows to integrate this data into automatic data analysis workflows, allow effective data management and increases the chance of data reuse by third party applications (e.g. computational modelling apps). These provide a strong, bottom up mechanism to ensure that data volume will increase over time. With sufficient data deposited, new users will more easily be attracted, continuously increasing the relevance of the database. Users without associations with the authors have already begun to deposit data on the site, so we believe that the critical mass of data has been reached to keep the site relevant and in use long term. We will facilitate this development by actively approaching authors of publications that produce data suited for deposition in the IBdb. To illustrate the momentum of the database usage and the steady increase in registered users, we have added a supplemental figure displaying the change of usage over time, providing a basis for extrapolation.

    1. This database doesn't seem to require registration of neurons to a standard brain of the species (Line 502). It is unclear how one can make visualization as in Fig. 4 without registration. It would be helpful to detail what one can/cannot do depending on the data type.

    We have now emphasized in the results section (data contribution) that no shared 3D view can be carried out with neurons that are not registered into a common standard. While we encourage registration for more advanced visualizations, we have deliberately not made registration a requirement in order to allow easier deposition of older data and data from species without existing reference atlas. This limit is now also highlighted when uploading a neuron reconstruction that is not registered. This information was also highlighted in the user guide.

    Read the original source
    Was this evaluation helpful?
  2. Evaluation Summary:

    This manuscript will be of interest to insect neuroscientists and broadly to the neuroanatomy community. It presents a new web resource that collects and displays neuron, brain region and species data in user-friendly ways. If taken up by the community, it has the potential to become an important data hub.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 agreed to share their name with the authors.)

    Read the original source
    Was this evaluation helpful?
  3. Reviewer #1 (Public Review):

    Heinze and colleagues present InsectBrainDatabase (IBdb), a resource that collects, displays and shares mostly neuroanatomical data from several insect species. Users are able to search and visualise neuronal morphologies in several ways, brain regions and species' information. On the whole, the site is very well built, with a clear intent on providing a good user experience, both for experienced insect researchers and naïve users. The authors have designed the site in a way that it could be used as a general data hub, pre-publication, with a clear versioning system, and this will be appealing for some researchers. The manuscript describes the site accurately, focusing on how a user would interact with it.

    Although the authors intend for IBdb to become a major resource for their community, it is not made clear how they will increase data deposition beyond simplifying this process. With a distributed curation approach, and the expectation that researchers will be submitting their own data, it is also not clear how they will ensure consistency, completeness and accuracy of data curation, an essential aspect to broaden the usage of their platform, guarantee transparency to users, while, at the same time, enriching the amount of searchable data.

    1. The authors use the term 'cell type' often in the manuscript, and a few times in the user guide. The concept of 'cell type' is an essential one for neuroscience researchers. However, I was unable to find any reference to it on the site, particularly in the information pages for neurons. This led to ambiguity about what data was displayed, what constitutes a cell type and how I would search for it. For example, a pair of neurons that look of the same type are am-AMMC-1 (NIN-0000159) and am-AMMC-1' (NIN-0000171), although they have distinct, yet related, full names. A similar case for am-AMMC-2' (NIN-0000172) and am-AMMC-2 (NIN-0000160) although it looks like this could be in fact the same cell - the neuron image caption of the former notes that it is a mirror image of the actual cell though this information is not given elsewhere. Lastly, the authors mention that they do not require neuron morphologies to be registered to a particular template. This point only makes it more essential for neuron types to be transparently labelled.

    2. I found the curation of data was often incomplete or inconsistent. This might be a consequence of the distributed strategy for curation together with the significant direct input from users. A few illustrative examples:
      a. Completeness: the curation of neuroanatomical information for neurons is often missing, although there is enough information to do this for 'soma location', 'fiber bundle', 'morphology description'.
      b. Consistency: (1) I found 3 different forms to designate AMMC, used in 'keyword'. Furthermore, I found functional terms used for 'keyword' ('mechanosensory'), not captured by a 'Modality' descriptor. (2) There is significant inconsistency also in the species's pages. Some have descriptions taken from the web, others cite academic literature while others are missing this section entirely. Often, no linked publications are available.

    3. The authors present the platform as a data hub for not only neuroanatomical but also functional data. There is of course potential, but currently there is very little functional data on the site. Thus I find the authors' claim on the abstract "by intimately linking data on structure and function" unproven at this point.

    Read the original source
    Was this evaluation helpful?
  4. Reviewer #2 (Public Review):

    Heinze et al. created a public online database for morphology and function of neurons in many insect species (IBdb). This database is a platform for shared and searchable data repository. It can visualize the morphology of multiple identified neurons in the context of neuropils with options to control various visualization features (i.e. color, transparency, etc.). The uniqueness of IBdb is to index brains and neuron data of many insect species, allowing comparative approaches. The structure and functions of the IBdb are uncomplicated and intuitive.

    All these features were described clearly, and the paper can also serve as an instruction of the database. My comments and concerns are therefore mainly about usability and future development.

    1. This comparative database does not include some species, most critically Drosophila melanogaster. This exclusion is a drawback, as searching homologous neurons of the Drosophila neurons in other insects, or vice versa, would be inspiring and promote further comparative approach. As the same neuropil nomenclature was used in the largest and probably most elaborate database with similar functions for the Drosophila brain, VirtualFlyBrain, and IBdb, it would be helpful to implement cross-species neuron search based on arbor areas (as mentioned in Line 508).

    2. More comprehensive 'preset' depository of published data would make this database more attractive, as users naturally tend to first go to the largest and most comprehensive one. VFB also made a big success in this respect by actively indexing massive data taken in different labs.

    3. There is a concern on sustainability, as administration/management (e.g. species, curation, approval) continuously need expertise. It would be powerful to come up with a mechanism to encourage participation of more active users.

    4. This database doesn't seem to require registration of neurons to a standard brain of the species (Line 502). It is unclear how one can make visualization as in Fig. 4 without registration. It would be helpful to detail what one can/cannot do depending on the data type.

    Read the original source
    Was this evaluation helpful?
  5. Excerpt

    A new InsectBrainDatabase is now available at https://insectbraindb.org/app/ , report Heinze and colleagues, so check and admire 3D reconstructions and add you own!

    Read the original source
    Was this evaluation helpful?