Whether the weather will help us weather the COVID-19 pandemic: Using machine learning to measure twitter users’ perceptions
This article has been Reviewed by the following groups
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
- Evaluated articles (ScreenIT)
Abstract
Article activity feed
-
-
SciScore for 10.1101/2020.07.29.20164814: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
No key resources detected.
Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:This research is subject to limitations. As mentioned, the tri-class of “effect,” “no effect,” and “uncertain” problem proved too difficult for machine learning. Indeed, part of this arose from annotator difficulty in …
SciScore for 10.1101/2020.07.29.20164814: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
No key resources detected.
Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:This research is subject to limitations. As mentioned, the tri-class of “effect,” “no effect,” and “uncertain” problem proved too difficult for machine learning. Indeed, part of this arose from annotator difficulty in separating “no effect” and “uncertain” tweets. Several tweets were found to straddle the border of these two categories, partially due to the similarity of words across the “no effect” and “uncertain” tweets. This partly explains why collapsing these two categories into one improved our analysis performance enough to present results, and our adjusted Effect Classifier was able to successfully recognize users who claimed an effect. An additional limitation in the effect annotation scheme was that we did not label for the magnitude of the effect. With this, we lose the nuance of whether tweets are claiming a strong, impactful or weak, inconsequential effect of the weather. One solution to this is to annotate for ‘weak’ or ‘strong’ effect or assign a numerical score for the strength of effect; with more ample training data it is plausible a model may successfully learn which tweets claim a strong effect or otherwise. One significant language pattern that helped train our NLP analysis was the use of certain geographical locations to support a claim. For example, annotators noticed that warm locations, such as Florida and Singapore, were typically mentioned amongst users as a counterexample to undermine the possibility that warm weather will reduce the spread of COVI...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-
