The “TOP Factor” is a journal scoring system for characterising the extent to which a journal is compliant with the Transparency and Openness Promotion (TOP) Guidelines.
Created by the Center for Open Science, TOP is a set of eight standards that journals can implement to improve the potential replicability or reproducibility of the research they are publishing.
EBT has not yet been formally assessed for TOP Factor, but is aiming for a score of 17. This would put EBT in the top 1% of over 3000 journals with a TOP Factor ranking.
The table below shows the level at which EBT aims to comply with the TOP guidelines, and explains our implementation of each standard.
*Indicates a modified TOP implementation for consistency with journal goals.
| TOP Guideline | EBT Score | EBT Policy |
|---|---|---|
| 1. Citation Standards | 3 | All data, program code and other methods must be appropriately cited. Such materials must be recognized as original intellectual contributions and afforded recognition through citation. Articles will not be published until the citations conform to these standards. |
1. All data sets and program code used in a publication must be cited in the text and listed in the reference section.
2. References for data sets and program code should include a persistent identifier, such as a Digital Object Identifier (DOI). Persistent identifiers ensure future access to unique published digital objects, such as a text or data set. Persistent identifiers are assigned to data sets by digital archives, such as institutional repositories and partners in the Data Preservation Alliance for the Social Sciences (Data-PASS). | | 2. Data Transparency
Analytic Methods (Code) Transparency
Research Materials Transparency | 2 for each | The policy of EBT is to publish papers only if the data, methods used in the analysis, and materials used to conduct the research are clearly and precisely documented and are maximally available to any researcher for purposes of reproducing the results or replicating the procedure.
1. Authors reusing data available from public repositories must provide program code, scripts for statistical packages, and other documentation sufficient to allow an informed researcher to precisely reproduce all published results.
2. Authors using original data must:
make the data available at a trusted digital repository (Note: If all data required to reproduce the reported analyses appears in the article text, tables, and figures then it does not also need to be posted to a repository.)
include all variables, treatment conditions, and observations described in the manuscript
provide a full account of the procedures used to collect, preprocess, clean, or generate the data
provide program code, scripts, codebooks, and other documentation sufficient to precisely reproduce all published results
provide research materials and description of procedures necessary to conduct an independent replication of the research
3. In rare cases, despite authors’ best efforts, some or all data or materials cannot be shared for legal or ethical reasons. In such cases, authors must inform the editors at the time of submission. This will be taken into account during the review process. Authors are encouraged to anticipate data and material sharing at the beginning of their projects to provide for these circumstances. It is understood that in some cases access will be provided under restrictions to protect confidential or proprietary information. Editors may grant exceptions to data and material access requirements provided authors:
explain the restrictions on the dataset or materials and how they preclude public access.
provide a public description of the steps others should follow to request access to the data or materials.
provide software and other documentation that will precisely reproduce all published results.
provide access to all data and materials for which the constraints do not apply.
4. Data, program code, research materials, and other documentation of the research process should be made available through a trusted digital repository. Trusted repositories adhere to policies that make data discoverable, accessible, usable, and preserved for the long term. Trusted repositories also assign unique and persistent identifiers. For example these services are offered by partners in the Data Preservation Alliance for the Social Sciences (Data-PASS) and most institutional repositories. Author maintained websites are not compliant with this requirement.
Dissemination of these materials may be delayed until publication. Under exceptional circumstances, editors may grant an embargo of the public release of data for at most one year after publication.
Articles accepted for publication will not be assigned a publication date until the above conditions have been met. Authors are responsible for ensuring that their articles continue to meet these conditions. Failure to do so may lead to an editorial expression of concern or retraction of the article. | | 5. Design and Analysis Transparency | 3* | The policy of EBT is to publish papers where authors follow standards for disclosing key aspects of the research design and data analysis. Authors are required to select one or more reporting standards that reflect contemporary consensus on reporting requirements for their study design.* At manuscript submission, authors must confirm, via a completed reporting checklist, that they followed those standards in the manuscript. The journal, or an entity acting on behalf of the journal, will verify that the appropriate standards were adopted and followed. Failure to follow the relevant standards may result in the paper not being published.
*Note modification to original TOP requirement, that specifies review of the Equator Network standards database for each study. EBT instead only requires that any selected reporting standard be contemporary and considered appropriate by the Editors and Reviewers of the journal (for example, there is no requirement to review systematic review reporting standards for each submitted SR Working Paper; instead, general consensus that e.g. the PRISMA 2.0 statement is appropriate would be sufficient). Additionally to TOP, EBT requires submission of a completed reporting checklist to facilitate compliance checks. | | 6. Preregistration of Studies
Preregistration of studies involves registering the study design, variables, and exposure or intervention conditions. Including an analysis plan involves specification of the sequence of analyses and/or the statistical model that will be reported.
A link to the preregistration in an institutional registry must be made available to the journal prior at point of submission. The journal, or an entity acting on behalf of the journal, will verify that preregistration adheres to the specifications for preregistration of study methods and/or analysis plan, and then provide certification of the preregistration in the article.
1. Authors must, in the description of their methods, indicate if they did or did not preregister the research with or without an analysis plan in an independent, institutional registry.
2. If an author did preregister the research with an analysis plan, the author must:
Confirm in the text that the study was registered prior to conducting the research with links to the time-stamped preregistrations at the institutional registry, and that the preregistration adheres to the disclosure requirements of the institutional registry or those required for the preregistered badge with analysis plans maintained by the Center for Open Science.
Report all pre-registered analyses in the text, or, if there were changes in the analysis plan following preregistration, those changes must be disclosed with explanation for the changes.
Clearly distinguish in text analyses that were preregistered from those that were not, such as having separate sections in the results for confirmatory and exploratory analyses.
*Note modification to original TOP requirement: EBT requires link to planned methods to be provided at point of submission (not publication), so any planned methods can be compared to the methods eventually used, and the extent to which methods are preplanned can be understood and used to fully contextualise the findings of the submitted research. | | 8. Replication | 1 | The policy of EBT is to encourage submission of replication studies, particularly of research published in this journal. | | Total Score | 17 | Estimated score based on our evaluation of compliance |