← home
Github: datasets/highwire.py

ir_datasets: Highwire (TREC Genomics 2006-07)

Index
  1. highwire
  2. highwire/trec-genomics-2006
  3. highwire/trec-genomics-2007

"highwire"

Medical document collection from Highwire Press. Includes 162,259 scientific articles from 49 journals.

This dataset is used for the TREC 2006-07 TREC Genomics track.

Note that these documents are split into passages based on paragraph tags in the HTML.

docs
162K docs

Language: en

Document type:
HighwireDoc: (namedtuple)
  1. doc_id: str
  2. journal: str
  3. title: str
  4. spans: Tuple[
    HighwireSpan: (namedtuple)
    1. start: int
    2. length: int
    3. text: str
    , ...]

Examples:

Python API 
import ir_datasets dataset = ir_datasets.load("highwire") for doc in dataset.docs_iter(): doc # namedtuple<doc_id, journal, title, spans>

You can find more details about the Python API here.

CLI 
ir_datasets export highwire docs
[doc_id]    [journal]    [title]    [spans]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt pt.init() dataset = pt.get_dataset('irds:highwire') # Index highwire indexer = pt.IterDictIndexer('./indices/highwire') index_ref = indexer.index(dataset.get_corpus_iter(), fields=['journal', 'title'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset dataset = prepare_dataset('irds.highwire') for doc in dataset.iter_documents(): print(doc) # an AdhocDocumentStore break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

Metadata 
{ "docs": { "count": 162259, "fields": { "doc_id": { "max_len": 8, "common_prefix": "" } } } }

"highwire/trec-genomics-2006"

The TREC Genomics Track 2006 benchmark. Contains 28 queries with passage-level relevance judgments.

queries
28 queries

Language: en

Query type:
GenericQuery: (namedtuple)
  1. query_id: str
  2. text: str

Examples:

Python API 
import ir_datasets dataset = ir_datasets.load("highwire/trec-genomics-2006") for query in dataset.queries_iter(): query # namedtuple<query_id, text>

You can find more details about the Python API here.

CLI 
ir_datasets export highwire/trec-genomics-2006 queries
[query_id]    [text]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt pt.init() dataset = pt.get_dataset('irds:highwire/trec-genomics-2006') index_ref = pt.IndexRef.of('./indices/highwire') # assumes you have already built an index pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25') # (optionally other pipeline components) pipeline(dataset.get_topics())

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset topics = prepare_dataset('irds.highwire.trec-genomics-2006.queries') # AdhocTopics for topic in topics.iter(): print(topic) # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
162K docs

Inherits docs from highwire

Language: en

Document type:
HighwireDoc: (namedtuple)
  1. doc_id: str
  2. journal: str
  3. title: str
  4. spans: Tuple[
    HighwireSpan: (namedtuple)
    1. start: int
    2. length: int
    3. text: str
    , ...]

Examples:

Python API 
import ir_datasets dataset = ir_datasets.load("highwire/trec-genomics-2006") for doc in dataset.docs_iter(): doc # namedtuple<doc_id, journal, title, spans>

You can find more details about the Python API here.

CLI 
ir_datasets export highwire/trec-genomics-2006 docs
[doc_id]    [journal]    [title]    [spans]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt pt.init() dataset = pt.get_dataset('irds:highwire/trec-genomics-2006') # Index highwire indexer = pt.IterDictIndexer('./indices/highwire') index_ref = indexer.index(dataset.get_corpus_iter(), fields=['journal', 'title'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset dataset = prepare_dataset('irds.highwire.trec-genomics-2006') for doc in dataset.iter_documents(): print(doc) # an AdhocDocumentStore break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
28K qrels
Query relevance judgment type:
HighwireQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. start: int
  4. length: int
  5. relevance: int

Relevance levels

Rel.DefinitionCount%
0NOT25K89.1%
1POSSIBLY1.2K4.4%
2DEFINITELY1.8K6.5%

Examples:

Python API 
import ir_datasets dataset = ir_datasets.load("highwire/trec-genomics-2006") for qrel in dataset.qrels_iter(): qrel # namedtuple<query_id, doc_id, start, length, relevance>

You can find more details about the Python API here.

CLI 
ir_datasets export highwire/trec-genomics-2006 qrels --format tsv
[query_id]    [doc_id]    [start]    [length]    [relevance]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt from pyterrier.measures import * pt.init() dataset = pt.get_dataset('irds:highwire/trec-genomics-2006') index_ref = pt.IndexRef.of('./indices/highwire') # assumes you have already built an index pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25') # (optionally other pipeline components) pt.Experiment( [pipeline], dataset.get_topics(), dataset.get_qrels(), [MAP, nDCG@20] )

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset qrels = prepare_dataset('irds.highwire.trec-genomics-2006.qrels') # AdhocAssessments for topic_qrels in qrels.iter(): print(topic_qrels) # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Hersh2006TrecGenomics}

Bibtex:

@inproceedings{Hersh2006TrecGenomics, title={TREC 2006 Genomics Track Overview}, author={William Hersh and Aaron M. Cohen and Phoebe Roberts and Hari Krishna Rekapalli}, booktitle={TREC}, year={2006} }
Metadata 
{ "docs": { "count": 162259, "fields": { "doc_id": { "max_len": 8, "common_prefix": "" } } }, "queries": { "count": 28 }, "qrels": { "count": 27999, "fields": { "relevance": { "counts_by_value": { "0": 24934, "1": 1237, "2": 1828 } } } } }

"highwire/trec-genomics-2007"

The TREC Genomics Track 2007 benchmark. Contains 36 queries with passage-level relevance judgments.

queries
36 queries

Language: en

Query type:
GenericQuery: (namedtuple)
  1. query_id: str
  2. text: str

Examples:

Python API 
import ir_datasets dataset = ir_datasets.load("highwire/trec-genomics-2007") for query in dataset.queries_iter(): query # namedtuple<query_id, text>

You can find more details about the Python API here.

CLI 
ir_datasets export highwire/trec-genomics-2007 queries
[query_id]    [text]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt pt.init() dataset = pt.get_dataset('irds:highwire/trec-genomics-2007') index_ref = pt.IndexRef.of('./indices/highwire') # assumes you have already built an index pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25') # (optionally other pipeline components) pipeline(dataset.get_topics())

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset topics = prepare_dataset('irds.highwire.trec-genomics-2007.queries') # AdhocTopics for topic in topics.iter(): print(topic) # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
162K docs

Inherits docs from highwire

Language: en

Document type:
HighwireDoc: (namedtuple)
  1. doc_id: str
  2. journal: str
  3. title: str
  4. spans: Tuple[
    HighwireSpan: (namedtuple)
    1. start: int
    2. length: int
    3. text: str
    , ...]

Examples:

Python API 
import ir_datasets dataset = ir_datasets.load("highwire/trec-genomics-2007") for doc in dataset.docs_iter(): doc # namedtuple<doc_id, journal, title, spans>

You can find more details about the Python API here.

CLI 
ir_datasets export highwire/trec-genomics-2007 docs
[doc_id]    [journal]    [title]    [spans]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt pt.init() dataset = pt.get_dataset('irds:highwire/trec-genomics-2007') # Index highwire indexer = pt.IterDictIndexer('./indices/highwire') index_ref = indexer.index(dataset.get_corpus_iter(), fields=['journal', 'title'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset dataset = prepare_dataset('irds.highwire.trec-genomics-2007') for doc in dataset.iter_documents(): print(doc) # an AdhocDocumentStore break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
36K qrels
Query relevance judgment type:
HighwireQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. start: int
  4. length: int
  5. relevance: int

Relevance levels

Rel.DefinitionCount%
0NOT_RELEVANT32K87.5%
1RELEVANT4.5K12.5%

Examples:

Python API 
import ir_datasets dataset = ir_datasets.load("highwire/trec-genomics-2007") for qrel in dataset.qrels_iter(): qrel # namedtuple<query_id, doc_id, start, length, relevance>

You can find more details about the Python API here.

CLI 
ir_datasets export highwire/trec-genomics-2007 qrels --format tsv
[query_id]    [doc_id]    [start]    [length]    [relevance]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt from pyterrier.measures import * pt.init() dataset = pt.get_dataset('irds:highwire/trec-genomics-2007') index_ref = pt.IndexRef.of('./indices/highwire') # assumes you have already built an index pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25') # (optionally other pipeline components) pt.Experiment( [pipeline], dataset.get_topics(), dataset.get_qrels(), [MAP, nDCG@20] )

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset qrels = prepare_dataset('irds.highwire.trec-genomics-2007.qrels') # AdhocAssessments for topic_qrels in qrels.iter(): print(topic_qrels) # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Hersh2007TrecGenomics}

Bibtex:

@inproceedings{Hersh2007TrecGenomics, title={TREC 2007 Genomics Track Overview}, author={William Hersh and Aaron Cohen and Lynn Ruslen and Phoebe Roberts}, booktitle={TREC}, year={2007} }
Metadata 
{ "docs": { "count": 162259, "fields": { "doc_id": { "max_len": 8, "common_prefix": "" } } }, "queries": { "count": 36 }, "qrels": { "count": 35996, "fields": { "relevance": { "counts_by_value": { "0": 31501, "1": 4495 } } } } }