💬SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres

🍎 The project is an official implementation for SPEECH model and a repository for OntoEvent-Doc dataset, which has firstly been proposed in the paper 💬SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres accepted by ACL 2023 main conference.

🖥️ We also release the poster and slides for better understanding of this paper.

🤗 The implementations are based on Huggingface's Transformers and also referred to OntoED & DeepKE.

🤗 The baseline implementations are reproduced with codes referred to MAVEN's baselines or with official implementation.

SPEECH is proposed to address event-centric structured prediction with energy-based hyperspheres.
SPEECH models complex dependency among event structured components with energy-based modeling, and represents event classes with simple but effective hyperspheres.

The structure of data and code is as follows:

SPEECH ├── README.md ├── ACL2023@Poster_Speech.pdf ├── ACL2023@Slides_Speech.pdf ├── requirements.txt # for package requirements ├── data_utils.py # for data processing ├── speech.py # main model (bert serves as the backbone) ├── speech_distilbert.py# main model (distilbert serves as the backbone) ├── speech_roberta.py # toy model (roberta serves as the backbone, not adopted in the paper and just for reference) ├── run_speech.py# for model running ├── run_speech.sh# bash file for model running  └── Datasets # data ├── MAVEN_ERE │   ├── train.jsonl # for training │   ├── test.jsonl # for testing │   └── valid.jsonl # for validation ├── OntoEvent-Doc │ ├── event_dict_label_data.json # containing all event type labels  │   ├── event_dict_on_doc_train.json# for training │   ├── event_dict_on_doc_test.json# for testing │ └── event_dict_on_doc_valid.json# for validation └── README.md 

• python==3.9.12

• torch==1.13.0

• transformers==4.25.1

• scikit-learn==1.2.2

• torchmetrics==0.9.3

• sentencepiece==0.1.97

1. Project Preparation:

Download this project and unzip the dataset. You can directly download the archive, or run git clone https://github.com/zjunlp/SPEECH.git in your teminal.

cd [LOCAL_PROJECT_PATH] git clone git@github.com:zjunlp/SPEECH.git 

2. Data Preparation:

Unzip MAVEN_ERE and OntoEvent-Doc datasets stored at ./Datasets.

cd Datasets/ unzip MAVEN_ERE unzip OntoEvent-Doc cd .. 

3. Running Preparation:

Install all required packages.
Adjust the parameters in run_speech.sh bash file.

pip install -r requirements.txt vim run_speech.sh # input the parameters, save and quit 

Hint:

• Please refer to main() function in run_speech.py file for detail meanings of each parameters.
• Pay attention to --ere_task_type parameter candidates:
  • "doc_all" is for "All Joint" experiments in the paper
  • "doc_joint" is for each ERE subtask "+joint" experiments in the paper
  • "doc_temporal"/"doc_causal/"doc_sub" is for each ERE subtask experiments only
• Note that the loss ratio λ1, λ2, λ3, for trigger classification, event classification and event-relation extraction depends on different tasks, please ensure a correct setting of these ratios, referring to line 56-61 in speech.py and speech_distilbert.py file for details. We also present the loss ratio setting in Appendix B in our paper.

4. Running Model:

Run ./run_speech.sh for training, validation, and testing.

./run_speech.sh # Or you can run run_speech.py with manual parameter input in the terminal. python run_speech.py --para... 

Hint:

• A folder of model checkpoints will be saved at the path you input (--output_dir) in the bash file run_speech.sh or the command line in the terminal.
• We also release the checkpoints for direct testing (Dismiss --do_train in the parameter input)

We briefly introduce the datasets in Section 4.1 and Appendix A in our paper.

MAVEN_ERE is proposed in a paper and released in GitHub.

OntoEvent-Doc, formatted in document level, is derived from OntoEvent which is formatted in sentence level.

The statistics of MAVEN-ERE and OntoEvent-Doc are shown below, and the detailed data schema can be referred to [./Datasets/README.md].

The data schema of MAVEN-ERE can be referred to their GitHub. Experiments on MAVEN-ERE in our paper involve:

• 6 temporal relations: BEFORE, OVERLAP, CONTAINS, SIMULTANEOUS, BEGINS-ON, ENDS-ON
• 2 causal relations: CAUSE, PRECONDITION
• 1 subevent relation: subevent_relations

Experiments on OntoEvent-Doc in our paper involve:

• 3 temporal relations: BEFORE, AFTER, EQUAL
• 2 causal relations: CAUSE, CAUSEDBY

We also add a NA relation to signify no relation between the event mention pair for the two datasets.

🍒 The OntoEvent-Doc dataset is stored in json format. Each document (specialized with a doc_id, e.g., 95dd35ce7dd6d377c963447eef47c66c) in OntoEvent-Doc datasets contains a list of "events" and a dictionary of "relations", where the data format is as below:

[a doc_id]: { "events": [ { 'doc_id': '...', 'doc_title': 'XXX', 'sent_id': , 'event_mention': '......', 'event_mention_tokens': ['.', '.', '.', '.', '.', '.'], 'trigger': '...', 'trigger_pos': [, ], 'event_type': '' }, { 'doc_id': '...', 'doc_title': 'XXX', 'sent_id': , 'event_mention': '......', 'event_mention_tokens': ['.', '.', '.', '.', '.', '.'], 'trigger': '...', 'trigger_pos': [, ], 'event_type': '' }, ... ], "relations": { // each event-relation contains a list of 'sent_id' pairs. "COSUPER": [[,], [,], [,]], "SUBSUPER": [], "SUPERSUB": [], "CAUSE": [[,], [,]], "BEFORE": [[,], [,]], "AFTER": [[,], [,]], "CAUSEDBY": [[,], [,]], "EQUAL": [[,], [,]] } } 

📋 Thank you very much for your interest in our work. If you use or extend our work, please cite the following paper:

@inproceedings{ACL2023_SPEECH, author = {Shumin Deng and  Shengyu Mao and  Ningyu Zhang and  Bryan Hooi}, title = {SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres}, booktitle = {{ACL} {(1)}}, publisher = {Association for Computational Linguistics}, pages = {351--363}, year = {2023}, url = {https://aclanthology.org/2023.acl-long.21/} }