sebastianruder--nlp-progress
Mehrdad Farahani
f3f86e3eec
* First version of summarization for persian * First version of summarization for persian docs * Fix misspell, add NER section, update ARMAN table * Update PEYMA table * Add NLI information for persian * Add new Vietnamese POS tagging results (#508) * Add new Vietnamese POS tagging results * Update vietnamese.md * Fix dataset title in semantic_parsing.md (#519) LDC2016E25 is the release limited to DEFT participants in 2016. LDC2017T10 is the general public release of this data and the name used in all recent papers. I believe the data is exactly the same (it definitely has the same number of train/test/dev samples). * Add New Result for Constituency Parsing on PTB (#516) Strongly Incremental Constituency Parsing with Graph Neural Networks Kaiyu Yang and Jia Deng Neural Information Processing Systems (NeurIPS) 2020 * add logical reasoning dataset ReClor (#515) * Update AMR parsing results (#514) I would like to "Create a new branch for this commit and start a pull request" * Added DRS parsing results for PMB 2.2.0 and 3.0.0 (#510) * add a new summarization result (#505) * add new update to Simplification (#504) * add new update to Simplification * Update english/simplification.md Co-authored-by: HaukurPall <haukurpalljonsson@gmail.com> Co-authored-by: Sebastian Ruder <sebastian@ruder.io> Co-authored-by: HaukurPall <haukurpalljonsson@gmail.com> * Add POS tagging result - social media (#503) * Add POS tagging result - social media * Apply suggestions from code review * Update english/part-of-speech_tagging.md Co-authored-by: Sebastian Ruder <sebastian@ruder.io> * Add a new task, NLI for Persian (#501) * Add a new task, NLI for Persian * Rename natural-language-inference to natural-language-inference.md Co-authored-by: Sebastian Ruder <sebastian@ruder.io> * Added joint entity and relation extraction task in relationship_extraction.md (#473) * Update relationship_extraction.md * Update relationship_extraction.md * Update relationship_extraction.md * Update semantic_parsing.md (#522) * Update semantic_parsing.md * Update semantic_parsing.md * Add dialogue implementations (#521) * Add SGNN implementation Add link to SGNN implementation * Add CASA implementation Add link to CASA implementation * Update word_sense_disambiguation.md (#517) * Update semantic_parsing.md (#518) Recently released AMR parser using a Seq-to-Seq model. Code can be found at... https://github.com/xdqkid/S2S-AMR-Parser Co-authored-by: Sebastian Ruder <ruder.sebastian@gmail.com> * First version of summarization for persian * First version of summarization for persian docs * Fix misspell, add NER section, update ARMAN table * Update PEYMA table * Add NLI information for persian * Rename nli, fix nli-results, fix typos in readme Co-authored-by: Huy Ngo <mayngulama1@gmail.com> Co-authored-by: Brad Jascob <bjascob@msn.com> Co-authored-by: Kaiyu Yang <kaiyuy@princeton.edu> Co-authored-by: Weihao Yu <1090924009@qq.com> Co-authored-by: Young-Suk Lee <ysuklee@us.ibm.com> Co-authored-by: Rik van Noord <31345314+RikVN@users.noreply.github.com> Co-authored-by: douglas01996 <437527550@qq.com> Co-authored-by: Billie <chuang211@163.com> Co-authored-by: Sebastian Ruder <sebastian@ruder.io> Co-authored-by: HaukurPall <haukurpalljonsson@gmail.com> Co-authored-by: Sara <59834747+sara-meftah@users.noreply.github.com> Co-authored-by: h-amirkhani <h.amirkhani.n@gmail.com> Co-authored-by: Tapas <tnk02.05@gmail.com> Co-authored-by: Andres Suarez <suarezandres@gmail.com> Co-authored-by: Gabor Berend <berendg@inf.u-szeged.hu> Co-authored-by: Sebastian Ruder <ruder.sebastian@gmail.com>
40 行
2.9 KiB
Markdown
40 行
2.9 KiB
Markdown
# Named entity recognition
|
|
|
|
Named entity recognition (NER) is the task of tagging entities in text with their corresponding type.
|
|
Approaches typically use BIO notation, which differentiates the beginning (B) and the inside (I) of entities.
|
|
O is used for non-entity tokens.
|
|
|
|
Example:
|
|
|
|
| Mark | Watney | visited | Mars |
|
|
| --- | ---| --- | --- |
|
|
| B-PER | I-PER | O | B-LOC |
|
|
|
|
### ArmanPersoNERCorpus
|
|
|
|
The [ArmanPersoNERCorpus](https://www.aclweb.org/anthology/C16-1319/) dataset contains 7,682 sentences with 250,015 tokens tagged in IOB format in six different classes, Organization, Person, Location, Facility, Event, and Product.
|
|
|
|
Download Links: [ARMAN](https://github.com/HaniehP/PersianNER/blob/master/ArmanPersoNERCorpus.zip)
|
|
|
|
| Model | F1 | Paper / Source | Code |
|
|
| ------------- | :-----:| --- | --- |
|
|
| ParsBERT (Farahani et al., 2020) | 99.84 | [ParsBERT: Transformer-based Model for Persian Language Understanding](https://arxiv.org/abs/2005.12515) | [Official](https://github.com/hooshvare/parsbert) |
|
|
| LSTM-CRF (Hafezi, Rezaeian, 2018) | 86.55 | [Neural Architecture for Persian Named Entity Recognition](https://ieeexplore.ieee.org/abstract/document/8700549) | - |
|
|
| mBERT (Taher et al., 2020) | 84.03 | [Beheshti-NER: Persian Named Entity Recognition Using BERT](https://arxiv.org/abs/2003.08875) | [Official](https://github.com/sEhsanTaher/Beheshti-NER) |
|
|
| Deep-CRF (Bokaei, Mahmoudi, 2018) | 81.50 | [Improved Deep Persian Named Entity Recognition](https://ieeexplore.ieee.org/abstract/document/8661067) | - |
|
|
| Deep-Local (Bokaei, Mahmoudi, 2018) | 79.19 | [Improved Deep Persian Named Entity Recognition](https://ieeexplore.ieee.org/abstract/document/8661067) | - |
|
|
| BiLSTM-CRF (Poostchi et al., 2018) | 77.45 | [BiLSTM-CRF for Persian Named-Entity Recognition](https://www.aclweb.org/anthology/L18-1701/) | - |
|
|
| SVM-HMM (Poostchi et al., 2016) | 72.59 | [PersoNER: Persian Named-Entity Recognition](https://www.aclweb.org/anthology/C16-1319/) | - |
|
|
|
|
### PEYMA
|
|
|
|
The [PEYMA](https://arxiv.org/abs/1801.09936) dataset includes 7,145 sentences with 302,530 tokens from which 41,148 tokens are tagged in IOB format in with seven different classes, Organization, Percent, Money, Location, Date, Time, and Person.
|
|
|
|
Download Links: [PEYMA](http://en.itrc.ac.ir/sites/default/files/pictures/NER.rar)
|
|
|
|
| Model | F1 | Paper / Source | Code |
|
|
| ------------- | :-----:| --- | --- |
|
|
| ParsBERT (Farahani et al., 2020) | 93.40 | [ParsBERT: Transformer-based Model for Persian Language Understanding](https://arxiv.org/abs/2005.12515) | [Official](https://github.com/hooshvare/parsbert) |
|
|
| mBERT (Taher et al., 2020) | 90.59 | [Beheshti-NER: Persian Named Entity Recognition Using BERT](https://arxiv.org/abs/2003.08875) | [Official](https://github.com/sEhsanTaher/Beheshti-NER) |
|
|
| Rule-Based-CRF (Shahshahani et al., 2018) | 84.00 | [PEYMA: A Tagged Corpus for Persian Named Entities](https://arxiv.org/abs/1801.09936) | - |
|