【深層学習】BERT #まとめ編 #01

BERT まとめ編 yhayato1320.hatenablog.com

Index

学習するドメインに対する研究

SciBERT

• [2019 Allen Institute]
• Scientific BERT / 科学

yhayato1320.hatenablog.com

BlueBERT

• [2019]
• Blue (Biomedical Language Understanding Evaluation / 生物医学) BERT

yhayato1320.hatenablog.com

BioBERT

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
  • [2019]
  • arxiv.org

ClinicalBERT

• Publicly Available Clinical BERT Embeddings
  • [2019]
  • arxiv.org

LEGAL-BERT

• LEGAL-BERT: The Muppets straight out of Law School
  • [2020]
  • 法律
  • arxiv.org

PubMedBERT

• [2020 Microsoft Research]

yhayato1320.hatenablog.com

Umls BERT

• UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus
  • [2021]
  • arxiv.org
  • BioBERT, Clinical BERTの発展形?