medcat.utils.relation_extraction.modernbert.tokenizer

Module Contents

Classes

TokenizerWrapperModernBERT_RelationExtraction

Wrapper around a huggingface ModernBERT tokenizer so that it works with the

Attributes

logger

medcat.utils.relation_extraction.modernbert.tokenizer.logger
class medcat.utils.relation_extraction.modernbert.tokenizer.TokenizerWrapperModernBERT_RelationExtraction(hf_tokenizers=None, max_seq_length=None, add_special_tokens=False)

Bases: medcat.utils.relation_extraction.tokenizer.BaseTokenizerWrapper_RelationExtraction

Wrapper around a huggingface ModernBERT tokenizer so that it works with the RelCAT models.

Parameters:

  • hf_tokenizers (transformers.PreTrainedTokenizerFast) – A huggingface Fast tokenizer.

  • max_seq_length (Optional[int]) –

  • add_special_tokens (Optional[bool]) –

name = 'tokenizer_wrapper_modern_bert_rel'
pretrained_model_name_or_path = 'answerdotai/ModernBERT-base'
classmethod load(tokenizer_path, relcat_config, **kwargs)
Parameters:
Return type:

TokenizerWrapperModernBERT_RelationExtraction