medcat.utils.relation_extraction.llama.model

Module Contents

Classes

LlamaModel_RelationExtraction

LlamaModel class for RelCAT

LlamaPooler

An attempt to copy the BERT pooling technique for an increase in performance.
class medcat.utils.relation_extraction.llama.model.LlamaModel_RelationExtraction(pretrained_model_name_or_path, relcat_config, model_config)

Bases: medcat.utils.relation_extraction.models.BaseModel_RelationExtraction

LlamaModel class for RelCAT

Parameters:
name = 'llamamodel_relcat'
log
__init__(pretrained_model_name_or_path, relcat_config, model_config)

Class to hold the Llama model + model_config

Parameters:

  • pretrained_model_name_or_path (str) – path to load the model from, this can be a HF model i.e: “bert-base-uncased”, if left empty, it is normally assumed that a model is loaded from ‘model.dat’ using the RelCAT.load() method. So if you are initializing/training a model from scratch be sure to base it on some model.

  • relcat_config (ConfigRelCAT) – relcat config.

  • model_config (LlamaConfig_RelationExtraction) – HF bert config for model.

output2logits(pooled_output, sequence_output, input_ids, e1_e2_start)
Parameters:

  • pooled_output (torch.Tensor) – embedding of the CLS token

  • sequence_output (torch.Tensor) – hidden states/embeddings for each token in the input text

  • input_ids (torch.Tensor) – input token ids.

  • e1_e2_start (torch.Tensor) – annotation tags token position

Returns:

torch.Tensor – classification probabilities for each token.

Return type:

torch.Tensor

forward(input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, head_mask=None, encoder_hidden_states=None, encoder_attention_mask=None, Q=None, e1_e2_start=None, pooled_output=None)

Forward pass for the model

Parameters:

  • input_ids (torch.Tensor) – input token ids. Defaults to None.

  • attention_mask (torch.Tensor) – attention mask for the input ids. Defaults to None.

  • token_type_ids (torch.Tensor) – token type ids for the input ids. Defaults to None.

  • position_ids (Any) – The position IDs. Defaults to None.

  • head_mask (Any) – The head mask. Defaults to None.

  • encoder_hidden_states (Any) – Encoder hidden states. Defaults to None.

  • encoder_attention_mask (Any) – Encoder attention mask. Defaults to None.

  • Q (Any) –

    1. Defaults to None.

  • e1_e2_start (Any) – start and end indices for the entities in the input ids. Defaults to None.

  • pooled_output (Any) – The pooled output. Defaults to None.

Returns:

Optional[Tuple[torch.Tensor, torch.Tensor]] – logits for the relation classification task.

Return type:

Tuple[torch.Tensor, torch.Tensor]

classmethod load(pretrained_model_name_or_path, relcat_config, model_config, **kwargs)

Load the model from the given path

Parameters:
Returns:

BaseModel_RelationExtraction – The loaded model.

Return type:

LlamaModel_RelationExtraction

class medcat.utils.relation_extraction.llama.model.LlamaPooler(hidden_size)

Bases: torch.nn.Module

An attempt to copy the BERT pooling technique for an increase in performance.

Parameters:

  • nn (nn.Module) –

    .

  • hidden_size (int) –

__init__(hidden_size)
Initialises the pooler with a linear layer of size:

self.model_config.hidden_size x self.model_config.hidden_size

Parameters:

hidden_size (int) – size of tensor

forward(hidden_states)
Parameters:

hidden_states (torch.Tensor) –

Return type:

torch.Tensor