medcat.utils.meta_cat.ml_utils

Module Contents

Functions

set_all_seeds(seed)

create_batch_piped_data(data, start_ind, end_ind, ...)

Creates a batch given data and start/end that denote batch size, will also add

predict(model, data, config)

Predict on data used in the meta_cat.pipe

split_list_train_test(data, test_size[, shuffle])

Shuffle and randomply split data

print_report(epoch, running_loss, all_logits, y[, name])

Prints some basic stats during training

train_model(model, data, config[, save_dir_path])

Trains a LSTM model (for now) with autocheckpoints

eval_model(model, data, config, tokenizer)

Evaluate a trained model on the provided data

Attributes

logger

medcat.utils.meta_cat.ml_utils.logger
medcat.utils.meta_cat.ml_utils.set_all_seeds(seed)
Parameters:

seed (int) –

Return type:

None

medcat.utils.meta_cat.ml_utils.create_batch_piped_data(data, start_ind, end_ind, device, pad_id)

Creates a batch given data and start/end that denote batch size, will also add padding and move to the right device.

Parameters:

  • data (List[List[int], int, Optional[int]]) – Data in the format: [[<[input_ids]>, <cpos>, Optional[int]], …], the third column is optional and represents the output label

  • start_ind (int) – Start index of this batch

  • end_ind (int) – End index of this batch

  • device (torch.device) – Where to move the data

  • pad_id (int) – Padding index

Returns:

  • x () – Same as data, but subsetted and as a tensor

  • cpos () – Center positions for the data

Return type:

Tuple

medcat.utils.meta_cat.ml_utils.predict(model, data, config)

Predict on data used in the meta_cat.pipe

Parameters:

  • data (List[List[List[int], int]]) – Data in the format: [[<input_ids>, <cpos>], …]

  • config (medcat.config_meta_cat.ConfigMetaCAT) – Configuration for this meta_cat instance.

  • model (torch.nn.Module) –

Returns:

  • predictions (List[int]) – For each row of input data a prediction

  • confidence (List[float]) – For each prediction a confidence value

Return type:

Tuple

medcat.utils.meta_cat.ml_utils.split_list_train_test(data, test_size, shuffle=True)

Shuffle and randomply split data

Parameters:

  • data (List) –

  • test_size (int) –

  • shuffle (bool) –

Return type:

Tuple

medcat.utils.meta_cat.ml_utils.print_report(epoch, running_loss, all_logits, y, name='Train')

Prints some basic stats during training

Parameters:

  • epoch (int) –

  • running_loss (List) –

  • all_logits (List) –

  • y (Any) –

  • name (str) –

Return type:

None

medcat.utils.meta_cat.ml_utils.train_model(model, data, config, save_dir_path=None)

Trains a LSTM model (for now) with autocheckpoints

Parameters:
Return type:

Dict

medcat.utils.meta_cat.ml_utils.eval_model(model, data, config, tokenizer)

Evaluate a trained model on the provided data

Parameters:
Return type:

Dict