:py:mod:`medcat.utils.regression.checking` ========================================== .. py:module:: medcat.utils.regression.checking Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: medcat.utils.regression.checking.RegressionCase medcat.utils.regression.checking.MetaData medcat.utils.regression.checking.RegressionChecker Functions ~~~~~~~~~ .. autoapisummary:: medcat.utils.regression.checking.get_ontology_and_version medcat.utils.regression.checking.fix_np_float64 Attributes ~~~~~~~~~~ .. autoapisummary:: medcat.utils.regression.checking.logger medcat.utils.regression.checking.UNKNOWN_METADATA .. py:data:: logger .. py:class:: RegressionCase Bases: :py:obj:`pydantic.BaseModel` A regression case that has a name, defines options, filters and phrases.s .. py:attribute:: name :type: str .. py:attribute:: options :type: medcat.utils.regression.targeting.FilterOptions .. py:attribute:: filters :type: List[medcat.utils.regression.targeting.TypedFilter] .. py:attribute:: phrases :type: List[str] .. py:attribute:: report :type: medcat.utils.regression.results.ResultDescriptor .. py:method:: get_all_targets(in_set, translation) Get all applicable targets for this regression case :param in_set: The input generator / iterator :type in_set: Iterator[Tuple[str, str]] :param translation: The translation layer :type translation: TranslationLayer :Yields: *Iterator[Tuple[str, str]]* -- The output generator .. py:method:: check_specific_for_phrase(cat, cui, name, phrase, translation) Checks whether the specific target along with the specified phrase is able to be identified using the specified model. :param cat: The model :type cat: CAT :param cui: The target CUI :type cui: str :param name: The target name :type name: str :param phrase: The phrase to check :type phrase: str :param translation: The translation layer :type translation: TranslationLayer :Returns: **bool** -- Whether or not the target was correctly identified .. py:method:: _get_all_cuis_names_types() .. py:method:: get_all_subcases(translation) Get all subcases for this case. That is, all combinations of targets with their appropriate phrases. :param translation: The translation layer :type translation: TranslationLayer :Yields: *Iterator[Tuple[str, str, str]]* -- The generator for the target info and the phrase .. py:method:: _get_specific_cui_and_name() .. py:method:: check_case(cat, translation) Check the regression case against a model. I.e check all its applicable targets. :param cat: The CAT instance :type cat: CAT :param translation: The translation layer :type translation: TranslationLayer :Returns: **Tuple[int, int]** -- Number of successes and number of failures .. py:method:: to_dict() Converts the RegressionCase to a dict for serialisation. :Returns: **dict** -- The dict representation .. py:method:: from_dict(name, in_dict) :classmethod: Construct the regression case from a dict. The expected stucture: { 'targeting': { 'strategy': 'ALL', # optional 'prefname-only': 'false', # optional 'filters': { : , # possibly multiple } }, 'phrases': ['phrase %s'] # possible multiple } Parsing the different parts of are delegated to other methods within the relevant classes. Delegators include: FilterOptions, TypedFilter :param name: The name of the case :type name: str :param in_dict: The dict describing the case :type in_dict: dict :raises ValueError: If the input dict does not have the 'targeting' section :raises ValueError: If the 'targeting' section does not have a 'filters' section :raises ValueError: If there are no phrases defined :Returns: **RegressionCase** -- The constructed regression case .. py:method:: __hash__() .. py:method:: __eq__(other) .. py:data:: UNKNOWN_METADATA :value: 'Unknown' .. py:function:: get_ontology_and_version(model_card) Attempt to get ontology (and its version) from a model card dict. If no ontology is found, 'Unknown' is returned. The version is always returned as the first source ontology. That is, unless the specified location does not exist in the model card, in which case 'Unknown' is returned. The ontology is assumed to be descibed at: model_card['Source Ontology'][0] (or model_card['Source Ontology'] if it's a string instead of a list) The ontology version is read from: model_card['Source Ontology'][0] (or model_card['Source Ontology'] if it's a string instead of a list) Currently, only SNOMED-CT, UMLS and ICD are supported / found. :param model_card: The input model card. :type model_card: dict :Returns: **Tuple[str, str]** -- The ontology (if found) or 'Unknown'; and the version (if found) or 'Unknown' .. py:class:: MetaData Bases: :py:obj:`pydantic.BaseModel` The metadat for the regression suite. This should define which ontology (e.g UMLS or SNOMED) as well as which version was used when generating the regression suite. The metadata may contain further information as well, this may include the annotator(s) involved when converting from MCT export or other relevant data. .. py:attribute:: ontology :type: str .. py:attribute:: ontology_version :type: str .. py:attribute:: extra :type: dict .. py:attribute:: regr_suite_creation_date :type: str .. py:method:: from_modelcard(model_card) :classmethod: Generate a MetaData object from a model card. This involves reading ontology info and version from the model card. It must be noted that the model card should be provided as a dict not a string. :param model_card: The CAT modelcard :type model_card: dict :Returns: **MetaData** -- The resulting MetaData .. py:method:: unknown() :classmethod: .. py:function:: fix_np_float64(d) Fix numpy.float64 in dictrionary for yaml saving purposes. These types of objects are unable to be cleanly serialized using yaml. So we need to conver them to the corresponding floats. The changes will be made within the dictionary itself as well as dictionaries within, recursively. :param d: The input dict :type d: dict .. py:class:: RegressionChecker(cases, metadata) The regression checker. This is used to check a bunch of regression cases at once against a model. :param cases: The list of regression cases :type cases: List[RegressionCase] :param metadata: The metadata for the regression suite :type metadata: MetaData :param use_report: Whether or not to use the report functionality (defaults to False) :type use_report: bool .. py:method:: __init__(cases, metadata) .. py:method:: get_all_subcases(translation) Get all subcases (i.e regssion case, target info and phrase) for this checker. :param translation: The translation layer :type translation: TranslationLayer :Yields: *Iterator[Tuple[RegressionCase, str, str, str]]* -- The generator for all the cases .. py:method:: check_model(cat, translation, total = None) Checks model and generates a report :param cat: The model to check against :type cat: CAT :param translation: The translation layer :type translation: TranslationLayer :param total: The total number of (sub)cases expected (for a progress bar) :type total: Optional[int] :Returns: **MultiDescriptor** -- A report description .. py:method:: __str__() Return str(self). .. py:method:: __repr__() Return repr(self). .. py:method:: to_dict() Converts the RegressionChecker to dict for serialisation. :Returns: **dict** -- The dict representation .. py:method:: to_yaml() Convert the RegressionChecker to YAML string. :Returns: **str** -- The YAML representation .. py:method:: __eq__(other) Return self==value. .. py:method:: from_dict(in_dict) :classmethod: Construct a RegressionChecker from a dict. Most of the parsing is handled in RegressionChecker.from_dict. This just assumes that each key in the dict is a name and each value describes a RegressionCase. :param in_dict: The input dict :type in_dict: dict :Returns: **RegressionChecker** -- The built regression checker .. py:method:: from_yaml(file_name) :classmethod: Constructs a RegressionChcker from a YAML file. The from_dict method is used for the construction from the dict. :param file_name: The file name :type file_name: str :Returns: **RegressionChecker** -- The constructed regression checker