Skip to content

MedCAT Documentation

mctokenizers

Initializing search

Home
Architecture
Migration guide (v2)
Breaking changes
API Reference

MedCAT Documentation

Home
Architecture
Migration guide (v2)
Breaking changes
API Reference
API Reference
- medcat
  medcat
  - cat
  - cdb
    
    cdb
    
    cdb
    
    concepts
  - components
    
    components
    
    addons
    
    addons
    
    addons
    
    meta_cat
    
    meta_cat
    
    data_utils
    
    mctokenizers
    
    mctokenizers
    
    bert_tokenizer
    
    bpe_tokenizer
    
    tokenizers
    
    meta_cat
    
    ml_utils
    
    models
    
    relation_extraction
    
    relation_extraction
    
    base_component
    
    bert
    
    bert
    
    config
    
    model
    
    tokenizer
    
    config
    
    llama
    
    llama
    
    config
    
    model
    
    tokenizer
    
    ml_utils
    
    models
    
    modernbert
    
    modernbert
    
    config
    
    model
    
    tokenizer
    
    pad_seq
    
    rel_cat
    
    rel_dataset
    
    tokenizer
    
    linking
    
    linking
    
    context_based_linker
    
    no_action_linker
    
    only_primary_name_linker
    
    two_step_context_based_linker
    
    vector_context_model
    
    ner
    
    ner
    
    dict_based_ner
    
    trf
    
    trf
    
    deid
    
    helpers
    
    model
    
    tokenizer
    
    transformers_ner
    
    vocab_based_annotator
    
    vocab_based_ner
    
    normalizing
    
    normalizing
    
    normalizer
    
    tagging
    
    tagging
    
    tagger
    
    types
  - config
    
    config
    
    config
    
    config_meta_cat
    
    config_rel_cat
    
    config_transformers_ner
  - data
    
    data
    
    entities
    
    mctexport
    
    model_card
  - deid
  - model_creation
    
    model_creation
    
    cdb_maker
    
    preprocess_snomed
    
    preprocess_umls
  - pipeline
    
    pipeline
    
    pipeline
  - plugins
    
    plugins
    
    catalog
    
    cli
    
    data
    
    downloadable
    
    installer
    
    loader
    
    registry
  - preprocessors
    
    preprocessors
    
    cleaners
  - stats
    
    stats
    
    kfold
    
    stats
  - storage
    
    storage
    
    jsonserialiser
    
    mp_ents_save
    
    schema
    
    serialisables
    
    serialisers
    
    zip_utils
  - tokenizing
    
    tokenizing
    
    regex_impl
    
    regex_impl
    
    tokenizer
    
    spacy_impl
    
    spacy_impl
    
    tokenizers
    
    tokens
    
    utils
    
    tokenizers
    
    tokens
  - trainer
  - utils
    
    utils
    
    cdb_state
    
    cdb_utils
    
    check_for_updates
    
    config_utils
    
    data_utils
    
    defaults
    
    download_scripts
    
    envsnapshot
    
    exceptions
    
    fileutils
    
    filters
    
    hasher
    
    import_utils
    
    iterutils
    
    legacy
    
    legacy
    
    conversion_all
    
    convert_cdb
    
    convert_config
    
    convert_deid
    
    convert_meta_cat
    
    convert_rel_cat
    
    convert_vocab
    
    fixes
    
    helpers
    
    identifier
    
    legacy_converter
    
    v2_beta
    
    matutils
    
    ner
    
    ner
    
    data_collator
    
    metrics
    
    transformers_ner
    
    postprocessing
    
    registry
    
    regression
    
    regression
    
    checking
    
    regression_checker
    
    results
    
    targeting
    
    utils
    
    usage_monitoring
    
    vocab_utils
  - version
  - vocab

medcat.components.addons.meta_cat.mctokenizers

Modules:

bert_tokenizer –
bpe_tokenizer –
tokenizers –

Made with Material for MkDocs