BjarneNPO/finetune_21_08_2025_11_16_02

SentenceTransformer based on Snowflake/snowflake-arctic-embed-m-v2.0

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m-v2.0 on the train dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-m-v2.0
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • train

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'GteModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("BjarneNPO/finetune_21_08_2025_11_16_02")
# Run inference
queries = [
    "fragt wie der Stand zu dem aktuellen Problem ist",
]
documents = [
    'In Klärung mit der Kollegin - Das Problem liegt leider an deren Betreiber. Die sind aber informiert und arbeiten bereits daran',
    'findet diese in der Übersicht der Gruppen.',
    'Userin muss sich an die Bistums IT wenden.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.2668, 0.0872, 0.0995]])

Evaluation

Metrics

Information Retrieval

• Dataset: Snowflake/snowflake-arctic-embed-m-v2.0
• Evaluated with scripts.InformationRetrievalEvaluatorCustom.InformationRetrievalEvaluatorCustom with these parameters:

  {
      "query_prompt_name": "query",
      "corpus_prompt_name": "query"
  }
  

Metric	Value
cosine_accuracy@1	0.3277
cosine_accuracy@3	0.4874
cosine_accuracy@5	0.5462
cosine_accuracy@10	0.6471
cosine_precision@1	0.3277
cosine_precision@3	0.2185
cosine_precision@5	0.2303
cosine_precision@10	0.2143
cosine_recall@1	0.006
cosine_recall@3	0.0227
cosine_recall@5	0.0348
cosine_recall@10	0.0741
cosine_ndcg@10	0.2372
cosine_mrr@10	0.4209
cosine_map@100	0.0889

Training Details

Training Dataset

train

• Dataset: train
• Size: 19,964 training samples
• Columns: query and answer
• Approximate statistics based on the first 1000 samples:

  	query	answer
  type	string	string
  details	min: 4 tokens mean: 27.77 tokens max: 615 tokens	min: 3 tokens mean: 22.87 tokens max: 151 tokens

• Samples:

  query	answer
  Wie kann man die Jahresurlaubsübersicht exportieren?	über das 3 Punkte Menü rechts oben. Mitarbeiter auswählen und exportieren
  1. Vertragsabschlüsse werden nicht übertragen
  2. Kinder kommen nicht von nach
  3. Absage kann bei Portalstatus nicht erstellt werden.	Ticket
  Userin gebeten sich an den Support zu wenden, da der Fehler liegt.
  Wird im Anmeldeportal nicht gefunden.	Die Schnittstelle war noch nicht aktiviert und Profil ebenfalls nicht.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 8
• learning_rate: 2e-05
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: False
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 8
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Snowflake/snowflake-arctic-embed-m-v2.0_cosine_ndcg@10
0.0641	10	2.1446	-
0.1282	20	2.1454	-
0.1923	30	1.8973	-
0.2564	40	1.7238	-
0.3205	50	1.7305	-
0.3846	60	1.5496	-
0.4487	70	1.4557	-
0.5128	80	1.406	-
0.5769	90	1.3067	-
0.6410	100	1.2727	-
0.7051	110	1.215	-
0.7692	120	1.1902	-
0.8333	130	1.218	-
0.8974	140	1.1271	-
0.9615	150	1.0909	-
1.0	156	-	0.2589
1.0256	160	1.0408	-
1.0897	170	1.0112	-
1.1538	180	1.0683	-
1.2179	190	0.9405	-
1.2821	200	0.933	-
1.3462	210	0.9533	-
1.4103	220	0.9144	-
1.4744	230	0.8618	-
1.5385	240	0.8624	-
1.6026	250	0.8649	-
1.6667	260	0.8646	-
1.7308	270	0.8307	-
1.7949	280	0.8522	-
1.8590	290	0.8566	-
1.9231	300	0.8389	-
1.9872	310	0.806	-
2.0	312	-	0.2421
2.0513	320	0.7134	-
2.1154	330	0.7545	-
2.1795	340	0.8033	-
2.2436	350	0.7402	-
2.3077	360	0.7876	-
2.3718	370	0.7185	-
2.4359	380	0.7391	-
2.5	390	0.7362	-
2.5641	400	0.7276	-
2.6282	410	0.6953	-
2.6923	420	0.6853	-
2.7564	430	0.7337	-
2.8205	440	0.7704	-
2.8846	450	0.6922	-
2.9487	460	0.7153	-
3.0	468	-	0.2372

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.11
• Sentence Transformers: 5.1.0
• Transformers: 4.55.2
• PyTorch: 2.8.0+cu129
• Accelerate: 1.10.0
• Datasets: 3.6.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}