This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Karcher Mean merge method.

Models Merged

The following models were included in the merge:

../Mira-v1.28-wave + ./Mira-v1.28-dpo-adapters/dpoq-1
../Mira-v1.28-wave
../Mira-v1.28-wave + ./Mira-v1.28-dpo-adapters/dpoq-2
../Mira-v1.28-wave + ./Mira-v1.28-dpo-adapters/dpoq-3

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: ../Mira-v1.28-wave+./Mira-v1.28-dpo-adapters/dpoq-1
  - model: ../Mira-v1.28-wave+./Mira-v1.28-dpo-adapters/dpoq-2
  - model: ../Mira-v1.28-wave+./Mira-v1.28-dpo-adapters/dpoq-3
  - model: ../Mira-v1.28-wave
merge_method: karcher
dtype: bfloat16
tokenizer_source: ../Mira-v1.28-wave
pad_to_multiple_of: 16