DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling
Paper • 2406.11617 • Published • 10
⚠️ Note: This model requires ChatML chat template. This version 0 is bugged and has early terminations, but is being released anyway for testing purposes.
della method. It was designed primarily for creativity and roleplay.
ChatML chat template to reduce (but not eliminate) the risk of this. A version 1 is being released once this is patched entirely, but it might require the removal of perhaps half of the donors.base_model: B:/12B/models--IntervitensInc--Mistral-Nemo-Base-2407-chatml
models:
# - model: B:/12B/models--aixonlab--Aether-12b
# - model: B:/12B/models--aixonlab--Zinakha-12b
# - model: B:/12B/models--allura-org--Bigger-Body-12b
# - model: B:/12B/models--allura-org--MN-12b-RP-Ink
# - model: B:/12B/models--allura-org--remnant-mn-12b
# - model: B:/12B/models--allura-org--Tlacuilo-12B
# - model: B:/12B/models--anthracite-org--magnum-v2-12b
# - model: B:/12B/models--anthracite-org--magnum-v2.5-12b-kto
# - model: B:/12B/models--anthracite-org--magnum-v4-12b
# - model: B:/12B/models--ArliAI--Mistral-Nemo-12B-ArliAI-RPMax-v1.2
# - model: B:/12B/models--axolotl-ai-co--romulus-mistral-nemo-12b-simpo
# - model: B:/12B/models--Babsie--Opulus-12B-v3
# - model: B:/12B/models--BeaverAI--mistral-doryV2-12b
# - model: B:/12B/models--cgato--Nemo-12b-Humanize-KTO-Experimental-Latest
# - model: B:/12B/models--cgato--Nemo-12b-Humanize-SFT-v0.2.5-KTO
# - model: B:/12B/models--ChaoticNeutrals--Mag-Mell-Reasoner-12B
# - model: B:/12B/models--crestf411--nemo-sunfall-v0.6.1
# - model: B:/12B/models--dphn--dolphin-2.9.3-mistral-nemo-12b
# - model: B:/12B/models--Elizezen--Himeyuri-v0.1-12B
# - model: B:/12B/models--Epiculous--Azure_Dusk-v0.2
# - model: B:/12B/models--Epiculous--Crimson_Dawn-v0.2
# - model: B:/12B/models--EpistemeAI2--Fireball-Mistral-Nemo-12B-Philos
# - model: B:/12B/models--EpistemeAI--Mistral-Nemo-Instruct-12B-Philosophy-Math
# - model: B:/12B/models--Fizzarolli--MN-12b-Rosier-v1
# - model: B:/12B/models--flammenai--Flammades-Mistral-Nemo-12B
# - model: B:/12B/models--flammenai--Mahou-1.3-mistral-nemo-12B
# - model: B:/12B/models--flammenai--Mahou-1.5-mistral-nemo-12B
# - model: B:/12B/models--GreenerPastures--Golden-Curry-12B
# - model: B:/12B/models--Gryphe--Pantheon-RP-1.5-12b-Nemo
# - model: B:/12B/models--Gryphe--Pantheon-RP-1.6.1-12b-Nemo
# - model: B:/12B/models--HumanLLMs--Human-Like-Mistral-Nemo-Instruct-2407
# - model: B:/12B/models--IIEleven11--Kalypso
# - model: B:/12B/models--intervitens--mini-magnum-12b-v1.1
# - model: B:/12B/models--IntervitensInc--Mistral-Nemo-Base-2407-chatml
# - model: B:/12B/models--jtatman--mistral_nemo_12b_reasoning_psychology_lora
# - model: B:/12B/models--KOOWEEYUS--BlackSheep-RP-12B
# - model: B:/12B/models--Lambent--Arsenic-Shahrazad-12B-v2
# - model: B:/12B/models--Lambent--Arsenic-Shahrazad-12B-v3
# - model: B:/12B/models--Lambent--arsenic-nemo-unleashed-12B
# - model: B:/12B/models--Lambent--Gilded-Arsenic-12B
# - model: B:/12B/models--LatitudeGames--Muse-12B
# - model: B:/12B/models--LatitudeGames--Wayfarer-12B
# - model: B:/12B/models--LatitudeGames--Wayfarer-2-12B
# - model: B:/12B/models--migtissera--Tess-3-Mistral-Nemo-12B
# # - model: B:/12B/models--mistralai--Mistral-Nemo-Instruct-2407
# - model: B:/12B/models--mpasila--Mistral-freeLiPPA-LoRA-12B
# - model: B:/12B/models--nbeerbower--Denker-mistral-nemo-12B
# - model: B:/12B/models--nbeerbower--Lyra-Gutenberg-mistral-nemo-12B
# - model: B:/12B/models--nbeerbower--Lyra4-Gutenberg-12B
# - model: B:/12B/models--nbeerbower--Merlina-ORPO-12B
# - model: B:/12B/models--nbeerbower--mistral-nemo-bophades-12B
# - model: B:/12B/models--nbeerbower--mistral-nemo-cc-12B
# - model: B:/12B/models--nbeerbower--mistral-nemo-gutenberg-12B-v2
# - model: B:/12B/models--nbeerbower--mistral-nemo-gutenberg-12B-v3
# - model: B:/12B/models--nbeerbower--mistral-nemo-gutenberg-12B-v4
# - model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Doppel-12B
# - model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Doppel-12B-v2
# - model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Encore-12B
# - model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Vitus-12B
# - model: B:/12B/models--nbeerbower--mistral-nemo-kartoffel-12B
# - model: B:/12B/models--nbeerbower--Mistral-Nemo-Prism-12B
# - model: B:/12B/models--nbeerbower--mistral-nemo-wissenschaft-12B
# - model: B:/12B/models--NeverSleepHistorical--lumi-nemo-e2.0
# - model: B:/12B/models--NeverSleep--Lumimaid-v0.2-12B
# - model: B:/12B/models--nothingiisreal--Celeste-12B-V1.6
# - model: B:/12B/models--nothingiisreal--MN-12B-Celeste-V1.9
# # - model: B:/12B/models--p-e-w--Mistral-Nemo-Instruct-2407-heretic-noslop
# - model: B:/12B/models--PocketDoc--Dans-DangerousWinds-V1.1.0-12b
# - model: B:/12B/models--PocketDoc--Dans-PersonalityEngine-V1.1.0-12b
# - model: B:/12B/models--PocketDoc--Dans-PersonalityEngine-V1.3.0-12b
# - model: B:/12B/models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
# - model: B:/12B/models--PygmalionAI--Eleusis-12B
# - model: B:/12B/models--PygmalionAI--Pygmalion-3-12B
# - model: B:/12B/models--rAIfle--Questionable-MN-bf16
# - model: B:/12B/models--ReadyArt--Dark-Nexus-12B-v2.0
# - model: B:/12B/models--ReadyArt--Forgotten-Safeword-12B-v4.0
# - model: B:/12B/models--ReadyArt--Omega-Darker_The-Final-Directive-12B
# - model: B:/12B/models--romaingrx--red-teamer-mistral-nemo
# - model: B:/12B/models--Sao10K--MN-12B-Lyra-v1
# - model: B:/12B/models--Sao10K--MN-12B-Lyra-v2a1
# - model: B:/12B/models--Sao10K--MN-12B-Lyra-v4
# - model: B:/12B/models--Sao10K--MN-BackyardAI-Party-12B-v1
# - model: B:/12B/models--schneewolflabs--A0l-12B
# - model: B:/12B/models--shisa-ai--shisa-v2-mistral-nemo-12b
# - model: B:/12B/models--shuttleai--shuttle-2.5-mini
# - model: B:/12B/models--SicariusSicariiStuff--Angelic_Eclipse_12B
# - model: B:/12B/models--SicariusSicariiStuff--Impish_Bloodmoon_12B
# - model: B:/12B/models--SicariusSicariiStuff--Impish_Longtail_12B
# - model: B:/12B/models--SicariusSicariiStuff--Impish_Nemo_12B
# - model: B:/12B/models--SicariusSicariiStuff--Sweet_Dreams_12B
# - model: B:/12B/models--sleepdeprived3--Christian-Bible-Expert-v2.0-12B
# - model: B:/12B/models--SuperbEmphasis--MN-12b-RP-Ink-RP-Longform
# - model: B:/12B/models--SuperbEmphasis--Omega-Darker_The-Final-Directive-Longform-Stage2-ERP-12B-v0.2
# - model: B:/12B/models--TheDrummer--Rivermind-12B-v1
# - model: B:/12B/models--TheDrummer--Rocinante-12B-v1
# - model: B:/12B/models--TheDrummer--Rocinante-X-12B-v1
# - model: B:/12B/models--Trappu--Nemo-Picaro-12B
# - model: B:/12B/models--Undi95--LocalC-12B-e2.0
# - model: B:/12B/models--UsernameJustAnother--Nemo-12B-Marlin-v5
# - model: B:/12B/models--UsernameJustAnother--Nemo-12B-Marlin-v6
# - model: B:/12B/models--UsernameJustAnother--Nemo-12B-Marlin-v8
# - model: B:/12B/models--VAGOsolutions--SauerkrautLM-Nemo-12b-Instruct
- model: B:/12B/models--aixonlab--Aether-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--aixonlab--Zinakha-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--allura-org--Bigger-Body-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--allura-org--MN-12b-RP-Ink
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--allura-org--remnant-mn-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--allura-org--Tlacuilo-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--anthracite-org--magnum-v2-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--anthracite-org--magnum-v2.5-12b-kto
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--anthracite-org--magnum-v4-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--ArliAI--Mistral-Nemo-12B-ArliAI-RPMax-v1.2
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--axolotl-ai-co--romulus-mistral-nemo-12b-simpo
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Babsie--Opulus-12B-v3
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--BeaverAI--mistral-doryV2-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--cgato--Nemo-12b-Humanize-KTO-Experimental-Latest
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--cgato--Nemo-12b-Humanize-SFT-v0.2.5-KTO
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--ChaoticNeutrals--Mag-Mell-Reasoner-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--crestf411--nemo-sunfall-v0.6.1
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--dphn--dolphin-2.9.3-mistral-nemo-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Elizezen--Himeyuri-v0.1-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Epiculous--Azure_Dusk-v0.2
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Epiculous--Crimson_Dawn-v0.2
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--EpistemeAI2--Fireball-Mistral-Nemo-12B-Philos
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--EpistemeAI--Mistral-Nemo-Instruct-12B-Philosophy-Math
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Fizzarolli--MN-12b-Rosier-v1
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--flammenai--Flammades-Mistral-Nemo-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--flammenai--Mahou-1.3-mistral-nemo-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--flammenai--Mahou-1.5-mistral-nemo-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--GreenerPastures--Golden-Curry-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Gryphe--Pantheon-RP-1.5-12b-Nemo
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Gryphe--Pantheon-RP-1.6.1-12b-Nemo
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--HumanLLMs--Human-Like-Mistral-Nemo-Instruct-2407
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--IIEleven11--Kalypso
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--intervitens--mini-magnum-12b-v1.1
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--IntervitensInc--Mistral-Nemo-Base-2407-chatml
- model: B:/12B/models--jtatman--mistral_nemo_12b_reasoning_psychology_lora
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--KOOWEEYUS--BlackSheep-RP-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Lambent--Arsenic-Shahrazad-12B-v2
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Lambent--Arsenic-Shahrazad-12B-v3
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Lambent--arsenic-nemo-unleashed-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Lambent--Gilded-Arsenic-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--LatitudeGames--Muse-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--LatitudeGames--Wayfarer-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--LatitudeGames--Wayfarer-2-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--migtissera--Tess-3-Mistral-Nemo-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
# - model: B:/12B/models--mistralai--Mistral-Nemo-Instruct-2407
- model: B:/12B/models--mpasila--Mistral-freeLiPPA-LoRA-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--Denker-mistral-nemo-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--Lyra-Gutenberg-mistral-nemo-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--Lyra4-Gutenberg-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--Merlina-ORPO-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--mistral-nemo-bophades-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--mistral-nemo-cc-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--mistral-nemo-gutenberg-12B-v2
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--mistral-nemo-gutenberg-12B-v3
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--mistral-nemo-gutenberg-12B-v4
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Doppel-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Doppel-12B-v2
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Encore-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Vitus-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--mistral-nemo-kartoffel-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--Mistral-Nemo-Prism-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nbeerbower--mistral-nemo-wissenschaft-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--NeverSleepHistorical--lumi-nemo-e2.0
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--NeverSleep--Lumimaid-v0.2-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nothingiisreal--Celeste-12B-V1.6
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--nothingiisreal--MN-12B-Celeste-V1.9
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
# - model: B:/12B/models--p-e-w--Mistral-Nemo-Instruct-2407-heretic-noslop
- model: B:/12B/models--PocketDoc--Dans-DangerousWinds-V1.1.0-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--PocketDoc--Dans-PersonalityEngine-V1.1.0-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--PocketDoc--Dans-PersonalityEngine-V1.3.0-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--PygmalionAI--Eleusis-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--PygmalionAI--Pygmalion-3-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--rAIfle--Questionable-MN-bf16
parameters:
density: 0.9
weight: 0.005 # -400%
epsilon: 0.099
- model: B:/12B/models--ReadyArt--Dark-Nexus-12B-v2.0
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--ReadyArt--Forgotten-Safeword-12B-v4.0
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--ReadyArt--Omega-Darker_The-Final-Directive-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--romaingrx--red-teamer-mistral-nemo
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Sao10K--MN-12B-Lyra-v1
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Sao10K--MN-12B-Lyra-v2a1
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Sao10K--MN-12B-Lyra-v4
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Sao10K--MN-BackyardAI-Party-12B-v1
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--schneewolflabs--A0l-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--shisa-ai--shisa-v2-mistral-nemo-12b
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--shuttleai--shuttle-2.5-mini
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--SicariusSicariiStuff--Angelic_Eclipse_12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--SicariusSicariiStuff--Impish_Bloodmoon_12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--SicariusSicariiStuff--Impish_Longtail_12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--SicariusSicariiStuff--Impish_Nemo_12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--SicariusSicariiStuff--Sweet_Dreams_12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--sleepdeprived3--Christian-Bible-Expert-v2.0-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--SuperbEmphasis--MN-12b-RP-Ink-RP-Longform
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--SuperbEmphasis--Omega-Darker_The-Final-Directive-Longform-Stage2-ERP-12B-v0.2
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--TheDrummer--Rivermind-12B-v1
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--TheDrummer--Rocinante-12B-v1
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--TheDrummer--Rocinante-X-12B-v1
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Trappu--Nemo-Picaro-12B
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--Undi95--LocalC-12B-e2.0
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--UsernameJustAnother--Nemo-12B-Marlin-v5
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--UsernameJustAnother--Nemo-12B-Marlin-v6
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--UsernameJustAnother--Nemo-12B-Marlin-v8
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
- model: B:/12B/models--VAGOsolutions--SauerkrautLM-Nemo-12b-Instruct
parameters:
density: 0.9
weight: 0.02
epsilon: 0.099
merge_method: della
parameters:
lambda: 1.0
normalize: false
int8_mask: false
dtype: float32
out_dtype: bfloat16
tokenizer:
source: "union"
tokens:
# Force ChatML EOS tokens
"<|im_start|>":
source: "B:/12B/models--SicariusSicariiStuff--Impish_Nemo_12B"
force: true
"<|im_end|>":
source: "B:/12B/models--SicariusSicariiStuff--Impish_Nemo_12B"
force: true
# Keep Mistral tokens
"[INST]":
source: "B:/12B/models--ArliAI--Mistral-Nemo-12B-ArliAI-RPMax-v1.2"
# source: "B:/12B/models--mistralai--Mistral-Nemo-Instruct-2407"
# The tokenizer system requires all models referenced in token configurations to be present in the merge's model list to build proper embedding permutations.
"[/INST]":
source: "B:/12B/models--ArliAI--Mistral-Nemo-12B-ArliAI-RPMax-v1.2"
# Force as fallback EOS
"":
source: "B:/12B/models--ArliAI--Mistral-Nemo-12B-ArliAI-RPMax-v1.2"
force: true
chat_template: "chatml"
name: 🦑 Kraken-12B-v0
# --copy-tokenizer --allow-crimes --out-shard-size 5B --trust-remote-code --lazy-unpickle --random-seed 420 --cuda --fix-mistral-regex
ID | Status | Delta Norm | Orig Size | Model Name
----------------------------------------------------------------------------------------------------
#1 | OK | 5.6771 | 74448896 | models--aixonlab--Aether-12b
#2 | OK | 5.8723 | 74448896 | models--aixonlab--Zinakha-12b
#3 | OK | 5.2058 | 74448896 | models--allura-org--Bigger-Body-12b
#4 | OK | 5.4643 | 74448896 | models--allura-org--MN-12b-RP-Ink
#5 | OK | 5.2060 | 74449920 | models--allura-org--remnant-mn-12b
#6 | OK | 1.9517 | 74449920 | models--allura-org--Tlacuilo-12B
#7 | OK | 1.0390 | 74448896 | models--anthracite-org--magnum-v2-12b
#8 | OK | 1.0390 | 74448896 | models--anthracite-org--magnum-v2.5-12b-kto
#9 | OK | 5.6574 | 74448896 | models--anthracite-org--magnum-v4-12b
#10 | OK | 5.2590 | 74448896 | models--ArliAI--Mistral-Nemo-12B-ArliAI-RPMax-v1.2
#11 | OK | 3.3112 | 74448896 | models--axolotl-ai-co--romulus-mistral-nemo-12b-simpo
#12 | OK | 6.6451 | 74448896 | models--Babsie--Opulus-12B-v3
#13 | OK | 0.1505 | 74448896 | models--BeaverAI--mistral-doryV2-12b
#14 | OK | 1.2048 | 74450944 | models--cgato--Nemo-12b-Humanize-KTO-Experimental-Latest
#15 | OK | 5.6528 | 74448896 | models--cgato--Nemo-12b-Humanize-SFT-v0.2.5-KTO
#16 | OK | 8.8050 | 74448896 | models--ChaoticNeutrals--Mag-Mell-Reasoner-12B
#17 | OK | 5.2159 | 74448896 | models--crestf411--nemo-sunfall-v0.6.1
#18 | OK | 1.8106 | 74449920 | models--dphn--dolphin-2.9.3-mistral-nemo-12b
#19 | OK | 1.0175 | 74448896 | models--Elizezen--Himeyuri-v0.1-12B
#20 | OK | 6.7114 | 74448896 | models--Epiculous--Azure_Dusk-v0.2
#21 | OK | 3.3770 | 74448896 | models--Epiculous--Crimson_Dawn-v0.2
#22 | OK | 2.3551 | 74448896 | models--EpistemeAI2--Fireball-Mistral-Nemo-12B-Philos
#23 | OK | 5.7042 | 74448896 | models--EpistemeAI--Mistral-Nemo-Instruct-12B-Philosophy-Math
#24 | OK | 0.1171 | 74448896 | models--Fizzarolli--MN-12b-Rosier-v1
#25 | OK | 3.3113 | 74448896 | models--flammenai--Flammades-Mistral-Nemo-12B
#26 | OK | 5.2062 | 74448896 | models--flammenai--Mahou-1.3-mistral-nemo-12B
#27 | OK | 3.3113 | 74448896 | models--flammenai--Mahou-1.5-mistral-nemo-12B
#28 | OK | 5.8125 | 74448896 | models--GreenerPastures--Golden-Curry-12B
#29 | OK | 4.9542 | 74448896 | models--Gryphe--Pantheon-RP-1.5-12b-Nemo
#30 | OK | 5.6554 | 74448896 | models--Gryphe--Pantheon-RP-1.6.1-12b-Nemo
#31 | OK | 5.2063 | 74448896 | models--HumanLLMs--Human-Like-Mistral-Nemo-Instruct-2407
#32 | OK | 5.7243 | 74448896 | models--IIEleven11--Kalypso
#33 | OK | 0.9360 | 74449408 | models--intervitens--mini-magnum-12b-v1.1
#34 | OK | 0.0000 | 74448896 | models--IntervitensInc--Mistral-Nemo-Base-2407-chatml
#35 | OK | 5.6078 | 74448896 | models--jtatman--mistral_nemo_12b_reasoning_psychology_lora
#36 | OK | 2.7004 | 74448896 | models--KOOWEEYUS--BlackSheep-RP-12B
#37 | OK | 3.0763 | 74448896 | models--Lambent--Arsenic-Shahrazad-12B-v2
#38 | OK | 3.0763 | 74448896 | models--Lambent--Arsenic-Shahrazad-12B-v3
#39 | OK | 3.0763 | 74448896 | models--Lambent--arsenic-nemo-unleashed-12B
#40 | OK | 3.0764 | 74448896 | models--Lambent--Gilded-Arsenic-12B
#41 | OK | 1.9517 | 74449920 | models--LatitudeGames--Muse-12B
#42 | OK | 3.9204 | 74449920 | models--LatitudeGames--Wayfarer-12B
#43 | OK | 2.3079 | 74449920 | models--LatitudeGames--Wayfarer-2-12B
#44 | OK | 0.5651 | 74450432 | models--migtissera--Tess-3-Mistral-Nemo-12B
#45 | OK | 5.3337 | 74448896 | models--mpasila--Mistral-freeLiPPA-LoRA-12B
#46 | OK | 3.3340 | 74448896 | models--nbeerbower--Denker-mistral-nemo-12B
#47 | OK | 2.8108 | 74448896 | models--nbeerbower--Lyra-Gutenberg-mistral-nemo-12B
#48 | OK | 5.2025 | 74448896 | models--nbeerbower--Lyra4-Gutenberg-12B
#49 | OK | 3.3363 | 74448896 | models--nbeerbower--Merlina-ORPO-12B
#50 | OK | 5.2063 | 74448896 | models--nbeerbower--mistral-nemo-bophades-12B
#51 | OK | 0.9367 | 74450432 | models--nbeerbower--mistral-nemo-cc-12B
#52 | OK | 0.9361 | 74449408 | models--nbeerbower--mistral-nemo-gutenberg-12B-v2
#53 | OK | 5.8718 | 74448896 | models--nbeerbower--mistral-nemo-gutenberg-12B-v3
#54 | OK | 5.8718 | 74448896 | models--nbeerbower--mistral-nemo-gutenberg-12B-v4
#55 | OK | 5.2061 | 74448896 | models--nbeerbower--Mistral-Nemo-Gutenberg-Doppel-12B
#56 | OK | 3.3112 | 74448896 | models--nbeerbower--Mistral-Nemo-Gutenberg-Doppel-12B-v2
#57 | OK | 5.2062 | 74448896 | models--nbeerbower--Mistral-Nemo-Gutenberg-Encore-12B
#58 | OK | 5.2063 | 74448896 | models--nbeerbower--Mistral-Nemo-Gutenberg-Vitus-12B
#59 | OK | 3.3338 | 74448896 | models--nbeerbower--mistral-nemo-kartoffel-12B
#60 | OK | 3.3338 | 74448896 | models--nbeerbower--Mistral-Nemo-Prism-12B
#61 | OK | 5.2062 | 74448896 | models--nbeerbower--mistral-nemo-wissenschaft-12B
#62 | OK | 0.9464 | 74449408 | models--NeverSleepHistorical--lumi-nemo-e2.0
#63 | OK | 0.9745 | 74449408 | models--NeverSleep--Lumimaid-v0.2-12B
#64 | OK | 5.2061 | 74448896 | models--nothingiisreal--Celeste-12B-V1.6
#65 | OK | 6.7418 | 74448896 | models--nothingiisreal--MN-12B-Celeste-V1.9
#66 | OK | 2.0064 | 74448896 | models--PocketDoc--Dans-DangerousWinds-V1.1.0-12b
#67 | OK | 2.2097 | 74448896 | models--PocketDoc--Dans-PersonalityEngine-V1.1.0-12b
#68 | OK | 6.3991 | 74448896 | models--PocketDoc--Dans-PersonalityEngine-V1.3.0-12b
#69 | OK | 2.8414 | 74448896 | models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
#70 | OK | 4.5633 | 74448896 | models--PygmalionAI--Eleusis-12B
#71 | HIGH MAG | 26.7912 | 74448896 | models--PygmalionAI--Pygmalion-3-12B
#72 | HIGH MAG | 12.9483 | 74448896 | models--rAIfle--Questionable-MN-bf16
#73 | OK | 6.0999 | 74448896 | models--ReadyArt--Dark-Nexus-12B-v2.0
#74 | OK | 6.0997 | 74448896 | models--ReadyArt--Forgotten-Safeword-12B-v4.0
#75 | OK | 5.2061 | 74448896 | models--ReadyArt--Omega-Darker_The-Final-Directive-12B
#76 | OK | 5.2061 | 74448896 | models--romaingrx--red-teamer-mistral-nemo
#77 | OK | 2.8108 | 74448896 | models--Sao10K--MN-12B-Lyra-v1
#78 | OK | 2.7925 | 74448896 | models--Sao10K--MN-12B-Lyra-v2a1
#79 | OK | 5.2024 | 74448896 | models--Sao10K--MN-12B-Lyra-v4
#80 | OK | 5.8457 | 74448896 | models--Sao10K--MN-BackyardAI-Party-12B-v1
#81 | OK | 3.3363 | 74448896 | models--schneewolflabs--A0l-12B
#82 | OK | 5.8740 | 74448896 | models--shisa-ai--shisa-v2-mistral-nemo-12b
#83 | OK | 1.4593 | 74449920 | models--shuttleai--shuttle-2.5-mini
#84 | HIGH MAG | 10.1439 | 74449920 | models--SicariusSicariiStuff--Angelic_Eclipse_12B
#85 | HIGH MAG | 10.1456 | 74449920 | models--SicariusSicariiStuff--Impish_Bloodmoon_12B
#86 | OK | 3.5326 | 74448896 | models--SicariusSicariiStuff--Impish_Longtail_12B
#87 | OK | 4.5888 | 74448896 | models--SicariusSicariiStuff--Impish_Nemo_12B
#88 | OK | 7.5975 | 74448896 | models--SicariusSicariiStuff--Sweet_Dreams_12B
#89 | OK | 5.2061 | 74448896 | models--sleepdeprived3--Christian-Bible-Expert-v2.0-12B
#90 | OK | 5.4655 | 74448896 | models--SuperbEmphasis--MN-12b-RP-Ink-RP-Longform
#91 | OK | 5.2071 | 74448896 | models--SuperbEmphasis--Omega-Darker_The-Final-Directive-Longform-Stage2-ERP-12B-v0.2
#92 | OK | 5.7491 | 74448896 | models--TheDrummer--Rivermind-12B-v1
#93 | OK | 5.8718 | 74448896 | models--TheDrummer--Rocinante-12B-v1
#94 | OK | 5.6795 | 74448896 | models--TheDrummer--Rocinante-X-12B-v1
#95 | OK | 3.6626 | 74448896 | models--Trappu--Nemo-Picaro-12B
#96 | OK | 1.8964 | 74448896 | models--Undi95--LocalC-12B-e2.0
#97 | OK | 5.4049 | 74448896 | models--UsernameJustAnother--Nemo-12B-Marlin-v5
#98 | OK | 5.7564 | 74448896 | models--UsernameJustAnother--Nemo-12B-Marlin-v6
#99 | OK | 1.5853 | 74448896 | models--UsernameJustAnother--Nemo-12B-Marlin-v8
#100 | OK | 5.4672 | 74448896 | models--VAGOsolutions--SauerkrautLM-Nemo-12b-Instruct
Status | Gen ID | Vocab ID | EOS Str | Model Name
----------------------------------------------------------------------------------------------------
FAIL | 2 | 2 | | models--aixonlab--Aether-12b
FAIL | 2 | 2 | | models--aixonlab--Zinakha-12b
FAIL | 2 | 2 | | models--allura-org--Bigger-Body-12b
FAIL | 2 | 2 | | models--allura-org--MN-12b-RP-Ink
FAIL | 2 | 2 | | models--allura-org--remnant-mn-12b
BROKEN | 131072 | MISSING | <|im_end|> | models--allura-org--Tlacuilo-12B
BROKEN | 2 | 15 | <|im_end|> | models--anthracite-org--magnum-v2-12b
BROKEN | 2 | 15 | <|im_end|> | models--anthracite-org--magnum-v2.5-12b-kto
FAIL | 2 | 2 | | models--anthracite-org--magnum-v4-12b
FAIL | 2 | 2 | | models--ArliAI--Mistral-Nemo-12B-ArliAI-RPMax-v1.2
FAIL | 4 | 4 | <|im_end|> | models--axolotl-ai-co--romulus-mistral-nemo-12b-simpo
FAIL | 2 | 2 | | models--Babsie--Opulus-12B-v3
FAIL | 2 | 2 | | models--BeaverAI--mistral-doryV2-12b
BROKEN | 2 | MISSING | <|im_end|> | models--cgato--Nemo-12b-Humanize-KTO-Experimental-Latest
BROKEN | 2 | 17 | <|im_end|> | models--cgato--Nemo-12b-Humanize-SFT-v0.2.5-KTO
FAIL | 15 | 15 | <|im_end|> | models--ChaoticNeutrals--Mag-Mell-Reasoner-12B
FAIL | 2 | 2 | | models--crestf411--nemo-sunfall-v0.6.1
BROKEN | 2 | MISSING | <|im_end|> | models--dphn--dolphin-2.9.3-mistral-nemo-12b
BROKEN | MISSING | 15 | <|im_end|> | models--Elizezen--Himeyuri-v0.1-12B
BROKEN | 2 | 15 | <|im_end|> | models--Epiculous--Azure_Dusk-v0.2
BROKEN | 2 | 15 | <|im_end|> | models--Epiculous--Crimson_Dawn-v0.2
FAIL | 2 | 2 | | models--EpistemeAI2--Fireball-Mistral-Nemo-12B-Philos
FAIL | 2 | 2 | | models--EpistemeAI--Mistral-Nemo-Instruct-12B-Philosophy-Math
FAIL | 2 | 2 | | models--Fizzarolli--MN-12b-Rosier-v1
FAIL | 4 | 4 | <|im_end|> | models--flammenai--Flammades-Mistral-Nemo-12B
FAIL | 2 | 2 | | models--flammenai--Mahou-1.3-mistral-nemo-12B
FAIL | 4 | 4 | <|im_end|> | models--flammenai--Mahou-1.5-mistral-nemo-12B
BROKEN | 2 | 15 | <|im_end|> | models--GreenerPastures--Golden-Curry-12B
FAIL | 128 | 128 | <|im_end|> | models--Gryphe--Pantheon-RP-1.5-12b-Nemo
FAIL | 128 | 128 | <|im_end|> | models--Gryphe--Pantheon-RP-1.6.1-12b-Nemo
FAIL | 2 | 2 | | models--HumanLLMs--Human-Like-Mistral-Nemo-Instruct-2407
FAIL | 2 | 2 | | models--IIEleven11--Kalypso
FAIL | 2 | 2 | | models--intervitens--mini-magnum-12b-v1.1
BROKEN | 2 | 15 | <|im_end|> | models--IntervitensInc--Mistral-Nemo-Base-2407-chatml
FAIL | 2 | 2 | | models--jtatman--mistral_nemo_12b_reasoning_psychology_lora
FAIL | 2 | 2 | | models--KOOWEEYUS--BlackSheep-RP-12B
BROKEN | MISSING | 2 | | models--Lambent--Arsenic-Shahrazad-12B-v2
FAIL | 2 | 2 | | models--Lambent--Arsenic-Shahrazad-12B-v3
FAIL | 2 | 2 | | models--Lambent--arsenic-nemo-unleashed-12B
FAIL | 2 | 2 | | models--Lambent--Gilded-Arsenic-12B
BROKEN | 131072 | MISSING | <|im_end|> | models--LatitudeGames--Muse-12B
BROKEN | 131072 | MISSING | <|im_end|> | models--LatitudeGames--Wayfarer-12B
BROKEN | 2 | MISSING | <|im_end|> | models--LatitudeGames--Wayfarer-2-12B
BROKEN | 2 | MISSING | <|im_end|> | models--migtissera--Tess-3-Mistral-Nemo-12B
FAIL | 2 | 2 | <|im_end|> | models--mpasila--Mistral-freeLiPPA-LoRA-12B
FAIL | 4 | 4 | <|im_end|> | models--nbeerbower--Denker-mistral-nemo-12B
FAIL | 2 | 2 | | models--nbeerbower--Lyra-Gutenberg-mistral-nemo-12B
FAIL | 2 | 2 | | models--nbeerbower--Lyra4-Gutenberg-12B
FAIL | 4 | 4 | <|im_end|> | models--nbeerbower--Merlina-ORPO-12B
FAIL | 2 | 2 | | models--nbeerbower--mistral-nemo-bophades-12B
BROKEN | 131074 | MISSING | <|im_end|> | models--nbeerbower--mistral-nemo-cc-12B
FAIL | 2 | 2 | | models--nbeerbower--mistral-nemo-gutenberg-12B-v2
FAIL | 2 | 2 | | models--nbeerbower--mistral-nemo-gutenberg-12B-v3
FAIL | 2 | 2 | | models--nbeerbower--mistral-nemo-gutenberg-12B-v4
FAIL | 2 | 2 | | models--nbeerbower--Mistral-Nemo-Gutenberg-Doppel-12B
FAIL | 4 | 4 | <|im_end|> | models--nbeerbower--Mistral-Nemo-Gutenberg-Doppel-12B-v2
FAIL | 2 | 2 | | models--nbeerbower--Mistral-Nemo-Gutenberg-Encore-12B
FAIL | 2 | 2 | | models--nbeerbower--Mistral-Nemo-Gutenberg-Vitus-12B
FAIL | 4 | 4 | <|im_end|> | models--nbeerbower--mistral-nemo-kartoffel-12B
FAIL | 4 | 4 | <|im_end|> | models--nbeerbower--Mistral-Nemo-Prism-12B
FAIL | 2 | 2 | | models--nbeerbower--mistral-nemo-wissenschaft-12B
FAIL | 2 | 2 | | models--NeverSleepHistorical--lumi-nemo-e2.0
FAIL | 2 | 2 | | models--NeverSleep--Lumimaid-v0.2-12B
FAIL | 2 | 2 | | models--nothingiisreal--Celeste-12B-V1.6
FAIL | 2 | 2 | | models--nothingiisreal--MN-12B-Celeste-V1.9
FAIL | 2 | 2 | | models--PocketDoc--Dans-DangerousWinds-V1.1.0-12b
FAIL | 2 | 2 | <|im_end|> | models--PocketDoc--Dans-PersonalityEngine-V1.1.0-12b
FAIL | 2 | 2 | <|endoftext|> | models--PocketDoc--Dans-PersonalityEngine-V1.3.0-12b
FAIL | 2 | 2 | <|im_end|> | models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
BROKEN | 2 | 15 | <|im_end|> | models--PygmalionAI--Eleusis-12B
BROKEN | 2 | 15 | <|im_end|> | models--PygmalionAI--Pygmalion-3-12B
FAIL | 2 | 2 | <|im_end|> | models--rAIfle--Questionable-MN-bf16
FAIL | 2 | 2 | | models--ReadyArt--Dark-Nexus-12B-v2.0
FAIL | 2 | 2 | | models--ReadyArt--Forgotten-Safeword-12B-v4.0
FAIL | 2 | 2 | | models--ReadyArt--Omega-Darker_The-Final-Directive-12B
FAIL | 2 | 2 | | models--romaingrx--red-teamer-mistral-nemo
BROKEN | MISSING | 2 | | models--Sao10K--MN-12B-Lyra-v1
BROKEN | MISSING | 15 | <|im_end|> | models--Sao10K--MN-12B-Lyra-v2a1
FAIL | 2 | 2 | | models--Sao10K--MN-12B-Lyra-v4
BROKEN | 15 | 4 | [/INST] | models--Sao10K--MN-BackyardAI-Party-12B-v1
FAIL | 4 | 4 | <|im_end|> | models--schneewolflabs--A0l-12B
FAIL | 2 | 2 | | models--shisa-ai--shisa-v2-mistral-nemo-12b
BROKEN | 2 | MISSING | <|im_end|> | models--shuttleai--shuttle-2.5-mini
BROKEN | 2 | MISSING | <|im_end|> | models--SicariusSicariiStuff--Angelic_Eclipse_12B
BROKEN | 2 | MISSING | <|im_end|> | models--SicariusSicariiStuff--Impish_Bloodmoon_12B
FAIL | 2 | 2 | <|im_end|> | models--SicariusSicariiStuff--Impish_Longtail_12B
FAIL | 2 | 2 | <|im_end|> | models--SicariusSicariiStuff--Impish_Nemo_12B
FAIL | 2 | 2 | <|im_end|> | models--SicariusSicariiStuff--Sweet_Dreams_12B
FAIL | 2 | 2 | | models--sleepdeprived3--Christian-Bible-Expert-v2.0-12B
FAIL | 2 | 2 | | models--SuperbEmphasis--MN-12b-RP-Ink-RP-Longform
FAIL | 2 | 2 | | models--SuperbEmphasis--Omega-Darker_The-Final-Directive-Longform-Stage2-ERP-12B-v0.2
FAIL | 2 | 2 | | models--TheDrummer--Rivermind-12B-v1
FAIL | 2 | 2 | | models--TheDrummer--Rocinante-12B-v1
FAIL | 2 | 2 | | models--TheDrummer--Rocinante-X-12B-v1
FAIL | 2 | 2 | | models--Trappu--Nemo-Picaro-12B
FAIL | 2 | 2 | | models--Undi95--LocalC-12B-e2.0
FAIL | 2 | 2 | <|im_end|> | models--UsernameJustAnother--Nemo-12B-Marlin-v5
FAIL | 2 | 2 | <|im_end|> | models--UsernameJustAnother--Nemo-12B-Marlin-v6
FAIL | 2 | 2 | <|im_end|> | models--UsernameJustAnother--Nemo-12B-Marlin-v8
BROKEN | MISSING | 2 | | models--VAGOsolutions--SauerkrautLM-Nemo-12b-Instruct