Commit Graph

17 Commits

Author SHA1 Message Date
Bruce MacDonald 5d3da85a16 remove out of date comments 2025-05-12 13:51:04 -07:00
Bruce MacDonald 684f0d9291 set default values for vision model in config 2025-05-12 13:51:04 -07:00
Bruce MacDonald 661bf04696 add picture prefix 2025-05-12 13:49:44 -07:00
Bruce MacDonald 2521a55ae6 fixes after rebase 2025-05-12 13:49:44 -07:00
Bruce MacDonald 32948ec952 increase rope base 2025-05-12 13:49:43 -07:00
Bruce MacDonald 16b13e0cfc Revert "ropeTheta should be 1e5"
This reverts commit cc1638b26763eae7daddd44e3975a885671ef9d3.

This reverts commit
b32385591307e2d33a8f43ce1626b529d2dac83e.
2025-05-12 13:49:43 -07:00
Bruce MacDonald 75441c56f3 add comment explaining rope theta 2025-05-12 13:49:43 -07:00
Bruce MacDonald 45f96e898d ropeTheta should be 1e5 2025-05-12 13:49:43 -07:00
Michael Yang f1257a7de4 update vision rope theta default 2025-05-12 13:49:43 -07:00
Bruce MacDonald ca981c8a49 full attn block indexes should be []int32 2025-05-12 13:49:43 -07:00
Bruce MacDonald 359e1d5b19 full attention layers 2025-05-12 13:49:42 -07:00
Bruce MacDonald eed0ac2948 clean up vision model forward pass 2025-05-12 13:49:42 -07:00
Michael Yang fb3c16f2a2 window index 2025-05-12 13:49:42 -07:00
Michael Yang 7e920c8d75 fix: patch merger and convert
convert:
- split patch embedding
- split qkv

remove duplicate PatchMerger
2025-05-12 13:49:42 -07:00
Bruce MacDonald 5ff0d538b0 wip: implementing rope 2025-05-12 13:49:41 -07:00
Bruce MacDonald 963531215e update convert 2025-05-12 13:49:41 -07:00
Bruce MacDonald c1f9bcb4dd restructure
image processing

Update model.go

Update model.go

Update model.go

no projector

no projector

vision model scaffold

...

...

wip

...

rebase

fix patch merger

tidy

...

Update model_vision.go

server: do not attempt to parse offset file as gguf

This logic was causing issues for me when importing a gguf that had some padding at the end of the file. The valid gguf would be read, but then it would try to read the offset as a different gguf file. This does not seem right.

Update process_image_test.go

apply norm

prompt processing

prompt processing

fix post tokenize

fix gguf padding + populate the split patch embeddings

...

...

another shot at patch embeddings

...

patch embedding

Update model_vision.go

split pixels
2025-05-12 13:49:41 -07:00