Bruce MacDonald
1a2c413225
move mask
2025-05-12 13:49:44 -07:00
Bruce MacDonald
57279f89a2
calculate block mask once, rather than in attention
2025-05-12 13:49:44 -07:00
Bruce MacDonald
9876c8453a
update exported functions for tests
2025-05-12 13:49:43 -07:00
Bruce MacDonald
16b13e0cfc
Revert "ropeTheta should be 1e5"
...
This reverts commit cc1638b26763eae7daddd44e3975a885671ef9d3.
This reverts commit
b32385591307e2d33a8f43ce1626b529d2dac83e.
2025-05-12 13:49:43 -07:00
Bruce MacDonald
45f96e898d
ropeTheta should be 1e5
2025-05-12 13:49:43 -07:00
Bruce MacDonald
39ee6d2bd0
ranges for lint
2025-05-12 13:49:43 -07:00
Michael Yang
698a92aa4a
reverse window
2025-05-12 13:49:43 -07:00
Michael Yang
150c499cae
use silu
2025-05-12 13:49:43 -07:00
Bruce MacDonald
b68af0370f
move sdpa to model forward pass
2025-05-12 13:49:43 -07:00
Bruce MacDonald
b3da8a319e
Update model_vision.go
2025-05-12 13:49:42 -07:00
Bruce MacDonald
359e1d5b19
full attention layers
2025-05-12 13:49:42 -07:00
Bruce MacDonald
ff1f74534b
block attention
2025-05-12 13:49:42 -07:00
Bruce MacDonald
104f802df1
remove todos
2025-05-12 13:49:42 -07:00
Bruce MacDonald
eed0ac2948
clean up vision model forward pass
2025-05-12 13:49:42 -07:00
Bruce MacDonald
fcfad744ff
fix patch merger
2025-05-12 13:49:42 -07:00
Michael Yang
fb3c16f2a2
window index
2025-05-12 13:49:42 -07:00
Michael Yang
88b231f903
use maxgridsize
2025-05-12 13:49:42 -07:00
Michael Yang
7e920c8d75
fix: patch merger and convert
...
convert:
- split patch embedding
- split qkv
remove duplicate PatchMerger
2025-05-12 13:49:42 -07:00
Bruce MacDonald
2af76d0e7a
default to 32 for vision block count
2025-05-12 13:49:42 -07:00
Bruce MacDonald
8d901825f0
reshape cos and sin
2025-05-12 13:49:41 -07:00
Bruce MacDonald
04936b719f
Update model_vision.go
2025-05-12 13:49:41 -07:00
Bruce MacDonald
0f0136d419
simplify by doing operations in Go rather than with tensors
...
Co-Authored-By: Michael Yang <2372640+mxyng@users.noreply.github.com >
2025-05-12 13:49:41 -07:00
Bruce MacDonald
80498f76de
fix build
2025-05-12 13:49:41 -07:00
Bruce MacDonald
5ff0d538b0
wip: implementing rope
2025-05-12 13:49:41 -07:00
Bruce MacDonald
eedc969c35
grid refactor
2025-05-12 13:49:41 -07:00
Bruce MacDonald
963531215e
update convert
2025-05-12 13:49:41 -07:00
Bruce MacDonald
3fe090f447
get patch embedding vals from config
2025-05-12 13:49:41 -07:00
Bruce MacDonald
1704072746
patch embeddings
2025-05-12 13:49:41 -07:00
Bruce MacDonald
c1f9bcb4dd
restructure
...
image processing
Update model.go
Update model.go
Update model.go
no projector
no projector
vision model scaffold
...
...
wip
...
rebase
fix patch merger
tidy
...
Update model_vision.go
server: do not attempt to parse offset file as gguf
This logic was causing issues for me when importing a gguf that had some padding at the end of the file. The valid gguf would be read, but then it would try to read the offset as a different gguf file. This does not seem right.
Update process_image_test.go
apply norm
prompt processing
prompt processing
fix post tokenize
fix gguf padding + populate the split patch embeddings
...
...
another shot at patch embeddings
...
patch embedding
Update model_vision.go
split pixels
2025-05-12 13:49:41 -07:00
