 Piotr Wilkin
|
75586ea36e
Delta.net chunked reimplemented
|
3 ماه پیش |
|
Piotr Wilkin
|
61fbeef88b
Attempt 246461
|
3 ماه پیش |
|
Piotr Wilkin
|
a51d4381d4
Like this.
|
3 ماه پیش |
|
Piotr Wilkin
|
54bb6f1eb9
argh again
|
3 ماه پیش |
|
Piotr Wilkin
|
20424d8785
argh
|
3 ماه پیش |
|
Piotr Wilkin
|
413652178f
attempt 2
|
3 ماه پیش |
|
Piotr Wilkin
|
c5dc442a5d
repeat_interleave
|
3 ماه پیش |
 Piotr Wilkin
|
a4fe12821b
Fix layer counting logic
|
3 ماه پیش |
|
Piotr Wilkin
|
610b0fede7
Wrong tensor for comparison
|
3 ماه پیش |
|
Piotr Wilkin
|
4d571eda07
Let's dump extra tensors
|
3 ماه پیش |
|
Piotr Wilkin
|
2cab86a09f
Let the debug out.
|
3 ماه پیش |
|
Piotr Wilkin
|
7eef0bd948
Rewrite recurrent delta + softmax to separate ops
|
3 ماه پیش |
|
Piotr Wilkin
|
554593d60d
Variable scopes are fun
|
3 ماه پیش |
|
Piotr Wilkin
|
0b301889bf
Stabilize tensor dump trigger for now with -n < 50
|
3 ماه پیش |
|
Piotr Wilkin
|
f0a07c1091
Add proper backend tensor printing, use double for accumulating the sum
|
3 ماه پیش |
|
Piotr Wilkin
|
4c8771d200
Print 5D tensors
|
3 ماه پیش |
|
Piotr Wilkin
|
10032affcf
More debug data
|
3 ماه پیش |
|
Piotr Wilkin
|
d300ce9eba
Hmmmm......
|
3 ماه پیش |
|
Piotr Wilkin
|
3f5994223b
Hmm...
|
3 ماه پیش |
|
Piotr Wilkin
|
7348546b5e
Missing cont()
|
3 ماه پیش |
|
Piotr Wilkin
|
5a161d9461
Remove unnecessary transposes/reshapes
|
3 ماه پیش |
|
Piotr Wilkin
|
572864287e
Handle case with more than one token per seq with elegant loop plus completely not crazy change to max nodes ;)
|
3 ماه پیش |
|
Piotr Wilkin
|
c2a82a1773
Move the norm shift to conversion, Gemma 2 style
|
3 ماه پیش |
|
Piotr Wilkin
|
5306640300
All's well that ends in a well
|
3 ماه پیش |
|
Piotr Wilkin
|
232ec56251
Yes, I finally managed to implement it with ssm_conv :>
|
3 ماه پیش |
|
Piotr Wilkin
|
aa8d6a21a3
Remove extra files cont.
|
3 ماه پیش |
|
Piotr Wilkin
|
e9a98f2af9
Remove extra files
|
3 ماه پیش |
|
Piotr Wilkin
|
22ee5a971b
Add gate_sigmoid to callback
|
3 ماه پیش |
|
Piotr Wilkin
|
ce87b7d78e
Yup, it's NeoX
|
3 ماه پیش |
|
Piotr Wilkin
|
df0b5bcf30
Proper order of attention operations
|
3 ماه پیش |
