 Jiahao Li
|
35195689cd
2x faster (rms) norm cuda kernels (3.7% e2e improvement) (#2985)
|
2 years ago |
 Engininja2
|
f04d002844
cuda : vsubss4 for older versions of ROCm/clang (#2942)
|
2 years ago |
 Johannes Gäßler
|
92b1bbd2ec
CUDA: fix RoPE asserts, block sizes (#2833)
|
2 years ago |
 Georgi Gerganov
|
eaa13a48ff
falcon : fix CUDA inference by making K and Q contiguous (#2830)
|
2 years ago |
 Kawrakow
|
a6d1189fdd
k_quants tuning for Falcon-7b (#2816)
|
2 years ago |
 Henri Vasserman
|
6bbc598a63
ROCm Port (#1087)
|
2 years ago |
|
Georgi Gerganov
|
3f460a2b72
cuda : add RoPE kernel for mode == 2 (NeoX) (#2760)
|
2 years ago |
|
Georgi Gerganov
|
cf658adc83
llm : add Falcon support (#2717)
|
2 years ago |
|
Johannes Gäßler
|
c63bb1d16a
CUDA: use mul_mat_q kernels by default (#2683)
|
2 years ago |
|
Jiahao Li
|
800c9635b4
Fix CUDA softmax by subtracting max value before exp (#2665)
|
2 years ago |
 slaren
|
1123f7fbdf
ggml-cuda : use graph allocator (#2684)
|
2 years ago |
|
Georgi Gerganov
|
ef3f333d37
ggml : sync latest (SAM + SD operators, CUDA alibi) (#2709)
|
2 years ago |
|
slaren
|
097e121e2f
llama : add benchmark example (#2626)
|
2 years ago |
|
Johannes Gäßler
|
1cd06fa25e
CUDA: launch_bounds, small q4_K, q5_K mmq refactor (#2596)
|
2 years ago |
|
Johannes Gäßler
|
f64d44a9b9
CUDA: Fixed OpenLLaMA 3b mmq, reduced compile time (#2590)
|
2 years ago |
|
Johannes Gäßler
|
25d43e0eb5
CUDA: tuned mul_mat_q kernels (#2546)
|
2 years ago |
|
Johannes Gäßler
|
f514d1b306
CUDA: faster k-quant mul_mat_q kernels (#2525)
|
2 years ago |
 Cebtenzzre
|
4329d1acb0
CUDA: use min compute capability of GPUs actually used (#2506)
|
2 years ago |
|
Cebtenzzre
|
02f9d96a86
CUDA: check if event is NULL before cudaStreamWaitEvent (#2505)
|
2 years ago |
|
Johannes Gäßler
|
468ea24fb4
CUDA: faster non k-quant mul_mat_q kernels (#2483)
|
2 years ago |
|
Johannes Gäßler
|
4f6b60c776
CUDA: Fix models with output size != 32000 (#2480)
|
2 years ago |
|
Johannes Gäßler
|
0728c5a8b9
CUDA: mmq CLI option, fixed mmq build issues (#2453)
|
2 years ago |
|
Johannes Gäßler
|
1215ed7d5c
CUDA: Implemented row flattening for non-glm RoPE (#2468)
|
2 years ago |
|
Johannes Gäßler
|
2dbf518911
CUDA: fewer memory bank conflicts for mul_mat_q (#2458)
|
2 years ago |
|
Johannes Gäßler
|
11f3ca06b8
CUDA: Quantized matrix matrix multiplication (#2160)
|
2 years ago |
|
Johannes Gäßler
|
9baf9ef304
CUDA: faster multi GPU synchronization (#2448)
|
2 years ago |
|
Kawrakow
|
129d844c87
Fix Q4_K and Q5_K for QK_K = 64 on CUDA (#2359)
|
2 years ago |
|
slaren
|
41c674161f
make rms_norm_eps a parameter (#2374)
|
2 years ago |
|
Georgi Gerganov
|
5b2b2dc6ae
ggml : sync (unary ops refactor, static-correctness) (#2370)
|
2 years ago |
|
Kawrakow
|
2f9cf974a0
Some more Q4_K and Q5_K speedup on CUDA (#2346)
|
2 years ago |
