 Andrew Godfrey
|
73bdcb395e
finetune : add -ngl parameter (#3762)
|
2 jaren geleden |
 Georgi Gerganov
|
2f9ec7e271
cuda : improve text-generation and batched decoding performance (#3776)
|
2 jaren geleden |
|
Georgi Gerganov
|
6961c4bd0b
batched-bench : print params at start
|
2 jaren geleden |
|
Georgi Gerganov
|
b2f7e04bd3
sync : ggml (conv ops + cuda MSVC fixes) (#3765)
|
2 jaren geleden |
|
Georgi Gerganov
|
2b4ea35e56
cuda : add batched cuBLAS GEMM for faster attention (#3749)
|
2 jaren geleden |
 Jan Ploski
|
f5f9121de1
llm : add MPT support (#3417)
|
2 jaren geleden |
|
Georgi Gerganov
|
db3abcc114
sync : ggml (ggml-backend) (#3548)
|
2 jaren geleden |
 slaren
|
f5ef5cfb18
ggml-cuda : perform cublas mat mul of quantized types as f16 (#3412)
|
2 jaren geleden |
|
slaren
|
16bc66d947
llama.cpp : split llama_context_params into model and context params (#3301)
|
2 jaren geleden |
|
Georgi Gerganov
|
ec893798b7
llama : custom attention mask + parallel decoding + no context swaps (#3228)
|
2 jaren geleden |
|
slaren
|
da0400344b
ggml-cuda : perform cublas fp16 matrix multiplication as fp16 (#3370)
|
2 jaren geleden |
 Johannes Gäßler
|
ee66942d7e
CUDA: fix peer access logic (#3231)
|
2 jaren geleden |
|
Johannes Gäßler
|
111163e246
CUDA: enable peer access between devices (#2470)
|
2 jaren geleden |
|
Johannes Gäßler
|
578d8c8f5c
CUDA: fix scratch malloced on non-main device (#3220)
|
2 jaren geleden |
 Vlad
|
5dbc2b3213
Enable build with CUDA 11.0 (make) (#3132)
|
2 jaren geleden |
|
Johannes Gäßler
|
0a5eebb45d
CUDA: mul_mat_q RDNA2 tunings (#2910)
|
2 jaren geleden |
|
Johannes Gäßler
|
4f7cd6ba9c
CUDA: fix LoRAs (#3130)
|
2 jaren geleden |
|
Johannes Gäßler
|
89e89599fd
CUDA: fix mul_mat_q not used for output tensor (#3127)
|
2 jaren geleden |
|
Johannes Gäßler
|
d54a4027a6
CUDA: lower GPU latency + fix Windows performance (#3110)
|
2 jaren geleden |
|
Johannes Gäßler
|
8a4ca9af56
CUDA: add device number to error messages (#3112)
|
2 jaren geleden |
|
Georgi Gerganov
|
b3e9852e47
sync : ggml (CUDA GLM RoPE + POSIX) (#3082)
|
2 jaren geleden |
 Jiahao Li
|
35195689cd
2x faster (rms) norm cuda kernels (3.7% e2e improvement) (#2985)
|
2 jaren geleden |
 Engininja2
|
f04d002844
cuda : vsubss4 for older versions of ROCm/clang (#2942)
|
2 jaren geleden |
|
Johannes Gäßler
|
92b1bbd2ec
CUDA: fix RoPE asserts, block sizes (#2833)
|
2 jaren geleden |
|
Georgi Gerganov
|
eaa13a48ff
falcon : fix CUDA inference by making K and Q contiguous (#2830)
|
2 jaren geleden |
 Kawrakow
|
a6d1189fdd
k_quants tuning for Falcon-7b (#2816)
|
2 jaren geleden |
 Henri Vasserman
|
6bbc598a63
ROCm Port (#1087)
|
2 jaren geleden |
|
Georgi Gerganov
|
3f460a2b72
cuda : add RoPE kernel for mode == 2 (NeoX) (#2760)
|
2 jaren geleden |
|
Georgi Gerganov
|
cf658adc83
llm : add Falcon support (#2717)
|
2 jaren geleden |
|
Johannes Gäßler
|
c63bb1d16a
CUDA: use mul_mat_q kernels by default (#2683)
|
2 jaren geleden |
