moritz/infrapuzzle
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

refactor(structure): reorganize monorepo with projects, terraform hierarchy, and strict k8s namespace rules

Browse Source
This commit is contained in:
Moritz Graf
2026-10-12 13:49:41 +02:00
parent 4831147087
commit 8f01b75bd6
4766 changed files with 551 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
k8s/llm/README.md
+24
View File
@@ -0,0 +1,24 @@
# llm-hosting
This is an extended article to not have to write everything in the main readme. This chapter takes care of hosting llm models on the server.
## deploy
```sh
kubectl apply -f llm/llama_cpp_hosting.yaml
```
## development
```sh
```
## links
Two examples of model files that are currently tried out:
* [https://huggingface.co/MaziyarPanahi/Meta-Llama-3-70B-Instruct-GGUF/resolve/main/Meta-Llama-3-70B-Instruct.IQ1_S.gguf?download=true](https://huggingface.co/MaziyarPanahi/Meta-Llama-3-70B-Instruct-GGUF/resolve/main/Meta-Llama-3-70B-Instruct.IQ1_S.gguf?download=true)
* From [this page](https://huggingface.co/MaziyarPanahi/Meta-Llama-3-70B-Instruct-GGUF/tree/main).
* [https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct.Q8_0.gguf?download=true](https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct.Q8_0.gguf?download=true)
* From [this page](https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/tree/main).