I will setup inference for your open source models


About this gig
I will build inference for your open source models using engines like TensorRT-LLM, vLLM and sGLang . For low latency workflows I will help you achieve a low latency through methods like quantization, model compilation, batching and streaming. I will also setup security measures for your inference endpoints with API keys and RAG support to help attach documents to your prompts. This service is inclusive for on premises and cloud both on CPU and GPUs.
Get to know Erick
Selfhosted solutions
- FromKenya
- Member sinceSep 2022
- Avg. response time19 hours
- Last delivery3 months
Languages
English, Swahili
1 reviews for this Gig
| (1) | ||
| (0) | ||
| (0) | ||
| (0) | ||
| (0) |
Rating Breakdown
- Seller communication level
- Quality of delivery
- Value of delivery
Sort By
B benrudidi82

Cameroon
Ongoing collaborationfor me it is difficult to find people like that I feel very confident working with him, he is very professional, he understands very quickly, he complies with all my requirements, he does things in a professional manner, despite the fact that I do not speak English, he adapts to the French, to the point...
$50-$100
Price
4 days
Duration
Helpful?
1 reviews for this Gig
| (1) | ||
| (0) | ||
| (0) | ||
| (0) | ||
| (0) |
Rating Breakdown
- Seller communication level
- Quality of delivery
- Value of delivery
Sort By
B benrudidi82

Cameroon
Ongoing collaborationfor me it is difficult to find people like that I feel very confident working with him, he is very professional, he understands very quickly, he complies with all my requirements, he does things in a professional manner, despite the fact that I do not speak English, he adapts to the French, to the point...
$50-$100
Price
4 days
Duration
Helpful?
