I will setup inference for your open source models

R
rickymuchiri
R
rickymuchiri
Erick
5.0
5.0

About this gig

I will build inference for your open source models using engines like TensorRT-LLM, vLLM and sGLang . For low latency workflows I will help you achieve a low latency through methods like quantization, model compilation, batching and streaming. I will also setup security measures for your inference endpoints with API keys and RAG support to help attach documents to your prompts. This service is inclusive for on premises and cloud both on CPU and GPUs.

Get to know Erick

Erick

Selfhosted solutions

4.9(10)
  • FromKenya
  • Member sinceSep 2022
  • Avg. response time19 hours
  • Last delivery3 months
  • Languages

    English, Swahili
I'm Erick, a Software Engineer and System Administrator specializing in self-hosted solutions. I deploy and manage secure server environments to help businesses operate independently from third-party SaaS platforms. My expertise includes Linux server administration, Docker containerization, virtualization with Proxmox, and VPN configuration using Tailscale, Wireguard and OpenVPN. I set up, troubleshoot, and optimize self-hosted services, ensuring efficient and reliable infrastructures tailored to your needs. I also provide Twilio services to help businesses efficiently reach out to customers.
Reviews

1 reviews for this Gig
5.0

(1)
(0)
(0)
(0)
(0)
Rating Breakdown
  • Seller communication level
    5
  • Quality of delivery
    5
  • Value of delivery
    5
Sort By
Most relevant
  • B

    benrudidi82

    CM

    Cameroon

    Ongoing collaboration
    5

    for me it is difficult to find people like that I feel very confident working with him, he is very professional, he understands very quickly, he complies with all my requirements, he does things in a professional manner, despite the fact that I do not speak English, he adapts to the French, to the point...

    $50-$100

    Price

    4 days

    Duration

    Helpful?
    Yes
    No
Reviews

1 reviews for this Gig
5.0

(1)
(0)
(0)
(0)
(0)
Rating Breakdown
  • Seller communication level
    5
  • Quality of delivery
    5
  • Value of delivery
    5
Sort By
Most relevant
  • B

    benrudidi82

    CM

    Cameroon

    Ongoing collaboration
    5

    for me it is difficult to find people like that I feel very confident working with him, he is very professional, he understands very quickly, he complies with all my requirements, he does things in a professional manner, despite the fact that I do not speak English, he adapts to the French, to the point...

    $50-$100

    Price

    4 days

    Duration

    Helpful?
    Yes
    No