Skip to content

@ModelCloud ModelCloud.ai

Change the repository type filter

All

    Repositories list

    2 repositories

    • GPTQModel

      Public
      Production ready LLM model compression/quantization toolkit with accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.
      Python
      •
      Apache License 2.0
      •30•142•9•4•Updated Dec 11, 2024Dec 11, 2024

    • Device-SMI

      Public
      Self-contained Python lib with zero-dependencies that give you a unified device properties for gpu, cpu, and npu. No more calling separate tools such as nvidia-smi or /proc/cpuinfo and parsing it yourself.
      devicecpugpusminpuxpu
      Python
      •
      Apache License 2.0
      •0•9•0•3•Updated Dec 10, 2024Dec 10, 2024