NVIDIA
Configure MyDeskBot to use NVIDIA NIM (NVIDIA Inference Microservices).
Overview
NVIDIA NIM provides optimized inference for AI models with enterprise-grade performance.
Supported Models
NVIDIA NIM supports various models:
- Llama 3.1 - Meta's models, NVIDIA optimized
- Mixtral - Mixture of Experts
- Mistral - High-quality models
- Phi-3 - Small, efficient models
- And more...
Get Access
- Visit build.nvidia.com
- Sign up or log in
- Navigate to NIM catalog
- Select a model to use
- Get API access
Configuration Options
Model Selection
| Model | Best For | Cost | Speed |
|---|---|---|---|
| Llama 3.1 70B NIM | Complex tasks | Medium | Fast |
| Llama 3.1 405B NIM | Most capable | High | Medium |
| Mixtral 8x7B NIM | Coding, complex | Low | Very Fast |
| Phi-3 NIM | Quick tasks | Low | Very Fast |
Parameters
- Temperature (0-2) - Controls randomness
- Max Tokens - Maximum response length
- Top P - Nucleus sampling
Pricing
NVIDIA NIM offers various pricing options. Check build.nvidia.com for current rates.
Why Choose NVIDIA?
- Optimized - GPU-optimized performance
- Enterprise - Enterprise-grade support
- Deployment - Deploy anywhere (cloud, on-prem, edge)
- Performance - High throughput
Deployment Options
- Cloud API - Use NVIDIA hosted inference
- On-Premises - Deploy NIM on your own infrastructure
- Edge - Deploy at the edge
Troubleshooting
Invalid API Key
- Verify your API key is correct
- Check your NIM access
Performance Issues
- Ensure adequate GPU resources for on-prem deployment
- Check network connectivity for cloud API