Groq
Configure MyDeskBot to use Groq for ultra-fast AI inference.
Overview
Groq provides incredibly fast AI inference using their custom LPUs (Language Processing Units).
Supported Models
Groq hosts many popular open-source models:
- Llama 3.1 - Meta's latest models
- Mixtral 8x7B - Mixture of Experts model
- Gemma 2 - Google's open models
- Whisper - Audio transcription
Get API Key
- Visit console.groq.com
- Sign up or log in
- Go to API Keys section
- Create a new API key
- Copy the key
Configuration Options
Model Selection
| Model | Best For | Cost | Speed |
|---|---|---|---|
| Llama 3.1 70B | Complex tasks | Low | Very Fast |
| Llama 3.1 8B | General tasks | Low | Very Fast |
| Mixtral 8x7B | Coding, complex | Low | Very Fast |
| Gemma 2 9B | General tasks | Low | Very Fast |
Parameters
- Temperature (0-2) - Controls randomness
- Max Tokens - Maximum response length
- Top P - Nucleus sampling
Pricing
Groq offers very competitive pricing. Check Groq Pricing for current rates.
Why Choose Groq?
- Lightning Fast - Custom LPU hardware for speed
- Low Cost - Competitive pricing
- Open Models - Open-source models hosted
- Low Latency - Real-time applications
Troubleshooting
Invalid API Key
- Verify your API key is correct
- Check your account status
Rate Limits
- Groq has generous rate limits
- Monitor usage in console