Troubleshooting

​

Common Issues

​

Engine Fails to Start with Memory Error

1. Verify your actual GPU memory
   nvidia-smi
   
2. Try the next lower preset
   # If using baseline-32g, try baseline-48g instead
   ai:
     preset: "baseline-48g"
   
3. Remove optional capabilities to reduce memory usage
   # Remove capabilities
   ai:
     preset: "baseline-48g"  # Instead of "baseline-48g,capabilities.multilingual"
   
4. Check for other applications using GPU memory
5. Reboot the machine

​

Poor Performance or Slow Responses

1. Ensure you’re using the correct preset for your hardware
2. Consider decreasing to a lower-tier preset
3. Communicate with Zylon engineers to understand what is happening

​

Pod in Failed or CrashLoopBackOff State

# Restart the deployment
kubectl rollout restart deploy/zylon-triton -n zylon

​

Advanced Issues

​

Startup Failures

​

Triton Inference Server Fails to Start

1. Check the Triton logs to identify which specific model is causing the failure
   kubectl logs deploy/zylon-triton -n zylon --tail=200
   
2. Verify memory allocation for the problematic model - adjust gpuMemoryUtilization if needed
3. If you’ve reduced memory allocation too much, reduce the contextWindow parameter for that model
4. Use nvidia-smi to check actual GPU memory usage and availability
   nvidia-smi
   

​

Unsupported Model Version

• Mistral Small 3 (2501) is supported
• Mistral Small 3 (2509) might not be supported yet

1. Check the supported model version in the documentation
2. Try an earlier version of the same model family if available
3. Check Zylon release notes for supported model versions
4. Contact Zylon engineers to confirm model compatibility

​

Memory Errors

​

Engine Fails to Start with “Out of Memory” Error

1. Verify total gpuMemoryUtilization does not exceed 0.95
   # Calculate total across all models
   ai:
     config:
       models:
         - id: llm
           gpuMemoryUtilization: 0.60
         - id: llmvision
           gpuMemoryUtilization: 0.25
         - id: embed
           gpuMemoryUtilization: 0.10
   # Total: 0.95 ✓
   
2. Reduce allocation for one or more models based on the crash logs
   kubectl logs deploy/zylon-triton -n zylon
   
3. Check actual GPU memory with nvidia-smi during startup
   watch -n 1 nvidia-smi