How to Set Up Local AI in Your Terminal with Wave and Ollama

Working in the terminal often means switching between searching for commands, debugging errors, and writing scripts — all while staying focused. Relying on cloud-based AI tools can help, but not everyone wants their data going to external servers. Some developers also work in offline environments where cloud services aren’t an option.
Local AI models offer a solution: you get on-demand assistance without compromising privacy or needing constant internet access. This guide shows how to set up local AI in a terminal environment using Wave Terminal and Ollama.
Key Takeaways
- Local AI lets you use AI assistance privately, without sending your queries to the cloud.
- Running your own model like Llama 2 avoids dependency on third-party servers or internet access.
- Wave Terminal and Ollama offer a way to set this up quickly, but the general approach applies elsewhere too.
The problem: relying on external AI isn’t always ideal
Today, many AI integrations in developer tools assume you’re online and comfortable sending data to third parties. This leads to some real problems:
- Privacy risks: Commands, logs, or sensitive queries may leave your machine.
- Connectivity issues: You can’t always rely on stable internet access, especially when working remotely or in secure environments.
- Cost concerns: Using paid API models like GPT-4 at scale can get expensive.
If you need fast, local, private AI assistance — for things like generating scripts, explaining errors, or brainstorming solutions — cloud-only options aren’t enough.
The solution: running AI locally in your terminal
Local models like Llama 2, run through tools like Ollama, give you full control. Instead of depending on external APIs, you install a model once and query it directly from your machine.
Wave Terminal makes this setup easier by providing built-in support for local AI connections through a simple interface.
You’re not locked into Wave or Ollama specifically — this approach is general: local model + terminal integration.
Let’s walk through how to set it up.
Setting up local AI: step-by-step
1. Install a local LLM runtime (Ollama)
First, install a tool that lets you run AI models on your own machine. Ollama is a simple choice for macOS, Linux, and Windows.
- Download and install Ollama from the official site.
- After installing, open a terminal and verify it’s working:
ollama list
You should see available models or an empty list ready to add some.
2. Download a model (like Llama 2)
Pick a model to run locally. A good starting point is Llama 2.
In your terminal:
ollama run llama2
This will download the model and spin up a local server on localhost:11434
.
You now have a language model ready to use without any cloud connection.
Model requirements:
- Disk space: 3-4GB for the 7B parameter model; larger variants need more
- RAM: Minimum 8GB (16GB recommended) for the 7B model; 32GB+ for larger variants
- Processor: Any modern multi-core CPU; GPU acceleration improves performance
- Internet: Required only for initial download
- Operating systems: macOS, Linux, Windows (may require WSL)
3. Configure Wave Terminal to talk to your local model
Now link Wave Terminal to your local Ollama server.
- Install and open Wave Terminal.
- Go to Settings > AI Providers.
- Add a new provider:
- Name: Local Llama2 (or whatever you prefer)
- URL:
http://localhost:11434
- Model:
llama2
- Save and make it your default AI provider.
Wave will now route AI queries to your local model instead of the cloud.
4. Test it
In Wave Terminal, open the AI panel and try a simple query:
Suggest a bash script that backs up a directory daily.
If everything is set up correctly, you’ll get a suggestion from your local model — no internet required.
Things to know
- Hardware requirements: Running models locally needs decent specs (at least 16GB RAM recommended for smoother Llama 2 usage).
- Model size considerations: Smaller models (e.g., 7B parameters) use less memory but may be less capable; larger ones offer better answers but are heavier.
- Performance trade-offs: Local models may be slower than cloud GPT-4 initially, but avoid latency and API limits.
This setup is not limited to Wave — any terminal tool or editor that supports custom AI backends could use a local Ollama server.
Conclusion
Setting up local AI in your terminal represents a significant shift in how developers can interact with AI tools. Rather than constantly sending data to cloud services, you gain complete control over your development environment while still benefiting from AI assistance.
The combination of Wave Terminal and Ollama makes this process remarkably straightforward, even for those new to local AI models. You get:
- Enhanced privacy: Your code and queries never leave your machine
- Offline capabilities: Work from anywhere without worrying about internet connectivity
- Cost efficiency: Eliminate API usage fees for your everyday AI assistance needs
- Customization options: As you become more comfortable, you can explore different models to find the one that best fits your workflow
While cloud-based AI services certainly have their place, this local setup provides a powerful alternative that respects your privacy and gives you independence from third-party services.
Start with the 7B model to get comfortable with the workflow, then explore larger models or specialized ones as your needs evolve. The initial setup might take some time, but the long-term benefits of having AI assistance that works on your terms make it well worth the effort.
FAQs
No, both are free to use. Wave Terminal is open-source, and Ollama provides free access to several models. If you want premium models later, those might have separate costs, but local running is free.
Yes, but it depends on your machine. For example, running a 13B model (larger than 7B) would require more RAM and might be slower on laptops. Start small if you’re unsure.
Yes. Both Wave Terminal and Ollama now support Windows. Just be sure to follow installation instructions carefully — WSL might be needed in some cases for Ollama.
Yes. Although this guide uses Wave, any terminal or editor that allows setting a custom AI endpoint can connect to Ollama. It’s just a matter of configuring the right URL.