How to Set Up Local AI in Your Terminal with Wave and Ollama

Working in the terminal often means switching between searching for commands, debugging errors, and writing scripts — all while staying focused. Relying on cloud-based AI tools can help, but not everyone wants their data going to external servers. Some developers also work in offline environments where cloud services aren’t an option.

Local AI models offer a solution: you get on-demand assistance without compromising privacy or needing constant internet access. This guide shows how to set up local AI in a terminal environment using Wave Terminal and Ollama.

The problem: relying on external AI isn’t always ideal

Today, many AI integrations in developer tools assume you’re online and comfortable sending data to third parties. This leads to some real problems:

• Privacy risks: Commands, logs, or sensitive queries may leave your machine.
• Connectivity issues: You can’t always rely on stable internet access, especially when working remotely or in secure environments.
• Cost concerns: Using paid API models like GPT-4 at scale can get expensive.

If you need fast, local, private AI assistance — for things like generating scripts, explaining errors, or brainstorming solutions — cloud-only options aren’t enough.

The solution: running AI locally in your terminal

Local models like Llama 2, run through tools like Ollama, give you full control. Instead of depending on external APIs, you install a model once and query it directly from your machine.

Wave Terminal makes this setup easier by providing built-in support for local AI connections through a simple interface.

You’re not locked into Wave or Ollama specifically — this approach is general: local model + terminal integration.

Let’s walk through how to set it up.

Setting up local AI: step-by-step

1. Install a local LLM runtime (Ollama)

First, install a tool that lets you run AI models on your own machine. Ollama is a simple choice for macOS, Linux, and Windows.

• Download and install Ollama from the official site.
• After installing, open a terminal and verify it’s working:

ollama list

You should see available models or an empty list ready to add some.

2. Download a model (like Llama 2)

Pick a model to run locally. A good starting point is Llama 2.

In your terminal:

ollama run llama2

This will download the model and spin up a local server on localhost:11434. You now have a language model ready to use without any cloud connection.

Model requirements:

• Disk space: 3-4GB for the 7B parameter model; larger variants need more
• RAM: Minimum 8GB (16GB recommended) for the 7B model; 32GB+ for larger variants
• Processor: Any modern multi-core CPU; GPU acceleration improves performance
• Internet: Required only for initial download
• Operating systems: macOS, Linux, Windows (may require WSL)

3. Configure Wave Terminal to talk to your local model

Now link Wave Terminal to your local Ollama server.

• Install and open Wave Terminal.
• Go to Settings > AI Providers.
• Add a new provider:
  • Name: Local Llama2 (or whatever you prefer)
  • URL: http://localhost:11434
  • Model: llama2
• Save and make it your default AI provider.

Wave will now route AI queries to your local model instead of the cloud.

4. Test it

In Wave Terminal, open the AI panel and try a simple query:

Suggest a bash script that backs up a directory daily.

If everything is set up correctly, you’ll get a suggestion from your local model — no internet required.

Things to know

• Hardware requirements: Running models locally needs decent specs (at least 16GB RAM recommended for smoother Llama 2 usage).
• Model size considerations: Smaller models (e.g., 7B parameters) use less memory but may be less capable; larger ones offer better answers but are heavier.
• Performance trade-offs: Local models may be slower than cloud GPT-4 initially, but avoid latency and API limits.

This setup is not limited to Wave — any terminal tool or editor that supports custom AI backends could use a local Ollama server.

Conclusion

Setting up local AI in your terminal represents a significant shift in how developers can interact with AI tools. Rather than constantly sending data to cloud services, you gain complete control over your development environment while still benefiting from AI assistance.

The combination of Wave Terminal and Ollama makes this process remarkably straightforward, even for those new to local AI models. You get:

• Enhanced privacy: Your code and queries never leave your machine
• Offline capabilities: Work from anywhere without worrying about internet connectivity
• Cost efficiency: Eliminate API usage fees for your everyday AI assistance needs
• Customization options: As you become more comfortable, you can explore different models to find the one that best fits your workflow

While cloud-based AI services certainly have their place, this local setup provides a powerful alternative that respects your privacy and gives you independence from third-party services.

Start with the 7B model to get comfortable with the workflow, then explore larger models or specialized ones as your needs evolve. The initial setup might take some time, but the long-term benefits of having AI assistance that works on your terms make it well worth the effort.

FAQs