Building a Custom AI Assistant: Using Gemini to Unlock the Power of Open WebUI

Introduction: The Journey Begins

For a couple of years now, I’ve been self-hosting Ollama + Open WebUI on my 2020 M1 Mac Mini. While I never spent too much time customizing it, I recently decided to see if I could recreate some helpful tools that I use at work similar to Gemini Gems or Cursor Rules. These tools make an AI chat more useful by giving the selected model persistent instructions, specialized roles, and project guardrails so you don’t have to re-type context every time.

I started by hunting for a high-performing system prompt on Open WebUI. I came across “The Personal Executive Assistant – Jenny v2.0,” which had a fantastic, highly structured framework for task management and strategic planning. It was a good starting point, but out of the box, it wasn’t what I needed. I wanted a digital assistant that had a shared context of topics that are top of mind for me: nonprofit systems, human-centered leadership, and a practical approach to serving others.

Crafting a Good Prompt

A well-crafted prompt is essential for creating a custom AI assistant that meets your needs and expectations. A good prompt should include clear goals, specific tasks, target audience, voice and tone, and rules and constraints. By including these elements in your prompt, you can create a more effective and personalized AI assistant that meets your specific needs and expectations.

Before crafting your own prompt from scratch, you can search Open WebUI’s website for existing prompts that others have created. This can be a great way to find inspiration and build upon the work of others. You can search by keywords, categories, or even browse through the top-rated prompts.

Instead of rewriting thousands of words of prompt instructions by hand, I brought the base prompt over to Gemini to act as my prompt architect. Together, we tailored the persona from the ground up:

• The Voice: Grounded, plain-spoken, warm, and outcome-first.
• The Rules: Strict anti-bloat constraints to strip out fluff and hyper-corporate jargon (say goodbye to buzzwords like “optimization” or “synergy”).
• The Capabilities: A multi-domain framework built to switch seamlessly between executive task planning, emotional clarity during high-stress weeks, legal analysis, and objective truth-seeking.

Gemini helped translate my leadership philosophy into precise, strict prompt rules that a local LLM could actually follow. And here’s the key point: I’m using Gemini in this process because it’s an incredibly powerful tool. But my goal is to use this power to build something private and self-sustaining.

Taking a complex prompt and putting it on a local 8B model (running on an M1 Mac Mini via Ollama) always brings real-world quirks. My first few test runs were… not ideal. Jarvis kept falling into “tool-recovery loops,” apologizing, or spitting out raw, robotic bullet lists instead of clean executive summaries.
Working back and forth with Gemini, we systematically debugged the setup step-by-step:

• Memory Optimization: Dialed in num_ctx to 8192 to give Jarvis a deep context window without maxing out the Mac Mini’s 16GB unified memory pool.
• Tool Stripping: Realized local models get “tool-happy.” By disabling native function calling and background tool schemas (search_chats, memory, tasks), we forced Open WebUI to feed my prompt directly to the LLM engine without wrapper errors.
• Formatting Rules: Rewrote the response template syntax to use literal Markdown examples (## Executive Summary), forcing the model into scannable, pristine outputs every single time.

After fine-tuning Jarvis, I was impressed by the noticeable improvement. Now, when I ask for a 30-day plan or help with balancing tasks, I get:

1. A Practical Executive Summary right at the top.
2. Explicit Module Activation (calling out executive planning or emotional clarity with confidence levels).
3. A Clean, Scannable Action Plan with zero buzzwords and zero tool errors.

Building your own local executive assistant isn’t just about grabbing a prompt off the web. It’s about dialing in the underlying engine to match your exact hardware and workflow. And sometimes, that means using powerful tools like Gemini to help you get there faster.

Different Use Cases for Your Custom AI Assistant

While I built my custom AI assistant, Jarvis, with a specific focus on executive task planning and strategic decision-making, the possibilities are endless. Here are some different use cases for creating your own AI assistant:

• The Health-Conscious Individual: Create a personalized AI assistant that can support your health and wellness goals, such as tracking exercise routines, monitoring nutrition intake, and providing stress management techniques. Use Gemini to ensure the prompt is grounded in best practices for maintaining physical and mental well-being.
• The Productivity Powerhouse: Design a custom AI assistant that can supercharge your productivity by helping you prioritize tasks, manage time effectively, and stay organized. Use Gemini to ensure the prompt is aligned with my personal productivity philosophy.
• The Financial Whiz: Create a personalized AI assistant that can help you manage your finances by tracking expenses, budgeting, and providing investment advice. Use Gemini to ensure the prompt is grounded in best practices for personal finance management.
• The Tech-Savvy Entrepreneur: Create a personalized AI assistant that can handle tasks such as scheduling meetings, managing emails, and tracking sales pipeline. Use Gemini to customize the prompt and ensure it’s aligned with my leadership philosophy.
• The Nonprofit Leader: Design a custom AI assistant that can support my nonprofit team with tasks like donor outreach, event planning, and grant writing. Use Gemini to ensure the prompt is grounded in our organization’s values and mission.
• The Executive Coach: Create a personalized AI assistant that can support my clients with tasks like goal-setting, accountability tracking, and performance feedback. Use Gemini to ensure the prompt is grounded in best practices for executive coaching.

These use cases illustrate how different personas, whether personal or professional, might approach the creation of a custom AI assistant using Gemini. By tailoring the prompt to the specific needs and values of each persona, you can create a more personalized and effective AI assistant that aligns with their goals and objectives.

Have you tried running tailored assistant prompts on your local setup? Let’s swap notes!