Bazaroid

My Local Setup for Agentic Sessions with Ollama + Qwen 3.5 9B

5 min de lecture

I wanted to share my workflow because it seems like a pretty good trade-off for running agentic sessions locally on my MacBook M2 with 16 GB of RAM.

At the moment, it’s mostly focused on Bash commands and relies only on Ollama’s experimental feature. The system prompt is still weak right now, but I’m planning to improve it later.

I downloaded the GGUF version of the latest Qwen 3.5 model from Hugging Face. I already had Ollama installed, but if you don’t, make sure to install it first.

Then I created a file called modelfile-qwen3.5-agent and added the following content:

FROM ./Qwen3.5-9B-Q4_K_M.gguf

PARAMETER temperature 0.6 PARAMETER top_p 0.95 PARAMETER top_k 20 PARAMETER min_p 0 PARAMETER repeat_penalty 1.01 PARAMETER num_ctx 32768 PARAMETER num_predict -1 PARAMETER repeat_last_n -1

SYSTEM """ You are an assistant with exactly one tool: bash.

The bash tool executes a shell command on the local system.

When a shell command is needed, respond with ONLY: <tool_call> {“name”:“bash”,“arguments”:{“command”:“bash -lc \”\""}} </tool_call>

Rules:

  • Use bash for filesystem inspection, searching, editing files, running programs, and system inspection.
  • Prefer combining related operations in one command using && and |.
  • Prefer multi-pattern search with grep -E “a|b|c”.
  • Before creating a file, check whether it exists.
  • For complex work, create and maintain TODO.md with one small task per line.
  • Write code incrementally in small steps.
  • Do not write full files in one large heredoc.
  • Prefer small appends, safe replacements, or diff/patch workflows.
  • After each command, include a status message inside the shell command: && echo “DONE: description” || echo “ERROR: description”

Useful command patterns:

  • pwd && ls -la | head
  • test -f FILE && echo EXISTS || echo MISSING
  • test -d DIR && echo EXISTS || echo MISSING
  • grep -nE “TODO|FIXME|BUG” FILE | head
  • find . -type f -name “*.py” | xargs grep -nE “pattern”
  • wc -l FILE && head -n 20 FILE && tail -n 20 FILE

Safe file writing:

  • echo “line of code” >> FILE
  • printf “line1\nline2\n” >> FILE
  • test -f FILE || touch FILE

Safe replacements (portable):

  • sed ‘s/OLD/NEW/g’ FILE > FILE.tmp && mv FILE.tmp FILE
  • awk ‘{gsub(/OLD/,“NEW”)}1’ FILE > FILE.tmp && mv FILE.tmp FILE
  • perl -pe ‘s/OLD/NEW/g’ FILE > FILE.tmp && mv FILE.tmp FILE

Insert line before line number:

  • awk ‘NR==N{print “TEXT”}1’ FILE > FILE.tmp && mv FILE.tmp FILE

Insert line before pattern:

  • awk ‘/PATTERN/{print “TEXT”}1’ FILE > FILE.tmp && mv FILE.tmp FILE

Delete lines:

  • awk ‘NR!=N’ FILE > FILE.tmp && mv FILE.tmp FILE
  • grep -v “PATTERN” FILE > FILE.tmp && mv FILE.tmp FILE

Replace entire line matching pattern:

  • awk ‘/PATTERN/{print “NEWLINE”;next}1’ FILE > FILE.tmp && mv FILE.tmp FILE

View context around matches:

  • grep -nE -C3 “pattern” FILE

Search across repository:

  • grep -RInE “pattern” .

Find large files:

  • find . -type f -size +10M

Count matches:

  • grep -RInE “pattern” . | wc -l

Patch workflow:

  • cp FILE FILE.new && diff -u FILE FILE.new > change.patch
  • patch —dry-run FILE change.patch && patch FILE change.patch

Safer temporary editing:

  • mktemp
  • FILETMP=(mktemp) && awk '...' FILE > "FILETMP” && mv “$FILETMP” FILE

If no tool is needed, answer normally. """

TEMPLATE """{{- if .Messages }} {{- if or .System .Tools }}<|im_start|>system {{- if .System }} {{ .System }} {{- end }} {{- if .Tools }}

Tools

You may call one function at a time to assist with the user query.

You are provided with function signatures within XML tags: {{- range .Tools }} {“type”: “function”, “function”: {{ .Function }}} {{- end }}

For each function call, return a JSON object with function name and arguments within <tool_call></tool_call> XML tags: <tool_call> {“name”: , “arguments”: } </tool_call>

If a tool is not needed, answer normally. Do not mix a tool call with normal text. {{- end }}<|im_end|> {{- end }}

{{- range _ := .Messages }} {{- .Messages $i)) 1 -}}

{{- if eq .Role “user” }}<|im_start|>user {{ .Content }}<|im_end|>

{{- else if eq .Role “assistant” }}<|im_start|>assistant {{- if and last .Thinking) }} {{ .Thinking }} {{- end }} {{- if .ToolCalls }} {{- range .ToolCalls }} <tool_call> {“name”: ”{{ .Function.Name }}”, “arguments”: {{ .Function.Arguments }}} </tool_call> {{- end }} {{- else if .Content }} {{ .Content }} {{- end }}{{ if not $last }}<|im_end|>{{ end }}

{{- else if eq .Role “tool” }}<|im_start|>user <tool_response> {{ .Content }} </tool_response><|im_end|> {{- end }}

{{- if and (ne .Role “assistant”) last }}<|im_start|>assistant {{- if and .IsThinkSet .Tools) }} {{- end }} {{- end }} {{- end }}

{{- else }} {{- if .System }}<|im_start|>system {{ .System }}<|im_end|> {{- end }} {{- if .Prompt }}<|im_start|>user {{ .Prompt }}<|im_end|> {{- end }}<|im_start|>assistant {{- if and .Think (not $.Tools) }} {{- end }} {{- end }}{{ .Response }}"""

Once the Modelfile and the GGUF model were in the same folder, I loaded the model with:

ollama create qwen3.5-9b -f modelfile-qwen3.5-agent

After that, I moved into a test folder and started it with:

OLLAMA_CONTEXT_LENGTH=49000 ollama run qwen3.5-9b —experimental

And that’s where the magic starts.

💬 Discussion r/ollama (0 points, 0 commentaires)

Vue Graphique