Project to run IA locally in VB.NET

LocalIAdotNet
LocalIAdotNet is a fully local Windows Forms application written in VB.NET for downloading, running, and comparing local large language models.

Features
Analyzes CPU, RAM, GPU, VRAM, instruction sets, and free disk space.
Recommends compatible GGUF models for the current computer.
Downloads models from Hugging Face with resume support.

Installs and runs llama.cpp automatically.
Supports CPU, Vulkan, and CUDA backends.

Provides a local chat interface with streaming responses.
Displays total response time, estimated input tokens, estimated output tokens, and output tokens per second.
Supports PDF, image, Excel, Word, PowerPoint, CSV, JSON, XML, HTML, SQL, source code, and plain-text attachments.
Extracts text locally from supported documents.
Renders scanned PDF pages as images for compatible vision models.
Detects already installed runtimes and models to avoid unnecessary downloads.
Supports runtime discovery for llama.cpp, Ollama, LM Studio, vLLM, Jan, GPT4All, Docker Model Runner, and Apple MLX.
Requirements
Windows 10 or Windows 11, 64-bit.
Visual Studio 2022 with the .NET desktop development workload.
.NET 8 SDK.
Internet access for restoring NuGet packages and downloading runtimes or models.
Build
Open LocalIAdotNet.sln, select Release and x64, then build the solution.

You can also run:

build.cmd
To create a self-contained Windows x64 publication:

publish-win-x64.cmd
The published application is generated under the standard bin\Release\net8.0-windows...\win-x64\publish folder.

First run
Start LocalIAdotNet.
Click Analyze PC.
Review the automatically ranked model list.
Install or select a runtime.
Download a compatible model if required.
Start the model.
Open the Local chat tab and send a message.
Downloaded content is stored under:

%LOCALAPPDATA%\LocalIAdotNet
Supported document formats
Local text extraction
PDF files containing a text layer.
XLSX and XLSM workbooks.
DOCX documents.
PPTX presentations.
TXT, CSV, MD, JSON, XML, HTML, SQL, VB, CS, PHP, and similar text-based files.
Vision analysis
With a compatible multimodal model, LocalIAdotNet can analyze:

PNG, JPG, JPEG, BMP, GIF, TIFF, and WEBP images.
Scanned PDFs or PDFs without an extractable text layer.
By default, only the first pages of a scanned PDF are rendered to limit memory use and context size.

Legacy XLS, DOC, and PPT files must first be saved in modern Office Open XML formats.

Privacy
After the initial runtime and model downloads:

Inference runs on the local computer.
Attached documents remain on the local computer.
The embedded server listens only on 127.0.0.1.
Documents are not uploaded to Hugging Face or GitHub.
A Hugging Face token is used only when a repository requires authentication or license acceptance.
The Hugging Face token is currently stored in the local settings.json file. For production distribution, consider protecting it with Windows DPAPI.

Local API
When a llama.cpp model is running, llama-server exposes an OpenAI-compatible endpoint:

http://127.0.0.1:18456/v1/chat/completions
The port can be changed in the Settings tab.

Example request body:

{
"model": "local-model",
"messages": [
{
"role": "system",
"content": "You are a precise local assistant."
},
{
"role": "user",
"content": "Summarize this text."
}
]
}

Acceleration
The application recommends:
CUDA for supported NVIDIA GPUs.
Vulkan for other compatible GPUs.
CPU when no suitable GPU backend is available.
If a CUDA package is not available for a specific llama.cpp release, the installer can fall back to Vulkan or CPU.

Limitations
Hardware recommendations are estimates and should be validated with real benchmarks.
WMI may report inaccurate VRAM values for some non-NVIDIA GPUs.
Large documents are truncated to protect the context window.
Password-protected documents cannot be read.
Local models can produce incorrect information; important output should be reviewed.
A built-in vector database and RAG knowledge base are not yet included in this version.

Next step, attaching RAG