Shop is live

We build Workstation AI machines for websites, AI agents, and LLM training — configure and buy online.

Get Trade Prices
AI Products
AI LabsOpenAI AgentsClaude AgentsGrok BotWorkstation AI native CRMMarketingAll Products
AI Solutions
Applied AIAI AgentsGenerative AIAI AutomationAI UpskillingAI SME PackagesEnterprise AI LabAI WorkstationsNVIDIA Hardware & SoftwareNVIDIA Accelerated ComputingPrivate AIGPU Private CloudGPU ClustersEdge AIAI by Industry
AI Services
Platform ModernisationDigital EngineeringData Foundations & AIAutonomous OperationsAI ConsultancyDevOps AutomationCyber SecuritySoftware DevelopmentAgent BuildingMLOps SetupGPU LeasingGPU Private Cloud
About Us
CompanyOur StoryPartnersCustomer Stories
AI Data Centres
AI Data CentresGPU Private CloudGPU ClustersGPU Leasing
AI InsightsJev seriesBenchmark LabModel selection
Technical Notes
Documentation
Linux OSUbuntuRed Hat Enterprise LinuxSUSE LinuxAlpine LinuxDebian, Rocky & AlmaLinux
AI Cloud PlatformsAWS & Amazon BedrockGoogle Cloud & Vertex AIMicrosoft Azure & AI FoundryGPU clouds & marketplaces
Private AI Data CentresFor law firmsFor accounting firmsOn-premises platformsFully secured cloud
ToolsRing PromoterJobshout
News
Blog
Contact UsSign in

AI workstations, AI Multi Agentic Software, GPU infrastructure, and intelligent agent solutions for modern businesses.

Contact Us

AI Solutions

Applied AIAI AgentsGenerative AIAI AutomationAI UpskillingAI SME PackagesEnterprise AI LabAI by Industry

Products

All ProductsWorkstation AI native CRM & ERPMarketingOpenAI AgentsRing PromoterJobshout

Company

About UsCompanyAI Data CentresWhy WorkstationPartnersCustomer StoriesPricingContactTenthMatrix

Resources

AI InsightsTechnical NotesDocumentationLinux OSAI Cloud PlatformsPrivate AI Data CentresBlogSearchSitemap
UK Head Office
77-79 Marlowes, Hemel Hempstead HP1 1LFDirections - Take Junction 20 off M25 Outer LondonCompany No: 11641870Mon - Fri: 9:00 AM - 6:00 PM GMT
+44 7515 356 146
Belgium Office
Workstation SRL, Rue Vanderkindere 34, 1180 Uccle, BrusselsBE 0751.518.683Mon - Fri: 9:00 AM - 6:00 PM CET
+32 492 45 67 46
India Office
#159 Sector 9, Pocket 1, DDA Flats, 110077 Dwarka, New Delhi
+91 98881 98841
Taiwan Office
3F, No. 2-4, Aisi Rd., Ren'ai Dist., Keelung City 200, Taiwan基隆市仁愛區愛四路2之4號3樓Mon - Fri: 9:00 AM - 6:00 PM Taiwan time (GMT+8)
+886 926 892 701+32 492 45 67 46
Estonia Office
Aleksander Puškini tn 30-25, 20306 Narva, EstoniaMon - Fri: 9:00 AM - 6:00 PM Estonia time (EET)
+32 492 45 67 46

© 2026 Workstation AI. All rights reserved.

PrivacyData DeletionCookiesTerms of ServiceWebsite Sitemap

NVIDIA, the NVIDIA logo, RTX, RTX PRO, Blackwell, CUDA, NVLink, MGX, Omniverse, Spectrum-X, Quantum, BlueField, DGX and HGX are trademarks and/or registered trademarks of NVIDIA Corporation in the US and other countries. AMD, the AMD Arrow logo, EPYC, Instinct, Ryzen and Threadripper are trademarks of Advanced Micro Devices, Inc. Dell and PowerEdge are trademarks of Dell Inc. or its subsidiaries. Intel and Xeon are trademarks of Intel Corporation or its subsidiaries. AMAX, AceleMax and LiquidMax are trademarks of AMAX Engineering Corporation. All other product names, logos and brands are the property of their respective owners. We use them only to identify the products we describe, supply or support; their use does not mean their owners endorse us.

Home / Technical Notes / Technology
AIHardwareLinuxLLMs

AI Developer Laptops: Running LLMs Locally on Linux with 16 GB and 24 GB GPUs (2026 Guide)

What fits on an RTX 5080 or RTX 5090 Laptop GPU, the memory arithmetic, the open models worth running, Ubuntu 26.04 setup with the NVIDIA stack, Ollama, llama.cpp and vLLM, coding assistants, LoRA fine-tuning, security, and when to move work to a server

November 18, 2026Technology15 min read

A practical guide for engineering leads and developers who want large language models (LLMs) running on the developer's own laptop, on Linux, rather than on a shared server or a cloud GPU. It covers what fits in 16 GB and 24 GB of laptop GPU memory, the arithmetic behind that, the open models worth running in November 2026, setting up Ubuntu 26.04 with the NVIDIA stack, running models with Ollama, llama.cpp and vLLM, plugging them into coding tools, fine-tuning with LoRA, power and heat, security, and when a job should move to a server or the cloud.

Illustration of an 18-inch laptop running gpt-oss:20b locally, with the RTX 5080 Laptop GPU's 16 GB, the RTX 5090 Laptop GPU's 24 GB, up to 192 GB of DDR5 and Ubuntu 26.04 LTS.

Summary. GPU memory decides what a laptop can run. An RTX 5080 Laptop GPU (16 GB) runs today's best 8–20B open models fully on the GPU, including OpenAI's gpt-oss-20b; an RTX 5090 Laptop GPU (24 GB) adds the 27–32B class, such as Gemma 3 27B, Qwen3.8 27B, Qwen3-Coder 30B and NVIDIA's Nemotron 3.5 Lightning. Bigger models (Llama 3.3 70B, gpt-oss-120b) run by splitting the work between the GPU and system memory, slower but useful, which is why 96–192 GB of DDR5 matters. Budget for the KV cache as well as the model: at long context it can be as big as the model itself. On Linux the laptop runs the same driver, CUDA and containers as your GPU servers, so the inner development loop stays on the desk and only the heavy jobs move to a server or the cloud.

This technote accompanies our blog post AI developer laptops: why every AI team should give developers their own GPU, which makes the business case. The example machine is the Juno Neptune 18", an 18-inch Linux laptop we sell. Model sizes are from the Ollama library and GPU figures from NVIDIA, checked on 18 November 2026. Speeds depend on the model, quantisation, context and power settings, so we explain how to measure rather than quote benchmark numbers.

1. Why put the GPU on the developer's desk

Most AI development is an inner loop: change a prompt, a retrieval step, a tool definition or a few lines of code, run it, read the output, repeat. That loop runs hundreds of times a day. When each run waits for a shared GPU, a cloud instance to start or an API with rate limits, developers batch changes and learn more slowly. A local model answers in seconds, works offline and costs nothing per call.

  • Privacy. Source code, customer data and prompts never leave a company-managed device. That simplifies UK GDPR assessments and client contracts that forbid sending data to third-party APIs.
  • Realism. Developers see the real limits of open models (context length, quantisation, memory) before production does.
  • Parity. Linux, the NVIDIA driver, CUDA and containers are the same as on your GPU servers, so a prototype moves to a server without changes.
  • Availability. No queue, no quota, no outage dependency during development.

A laptop does not replace servers. Training, large fine-tunes, long evaluation runs and production serving belong on a shared GPU server or a cluster (section 11).

2. The machine

The figures that matter for AI work, for the Neptune 18" as configured in our shop. GPU figures are NVIDIA's; the rest are the manufacturer's.

PartSpecificationWhy it matters
GPU (standard)NVIDIA GeForce RTX 5080 Laptop GPU: 16 GB GDDR7, 256-bit, 7,680 CUDA cores, 1,334 AI TOPSHolds 8–20B models and their KV cache on the GPU
GPU (on request)NVIDIA GeForce RTX 5090 Laptop GPU: 24 GB GDDR7, 256-bit, 10,496 CUDA cores, 1,824 AI TOPSAdds the 27–32B class and longer context
GPU power150 W + 25 W Dynamic Boost in this chassis (NVIDIA's ranges: 80–150 W for the RTX 5080, 95–150 W for the RTX 5090 Laptop GPU)Sustained speed depends on power; run long jobs plugged in
CPUIntel Core Ultra 9 290HX Plus, 24 cores (8P + 16E), up to 5.5 GHz; 55 W base, 160 W maxTokenising, data loading and CPU offload of model layers
Memory32 to 192 GB DDR5 (2 modules at 5600 MHz up to 96 GB; 4 modules at 4000 MHz for 128 and 192 GB)Room for models bigger than GPU memory (section 4)
Storage1 × PCIe Gen 5 + 3 × PCIe Gen 4 M.2 2280, up to 16 TBModel files are 5–140 GB each; keep a library locally
Display18" 16:10, QHD+ 240 Hz (RTX 5080) or UHD+ 200 Hz (RTX 5090), 100% DCI-P3Room for code, terminal and notebook side by side
Ports2 × Thunderbolt 4 / USB-C, 2 × USB-A, HDMI and Mini DisplayPort (wired to the NVIDIA GPU), 2.5 Gb Ethernet, Wi-Fi 7Fast transfers of models and datasets
Power99 Wh battery, 330 W chargerGPU work is a plugged-in activity
SoftwareKubuntu or Ubuntu 26.04 LTS pre-installedThe same OS family as most GPU servers

3. What fits: GPU memory decides

When a model's weights, its KV cache and the runtime all fit in GPU memory, generation is fast. When they don't, part of the model runs from system memory over PCIe and from the CPU, and speed drops sharply. So the first question for any model is its size.

Bar chart of model download sizes against 16 GB and 24 GB lines. Fit 16 GB: qwen3:8b 5.2 GB, qwen3:14b 9.3 GB, gpt-oss:20b 14 GB. Need 24 GB: gemma3:27b 17 GB, qwen3.8:27b 18 GB, qwen3-coder:30b and nemotron-3.5-lightning:30b 19 GB, qwen3:32b 20 GB. Need system RAM too: llama3.3:70b 43 GB, gpt-oss:120b 65 GB, qwen3:235b 142 GB.

Model (Ollama tag)DownloadRTX 5080 (16 GB)RTX 5090 (24 GB)Good for
qwen3:8b5.2 GBOn the GPU, long contextOn the GPUFast general assistant, agents, RAG
qwen3:14b9.3 GBOn the GPUOn the GPUStronger reasoning at good speed
gpt-oss:20b (OpenAI, MoE)14 GBOn the GPU, short to medium contextOn the GPU, longer contextReasoning, tool use, general coding
gemma3:27b (Google)17 GBPartly offloadedOn the GPUWriting, multilingual, images as input
qwen3.8:27b18 GBPartly offloadedOn the GPUCoding, research and agent tasks, text and images
qwen3-coder:30b (MoE, about 3B active)19 GBExperts offloaded, still usableOn the GPUCode completion and coding agents
nemotron-3.5-lightning:30b (NVIDIA, MoE, 3B active)19–25 GBExperts offloadedSmaller quantisation on the GPUAlways-on agents, up to 1M context
qwen3:32b20 GBPartly offloadedOn the GPU, short contextDense 32B quality
llama3.3:70b (Meta)43 GBGPU + system RAM (64 GB or more): slow, fine for checksComparing with a server-class model
gpt-oss:120b (OpenAI, MoE, 5.1B active)65 GBGPU + system RAM (96 GB or more): usable thanks to MoENear-frontier open reasoning on a laptop
qwen3:235b (MoE)142 GBGPU + 192 GB RAM: experimentalTrying a very large model before buying a server

Sizes are the default Ollama tags, mostly 4-bit quantisations. Licences differ (Apache 2.0, the Gemma terms, Meta's Llama licence, NVIDIA's open model licence and others): check each model card before using a model commercially.

4. The memory arithmetic

Three things share GPU memory: the weights, the KV cache and the runtime.

Weights

weights ≈ parameters × bits per weight ÷ 8. A 14B model is 28 GB at 16-bit, about 14–15 GB at 8-bit and about 8–9 GB at 4-bit (4-bit formats store a little extra per block, so real files are slightly larger than half a byte per parameter). Quality loss at a good 4-bit quantisation is small for most development work; 8-bit is close to the original. Blackwell GPUs, including the RTX 50 Series, also support FP4 and FP8 in hardware, which runtimes such as TensorRT-LLM and vLLM can use.

KV cache

For every token in the context, each layer stores a key and a value: KV bytes per token = 2 × layers × KV heads × head size × bytes per value. For Llama 3.1 8B that is 2 × 32 × 8 × 128 × 2 = 131,072 bytes (128 KiB) at 16-bit. It grows linearly with context length and with the number of requests served at once.

Stacked bars for Llama 3.1 8B at 4-bit: at 8K tokens, 4.9 GB weights, 1.1 GB KV cache and 1 GB runtime make 7.0 GB. At 32K, 4.3 GB of cache makes 10.2 GB. At 128K, 17.2 GB of cache makes 23.1 GB, over 16 GB and just under 24 GB.

So an 8B model that is 5 GB on disk needs about 23 GB at its full 128K context. Three ways to fit:

  • Use the context you need. Most coding and RAG work is fine at 16–32K. Runtimes reserve the cache for the context you set, so set it.
  • Quantise the cache. An 8-bit KV cache halves it with little quality loss; 4-bit quarters it. In Ollama set OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0; in llama.cpp use --cache-type-k q8_0 --cache-type-v q8_0.
  • Prefer models with grouped-query or other compact attention. Fewer KV heads mean a smaller cache; the model card lists them.

Runtime

Allow 1–2 GB for the CUDA context, activations and buffers, and remember the desktop uses some GPU memory too when displays run from the NVIDIA GPU. Check with nvidia-smi before loading a model.

Speed, and why system memory still matters

Generating each token reads every active weight once, so generation speed is limited mainly by memory bandwidth: GDDR7 on the GPU is several times faster than DDR5 in the laptop. That is why a model that spills into system memory slows down, and why mixture-of-experts (MoE) models are the exception: gpt-oss-120b has 117B parameters but only about 5.1B are active for each token, so keeping the shared layers on the GPU and the experts in system memory still gives usable speed. For that kind of work, more and faster system memory helps: two modules at 5600 MHz are faster than four at 4000 MHz, so choose 96 GB (2 × 48 GB) unless you need 128 or 192 GB for capacity.

Measure on your own work: ollama run <model> --verbose prints prompt-processing and generation rates in tokens per second, and llama-bench does the same for llama.cpp.

5. Setting up Ubuntu 26.04 for AI work

The Linux AI stack in seven layers: your work; model runtimes such as Ollama, llama.cpp, LM Studio, vLLM, SGLang and TensorRT-LLM; frameworks such as PyTorch, Transformers, PEFT and Unsloth; Docker or Podman with the NVIDIA Container Toolkit; the NVIDIA driver, CUDA, cuDNN and NCCL; Ubuntu 26.04 LTS; and the hardware.

Machines from us arrive with the NVIDIA driver installed and checked. To check or rebuild one:

# Is the NVIDIA GPU visible, and which driver is in use?
nvidia-smi

# Ubuntu's recommended NVIDIA driver (signed for Secure Boot)
sudo ubuntu-drivers list
sudo ubuntu-drivers install
sudo reboot

# Laptops with two GPUs: run everything on the NVIDIA GPU, or only what asks for it
prime-select query
sudo prime-select nvidia       # or: on-demand

With Secure Boot on, a driver built locally needs its signing key enrolled (MOK) at the next boot; Ubuntu's packaged drivers avoid that step. Keep the driver from Ubuntu's archive or NVIDIA's repository, not both.

Containers with the GPU

Containers keep CUDA versions per project and match what runs on servers. Install Docker (or Podman), then the NVIDIA Container Toolkit from NVIDIA's apt repository following NVIDIA's install guide, and wire it in:

sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Test: the container should list the laptop GPU
docker run --rm --gpus all ubuntu nvidia-smi

Python

Use one virtual environment per project (uv or venv) and install PyTorch with the CUDA build its install page recommends for your driver. Check it with python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))".

6. Running models

Ollama: the quickest start

curl -fsSL https://ollama.com/install.sh | sh
ollama run gpt-oss:20b            # downloads once, then runs locally
ollama ps                         # shows how much of the model is on the GPU vs the CPU

ollama ps is the first check: "100% GPU" is what you want. If it shows a CPU share, reduce the context, quantise the KV cache or pick a smaller model. Ollama listens on 127.0.0.1:11434 and serves an OpenAI-compatible API at /v1, so most tools can use it unchanged.

llama.cpp: full control

# All layers on the GPU, 32K context, 8-bit KV cache, OpenAI-compatible server on :8080
llama-server -m qwen3-14b-q4_k_m.gguf --n-gpu-layers 99 --ctx-size 32768 \
  --cache-type-k q8_0 --cache-type-v q8_0 --host 127.0.0.1 --port 8080

# A MoE model bigger than GPU memory: keep shared layers on the GPU, put experts in system RAM
# (recent llama.cpp builds; raise the number until it fits)
llama-server -m gpt-oss-120b.gguf --n-gpu-layers 99 --n-cpu-moe 24 --ctx-size 16384

vLLM: closest to production serving

pip install vllm
vllm serve Qwen/Qwen3-8B --max-model-len 16384 --gpu-memory-utilization 0.85
# OpenAI-compatible API on http://127.0.0.1:8000/v1

vLLM pre-allocates GPU memory for its KV cache, so on a 16 GB laptop GPU keep models small and set --max-model-len. Its value is parity: the same server, flags and API as on your GPU servers.

LM Studio is a desktop app for the same job, with a model browser and a local API, useful for people who don't live in a terminal.

7. Coding assistants and agents on a local model

Most IDE assistants, coding agents and agent frameworks accept an OpenAI-compatible base URL. Point them at http://127.0.0.1:11434/v1 (Ollama), :8080/v1 (llama.cpp) or :8000/v1 (vLLM). Practical choices in November 2026:

  • Completion while typing: a small, fast model (qwen3:8b or similar). Latency matters more than size.
  • Chat and refactoring: gpt-oss:20b on 16 GB; qwen3-coder:30b or qwen3.8:27b on 24 GB.
  • Agents with tools: models trained for tool calling (gpt-oss, Qwen3 family, Nemotron 3.5 Lightning), with enough context for the tool results, typically 32K or more.

Keep a hosted frontier model for the hardest problems if your policy allows it; the local model handles the everyday volume privately. Many teams route by task: local by default, hosted on request.

8. Fine-tuning on a laptop

Full fine-tuning of even a 7B model needs far more than 24 GB. LoRA and QLoRA train small adapter matrices on top of frozen weights (QLoRA keeps the base model in 4-bit), which brings fine-tuning onto a laptop:

  • 16 GB (RTX 5080): QLoRA on 7–8B models at modest sequence lengths; up to about 14B with memory-saving tooling and short sequences.
  • 24 GB (RTX 5090): QLoRA on 14B comfortably, and up to about 30B with short sequences.
  • Beyond that: 70B fine-tunes and full fine-tunes go to a server (section 11).

Use Hugging Face PEFT and TRL, or Unsloth for lower memory use, with gradient checkpointing and a batch size of one or two plus gradient accumulation. These ranges depend on sequence length, rank and tooling: start small, watch nvidia-smi and grow. The laptop is ideal for checking data formats, hyperparameters and evaluation scripts on a small run before paying for a long one.

9. Power, heat and battery

  • Plug in for GPU work. On battery the GPU runs at reduced power; the 330 W charger is what sustains 150 W+ GPU loads.
  • Watch it: nvidia-smi dmon -s pucm streams power, utilisation, clocks and memory; nvidia-smi -q -d POWER,TEMPERATURE shows limits.
  • Long jobs: keep vents clear, use the performance power profile, and expect fans. For multi-hour jobs a desktop or server is the better tool.

10. Security and governance

  • Encrypt the disk. Full-disk encryption (LUKS) protects code, data and model files if the laptop is lost; we set it up before dispatch when ordered.
  • Keep model servers local. Ollama, llama.cpp and vLLM have no user authentication by default. Bind them to 127.0.0.1; never expose them on 0.0.0.0 on office or public Wi-Fi without a proxy that authenticates.
  • Know where models come from. Download from the publisher's official repositories, prefer safetensors and GGUF over formats that can run code when loaded, and keep a list of approved models and their licences.
  • Manage the device. Updates, disk encryption and screen locks under your normal device management; data processed locally is still personal data under UK GDPR.

11. When to move work off the laptop

Three columns: on the laptop, the daily inner loop of prompts, coding assistants, RAG prototypes, tests, small LoRA fine-tunes and offline work; on a shared GPU server, weekly team evaluations, 30 to 70B fine-tunes, a team endpoint, long-context jobs and models over 24 GB; on a cluster or cloud, burst work such as pre-training, production serving, load tests and multi-node jobs.

SignalMove to
The model needs more than 24 GB at full speed, or long context for many usersA workstation or server with RTX PRO 6000 (96 GB) or a DGX system
A job runs for hours, or blocks the developer's machineA shared GPU server, scheduled
Several developers need the same model all dayOne team inference endpoint instead of many copies
Multi-GPU training or production servingCluster or cloud, with UK data residency where needed

Not ready to buy? Workstation Rentals rents AI laptops and GPU workstations by the week or month, and GPU Leasing explains leasing them.

12. Which configuration for which developer

RoleGPUSystem memoryStorage
Application developer using AI (RAG, agents, assistants)RTX 5080, 16 GB64 GB2 TB
AI engineer (evaluation, fine-tuning, 27–32B models)RTX 5090, 24 GB96 GB (2 × 48 GB, fastest)4 TB
Researcher trying very large MoE models locallyRTX 5090, 24 GB128–192 GB4 TB or more
Data scientist (notebooks, classic ML, small models)RTX 5080, 16 GB64–96 GB2–4 TB

13. Frequently asked questions

Can a laptop really run a 120B model? gpt-oss-120b, yes, with 96 GB or more of system memory, because only about 5.1B parameters are active per token. Dense 70B models also run, but slowly; use them to compare answers, not for interactive work.

16 GB or 24 GB? 16 GB covers the 8–20B class, which handles most development work. Choose 24 GB if you work with 27–32B models, long contexts or fine-tuning above 8B.

Why Linux rather than Windows? Servers, containers and most AI tooling are Linux-first. Developing on the same OS removes a class of "works on my machine" problems.

Does a local model replace hosted frontier models? Not for the hardest tasks. It takes the everyday volume privately and cheaply; keep a hosted model for what needs it, where policy allows.

What about Apple silicon? Macs with large unified memory run big models well with MLX, but without CUDA. If your servers are NVIDIA and Linux, an NVIDIA Linux laptop keeps the stack identical.

Sources

  • NVIDIA, GeForce RTX 50 Series laptop specifications (CUDA cores, AI TOPS, memory, interface width, GPU power range), checked 18 November 2026.
  • Ollama library pages for qwen3, gpt-oss, gemma3, qwen3.8, qwen3-coder, nemotron-3.5-lightning and llama3.3 (download sizes and descriptions), checked 18 November 2026.
  • OpenAI, gpt-oss model card (117B total and about 5.1B active parameters for gpt-oss-120b).
  • Meta, Llama 3.1 8B configuration (32 layers, 8 KV heads, head size 128), used for the KV cache example.
  • Juno Computers, Neptune 18" specifications; Workstation shop listing (configuration, price and dispatch time).
  • Commands follow the Ubuntu, NVIDIA Container Toolkit, Ollama, llama.cpp and vLLM documentation; flags change between releases, so check the current docs.

Acknowledgement of rights: NVIDIA, GeForce, RTX, CUDA, cuDNN, NCCL, TensorRT, DGX, Blackwell and Nemotron are trademarks of NVIDIA Corporation. Intel and Core Ultra are trademarks of Intel Corporation. Ubuntu and Kubuntu are trademarks of Canonical Ltd. Juno and Neptune are names of Juno Computers Ltd. Llama is a trademark of Meta Platforms, Inc.; Gemma of Google LLC; Qwen of Alibaba Cloud; gpt-oss and OpenAI of OpenAI. Other names are the property of their respective owners. This article is independent and not endorsed by any of these companies.

Share this article

More in Technology

Testing ML and LLM Systems You Can't Prove Correct: A Python Evidence Kit

Testing ML and LLM Systems You Can't Prove Correct: A Python Evidence Kit

Evidence instead of proofs, in about 400 lines of Python with real output: slice metrics with bootstrap intervals and a pass/fail contract, Hypothesis property and metamorphic tests, calibration and conformal prediction, Pydantic contracts and regression evals for LLM output, drift checks, a CI gate, and exact verification of a small neural network

Read more
TorchCurves in Practice: Learnable B-spline and Legendre Curves in PyTorch

TorchCurves in Practice: Learnable B-spline and Legendre Curves in PyTorch

Install torchcurves, fit and size a curve, choose B-spline or Legendre and the input map, build a monotone calibration curve and an additive model on tabular data, compare with SciPy, scikit-learn, a KAN and an MLP, export coefficients to JSON, test and run it in production, with real outputs

Read more
Run Agentic, Voice and Vision Models On-Device with Cactus: A Hands-On Guide (2026)

Run Agentic, Voice and Vision Models On-Device with Cactus: A Hands-On Guide (2026)

Install cactus-compute, make tool calls, gate on confidence with a fallback to a bigger model, extract structured data, run speech and vision, measure latency and memory, and ship an offline container, with every example run and a comparison with llama.cpp, ONNX Runtime, ExecuTorch, MLC LLM and TensorRT Edge-LLM

Read more