RoboTF AI

@UCHkAlM1-h5kvInnaCpTPxSg - 3.9K subscribers

Just another engineer fooling around in a GPU powered Kubernetes based AI/LLM Lab. We play around with different GPUs, different Large Language Models, run tests and see what we can learn together. I am not an expert, and make bad decisions, please do not follow anything I do on this channel.

Home Videos Live Playlists

Building Ubuntu Server for AI LLMs Part 4: LLM Tools companion app for LocalAI, Model Config Editing

Building Ubuntu Server for AI LLMs Part 4: LLM Tools companion app for LocalAI, Model Config Editing RoboTF AI

641 views - 10 months ago

Updates, News, Local AI 2.27.0, Sycl Support, Mistral Small 24B coffee session ☕️

Updates, News, Local AI 2.27.0, Sycl Support, Mistral Small 24B coffee session ☕️ RoboTF AI

386 views - 10 months ago

LocalAI LLM Testing: Deepseek R1 Distill Llama 70B Deepdive VRAM at different context levels!

LocalAI LLM Testing: Deepseek R1 Distill Llama 70B Deepdive VRAM at different context levels! RoboTF AI

1.5K views - 1 year ago

Nvidia cutting support for Maxwell, Pascal, and Volta cards in a future release! 🪦

Nvidia cutting support for Maxwell, Pascal, and Volta cards in a future release! 🪦 RoboTF AI

1.1K views - 1 year ago

Behind the Scenes: Unboxing MinisForum MS-01, Major Network upgrades Sodola switches, Deepseek R1!

Behind the Scenes: Unboxing MinisForum MS-01, Major Network upgrades Sodola switches, Deepseek R1! RoboTF AI

749 views - 1 year ago

Building Ubuntu Server for AI LLMs from scratch Part 3: ComfyUI, Open WebUI, Flowise, n8n, and more!

Building Ubuntu Server for AI LLMs from scratch Part 3: ComfyUI, Open WebUI, Flowise, n8n, and more! RoboTF AI

1.5K views - 1 year ago

Building Ubuntu Server for AI and LLMs from scratch Part 2: Nvidia Cuda Drivers, Toolkit, LocalAI!

Building Ubuntu Server for AI and LLMs from scratch Part 2: Nvidia Cuda Drivers, Toolkit, LocalAI! RoboTF AI

3.1K views - 1 year ago

Building Ubuntu Server for AI and LLMs from scratch Part 1: Hardware build, Open Air Case, Multi-GPU

Building Ubuntu Server for AI and LLMs from scratch Part 1: Hardware build, Open Air Case, Multi-GPU RoboTF AI

4.9K views - 1 year ago

Cutting up a Dell R720xd to add 2x Nvidia Tesla M40 24GB GPUs - Llama 3.3 70B, QwQ 32B, and more!

Cutting up a Dell R720xd to add 2x Nvidia Tesla M40 24GB GPUs - Llama 3.3 70B, QwQ 32B, and more! RoboTF AI

1.5K views - 1 year ago

3 Uncensored AI comedians do the 12 Days of Christmas and roast AI Engineers with homelabs! 🎄

3 Uncensored AI comedians do the 12 Days of Christmas and roast AI Engineers with homelabs! 🎄 RoboTF AI

170 views - 1 year ago

LocalAI LLM Testing: Llama 3.3 70B Q8, Multi GPU 6x A4500, and PCIe Bandwidth during inference

LocalAI LLM Testing: Llama 3.3 70B Q8, Multi GPU 6x A4500, and PCIe Bandwidth during inference RoboTF AI

3.8K views - 1 year ago

3 AI Comedians talk about Running a GPU Homelab for AI and Machine Learning using uncensored models!

3 AI Comedians talk about Running a GPU Homelab for AI and Machine Learning using uncensored models! RoboTF AI

281 views - 1 year ago

Project: Count your tokens for Huggingface models! 🪙 AutoTikTokenizer and RoboTF LLM Token Estimator

Project: Count your tokens for Huggingface models! 🪙 AutoTikTokenizer and RoboTF LLM Token Estimator RoboTF AI

238 views - 1 year ago

Mistral 7B LLM AI Leaderboard: Gigabyte AMD Radeon RX 7600 XT 16GB uses ROCm to hit leaderboard!

Mistral 7B LLM AI Leaderboard: Gigabyte AMD Radeon RX 7600 XT 16GB uses ROCm to hit leaderboard! RoboTF AI

3.7K views - 1 year ago

LLM Testing Behind the Scenes AMD Radeon 7600 XT ROCm set flash_attention to true! 🤦‍♂️

LLM Testing Behind the Scenes AMD Radeon 7600 XT ROCm set flash_attention to true! 🤦‍♂️ RoboTF AI

1.8K views - 1 year ago

LLM Testing Behind the Scenes Unboxing Gigabyte AMD Radeon 7600 XT 16GB ROCm vs Vulkan battle!

LLM Testing Behind the Scenes Unboxing Gigabyte AMD Radeon 7600 XT 16GB ROCm vs Vulkan battle! RoboTF AI

2.7K views - 1 year ago

Mistral 7B LLM AI Leaderboard: Unboxing AsRock Intel ARC A770 16GB and running it through our tests!

Mistral 7B LLM AI Leaderboard: Unboxing AsRock Intel ARC A770 16GB and running it through our tests! RoboTF AI

5K views - 1 year ago

Mistral 7B LLM AI Leaderboard: Apple M1 Max MacBook Pro uses Metal to take on GPU's for their spot!

Mistral 7B LLM AI Leaderboard: Apple M1 Max MacBook Pro uses Metal to take on GPU's for their spot! RoboTF AI

4.5K views - 1 year ago

Mistral 7B LLM AI Leaderboard: i5-12450H 32GB DDR4 Mini PC takes on the leaderboard?

Mistral 7B LLM AI Leaderboard: i5-12450H 32GB DDR4 Mini PC takes on the leaderboard? RoboTF AI

1.5K views - 1 year ago

Halloween Stories via Streamlit, Langchain, Python, and LocalAI (or OpenAI) with Text to Speech!

Halloween Stories via Streamlit, Langchain, Python, and LocalAI (or OpenAI) with Text to Speech! RoboTF AI

271 views - 1 year ago

Mistral 7B LLM AI Leaderboard: Nvidia RTX A4500 GPU 20GB Where does prosumer/enterprise land?

Mistral 7B LLM AI Leaderboard: Nvidia RTX A4500 GPU 20GB Where does prosumer/enterprise land? RoboTF AI

1.1K views - 1 year ago

LocalAI LLM Tuning: WTH is Flash Attention? What are the effects on memory and performance? Llama3.2

LocalAI LLM Tuning: WTH is Flash Attention? What are the effects on memory and performance? Llama3.2 RoboTF AI

641 views - 1 year ago

Mistral 7B LLM AI Leaderboard: Unboxing an Nvidia RTX 4090 Windforce 24GB Can it break 100 TPS?

Mistral 7B LLM AI Leaderboard: Unboxing an Nvidia RTX 4090 Windforce 24GB Can it break 100 TPS? RoboTF AI

3.4K views - 1 year ago

Mistral 7B LLM AI Leaderboard: The King of the Leaderboard? Nvidia RTX 3090 Vision 24GB throw down!

Mistral 7B LLM AI Leaderboard: The King of the Leaderboard? Nvidia RTX 3090 Vision 24GB throw down! RoboTF AI

1K views - 1 year ago

Mistral 7B LLM AI Leaderboard: Unboxing an Nvidia RTX 4070Ti Super 16GB and giving it run!

Mistral 7B LLM AI Leaderboard: Unboxing an Nvidia RTX 4070Ti Super 16GB and giving it run! RoboTF AI

2.5K views - 1 year ago

Mistral 7B LLM AI Leaderboard: GPU Contender Nvidia RTX 4060Ti 16GB

Mistral 7B LLM AI Leaderboard: GPU Contender Nvidia RTX 4060Ti 16GB RoboTF AI

1.4K views - 1 year ago

Mistral 7B LLM AI Leaderboard: GPU Contender Nvidia Tesla M40 24GB

Mistral 7B LLM AI Leaderboard: GPU Contender Nvidia Tesla M40 24GB RoboTF AI

1.2K views - 1 year ago

Mistral 7B LLM AI Leaderboard: GPU Contender Nvidia GTX 1660

Mistral 7B LLM AI Leaderboard: GPU Contender Nvidia GTX 1660 RoboTF AI

1K views - 1 year ago

Mistral 7B LLM AI Leaderboard: Rules of Engagement and first GPU contender Nvidia Quadro P2000

Mistral 7B LLM AI Leaderboard: Rules of Engagement and first GPU contender Nvidia Quadro P2000 RoboTF AI

543 views - 1 year ago

Mistral 7B LLM AI Leaderboard: Baseline Testing Q3,Q4,Q5,Q6,Q8, and FP16 CPU Inference i9-9820X

Mistral 7B LLM AI Leaderboard: Baseline Testing Q3,Q4,Q5,Q6,Q8, and FP16 CPU Inference i9-9820X RoboTF AI

588 views - 1 year ago

Mistral 7B LLM AI Leaderboard: Baseline Testing Q3 CPU Inference i9-9820X

Mistral 7B LLM AI Leaderboard: Baseline Testing Q3 CPU Inference i9-9820X RoboTF AI

461 views - 1 year ago

LocalAI LLM Testing: Part 2 Network Distributed Inference Llama 3.1 405B Q2 in the Lab!

LocalAI LLM Testing: Part 2 Network Distributed Inference Llama 3.1 405B Q2 in the Lab! RoboTF AI

2.5K views - 1 year ago

LocalAI LLM Testing: Distributed Inference on a network? Llama 3.1 70B on Multi GPUs/Multiple Nodes

LocalAI LLM Testing: Distributed Inference on a network? Llama 3.1 70B on Multi GPUs/Multiple Nodes RoboTF AI

11K views - 1 year ago

LocalAI LLM Testing: Llama 3.1 8B Q8 Showdown - M40 24GB vs 4060Ti 16GB vs A4500 20GB vs 3090 24GB

LocalAI LLM Testing: Llama 3.1 8B Q8 Showdown - M40 24GB vs 4060Ti 16GB vs A4500 20GB vs 3090 24GB RoboTF AI

21.8K views - 1 year ago

LocalAI LLM Testing: How many 16GB 4060TI's does it take to run Llama 3 70B Q4

LocalAI LLM Testing: How many 16GB 4060TI's does it take to run Llama 3 70B Q4 RoboTF AI

45.8K views - 1 year ago

LocalAI LLM Testing: Can 6 Nvidia A4500's Take on the WizardLM 2 8x22b?

LocalAI LLM Testing: Can 6 Nvidia A4500's Take on the WizardLM 2 8x22b? RoboTF AI

1.6K views - 1 year ago

LocalAI LLM Testing: Viewer Questions using mixed GPUs, and what is Tensor Splitting AI lab session

LocalAI LLM Testing: Viewer Questions using mixed GPUs, and what is Tensor Splitting AI lab session RoboTF AI

5K views - 1 year ago

What's on the Robotf-AI Workbench Today?

What's on the Robotf-AI Workbench Today? RoboTF AI

870 views - 1 year ago

LocalAI LLM Testing: i9 CPU vs Tesla M40 vs 4060Ti vs A4500

LocalAI LLM Testing: i9 CPU vs Tesla M40 vs 4060Ti vs A4500 RoboTF AI

30.2K views - 1 year ago

LocalAI Testing: Viewer Question LLM context size, & quant testing with 2x 4060 Ti's 16GB VRAM

LocalAI Testing: Viewer Question LLM context size, & quant testing with 2x 4060 Ti's 16GB VRAM RoboTF AI

2.8K views - 1 year ago

LocalAI LLM Single vs Multi GPU Testing scaling to 6x 4060TI 16GB GPUS

LocalAI LLM Single vs Multi GPU Testing scaling to 6x 4060TI 16GB GPUS RoboTF AI

24.3K views - 1 year ago