Coder Radio Discord
The Mad Botter Data Platform
Mike's Blog

Alice Promo

1. AI on Red Hat Enterprise Linux (RHEL)

Trust and Stability: RHEL provides the mission-critical foundation needed for workloads where security and reliability cannot be compromised.

Predictive vs. Generative: Acknowledging the hype of GenAI while maintaining support for traditional machine learning algorithms.

Determinism: The challenge of bringing consistency and security to emerging AI technologies in production environments.

1. Rama-Llama & Containerization

Developer Simplicity: Rama-Llama helps developers run local LLMs easily without being "locked in" to specific engines; it supports Podman, Docker, and various inference engines like Llama.cpp and Whisper.cpp.

Production Path: The tool is designed to "fade away" after helping package the model and stack into a container that can be deployed directly to Kubernetes.

Behind the Firewall: Addressing the needs of industries (like aircraft maintenance) that require AI to stay strictly on-premises.

1. Enterprise AI Infrastructure

Red Hat AI: A commercial product offering tools for model customization, including pre-training, fine-tuning, and RAG (Retrieval-Augmented Generation).

Inference Engines: James highlights the difference between Llama.cpp (for smaller/edge hardware) and vLLM, which has become the enterprise standard for multi-GPU data center inferencing.

Plus the state of browser extension development, the value of non-technical advice, and your feedback.

Links:

• Feedback: good mic for voice recording? — I'm looking for a good mic for voice recording since I will be a guest on a podcast soon. Since you sound good in your shows, can you share what mics you are using?
• Amazon.com: Audio-Technica ATR2500-USB Cardioid Condenser USB Microphone: Musical Instruments — Side-address condenser microphone with USB output for easy connection to your computer.
• Google and Mozilla are failing to support browser extension developers · Armin Sebastian — We are witnessing the failure of browser vendors to recognize the value of our labor and the important role it plays in a healthy browser ecosystem.
• Half of all Google Chrome extensions have fewer than 16 installs — All in all, about 50% of all Chrome extensions have fewer than 16 installs, meaning that half of the Chrome extension ecosystem is actually more of a ghost town, according to a recent scan of the entire Chrome Web Store conducted by Extension Monitor.
• All the best engineering advice I stole from non-technical people — As I focus on becoming a better manager of engineers, I have been reflecting more and more on the advice that produced a 10X boost in my abilities at that same stage. More often than not the best advice, the things that stuck with me, came from people who had no background at all in software.
• Overview of Docker Compose | Docker Documentation — Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.
• Podman — What is Podman? Podman is a daemonless container engine for developing, managing, and running OCI Containers on your Linux System. Containers can either be run as root or in rootless mode. Simply put: `alias docker=podman`.
• Buildah — A tool that facilitates building OCI container images.
• skopeo — skopeo is a command line utility that performs various operations on container images and image repositories.
• From 30 to 230 docker containers per host — I could not find much information on the interwebs how many containers you can run per host. So here are mine and the issues we ran into along the way.