Mac Mini on Melabit

LM Studio, an LLM on your computer

Mon, 31 Mar 2025 06:00:00 +0000

As some of you may already know, I use LLMs (Large Language Models) for what they’re really good at, but I’m pretty skeptical about whether they’re truly intelligent or can solve any problem, as the folks at OpenAI, Microsoft, Google, and Meta keep telling us every day. They’ve invested a ton of money in LLMs, and they obviously have a big stake in getting everyone to use them all the time.

LLMs, or (a lot of) statistics at work

LLMs don’t really understand the meaning of the texts they generate. All they do is predict each word, picking it from the billions of documents they’ve analyzed. The statistical models they’re based on can’t verify the truthfulness of information, nor do they have a real awareness of the sentence’s context, but rely exclusively on the probability of each word’s appearance in a text.

This often leads to errors, which are sometimes just ridiculous, but can also become very dangerous.

Basically, LLMs are students who have memorized the lesson, but can’t go beyond what’s written in the textbook. And sometimes they even get the page wrong!

Once these limitations are understood, LLMs prove very useful if we consider them computer tools like any other and exploit their enormous language processing capacity to analyze a text, summarize or expand it, translate it into other languages, and so on.

LLMs on the web

The most common way to use an LLM is in the form of a chatbot, that is, an interactive system to which we asks questions and gets answers, simulating a normal conversation between humans. In most cases, this happens through a web interface like that of ChatGPT, Copilot, Gemini, Claude or Mistral AI. Not forgetting DeepSeek, which has generated so much controversy in the last two months (and which we will use shortly).

Some chatbots also have native applications for desktop or mobile devices (such as ChatGPT or Claude), but I don’t think I’m wrong in saying that they are little more than interfaces for quick access to the web version.

In all cases, the generation process always takes place on a remote server in a data center located somewhere out there. An almost always irrelevant detail, but one that can create many problems if we use an LLM to process personal data or confidential information.

Local LLMs

But we are not forced to use an LLM on the web! Many LLMs can be used locally, directly on our computer. And we don’t even need to mess around with the Terminal, as with ollama, but we can comfortably stay in the familiar environment of a normal program for our Mac, or for a PC running Windows or Linux.

There are an infinite number of these programs, but they all have common traits: once the program is installed, we first need to download one (or more) models to use;¹ after which we can interact with the model directly through the program’s interface, without even a byte of what we do leaving our computer for the increasingly hostile world of the network.

LM Studio

But which program to choose among the many available, like Msty, LM Studio, GPT4All, Jan, LibreChat?

I don’t know about you, but LM Studio made a good impression on me right from the beginning. LM Studio runs on the three major operating systems, but to use it on macOS you need an Apple Silicon processor, while on Linux you need an Intel processor. Only the Windows version supports both Intel and ARM processors (but I’d be really curious to see how it works under Windows for ARM).

The current version is 0.3.14 and the installation on the Mac requires, as usual, double-clicking on the .dmg file and dragging the program icon into the Applications folder. On the Mac, the application takes up a good 1.26 GB, so it’s not exactly a light program. After all, it’s based on Electron, and that explains everything.

On Linux, the installation process is very similar, because the program is available in the AppImage format. This means that, in addition to the executable itself, it includes all the necessary support libraries and files, just like it has been done on macOS for decades. For Windows, however, you have to run the usual installer, which will spread junk in every corner of the operating system.

Discovering LM Studio

Having reached this point, all that remains is to run LM Studio for the first time. The very first thing to do is to download a model, so that we can start using the program.

I opt for the default suggested model, which happens to be a distilled version of DeepSeek with only 7 billion parameters (the full model has 100 times more). The model takes up 4.7 GB and is downloaded to the hidden directory ~/.lmstudio/models/.

The download process is not fast at all, so we need a good dose of patience.

As suggested by the program itself, while the download continues I start to explore the interface, which is the classic one we are now used to, with a large central area dedicated to the dialogue with the chatbot, while the sidebar houses some icons that, from top to bottom, allow to activate the Chat or Developer mode and to check which models are installed on our machine and which other models are available.

The last icon in the lower left corner shows the model download status. Clicking on it will display the amount of data already downloaded, the estimated completion time and the possibility to pause or stop the download if necessary. If the download process times out, it is also possible to resume it from where it left off.

This happened to me several times with the selected model, so check it from time to time. Once the model download is complete, we need to load it into LM Studio to be able to use it. We can use one of the two Load Model buttons to do this, so it’s hard to forget.

Once the first model is loaded, the LM Studio interface changes slightly, showing some sample prompts in the center of the window. In the top bar, the name of the loaded model appears in the center, flanked by two icons: the one on the left allows you to configure the model parameters, while the one on the right allows you to replace the current model with another. An icon on the right that looks like a glass flask provides access to the advanced configuration settings.

The bottom bar of the program displays some useful information, on the left the LM Studio version and on the right the amount of RAM and CPU in use.

The last icon on the right of the bottom bar allows to access the program settings. And it’s also the only way to do it because, strangely, there is no menu item dedicated to the program settings.

The same bottom bar also allows to select the usage mode of the program, choosing between User (which hides the side icons), Power User (the default mode) and Developer (which apparently does not change the interface).

Putting LM Studio to the test

All that remains is to put LM Studio (and DeepSeek) to the test, perhaps using one of the example prompts. I don’t need to ask an AI what the capital of France is, the Rubik’s cube is too 80s, let’s see how it handles mathematics.

Deepseek thinks about it for a minute, but then comes up with a nice proof of the Pythagorean theorem based on proportions. As I explicitly asked, it also formats the equations in LaTeX, which is always a good thing.

And if I click on the little triangle in the Thoughts box, it even shows the reasoning it followed to arrive at that proof. Not bad at all!

So far I’ve been using a Mac Mini M1 with 16 GB of RAM. But I also have a Mac Studio M2 Ultra with 192 GB of RAM available, how long will it take?

Making a comparison is not easy because, even if you use exactly the same prompt, the answers from the Mac Mini and the Mac Studio will always be different, demonstrating the purely statistical nature of the reasoning done by LLMs. You can see this in the two images below, where the Mini is recognizable by the light theme I’ve used so far, while the Mac Studio is configured to use a dark theme.

If the same question is repeated five times on the Mac Mini, the response time varies between 80 and 120 seconds, with a constant speed of 10-11 tokens per second²; on the Mac Studio, on the other hand, the responses are generated in 15-45 seconds, with a speed of 60-70 tokens per second. So, roughly speaking, the text generation speed on the Mac Studio is about 6-7 times faster than on the Mac Mini.

Lowering the temperature

But I’m stubborn, and to make the comparison more accurate, I want the two Macs to always give me the same answer. To do this, I have to click on the glass Flask icon in the top right corner and set Temperature to zero (the default is 0.8).

Under these conditions, the Mac Mini takes 75 to 140 seconds to process its responses, while the Mac Studio does it in 11-22 seconds. The strange thing is that although the answers are always the same, the number of tokens generated changes each time, so even in this case the Mac Mini’s speed is always 10-11 tokens per second, while the Mac Studio is less constant, generating 60-80 tokens per second.

For a more scientific study, it would be necessary to work under much more controlled conditions, but for now we can be content to say that, even with this configuration, the Mac Studio is at least 6-7 times faster than the Mac Mini.

What about ChatGPT?

For comparison, how long does ChatGPT take? When I ask it the usual question about the Pythagorean theorem with the Reason option enabled, ChatGPT gives me two different, very detailed and well-written answers. And it takes just 28 seconds to do it, which is a time comparable to that of the Mac Studio.

I have to admit that I’m impressed by the speed of ChatGPT. It’s true that ChatGPT runs on servers that have nothing to do with my two Macs, but it’s also true that they have to respond to thousands of requests in parallel with mine, while the Macs are only at my service.

Conclusions (for now)

In any case, the goal of this article is not to compare ChatGPT with other LLMs, but to verify if it is possible to use an LLM directly on our computer, without sending confidential information over the network.

It is also true that both the Mac Mini and the Mac Studio don’t even notice that LM Studio is doing its processing, so there should be room for optimization. But we will see this (and more) in the next article.

A “model” is the set of weights, mathematical rules and neural structures that constitute a specific LLM. ↩︎
A “token” is the basic unit of text processed by the model and can correspond to a whole word, a part of a word, a space or a punctuation mark. The choice to represent a word with a single token or with multiple tokens depends on its frequency of use. If a word is very common, a single token will be used to represent it. On the other hand, if a word is rarely used, it doesn’t make sense to add the whole word to the model’s vocabulary, but it will be broken down into more common subwords. ↩︎

Apple, we have a problem: a look at macOS Sonoma bugs (update)

Fri, 31 May 2024 06:00:00 +0000

– Image generated by Microsoft Designer AI.

A couple of months ago, I listed some more or less serious bugs in Sonoma that I noticed while getting familiar with the latest version of macOS, first on the new Mac Studio M2 Ultra and then on the household Mac Mini M1.

At that time, I was using macOS Sonoma 14.3, which I soon updated on the Mini to version 14.3.1. With this minor release, Apple fixed a couple of the bugs I described, specifically the one about emptying the Trash into a random Space and the issue that prevented giving decent names to PDF files generated by the Print function.

Since I primarily use the Print (virtual) function for PDFs in Firefox, I suspected this bug might be tied to the browser rather than the operating system. But that’s not the case. On the updated Mini, the issue disappeared, but the Mac Studio, which stayed on version 14.3 longer, continued to bother me despite having the exact same browser as the Mini.

Now both machines have macOS 14.5 installed, and the PDF naming issue has disappeared on the Mac Studio as well. However, all the other bugs I reported still happily persist. In fact, with this post, I’m adding two new bugs, one of which immediately climbed to the top of my list of annoyances.

The first bug was pointed out to me by Mimmo (thanks!) back in the days of Sonoma 14.3. It involved Safari occasionally hiding the text within a selection, as shown in the image below.

It’s not a major issue, I admit, but it’s surprising in a product from a company as detail-oriented as Apple. Apparently, Cupertino thought so too, as it seems to have been fixed in the transition from version 14.3 to 14.5. That’s better.

The other bug is so obvious that I can’t understand how I missed it before: every time I open a folder with a large number of files, Sonoma’s Finder takes an eternity to display them.

For years, I’ve used an external USB drive to store everything I want to keep but don’t need daily. Some folders contain thousands and thousands of files, but until now, viewing their contents was always instantaneous, or nearly so. And this was even with a mechanical drive—high quality but still mechanical.

With Sonoma, everything suddenly changed: every time I open a very full folder, I have to wait several dozen seconds before seeing its contents. And when I say dozens of seconds, I mean 40, 50, even 60 seconds for a folder with 1,000–2,000 items. That’s a lot, sure, but it’s not the end of the world.

What’s strange is that until Monterey, with the same external drive, viewing was practically instantaneous, so I don’t think it’s a disk or connection speed issue.

To rule out doubts, I even purchased a 2 TB NVMe SSD, which I installed in an external hub connected to the Mini via USB-C. Now folder contents load faster (as expected!) but still much slower than what would normally be expected. Roughly speaking, it takes half the time it did before, which for an NVMe SSD—even an external one—still seems unacceptable to me.

I hope Apple has noticed the problem and will fix it with macOS 15. Otherwise, I might actually have to file a bug report, and I really don’t feel like doing that.

From Intel to Apple Silicon

Mon, 11 Mar 2024 06:00:00 +0000

– Source: Viktor Forgacs on Unsplash.

The transition is complete: since a few days, all the computers I use for work are running on Apple Silicon ARM processors. This includes a Mac Studio M2 Ultra, which I’ve already talked about extensively and which sits on my office desk; a Mac Mini M1 with 16 GB RAM and 1 TB SSD – previously neglected on a shelf for reasons I won’t go into here – now in my home office; and a very basic MacBook Air M1 (just 8 GB RAM and 256 GB SSD, half the specs of my wife’s) for light use and when I’m on the go.

Oh, and let’s not forget: since last summer, I’ve also been using an 11" iPad Pro M1, which is an absolute gem and deserves a post of its own.

The Mac Studio and Mini are running Sonoma, which I’m not thrilled about, but it’s here to stay. Meanwhile, the Air is still on Monterey. I’ll update it to Sonoma soon to ensure consistency across all my machines.

Needless to say, compared to the Intel models I’ve used up until now, the performance is on another level. The benchmarks already say a lot, but the daily experience speaks volumes more: programs that open instantly, the fluidity of even the most resource-hungry applications, Python or R scripts that used to take minutes now finishing almost before I can blink.

Another clear indication of the speed of these machines is the time it takes to install (as usual) the operating system and all the applications I need from scratch (and there are quite a few).

With the Mini, I completed everything in a single day (around 100 applications to download, register, and configure one by one). I spent a second day on finer details, such as transferring files between Macs, tweaking system settings (a rather complicated process that, thankfully, most users can skip), and installing essential command-line tools like homebrew and conda for my work.

Rosetta installed itself automatically the first time I opened an Intel-only application, and so far, all the apps I’ve installed run without issue – even the more niche ones that require specialized libraries. After all, it’s been over three years since the introduction of the first M1 Macs, and the early challenges should have been resolved long ago.

In short, the transition from Intel to Apple Silicon has been smooth – easier than I initially expected and perhaps even simpler than the previous switch from PowerPC to Intel. And in any case, much easier than crossing a busy street in Rome in the 1960s.

P.S. What about my old Macs? As I’ve mentioned elsewhere, I have a new project in mind to repurpose them effectively. The initial tests look very promising, but I’d rather be sure about certain details before writing about it. Stay tuned!

Mac Studio!

Sun, 04 Feb 2024 06:00:00 +0000

And here it is, the Mac Studio. I had to go through a long bureaucratic process to get it, but I finally managed, and now it’s right here in front of me.

The first thing you notice is that the box is heavy, very heavy. The specs state that the Mac Studio is large and weighs about as much as three Mac Minis stacked on top of each other (19.7 x 19.7 x 9.5 cm^3 and 3.6 kg for the Mac Studio versus 19.7 x 19.7 x 3.6 cm^3 and 1.2 kg for the Mac Mini). I’ve never held three Mac Minis together, but the box gives a strong impression of solidity, which is definitely a positive for such an expensive device.

The Mac Studio is indeed not cheap. The configuration I chose is almost top-of-the-line — M2 Ultra processor with a 24-core CPU, 60-core GPU, and 32-core Neural Engine, 128 GB of RAM, and a 1 TB SSD. The total cost was just under €7,000 — the price of two Vision Pros.

But quality comes at a price, and for an equivalent workstation from HP, Dell, or Lenovo, you’d have to shell out €1,000 to €4,000 more.¹ In this case, Apple is even competitive.

Unboxing

I’m not a fan of unboxing rituals, but the Mac Studio deserves this initiation ceremony. The box is a tall, sturdy cardboard parallelepiped, more developed in height than width.

Opening the box is incredibly easy: just pull the tab and fold one side of the box open, reminiscent of the opening mechanism of the Mac Pro G4.

Seeing the Mac Studio lying on its side is a bit unsettling — it looks like it could fall at any moment. But the side flaps are much sturdier than they appear, and the Mac Studio is far too large to slip out of their grip.

At this point, you need to lay the box on its side, spread the flaps in the direction of the arrows, remove the protective paper wrapping, and finally hold the Mac Studio in your hands.²

It really does look like a Mac Mini stretched in height, and now it seems appropriately weighted for its size. The additional weight is entirely due to the box, which is clearly designed to provide the best protection for the valuable item inside.

Even the power cable is a piece of engineering — thick yet flexible, with a fabric sheath that reminds me of those on old irons. This one, however, has a much tighter weave and is unquestionably more elegant. Too bad about the plug — I would have preferred a Schuko, but so be it.

Connecting the power cable and the USB-C cable for the monitor is a breeze. Even pairing my wireless keyboard and mouse (Logitech, both excellent) was simple: I just removed the USB dongle from my usual Mac and plugged it into one of the Mac Studio’s ports — no need to configure anything or fuss with Bluetooth.

Finding the power button was a bit trickier. It’s on the bottom-left corner, perfectly flush with the body of the device, and practically imperceptible to touch. But once pressed, badaboom! In a few seconds, the macOS welcome screen appears. A quick system configuration (since I’ll be reinstalling anyway), and the Sonoma desktop appears.

Reinstalling macOS

Call me obsessive, but I always reinstall macOS from scratch on any new Mac I get my hands on. Maybe it’s a habit from the past, a fixation, or just a desire for control, but I’ve never used a Mac without personally installing macOS.

On Apple Silicon Macs, you need to hold down the power button until the Options icon (a gear symbol) appears, usually alongside one or more icons representing the Mac’s disks. Clicking on Options and confirming loads the macOS Recovery utility.

Once Recovery loads, I first initialize the system disk with Disk Utility, giving it a more meaningful name than the default Macintosh HD (usually the same name I use on the network for easy identification). Then, I reinstall macOS using the straightforward guided procedure.

As always, the estimated installation time is wildly inaccurate: the three-plus hours initially shown turn out to be just about 40 minutes — not short, but not excessive for installing a complex OS like macOS.

It’s also curious that after the mandatory reboot, the screen resolution drops significantly, only to return to normal once the macOS welcome screen appears. I’ve never seen this happen with other macOS versions, but it’s worth noting that most of my (re)installations are done on MacBooks, which have more consistent hardware than desktop systems.

After carefully completing the initial macOS setup, I was back on the Sonoma desktop within minutes. Now, I could finally start experiencing how the Mac Studio performs.

How Does the Mac Studio Perform?

The Mac Studio runs beautifully (but was there any doubt?). Everything is smooth, without the slightest hiccup. While this is true for all Apple Silicon Macs, it’s immediately clear that the Studio is a cut above — perhaps even two.

Application installation is instantaneous: double-click the .dmg, drag the icon to the Applications folder, and… done. Even with larger apps like GNU Emacs, GIMP, or Miniconda, the wait is just two or three seconds.

The only app that put up some resistance was QGIS, whose installation took two and a half minutes on the Mac Studio. But that’s understandable for a 3 GB behemoth containing thousands of small files that need to be copied one by one.

I didn’t bother with Geekbench or similar benchmarks to measure the Mac Studio’s performance. The web is already flooded with such data, and my results would only be redundant. Instead, I’m more interested in how the Mac Studio handles demanding applications — I have some early impressions but will wait to share more.

Sonoma, however, is a drag. It’s clear from the start that this operating system is still full of bugs and inconsistencies (which I’ll showcase in the next post), and it doesn’t do justice to what the Mac Studio can really achieve.

Apple has accustomed us to a tick-tock approach for its operating systems (similar to the one Intel followed for years with its processors): one year, they release an innovative but buggy and slow OS, followed by a version focusing almost exclusively on bug fixes and optimizations. This happened with Leopard and Snow Leopard, then with Lion and Mountain Lion. Later, the pattern continued with El Capitan, practically perfect after the horrendous Mavericks and Yosemite, and Mojave, following two mediocre releases like Sierra and High Sierra. Most recently, Monterey was excellent after the disaster of Catalina. Sonoma, theoretically the bug-fix version of Ventura, is a disappointment. Hopefully, Apple will fix things at the next WWDC in June — and big time.

Conclusion

I did all the unboxing and setup of the Mac Studio at home, so I could work in peace away from the usual chaos at the institute. But after a week, the Mac Studio moved to my office, taking pride of place on my desk amid monitors, keyboards, mice, cables, and various gadgets.

And my old High Sierra computer? That’s destined for a new project, and I’m really curious to see what comes of it. Stay tuned!

It’s worth noting that the aforementioned workstations are, at least theoretically, more expandable than the Mac Studio. However, expandability isn’t a critical factor for my specific needs. ↩︎
There’s not a single piece of plastic in the box. It may not be much from an environmental perspective, but it shows that it can be done. ↩︎

Jony Ive quits Apple: a tragedy or a blessing?

Wed, 10 Jul 2019 06:00:00 +0000

– Source: The London Standard.

Jonathan “Jony” Ive, after thirty years, is leaving Apple to start his own company, LoveFrom, which will have Apple as its first client. The news has filled tech (and non-tech) headlines worldwide for days (as you can read here, here, and here). Almost everyone, after overcoming the surprise of the announcement, expressed hope that the collaboration between the English knight¹ and the California-based company could continue just as before.

I know this might be an unpopular opinion, but I think that would be a mistake. Jony Ive was overdoing it.

Jony Ive is (or was?) a great designer, and over the course of his long career, he created outstanding products. I’m thinking of the various iMacs, from the G3 that marked Apple’s revival to the lamp-like G4 and the all-in-one G5. I’m thinking of the iPod, the iPhone, and iOS 7. But, like many star designers, at some point, he went overboard, embracing an anorexic aesthetic that prioritized design over functionality. Everything became too thin and minimalist, with zero accessibility and repairability.

This gave us the ultra-thin MacBook Pro, aesthetically perfect but technically incomprehensible – a laptop that forces you to carry around a plethora of adapters just to connect a basic USB stick. Or the horrendous butterfly keyboard, whose keys feel like tapping on glass and can be rendered unusable by a speck of dust, forcing you to replace the entire top case (and sometimes the whole laptop!) for a single malfunctioning key.

Then there are the AirPods, which, when their battery dies (two years if you’re lucky), have to be thrown away because even Apple can’t replace the battery without destroying them. Or the iMacs, Mac Minis, and MacBook Airs with soldered RAM that can’t be upgraded. Whatever configuration you choose at purchase is what you’re stuck with forever.²

And let’s not forget the most spectacular flop of all: the Mac Pro, now good only as a (very expensive) trash can. A professional computer so minimal that, to truly use it, you need a bunch of external accessories, all precariously connected via cables. A professional computer that’s not upgradeable (it’s still stuck in 2013), a true contradiction for anyone wanting to preserve the hefty investment it requires. A professional computer that overheats, overheats a hell of a lot!, making it unsuitable for the heavy workloads it’s supposed to handle. Is this the fault of engineers who can’t do thermal calculations or a designer who cares little about the actual function of the product?

Is it just a coincidence that the latest Mac Mini once again offers easily upgradable RAM? Or that the new Mac Pro has returned to its old design — a large, perforated metal case for better heat dissipation, easy to open, with plenty of space for adding drives, RAM, and interface cards? Or that the MacBook Pro keyboard is being redesigned for the umpteenth time in three years?

Perhaps Tim Cook and Apple’s board concluded that Jony Ive had gone too far and decided to finally get rid of him, despite the narrative of a mutual agreement.

It was about time. As Apple users, we deserve better.

– Source: The London Standard.

Jony Ive was knighted in 2012. ↩︎
A technical absurdity: the amount of RAM needed to keep a computer running optimally increases over time with advancements in operating systems and applications. Upgrading RAM is one of the most effective ways to significantly extend the life of any computer. ↩︎