Blog Posts

…that is the question.

Ever since the introduction of deduplication into ZFS, users have been divided into two camps: One side enthusiastically adopted deduplication as a way to save storage space, while the other remained skeptical, pointing out that dedupe has a cost, and that it may not be always the best option.

Let’s look a little deeper into the benefits of ZFS deduplication as well as the cost, because ultimately it boils down to running a cost/benefit analysis of ZFS deduplication. It’s that simple.

More than a while ago, I wrote about the birth of Illumos, a project that aims at substituting the last non-open-source bits from the OpenSolaris kernel with replacements, in order to create a 100% open source Solaris kernel.

On May 20th, I had the opportunity to attend the Nexenta European User Conference 2011 (no link, page no longer exists) in Amsterdam, where Solaris and storage enthusiasts from all over the world met to discuss their favorite technology: ZFS. Of course there was also a lot of talk about Illumos and related projects.

Now I’ve given a lot of Solaris presentations to customers, always highlighting the big, growing and powerful community behind the Solaris OS. But this conference added a new dimension to the Solaris Eco-System for me!

First of all: Apologies for not posting for a long time. The reason? I was having too much fun with node.js and the Joyent Cloud :).

What started as a small experimental hack turned quickly into an exciting new pet project involving the good old Mandelbrot Set, as a web service, running in the Joyent cloud, programmed in node.js.

But first things first: Let’s take a look at node.js as a language and programming model, at the Joyent Cloud and how it relates to Solaris and finally some details on how the picture you see was rendered inside the Joyent Cloud, including an interactive Mandelbrot Set explorer you can play with now, written as a web app.

Last week during WorldHostingDays, I had the opportunity to visit Tom (@tomme), a former colleague of mine who came with Q-Layer to Sun, then to Oracle. Today, he works for a new Belgian startup called Amplidata (no link, amplidata.com no longer exists), a company that specializes in building storage clouds. He introduced me to Wim, their CEO and we discussed their optimized object storage technology, some parallels to ZFS and the newest trends in cloud computing storage. Amplidata is a spin-off of Incubaid, a technology incubator which is responsible for the success of two good old Sun friends: Innotek (VirtualBox) and Q-Layer (The company that powered the Sun Cloud).

One of the first things to do when setting up a new system is to mirror your boot disk. This protects you against system disk failures: If one of the two mirrored boot disks fails, the system can continue running from the other disk without downtime. You can even boot from the surviving mirror half and continue using the system normally, until you have replaced the failed half.

At the currently low prices for boot drive sized disks, this is a no-brainer for increasing your system’s availability, even for a home server system.

Unfortunately, the steps to complete until you’re running off a mirrored ZFS root pool are not yet a no-brainer. While there is a piece of documentation entitled How to Configure a Mirrored Root Pool, it only covers how to add a second disk to your root pool, it does not cover how to prepare and layout a fresh disk so Solaris will accept it as a bootable second half of an rpool mirror.

Which, for historic reasons, is slightly more complicated than just saying zpool attach.

Over the weekend, I sat down and played a bit with the current Oracle Solaris 11 Express (no link, page no longer exists) release in VirtualBox and tested, re-tested and investigated all currently necessary steps to get your root pool mirrored, including some common issues and variations.

Here’s a complete, step-by-step guide with background information on how to mirror your ZFS root pool:

A few weeks ago, a reader asked me a couple of questions about SSDs and ZFS, hinting that this might be a good topic to write a blog post about.

Sure enough, just last week, a couple of similar questions came up, this time from a customer and a colleague at work.

Well, if that’s not a sign from heaven, I don’t know what is, so here’s a collection of frequently asked questions about flash memory (also known as solid state disks, or SSDs) and ZFS, with answers and some useful links, and an index, too.

A deserted street in the docks. The rain is pouring.

A small, old, flower-power style painted Volkswagen van drives by at high speed, then stops below a lifting crane, tires screeching.

The door opens and a woman stumbles out of the vehicle, as if trying to escape. A tall man in uniform catches her violently. Intense exchange of words, probably four-lettered. She hesitates, then follows him back into the van.

“Cut!”

My brother and I are watching from the side of the road. It’s a scene from the upcoming movie Iron Sky (no link, ironsky.net no longer exists), a science-fiction comedy about the Nazis who escaped with UFOs to the dark side of the moon and who come back in 2018 to conquer the earth.

What?

It’s a long, fun story, but you better see it for yourself. Early 2012, that is, when the movie is done.

About 4 years ago, a few colleagues and myself got together and we created a short video about the coolness of two of the most innovative products from Sun of the last decade: ZFS and the X4500 Server.

Today, nearly 4 years later, the video has been downloaded more than 100,000 times (across the original German and the English dubbed version, plus the full resolution downloadable files) and shown to a lot more people during tradeshows, customer demos, etc.

Now YouTube and Google Video (remember?) don’t allow for highest video quality and the old Sun Mediacast server, where we hosted the original MP4 file, no longer exists. Instead, Vimeo has emerged as my video hoster of choice for a variety of projects (check out my video collection on Vimeo) and so it was time to give this video a new home.

A while ago, we discussed some performance analysis basics:

• Define what your problem is.

• Figure out your goal: What metric needs to be in what ballpark for you to declare victory?

• Analyze your system from the inside out: CPU, RAM, Disk, Network. Your Bottleneck is always in one of these 4 regions.

So what are the best commands for finding bottlenecks in each of the four categories above? Here’s part two of my Oracle Solaris Performance cheat sheet with some favorite tricks.

The year 2011 is almost two weeks old, and by now you’ve probably read every other review/resolutions/plans blog post out there. Now you have time to read this one :).

But first of all: Apologies for not having blogged for weeks. I’ve been moving to a new home over the last two months, the holiday season took its toll and there were a few other personal and job projects that demanded my attention.

Now, normality (whatever it is) is starting to come back and I can start devoting more time to this blog again.

This blog is now one year old, although I had been blogging for more than five years before. Time to reflect, plan and flip some switches for the future.

Here’s a quick recap of 2010, some plans and ideas for this blog, a call for feedback and hints to some other projects for 2011.

Whenever you run a professional server, a home server, a hosted virtual server or just a blog on a shared web space, you run a service that provides something useful to your users, readers or whatever consumers your service caters to.

Too bad if things go wrong and you’re the last to notice.

Enter Monitis.com: This service allows you to easily track your servers, websites, blogs or anything else that can be accessed through the internet, or that is able to run a simple agent. Monitoring from the cloud, if you will, at reasonable prices. And if all you need is some basic monitoring, then there’s a free version called mon.itor.us, too.

Let’s check this out in some detail:

At last, it is here: After lots of waiting, speculation, community self-help efforts, future directions, anticipation and more, we’re now holding it in our hands: Oracle Solaris 11 Express is here (no link, page no longer exists)!

While you’re testing/installing/upgrading to this new release, let’s check out the following top 7 cool things about Oracle Solaris 11 Express:

Before we continue our little Performance Analysis Series, let’s look at some current news:

The Bad News: HTTP is dead. Get over it. The killer? It’s called Firesheep, a free Firefox extension that makes it trivially easy for that kid sitting next to you in that Wifi hotspot to steal your Facebook, Twitter or other web services’ identity.

The Victims: The first line of victims are of course millions of unsuspecting users that are sitting in WLAN areas, not knowing that their web identities can be stolen at the click of the button. But the real victims are hundreds, if not thousands of website owners, starting with the who-is-who of web companies, who are now (rightly so) faced with the challenge of upgrading their web infrastructure to HTTPS as soon as possible, preferably overnight.

The Good News: Adding encryption to your web servers used to be an additional burden on the CPU, negatively impacting performance by as much as 2-3x. Fortunately, the new SPARC T3 processors enable you to switch SSL encryption on for your web applications, without any performance impact. This is possible through built-in encryption engines at the core level. And thanks to the Oracle Solaris Cryptographic Framework, it’s easy to take advantage of hardware encryption for any application that needs it.

Wanna learn more? Read on!

Over time, you tend to learn a Solaris performance trick or two. Or three. Or more. That’s cool, it’s how stuff works: You learn, you do, you remember.

Performance analysis and tuning is just like that: You learn a trick from a person that is more senior than you are, you apply it, you feel like a hero, you learn the next trick.

But having a bag of tricks is not enough. Because then you start trying out stuff without a system, and spend useless time hunting that problem with a hit-and-miss approach, gut-based only.

Therefore, I’m always glad to listen to Ulrich Gräf when he does one of his famous performance tuning workshops (if you’re lucky, you can catch Uli blogging in German here (no link, sun.com no longer exists)), because he’ll give you the full view, the context and the system too, when it comes to performance analysis.

So here’s my personal cheat sheet for Oracle Solaris Performance Analysis, including some guideance on how to systematically catch that elusive bottleneck.

A guy I know once said: “If you follow the herd, you’ll end up as lunch.” (Actually, he said “Schnitzel”, since he’s German, but you get the idea).

Well, here’s a guide, a manual if you wish, for avoiding the fate of leading a dull, boring and unremarkable life. This is not just a self-help or success guide type of book, it’s much more. It’s a manifesto for personal freedom that can apply to all of us, if we choose to follow it.

In some ways, it’s like the red pill/blue pill thing from The Matrix: Do you want to stay in the normal world, do normal, boring things like getting a job, applying for a mortgage, going on vacation once or twice a year, and feeding the ducks in the park after you retire?

Or do you want to decide for yourself what to do with your life, create your own rules and live your life the way you want?

For my last article, I wanted to include a couple of videos from YouTube.

First, I pondered using the standard YouTube embed code, but I wanted a more simple, elegant solution. The Drupal RepTags module supports powerful macros and it comes with some pre-built YouTube macros, so that wasn’t bad.

But I wanted more: HTML5. Read on and see how easily you can create your own Drupal RepTags, including the code I use to add HTML5 support for the Drupal YouTube RepTag.

The recently announced Oracle Solaris 10, 09/10 release (no link, page no longer exists) introduced a number of significant upgrades to the ZFS file system.

Ironically, Solaris 10 now comes with a higher ZFS pool version (19 (no link, opensolaris.org no longer exists), at least) than OpenSolaris 2009.06 (14 (no link, opensolaris.org no longer exists)).

So let’s look at some of the key ZFS improvements that came in this update and figure out why they’re so useful.

In this article, you’ll learn more about LUN Expansion, Snapshot Holds, Triple Parity RAID-Z, Log Device Improvements, Pool Recovery, Splitting Mirrors and we’ll discover a new scheduler class!

And as a bonus, we’ll get to watch some videos that explain these features in further detail.

Oracle Solaris 11 is the future of enterprise IT, that is now clear.

Still, we need to wait a year until it is officially released. What can we do now? Well, quite a lot, it turns out. Even if the preview version (due later this year) hasn’t been relased yet, there are a lot of things you can do to prepare for the big OS upgrade.

Here’s a list of 11 things you can do now to start enjoying the benefits of Solaris 11, get ahead of your system peers and be a part of the future of Solaris now!

My last post about Solaris 11 triggered an interesting discussion with UX-admin about IPS and the lack of pre-/post-scripts for installation/removal.

Assuming that Solaris 11 will be based on IPS (no link, opensolaris.org no longer exists) just like OpenSolaris, it is certain that sysadmins will have to change a lot in how they create and manage packages, because IPS is fundamentally different from the good old System V packaging system.

So let’s explore the lack of scripting hooks in IPS and see if we can find some ways of working around them:

Lots of speculation about Solaris and OpenSolaris is happening right now, with an allegedly leaked email being the latest generator of buzz, rumors and troll-ism.

But is that any useful? No.

So let’s cut through the shiitake, do some due diligence and focus on some real facts instead.

In this article, we’ll check out some real and authoritative sources of Solaris direction, mainly John Fowler’s recent webcast about Solaris 11. Then we’ll see what our future opportunities as members of the Solaris community are, and close with some pointers to other opinions on Solaris 11.