Posts tagged "opensolaris"

More than a while ago, I wrote about the birth of Illumos, a project that aims at substituting the last non-open-source bits from the OpenSolaris kernel with replacements, in order to create a 100% open source Solaris kernel.

On May 20th, I had the opportunity to attend the Nexenta European User Conference 2011 (no link, page no longer exists) in Amsterdam, where Solaris and storage enthusiasts from all over the world met to discuss their favorite technology: ZFS. Of course there was also a lot of talk about Illumos and related projects.

Now I’ve given a lot of Solaris presentations to customers, always highlighting the big, growing and powerful community behind the Solaris OS. But this conference added a new dimension to the Solaris Eco-System for me!

At last, it is here: After lots of waiting, speculation, community self-help efforts, future directions, anticipation and more, we’re now holding it in our hands: Oracle Solaris 11 Express is here (no link, page no longer exists)!

While you’re testing/installing/upgrading to this new release, let’s check out the following top 7 cool things about Oracle Solaris 11 Express:

My last post about Solaris 11 triggered an interesting discussion with UX-admin about IPS and the lack of pre-/post-scripts for installation/removal.

Assuming that Solaris 11 will be based on IPS (no link, opensolaris.org no longer exists) just like OpenSolaris, it is certain that sysadmins will have to change a lot in how they create and manage packages, because IPS is fundamentally different from the good old System V packaging system.

So let’s explore the lack of scripting hooks in IPS and see if we can find some ways of working around them:

Lots of speculation about Solaris and OpenSolaris is happening right now, with an allegedly leaked email being the latest generator of buzz, rumors and troll-ism.

But is that any useful? No.

So let’s cut through the shiitake, do some due diligence and focus on some real facts instead.

In this article, we’ll check out some real and authoritative sources of Solaris direction, mainly John Fowler’s recent webcast about Solaris 11. Then we’ll see what our future opportunities as members of the Solaris community are, and close with some pointers to other opinions on Solaris 11.

Ever since Oracle announced that they’ll buy Sun (no link, page no longer exists), there has been a lot of discussion about the future (some would say “fate”) of OpenSolaris in the “community”.

In fact, the last 15 months have been very instructive in terms of how communities work, or how they don’t.

Let’s check out what a community is supposed to do, and what it is not supposed to do, in the light of the latest OpenSolaris announcement: Illumos.

When talking to customers, partners and colleagues about Oracle Solaris ZFS performance, one topic almost always seems to pop up: Synchronous writes and the ZIL.

In fact, most ZFS performance problems I see are related to synchronous writes, how they are handled by ZFS through the ZIL and how they impact IOPS load on the pool’s disks.

Many people blame the ZIL for bad performance, and they even try to turn it off (no link, solarisinternals.com no longer exists), but that’s not good. Actually, the opposite is true: The ZIL is there to help you.

In this article, we’ll learn what synchronous writes are, how they’re processed by ZFS, what the ZIL is, how it works, how to measure ZIL activity and how to accelerate synchronous write performance, which is at the root of many, if not the majority of ZFS performance problems.

One of the best information sources for any topic are blogs, and the Oracle Solaris operating system in all its variants (Solaris 10 and OpenSolaris) is no exception. Most of what I learned about OpenSolaris was through blogs, or through interacting with Solaris bloggers.

As a way of saying “Thank You”, I did some research and came up with a list of the top ten Solaris related blogs with the highest traffic on the Internet.

But first, let’s clear up some basic rules.

The developers of ZFS are a funny bunch of people. You can tell that by watching the “ZFS: The Next Word” talk, meeting them on conferences, reading their blogs or their comments on mailing lists.

And there are also some funny parts in the ZFS source code, too. In fact, if you use ZFS, you’ll have a funny joke sitting on your disk, right under your nose!

I was reminded about this particular joke while listening to Ulrich Gräf’s excellent talk on ZFS internal data structures during OSDevCon 2009 (no link, osdevcon.org no longer exists) (watch a video of Ulrich’s talk here (no link, sun.com no longer exists)).

But first, we need to dig a little bit into the world of ZFS data structures.

When looking at the mails and comments I get about my ZFS optimization and my RAID-Greed posts, the same type of questions tend to pop up over and over again. Here’s an example from a reader email: “I was reading about ZFS on your blog and you mention that if I do a 6 drive array for example, and a single RAID-Z the speed of the slowest drive is the maximum I will be able to achieve, now I thought that ZFS would be better in terms of speed. Please let me know if there is a newer ZFS version that improved this or if it does not apply anymore.” This is just an example, but the basic theme is the same for much for the reactions I see: Many people think that RAID-Z will give them always good performance and are surprised that it doesn’t, thinking it’s a software, an OpenSolaris or a ZFS issue.

In reality, it’s just pure logic and physics, and to understand that we should look a little closer at what vdevs are in ZFS and how they work.

In the last couple of posts, we used DTrace to notify our media servers and perfected our script a bit.

But the script is still not ready to be used on our home servers yet: It requires manual start and stop, not quite the service oriented automatism we’re used to in the Oracle Solaris world.

The next step is to wrap our DTrace script inside a Service Management Facility (SMF) service, then wrap everything into a shell script that will easily install or remove the service whenever we need it.

A few weeks ago, we discussed using DTrace for automatically updating media servers when you upload new content.

Yesterday though, I discovered that my D script didn’t work any more. I uploaded new songs to my home server, and expected the music daemon to re-scan the music directory, but nothing happened.

That teached me an important lesson about DTrace, and here’s what I learned:

This is a long article, but I hope you’ll still find it interesting to read. Let me know if you want me to break down future long articles into multiple parts instead.

One of the most frequently asked questions around ZFS is: “How can I improve ZFS performance?”.

This is not to say that ZFS performance would be bad. ZFS can be a very fast file system. ZFS is mostly self-tuning and the inherent nature of the algorithms behind ZFS help you reach better performance than most RAID-controllers and RAID-boxes - but without the expensive “controller” part.

Most of the ZFS performance problems that I see are rooted in incorrect assumptions about the hardware, or just unrealistic expectations of the laws of physics.

So let’s look at ten ways to easily improve ZFS performance that everyone can implement without being a ZFS expert.

During OSDevCon 2009 in Dresden (no link, osdevcon.org no longer exists), I had the honor to present on some lessons learned while putting together a ZFS Automatic-Scrubbing SMF service.

Today, Deirdre was so kind to publish the video recording of my presentation on The Oracle Solaris Video Blog (no link, sun.com no longer exists).

From the description:

This talk walks you through the implementation of a simple SMF service. What sounds "simple" at first, develops a life of its own when you consider installation/de-installation, security considerations, error handling and debugging, and of course unexpected little bugs and shortcomings. Finally, we add a GUI to our service by discovering the OpenSolaris Visual Panels project. This "lessons learned" talk is intended to be a practical roundup of things to consider for developers interested in integrating with SMF.

When I blogged about my OpenSolaris Home Server a while ago (no link, sun.com no longer exists), little did I know that this would become my most popular entry in my old blog!

In fact, R.G. (no link, page no longer exists) was so kind to call my setup “perilously close to being an AMD reference design (no link, opensolaris.org no longer exists)”. Thanks, R.G.! Read about his final setup here (no link, opensolaris.org no longer exists). And BTW, if you’re into e-guitars, check out his GEOFEX page, a great resource for guitar effects.

So let’s review our reference design and discuss some modifications to better suit your needs:

Since November 1st, 2009, when ZFS Deduplication was integrated into OpenSolaris (no link, genunix.org no longer exists), a lot has happened: We learned how it worked, people got to play with it, used it in production and it became part of the Oracle Sun Storage 7000 Unified Storage System (no link, sun.com no longer exists).

Here’s everything you need to know about ZFS Deduplication and a few links to help you dig deeper into the subject:

Last week, we looked at how essential scripting is for administering home servers (one of the 7 tips for home server bliss) and we wrote us a little script for enabling automatic snapshots.

Another thing that you’ll almost certainly want to do on your OpenSolaris home server is enabling power management. This will ensure your server spends as little power as possible when idle, while still being powerful when needed.

Firefox 3.6 has introduced a new feature (that was in Beta for some time): Personas.

What’s a Persona? From Firefox’ own description: Personas are easy-to-use themes that let you personalize the look of your Firefox There’s a very nicely done video on the Firefox Personas site to introduce the feature.

If you (or your friendly sysadmin) haven’t upgraded to Firefox 3.6 yet, you can still install the Personas Plus Add-On (no link, page no longer exists) to enjoy all of the Personas goodness.

One of my OpenSolaris Home Server Tips is to script everything. That triggered quite some interest. So let’s start a short series around OpenSolaris home server scripting.

Today, we’ll talk a little bit about the “why?” of home server scripting, then run into a small surprise while we write a small script that will enable/disable the OpenSolaris ZFS Auto-Snapshot Service for us.

A lot of people have read last year’s article “A Small and Energy-Efficient OpenSolaris Home Server” (Thanks a lot to Andre Lue from the EON project for linking to it!) and there was quite some discussion on different RAID options as a result of my RAID-Greed article.

So let’s continue the theme and have a look at the following home server tips that helped me a lot during my own home server planning, building and installing:

After moving my blog to its new home and getting my hands dirty with Drupal, it’s time to continue my series of blog articles about setting up a home server. Remember? We talked about home server requirements (no link, sun.com no longer exists), then I presented to you my small and energy-efficient, still ECC-protected and powerful AMD-based home server (no link, sun.com no longer exists). Now it’s time to explore some different ZFS disk pool RAID strategies.