Posts tagged "storage"

While most customers immediately understand how the magic of Oracle’s Hybrid Columnar Compression, intelligent storage servers and flash memory make Exadata uniquely powerful against home-grown database systems, some people think that Exalogic is nothing more than a bunch of x86 servers, a storage appliance and an InfiniBand (IB) network, built into a single rack.

After all, isn’t this exactly what the High Performance Computing (HPC) world has been doing for decades?

On the surface, this may be true. And some people tried exactly that: They tried to put together their own version of Exalogic, but then they discover there’s a lot more to building a system than buying hardware and assembling it together. IT is not Ikea.

Why is that so? Could it be there’s more going on behind the scenes than merely putting together a bunch of servers, a storage array and an InfiniBand network into a rack? Let’s explore some of the special sauce that makes Exalogic unique and un-copyable, so you can save yourself from your next 6- to 12-month science project that distracts you from doing real work that adds value to your company.

Ever since I’ve been using a Mac, I enjoy using Sparse Encrypted Disk Images for a variety of tasks, for instance securely storing data that can be backed up somewhere else, say on a hosting server.

In fact, most of my project/personal data on my Mac sits on sparse encrypted disk images that are regularly rsynced to an external storage service, Strato’s in particular.

The beauty of this solution lies in it simplicity:

Sparse encrypted disk images show up just like any other hard drive. But on the back end, they translate into a bunch of flat files that store all the data in an encrypted manner. By rsyncing the backing store, sparse encrypted disk images can be easily backed up across the net, while ensuring privacy and convenience.

Here’s how to do similar things with Solaris and ZFS, including some extra data integrity magic:

Solaris 11 is here!

And together with the official launch activities, a lot of Oracle and non-Oracle bloggers contributed helpful and informative blog articles to help your datacenter go to eleven.

Here are some notable blog postings, sorted by category for your Solaris 11 blog-reading pleasure:

Maybe I should write more frequently, though that would mean shorter, less elaborate articles. This is the first one of that kind. Let me know what you think!

Recently, the Oracle Solaris 11 Early Adopter Release (no link, page no longer exists) became available on the Oracle Technology Network (BTW, can I have a date with that Java Developer, please?). Here’s the gist:

…that is the question.

Ever since the introduction of deduplication into ZFS, users have been divided into two camps: One side enthusiastically adopted deduplication as a way to save storage space, while the other remained skeptical, pointing out that dedupe has a cost, and that it may not be always the best option.

Let’s look a little deeper into the benefits of ZFS deduplication as well as the cost, because ultimately it boils down to running a cost/benefit analysis of ZFS deduplication. It’s that simple.

More than a while ago, I wrote about the birth of Illumos, a project that aims at substituting the last non-open-source bits from the OpenSolaris kernel with replacements, in order to create a 100% open source Solaris kernel.

On May 20th, I had the opportunity to attend the Nexenta European User Conference 2011 (no link, page no longer exists) in Amsterdam, where Solaris and storage enthusiasts from all over the world met to discuss their favorite technology: ZFS. Of course there was also a lot of talk about Illumos and related projects.

Now I’ve given a lot of Solaris presentations to customers, always highlighting the big, growing and powerful community behind the Solaris OS. But this conference added a new dimension to the Solaris Eco-System for me!

Last week during WorldHostingDays, I had the opportunity to visit Tom (@tomme), a former colleague of mine who came with Q-Layer to Sun, then to Oracle. Today, he works for a new Belgian startup called Amplidata (no link, amplidata.com no longer exists), a company that specializes in building storage clouds. He introduced me to Wim, their CEO and we discussed their optimized object storage technology, some parallels to ZFS and the newest trends in cloud computing storage. Amplidata is a spin-off of Incubaid, a technology incubator which is responsible for the success of two good old Sun friends: Innotek (VirtualBox) and Q-Layer (The company that powered the Sun Cloud).

A few weeks ago, a reader asked me a couple of questions about SSDs and ZFS, hinting that this might be a good topic to write a blog post about.

Sure enough, just last week, a couple of similar questions came up, this time from a customer and a colleague at work.

Well, if that’s not a sign from heaven, I don’t know what is, so here’s a collection of frequently asked questions about flash memory (also known as solid state disks, or SSDs) and ZFS, with answers and some useful links, and an index, too.

At last, it is here: After lots of waiting, speculation, community self-help efforts, future directions, anticipation and more, we’re now holding it in our hands: Oracle Solaris 11 Express is here (no link, page no longer exists)!

While you’re testing/installing/upgrading to this new release, let’s check out the following top 7 cool things about Oracle Solaris 11 Express:

The recently announced Oracle Solaris 10, 09/10 release (no link, page no longer exists) introduced a number of significant upgrades to the ZFS file system.

Ironically, Solaris 10 now comes with a higher ZFS pool version (19 (no link, opensolaris.org no longer exists), at least) than OpenSolaris 2009.06 (14 (no link, opensolaris.org no longer exists)).

So let’s look at some of the key ZFS improvements that came in this update and figure out why they’re so useful.

In this article, you’ll learn more about LUN Expansion, Snapshot Holds, Triple Parity RAID-Z, Log Device Improvements, Pool Recovery, Splitting Mirrors and we’ll discover a new scheduler class!

And as a bonus, we’ll get to watch some videos that explain these features in further detail.

When talking to customers, partners and colleagues about Oracle Solaris ZFS performance, one topic almost always seems to pop up: Synchronous writes and the ZIL.

In fact, most ZFS performance problems I see are related to synchronous writes, how they are handled by ZFS through the ZIL and how they impact IOPS load on the pool’s disks.

Many people blame the ZIL for bad performance, and they even try to turn it off (no link, solarisinternals.com no longer exists), but that’s not good. Actually, the opposite is true: The ZIL is there to help you.

In this article, we’ll learn what synchronous writes are, how they’re processed by ZFS, what the ZIL is, how it works, how to measure ZIL activity and how to accelerate synchronous write performance, which is at the root of many, if not the majority of ZFS performance problems.

The developers of ZFS are a funny bunch of people. You can tell that by watching the “ZFS: The Next Word” talk, meeting them on conferences, reading their blogs or their comments on mailing lists.

And there are also some funny parts in the ZFS source code, too. In fact, if you use ZFS, you’ll have a funny joke sitting on your disk, right under your nose!

I was reminded about this particular joke while listening to Ulrich Gräf’s excellent talk on ZFS internal data structures during OSDevCon 2009 (no link, osdevcon.org no longer exists) (watch a video of Ulrich’s talk here (no link, sun.com no longer exists)).

But first, we need to dig a little bit into the world of ZFS data structures.

When looking at the mails and comments I get about my ZFS optimization and my RAID-Greed posts, the same type of questions tend to pop up over and over again. Here’s an example from a reader email: “I was reading about ZFS on your blog and you mention that if I do a 6 drive array for example, and a single RAID-Z the speed of the slowest drive is the maximum I will be able to achieve, now I thought that ZFS would be better in terms of speed. Please let me know if there is a newer ZFS version that improved this or if it does not apply anymore.” This is just an example, but the basic theme is the same for much for the reactions I see: Many people think that RAID-Z will give them always good performance and are surprised that it doesn’t, thinking it’s a software, an OpenSolaris or a ZFS issue.

In reality, it’s just pure logic and physics, and to understand that we should look a little closer at what vdevs are in ZFS and how they work.

A few weeks ago, we discussed using DTrace for automatically updating media servers when you upload new content.

Yesterday though, I discovered that my D script didn’t work any more. I uploaded new songs to my home server, and expected the music daemon to re-scan the music directory, but nothing happened.

That teached me an important lesson about DTrace, and here’s what I learned:

Since November 1st, 2009, when ZFS Deduplication was integrated into OpenSolaris (no link, genunix.org no longer exists), a lot has happened: We learned how it worked, people got to play with it, used it in production and it became part of the Oracle Sun Storage 7000 Unified Storage System (no link, sun.com no longer exists).

Here’s everything you need to know about ZFS Deduplication and a few links to help you dig deeper into the subject:

A lot of people have read last year’s article “A Small and Energy-Efficient OpenSolaris Home Server” (Thanks a lot to Andre Lue from the EON project for linking to it!) and there was quite some discussion on different RAID options as a result of my RAID-Greed article.

So let’s continue the theme and have a look at the following home server tips that helped me a lot during my own home server planning, building and installing:

After moving my blog to its new home and getting my hands dirty with Drupal, it’s time to continue my series of blog articles about setting up a home server. Remember? We talked about home server requirements (no link, sun.com no longer exists), then I presented to you my small and energy-efficient, still ECC-protected and powerful AMD-based home server (no link, sun.com no longer exists). Now it’s time to explore some different ZFS disk pool RAID strategies.