Now available: DB2 Version 11 for Linux, UNIX, and Windows

The new version 11.1 of DB2 for Linux, UNIX, and Windows (DB2 LUW) is now available. Enjoy many product improvements for analytic and OLTP scenarios. Here is how to get started:

• This DB2 support site lists all available DB2 versions and fixpacks from DB2 9.1 fixpack 1 to the new DB2 11.1.
• As usual, I recommend by taking a look at the "What's new", the "What's changed" and the "Highlights of DB2 Version 11.1" pages in the DB2 documentation for an overview and links to more detailed information.
• Another good introduction is provided by recent editions of the DB2Night Show which had several shows dedicated to new features of DB2 11.1.
• As part of the security enhancements centralized key managers for the native encryption are now supported.
• If you are into Oracle or Netazza you will find this overview page for "SQL Compatibility Enhancements" useful.
• Last but not least, I always take a look at "functionality in DB2 product editions and DB2 offerings" to understand what editions/offerings are available and what is included.

With that, let's get started and have a save and successful journey with the newest DB2 version, DB2 11.1.

P.S.: Don't forget to try out IBM dashDB, the DB2-based cloud offering.

DB2 pureScale for the Masses

DB2 pureScale Cluster (nanoCluster)

It is almost four years since I wrote about my exciting tests with a DB2 pureScale demo cluster (see picture). At that time the pureScale feature was still fairly new and was supported on a limited choice of hardware/software. Since then the requirements to put DB2 pureScale into production have been reduced dramatically and at the same time many useful features have been added. Let me give a quick overview of why customers choose DB2 with pureScale for both scaling out systems as well as a consolidation platform. Today, DB2 pureScale really is something for the masses...

The following is an unsorted and incomplete list of features and good-to-know items that come to mind when talking about DB2 pureScale:

• License: One of my favorite pages in the DB2 Knowledge Center is the "Functionality in DB2 product editions and DB2 offerings". It shows that pureScale is part of or can be purchased as add-on (Business Application Continuity/BAC Offering) for almost all DB2 editions.
• Application Transparency: How about writing a pureScale-enabled application without knowing about it? There are no special needs to be considered when writing an application to be run on a pureScale cluster.
• Cluster Size and Scale-Out: You can start small and increase the cluster size, even online, depending on your needs.
• Availability: DB2 supports continuous availability of the pureScale cluster by its system design and by features like rolling fixpack updates. Depending on requirements the cluster can span data centers and it is then called Geographically Dispersed pureScale Cluster (GDPC or "stretch cluster"). Basically, if one data center becomes unavailable, DB2 databases continue to be available and are managed from the hardware in the second data center.
  Two pureScale clusters can be linked by, e.g., the built-in HADR feature where transaction logs are shipped to and applied to the standby-cluster, increasing availability even further.
  For increased availability network adapters on each machine and network switches can be redundant.
• Hardware/OS: DB2 runs on the POWER and Intel platform on AIX, Red Hat, and SuSE operatings. Infiniband adapters can be used for highest network performance, but even the GDPC/stretch cluster version of DB2 pureScale only requires a regular TCP/IP network ("vanilla ethernet"). If you don't want to run pureScale on dedicated hardware, no problem, virtual machines (VMware and KVM) and even VM mobilitity are supported, too.
• Smart Workload Management and Processing: Depending on requirements and available resources, applications or specific workloads can be tied to a single node or to a group of computers in the pureScale cluster. This is great for consolidation. Workload balancing distributes the load among the nodes within a group. Over the past years several performance enhancements have also been added to DB2 that cater to consolidation scenarios.

All of the above boils down to cost savings and higher flexibility, an important driver behind opting for pureScale. Having said that I need to mention that earlier this week I decided against pureScale. I gave a nanoCluster, the same type as pictured above, away to make some room in my home office.

BTW: I have been asked whether a DB2 pureScale cluster can brew a good coffee. What would be your answer...?

My DB2 pureScale / Parallel Sysplex Moment

Last week, members of the German union for train operators/engineers were on strike and only a fraction of trains were operating. I had to go to Zurich, Switzerland, but most of the trains on my usual route were out of service. That's when I had my DB2 pureScale or DB2 Parallel Sysplex moment.

What is special about a DB2 Parallel Sysplex or DB2 pureScale configuration? It is a data sharing cluster. If one node in the cluster or a network connection goes down and is inaccessible, the other components can pick up the work. All this is usually transparent to the application. In my case I was the application and data at the same time: Try to get me to Zurich, discuss some issues in person, get back home.

New fixpack for DB2 10.5 brings in-memory analytics to Windows and zLinux

The new DB2 10.5 Fixpack 5 is available since today. A high-level overview of new features and enhancements can be found in the fixpack summary in the DB2 Knowledge Center. The list of all available DB2 fixpacks is available in the IBM Support Portal for DB2. There you will also find the links to download this new fixpack and a list of fixed bugs.

After this introduction I would like to point out two product enhancements that are included in this fixpack:

As you may know, "BLU Acceleration" is the technology codename for highly optimized in-memory analytics that is deeply integrated into the supported platforms. It is not just another column store, but optimizes the data flow from disk to the CPU registers to efficiently use the available processing power and memory resources. DB2 is also exploiting special CPU instruction sets, e.g., on the POWER platform, for faster data processing. With the fixpack 5 this technology is available now on Microsoft Windows and for Linux on zSeries.

Another feature enhancement is the new ability to specify which network interface cards (NICs) DB2 should use, if you have multiple. A new file nicbinding.cfg can be used to set up the bindings. If you had to deal with db2nodes.cfg before, then the syntax will look familiar.

That's all for my quick summary. Enjoy the weekend AND DB2.

New DB2 Cancun Release (Version 10.5 Fixpack 4) offers many enhancements

The fixpack 4 for DB2 10.5 is available since end of last week. Because it has an unusually long list of product enhancements and new features the fixpack even has the codename or nickname "Cancun Release". For those of you not too familiar with Northern American vacation culture, Cancún is a favorite vacation/tourist destination in Mexico, located at the Carribean Sea. So "Cancun Release" may suggest relaxation, recreation, and a dream come true because of ease of use, simplification and major performance enhancements for the DB2 in-memory database feature (BLU Acceleration), the broadened pureScale support, and other nice to haves.

A good start for approaching the new DB2 Cancun release is the fixpack summary in the Knowledge Center. It lists new features by category, my personal highlights are:

• For the in-memory database support (referred to as "column-organized tables" and known as "BLU Acceleration" some bigger items include so-called shadow table to improve analytic queries in an OLTP environment, lifting of several DDL restrictions, and major performance improvement by adding CHAR and VARCHAR columns to the synopsis table. An in-memory database can be made highly available with the HADR feature.
• DB2 pureScale clusters can be deployed in virtualized environments (VMware ESXi, KVM), on low-cost solutions without the RDMA requirement, and geographically dispersed cluster (2 data centers) can be implemented on AIX, Red Hat, SuSE with just RoCE as requirement.
• As part of the SQL compatibility DB2 now supports string length definitions by characters, not just by bytes as before.
• Installation of DB2 in so-called thin server instances.
• A SECADM can enforce encryption of backups.
• db2audit can be used to transfer audit records to syslog for simpler analyzation with, e.g., Splunk.
• db2look has been improved to generate the CREATE DATABASE statement and export the configuration (see my earlier blog article on that db2look improvement in DB2 10.1).
• Official support for POWER8.

I plan to blog about some of the new functionality over the next weeks. Until then you can take a look at the new items yourself. Fixpacks can be downloaded from this IBM support website. If you have an IBM Bluemix account or plan to create one, you can use the improved DB2 as part of the Bluemix Analytics Warehouse service. Check out my earlier post about how to set it up and connect to it using a local DB2CLP.

Last but not least: What is your favorite vacation destinations? Suggest new codenames as comment and don't forget new DB2 features you want to see...

Gradual Adjustments: Back to school, epilepsy medication, and DB2

This week school started again for the kids (and parents). It means to adjust to getting up early in the morning (where is the extra hour of sleep?), getting used to new and different schedules, and coping with homework. It is a process that isn't done in the blink of an eye, but which takes time. Most adjustments are "in the system" within a day or two, the rest is a matter of few weeks until everybody is really back in school mode/mood.

As I wrote before, one of my sons has/had epilepsy. Since middle of June we are happy to phase out his medication. Every 4-5 weeks we are reducing the dose of tablets he has to take, so that his body can adjust to the changed "chemical cocktail". After each change it takes a week to get used to it, then the remaining 3-4 weeks to really adapt to the new dose. Getting rid of medication is not an abrupt event, it takes a while.

Guess what DB2' autonomic features, especially the self-tuning memory manager (STMM) are doing? For a change in the workload characteristics STMM tries to adjust the configuration, the amount of available memory for different consumers. First it usually is a bigger adjustment, then it gradually moves to the final state. In a pureScale environment, previously a single member was in charge of choosing the memory configuration if STMM was active. Starting with DB2 10.5, you can specify whether each member is adjusting the memory configuration on its own, or a single member should do it and whether you pick that member or DB2 dynamically picks it.
So is it like "back to school"? I guess not, every family member has to adjust at the same pace and over night...

DB2 in virtualized environments

I had written about DB2 and virtualization in the past. You can use DB2 in many virtual environments and there are already kind of "dated" redbooks explaining details about DB2 with VMWare, PowerVM, and others. Today I want to point you to two interesting resources:

• You can even virtualize a DB2 pureScale cluster on Linux: The "yes, it is supported" can be found in the Information Center when looking for KVM. All the technical details of setting such a cluster up are discussed in this developerWorks article: Virtualize the IBM DB2 pureScale Feature on Linux using Kernel-based Virtual Machine
• An overview of all supported DB2 virtualization environments, e.g., which hypervisor, which guest operating systems, etc., is provided on this developerWorks page.

WLB, ACR, and Client Affinity in a DB2 pureScale environment

End of April a longer article was published on IBM developerWorks describing Workload balancing, automatic client reroute, client affinities concepts and administration for the DB2 pureScale feature. The mentioned technologies are key to setting up a shared disk DB2 cluster for a 24x7 availability. So if you want to learn about server lists, member priorities, connection timeouts, and other related concept, this is a good place to start reading.

It is important to note that right now the article is based on DB2 10.1. The new release DB2 10.5 adds so-called member subsetting to the above concepts, so that an application can be configured to use a subgroup of available machines.

New page with DB2 pureScale-related acronyms

I started a new permanent page with DB2 pureScale-related acronyms. This is some work in progress as I plan to add links and some more terms. However, as I was searching for such a list, I wanted to point you to it as early as possible. Let me know which other terms to add.

Updated Redbook for DB2 10: High Availability and Disaster Recovery Options

Well, there is not much to say about this existing Redbook that has been updated to reflect DB2 10.1 for Linux, UNIX, and Windows and current technologies. The "High Availability and Disaster Recovery Options for DB2 for Linux, UNIX, and Windows" Redbook describes and explains technologies like IBM Tivoli TSA, PowerHA SystemMirror, Microsoft Windows Failover Cluster, WebSphere Q Replication or InfoSphere CDC.

With close to 600 pages it also requires your high availability...

IBM PureData follows PureFlex and PureApplication Systems

IBM PureData System for Transactions and IBM PureData System for Analytics are two new categories of IBM's expert integrated systems, the so-called PureSystems.

The PureData System for Transactions is built on the DB2 pureScale technology and hence provides scalability and operational continuity. Using DB2's SQL compatibility mode it is also an attractive offer for customers wishing to move away from Oracle database systems.
The PureData System for Analytics actually distinguishes between Analytics and Operational Analytics. The former is built using Netezza technology, the later follows the tradition of InfoSphere Warehouse and IBM Smart Analytics System.

The common benefit of all three new systems is that they provide simplification from A-Z, starting from procurement (one HW/SW piece only) over getting ready for production (my colleagues are claiming hours) to maintenance/administration. They all integrate best practices (like those found here) from experience in the field when working with customers.

BTW: The new systems are nothing for you if you like to fiddle around with OS and DB2 configuration paramaters and want to personally tune everything. That was already done by IBM experts.

PureSystems meets Data: Register for October 9th event

Something cool is coming up on October 9th. The following video includes a registration link for the announcement event. I am not allowed to tell you what it is, but it is neither a new iPhone, electric car, next version of DB2, nor anything related to any elections coming up. It is related to expert knowledge, hardware and software integration, and your future in IT...

New Redbook on pureScale: Delivering Continuity and Extreme Capacity with the IBM DB2 pureScale Feature

A new redbook covering many aspects of using pureScale with DB2 10.1 has been published last Friday. All the details on the related website.

Vacation: Time to REBALANCE (and to look at DB2 10)

It is vacation time, time to recharge and to rebalance life (BTW: Some years back some companies talked about work-life balance, now it is "work/life integration" - towards always on). When I thought about "rebalance", some new DB2 features came to mind. You see, I am still in work mode...

When changing storage in DB2, e.g., adding containers to a tablespace or removing them from it, the tablespace is rebalanced to keep it evenly striped. DB2 10 introduced the concept of storage groups. It allows to group tablespaces with similar properties or "tasks" together. Using ALTER TABLESPACE it is possible to change the associated storage group. To move the data from the former to the new storage (group), data is rebalanced in the background, i.e., asynchronously. To have greater control over those background tasks, DB2 10 adds SUSPEND and RESUME to the ALTER TABLEPACE ... REBALANCE statements. So you can decide when to take a break...

As you might know, there is another rebalance operation available in DB2, storage-related again. It is used in pureScale environments after changes to the shared file system (cluster file system) and rebalances storage utilization. The rebalance option is part of the db2cluster command which is used to interact with the reliable, scalable cluster infrastructure (RSCT) and the cluster file system GPFS.

As both rebalance operations are recommended for "periods of low system activity", this could mean vacation time - bringing me back to what I am preparing for...

High availability, epilepsy, functional MRI, and DB2 commands

In the past I had written a couple times about epilepsy and some computer-related aspects. Today, I want to point you to some more interesting aspects of your brain and high availability. As written earlier, one of my sons has epilepsy which was caused by a brain tumor. Last January, my son and I had a very interesting session in the hospital where a functional MRI (fMRI or fMRT) was performed. The reason has to do with the high availability of the brain.

What happens when you use DB2 with HADR or pureScale and one machine fails? If all is configured right and it indeed works, tasks should move over to one of the machines still up. When a child has epilepsy, depending on the type and seriousness, parts of the brain can get damaged. However, the brain is flexible and to some degree self-repairing. As a result, the functional "processors" of the impacted area can move to a different part of the brain. In DB2 you can monitor the HADR environment and query the state of the pureScale cluster: Who is primary, who is in peer state, on which machine are what services active? But how do you find out where in your brain the speech center is located (actually one of many)? The solution that helped in our case was to perform a functional MRI. It showed where important parts were located and whether it was safe later during surgery to operate in those areas where planned.

Conclusions: DB2 is simple to administrate compared to planning brain surgery.

BTW: Many hospitals use DB2 for patient records and much more.

"Remember to flush" - Your options in DB2

When I hear the word "flush", I always have to think about the movie "The Man Who Knew Too Little" and the scene where Wallace/Spenser is told "Remember to flush".

In DB2 there are statements to flush the event monitors, to flush the package cache, and to flush the optimization profile cache. Well, with DB2 pureScale, the feature for application cluster transparency, you have one more option. It is for another cache, a big data cache. The new statement is FLUSH BUFFERPOOLS. It writes out all dirty pages from the buffer pools to disk. The less dirty data is buffered, the shorter the recovery time in the event of failures. Depending on your strategy and configuration for the page cleaners, this is a new statement to remember. Remember to flush...

Things for the curious: db2greg

It's Friday and it is a slow day. How about I tell you about a DB2 tool that until 2 weeks ago I had never heard of and never had used before? It is a tool that has been in DB2 for an eternity (I found an entry in the Information Center for version 8). I am talking about db2greg which is used to view and change the DB2 global registry.

hloeser@BR857D67:~/Downloads$ db2greg -dump
V,DB2GPRF,DB2SYSTEM,HENRIK,/opt/ibm/db2/V9.7,
V,DB2GPRF,DB2ADMINSERVER,dasusr1,/opt/ibm/db2/V9.7,
I,DB2,9.7.0.4,hloeser,/home/hloeser/sqllib,,1,0,/opt/ibm/db2/V9.7,,
V,DB2GPRF,DB2INSTDEF,hloeser,/opt/ibm/db2/V9.7,
V,DB2GPRF,DB2FCMCOMM,TCPIP4,/opt/ibm/db2/V9.7,
S,DB2,9.7.0.4,/opt/ibm/db2/V9.7,,,4,0,,1305817517,0
S,DAS,9.7.0.4,/opt/ibm/db2/V9.7/das,lib/libdb2dasgcf.so,,4,, ,,
I,DAS,9.7.0.4,dasusr1,/home/dasusr1/das,,1,0,/opt/ibm/db2/V9.7/das,,

Using the "-dump" option as shown above lists the current entries, here for a DB2 9.7. But why would I have to use db2greg and how did I find out about it? The reason is DB2 9.8 which basically is the pureScale feature that brings application cluster transparency. By lack of coffee and sleep and too much enthusiasm I had pulled too many power cables at the same time on a demo machine (nanoCluster). That resulted in some "extra time" in bringing back the machine to "fully operational". In that process I had to clean up some system entries, e.g., like shown in this example in the Information Center.

You will notice in the example that some information DB2 needs to know about the GPFS and the RSCT clusters are stored in the global registry (PEER_DOMAIN, GPFS_CLUSTER, etc.). If parts of a system are manually (re-)build, the registry may become inconsistent and that's when db2greg options like "-delvarrec" and "-addvarrec" are needed to patch up the registry.

For me the mishap ended up with some extra work, but lots of new things learned. And remember, patch responsibly...

Epilepsy: When computer science and medical science meet

Over the next couple of weeks I plan to write some blog posts dealing with epilepsy. One of my sons - maybe it is fair to say, the entire family - has been fighting epilepsy for some years now. We don't know when his epilepsy really started, it has been at least 2 years with visible impact. We expect that it is gone now because he had surgery about 3 weeks ago to remove a brain tumor which caused his epilepsy.

During our visits to doctors and clinics, by talking with other parents and meeting kids with various forms of epilepsy, and by reading we learned a lot. The more we learned, the clearer it got to me how close both computer science and epilepsy are. In computer science, e.g., for database systems, we care about cluster and high availability technologies. Epilepsy is a neurological disorder that impacts the CPU, the main board, and the I/O system that we humans utilize.

What I find interesting is that about one in every hundred has or had epilepsy and that there are more than 40 different kinds of epilepsy syndromes. Some can be dealt with (controlled) by drugs, some cured by surgery, but for many neither works. We learned that the human brain uses advanced technologies to guarantee high availability and to recover from failures. Often, single point of failure is avoided. For computer issues we call in service and have parts replaced. For us humans, however, we should be grateful when our core system is running "normal" - many don't share this fortune...

How to stretch a pureScale cluster: Configuring geographically dispersed DB2 pureScale clusters

Recently, two very interesting new articles were published on developerWorks. One deals with building geographically dispersed DB2 pureScale clusters, the other has an in-depth look on how to upgrade from DB2 9.7 to the DB2 pureScale feature. In both papers, a detailed description of each step is included.

Geographically dispersed clusters, sometimes referred to as stretch clusters, span multiple locations to be able to continue with processing even when an entire site is down. When all involved sites are up, the processing power of the available sites is used (active/active).

New IBM Redpaper: Highly Available and Scalable Systems with IBM eX5 and DB2 pureScale

A new IBM Redpaper discussing DB2 pureScale on System x eX5 machines has been published this week. The paper has a high-level introduction to pureScale and its benefits and talks about the situation at a specific customer. It's not that deeply technical, but a good introduction and with some links to more resources.