It's Not the Size of the Database that Counts...

...it's how you ... it!

Hello again from CyberSpace!

This post is all about big data, what it is, why it's important, and why it matters. I've done some research into Hadoop before, but that work wasn't posted here, and it was limited to one big data solution. So let's get started!

First, it will probably help to define big data. Webster's Dicti.... wait, isn't that too 20th century? Wikipedia defines 'big data' as data that is so large that it be difficult to work with using on-hand database management tools. This can also mean that the data isn't stored anywhere on your system, but on the Internet itself. Big data isn't new: fields such as meteorology and genomics (genetic research) find that there is more data than can be easily stored and queried using standard relational databases.

Being an SQL fan myself, big data had me somewhat curious as to how one would go about extracting any information if it isn't stored locally. The answer is a "No SQL" database - a database that does not use SQL as its primary query language. These databases typically use a key-value pairs to find data, cannot be join tables, and scale horizontally. This sounds impressive, but given the limited uses for this type of search, there are only a few companies that can make good use of the technology. Just ask these fuzzy little guys. WARNING!! Video contains harsh language. Even so, these types of databases allow Google to search millions of web pages in seconds. Nothing to scoff at, as some have suggested Google saved the Internet.

I know what you're thinking - "So I know what big data is, and that No SQL databases are used in big data, but I'm still a little iffy. Do you have any concrete examples for me to wrap my brain around?" Glad you asked, because I'll tell you about not just one, but three - Hadoop, Pig, and Hive. All three are offered by Apache, and by the sounds of their products, they are going into the farming business. Obviously they aren't, so let's examine what each one of these are:

Hadoop

Hadoop is software framework that allows the coordination of multiple servers to process resource intensive work - just like big data. Hadoop is scalable, so that the program go from using one server to thousands with little effort on the programmer's part. In relation to big data, Hadoop is the software that allows programs to be written that can process petabytes of data quickly.

Pig

If Hadoop is the data gathering side of big data, Pig is the analysis side of big data. Pig is fairly new, but has some great potential. Let's face it - what good is collecting a petabyte of data if you can't do anything with it?

Hive

The most aptly named of the three programs, Hive stores the data collected by Hadoop, just like a hive stores honey (and bees!). Interestingly enough, Hive uses HiveQL, which is supposed to be like SQL.

Now that we know the what and how, let's that a look at why. Why is big data important? As I already said, Google used the idea to change search engines, but that was years ago, and only a few companies today are in the business of big data. But as we move into the future, we are going to be dealing with more data - a LOT more. According to the SSI, we are adding data by 60% annually. As we continue to add more data to our lives, and machines start to gather more data about their existence, we have to find better ways of using the data. Considering how cheap data storage is becoming, and how many things have computers in them now, big data is really just a natural response. We have tons of data collected by millions of devices, and curious human minds want to know what all that data has to tell us!

Impressions of Business Process Management

Hello again fro CyberSpace!

As promised in part 1, here are my comments on the article What is Business Process Management? by Michael Hammer.

I rather enjoyed this article, and found that it took a different view of the idea of improving processes than some other articles. Rather than an intellectual analysis, or a how-to in-depth discussion on how to use a particular method, this article gave an historical perspective on process management. The setup framed the rest of the article nicely, and was honest about the shortcomings of BPM.

It feels like this article was aimed at those with experience in the field of Business Process Improvement, but I want to recommend this article to anyone just starting out. The overview gives the reader everything one would need to know at a high level before starting BPM on a large scale with a desire to succeed, not just a one-off attempt at process improvement.

The section discussing the principles of process management felt a little like padding to me, and didn't really offer much in the way of moving the article forward, but I enjoyed the section on the Enablers of Process. This section does an excellent job of outlining some things that many companies inexplicably struggle with - like the elephant in the room that no one is talking about.  It is nice to see these items listed out, since it can be difficult to call out leadership for their lack of conviction, or their inability to change the company culture.

Well thought out and thorough, I found this to be a good summary of Business Process Management. The author balanced details and brevity well, and gave the reader a good overview of what it means to embark on the path of BPM.

Hammering Home Process Management

Hello Again from CyberSpace!

In much the same vein as the last posts, this is a two-parter. This first post is a summary of the article What is Business Process Management? by Michael Hammer.  Typically, I like to link to the article so that you can enjoy it for yourself, but unfortunately, I am unable to locate this one in CyberSpace. Our professor handed out the article, so you'll just have to take my word for it.

The article is broken down into eight sections:

1. Origins of BPM

According to the article, the predecessors of modern process modelling are Six Sigma and Business Process Engineering. Six Sigma is focused on using statistical methods to reduce variance, whereas Business Process Engineering, created by the author of this article, is focused on mapping processes on a large scale.

2. The Process Management Cycle

Current Business Process Management has been shaped by the previous methods of improving business processes. The focus is now on plotting a business process end-to-end, and always cycling back to continue improvement process.

3. The Payoffs of Process Management

There are many reasons to embark on the path of BPM. Some examples are higher productivity, reduced costs, greater consistency, and improved customer service. One benefit, though, isn't as easy to quantify - the ability to respond to rapid change. By having greater control of your processes, it is easier to adapt them as customers demand.

4. The Enablers of Process

Despite the benefits of BPM, not every company has success using it. According to the article, there are five enablers required to gain the benefits of BPM:

1. Process Design - Details of the process in question
2. Process Metrics - End-to-end metrics that measure targets important to customers
3. Process Performers - Individuals skilled in working within these parameters
4. Process Infrastructure - Technology and HR support to allow performers to do their jobs
5. Process Owner - The person who has the responsibility of making the process better

5. Organizational Capabilities for Process

There are four capabilities a company must have in order to install the enablers of success:

1. Leadership - Company leaders must be passionately interested in driving change
2. Culture - The culture of the company must be one that values the customer, and this is shaped by leadership
3. Governance - A body of mechanisms which monitor all process across multiple departments
4. Expertise - Process Management is not particularly easy, so a company must have many people well versed in these methods

6. The Principles of Process Management

There are several principals to process management, including:

• All work is process work
• Any process is better than no process
• A good process is better than a bad process
• One process is better than many
• Even a good process must be performed effectively
• Even a good process can be made better
• Every good process eventually becomes a bad process

7. The EPM as a Management Tool and BPMS

EPM, or Enterprise Process Model, is the foundation of the process model. This is the high-level map of a process, rather than a narrow view of changing a process. This high level view gives a management tool for defining what a company actually does.

8. The Frontiers of BPM

Even with all the benefits listed in the article, BPM is not perfect and has issues as well. According to the article, some of these include management structure and responsibility, IT support, and industry structure.

There you have it - the article in a nutshell. In my next post, I'll give my impressions.

Thoughts on Lean Primer

Hello again from Cyberspace!

Welcome to second post on the article Lean Primer. You can find the part one in my earlier posts. This post is dedicated to my opinion of the article.

This was a pleasant read, and I was especially interested in the article since I have done some work with Six Sigma, and I am enamored with the idea of making things easier. I saw some similarities between Six Sigma and Lean, but mostly in processes and not mindset changes. Six Sigma focuses heavily on statistics, where Lean is about transforming the entire way a company goes about doing business.

Even without the mindset changes, there are several Lean methods useful in Six Sigma, such as The 5 Whys (graph pictured above). This kaizen principle is used to find out the root cause of a problem, and is implemented simply by asking "why?" five times when examining at a problem. I have annoyed several people at work using this method, mostly because it only takes about 3 "why"s before people start to understand there isn't a good reason they have been doing things the hard way.  It has been my experience that it often takes less than five "why"s to find the root cause of a problem, and I am surprised at how effective the method can be at removing the real issue at hand.


Something else that struck my interest was a comment made on page 20:

"Imagine a customer was observing the work in your office. At what moments would they be willing to reach into their pocket, pull out money, and give it to you?"

Currently, I work in an analytical role, providing ad-hoc reports to our executive team, and this really made me stop and think. While I'm sure what I do provides value, it was tough for me to imagine someone handing me cash just for that beautiful bar chart.  Keeping this in the back of my mind has helped me stay on task and guide my actions already.

Along the same lines, I enjoyed the section on higher-value information, found on page 35. Sometimes I a struggle to decide where the focus of the next analysis should be, so I found the ideas of focusing on the unknown and focusing on early feedback quite helpful. I must note that Lean Primer discusses knowledge in reference to process development, but these principles are still useful in my current role. Trying to find the answer to a un-answered questions (which isn't as easy as it may sound), and doing some early high-level initial analysis to verify that this is even worth answering are both ideas I plan to implement at work.

And there you have it! I hope you enjoyed my summary and comments on the Lean process used by Toyota. Not only did I learn about a method for process improvement, but also several things I can implement in my daily job.

Does This Mindset Make My Process Look Lean?

Hello again from Cyberspace! Time for a two-part post!

These posts are dedicated to Japan's favorite car company, Toyota. But we aren't here to talk cars - instead we are going to examine the method of doing business developed by Toyota, lovingly referred to as "Lean". For the first entry, I will be summarizing an article called Lean Primer written by Craig Larman and Bas Vodde, which can be downloaded from the Lean Primer website.

The term "Lean Production" was coined by MIT researchers, and the intention is to contrast the idea of "Mass Production". Often confused for reduction of waste, the true focus of Lean is to "watch the baton, not the runner". In other words, companies should be concerned about product quality, not the actions of the employees. Often in the west, we are more concerned with employee activity, and I'm sure you've had a manager that hated the idea of you not having anything to do.

There's more to it than baton-watching, though. Really, Lean is a way of thinking, with an emphasis on respect for the customer. The customer could be the person purchasing your product, but it can also be a supplier or the next person on the assembly line. When you respect the customer and do your best to make their experience as simple and hassle-free as possible, you will find problems and endeavor to fix them. That leads to continuous improvement, and all the other processes associated with Lean. The goal is that, when you respect your customer, all of the 'reduction in waste' and 'challenge everything' comes from a desire to make life excellent for your customer.

Unfortunately, there are some misconceptions about what Lean is. Some have tried to take pieces of Lean and implement them without the overall culture shift that is at the heart of Lean. For example, the Management Tools associated with Lean, or overall waste reduction. While these are a part of lean, they do not encompass all of Lean. Without changing the mindset to respect others and empowering your employees to improve the processes all around them, you are not truly using a Lean methodology. Lean is really a a mental shift, both in how to treat your customers and employees, and any improvements or processes are born from this shift, rather than for the sake of a better process.

The respect for people is shown in the continuous learning of it's employees. Each employee is trained in the Lean way of thinking and how to spot waste and issues to report them, and all of this is taught by the managers. Rather than sitting behind a desk and passing down verdicts, managers are expected to go see the issues for themselves. The saying that particularly struck me was "My manager can do my job better than me". I wonder how many American employees can say that about their current managers?

Another major term associated with Lean is Kaizen (改善), the relentless focus on continuous improvement. The improvement is for the customer, but goes beyond that - it is improvement just for the sake of improvement. Stated another way: the relentless pursuit of perfection. 

Of course, there's much more to Lean than what can be outlined here. Hopefully, this overview gave you a good idea of what Lean is all about. In the next post, I will give you my thoughts on the article. Stay tuned!