Throwing Darts

Hello again from Cyberspace!

It's been a while, but I'm back with some cool news from Google I/O 2012. This annual conference of developers highlights new developments in the tech world from Google. There were many new technologies on display at this year's convention, and I was particularly interested in the new web programming language Dart.

This is a web programming language, presented by the lead programmers Lars Bak and Kasper Lund, and offered as a replacement to Javascript. There are several issues with Javascript, according to the video, including difficult syntax, error forgiveness, and difficulty creating web apps for optimal performance.

Dart is designed to specifically fix these issues. The syntax was designed to be easy for a C#/Java programmer to pick up, and the error-checking is very strict, enforcing higher coding standards and more functional web apps. They also showed the ability to convert Dart to Javascript, so that this new language doesn't "break the web".

A portion of the video spent on performance enhancements Dart brings. This part of the video, while mostly over my head, will be the most impactful to the web as we know it.  One feature that did interest me was the idea of "tree-shaking", where Dart drops any unused functions and variables not used at compile time, making programs smaller and run faster.  The presenters were concerned with trying to drop 1 second load times to 10 milliseconds, and the implications for web apps here are amazing.  This gives developers the opportunity to create larger, more intensive web apps with less lead time, giving us more options to visit websites, rather than downloading or installing local software.

Every bit of Dart appears to be free and open source, which begs the question "Why launch this cool stuff and not charge for it?". From what I could tell, Google is trying to follow in Sun's footsteps with Java. While Java is free for the casual user, the language itself drives companies to do business with Sun. While the article referenced is a little dated, I can certainly see the sense in Google trying to capture more users of its services, given their advertising-driven business model.

Based on what the developers presented, lots of time and energy went into creating this language, and has a place among today's technologies. While this isn't a complete picture, it looks compelling from their presentation. If it works as well as claimed, business could even see benefits from using this language to develop sites that load very quickly and are easy to maintain based on the readability of the code.

Overall, the presenters did a good job of selling Dart. That said, we'll have to wait and see if Google can adequately differentiate its product from the myriad of other web programming languages already available.

Connecting to AWS using the the Terminal on a Mac

Hello again from CyberSpace!!

This post is not class oriented! Yay!

Here I'm going to post how to connect to an Amazon Web Services (AWS) BitNami LAMP instance using the Terminal on a Mac using OSX Snow Leopard 10.6.8. I'm a total noob when it comes to server connections, so hopefully this will help other web beginners like myself. If you are looking for an in-depth or full explanation, turn back now. I'm just trying to help other poor souls trying to hack their way into a server for the first time.

First off, you don't need any new software on a Mac. SSH, the security thingy, comes installed with a Mac, whereas Windows users typically use PuTTY and WinSCP. Connecting from windows is pretty simple. In fact, Amazon has a tutorial for connecting using PuTTY at the time of this post. On a Mac, however, the connection is a little less publicized.

First, you will need a code key .pem file. This should be issued by Amazon when you start an instance. You can check the code key pair associated with the AWS instance from your user console, or download a new one.

Next, you need your terminal. I used spotlight find the terminal, but I believe it is located in the applications folder. Once you are in your tsserminal, type the following:

ssh -i THE FILE PATH TO YOUR CODE KEY ec2-user@AMAZON WEB PATH

The file path should be where you saved your codekey.pem file, and the Amazon Web Path should the from Amazon. You can copy path this by checking your instance and clicking 'Connect'.

Once I did that, I got a happy little error:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/users/___/codeKey.pem' are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: /users/___/codeKey.pem
Permission denied (publickey).


DENIED1!?!?! Crap. Here's where things got a little dicey for me, and I'll warn you as well, in case you are trying something a little different. There are codes in a Mac (maybe PC, but I've never encountered them), that determine the publicity of a file, and who can access it. Apparently, my default permissions for this file were set to 644, and that made ssh sad.

There are lots of different levels of permission out there, as a quick Google search will reveal, but not all are suited for connecting to AWS. To change the code of the file, type the following

sudo chmod 700 /users/____/CodeKey.pem

As in the code before, the '/users/____/CodeKey.pem is the path to your codekey. This changes the permissions for this file to 700, but you can use any number. Hit a couple of help message boards, and you'll find plenty of other security codes out there. I had success with 700 for my BitNami LAMP AWS EC2 instance.

You will be prompted for a password, then the security will change. Type in the connection command (first one on this blog, or press 'Up' twice on your keyboard), and you should connect as happy as you please.

Good luck, and happy server surfing to you!

It's Not the Size of the Database that Counts...

...it's how you ... it!

Hello again from CyberSpace!

This post is all about big data, what it is, why it's important, and why it matters. I've done some research into Hadoop before, but that work wasn't posted here, and it was limited to one big data solution. So let's get started!

First, it will probably help to define big data. Webster's Dicti.... wait, isn't that too 20th century? Wikipedia defines 'big data' as data that is so large that it be difficult to work with using on-hand database management tools. This can also mean that the data isn't stored anywhere on your system, but on the Internet itself. Big data isn't new: fields such as meteorology and genomics (genetic research) find that there is more data than can be easily stored and queried using standard relational databases.

Being an SQL fan myself, big data had me somewhat curious as to how one would go about extracting any information if it isn't stored locally. The answer is a "No SQL" database - a database that does not use SQL as its primary query language. These databases typically use a key-value pairs to find data, cannot be join tables, and scale horizontally. This sounds impressive, but given the limited uses for this type of search, there are only a few companies that can make good use of the technology. Just ask these fuzzy little guys. WARNING!! Video contains harsh language. Even so, these types of databases allow Google to search millions of web pages in seconds. Nothing to scoff at, as some have suggested Google saved the Internet.

I know what you're thinking - "So I know what big data is, and that No SQL databases are used in big data, but I'm still a little iffy. Do you have any concrete examples for me to wrap my brain around?" Glad you asked, because I'll tell you about not just one, but three - Hadoop, Pig, and Hive. All three are offered by Apache, and by the sounds of their products, they are going into the farming business. Obviously they aren't, so let's examine what each one of these are:

Hadoop

Hadoop is software framework that allows the coordination of multiple servers to process resource intensive work - just like big data. Hadoop is scalable, so that the program go from using one server to thousands with little effort on the programmer's part. In relation to big data, Hadoop is the software that allows programs to be written that can process petabytes of data quickly.

Pig

If Hadoop is the data gathering side of big data, Pig is the analysis side of big data. Pig is fairly new, but has some great potential. Let's face it - what good is collecting a petabyte of data if you can't do anything with it?

Hive

The most aptly named of the three programs, Hive stores the data collected by Hadoop, just like a hive stores honey (and bees!). Interestingly enough, Hive uses HiveQL, which is supposed to be like SQL.

Now that we know the what and how, let's that a look at why. Why is big data important? As I already said, Google used the idea to change search engines, but that was years ago, and only a few companies today are in the business of big data. But as we move into the future, we are going to be dealing with more data - a LOT more. According to the SSI, we are adding data by 60% annually. As we continue to add more data to our lives, and machines start to gather more data about their existence, we have to find better ways of using the data. Considering how cheap data storage is becoming, and how many things have computers in them now, big data is really just a natural response. We have tons of data collected by millions of devices, and curious human minds want to know what all that data has to tell us!

Impressions of Business Process Management

Hello again fro CyberSpace!

As promised in part 1, here are my comments on the article What is Business Process Management? by Michael Hammer.

I rather enjoyed this article, and found that it took a different view of the idea of improving processes than some other articles. Rather than an intellectual analysis, or a how-to in-depth discussion on how to use a particular method, this article gave an historical perspective on process management. The setup framed the rest of the article nicely, and was honest about the shortcomings of BPM.

It feels like this article was aimed at those with experience in the field of Business Process Improvement, but I want to recommend this article to anyone just starting out. The overview gives the reader everything one would need to know at a high level before starting BPM on a large scale with a desire to succeed, not just a one-off attempt at process improvement.

The section discussing the principles of process management felt a little like padding to me, and didn't really offer much in the way of moving the article forward, but I enjoyed the section on the Enablers of Process. This section does an excellent job of outlining some things that many companies inexplicably struggle with - like the elephant in the room that no one is talking about.  It is nice to see these items listed out, since it can be difficult to call out leadership for their lack of conviction, or their inability to change the company culture.

Well thought out and thorough, I found this to be a good summary of Business Process Management. The author balanced details and brevity well, and gave the reader a good overview of what it means to embark on the path of BPM.

Hammering Home Process Management

Hello Again from CyberSpace!

In much the same vein as the last posts, this is a two-parter. This first post is a summary of the article What is Business Process Management? by Michael Hammer.  Typically, I like to link to the article so that you can enjoy it for yourself, but unfortunately, I am unable to locate this one in CyberSpace. Our professor handed out the article, so you'll just have to take my word for it.

The article is broken down into eight sections:

1. Origins of BPM

According to the article, the predecessors of modern process modelling are Six Sigma and Business Process Engineering. Six Sigma is focused on using statistical methods to reduce variance, whereas Business Process Engineering, created by the author of this article, is focused on mapping processes on a large scale.

2. The Process Management Cycle

Current Business Process Management has been shaped by the previous methods of improving business processes. The focus is now on plotting a business process end-to-end, and always cycling back to continue improvement process.

3. The Payoffs of Process Management

There are many reasons to embark on the path of BPM. Some examples are higher productivity, reduced costs, greater consistency, and improved customer service. One benefit, though, isn't as easy to quantify - the ability to respond to rapid change. By having greater control of your processes, it is easier to adapt them as customers demand.

4. The Enablers of Process

Despite the benefits of BPM, not every company has success using it. According to the article, there are five enablers required to gain the benefits of BPM:

1. Process Design - Details of the process in question
2. Process Metrics - End-to-end metrics that measure targets important to customers
3. Process Performers - Individuals skilled in working within these parameters
4. Process Infrastructure - Technology and HR support to allow performers to do their jobs
5. Process Owner - The person who has the responsibility of making the process better

5. Organizational Capabilities for Process

There are four capabilities a company must have in order to install the enablers of success:

1. Leadership - Company leaders must be passionately interested in driving change
2. Culture - The culture of the company must be one that values the customer, and this is shaped by leadership
3. Governance - A body of mechanisms which monitor all process across multiple departments
4. Expertise - Process Management is not particularly easy, so a company must have many people well versed in these methods

6. The Principles of Process Management

There are several principals to process management, including:

• All work is process work
• Any process is better than no process
• A good process is better than a bad process
• One process is better than many
• Even a good process must be performed effectively
• Even a good process can be made better
• Every good process eventually becomes a bad process

7. The EPM as a Management Tool and BPMS

EPM, or Enterprise Process Model, is the foundation of the process model. This is the high-level map of a process, rather than a narrow view of changing a process. This high level view gives a management tool for defining what a company actually does.

8. The Frontiers of BPM

Even with all the benefits listed in the article, BPM is not perfect and has issues as well. According to the article, some of these include management structure and responsibility, IT support, and industry structure.

There you have it - the article in a nutshell. In my next post, I'll give my impressions.

Thoughts on Lean Primer

Hello again from Cyberspace!

Welcome to second post on the article Lean Primer. You can find the part one in my earlier posts. This post is dedicated to my opinion of the article.

This was a pleasant read, and I was especially interested in the article since I have done some work with Six Sigma, and I am enamored with the idea of making things easier. I saw some similarities between Six Sigma and Lean, but mostly in processes and not mindset changes. Six Sigma focuses heavily on statistics, where Lean is about transforming the entire way a company goes about doing business.

Even without the mindset changes, there are several Lean methods useful in Six Sigma, such as The 5 Whys (graph pictured above). This kaizen principle is used to find out the root cause of a problem, and is implemented simply by asking "why?" five times when examining at a problem. I have annoyed several people at work using this method, mostly because it only takes about 3 "why"s before people start to understand there isn't a good reason they have been doing things the hard way.  It has been my experience that it often takes less than five "why"s to find the root cause of a problem, and I am surprised at how effective the method can be at removing the real issue at hand.


Something else that struck my interest was a comment made on page 20:

"Imagine a customer was observing the work in your office. At what moments would they be willing to reach into their pocket, pull out money, and give it to you?"

Currently, I work in an analytical role, providing ad-hoc reports to our executive team, and this really made me stop and think. While I'm sure what I do provides value, it was tough for me to imagine someone handing me cash just for that beautiful bar chart.  Keeping this in the back of my mind has helped me stay on task and guide my actions already.

Along the same lines, I enjoyed the section on higher-value information, found on page 35. Sometimes I a struggle to decide where the focus of the next analysis should be, so I found the ideas of focusing on the unknown and focusing on early feedback quite helpful. I must note that Lean Primer discusses knowledge in reference to process development, but these principles are still useful in my current role. Trying to find the answer to a un-answered questions (which isn't as easy as it may sound), and doing some early high-level initial analysis to verify that this is even worth answering are both ideas I plan to implement at work.

And there you have it! I hope you enjoyed my summary and comments on the Lean process used by Toyota. Not only did I learn about a method for process improvement, but also several things I can implement in my daily job.

Does This Mindset Make My Process Look Lean?

Hello again from Cyberspace! Time for a two-part post!

These posts are dedicated to Japan's favorite car company, Toyota. But we aren't here to talk cars - instead we are going to examine the method of doing business developed by Toyota, lovingly referred to as "Lean". For the first entry, I will be summarizing an article called Lean Primer written by Craig Larman and Bas Vodde, which can be downloaded from the Lean Primer website.

The term "Lean Production" was coined by MIT researchers, and the intention is to contrast the idea of "Mass Production". Often confused for reduction of waste, the true focus of Lean is to "watch the baton, not the runner". In other words, companies should be concerned about product quality, not the actions of the employees. Often in the west, we are more concerned with employee activity, and I'm sure you've had a manager that hated the idea of you not having anything to do.

There's more to it than baton-watching, though. Really, Lean is a way of thinking, with an emphasis on respect for the customer. The customer could be the person purchasing your product, but it can also be a supplier or the next person on the assembly line. When you respect the customer and do your best to make their experience as simple and hassle-free as possible, you will find problems and endeavor to fix them. That leads to continuous improvement, and all the other processes associated with Lean. The goal is that, when you respect your customer, all of the 'reduction in waste' and 'challenge everything' comes from a desire to make life excellent for your customer.

Unfortunately, there are some misconceptions about what Lean is. Some have tried to take pieces of Lean and implement them without the overall culture shift that is at the heart of Lean. For example, the Management Tools associated with Lean, or overall waste reduction. While these are a part of lean, they do not encompass all of Lean. Without changing the mindset to respect others and empowering your employees to improve the processes all around them, you are not truly using a Lean methodology. Lean is really a a mental shift, both in how to treat your customers and employees, and any improvements or processes are born from this shift, rather than for the sake of a better process.

The respect for people is shown in the continuous learning of it's employees. Each employee is trained in the Lean way of thinking and how to spot waste and issues to report them, and all of this is taught by the managers. Rather than sitting behind a desk and passing down verdicts, managers are expected to go see the issues for themselves. The saying that particularly struck me was "My manager can do my job better than me". I wonder how many American employees can say that about their current managers?

Another major term associated with Lean is Kaizen (改善), the relentless focus on continuous improvement. The improvement is for the customer, but goes beyond that - it is improvement just for the sake of improvement. Stated another way: the relentless pursuit of perfection. 

Of course, there's much more to Lean than what can be outlined here. Hopefully, this overview gave you a good idea of what Lean is all about. In the next post, I will give you my thoughts on the article. Stay tuned!

Pegasystems BPM

Hello again from Cyberspace!

This post is dedicated to the Business Process Management (BPM) company Pegasystems. Pegasystems, or Pega according to their website, is one of the world leaders in using BPM to help companies streamline existing processes and create new systems. Not only do they offer BPM consulting, but they also work with Customer Relationship Management (CRM).

Started in 1983 by Alan Trefler, Pega works with its own version of BPM modelling to improve business procedures. Their motto, Build for Change, is demonstrated through their consulting services. They focus on training employees quickly to enable change to continue throughout the organization, according to their website.

Pega offers several services, including Consulting, University of Pega, and Pega Product Support. Consulting, as already mentioned, offers in-depth consulting support for updating a company's existing processes or creating new ones. The consultants help with Design Review, Application Readiness, Performance Health Checks, Implementation, Usability, and more.

The University of Pega is an educational environment focused on teaching the Pega method of BPM. The offerings come in the form of eLearning, Certification Programs, and Instructor-led programs, both online and in-person. These offerings can be done on-site as well as individually at one of their training centers. If you're in the States like me, grab that frequent flyer card since most of them are offered in England. If that doesn't work, you can always hop a jet to Tokyo to find more classes.

The final offering is Pega Product Support. If you are like me, the title may be a little misleading. I assumed it was some sort of call center, but it appears to be a collection of FAQ's, a knowledge base, Product Update listings, and discussion forums. All this good stuff requires a username and password, so I wasn't able to peruse the offerings. According to Pega's Product Support page, the support site is also monitored by Pega experts ready to answer your questions.

Given their location in the Gartner Magic Quadrant and the Pegasystems website, it seems like Pega is good at what it does. Before researching the company, I must admit I had not heard of them. Now that I have looked into them, I must say they appear to know what they are doing. I was especially impressed that a British company had so many American success stories posted on their website. If you need some processes mapped, you would do well to give Pega consideration.

SAD SWEBOK

Hello again from CyberSpace!

No, that isn't a picture of a sad swebok we will be discussing. This post is dedicated to Software Analysis & Design (SA & D), as outlined in our course textbook and Software Engineering Body of Knowledge (SWEBOK) methods of process design. I'll review the two methods, highlighting the differences between the two.

To start, we'll outline the different approaches. The textbook version has 6 steps:

1. Identify the problem and obtain approval to proceed
2. Plan and monitor the project
3. Discover and understand the details of the problem
4. Design the system
5. Build and test the system components
6. Complete the test and deploy the solution

SA & D also is a proponent of working the problem in phases, with emphasis on getting a bare-bones functional prototype operational as soon as possible. Additional phases are then worked to refine the prototype and add further functionality.

SWEBOK is a ten step process, and focuses more heavily on the software aspect of design:

1. Software requirements
2. Software design
3. Software construction
4. Software testing
5. Software maintenance
6. Software configuration management
7. Software engineering management
8. Software engineering process
9. Software engineering tools and methods
10. Software quality

As you can see, the word Software is in each topic here, taken from the SWEBOK guide.

At the outset, we can expect that several of these to be different just based on numbering. In fact, half of the lists are the same: Identify and understand requirements, design the software, build the software, then test and launch the software. #2 in SA & D can arguably be added into #1 in SWEBOK given the breadth and depth of requirements setup in SWEBOK. And while Understanding the Issue isn't expressly listed in SWEBOK, the depth of planning and designing in SWEBOK cannot be achieved without a thorough understanding of the issue, even that isn't expressly listed as a step.

That brings us to the items in SWEBOK that are not present in SA & D, specifically items #5 - #10. A major difference in the methodologies is the scope; SA & D is broad enough that it could apply to many process designs, not just software, where SWEBOK is much more focused on developing software. Since these are not covered in SA & D, let's briefly cover them:

Software Maintenance

This is basically what it sounds like: maintaining software after release. Included are requests for upgrades and corrections, along with the methods of tracking and logging such requests.

Configuration Management

Configuration management is the discipline of coordinating software, hardware, firmware, and versions of a program and how they interact at a given point in time, according to the SWEBOK website. When discussing maintenance of software at the level SWEBOK is concerned, this is as important as maintaining the code itself.

Engineering Management

#7 in the SWEBOK list is defined as "planning, coordinating, measuring, monitoring, controlling, and reporting—to ensure that the development and maintenance of software is systematic, disciplined, and quantified", according to the SWEBOK website. While this is seventh in the list of SWEBOK, it sounds very similar to #2 in SA & D. The largest difference, as far as I can see, is the placement within the methodologies.

Engineering Process

The engineering process is divided into two levels within SWEBOK: the managerial level and the detail level. Level one mirrors #2 in SA & D process, trying to manage the people and span of the project, where the second level focuses on the minutia of a project and measuring the effectiveness of the solution.

Engineering Tools & Methods

I'm a little surprised that this is a step, rather than a detail of working with software. Tools and Methods focuses on the development tools used in the design of a solution and the syntax and structure of the coding. This seems more like an overall guideline for a process, rather than an actual step to designing a system.

Software Quality

Once again, not really a step in a process, Software Quality is concerned with designing software such that the customer is pleased with the results. Specifically, quality in this area of SWEBOK is the techniques used to achieve and maintain quality as a standard practice across many projects.

There you have it - two different ways to design a software system. They are very similar in topics, and really only differ in the order of planning. SA & D is more focused on planning and control before design, where SWEBOK manages this aspect of design later in process.

MIST 7530 - The Most Expensive Bullets

Hello again from CyberSpace!

It's a new semester, and that means time for more blog posts! The first installation will be about silver bullets and werewolves! Well, only metaphorical silver bullets and no werewolves...

The assignment was to read No Silver Bullet - Essence and Accident in Software Engineering by Frederick P. Brooks, Jr., which discusses the inherent difficulties in improving software programming, and the absence of 'silver bullets' that will advance programming by an order of magnitude, then come up with my own silver bullets, hypothetical as they may be. So without further ado, here is my top three list:

Silver bullet #1: Outcome Visualizer

Like a window into the future, this little gadget will accept a plan, detailed or otherwise, and show you that plan in action. You can see all the good your design creates, as well as any of the shortcomings. Using such a tool with help you to refine your designs, remove the most harmful pieces, and plan for the inevitable damage you will cause. TANSTAAFL; everything we do will surely have issues for the next project team to tackle.

Silver bullet #2: Specification Sniffer

Modern software programming environments, like Visual Studio 2010, have auto-complete functions which will assist the programmer by suggesting the correct function or property from the code. Intellisense, for example, will provide the most common option for the code being typed. Having a similar knowledge base for coming up with project specifications will assist in development. The system will go a step further by suggesting other similar actions, and even negative results associated with the recommended action. If a sniffer can help find common errors at design, the programming process becomes much simpler and more robust.

Silver bullet #3: Organic code

One of the largest issues in writing code is its static nature - the code you write can only do what it's told to do. There are techniques and styles to help reduce the issue, but the problem remains: code can't adapt on its own to changing demands and needs. If code could organically change or repair itself, we would actually need less system design, since systems would be able to grow and meet the current demands.

There you have it - three ways to help software develop as quickly as hardware. Even though none of them exist yet...