Sunday, February 25, 2007

Microsoft's Real Plan?

Microsoft is in trouble. They've taken way to long to deliver the next generation their primary product (Windows), and the development cycle with it was horrendous - so bad they cannot do development that way again and survive. On top of that, they have placed the Computer Industry (and users) at a cross-roads - Office 2007 requires so much retraining that upgrading to OpenOffice 2.x is cheaper than upgrading to Office 2007 based on upgrade costs alone. (OpenOffice 2.x's user interface is familiar enough to most users that they will barely know they switched unless they are really big power users and use all the VBA functionality.) Moreover, the industry is moving towards standardized document formats, which Microsoft refuses to natively support. So, the Computer Industry is currently primed to chose a different OS and a different Office Productivity Suite.

So what is Microsoft's real plan? A few years back, after Microsoft was told they could not embrace and extend Java, they created .Net, and made it a central focus to move everything Microsoft related to be based on .Net. Today, they have achieved quite a lot in that arena. Add to it the recent Microsoft/Novell deal concerning Linux (which the Linux community generally does not like) and perhaps some of the pieces of where Microsoft is heading will start to fall together.

More than anything else, .Net is an enabling technology for Microsoft. Since they created it and wrote it, they could tie it to their current offerings (e.g. Windows on x86). But they also seem to have planned into it the ability to remove Windows from the stack. That is, like Java, .Net is a framework that allows programs to be written independently of the OS, theoretically, and hardware. .Net is tied to the way Microsoft does things, but as the dotGNU and Mono projects have shown the .Net framework simply needs to be ported to a different OS and architecture platform to move the applications (at the source level) to that new OS and architecture. So now Microsoft's own code is "highly portable" to something other than Windows (again, theoretically). True, Microsoft is not supporting .Net on other platforms than their own (Win32, Win64), but they could if they wanted to. (Frankly, it would be surprising if they did not already have a port of .Net to other platforms.)

Add to all of this the recent report of Microsoft selling Novell's SuSE Linux faster than expected.

So what is going on here?

It has been my speculation that .Net was the start of Microsoft's plan for how they will survive in a post Windows world. Yes, their Windows platform is currently the most widely distributed - but (a) people have a lot of issues with it, and (b) Microsoft can't continue to develop it any longer the way they have. Perhaps they will surprise us with a new development model for it, or perhaps the surprise is something even bigger...

Imagine (for a moment) Microsoft releasing a new version of Windows - Windows NG (for Next Generation) - that does not provide any backwards compatibility whatsoever. If Microsoft did this, they would need to be able to quickly push a lot of people to support their new system; or they could ride on the shoulders of giants - existing OS's that are already out there that have a lot of software and they would only need to push their major third party vendors over and the rest would be a piece of cake. Of course, to do this, they would only need to release .Net NG with support for both the latest version of Windows (e.g. Windows Vista) and Windows NG, release a new version of Visual Studio to support both but only allow it to compile to .Net's CLR, forcing the .Net framework to do it upon install or on first run. Most Windows developers use Visual Studio - and the majority of those being trained are only being trained on .Net - so this would quite easily move everyone to the new Windows NG platform.

So, then if it is so simple for Microsoft to move people to a new platform in a few short years (yes, it would take one to three years for this all to happen), then how could Microsoft use an existing OS? What would there be for them to use? Well, there is always the BSD's, but then Microsoft would have to fork and support their own - kind of like Apple did; which could be costly. Or, Microsoft could chose a Linux Distribution (Novell's SuSE?) and make it its primary back end; add on the extra tools to move their infrastructure over (Vista's User Mode Sound and Video drivers, and .Net) and a user interface to make it look like Windows (so user's can't tell), and Microsoft could quickly find themselves with (a) a very large set of applications that are already developed and running their system, (b) an easy way to support old applications (official Microsoft support of WINE?), (c) little trouble with moving their existing customer and developer base over (ala .Net), and (d) user's that are much happier. On top of it all, they could get out of the Monopoly they are currently in with Windows, as there would be a lot of vendors out there competing (Oracle, Red Hat, Novell, Slackware, Debian, etc.), and they can finally claim better security.

Now, if this actually happened, I would not expect Microsoft to open source .Net or any part of the infrastructure or interface support they would have to provide to the Linux distribution. (Shared Source? May be, but it wouldn't be very GPL friendly wherever they could get away with it.)

So, what is Microsoft's Real Plan?

Honestly, I am not an insider with Microsoft in any way, but looking at how they have set themselves up over the last few years, how they have separated out their Windows platform internally, and their deal with Novell and its sales results; it may very well be that their real plan is to make the next version of Windows have no backwards compatibility and be based on Linux or a BSD.

Here's hoping.

Tuesday, February 06, 2007

Consumers and Customers...

Perhaps one of the things that annoys me the most about businesses today is that they no longer have a concept (or at least most do not have a concept) any more of a customer. They view everyone as a consumer. It's bad enough that it is not just business any more, but culture itself. We've convinced ourselves that we are not anyone's customers, so much as we are consumers of goods. As a result we try to consume as much as we can, without giving thought to whether we really need to do so.

In truth, we are customers and we deserve to be treated as such. When customers are viewed simply as consumers, they are treated poorly. More importantly, they are made to believe the "more for less is better" - which is true for consumers, but not necessarily for customers.

For example, let's look at a restaurant. If a restaurant views their customers as consumers, then they will try to stuff as much into their meal as possible - regardless of its impact on the customer. This, in part, leads to customers overeating, gaining weight, and ultimately health problems. However, if a restaurant views their customers as customers, then they will serve a meal that is properly proportioned and ultimately healthy - it won't be too much, or too little.

Some simply argue that it is market forces at work. However, that is pure BS. Yes, if customers demanded to be customers instead of consumers some businesses will listen. However, customers are ingrained with this idea not entirely from the market - but also from education - specifically the economics classes and instruction.

Economists take the view of customers as consumers because it is easy for their subject to view it as such; just like they view businesses as producers. While this view has limited benefits in the subject, it has a lot more devestating impact on society at large, as society takes on the view and ultimately kills itself and everything around it trying to live up to it.

Please, treat us all - people and businesses alike - with the respect we deserve. We are customers. Yes, we do 'consume' a certain amount of goods, and we need to restrain ourselves as well, but we desire the respect of being a customer.

Sunday, February 04, 2007

Software Being Hard, and Engineering

A story ran today on Slashdot and also on OS News about why software is hard. It was interesting reading the responses - especially on OS News - as everyone seemed to offer their own opinion on what was hard about software. Both stories link back to a single article entitled Why Software Is Hard. However, the irony was primarily at OS News, where several discussions got off about how Software is NOT Engineering, and that is where I'd like to focus today...

Programmers are the hardest people to deal with when it comes to getting them to do actual work, and do it the right way; and then, when the product comes out and is handed over to the customer they will offer up hundreds of excuses as to why it wasn't what was promised - and there are a number of reasons that are truly valid. However, they seem to miss the entire point of how to solve the majority of those problems.

On the other hand, properly trained Software Engineers[1] seem to be able to do a lot better at delivering products on time and on budget - and yet they are also programmers. So what's the difference?

The difference is in how they both approach the problem of writing software. A programmer will look at the requests and say "yeah, I can do that"; and the software engineer will sit back and discuss requirements, architecture, concept of operations, design, time line, budget, and much more. So how does this really affect a project?

The Software Engineer's approach of discussion the requirements, architecture, concept of operations, design, time line and budget will reveal to them a lot more about the product - and they will (if the customer participated in the discussions all along) more likely than not provide exactly what the customer asked for and quite likely on time and within budget; but even when things are not on-time or outside of budget - the reasons for it are clearly documented and can be improved upon later on. I.e. the Software Engineer can learn from one project to improve the next project.

The programmer, however, will ask a few questions, get some basic requirements, take a wild guess at the budget and time line, and then get lost in the mess of work that ensues. They will also get lost in "requirement creep" and make a number of mistakes that require additional time to fix - and more money to do so - and more likely than not deliver a product that does not meet the customer's needs, expectations, and may get thrown out. The worse part is that the programmer has no way of gathering metrics to improve their work the next time around; so they make the same mistakes on the next project again and again - and they don't even know they are making the mistakes.

One of the arguments used by programmers is "writing software is an art, not engineering". This argument lies in the history of the field. First, a lot of early computer people started off in electrical engineering; they built the hardware, wrote the basic systems; and then moved on to build new hardware and became computer engineers. This part of the field still exists within the Electrical Engineering discipline. As those early computer people got more involved in hardware and improving computers, more computer people came out of the mathematical discipline (which was heavily used due to the Electrical Engineering involved) and took over more of the software that ran on the computer. And this is more or less since early computer usage was primarily for performing either very hard computations that took a long time, or quickly spitting out a lot of easy calculations. In either respect, these mathematicians did not like the engineering side, and saw their mathematics as more of an art, which they then pushed into the software.

However, since then software has grown quite a bit. Languages have formulized from assembly and a very chaotic method of programming to functional/structural and to more object oriented methodologies. Each time the field moved away from chaos it also gained a level of structure to how it was done, yet the mindset of the program writers - programmers - remained much the same. "It is an art."

In the last decade or so, however, business is realizing that there really is more to programming than "art" - specifically those businesses that are involved in a lot of engineering - mechanical, electrical, computer, etc. As a result, they are pushing back and saying "we can do this in our engineering houses, why don't we apply the general ideas behind it to our software programming houses?" This bore the software engineering field.

The key difference, however, is that the Software Engineer realizes that the "programming process" is just the implementation phase of creating software; and that there is a lot more to be done before the implementation phase can even begin. Comparitively, the programmer wants to just jump in and start writing code as soon as they have been handed a task, skipping the rest of the process, and possibly even ignoring any part of that process if anything from it was handed to him/her.

Thus, programmers can be seen as simple "grease monkeys" that are needed to put the bolts together; but will have a hard time rebuilding the entire system in a working order without much pain. Such jobs can easily be outsourced, and even sent overseas. However, software engineers hold a valuable value-add over programmers - they know the details of how the entire system works, and how it all fits together, and can repeatably build it to work; thus their jobs are not so easily outsourced or sent overseas.

To bring it back to the OS News and Slashdot articles, I noticed on the OS News responses that most all agreed upon the problems - bad management, too many unknowns, bad communication, etc. Yet the programmers could not offer any solution to the problems. It is exactly these kinds of problems that Software Engineernig aims to fix.

A few comments stuck out to me in reading over OS News.

Comment One:

I've always had a saying that goes something like this:

Writing software will be comparable to engineering when A) 10 different separate product management groups can come up with an identical spec for an application and B) can hand that spec to 10 different development groups and get an identical product back in terms of usability, bugs per LOC, performance, and price.

You do that today, and you'll get 100 different applications.

In other words, when results are predictable to within the 90th+ percentile in writing software, then it will be an engineering field. Until then, it's very much an art as Knuth outlines.
To this, I must say: Yes, you may get different specs, but each program - if engineered instead of merely programmed - will come back adhering to the spec, and likely with similar bugs per SLOC, with similar performance, price, and usability. They will differ some between them, but in the end - from a programmatic view point - they will be so similar it won't matter. Software Engineering applies itself not simply in the process of getting to the program, but also in the HMI (Human-Machine-Interface), MMI (Machine-Machine-Interface), Programming Structure, and Programming Logic. Programming is only a small part of Software Engineering.

Comment Two:

Whatever you want to call it, there is a process that should happen before the first lines of code get entered into your editor. It comprises some consideration of the problem, generation of requirements, specification of the interface, design of the data structures and algorithms, and representation of the control logic. These processes might not be so clearly identifiable in many cases, but they happen. Or at least they should.

I know exactly what you mean and we even studied these disciplines and let me tell you one thing.. in 99% of the time and situations, this is the biggest waste of time imaginable.

The reason?

Because you'll end up after hours of careful planning with an implementation block which will require utter redesign.

Been there, done that. I'm programming for 5 years actively now (I know it's not mastery) and honestly haven't been in big teams on big projects, but I did try to go this route and it usually backfires in a bad way.

You might say that the planning was poor etc. but especially if you are about to use new stuff (eg: new database, sure it's just another SQL but different from the one you used before) you WILL stumble on the way in a bad way.

My personal opinion is that a bit of good verbal (unless it's a huge planning) discussion about the internal logic BETWEEN PROGRAMMERS (no I don't want to talk about that with customer or marketing, it wouldn't make sense to them anyhow) is a good thing, but keep it realistic.

Also there should be "tries" of basic stuff before ventures take place. Eg: try out the new DB if it can do all the major stuff first exactly the way we think it can etc. Documentation is never good enough source of capabilities and pitfalls.

To this I must say: Yes, Engineering Software does add overhead. However, it will also drive down the cost. I've been programming for a decade now, and can see a true benefit to Software Engineering. It helps solve a lot of problems, and helps the implementators of the product do their job many times betters, on or under budget, and on time or even early. Also, Software Engineering is not so much about the technology you use, but the process you use to do to it. Moreover, the manager of the engineering team should ensure that they have the right people, with the right skill sets, to do the job. Per the quoter's DB scenerio - you bring in someone who knows the DB already instead of training the team on the DB, and so forth. (And you build that kind of stuff into your costs - cost for the project, and the cost of doing business.) Finally, with Software Engineering (as I said above) you learn from your mistakes; sure this time around you may have had to redesign it for some reason (likely because the process was not held to strictly enough, or communications were not held up well enough); but you learn the next time and you can fix the problem - ensure you have better communications, say no more to feature creep, etc. Unlike programming, Software Engineering has a feedback loop that helps it improve itself over time.

Now there is another, newer field that is starting to crop up - Systems Engineering. Simply put, Software Engineers simply deal with the software parts of the system, where as Systems Engineers extrapolate a level higher to get a full view of the entire system (e.g. hardware, software, server components, etc.). Many Software Engineers probably do a good bit of Systems Engineering along their way too.

[1] I say properly trained since unlike any other Engineering Discipline, there is no barrier to someone calling themselves a Software Engineer - unless they are from the State of Texas, which requires Software Engineers to pass tests just like any other Engineering discipline in order to call themselves a Software Engineer. Honestly, this is probably a good thing; even though it would also be a pain in the butt to do, and expensive too.