Is software engineering immature?

Is software engineering an "immature" engineering discipline? I have often heard this both in presentations and in reading, for example SEMAT states on the homepage:

"Software engineering is gravely hampered today by immature practices."

If I draw parallels between software engineering and civil engineering (arguably the most mature engineering discipline) my spontaneous conclusion was: Yes, software engineering seems to be more like how cathedrals were built in medieval times. The construction of them were based on rules of thumb and the practical skills and experience of architects and stone masons instead of the type of engineering practices taught at universities today. Interestingly enough, one can still see the "successful" cathedrals still standing several centuries later. But it would have been impossible for the architect of  Lund Cathedral to build something like the nearby Turning Torso. So the engineering in civil engineering has certainly "matured".
But where is software engineering failing? Everyone has heard of software projects running over time, over budget or have not been used as intended (or not used at all). But wait a minute! Exactly the same thing is common within civil engineering as well. There are many examples from a single contractor renovating a small house up to world known buildings such as the Sydney Opera House:

"The original cost estimate in 1957 was £3,500,000 ($7 million). The original completion date set by the government was 26 January 1963 (Australia Day).[16] Thus, the project was completed ten years late and over-budget by more than fourteen times."

So in this respect software engineering is as immature as other engineering disciplines.
So what is the fuss about? I agree that there is no consensus in the software community on how to do the equivalence of structural mechanics calculations or stress tests. Are we more hampered by this lack of consensus on how to do engineering tasks than by the difficulty to run large projects, which seem to be common to other engineering disciplines?

Architects as managers and leaders?

It is not often I write about management here, in spite of the title of the blog: "Managing..."'

I do believe that architects should not be people managers, at least not simultaneously.
However they should be leaders. An architect should have a vision of what he wants to accomplish and should lead by example. If the architect is successful it seemed like he did not do anything and everything was accomplished jointly by the developers, so don't expect too much credit as a leader...

Back to management: A former student of mine wrote about the difference between good and great companies to work for in his blog and directed me to this series of blog articles:12 Things Good Bosses Believe from Harvard Business Review.

Is software a product of creativity? Or?

I think the phrase "development process", or rather the concept of development as a process has become a paradigm in software. A google search on software development process results in 724 000 000 hits!

The view of creating software is done through a "development process" is so established, at least in the software industry, that no-one seem to question if this is a valid view on how to design stuff. This regardless if one talks about waterfall, spiral, agile or lean.

Since I am not a native English speaker it could be a lack understanding the connotation of the words, but I'll try to explain what I mean.

First, software is developed. No-one talks about a software creation process. To me development is more akin to refinement, exploitation or progress. I think software is created, it is one of the very few intellectual pursuits besides the arts where something is created from nothing.

Second, the development (or creation) is done through a process, something which in an engineering perspective means that by combining the right inputs and having the right circumstances, a desired output is achieved. And the process is repeatable as long as the inputs and circumstances are the same. I don't think anyone disagrees when I say that not two software projects are alike, regardless how similar one tries to keep the process.

So the question is how should we view software creation? I think it would be helpful to see it as a creative endeavour similar to what a writer does. There is usually some idea of what the end result should be, but that may or may not be very detailed when starting to make sentences. Some write in a very linear fashion, starting with the first chapter. Some start with an outline which gets more and more refined until it actually is the full text. Some writers are very disciplined and can write for 8 hours day, some can just write when they are inspired. And everybody has heard about "writer's block"...

If software is not developed in a process, but seen as a work of creativity I think efforts should be to make the people writing software as creative as possible (and therefore also productive). Maybe this can be used as a "lean" principle on how to manage a software project.

Is an archtiecture description always necessary?

I wrote in an earlier post about the importance of being able to write in a concise manner, based on the assumption that key issue for an architecture description is to convey an common understanding of the architecture (more on the validity of that assumption later in the blog).

What would happen if that line of thought is taken to it's extreme, i.e. nothing is written down about the architecture? How would one reach a degree of understanding among the team members? And especially a common understanding? Does it even need to be common?

Answering these questions could easily expand to an entire literature review of organisational learning and I'll try to avoid that. But an architecture description is a way to convert the tacit or internal knowledge (of the architects?) to explicit knowledge which in turn other stakeholders can internalise again. The question is of this transfer of knowledge can be spread through the organisation by other means?
I certainly think so, the same way I learned about electrical system in vehicles "on the job" rather than learning from books, but this requires working side by side on a recurring (daily?) basis. Feasible if the team is co-located in the same room, but it does not really scale if the desks are to far apart. One architect and 15 other developers is not a problem. One architect and 200 other developers is a problem.
I have a gut feel that this is related to how well agile practices scale to big projects...

After reading this post I realised I made an assumption that the architect is the main person having the knowledge about the architecture. Of course this must not always be the case. One alternative is the committe architectures I wrote about earlier, but there of course must exist other alternatives...

What is a project...

I'm taking a course in project management and leadership for PhD students. Quite interesting...

One of my immediate observations is the discrepancy between the academic definition of a project and what we call a project in the car industry. According to our Swedish textbook a project is an activity with

• a well defined goal
• during a well defined time period
• with pre-defined resources
• under particular work model

A vehicle project, for example a model year update, certainly fulfills the first three, but the work model is/should be the same as the one we use for a model year update of another vehicle (not including any kaizen).
There is not something unique working with a particular vehicle model that warrants creating a project, it is just another task. A huge task, but nevertheless...
A research project is a different thing, for example developing the architecture for a new generation of electrical systems. There most work and deliverables are new and have not been done before or at least not in the last 10 years...

The other thing that I noticed as major difference is that the textbook states it is (or should be) impossible to participate in more than one, or at most two, projects simultaneously. An engineer at an EESE department usually works in more projects than that...

This post is in a response to an email I got about my previous post on more personal observations...