Song of the Moment

Posted by Will Strye

One of the Beatles (I think it was George) used to tell the story of the day they got to meet Elvis Presley. He said when they got to Graceland Elvis was listening to a song that he let play over and over. When they asked why, he said it was the song of the moment. Elvis would listen to the same same song over and over, until he had internalized it to the point where it became a part of him. It represented where he was at that point in time, and became a touchstone.

As an artist (Particularly in my Theatre days), I would often latch on to themes and mechanism which I would replay over and over. As a technical architect in the software industry, I do something similar when learning a new technology or experimenting with a new technique. I’ll find I’ve emerged myself entirely in that technology, design pattern or methodology. I particularly do this on personal projects where can truly experiment and push the boundaries, sometimes beyond practical purposes.

As developers and software artists I think it’s important to sometimes do these types of things. Only by practicing, pushing and challenging our art can we find new paths. With that in mind I’d like to propose the “Technique of the Moment.”

Technique of the Moment

If you want to learn a new technique, and learn it thoroughly, why not fire up a personal project or 5 and use that technique over and over until you not only understand what you can and can’t do with it, but keep working with it until you have internalizes it. Push your understanding to the point of being able to quickly pull it out when needed, and know without a doubt that it is the right tool, for the right job, at the right time.

Okay, this one is a bit out there, but it’s late, and this notion of the song of the moment has been running through my mind all day.

SOA As A Distributed Neural Network

Posted by Will Strye

As part of a casual conversation a few weeks ago I started thinking about how I’d go about programming a large scale distributed AI. In the scenario being discussed the AI would be running the combat simulations for a starship. This was one of these, “How would you create this piece of SciFi tech?” conversations.

Now let me start with the caveat that I’m not an AI specialist. I toyed around in the space some years ago, but there are certainly those in the world who know much more about this than I. In fact what I’m going to talk about has almost certainly been detailed out in someones masters or doctoral thesis, and I have no illusions that this is anything original.

So with all of the disclaimers out of the way, lets jump in and take a look at what this type of SOA/NN AI might look like. This will be at a high level and really talk more about commonalities in order to intertwine the two concepts. Afterwards I’ll delve into some possibilities of why we might want to create such a beast.

I think that the number one thing that even makes this a possibility is the autonomous principle of SOA. Each node of a SOA should encapsulate all of it’s logic and be call-able for a single function/purpose. This simple concept can be seen in the node of a neural network. In a neural network a single node takes an input, analyzes the data and passes on an output to the next node. Once the output is passed on, the node is completed. Now there are some differences. For example, it’s generally expected that a node in the higher levels of a SOA may need to call into the lower level services in order to complete it’s function. Think about a piece of business logic that needs to call down into the application layers of one or more applications in order to get the data needed for the operation. This isn’t the norm for a neural network node, which instead expects that all data needed for the operation be passed in as inputs. While there are obviously other differences let’s look as one possible way to bridge these two concepts.

Now in order to bridge some of the gaps between the two, what if we created a series of services that lived behind our normal SOA services? This extra dimension of our SOA would serve to facilitate more organic processing. In practice this is just a special implementation of the synapses in an artificial Neural Network. The purpose of which is to use a set of weighted parameters in order to pass the outputs of one service (node) to the next in the chain. Now, at first I had thought of this as just a more detailed orchestration layer, since one of it’s major jobs would be to usher the inquiry through the process, but then I realized that it didn’t really act the same as a SOA orchestration. An orchestration sits above the worker layers of the SOA, and calls down into it. What I needed was something that filled the gaps between the lower level services.

Another factor that would probably be needed, are some specialty services that might wrap around normal SOA services in order to plug them into our synapses properly. Think of it as a hidden layer of dark matter that allows our universe to balance. In fact that’s exactly what I’m going to call it, the “dark matter layer.”

Now at this point hopefully I’ve painted a picture that allows you to accept the premise of an artificial Neural Network that can be based on an existing SOA. I may explore more details of the technology in future posts, but for now I’d like to talk about why we’d ever consider even doing this.

So, we have our existing SOA, and we’ve setup a framework to attach an artificial Neural Network onto it. What could we do with something like this?

The most common use might be to create predictive models based on real time data derived from the corporate SOA. Using AI for predictive modeling is not a new concept by any means. However, these are traditionally programs the chunk through extremely large amounts of data stored in a data-warehouses, and are designed to track trends over longer periods of time such as months and years. What if instead we looked for predictions of what a workload might look like over the next few hours, based on realtime data. An AI routine like this might be running in the background all the time, constantly updating the predictive model.

With the above node in place we might then create another AI system that would be a consumer of this model ,and would take some form of action based on it’s results. For example a complex routing system might utilize the predictive model in order to determine the most efficient way to route work. Now instead of a simplistic First In First Out (FIFO) work queue, we now have a much more dynamic queue, that will assign work based on the predictive model and perhaps job priority. The assessment of job priority might in and of itself be a neural network that is basing it’s judgements on realtime data.

While this might not be a good approach for all situations, it is a viable way of thinking about enterprise systems and SOA’s for any company that wants to not only be able to react to changing situations quickly, but wants to have an infrastructure that can react for them.

Thanks for indulging on this one, and as always, keep thinking malignant thoughts.

Agile As a Sequential Art Team

Posted by Will Strye

Last time I ended by bringing up the metaphor of the agile team as a team of sequential artists with the Customer in the role of writer. This time I’d like to further explore that metaphor and see if we can find something viable here we can latch onto. So while this is a thought exercise I’m always looking for ways to explain how agile teams work to those not familiar with the methodology. I’m also always looking for ways to improve how our agile teams work. Let’s see where this metaphor can take us…

For the sake of our analysis I’m going to pick the most straightforward form of sequential art, the graphic novel or as it’s colloquially known, the comic book. I’m making an assumption that I don’t need to write a definition here, but if you feel you need refresher here is a link to what wikipedia has to say on the topic.

In order to dig deeper into this metaphor let’s try by defining the different roles in a typical comic team and compare them to an equivalent on an agile development team.

Writer: The writer is the one who owns the story. They define who the characters are, what motivates them, and what they do in the world of the story.

Agile Equivalent: Customer. As stated before, this is our agile customer. They own and define our agile stories. Too often I think people see the Customer as the Editor and place the Architect in this role. Unfortunately this places the Customer in an after the fact role and distances them from the team. We’ll talk more about the role of the editor later and I think you’ll better understand what I mean.

Penciler: Sometimes, though not always, referred to as the artist. This person takes the writers script and gives live to the story. They work closely with the writer and together are typically seen as the creative team of a comic.

Agile Equivalent: All Developers. I’d really like to make a case here that every developer has the potential of playing this role on a project. Remember that in agile we have many stories, and the execution or development of each story each story may be owned by a different developer. In this way each developer has the potential of being the penciler of a story with support from one or more other developers in the following roles.

Inker, Letterer and Colorist: I’ve lumped these three roles together not to diminish them, but rather to set them up as both support and specially roles. Indeed an inker on one store may be the penciler on another. And understanding how the application of certain colors will look in the final print copy is invaluable.

Agile Equivalent: All Developers. Again because agile is many stories, within the course of one project each developer may not only play the role of penciler, but will also play supporting roles for others stories, as their skills warrant.

Cover Artist: This is usually an artist of some renowned. While they may not be a part of the permanent team they are brought in to create the covers of comics. These are usually given more time and attention to detail as they are intended to draw potential readers in and entice them to purchase the comic.

Agile Equivalent: Change Management Specialist. It would be easy to say that this role is equal to a technical specialist brought in to solve some particularly challenging problem, and I can’t argue that this is a valid analogy. However; I think that a more useful correlation is with change management. A good change management plan is the number on away to improve adoption of a system, which after all is the goal of cover art.

Editor: The editor is responsible for the quality of the product. They need to make sure that the work begin done meets the needs of the publishers overall goals, and also help to endure that everything gets done on time. Editors also help to remove roadblocks and sooth artist egos when needed.

Agile Equivalent: Scrum Master. Nuff said.

Art Breakdowns: This role is not often seen in comics as the penciler will normally take on this duty. However; on occasion an artist will be brought in to sketch out all of the flows within a comic. this is usually done in cases where multiple creative teams are weaving a series of stories together. Each team is telling their own story , but this over arching artist is preparing the page layouts and weaving all of the story arches into a seamless whole. These breakdowns are usually rough sketches so as not to override the vision of any one team, and in fact the final art, while adhering to the page layouts will look dramatically different from the sketches provided. In many if not all cases the artist doing the breakdowns works closely with all teams to ensure that each teams vision is respected while at the same time guaranteeing the overall quality of the whole.

Agile Equivalent: I think there are two answers to this. The first is a bit more traditional and really only applies in certain types of projects, while the second tends to be overlooked or out and out dropped from scope, but in my not so humble opinion is potentially more important.

Enterprise Architect. In a large enterprise project or when dealing with multiple competing platforms, it is the job of the EA to make sure that all the pieces flow from one to another properly, each performing it’s part in the whole. In some ways this role can also be considered a part of the Editors role as an editor is often a key figure in preserving continuity across story arcs. Think governance and compliance.

Experience Architect. This is a relatively new role in the software world, and suffers from much of the same ambiguity that enterprise architecture did a few years back. Some feel it belongs more in the domain of the designer and others feel it belongs more in the world of the business analyst. Personally I don’t think either is truly accurate. This is a role that’s emerging out of a gap in the current skills and roles as we have them defined in software development. I’ll try to post more about the the definition of this role in a subsequent post, but for purposes of this topic, the individual filling this role needs to make sure that all of the various moving pieces come into play in a way that creates a satisfying experience for the user, and that going from one part of a system or application to another is not jolting or abhorrent. This means understanding all components of a system, and understanding how both design and technical decisions will impact the experience.

While this metaphor isn’t 100% accurate I think it offers an interesting slant on the agile team and it’s dynamics. Particularly in illustrating the importance of certain non technical roles. I especially like how it points out the importance of the customer as a vital and active part of the team. To often clients say they understand their role in an agile project, but when it comes time to start work they don’t understand why the team wants so much of their time.

My company uses the slogan “The art and science of winning.” In that spirit I think this metaphor helps point out the fact that the methodology can truly be seen as the art of science as it pertains to delivering software.

Thanks for reading. I know this was a long one. And, as always, keep thinking malignant thoughts.

To Architect Or Not To Architect

Posted by Will Strye

OR… question = (2_architect | !2_architect);

I just posted to a thread on linked in about whether or not you should blow out a full architecture on a project to let the design evolve over time. I’m reposting and expanding may response here because I think there are some important elements to the topic worth consideration.

This is a topic that I’ve seen pop up again and again since I’ve started doing Agile.

The basic premise is that if you’re using sound design principals/patterns, and have a solid understanding of what the customer wants, (Remember you’ve gone through release planning with the customer, and have hopefully developed a solid product backlog) then you can allow the architecture to evolve iteration after iteration. Using techniques, such as refactoring and strangulation, you can keep the code base moving forward without ripping things out and starting over.

At this point let me point you to an article that Martin Fowler wrote back in 2003 called, “Who Needs an Architect?” (I told you this debate has been going on for a while.) In the article he describes the role of the architect within an agile project, and even has a section towards the end called, “Getting rid of software architecture.” Whether you agree with him or not, it is an interesting read, and one of the best arguments I’ve ever heard for not doing a lot of architecture up front.

Personally I always try to do as little as I can get way with up front, in order to allow for future scenarios I can’t see today. I always have an over-arching architecture in my head, but I don’t want to make it real until I need to. That way if new requirements pop up later, the project has not backed itself into a corner that might cause major overhaul. “What do you mean we’re going to deploy all the services as workers in the cloud? We just spent two weeks setting up an application server farm!” is something I hope never to hear because wrote something down on paper too soon.

When I look at the times where I “Over-Architected” a solution, it was usually because I had to have the full architecture documented before the team could write code. As a consultant I don’t always get to do things my way, and this includes the use of Agile. In these situations it can be easy to overthink things, as there tends to be pressure to “Get it right up front,” and “Account for all eventualities.” In my humble opinion this is one of the top reasons projects fail. In cases where requirements are simple, or very well defined, it’s ok, but oftentimes customers don’t really know what they want up front. A good indicator of this is when you start getting conflicting requirements.

If you’ll allow me the metaphor, I like to think of the software architecture as a collaborative art installation, and the role of the architect as one more of chief artist. As chief artist I select the medium we’ll work in, wood, canvas, marble, etc. If we’re working on canvas I will usually step in and take the requirements of the customer and begin to paint in the larger areas. I may lay down a patch of azure across the top of the canvas for the sky, and then work with the other artists to begin painting clouds, birds, balloons, or whatever else is needed. As we reach stories that involve a landscape or ocean, I may decide the placement, size and/or shape. Again blocking the pieces in, then working with the team to fill in the detail.

Now there is a place where this metaphor begins to break down for me, and that is in regards to the customer. In our metaphor the customer is exactly that, a customer, and our installation piece begins to be a commissioned piece of art. The reason that this bothers me is that commissioned works of art tend to be seen as somehow lacking in quality in comparison to art created for the sake of the artist’s own vision. At it’s worst I think this type of commissioned art can be equated to purchasing a painting to hang in your living room that matches your couch and curtains. In our tech metaphor this might be buying off the shelf software, or utilizing inexpensive labor. You tell them to draw a line, they’ll draw a line, and it will be mostly straight. While there is certainly a place for this type of art it leaves me wanting more. So lets see if we can’t fix our metaphor to make it more appealing.

So what if our customer is more than our customer? So what if instead of our band of artists creating just any type of art, we are instead are focused on creating sequential art? We can still keep our artistic mediums open. For example instead of sequential images that tell a story we might install a series of life-sized dioramas along a park trail that reveal the story. By making this simple change we can now put our customer in the role of writer. Our customer has now magically been transformed from patron and bill payer to an integral and active member of the artistic team. Now we’ve got a metaphor we can really work with, and in true agile fashion we’ve gotten there through iteration and a desire to keep our customer integrated with the team.

Next time I’ll explore the metaphor further and see if we can’t iterate against it even further.

Until then, keep thinking malignant thoughts.

Year of The Lemur

Posted by Will Strye

If I were an international spy or super villain, I’d want to be known as “The Lemur.” I’d get a lemur, name him Maurice, and train him to discretely take care of my enemies… This post has nothing to do with lemurs. Nor does it have anything to so with my plans to rule the Evil League of Evil (E.L.E). Sorry.

2009 was not a good year for me. I don’t feel the need to go into any details, and all things being equal it certainly doesn’t compare to the year my father passed away. However, as we head into the end of January I’m feeling good about 2010, and wanted to share some of the reasons why.

In 2009 I spent too much time trying to do what I thought was right for others rather than simply what I thought was right. When I tried to fit into the mold others wanted to put me in, I was unhappy and, surprise, failed. I’ve gotten to where I am because I don’t fit the mold. This year is about being my non-conformist self again, with no apologies.

I’m revitalizing my interest in technology and solutions I care about, and continuing my personal journey and exploration of how technology can be used to communicate. I’ll have more to say on this later.

I’m going to be writing more. This is one area where I actually thought I had kept momentum up last year. Then I realized that the majority of writing I did in 2009 was either at the beginning of the year or at the end. My goal this year is to not loose momentum. This also extends into my work with sequential art. In addition to the blog I also have some side projects that marry both technology and writing. I’m actually very excited about this and hopefully will be publishing further information about this in the next month or so.

This year more than any other I feel like the confluence of my skills, knowledge and interests has great potential. It’s this potential confluence that I’d like to spend the rest of the post exploring. When I mentioned my exploration of technology and communication earlier it was a bit myopic. Before I got involved in the technology sector I was a trained stage director and playwright. One of the primary themes I continually touched on was how people communicate, or rather how they don’t communicate.

This theme was fueled by something I see as a part of the human condition. It isn’t just a need to communicate with one another, but also a need to commune with one another, to understand in an unspoken way. This is the same need that spurs us to create art. Now there are some very practical reasons this need is baked into our dna. Humans are social animals, and it is this need to live, work and communicate that allowed us to survive in the face of nature and other predators. Humans are not the only social animals on the planet, not even the most socially centered. The insect world beats us hands down when it comes to structured social behavior. However I think it was the way this need to commune with one another combined together with a capacity for knowledge acquisition and application, which turned our tribes into cities. As our technology takes us beyond our physical communities and borders, we still have a need to bond and communicate. I think we take for granted how much interaction and communication is non-verbal. The saying that a picture is worth a 1000 words is not only true, but I’d bet that as you read it you pulled up a mental picture that demonstrated the concept. When you talk to someone on the phone do you mentally picture that person as they talk? As our communications get briefer how many of us resort to emoticons and lolspeak to help fill in the gaps and communicate mood? We are at a point where we need to look at how we communicate and interact in our new cyber communities. We need to look not only at how to use the tools we have but also to discover new tools to move us forward.

We all traverse along our own journey though our lives whether we choose to recognize and embrace it, or simply go along for the ride. Mine has taken me down a path from art to socio-cultural anthropology (for lack of a better term), and then turned down the path of technology. As an artist I explored how we communicate, which got me interested in human culture and social interactions. As a technologist I have been interested not only in how we interact with our technology, but also how our technology interacts with other bits of technology on our behalf. My hobbyist interest in AI was never been to be able to build sky-net, but to use it as a model to better understand human thought and motivation. I feel that there is an opportunity, and in some ways a need, to bring all of these interests and disciplines together that excites me.

So. With all of this in mind, I hereby declare 2010 the year of The Lemur.

PS: I used the word “confluence” a few times in this post. I feel this word, more than any other, helps describe the themes I’m talking about here. To show how long I’ve been ruminating on some of these topics, when I ran my own theatre company back in the mid 90’s I named it the Wormwood Theatre Confluence. I’ve also owned the domain dataconfluence since 2003, though I’ve never done anything with it.

The malignancy of genius – or – what’s in a name?

Posted by Will Strye

I wanted to talk a little bit about the name for the blog, how it came about and what it means. The title Malignant Genius is on the surface exactly that. To be honest I was actually looking for something along the lines of Evil Genius, but of course all of those were taken. I stumbled across Malignant Genius in my quest and liked the sound of it. In fact the more I thought about it the more I liked it, and it eventually turned into a philosophical title for me.

So as I said on the surface the title is exactly as stated. In fact the tagline is, “When you think like a genius, think malignant thoughts.” Not sure it anyone has caught that as it’s tucked away in various places on the site and not in your face. Under the surface this title has become much more to me. Simply put I feel that in some respects genius itself is a malignancy. Perhaps not something to be cut out like a tumor, but at times detrimental all the same. (And before you ask, yes I do meet the technical qualifications of genius and am a member of American Mensa.)

By way of an example of the malignancy, a few of my co-workers joke about what I’m like before my first cup of coffee. Until I’ve had at least some form of stimulant to start my day I’m pretty much incoherent and a bit belligerent. Now this isn’t because I drink so much coffee that I can’t function without it. I’ll let you in on a little secret. While I walk around with a coffee cup all day long I only drink 2-3 cups in a an average day, and 1 when I’m not at work. No the reason I can’t function without my morning libation is because I don’t fall asleep until 1 or 2 in the morning. Most nights I can’t shut off my brain long enough to fall asleep. I can be absolutely exhausted and as soon as my head hits the pillow my brain kicks into overdrive and starts running down scenarios and chasing after flights of fancy. About mid week my body generally can’t take it anymore and overrides my brain sending me straight to sleep, but the next night the whole thing starts over again. This is not a good thing.

Another example is the fact that I sometimes wish I was ignorant of what was going on around me. A few weeks ago one of my devs replied to a question I asked him in email. At the end of his response he asked if he could get a few minutes of my time the next day. My brain instantly jumped to the conclusion that he was going to give his notice. Guess what my brain did until 2 am that night. The next day sure enough he tendered his resignation. <sigh> Ignorant people seem so happy.

That’s not to say I would have the malignancy cut out, just that I wish I could shut it off once in a while. I love the fact that when presented with a problem by brain fractures into a million possibilities, each informing the others. Expanding some and collapsing others. I like the fact that my value is in presenting the non intuitive answer or piece of the puzzle that can push a solution from good to great. But god I wish I could sit in a meeting and not be analyzing the subtle interactions of the participants. Watching the social games play out and gauging not only where I stand at any given moment, but how everyone else does as well. Kicking myself because of some perceived slip on my part that most likely meant nothing to anyone else. Or worse, kicking myself because some intended action was lost unnoticed in the white noise of social communication.

With this post I’m starting a new category called Malignancy. It will be inclined towards the more philosophical and anthropological as apposed to the technical and artistic.