Browsed by
Tag: tools

Where Do We Go Wrong With Agile

Where Do We Go Wrong With Agile

A few days ago I published this blog mocking the ‘Ceremony Focused Agile’ teams. But it is pointless to state what one thinks is not right, without also commenting on what one thinks is right! So let us do that today. Here is a conversation, which I am sure many of us have witnessed (or been part of):

A: “I have assigned a ticket to you, what is the status of it?”

B: “”Ticket, what ticket?”

A: “It’s in JIRA, check your board.”

B: “Okay, but what is it about?What is the context?”

A: “It’s in JIRA”

B: “huh?”

In Agile teams, people believe in using JIRA to track their work, hours spent, communication / discussion about a feature, proposed features, discarded features, its this amazing one-stop shop for all Agile stuff.. What a tool! (pun not intended!) We are told that using JIRA helps us track time, keeps everything organised, ensures no one can go back on their words or commitments. Which is true, using a single, capable tool for tracking everything related to a task would ensure that everything that happened in the context of a task. But let us not mistake using JIRA as practising Agile.

Agile Manifesto

Agile manifesto states in four, very clear and concise statements:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

Yet, the reality is that these statements are understood by everyone differently. Some folks took away a different meaning from the word Agile itself, some did from the manifesto. Over time certain practices appeared, which seemed to work for certain use cases effectively. And many of us seem to have assumed their success is absolute, and expect them to work outside of the teams they were successful in. Somewhere we forgot, every team is different, so is agile for every team. It is certainly not a set of ‘mandatory rules’ that apply to every team the same way.

Then What Is Agile

AgilE to me is AtoE, or ‘Adapt to Evolve’. All the statements in the manifesto promise that when you change focus from tools/processes to people, from documentation to software, from contract to customer and form fixed plan to flexible plan, you will be able to evolve to match market needs and deliver a successful product. But then, how do we do it? Does it mean that we should stop using all tools/processes? Or signing contracts? Certainly not, these things are still important, but there is only so much that they can do. The manifesto tries to bring priority to them, stating that a doing is more important than planning, recording and tracking. That does not mean these are unimportant, but only that they are of less importance that the action itself.

When we put focus on people, and reduce the importance of processes and tools in the day to day life of these people, they interact more and better. This interaction results in better understanding and integration of the team itself. This results in better delivery. There is no tool that can match a person to person interaction. I understand, the interactions at times may not be face to face, but people have always communicated better when they talk than when they write. ‘Intent’ is always difficult to convey in writing, writing lacks tones. Also, these tools are asynchronous, adding further delays.

In the words Working Software, ‘working’ I believe is the important term. Working does not just mean ‘able to execute’, working means software fulfilling its purpose. A software is built to address a problem statement. As time passes the problem statement evolves, may be to adapt to the market, or to the changing landscape, or to fill a niche, but always with the intent of making solution more relevant. And hence the definition of ‘working’ keeps changing. And so the software should too. A Software is a way of specifying the requirements. This specification is a set of instructions, given to a machine to perform an algorithm, in a way that they can be clearly understood by others contributing to the specification. Also, if the software is supposed to replace the comprehensiveness of the documentation, without impacting the intended purpose of the documentation itself, then the same job must be done by the software equally well! In other words, the specification i.e. code, should read like a documentation! Now these, I believe constitute the ‘working’ of the software.

Engaging with the customer helps teams better understand their perspective which reflects in software. Contracts are required to ensure the business aspect of the software development, but they should not be a hindrance to value creation. The first and foremost job of an Agile team is to bring value to the customer, also referred to as Stakeholder. In the rawest of terms, it is more important to see if and how a task being done will bring value to the customer than looking for a way to create a new Change Request. 😉 Customers in software services industry are mostly external to the company or team and so collaboration becomes all the more important. In poetic terms, it is important to think of the software as ‘for us’ and not ‘for client’, and involving the customer aids this thinking.

It is a bit cliche to mention this, but ‘Change is the only constant’. There is no defined long term plan that can be executed exactly as expected, especially when the definition of ‘working’ itself is so prone to change. We can define a direction, and end goal, but we cannot define strict plan and even if we could, not sure it would help the changing context, except when ‘change is the plan’! Being able to incorporate change is the primary goal of an Agile team. Interestingly, when comparing different Agile frameworks you can see that one of the most important differentiating factor is the ‘frequency at which to expect change’. An Agile team should be ready and open to change than being strict about adhering to a plan. There are still plans, but in much smaller scale.

The Unsaid Requirements Of Agile

With all the points above, agile is said to give control to the team. An overused, always misunderstood term, “giving control”. And what does it mean? The ability to start or stop something, the ability to take decisions and executing them, that is control. Now I wonder, is it possible to give control, without having ‘trust‘? I do not see it possible, ever to do so. Would you ever give control of your car (project execution) to someone and ride along, without first trusting them to (intentionally or unintentionally) not kill you? Not possible. Traditionally, the management has always enjoyed this control. They have always believed that them making the decisions and team obeying and executing them is the right way. Agile requires them to relinquish this control, and it is certainly not going to be easy unless they trust the team to do the right thing.

So, the ‘management’ of an Agile team needs to first trust the team.

Now, would you trust a driver with no driving skill? Nope! The team needs to have the skill to make right choices, execute and deliver. Without this skill it is not possible for the management to trust the team. and without trust it is not possible for a team to be Agile.

An Agile team needs to have the required skill-set to make the project a success.

Would you trust a skilled, but irresponsible driver with your car and life? Would you trust a skilled, responsible yet unwilling driver, to take you to your destination? Certainly not, you and your car might end up in a ditch with the driver unscathed will move on to drive a different car! (analogy!) It is irrelevant if the driver is skilled, unless the driver is willing to take you to your destination, her skills are useless. The goals must align.

An Agile team must believe in the same goal as the management, must be willing to do what is needed to get there.

I always see the last requirement as a bit tricky. Why would a team of skilled, free-thinking individuals believe in someone else’s goal? This is where the ‘people interactions’ come in. It is not going to happen unless the team trusts the management to do right by them. Ah, it’s a game of trust, skill and will. There is no Agile without these. These assumptions should have been recorded somewhere, because this is the part that many teams and many traditional managers fail to understand. What this results into is a ceremony we like to call Agile.

The Ceremony

There are many ‘frameworks’ of Agile. Many different ways that can help you implement the manifesto better and they have different guidelines, like all thought-processes do. But these are guidelines, not rules. They cannot be forced on an unwilling team to beat them into being Agile. The core concept of Agile is that the team decides the practices they want to follow, in which form, to identify flows and improve on them. (Remember, we have already trust that the team is skilled and is willing to do right by this project.)

When we see meetings, call timings and statements forced upon team, they become mere ceremony, they lose their meaning, purpose and the result is a failure. A failure to achieve the goal, failure to build the team and a failure of the practice itself.

There are certain tools and practices though, that explained as being a part of Agile. These again are not rules, but arise from the need to respond to change rapidly. We need to deliver fast, and to do that. we need faster verification of software hence the need for Continuous Integration. To be able to deploy fast we need Continuous Delivery, so the artefacts are ready to go live, daring teams can even try Continuous Deployments. To deploy fast, we need to make the provisioning and configuration of systems automated and simplified, hence the need for DevOps. A car can go only as fast as the breaks can allow, so to deliver fast we also need to be able to revert fast, hence the need for artefact repositories and blue-green deployments. We need to change the specification quickly and hence the need to verify the specification at granular level, hence the need for Unit Tests. Since we are so focused on time, we should write specifications only for what is required, hence the TDD/BDD. Since the specification’s job is to convey intent to others, we need to have more than one person on the team who can understand the code, and to save time and effort, we have Pair Programming. Again, just following such does not make you Agile, similar to how not following some of them does not make you ‘not Agile’. Using Jenkins/GoCd, JIRA, Artifactory/Nexus etc tools does not make you Agile, and not using them, for a better alternative your team has established which allows you to act faster, does not make you ‘less Agile’.

And many other terminologies you would hear in Agile, know that are not part of the specification or requirement or some rule. Some of these things may help you be Agile, but the Agility is always in the context of what your team thinks is necessary to achieve the goal for your Customers.

The Markers

Enough theory, how do I tell if my team is Agile? Well, I have tried to build a list of markers that I have seen in non-agile Agile teams. Now this is certainly not an exhaustive list, and it is certainly not a rule, but indicators that can help identify the ceremony than agility.

  • You have a “manager”.
  • Your manager/scrum master, or someone ‘assigns’ you ‘tasks’, rather than you picking them.
  • This someone asks you for ‘status’ in your daily meetings. This is a bit tricky, remember you have control, and so you have responsibility to convey the status of the work you picked. It might be a failure on your side or management’s.
  • If your team has members reporting to different people, not in hierarchy.
  • If you learn about your tasks only via a tool, and also report via a tool.
  • If you prefer a tool or email over talking/quick calls when you need to discuss with your team.
  • If you have to jump through hoops and cc ten people to be able to talk to the customers.
  • If your productivity is measured solely in terms of ‘tickets’ fixed or moved.
  • If you as a team never meet to discuss what can be improved or you conclude that nothing can be!
  • If you do not know what others in your team are working on, blocked on and you are not helping to unblock them.
  • If you as a team are not driving to complete the goal of the iteration as a whole, and instead focus on finishing your work alone.
  • If you as an individual are not learning any new skill required by your work or performed in the team.
  • You have not changed the way you are working / following Agile practices in a long long time!

 

Mute Mic With Keyboard Shortcut On Ubuntu Or Linux Mint

Mute Mic With Keyboard Shortcut On Ubuntu Or Linux Mint

Here is a quick tip for all the automation buffs like me. Turn your mic on and off with just a keyboard combo.

I do all my work remotely. Which is also to say I have a lot of conference calls. And like you, I hate it when people do not mute their phones / mic on laptops when not speaking! (cue in the obligatory meme about not putting your phone on mute during a call!)

I always wished for a hotkey of some sort to mute / un-mute myself during a call. So here is a way to do it.

  1. Use a Linux machine. (This in itself is a great tip! 😉 ) These steps in particular are for an Ubuntu / Linux Mint machine.
  2. Put the following snippet in a bash script file and add it to path. You can also define it as a bash alias and load it from your custom bash profile, but then assigning it a shortcut may not be that easy.
  3. Set a keyboard shortcut to trigger this script. I use Meta+M for this.

Here are the two variants of the script for that:

This script toggles mic state, shows a nice (transient) notification of the changed state, with an intuitive icon! It should also replace previous notification quickly, but somehow it does not seem to work yet.

(cue in the meme about speaking on mute! ;))

Docker As Application Registry

Docker As Application Registry

Docker As Application Registry

Docker is great and solves a lot of problems with deployments. It taught VMs to share the resources, like how VMs taught hardware to share resources! Along with production, I have found that docker can work great as an application registry in a local development environment.
By applications I mean software that you install on your OS and launch them with shortcuts and they continue to live and retain state till you uninstall them; not exactly what containers are designed for but can work as. Something like snap or flatpak but with docker and for servers as well, not just for UI apps.

One advantage this has over using standard installers (like apt-get) is you are at complete liberty to start and stop the background processes, like mysql. If you installed mysql this way, you do not need to go and disable the autostart for it, it just does not matter! Similarly for your SonarQube server, you do not need to install it as a daemon, neither do you need to remember where you downloaded it to be able to restart it. Another advantage is most of such applications do have official docker images, it is the intended way to use them now!

One disadvantage of this method is that you always need to address them with their IP, you will not be able to bind them on host network then. But in my view, it is always better to have a dedicated IP, it emulates production scenario better and does not clutter your local machine ports.
So for apps what you need is containers to live long, be able to identify them with name, start and stop them easily and have a dedicated, static address to be able to reach to them. Most of these things are easy, except for a static IP. But once you create a network, you are set. That is it, it is that easy! Create a virtual network, and start your dockers with a name and static IP in that network. Simple!

To create a network:

docker network create -d bridge --subnet="172.30.0.0/24" --gateway="172.30.0.100" --ip-range="172.30.0.0/24" permanet

Now any app you need, just specify this network and a static IP of your choice; like this:

docker run --name mysql-server --network="permanet" --ip="172.30.0.1" -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql/mysql-server:5.7

More complex containers can be created like this:

docker run -it --name gocd-server --network="permanet" --ip="172.30.0.5" -v /yourhome/docker-volumes/gocd/godata:/godata -v /yourhome/docker-volumes/gocd/home:/home/go gocd/gocd-server

I keep a dedicated directory in my home for docker volumes, so I can back it up and use them as is when I change machines or OS. I also have a script where I add all the containers I need, so it is just a matter of copying the volume directory and running the script to create identical setups. Then even your .desktop files work as is!

Here is an interesting setup script for Jenkins (gist), it externalises all data directories from Jenkins, including the plugins and users and mounts your local m2 repository inside Jenkins so as to avoid downloading the libs again :

docker run -it --name jenkins --network="permanet" --ip="172.30.0.6" 
-v /yourhome/.m2:/var/jenkins_home/.m2
-v /yourhome/docker-volumes/jenkins/workspace:/var/jenkins_home/workspace
-v /yourhome/docker-volumes/jenkins/jobs:/var/jenkins_home/jobs
-v /yourhome/docker-volumes/jenkins/plugins:/var/jenkins_home/plugins
-v /yourhome/docker-volumes/jenkins/users:/var/jenkins_home/users
jenkins:2.32.3-alpine

You can create as many such networks as you wish for logical separation of groups of such apps; in my case this is the third network (172.30 for that reason), since first two were taken up by some compose scripts.

A list of few such containers I use: mysql (different versions), SonarQube, hystrix-dashboard, zipkin, swagger-ui, a redis-cluster for local use, gocd-server, Jenkins, portainer, postgres, pgadmin etc. I even have a couple Windows software running on wine in such containers, we shall talk about it some day.

Habit-Firebug Saver: SmartLogger

Habit-Firebug Saver: SmartLogger

How many of the web developers do not depend on Firebug or the chrome’s console… Just wondering..

BTW, its plain fun to work with firebug, makes life a lot easier.. Its a different matter all together that the other browser that you have to develop for does not have a powerful enough tool. (Name deliberately avoided to avoid the eminent flame-war!) Yes the current versions have a quite powerful debug and development tools but (hopefully) few developers working on products still have to consider some 10 year old versions (namely 6, 6.5 and 7). Ah, the pain.. Anyways, we are not discussing that..

What we are talking about is the issues that we face when testing our changes to a thousand lines JavaScript code on multiple browsers, especially after we are accustomed to the ease of firebug. 🙂

I spent much of my time commenting my console.log() statements before I could dare to open the page in IE. Well, fear not, the days have passed! The pain drove me to write a logger object that can not only sense presence of console object but can do much more than that, like ability to assert, selective logging and more..

I call it the SmartLogger.

//Global Logger Object, for use during development, configurable logger.
var SmartLogger = function(options) {

var sl = {}; // Logger Object

// Accepting passed params.
options = options || {};
sl.enableLogger = options.enableLogger!==undefined?options.enableLogger:true;
sl.enableAssert = options.enableAssert!==undefined?options.enableAssert:true;
sl.loggerOutput = options.loggerOutput!==undefined?options.loggerOutput:undefined; //'console', 'alert', undefined
sl.selectiveEnable = options.selectiveEnable!==undefined?options.selectiveEnable:'';
sl.selectiveDisable = options.selectiveDisable!==undefined?options.selectiveDisable:'';

// Logger properties
sl.name = "SmartLogger";
sl.whoami = function(){ return "SmartLogger_"+sl.enableLogger+"_"+sl.enableAssert+"_"+sl.loggerOutput+"_"+sl.selectiveEnable+"_"+sl.selectiveDisable;}
sl.version = '0.7';

// Checks if console object is defined. Checked only at the time of instantiation.
var hasConsole = (typeof console === "object");

// Checks if logging should be done to console.
function logToConsole(){
if (sl.loggerOutput){
if (sl.loggerOutput === 'console') return true;
} else {
if(hasConsole) return true;
}
return false;
}

// Handles the logging intelligence
function handleLogging(logMethod, logString, strId){
if(!sLog(strId)) {return;}
// Decides if to log and logs or alerts appropriately.
if(sl.enableLogger){
if (logToConsole()){ // && hasConsole
if(hasConsole)console[logMethod](logString);
} else {
alert(logString);
}
}
};

// Handles the selective logging functionality
function sLog(strId){
var allowLog = true;
if (sl.selectiveEnable) {
allowLog = strId === sl.selectiveEnable;
} else if (sl.selectiveDisable) {
allowLog = !(strId === sl.selectiveDisable);
}

return allowLog;
};

// Returns a formatted object structure with current values to complete depth.
function printString(obj, name, str, strEnd){
var stringified;
name = name?name:"Object",
str = str?str:"";
strEnd = strEnd?strEnd:"";
stringified = str+name+" : {n";
for (var a in obj){
if (typeof obj[a] === 'object'){
stringified+= printString(obj[a],a,"t",",");
} else {
stringified+= str+"t"+a +" : "+obj[a]+",n";
}
}
stringified += str+"}"+strEnd+"n";
return stringified;
};

// Exposed methods of the object
//log a string to console/alert
sl.log = function(str, strId){
handleLogging('log', str, strId);
};

//debug logging a string to console/alert
sl.debug = function(str, strId){
handleLogging('debug', str, strId);
};

//write an information string to console/alert
sl.info = function(str, strId){
handleLogging('info', str, strId);
};

//throw error string to console/alert
sl.error = function(str, strId){
handleLogging('error', str, strId);
};

//Assert an assumption
sl.assert = function(str, strId){
if(sl.enableAssert){
handleLogging('log', 'Assumption: true', strId);
if(!str){
handleLogging('error', 'Assumption failed!', strId);
debugger;
}
}
};

// Logs the formatted object structure with current values to console/alert
sl.stringToConsole = function(obj, str){
sl.log(printString(obj, str));
};

return sl;
};

var sl = new SmartLogger();

Features:

  • Multiple logging profiles can be maintained at the same time with different properties.
var sl = new SmartLogger();
var sl2 = new SmartLogger({selectiveEnable: 'block1'});
  • Proprieties can be set at the time of instantiation or even later.
var sl = new SmartLogger();
sl.loggerOutput = 'console';
var sl2 = new SmartLogger({loggerOutput: 'console'});
  • name, version number and whoami to identify the logger with a string of its current properties.
var sl = new SmartLogger();
sl.name // SmartLogger.
sl.version // 0.7
sl.whoamI() // Returns a string of its properties with the name of the object in a specific sequence:
// "SmartLogger_"+ enableLogger +"_"+ enableAssert+"_"+ loggerOutput+"_"+ selectiveEnable+"_"+ selectiveDisable;
// Example: SmartLogger_true_true_console__b
// We will see what these properties are in some time..
  • Enable or disable logging altogether: enableLogger controls if the statements should ever be logged.
var sl = new SmartLogger();
sl.log('gets logged');
sl.enableLogger = false;
sl.log('never gets logged');
  • Intelligently decides where the logging statements should go…
sl.loggerOutput = undefined; //default
/* Decides based on presence of 'console' object.
If console is present statements will be logged to console,
else like in case of IE, will be 'alerted' to the user.
Now at times this can get messy, with loads of log statements alerting on our face..
But wait, we have ways to handle that.*/

sl.loggerOutput = 'console';
// Plain instruction, no intelligence, all statements will always go to console.
// If console is not present statements will just be eaten-up.

sl.loggerOutput = 'alert';
// Another plain instruction, all statements will always be alerted.
// Will not bother to check if console exists or not.
  • Log formatted objects to console. Now you wont need that much with firebug but to see the entire contents of the object, well formatted you can just say stringToConsole.
// Just a sample object with unknown properties.
var obj = {prop1: 'value',functProp:function(){return "this is a function that returns me!";}, propObj:{prop2:'value2'}};
sl.stringToConsole(obj); // You say this.

// On console or in the alert prompt, you get this
Object : {
prop1 : value,
functProp : function () {
return "this is a function that returns me!";
},
propObj : {
prop2 : value2,
},
}
  • Assert your assumptions. Checks that the assumption is true, if yes, logs so. If assumption fails, will write out an error on the console and invoke the debugger so the user can check in the stack exactly where the assumption failed and why.
sl.assert(Obj.str===Obj2.str)
sl.assert(1==1); // logs 'Assumption: true' to console.
sl.assert(1==2); // Logs error 'Assumption: failed!' and invoke debugger to the assert line in SmartLogger.

//Now you can go and check in the stack and watch to panels to check value and call stack.
  • Has a wrapper for 4 of the logging APIs from firebug and adding new is not much of a task. What it already has:
    • log
    • debug
    • info
    • error
  • Has ability of selective logging.
Now this thing is a live saver. The properties selectiveEnable and selectiveDisable control what statements to log. While these are not mandatory inputs to all the wrappers but I suggest you set them always. These are logging context that can be used to selectively enable logs for only partial of the code, the code that currently interests you..
// Suppose we were working on a defect number 101 and now we are developing a functionality for
// automating welcome messages to users and are asked to urgently fix defect 203.
// Ah, complex scenario, but it will only help understand the purpose.

// When we are working on defect 101, we had the logger configured as statements as:
sl.log("reached here"); // worst way to log: who knows wheres here! but just an example.

// Now we are working on the functionality and
// we would not want those 10 logging statements added while we were working on the defect.
// We can remove them or simply enable the 'selective logger'!
sl.selectiveEnable = 'welcomer';
sl.log("fetching message", "welcomer");
// And voila, only the 'welcomer' messages will be logged.

// Now we get the next urgent defect.
sl.selectiveEnable = 'defect203';
sl.log("value in Obj1.str"+Obje1.str, "defect203");
// We get only the defect203 logs!

// Now some of our new changes depend on the changes we made in defect101, but we cant get the logs from those..
// What do we do? If someone did not enable selective logger and removed the statements, please add them back (:p),
// remove statements for 'welcomer' functionality. Or simply, disable 'welcomer' messages..!
sl.selectiveEnable = '';
sl.selectiveDisable = 'welcomer';
sl.log("value in Obj1.str"+Obje1.str, "defect203");
sl.log("value in Obj2.varInt1"+Obj2.varInt1, "defect101");
// Ha ha! Log statements for 'welcomer' gone and we get the rest!
While using the SmartLogger, I suggest you always pass the string identifier, so that you can control the logs at any point of time later.

What can you expect next in SmartLogger:

  • Use assert from firebug itself.
  • Check that the function exists in the logger before calling it.
  • Make the selective logger take arrays.
  • In stringToConsole, handle functions too to remove that glitch in no closing bracket.

Let me know what you think about the SmartLogger, if you would like any additions to its behaviour and also if you find any defects in the comments below.