A Definitive Guide: How To Migrate From Zoho to GSuite (Or any other service)

A Definitive Guide: How To Migrate From Zoho to GSuite (Or any other service)

I do not want to get into the reasons of why someone would want to migrate their email account from Zoho, a free/cheaper service, to GSuite, a paid service. For small businesses, Zoho is a really good service provider, but it has some drawbacks and so, if someone needs to migrate away from Zoho to GSuite, here is a tried and tested way to reduce your hassles with some handy tips along the way.

Some Considerations

Now this is your business email, which means there is no reason you should trust an off the shelf, free/freemium, closed source product with a website filled with bright colored, blinking stickers saying “Free”.

You would not want to use a hundred different tools for conversions, it is just too much of an effort.

You would not want to give a tool full fledged access to your business email or account.

As you are migrating from Zoho, to GSuite, let us assume you do not use a Microsoft Windows machine, so random tools to convert various different email formats or ‘one-click’ transfer are not really for you, good riddance.

With that, here we go.

Step 1: Export Zoho Mails

Go to your Zoho email account, into the settings and select “Export History”. Now here, you have to select each of your folders individually to export and create a downloadable package. Select “All emails” as the date range. Wait for the archive to be ready, and download. Now you have, say 3 different zip files each containing all emails from specific folders in ’eml’ file format.

Name each zip conveniently so you know which one has which folder, without having to open it every time.

Step 2: Setup your GSuite Account

Go to GSuite home, and click “Start free trial”. GSuite shall then take you on a guided tour and ask you all the details about the account you wish to create.

Provide your domain name, if you already have one. Don’t worry about the migration of MX records, GSuite does it for you automatically, if they have integration with your host, if not there is a simple documentation on how to do it when you reach that stage.

Choose your plan. If you are coming from Zoho, you likely were on free plan. So go for the “basic” GSuite plan. Finish the setup.

At the end of the setup, you should now start receiving new emails on Gsuite instead of Zoho mail. And you will see that Zoho sends a security warning email, stating that the MX records seem to have changed, little do they know we are doing it deliberately.

When asked for billing in GSuite, there is a concept of Billing profile. A billing profile is like a billing record, linked to a Google account which can be chosen for certain payments. Some services let you create multiple profiles. Here though, it asks if this account is for a Business or an Individual. Since this payment profile will be used for paying on behalf of the business, select Business and proceed. Also, do check the tax implications of this by clicking the little info icon in front of the option.

Step 3: Setting up Thunderbird

Now on to the hard part. GMail/GSuite cannot directly import the ’eml’ files generated by Zoho export tool. This is where most of the confusion comes from in this migration. There are some tools offering to convert eml to mbox and some other tools offering to import mbox into GSuite; but none of these pass of considerations/criteria we started off with. So no game.

Thunderbird is an open source email client form Mozilla, and a brilliant one at it. After the setup if you can even consider keeping it around, for now. For now, we are just concerned with importing the Zoho dump into GSuite, so just download Thunderbird from their official website and launch it.

Thunderbird shall ask you login credentials to access the new email account. Once you enter your credentials, Thunderbird should autodiscover that the account is a GSuite account and pick imap.gmail.com/smtp.gmail.com as addresses. If this does not happen, it is mostly because the DNS records are still not synchronized and you should try again in some time.

Once you logged in, you should see 2 mailboxes in Thunderbird, one is your GSuite box and the other is something Thunderbird calls ‘Local Folders’ Local folders are, well, local and not synchronized with any online service. Go to Thunderbird’s file menu -> tools and open add-ons section and install an add-on called ImportExportTools.

For each of your exported directories, create a new folder under “inbox” in GSuite account. The folder gets translated as a label in GMail called “INBOX/<dirName>”. Create a new folder called “placeholder” under your Local Folders, we are going to re-use this folder.

Step 4: Importing EMails

In case you received any new mails in your new GSuite account, take a back up of those mails by exporting them from email options in GMail.

The next is a tedious, but critical step.

Unpack your Zoho packages into separate directories. Right click the “placeholder” folder and select “ImportExportTools” -> Import all messages from a directory -> Just from the directory. And navigate to one of your unpacked directories and click “Open”. This should import all the eml files in that directory into your local folder. Once the process is finished, select all the files in the folder and drag-and-drop them into the corresponding folder under your GMail/GSuite account.

Wait for the process to finish, it may take a few minutes depending on number of emails and network speed etc, but we need to do this one at a time. Creating multiple placeholder folders does not help, one of the process halts and then it needs to be repeated.

Creating a folder under online inbox helps in case something goes wrong, you can select the folder as a label in GMail and delete all the emails that were just imported.

GMail’s conversation identification works brilliantly in mapping the sent/received emails in conversation, so don’t worry about that. If you have to re-import a directory, Thuderbird automatically de-duplicates the email, so no need to worry about that either.

Now why did we import to local folders and then dragged the mails to online folder? Well, there is a known bug that happens if you directly import emails into an online folder, they lose the their initial timestamps. It is talked about in a lot of forums related to GMail, IMAP, in general. ALL EMAILS will look like they were received when they are actually uploaded to GMail and so the Local Folders step is crucial for this to work.

Step 5: Rearrange emails

Now that you have all your emails uploaded to GMail, they have correct time stamps, the conversations are mapped, the issue is these emails now have a weird label. Get rid of the label and move the emails to inbox by adding the inbox label to incoming emails.

You should now have a functioning mailbox in GSuite.

Practical Git

Practical Git

This is a discussion on Git, the version control system. It is intended for folks with:

  1. Limited or no knowledge of any version control system
  2. Experience with Centralized Version Control Systems
  3. Experience with DVCS, but specifically git.

It covers the basics of version control, DVCS, and then goes on to describe Git in a practical way, with exercise to get the team quickly started on using Git.

It also talks a bit about how Git works internally, giving insights into the .git directory in each repository in order to demystify the commands.

It then moves on to Git workflows usable for various teams and also covers a one-line glossary of terms a person is most likely to encounter when using Git via CLI or through any UI tool.

Text Transcript

Slide 1:   Practical Git
Introduction (And Beyond)
                          @nikhilwanpal | NikhilWanpal

Slide 2: Version Control
 (The non-developer way)
                         @nikhilwanpal | NikhilWanpal

Slide 3:                       “With review comments from Guide”
                      “Modifications based on new
                      “Changes for team presentation”
                      “Changes for presentation at
                      conference, for novice audience”
                      “Merged changes: team and
                      “Cosmetic changes”
Version Control
  (The developer way)
                                       @nikhilwanpal | NikhilWanpal

Slide 4: Version Control Systems
1.  Local
2.  Centralized (CVCS): svn, cvs
3.  Distributed (DVCS): git, mercurial
                                       @nikhilwanpal | NikhilWanpal

Slide 5: Once upon a time..
● There was a Linus
● Then came Linux
● Then came the interest of the community and contributor boom
● Contributors distributed through time and space.
● The central person would become overworked, loaded.
● Need for distributed management.
● (skipping some events..) Birth of Git: To handle far complex, large and distributed
  teams. (than us)
● ..and then, came the interest of the community and contributor boom… (git
  uses git to version control git!)
                                                                     @nikhilwanpal | NikhilWanpal

Slide 6:                 CVCS                         Vs                   DVCS
1. Central Server, that manages the truth.      1. Distributed, no single location of truth.
2. Clients do checkout snapshots.               2. All clients are mirrors; servers; truth.
3. Weak clients                                 3. Fully functional clients
4. Server needs tending to: backups,            4. Self maintaining, recoverable from mirrors;
   maintenance etc.                                (of course should have backups.)
5. File locking / conflict handling.            5. No Locking; conflicts are less frequent.
6. Constant connection required.                6. Connection required only when sharing.
7. Defined workflow.                            7. Highly flexible workflows possible. With
8. Restricts free development of open source       subteams and sharing and merging before
   projects.                                       final publishing.
                                                8. Promotes Open Source development.
                                                                             @nikhilwanpal | NikhilWanpal

Slide 7: Understanding Git (coming from a world of CVCS)
● It is distributed: Things appear to be two step.
● Think of ‘torrents’, P2P networks. (It’s not a P2P and won’t help you download GOT!!)
● Like installing your own SVN server that can communicate with other SVNs.
● Branching is dirt cheap; not a task.
● Merging is easy; not an activity. (If you know what you are merging!)
● It’s not difficult, just different. (unless our glass is full)
                                                                      @nikhilwanpal | NikhilWanpal

Slide 8: Git Hosting
●  GitHub Vs GitLab Vs BitBucket
●  Git : GitHub = water : packaged drinking water
●  All hosting providers add proprietary features to git, ex: pull request.
●  Hosting providers do, can provide other VCS as well.
●  Git does not need hosting, or server or background process
●  Git can work with a shared folder as remote
●  Hosting makes corporate workflows easy
                                                                     @nikhilwanpal | NikhilWanpal

Slide 9: Your Git
● Install Git
● Setup
  git config --global user.name "Nikhil Wanpal"
  git config --global user.email "nikhil@dontwasteyourtimereading.com"
                                                                       @nikhilwanpal | NikhilWanpal

Slide 10: Your Git, Your First Repository
In a new directory: practical-git/1/
 ●   git init
 ●   Create file, git add, git commit -m
 ●   Modify file, git add, git commit -m
 ●   git log
 ●   Modify file, git add, git commit --amend
 ●   Modify file, git add
 ●   git reset
                                              @nikhilwanpal | NikhilWanpal

Slide 11: Your Git, Your First Repository with a server
In a directory: practical-git/central-repo.git/
      ●   git init --bare
In a directory: practical-git/personal-repo/
      ●   git clone ../central-repo.git/ .
      ●   git add
      ●   git commit -m
      ●   git push
                                                @nikhilwanpal | NikhilWanpal

Slide 12: Your Git, Your First Repository with a server and a
dual personality
In a directory: practical-git/central-repo.git/
      ●   git init --bare
In a directory: practical-git/personal-repo/
      ●   git clone ../central-repo.git/ .
      ●   Create file, git add, git commit -m
      ●   git push
                                                @nikhilwanpal | NikhilWanpal

Slide 13: Your Git, Your First Repository with a server and a
dual personality (v1)
In a directory: practical-git/colleague-repo/
      ●   git clone ../central-repo.git/ .
      ●   Surprise!
      ●   Create file, git add, git commit -m
      ●   git push
In a directory: practical-git/colleague-repo/
      ●   git pull
                                              @nikhilwanpal | NikhilWanpal

Slide 14: Branches and Tags
                             @nikhilwanpal | NikhilWanpal

Slide 15: V1.1 Branches
● Threads of development
● String of thought
● Series of changes with similar purpose
● A diversion
  git branch myFirstBranch
● Create a file, git add, git commit -m, git push
  git checkout master
                                                  @nikhilwanpal | NikhilWanpal

Slide 16: V1.2 Merges
● Tie the threads together
● Bring together different thoughts and ideas
● Achieve the purpose of your branch
  git merge myFirstBranch
● Git’s intelligent merge
  git checkout myFirstBranch, modify line 1, push
  git checkout master, modify line 2, merge
                                                  @nikhilwanpal | NikhilWanpal

Slide 17: V1.3 Conflicts
● What happens when two different lines of thought try to merge?
● Branches need a judge!
● You!
  git checkout myFirstBranch, modify line 2, push
  git checkout master, modify line 2, merge!
● Now try that across repositories, you have 2.
                                                              @nikhilwanpal | NikhilWanpal

Slide 18: V1.4 Rebase
 ●   Rewriting history
(after same steps as a conflict..)
     git pull origin master --rebase
                                     @nikhilwanpal | NikhilWanpal

Slide 19: What is a:
● Git commits storage: a rooted, Directed Acyclic Graph of patches.
● Patch: the delta between two commits. (or more)
● Staging/Index: Selecting relevant changes for commit.
● Commit: The delta store in git filesystem with name as the SHA.
● Branch HEAD: A pointer in the graph for ease of access.
● Branch: The path from root to branch head.
● Stash: Stash aside the state for now.
● HEAD: a variable, a pointer to current pointer of the current branch.
● Detached HEAD: state of repository when a commit is checked-out, which is
  not pointed to by any of the HEADs
                                                                 @nikhilwanpal | NikhilWanpal

Slide 20: What is a:
● Merge: Delta from the ‘common ancestor’ added together.
● Fast - Forward Merge: Pointer updated to the latest head.
● Merge commit: the commit that identifies a merge.
● Rebase: Rewrite the history to change the branching point, and reapplying
  the changes over. No more the same commits. (creating new history is that easy..!)
● Reset: undo, hard vs soft.
● Remote: The different repositories, tracked branches, not same branches.
● Pull: fetch + merge (rebase!)
                                                              @nikhilwanpal | NikhilWanpal

Slide 21: Git Folders:
 ●   HEAD: pointer to current branch’s head
 ●   index: staging info
 ●   refs: commit objects. Basically branch data.
 ●   objects: blobs of files and tree objects.
Git Objects:
 ●   blobs: or git objects contain the contents of checked files. Key-value file
 ●   trees: pointers to blobs by filenames and other trees.
 ●   Commit: top level tree, user, additional info regarding commit, message etc.
 ●   Packfiles and git gc | auto gc.
                                                                    @nikhilwanpal | NikhilWanpal

Slide 22: To Branch or not to branch..
What is a branch? (There is no such thing as a branch! It’s the path from the head to root,
traversed through ‘parent’ pointers.)
Git References:
       ○   Branches
       ○   Tags
       ○   HEAD
       ○   Remotes
Are you coming back to it? Then you need a branch!
                                                                            @nikhilwanpal | NikhilWanpal

Slide 23: Git Flow
●  A branching model suggested by Vincent Driessen in his blog.
●  develop and master: The only long lived branches.
●  Features, Releases and Hot-fixes
●  Convention: feature/; release/ and hotfix/
●  Life-cycle of:
    ○   Feature: develop → develop
    ○   Release: develop → (master + tag | develop)
    ○   HotFix: master → (develop + master(tag?))
                                                              @nikhilwanpal | NikhilWanpal

Slide 24: Git Shortcuts
Bash / shell aliases:        Git aliases:
 ●  alias gs='git status '    ●   git config --global alias.st status
 ●  alias ga='git add '       ●   git config --global alias.a add
 ●  alias gc='git commit'     ●   git config --global alias.ci commit
 ●  alias gb='git branch '    ●   git config --global alias.br branch
 ●  alias go='git checkout '  ●   git config --global alias.co checkout
                                                    @nikhilwanpal | NikhilWanpal

Slide 25: Git Best Practices
●  Commit often (every 30 mins), push once. Clean-up before push.
●  Prefer to code on new branch locally, never push such branches. Share
   among developers but not to central.
●  One change per commit. Not more.
●  Describe the commit well.
●  Consider rebase before push or pull, follow up with a --no-ff commit.
●  Don’t break the development tree.
●  Review merges. Build and fix post a merge before push.
●  Avoid force-delete (-D) when deleting branches.
●  Consider using shortcuts/aliases.
●  NEVER rebase pushed commits.
                                                                  @nikhilwanpal | NikhilWanpal

Slide 26: }
  @nikhilwanpal | NikhilWanpal

Using the link below, you can also download the PDF file for offline reference. Although, I prefer to revisit and update my presentations, you can always refer to the latest version here.

Setting up VPN on ubuntu 19.10 and getting DNS to work correctly

Setting up VPN on ubuntu 19.10 and getting DNS to work correctly

A bit late to post this? May be. But it is still relevant, so here goes. We shall see how to setup openvpn client on Ubuntu versions greater than 18.10 (or was it 18.04?) It certainly matters for 19.10 though!

A few things changed back then that caused issues in openvpn connection and DNS resolution. There was a fix found, and package released bundling the scripts so that normal users like us can install and enable those scripts. This solved the CLI problem. But I personally, do not like the idea of having to run a sudo command every time I need to use VPN and keep a terminal running. I like the flexibility Network Manager gives and also the fact that it nicely indicates with an icon if I am connected to VPN or not.

But to achieve this setup, a few steps are required:

  • Download your ovpn config file. This might require you to login to your OpenVpn server and download the ‘autologin’ config file.
  • Run this command: apt install openvpn-systemd-resolved. This will install the packages / scripts required to handle DNS resolution on VPN.
  • modify your ovpn file, add these lines before ca certs and after setenv:
    script-security 2
    up /etc/openvpn/update-systemd-resolved
    down /etc/openvpn/update-systemd-resolved
  • At the end of the file, after all certs and everything, add: dhcp-option DOMAIN-ROUTE .
    (note the “dot” in the command above.)
  • Connect to vpn using command: sudo openvpn --config <path to modified config file>

We have now achieved the basic setup, you can stop here. Just open terminal when you need to connect to VPN and run the above command; press Ctrl+C to disconnect from VPN. To integrate with Network Manager, follow through next steps:

  • We now need to import this file into Network Manager and use Network Manager UI to connect/disconnect from VPN.
  • Install sudo apt install network-manager-openvpn-gnome. Even if you are running KDE, the same command works.
  • Next, run this command to import the config file: sudo nmcli connection import type openvpn <modified config file> . On ubuntu 21.04 or 21.10, you might need to use this command instead: sudo nmcli connection import type openvpn file <modified config file>
  • This should bring a new network in your Network Manager Settings UI, under VPN header. The name of the connection will be same as the name of the file.
  • If this is a work VPN, which means you will need to use this VPN for connecting to work machines only, and rest of the traffic should go outside of VPN. To enable this, go to IPV4 settings, and select ‘Use this VPN for resources on this network only‘. On most work VPNs, if you do not enable this, you will lose ability to use anything other than vpn servers, even google wont work.
  • Now the next, tricky part. This sets us up for most of the things, but in my experience DNS servers still do not resolve properly. To enable DNS resolution via Network Manager, in IPV4 settings of the VPN network add “~.” (without quotes) to “Search Domains“. In my experience, without the ~. as search domains, the DNS resolution did not work.

This is it. The benefit is a built in, native, visual representation of VPN connectivity right on your System Tray!

[TechNggets] Podcast Is Moving To tnt-podcast.com

[TechNggets] Podcast Is Moving To tnt-podcast.com

This is not an episode of the Tech Nuggets And Thoughts podcast, this is just an update.

The podcast is moving to a new home: https://www.tnt-podcast.com

Please follow the new website or various other podcast aggregators for updates on new episodes and topics. You can find all the applicable links in sidebar or on the new website.

The blog will continue to be the vessel for my personal ramblings as usual.

[TechNggets] Episode 4: Delivering With Confidence: Unit Testing – Part – I

[TechNggets] Episode 4: Delivering With Confidence: Unit Testing – Part – I

This is the forth episode of podcast “Tech Nuggets and Thoughts”.

In this episode, we talk about Unit Testing covering points such as:

  1. What is testing?
  2. What are software requirements?
  3. The test pyramid
  4. Importance of business in software and what it means for testing.
  5. Test principles: F.I.R.S.T
  6. Test First approach
  7. What is a unit, in unit testing?

To get updates, you can subscribe to the podcast on: Apple iTunesGoogle Podcasts, player.fmRSS feed, FeedBurner, Stitcher Radio or TuneIn . If you have any suggestions, thoughts or recommendations, please feel free to comment below. You can also reach me on podcast’s twitter handle @TechNggets or my personal account @nikhilwanpal.

(If the fancy player above does not work, try the bare bones player below.)

Understanding Microservices: Exploding A Monolith

Understanding Microservices: Exploding A Monolith

This is a myth-buster exercise. When it comes to microservices, the talks and blogs around are riddled with fancy and scary terms. This presentation is takes a different take on it. Instead of learning the terms, we discover the solutions to various problems and then learn what those solutions are popularly referred to as.

We go through the history of software architecture to understand how it evolved in the good old days. Based on which we draw some rules of evolution. These rules when applied to a monolith we evolve to a microservices based architecture as a natural or obvious evolution. Attendees do this as an exercise to understand the importance of the components in a microservices stack.

Using the link below, you can also download the PDF file for offline reference. Although, I prefer to revisit and update my presentations, you can always refer to the latest version here.

Who do we write code for?

Who do we write code for?

Recently, while reviewing some code, I came across the snippet below.

Now a keen mind can spot many issues with this.

The ones I am referring to are something specific and relevant to the business though. The code is supposed to fetch all active clients, loop through, get job specifications from some DataSources defined for those clients, build a config object and return those values to calling function so that those jobs can be fired. Interestingly DataSourceInstance, DataSourceOriginConfig and DataSourceJobConfig definitions have nothing named as triggers, report or client_id_internal! Moreover, destination_params is a property of Client object, not of DataSourceJobConfig. I could not make heads or tails of it. To top it, I was told the code is already working and tested, review is just a formality (like it usually is in many places).

When I started digging through the code further, I found that getDataSourceJobConfig, was accepting list of dsConfigs, but instead of returning a list of jobConfigs it was enriching objects in dsConfigs and returning the same list. Further digging revealed that the same was being done in both the methods above. Effectively, all these variables were the exact same list of exact same objects, mutated in every call, were actually the list of Clients!

Now, I understand that JavaScript is loosely typed, you can modify objects on the fly without adhering to any defined structure. But because one can, does not mean one should. This code was so cryptic that I was forced to ask the question: “Who have you written this code for”, and the answer was unexpected, “the clients who are paying for it”. I gathered courage and asked again, “What do you hope to achieve specifying these steps in code”, this time the answer was expected: “So the program does so and so”.

This is a question I always loved when I first read Clean Code by Robert Marin (Uncle Bob). The answer takes some thought, but once you realise it, it all makes sense. We write code to tell our team members what we are trying to instruct the machine to do. The primary consumers of the code, even before the destined machine are the author and the peers who see, read, understand, refactor and maintain the code. If all one wants is for the machine to get the message, then one can talk in binary, computer languages are for people to convey to each other what they want the machines to do, along with instructing the machines.

To prove this point I have built a small snippet (and a cryptic language) and compiler. Here it is, can you guess what this code does:

Before you say it is invalid, gibberish and would just fail, let me assure you that every word in that snippet is useful, and has an appropriate meaning for what it is doing. If I were to compile this and execute it, it will work. In fact I used this snippet in a presentation to make this very point, when it worked, the output was even more confusing. Now I would be lying if I said that the compiler just compiled the java like file, it also did some transformations, like ‘afdrukken’ (Google Translate says that is dutch for ‘printing’) was being replaced with System.out.println, as if it was a function similar to python’s print.

The point I am trying to make is that instructions to a computer can be given is many ways, we use ‘Computer’ languages, with meaningful constructs and words so we understand what others are trying to instruct the computer to do.

Even before it executes, the code is read first by the writer, is read often by peers and so the goal of the code is to get the intent correctly across not only to the computers but also to other humans.

We code first for humans and then for machines.

[TechNggets] Episode 3: Circular Dependency

[TechNggets] Episode 3: Circular Dependency

This is the third episode of podcast “Tech Nuggets and Thoughts”.

To get updates, you can subscribe to the podcast on: Apple iTunesGoogle Podcasts, player.fmRSS feed, FeedBurner, Stitcher Radio or TuneIn . If you have any suggestions, thoughts or recommendations, please feel free to comment below. You can also reach me on podcast’s twitter handle @TechNggets or my personal account @nikhilwanpal.

(If the fancy player above does not work, try the bare bones player below.)

Where Do We Go Wrong With Agile

Where Do We Go Wrong With Agile

A few days ago I published this blog mocking the ‘Ceremony Focused Agile’ teams. But it is pointless to state what one thinks is not right, without also commenting on what one thinks is right! So let us do that today. Here is a conversation, which I am sure many of us have witnessed (or been part of):

A: “I have assigned a ticket to you, what is the status of it?”

B: “”Ticket, what ticket?”

A: “It’s in JIRA, check your board.”

B: “Okay, but what is it about?What is the context?”

A: “It’s in JIRA”

B: “huh?”

In Agile teams, people believe in using JIRA to track their work, hours spent, communication / discussion about a feature, proposed features, discarded features, its this amazing one-stop shop for all Agile stuff.. What a tool! (pun not intended!) We are told that using JIRA helps us track time, keeps everything organised, ensures no one can go back on their words or commitments. Which is true, using a single, capable tool for tracking everything related to a task would ensure that everything that happened in the context of a task. But let us not mistake using JIRA as practising Agile.

Agile Manifesto

Agile manifesto states in four, very clear and concise statements:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

Yet, the reality is that these statements are understood by everyone differently. Some folks took away a different meaning from the word Agile itself, some did from the manifesto. Over time certain practices appeared, which seemed to work for certain use cases effectively. And many of us seem to have assumed their success is absolute, and expect them to work outside of the teams they were successful in. Somewhere we forgot, every team is different, so is agile for every team. It is certainly not a set of ‘mandatory rules’ that apply to every team the same way.

Then What Is Agile

AgilE to me is AtoE, or ‘Adapt to Evolve’. All the statements in the manifesto promise that when you change focus from tools/processes to people, from documentation to software, from contract to customer and form fixed plan to flexible plan, you will be able to evolve to match market needs and deliver a successful product. But then, how do we do it? Does it mean that we should stop using all tools/processes? Or signing contracts? Certainly not, these things are still important, but there is only so much that they can do. The manifesto tries to bring priority to them, stating that a doing is more important than planning, recording and tracking. That does not mean these are unimportant, but only that they are of less importance that the action itself.

When we put focus on people, and reduce the importance of processes and tools in the day to day life of these people, they interact more and better. This interaction results in better understanding and integration of the team itself. This results in better delivery. There is no tool that can match a person to person interaction. I understand, the interactions at times may not be face to face, but people have always communicated better when they talk than when they write. ‘Intent’ is always difficult to convey in writing, writing lacks tones. Also, these tools are asynchronous, adding further delays.

In the words Working Software, ‘working’ I believe is the important term. Working does not just mean ‘able to execute’, working means software fulfilling its purpose. A software is built to address a problem statement. As time passes the problem statement evolves, may be to adapt to the market, or to the changing landscape, or to fill a niche, but always with the intent of making solution more relevant. And hence the definition of ‘working’ keeps changing. And so the software should too. A Software is a way of specifying the requirements. This specification is a set of instructions, given to a machine to perform an algorithm, in a way that they can be clearly understood by others contributing to the specification. Also, if the software is supposed to replace the comprehensiveness of the documentation, without impacting the intended purpose of the documentation itself, then the same job must be done by the software equally well! In other words, the specification i.e. code, should read like a documentation! Now these, I believe constitute the ‘working’ of the software.

Engaging with the customer helps teams better understand their perspective which reflects in software. Contracts are required to ensure the business aspect of the software development, but they should not be a hindrance to value creation. The first and foremost job of an Agile team is to bring value to the customer, also referred to as Stakeholder. In the rawest of terms, it is more important to see if and how a task being done will bring value to the customer than looking for a way to create a new Change Request. 😉 Customers in software services industry are mostly external to the company or team and so collaboration becomes all the more important. In poetic terms, it is important to think of the software as ‘for us’ and not ‘for client’, and involving the customer aids this thinking.

It is a bit cliche to mention this, but ‘Change is the only constant’. There is no defined long term plan that can be executed exactly as expected, especially when the definition of ‘working’ itself is so prone to change. We can define a direction, and end goal, but we cannot define strict plan and even if we could, not sure it would help the changing context, except when ‘change is the plan’! Being able to incorporate change is the primary goal of an Agile team. Interestingly, when comparing different Agile frameworks you can see that one of the most important differentiating factor is the ‘frequency at which to expect change’. An Agile team should be ready and open to change than being strict about adhering to a plan. There are still plans, but in much smaller scale.

The Unsaid Requirements Of Agile

With all the points above, agile is said to give control to the team. An overused, always misunderstood term, “giving control”. And what does it mean? The ability to start or stop something, the ability to take decisions and executing them, that is control. Now I wonder, is it possible to give control, without having ‘trust‘? I do not see it possible, ever to do so. Would you ever give control of your car (project execution) to someone and ride along, without first trusting them to (intentionally or unintentionally) not kill you? Not possible. Traditionally, the management has always enjoyed this control. They have always believed that them making the decisions and team obeying and executing them is the right way. Agile requires them to relinquish this control, and it is certainly not going to be easy unless they trust the team to do the right thing.

So, the ‘management’ of an Agile team needs to first trust the team.

Now, would you trust a driver with no driving skill? Nope! The team needs to have the skill to make right choices, execute and deliver. Without this skill it is not possible for the management to trust the team. and without trust it is not possible for a team to be Agile.

An Agile team needs to have the required skill-set to make the project a success.

Would you trust a skilled, but irresponsible driver with your car and life? Would you trust a skilled, responsible yet unwilling driver, to take you to your destination? Certainly not, you and your car might end up in a ditch with the driver unscathed will move on to drive a different car! (analogy!) It is irrelevant if the driver is skilled, unless the driver is willing to take you to your destination, her skills are useless. The goals must align.

An Agile team must believe in the same goal as the management, must be willing to do what is needed to get there.

I always see the last requirement as a bit tricky. Why would a team of skilled, free-thinking individuals believe in someone else’s goal? This is where the ‘people interactions’ come in. It is not going to happen unless the team trusts the management to do right by them. Ah, it’s a game of trust, skill and will. There is no Agile without these. These assumptions should have been recorded somewhere, because this is the part that many teams and many traditional managers fail to understand. What this results into is a ceremony we like to call Agile.

The Ceremony

There are many ‘frameworks’ of Agile. Many different ways that can help you implement the manifesto better and they have different guidelines, like all thought-processes do. But these are guidelines, not rules. They cannot be forced on an unwilling team to beat them into being Agile. The core concept of Agile is that the team decides the practices they want to follow, in which form, to identify flows and improve on them. (Remember, we have already trust that the team is skilled and is willing to do right by this project.)

When we see meetings, call timings and statements forced upon team, they become mere ceremony, they lose their meaning, purpose and the result is a failure. A failure to achieve the goal, failure to build the team and a failure of the practice itself.

There are certain tools and practices though, that explained as being a part of Agile. These again are not rules, but arise from the need to respond to change rapidly. We need to deliver fast, and to do that. we need faster verification of software hence the need for Continuous Integration. To be able to deploy fast we need Continuous Delivery, so the artefacts are ready to go live, daring teams can even try Continuous Deployments. To deploy fast, we need to make the provisioning and configuration of systems automated and simplified, hence the need for DevOps. A car can go only as fast as the breaks can allow, so to deliver fast we also need to be able to revert fast, hence the need for artefact repositories and blue-green deployments. We need to change the specification quickly and hence the need to verify the specification at granular level, hence the need for Unit Tests. Since we are so focused on time, we should write specifications only for what is required, hence the TDD/BDD. Since the specification’s job is to convey intent to others, we need to have more than one person on the team who can understand the code, and to save time and effort, we have Pair Programming. Again, just following such does not make you Agile, similar to how not following some of them does not make you ‘not Agile’. Using Jenkins/GoCd, JIRA, Artifactory/Nexus etc tools does not make you Agile, and not using them, for a better alternative your team has established which allows you to act faster, does not make you ‘less Agile’.

And many other terminologies you would hear in Agile, know that are not part of the specification or requirement or some rule. Some of these things may help you be Agile, but the Agility is always in the context of what your team thinks is necessary to achieve the goal for your Customers.

The Markers

Enough theory, how do I tell if my team is Agile? Well, I have tried to build a list of markers that I have seen in non-agile Agile teams. Now this is certainly not an exhaustive list, and it is certainly not a rule, but indicators that can help identify the ceremony than agility.

  • You have a “manager”.
  • Your manager/scrum master, or someone ‘assigns’ you ‘tasks’, rather than you picking them.
  • This someone asks you for ‘status’ in your daily meetings. This is a bit tricky, remember you have control, and so you have responsibility to convey the status of the work you picked. It might be a failure on your side or management’s.
  • If your team has members reporting to different people, not in hierarchy.
  • If you learn about your tasks only via a tool, and also report via a tool.
  • If you prefer a tool or email over talking/quick calls when you need to discuss with your team.
  • If you have to jump through hoops and cc ten people to be able to talk to the customers.
  • If your productivity is measured solely in terms of ‘tickets’ fixed or moved.
  • If you as a team never meet to discuss what can be improved or you conclude that nothing can be!
  • If you do not know what others in your team are working on, blocked on and you are not helping to unblock them.
  • If you as a team are not driving to complete the goal of the iteration as a whole, and instead focus on finishing your work alone.
  • If you as an individual are not learning any new skill required by your work or performed in the team.
  • You have not changed the way you are working / following Agile practices in a long long time!


Revolutionising Agile With Head-Stand-Ups

Revolutionising Agile With Head-Stand-Ups

So here’s a true story. I work with a normal sized team as per Agile/Scrum guidelines, about 8 people. We have our usual stand-ups every day, at about 10:30 in the morning. As a general rule, people have to join this meeting. Everyone speaks following the Scrum rules, just what is required: What I did, what I am going to do today and if I am blocked. And that’s it. Yet, our discussions diverge, others jump in to help whenever someone mentions why they are blocked, suggesting how it can be solved. It is helpful, it speeds us up in ways because it avoids need for a one-on-one meetings later, and team generally returns feeling the meeting was fruitful.

Yet, it results in our meetings getting extended, we have our stand-ups almost 20, at times 30 minutes long. This is something that has been on our mind for quite some time now, as a general rule, the stand-up should not be longer than 10 minutes, that is why we ‘stand up’, so we realise the time being spent physically, with a minor discomfort and that drives us to close the meeting sooner.

This is when I read about the “plank meetings”. Excited, I suggested to the team and everyone agreed agreed that this could help us reduce the time we spend in stand-up considerably, while giving everyone a healthy boost. We agreed to put it in practice. Now, for those who do not know what plank meetings are here are couple links explaining them. We decided we would start the next Monday, with the next sprint. But destiny seemed to have some different plans. During my yoga session on Sunday morning, it dawned on me that there could be something much more effective than this. It was the Shirshasana that gave me this idea! Shirshasana is an asana in yoga, where you stand on your head and keep the body and legs straight up, like a plank but upside down. This! This could be the thing we need to keep the timings in check.

Now Shirshasana, as a yoga form, has many, many benefits of its own. It is known to direct the blood flow to head and eyes, relieving stress, improving focus, and digestion. But the quality we are looking for most was that it makes us temporarily uncomfortable! Perfect! On the planned day, we started with this instead. There was just one simple rule:

  • No one speaks without first assuming Shirshasana form.

Now you see, there is no rule about when to speak, or who speaks. That is because on the first day we observed that entering this form and exiting from it is not easy, we needed help from others. Meaning it was intuitive that anyone who wanted to speak would have to wait for their turn and to get help from others to speak! This caused a revolution amongst us, we realised that this one change alone has brought us many benefits. Here are our observations:

  • First and foremost, meetings became short.
  • Meetings became fun.
  • Our habits changed to be more healthy. It is not advisable to consume food before performing Shirshasana, and hence we automatically started following better timings for breakfast.
  • Our team found that we were more interactive throughout the day. Somehow activity of helping each-other daily, and needing help from others was turning into a team-building exercise of sort. We may just have saved our company thousands of dollars.
  • More people around us took interest in this seemingly odd practice of standups and that created awareness about health and yoga across the organisation.

We as a team have never been better! Who needs stand-ups when ‘head-stand-ups’ are so rocking!

PS: Reading between the lines is an important skill, for others there are warnings in F12/console.