Browsed by
Category: tools

Docker As Application Registry

Docker As Application Registry

Docker As Application Registry

Docker is great and solves a lot of problems with deployments. It taught VMs to share the resources, like how VMs taught hardware to share resources! Along with production, I have found that docker can work great as an application registry in a local development environment.
By applications I mean software that you install on your OS and launch them with shortcuts and they continue to live and retain state till you uninstall them; not exactly what containers are designed for but can work as. Something like snap or flatpak but with docker and for servers as well, not just for UI apps.

One advantage this has over using standard installers (like apt-get) is you are at complete liberty to start and stop the background processes, like mysql. If you installed mysql this way, you do not need to go and disable the autostart for it, it just does not matter! Similarly for your SonarQube server, you do not need to install it as a daemon, neither do you need to remember where you downloaded it to be able to restart it. Another advantage is most of such applications do have official docker images, it is the intended way to use them now!

One disadvantage of this method is that you always need to address them with their IP, you will not be able to bind them on host network then. But in my view, it is always better to have a dedicated IP, it emulates production scenario better and does not clutter your local machine ports.
So for apps what you need is containers to live long, be able to identify them with name, start and stop them easily and have a dedicated, static address to be able to reach to them. Most of these things are easy, except for a static IP. But once you create a network, you are set. That is it, it is that easy! Create a virtual network, and start your dockers with a name and static IP in that network. Simple!

To create a network:

docker network create -d bridge --subnet="172.30.0.0/24" --gateway="172.30.0.100" --ip-range="172.30.0.0/24" permanet

Now any app you need, just specify this network and a static IP of your choice; like this:

docker run --name mysql-server --network="permanet" --ip="172.30.0.1" -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql/mysql-server:5.7

More complex containers can be created like this:

docker run -it --name gocd-server --network="permanet" --ip="172.30.0.5" -v /yourhome/docker-volumes/gocd/godata:/godata -v /yourhome/docker-volumes/gocd/home:/home/go gocd/gocd-server

I keep a dedicated directory in my home for docker volumes, so I can back it up and use them as is when I change machines or OS. I also have a script where I add all the containers I need, so it is just a matter of copying the volume directory and running the script to create identical setups. Then even your .desktop files work as is!

Here is an interesting setup script for Jenkins (gist), it externalises all data directories from Jenkins, including the plugins and users and mounts your local m2 repository inside Jenkins so as to avoid downloading the libs again :

docker run -it --name jenkins --network="permanet" --ip="172.30.0.6" 
-v /yourhome/.m2:/var/jenkins_home/.m2
-v /yourhome/docker-volumes/jenkins/workspace:/var/jenkins_home/workspace
-v /yourhome/docker-volumes/jenkins/jobs:/var/jenkins_home/jobs
-v /yourhome/docker-volumes/jenkins/plugins:/var/jenkins_home/plugins
-v /yourhome/docker-volumes/jenkins/users:/var/jenkins_home/users
jenkins:2.32.3-alpine

You can create as many such networks as you wish for logical separation of groups of such apps; in my case this is the third network (172.30 for that reason), since first two were taken up by some compose scripts.

A list of few such containers I use: mysql (different versions), SonarQube, hystrix-dashboard, zipkin, swagger-ui, a redis-cluster for local use, gocd-server, Jenkins, portainer, postgres, pgadmin etc. I even have a couple Windows software running on wine in such containers, we shall talk about it some day.

Jetbrains messed up our license: Jetbrains still rocks!

Jetbrains messed up our license: Jetbrains still rocks!

Jetbrains messed up our license: Jetbrains still rocks!

It is story time today. We are a small startup, by small I mean like 4 people on the team and we established barely a couple months ago. It was time to start development and like all good Java developers depend on their IDE for their life, we did too. Too soon to go off-topic, but I wonder sometimes how large a program can I write without an IDE.

There is no better IDE for Java or Javascript than Intellij Idea from Jetbrains, and like all developers who know this, we went ahead and bought the Idea license. Lucky for us, being a startup, we were eligible for a 50% off offer. Jetbrains’s sales team was kind enough to approve it and we did get the license without much of a problem. My CTO asked me to get it set-up for myself, it was easy. There was a link in the email, I clicked, logged in and I got the license added in my Jetbrains account. It was almost smooth, except for a short to and fro on email to get the offer to reflect on the checkout page. All said and done, we loved the experience. Who would not love to get Idea at 50% off!??

So, done with the story? Why would I write it up if it was all so smooth? Read on..

A week went by without a problem and all of a sudden one morning my Idea closed on me complaining that I had no license! This was a shocker, we bought it, I had seen in it in my account and it was working for a week! I wondered if my CTO could have accidentally reallocated it, but of course he has better things to do than poke around in Jetbrains account configuration. I logged in to my account and turns out there was actually no license!

Since the CTO and I work in different time-zones, it was almost end of the day when we could chat and I could request him to check the matter. (A day saved by sublime-text.) Like I thought, he told me he had better things to do but we decided to have a look at the account anyway. Since it was a single link click for me to setup the account, he had not even created a Jetbrains account till then. We created the account and logged in, and were shocked to see some 31 licenses in our name. Something was certainly wrong.

Clicking through a couple of menus revealed that we were actually seeing the licenses/account of a different company! And our CTO, had become an admin for them. We had complete control over all the licenses, our as well as their licenses, and we could even decide who becomes an admin. We could remove their admin. While it was all fascinating (the devil!), we needed to know what happened to our license and we found it. This company’s admin had revoked my license and allocated to someone on their team! This company had a name similar to ours (Ours is one word, their was two), it was obvious that the person creating our license mistook us as them and instead of creating a new customer account amended theirs.

What do we do now? I contemplated our options and risks, contacting Jetbrains sale and support team and asking them to fix this was the best and the obvious option, but I had no idea how long it would take. What would AI do till then, being locked out of my own license, and when Idea keeps shutting down spontaneously. Can they even track it? Will they do it? Or would they ask us to buy a new one? There were a lot of unknowns, even when I knew Jetbrains would not leave us in the lurch. What if we wrote an email to this company’s admin and explain what has happened and request them to free up our license in good faith. But then they had their chance, they chose to to re-allocate a license they had not paid for to someone in their team, their admin would of course know this! They had their chance to see where the license came from, why it was allocated to someone outside, but they just revoked and used it, can we trust them to act on good faith? What if they just kicked our admin out, we would have no visibility into what was happening. Scary!

We quickly wrote an email to jetbrains, basically responded on the previous mail chain explaining this and requesting a resolution. Also attached a couple of screenshots showing what exactly had gone wrong, one showing the list of unrelated licenses and one showing our license being reallocated to someone else. Also tweated at Jetbrains to help us out, a good fellow working with Jetbrains responded and asked us to write to support email as well. We forwarded the email to support address and waited.

It was not long before we heard back from the sales team, some 4 hours or so. They quickly separated our company account, moved our license, assigned our admin and responded with a new link to claim license. We could now see a single un-allocated license in our account and allocate it. What they did great was that they also looped in this second company’s admin and wrote to them explaining the situation and offering a discount for one license. In my view, it was a good gesture at amending a mistake.

Mistakes happen, how you fix them is what decides if you retain your customers. And Jetbrains you certainly have retained us.

Habit-Firebug Saver: SmartLogger

Habit-Firebug Saver: SmartLogger

How many of the web developers do not depend on Firebug or the chrome’s console… Just wondering..

BTW, its plain fun to work with firebug, makes life a lot easier.. Its a different matter all together that the other browser that you have to develop for does not have a powerful enough tool. (Name deliberately avoided to avoid the eminent flame-war!) Yes the current versions have a quite powerful debug and development tools but (hopefully) few developers working on products still have to consider some 10 year old versions (namely 6, 6.5 and 7). Ah, the pain.. Anyways, we are not discussing that..

What we are talking about is the issues that we face when testing our changes to a thousand lines JavaScript code on multiple browsers, especially after we are accustomed to the ease of firebug. 🙂

I spent much of my time commenting my console.log() statements before I could dare to open the page in IE. Well, fear not, the days have passed! The pain drove me to write a logger object that can not only sense presence of console object but can do much more than that, like ability to assert, selective logging and more..

I call it the SmartLogger.

//Global Logger Object, for use during development, configurable logger.
var SmartLogger = function(options) {

var sl = {}; // Logger Object

// Accepting passed params.
options = options || {};
sl.enableLogger = options.enableLogger!==undefined?options.enableLogger:true;
sl.enableAssert = options.enableAssert!==undefined?options.enableAssert:true;
sl.loggerOutput = options.loggerOutput!==undefined?options.loggerOutput:undefined; //'console', 'alert', undefined
sl.selectiveEnable = options.selectiveEnable!==undefined?options.selectiveEnable:'';
sl.selectiveDisable = options.selectiveDisable!==undefined?options.selectiveDisable:'';

// Logger properties
sl.name = "SmartLogger";
sl.whoami = function(){ return "SmartLogger_"+sl.enableLogger+"_"+sl.enableAssert+"_"+sl.loggerOutput+"_"+sl.selectiveEnable+"_"+sl.selectiveDisable;}
sl.version = '0.7';

// Checks if console object is defined. Checked only at the time of instantiation.
var hasConsole = (typeof console === "object");

// Checks if logging should be done to console.
function logToConsole(){
if (sl.loggerOutput){
if (sl.loggerOutput === 'console') return true;
} else {
if(hasConsole) return true;
}
return false;
}

// Handles the logging intelligence
function handleLogging(logMethod, logString, strId){
if(!sLog(strId)) {return;}
// Decides if to log and logs or alerts appropriately.
if(sl.enableLogger){
if (logToConsole()){ // && hasConsole
if(hasConsole)console[logMethod](logString);
} else {
alert(logString);
}
}
};

// Handles the selective logging functionality
function sLog(strId){
var allowLog = true;
if (sl.selectiveEnable) {
allowLog = strId === sl.selectiveEnable;
} else if (sl.selectiveDisable) {
allowLog = !(strId === sl.selectiveDisable);
}

return allowLog;
};

// Returns a formatted object structure with current values to complete depth.
function printString(obj, name, str, strEnd){
var stringified;
name = name?name:"Object",
str = str?str:"";
strEnd = strEnd?strEnd:"";
stringified = str+name+" : {n";
for (var a in obj){
if (typeof obj[a] === 'object'){
stringified+= printString(obj[a],a,"t",",");
} else {
stringified+= str+"t"+a +" : "+obj[a]+",n";
}
}
stringified += str+"}"+strEnd+"n";
return stringified;
};

// Exposed methods of the object
//log a string to console/alert
sl.log = function(str, strId){
handleLogging('log', str, strId);
};

//debug logging a string to console/alert
sl.debug = function(str, strId){
handleLogging('debug', str, strId);
};

//write an information string to console/alert
sl.info = function(str, strId){
handleLogging('info', str, strId);
};

//throw error string to console/alert
sl.error = function(str, strId){
handleLogging('error', str, strId);
};

//Assert an assumption
sl.assert = function(str, strId){
if(sl.enableAssert){
handleLogging('log', 'Assumption: true', strId);
if(!str){
handleLogging('error', 'Assumption failed!', strId);
debugger;
}
}
};

// Logs the formatted object structure with current values to console/alert
sl.stringToConsole = function(obj, str){
sl.log(printString(obj, str));
};

return sl;
};

var sl = new SmartLogger();

Features:

  • Multiple logging profiles can be maintained at the same time with different properties.
var sl = new SmartLogger();
var sl2 = new SmartLogger({selectiveEnable: 'block1'});
  • Proprieties can be set at the time of instantiation or even later.
var sl = new SmartLogger();
sl.loggerOutput = 'console';
var sl2 = new SmartLogger({loggerOutput: 'console'});
  • name, version number and whoami to identify the logger with a string of its current properties.
var sl = new SmartLogger();
sl.name // SmartLogger.
sl.version // 0.7
sl.whoamI() // Returns a string of its properties with the name of the object in a specific sequence:
// "SmartLogger_"+ enableLogger +"_"+ enableAssert+"_"+ loggerOutput+"_"+ selectiveEnable+"_"+ selectiveDisable;
// Example: SmartLogger_true_true_console__b
// We will see what these properties are in some time..
  • Enable or disable logging altogether: enableLogger controls if the statements should ever be logged.
var sl = new SmartLogger();
sl.log('gets logged');
sl.enableLogger = false;
sl.log('never gets logged');
  • Intelligently decides where the logging statements should go…
sl.loggerOutput = undefined; //default
/* Decides based on presence of 'console' object.
If console is present statements will be logged to console,
else like in case of IE, will be 'alerted' to the user.
Now at times this can get messy, with loads of log statements alerting on our face..
But wait, we have ways to handle that.*/

sl.loggerOutput = 'console';
// Plain instruction, no intelligence, all statements will always go to console.
// If console is not present statements will just be eaten-up.

sl.loggerOutput = 'alert';
// Another plain instruction, all statements will always be alerted.
// Will not bother to check if console exists or not.
  • Log formatted objects to console. Now you wont need that much with firebug but to see the entire contents of the object, well formatted you can just say stringToConsole.
// Just a sample object with unknown properties.
var obj = {prop1: 'value',functProp:function(){return "this is a function that returns me!";}, propObj:{prop2:'value2'}};
sl.stringToConsole(obj); // You say this.

// On console or in the alert prompt, you get this
Object : {
prop1 : value,
functProp : function () {
return "this is a function that returns me!";
},
propObj : {
prop2 : value2,
},
}
  • Assert your assumptions. Checks that the assumption is true, if yes, logs so. If assumption fails, will write out an error on the console and invoke the debugger so the user can check in the stack exactly where the assumption failed and why.
sl.assert(Obj.str===Obj2.str)
sl.assert(1==1); // logs 'Assumption: true' to console.
sl.assert(1==2); // Logs error 'Assumption: failed!' and invoke debugger to the assert line in SmartLogger.

//Now you can go and check in the stack and watch to panels to check value and call stack.
  • Has a wrapper for 4 of the logging APIs from firebug and adding new is not much of a task. What it already has:
    • log
    • debug
    • info
    • error
  • Has ability of selective logging.
Now this thing is a live saver. The properties selectiveEnable and selectiveDisable control what statements to log. While these are not mandatory inputs to all the wrappers but I suggest you set them always. These are logging context that can be used to selectively enable logs for only partial of the code, the code that currently interests you..
// Suppose we were working on a defect number 101 and now we are developing a functionality for
// automating welcome messages to users and are asked to urgently fix defect 203.
// Ah, complex scenario, but it will only help understand the purpose.

// When we are working on defect 101, we had the logger configured as statements as:
sl.log("reached here"); // worst way to log: who knows wheres here! but just an example.

// Now we are working on the functionality and
// we would not want those 10 logging statements added while we were working on the defect.
// We can remove them or simply enable the 'selective logger'!
sl.selectiveEnable = 'welcomer';
sl.log("fetching message", "welcomer");
// And voila, only the 'welcomer' messages will be logged.

// Now we get the next urgent defect.
sl.selectiveEnable = 'defect203';
sl.log("value in Obj1.str"+Obje1.str, "defect203");
// We get only the defect203 logs!

// Now some of our new changes depend on the changes we made in defect101, but we cant get the logs from those..
// What do we do? If someone did not enable selective logger and removed the statements, please add them back (:p),
// remove statements for 'welcomer' functionality. Or simply, disable 'welcomer' messages..!
sl.selectiveEnable = '';
sl.selectiveDisable = 'welcomer';
sl.log("value in Obj1.str"+Obje1.str, "defect203");
sl.log("value in Obj2.varInt1"+Obj2.varInt1, "defect101");
// Ha ha! Log statements for 'welcomer' gone and we get the rest!
While using the SmartLogger, I suggest you always pass the string identifier, so that you can control the logs at any point of time later.

What can you expect next in SmartLogger:

  • Use assert from firebug itself.
  • Check that the function exists in the logger before calling it.
  • Make the selective logger take arrays.
  • In stringToConsole, handle functions too to remove that glitch in no closing bracket.

Let me know what you think about the SmartLogger, if you would like any additions to its behaviour and also if you find any defects in the comments below.