"Continuous Integration" - Should I go for it?

Leave "How" for a moment. Think why you should adopt Continuous integration.

Let’s set the context of the article first. There are lot of doers around who can do things as said, but there are very few special ones who are decision makers. What makes later so special, it’s the way he-she thinks about stuffs around. They ask questions in their capacity and do self-research, they never rely blindly on things said by great ones, they challenge and doing so they achieve mastery in it and mold themselves into a decision maker.

Aligning with said philosophy, article will focus on "why I should go for it" than "How". Here "it" refers to "Continuous integration-build & deployment". I have added deployment explicitly in widely used term as I feel it’s a quite connected term.

I won’t go too much in boring theory, but for sake - will wrap it in simple words. It’s an automated process which continuously build your project and deploys it.

Let’s try to answer our question which we should be bothered of i.e. "why I should go for it".

There are many drivers for adopting Continuous integration, build and deployment - will visit them one-by-one. Don't think about “how” while reading this article. There are various online resources available which most of us are smart enough to utilize for that. Very few will feed you knowledge to decide whether it fits your scenario or not. At the end of this article, I am certain that many of you would be in better position to make decision in this regard.

I will refer Continuous integration - build and deployment as CIBD here onwards (yes I’m lazy :)

Driver 1: Minimize Efforts spent

Think of efforts spent by our poor integrator and deployment team. Whenever any release/patch has to be thrown, our poor folks has to spend their nights in office to achieve pity success. Real miserable pains are build time, finding-coordinating for errors fixes, unit testing, smoke test, testing team coordination, create-deploy deployment package & notify stake holder etc. Quite a long list. A mid-size (10k lines of code) web based application needs around least 2-3 person days of efforts for all listed activities for one iteration & it’s quite frequent. There is a serious need of automation, answer is CIBD adoption.

Driver 2: Monitor glitches and rectify as it discovered

Glitches refers to build errors, broken links or any other horrifying error. Glitches give a very hard time, first challenge is to find them and bigger one is to resolve them. Especially in less frequent build requirement scenario. In such cases, after ages when integrator starts integrating all modules, discovers lot of fatal issues in it and then everything will pile up to an extent which results into postponing delivery. Sometime, it creates a critical business risk for end client. To rectify this, CIBD is the solution .CIBD will continuously build your code chunk preferably with each code commit in source-control server and throws error if any glitches. Culprit (one chap from us) will be made to fix it. With CIBD will surely rectify 11th hour horror discovery and help us to play a smooth delivery. 

Driver 3: Sense of ownership with fear of misdeeds

With CIBD in place, a culprit can be caught very easily using source control commit log for an instance which made the build fail. This will embed a positive fear in team members mind - no one would want his/her name to be highlights for misdeeds and face the exaggerated blame, makes them extra cautious regarding code and unit testing before committing anything on server. This rectifies lot of rework.

Driver 4: Ancillary additions scope

To achieve wide scale automation, we can dope in few ancillary things like Static Code Analysis tool, Automated Unit Testing tools, Code Coverage Tools, Automated System Testing tools, Health Monitors, Configuration Management utilities. Adding these will make your overall process a resilient & adds further saving in time-effort-cost. 

Driver 5: Statistics

It’s the era of analytics - predictive, descriptive whatever any *tive. CIBD will give you stats of the build with success, failure ratios, and team based segregations whatever you think off, in very fancy charts. This will be seriously useful while flaunting reports to management.

Driver 6: Ancient ones - Code Coverage, Unit Testing blah blah blah...

And there are more to add which you will find in any CIBD article. Naming few ones here - code coverage, unit testing etc. I think they are self-explanatory – no explanation needed.

Finally, With these six drivers, post assertion against all six for your particular scenario see how much of them are seriously “Yes” in your case if majority are, then CIBD is the way to go.

"Automation"- an old but reborn disruption

Damn it’s very old term, but reborn. Recently, Automation happened to be a core part of many Major Software firm CIO's strategies. Lot have been said about it. After hearing so much and seeing significant inclination of technology world towards automation, I have decided to apprise myself and find answers for basic naive questions I had in my little mind, simple famous 3 W’s

        

     What is it?

        Why do we need it?

        Where do we need it?

Bear with me for few minutes, I am certain that you would admire this rookie for honest effort spend on this basic automation expedition.

What is it?

Any action which is being done manually and recursively (manually refers to actions performed by humans), process of transforming that manual deeds into an action executed by a machine removing human intervention is Automation.

Still Gray, Let’s try to throw some light over it. Consider a layman example of a garden: You are rich (unlike me) owning a big garden with around 300 plants. You have 1 gardener who is responsible for watering, fertilizing, spraying pesticide, new plantation and cleaning garden. Doing so, this hard working chap complete watering task every day in around 4 hours. Due to watering work, gardener is not able to spare required time for other must activities like fertilizing, cleaning, pesticide spray, planting new plants etc. To overcome this, you got an idea in your bright mind. You installed a water drip system in your garden which takes care of watering plants every day and with no waste. Here, Water drip system installation is the automation introduced to remove manual work done by gardener. This is a simplest example of Automation.

Nowadays, Automation is everywhere. Cars, Traffic lights, Railways, Manufacturing facilities, blah blah blah. You Name a field, with little bit inspection you will find automation present there. These automations are result of decades of research in those particular field. My bread-n-butter machine i.e. Computer is also example of automation which was invented to automate calculations, which in turn doing lot of other things nowadays.

Why do we need it?

A million pound (Havier than $) question - why the hell we need it? Answer is to speed up overall execution, reduce waste- cost & increase efficiency. By following as-is process, we would certainly end up doing job with acceptable output , but using Automation same job can be completed in hilariously less time and that too with sparing free resources to take up next queued job.

Software field being comparatively young, until now it was in evolving phase. I won’t call it mature though. To achieve next maturity level, software industry scientists (developers like us) are finding ways to introduce automation. A manufacturing factory has different assembly lines, similarly software development (factory) has phases like Requirement Gathering, Design, Development, Testing and Deployment. We have quite a big human resource base in software field, but industry is engrossed with challenges like tight budget and market volatility. Believe me, software Industry is in deep need of optimized resource utilization & cost cutting. To achieve this, automation is the way to go.

Where do we need it?

A question that must be mingling in your mind from start.

To ease this, a pity scientist like me tried inventing something simple. I have devised a basic methodology following which one can find potential tasks in overall execution which can be automation candidates. A Task can be fully automated or partially depending on task .To automate task, we need to carefully assess it in detail. Good way is to have a brainstorming session with important folks for this assessment. My tiny methodology not confined to any specific field, having said that I must mention that I have designed it with software development life cycle in mind. So beware …just kidding

This Methodology will help you in basic decision making, may need to be followed by further deep analysis.I will call it "Automation Candidature Assessment Methodology (ACAM)". OMG... Big Name.

Let’s start understanding it.

ACAM Methodology has 2 phases. Every phase is explained first and followed by illustration.

Phase 1: Task Collection & sorting

In phase 1, focus will be on existing process analysis, Task identifications and few outcomes. Outcomes will be used in later phase. Following are the steps to be followed during phase 1.

Step 1: Calculate Total efforts required for complete job execution

Step 2: Divide overall job into well-defined autonomous tasks which are quantifiable in efforts

Step 3: List down tasks with efforts needed for completing it once

Step 4: Calculate weight of each task with formula as

              Weight = (efforts in person hours * frequency per month)/ Total Number of tasks

Step 5: Arrange Tasks in the descending order of weight, highest weight task will come first. 

Phase 1 Illustration:  Let’s consider a garden example with 1 person working for 8 hours per day.

Total garden maintenance work needed efforts per day is = Number of person working * working hours per day = 1*8 = 8 person hours. So Total Job efforts = 8 Person Hours

I have identified four autonomous tasks for our garden example which are Watering Plants, Fertilizing & Spraying Pesticide, New Plantation and Cutting-Shaping.

After dividing job into autonomous tasks – here is what we get:

Task	Efforts for completing  job by one person
Watering	4
Fertilizing & Spraying Pesticide	2
Cutting - Shaping	1.5
New Plantation	0.5
Total effort	8

Now, let’s calculate weight for Watering Plants Task.

Formula is “Weight = (efforts in person hours * frequency per month)/ Total Number of tasks”

Efforts in person hours = 4 hours

Frequency Per month = 30 as its daily task

Watering Plants Task’s Weight = (4*30)/4 = 30

Similarly, Post weight calculation for other tasks and sorting according to descending weight– here is what we get:

Task	Weight
Watering Plants	30
Pesticide Spread	15
Cutting & Shaping	11.25
New Plantation	3.75

Phase 2: Automation Candidature decision making

Let’s move on to next phase of our methodology where we will decide on whether task is worth automation or not. Following are the steps to be followed.

Step 1: Consider only tasks who has weight greater than 5. Reason behind this decision is tasks which is of low efforts and less number of frequency will not be a candidate for automation.

Step 2: Analyze task, see if there is any human manual work is done it. List down manual work, will be used in later steps.

Step 3: If any manual work done then consider task for further steps else discard it.

Step 4: Check feasibility for automating manual work discovered in earlier steps. This can be simple gut feeling or can be verified by a small proof of concept if required.

Step 5: If feasible for automation, then consider it else discard it.

Step 6: Calculate estimated saving that can be achieved post automation. This can have 10% deviation as we don’t have automation ready. To achieve close to reality figure, POC is recommended. This figure will work like a weapon for convincing leadership for further approvals (I know that’s PIA job). With full automation, you may zero down efforts required or with partial automation certain percentage of it. This depends on the task under consideration. Be careful with your claims, do have proper reasoning for all claims. Formula to calculate saving is simple.

Step 7: If saving is more than 30 then consider it for automation else discard it.

Following flow diagram depicts Automation candidature decision making decision flow.

Phase 2 Illustration:

Continuing with Garden example, let’s consider Watering Plants Task.

Step 1: Watering Plants Task has weight 30 (Calculated in Phase 1), making it eligible for further steps.

Step 2: Analyze Watering Plants Task. Post analysis we found that it has a human manual work in it. Qualifies for next step.

Step 3: It has manual work, so qualified for next step.

Step 4: Feasibility for automating manual work done. Yes, it can be done using “Water drip system”. Qualifies for next step.

Step 5: Implementing “Water drip system” automation is feasible, qualifies for next step.

Step 6: Full automation possible in this case. No human efforts needed post installation of water dripping system except maintenance which is merely 1 hour per month , earlier watering is done 30 times in a month so, for watering job per day will be completed in (1 hour/30) = 0.033 hrs.

Saving = ((4-0.033)/4)*100 = 99.17

Step 7: If saving is more than 30 so we must go ahead with automation.

Post performing above steps, on all tasks we will get following

Task	weight	Saving	Automation Eligible	Remark
Watering Plants	30	99.17	Yes	Full Automation
Pesticide Spread	15	NA	No	No Automation possible
Cutting & Shaping	11.25	NA	No	No Automation possible
New Plantation	3.75	NA	No	Rejected – weight < 5

So, only watering task qualifies for automation. This way we can map any job that we do in normal development projects across Software life development life cycle. Just to give you folks a brain starter, deployment automation using Continuous Integration process, map deployment process using ACAM, follow steps as mentioned, convince leadership or client with clear numbers, I am sure they will be amazed.

Final Word:

This is all folks, Hope my pity ACAM Methodology provides a simple but realistic basic methodology to narrow down tasks for automation candidature.

Finally, I must mention, methodology explained is created based on my experience and using this approach I have created many automation accelerators. It may or may not exactly same for every scenario but modified version can be created. I would urge everyone to give more emphasize on automation as in near future that’s - what industry will largely inclined to.

Feel free to comment.