Posts

MindNet

Image
An internet of minds Aldous Huxley in his book, The Doors of Perception, a travelogue of his mescaline-fuelled trip through the mental landscape, compares the mind to a reducing valve. Every sentient individual is in touch with and is aware of all that is happening anywhere else but to preserve sanity in the face of an intolerable information overload, the mind chokes off most of the information and restricts itself to the safety and comfort of his local environment. The concept of a universal mind or consciousness, the brahman , of which an individual or atman is but an image is also the foundation of Sankara’s Advaita Vedanta. Both Huxley and Sankara in their own way suggest that all “minds” in the universe are somehow connected together and are in a position to share information. Doesn’t this ring a bell somewhere ? image Peace of Mind by Dokon taken from urbantimes.co A telephone bell perhaps ? Since every telephone on this planet is connected, or certainly connecta...

Introduction to Hadoop

My effort to learn Hadoop and Map Reduce resulted in the presentation. If you like it and find it useful do leave a comment.

Forecasting Retail Sales - Linear Regression with R and Hadoop

Image
image from Responsemagazine.com A retail store tracks the volume of sale for each stock-keeping-unit (SKU) that the store deals with. Given the sales for days 1 through 5, is it possible to predict the sales on days 6 through 10 ? Common sense dictates that sales will remain constant and the average sales per day for the first 5 days will be the same as the average sales per day for the next 5 days. However this may not always be the case, if there is a rising or falling trend. If there is a strongly rising trend, caused by a some strong promotional activity, then the assumption of constant sale will lead to a stock-out and loss of potential business. Similarly, if there is a strongly falling trend, then a similar assumption will lead to accumulation of dead stock and hence a loss related to excessive inventory. Instead of days, the same analysis can be done on the basis of weeks, fortnights or even month. Net-net given the sales over 5 periods of time, it is useful to be able to ...

HIVE and PIG to simplify Hadoop

Image
[Note]   -- Hadoop, IMHO, is history. Rather than waste time with all this, suggest you check up my blog post on  Spark with Python . When I was doing engineering at IIT, Kharagpur, the computers that we had were not even as powerful as a low-cost non-smart phone today and other than the basic concept of programming, nothing that we learnt is of any relevance today. So when we start a teaching a course on Business Analytics , that lies at the bleeding edge of  current technology and business practices, there is simply no option but to take the Do-It-Yourself approach of first learning a subject and then teaching it to students. Fortunately, there are many kind and knowledgeable souls on this planet who have taken the pains to explain new and difficult concepts to ancients like us and thanks to Google, it is not too difficult to locate them. Using this route, I first learnt what is Data Science and then created this compilation of tutorials and training materials th...

Consultant for Criminals or Criminal Consultants

Image
In an earlier post on Extreme Konsulting , I had explored the characteristics and challenges of delivering management consulting services to reluctant and criminally corrupt government offices, but Sameer Kamat in his book Business Doctors , goes far, far deeper. In this new novel set in US West Coast, Sameer explores what it would, or could, mean to offer professional management consulting services to actual, law-breaking, criminals ! As one who has been in the consulting profession for a long time, Sameer knows too well that the business is not about rocket-science, earth-shattering ideas, but the ability to locate an action-template that has worked in the past and apply it, with some creative tweaks, to address the situation at hand. This simple point-of-view has been expanded on and converted into a novel that explores the mutually awkward relationship that develops between an Ivy League educated, laptop-carrying management consultant and a mafia mob leader and his henchmen w...

Demystify Map Reduce and Hadoop with this DIY tutorial

Image
[Note] -- Hadoop, IMHO, is history. Rather than waste time with all this, suggest you check up my blog post on Spark with Python . In an earlier post on Data Science - A DIY approach , I had explained how one can initiate a career in data science, or data analytics, by using free resources available on the web. Since Hadoop and Map Reduce is a tool and a technique that is very popular in data science, this post will get you started and help you Install Hadoop 2.2, in a single machine cluster mode on a machine running Ubuntu Compile and run the standard WordCount example in Java Compile and run another, non WordCount , program in Java Use the Hadoop streaming utility to run a WordCount program written in Python, as an example of a non-Java application Compile and run a java program that actually solves a small but representative Predictive Analytics problem All the information presented here has been gathered from various sites on the web and has been tested on dual-boot ...

Tawang @ Arunachal Pradesh

Image
Arunchal Pradesh is said to be one of the most beautiful parts of India but because it is so very remote I have never had the opportunity to visit it in the past. This summer, when Calcutta was sweating at 43 Celsius, we decided to pay a visit. Arunachal is a vast state with an extreme diversity of culture. The western end, around Tawang, shares the Buddhist culture of Tibet / Bhutan while the eastern end is primarily tribal and animistic. Unless you have a lot of time and energy it is better to stick to one end. We chose the Tawang end -- the scene of the great 1962 debacle, where an under prepared and ill equipped Indian army was badly bruised by China despite many instances of personal bravery by our Jawans. We started off from Guwahati / Tezpur and this was the route that we took We followed the Kameng River and spent the night at Prashanti Cottage, Bhalukpong, a beautiful resort on the river bank where from the watch tower attached to the restaurant, you just might ge...