Category: Analytics

An Algorithm is essentially a method or plan for solving a problem, based on following a sequence of specified activities. And an algorithm is something that is used to train a model, all the decisions that a model is supposed to take are based on the input given by the algorithm to produce an expected outcome. Now going further, an analytical model is a statistical model, designed to perform a specific or to predict the probability of a specific event. Now the same is applied to businesses to determine the solution when in quandary. Algorithms are generally used throughout all areas of IT. A simple example would be a search based algorithm, it takes the keyword and searches its associated database for relevance and presents the results. So basically there are a variety of algorithms that can be used for different scenarios, it is usually variable dependent, that is based on a variety of variables, a data scientist can choose which algorithm to use.
It is not recommended that a plug and play of available algorithms are used by the data scientist, as the variables like, the size of data, industry type, its application, could produce results that might not be accurate. Hence a data scientist should train themselves on the most popular and important algorithms.

Awareness of supervised v/s unsupervised learning models

On the subject of algorithms, it is interesting to understand the difference between the two. A supervised learning model is essentially a model with a clear difference between explanatory and dependent variables, which means that the model’s outputs are known in advance.
Some examples of these models which are popular would be……

• Prediction – Linear Regression
• Classification – Decision Tree
• Time Series Forecasting

Simply an opposite of this would be the unsupervised learning model, where the model’s outputs are unknown, and there is no target characteristics. These models are built with the objective of finding out the fundamental structure of data, for example,

• Association Rules
• Cluster Analysis

The top 10 algorithms used by data scientist according to a poll are as follows……

1. Regression
2. Clustering
3. Decision Trees
4. Data Visualisation
5. K-Nearest Neighbours
6. PCA
7. Statistics
8. Random Forests
9. Time Series / Sequencing
10. Text Mining

Some new options to the list of most commonly used algorithms which are recently gaining popularity are….

• Neural Networks – Deep Learning
• Singular Value Decomposition

You will notice, that most of the popular algorithms are supervised learning algorithms in nature. Unsupervised clustering algorithms can be used to detect a relationship between an organisations data set. Through these algorithms, you can find different types of groupings within a customer base. At times an unsupervised clustering can offer specific advantages when compared to supervised learning models. One obvious example is a way new applications can be observed by studying, how the connections are grouped when a new cluster is formed.
As can be seen, these are only the most popular algorithms, however, here are a host of other machine learning and data mining algorithms available, which can be used to create value to any analytical program. There are specific algorithms that are designed and developed to specifically deal with business challenges. These algorithms can almost assist in doing anything, from recognising faces too, dispensing drinks from a vending machine to reminding you about your meetings.
To learn more about the algorithms join our an online data science course, which you can do anywhere and anytime .This program is co-created with Genpact as Knowledge Partner. This program helps you with a deep understanding of Data Analysis and Statistics, along with business perspectives and cutting-edge practices using SAS, R, Python, Hive, Spark, and Tableau.

The position of a data scientist is most sought after in recent times, mainly because of the boom in information, and the quick need to make sense of the data, extracting insights which could positively impact business. The buzz around data science, at times, confuses the link between the candidate and the company, creating a gap between what the company needs as opposed to what is perceived by the applicant.

So essentially what an IT company might be looking for in a data scientist might not be very different from what another line of business might be looking for in a data scientist.

It could be true that a tech company might be absolutely aware of the specific technical skills, required and their job description might be more evolved as they have the other skill sets in place, or have the functional knowledge about expectations out of qualifications, which for a non-tech organisation, might have blurred boundaries.

Educational Background

Irrespective of the industry, one solid fact that will be looked for while hiring a data scientist is the educational background or the technical know-how. Traditionally a data scientist needs to have a strong foundation in statistics or mathematics, this will be an added advantage, specifically from an IT point of view, as they would have many engineers with the Machine Learning capabilities, but a strong foundation would ensure that the data scientist will use or create the right algorithm by understanding the technical details.

Programming Skills

It is a given that a good data scientist will/should be an excellent programmer. Activities such as Sampling, Pre-Processing data, Model Estimation & Deployment, Sensitivity Analysis, Back-Testing, etc…, are used frequently by a data scientist so that these steps are successfully performed, programming needs to be done.

Hence a data scientist should have sound working knowledge of prototyping languages like SAS, Python, R, Deployment Language such as C++, C#, etc…, and lastly big data languages like Scala or Spark.

Exceptional Communication and Visualisation Skills

At present, there is a huge gap between analytical models and business users or stakeholders. It is imperative that the data scientist not only understands the business know-how, which he or she is associated with, but also explains the analytical models along with the involved statistics, and reports in a non-technical manner to the stakeholders. If this is successfully done then the business user will be able to appreciate the advantages of big data, which will further improve their acceptance and attitude towards big data.

Line management skills are also what sets apart a data analyst from a data scientist, the ability to work with people and not in a silo is imperative.

Creativity

Finally, anyone looking to hire a data scientist, in some sense expects them to be creative. They need to be creative on the technical level, with regards to feature selection, data treatment etc…, the steps of knowledge discovery process, or the ability to take the right guess or select the right approach makes a huge amount of difference, after all it’s the same data set, how you treat it is important. Not only technical creativity but also the know-how about the ever-evolving

data field, where they are up to date with current and future possibilities and technologies is important.

So a data scientist should literally be a master of all, with a mix of skills ranging from programming, quantitative modelling, communication, visualisation, to business acumen and creativity, as the field of data science and analytics is multidisciplinary in nature.

To make a career as a Data Scientist opt for an online data science course, which you can do by seating at home. This program is co-created with Genpact as Knowledge Partner. This program helps you with a deep understanding of Data Analysis and Statistics, along with business perspectives and cutting-edge practices using SAS, R, Python, Hive, Spark, and Tableau.

The Internet of Things and other various channels through which information and consumer data can be captured is on the rise, as a result, phenomenal amounts of data are being captured every second. Big Data has great insights which can be used to enhance end-user experience and aid business growth. It is established that extracting information out of this complex web of information is the tricky part, which has in turn given rise to new departments with specific skills targeted in getting the order out of chaos, across organisations.
Initially, any professional working in the capacity of big data, in the data science department was called a data scientist. Over the last few years, as the field is getting more complex and advanced and with the diverse possibilities, one can see companies hire specific professionals under various titles, like, Data Engineers, Data Architects, Business Analyst, Data Analyst, Machine Learning Engineers, Big Data Engineers, etc…,
Let us get a quick overview of the common roles within the Big Data scope and a general understanding of their responsibilities.

Data Analyst

This role is a branch out of the Data Scientist role, most organisations also call it the Junior Data Scientist. They play a vast role in the entire data science life cycle, from acquiring massive amounts of data to processing it, analysing it and also summarising the findings. They need to be involved in data scrapping and maintaining the quality of data. The skill set of a data analyst is vast, from knowledge of SQL to programming languages like Python, R, SAS, proficiency in statistics, math, techniques like data mining, data warehousing, and data visualisation, designing and deploying algorithms, knowledge of extrapolating data using advanced computer modelling, fixing code issues and pruning data, can be described as a day’s work of a data analyst.

Data Scientis

A data scientist does all that a data analyst is expected to do, however in terms of scope of the role, a data scientist has more responsibilities and is expected to have greater knowledge. Referred to as the sexiest job of the 21^st century, a data scientist is perhaps the most in-demand role. Besides being a pro in R, SAS, Python SQL, they mostly have a higher degree in quantitative subjects like math or statistics and should be great with big data analytical tools and technologies. They need to be up to date with not only the technical know-how but are also expected to be proficient with the business know-how of the industry they are associated with. They become the bridge between understanding the business challenges, adding value by deriving insights from the data, using predictive analysis and communicating the finding to the business stakeholders so that data-driven decision making, based on models and prototypes can be done.

Data Engineers/ Data Architect

They are the builders and managers of the big data infrastructure; they are responsible for making sure that the big data ecosystem is functioning smoothly. The essentially build, test and maintain data management systems. Their role is not to arrive at any solution for the business, they work on the linear path to ensure that existing system can be improved by integrating it with new data management technologies. Besides being programming wizards they also must have knowledge of ETL tools, API’s, data modelling, data warehousing etc…,

Machine Learning Engineers

ML are quite in demand in the recent times due to the booming field. Skills required to become an ML are, JavaScript, C++, Scala, one should have knowledge in highly scalable distributed systems. ML engineers have a strong foundation in math and statistics, hence are data and metric driven. Like data scientist, even ML need to be great communicators to interpret complex ML concepts to non-experts. An ML engineer essentially designs and implements ML algorithms, such as clustering, anomaly detection and prediction to address business challenges, they then have to monitor the reliability of the ML system in the organisation.
Data science is coined as the most promising profession of the century, people who are skilled in data literacy and strategic thinking, are inquisitive, who have the ability to look at data and spot trends, are key professionals to any organisation dealing with Big Data. the roles can be overlapping but exciting nevertheless.
Based on your skill sets and interest you can identify the role most suited to you, or you can pick up one of data analytics course to build the skill set to become more viable.

It is quite a gamble these days when as a part of your travel itinerary you have to book the airline tickets. Most people in recent times have given up the traditional travel agents for booking tickets and prefer booking tickets themselves. Booking airline tickets has become a massive decision making activity, where you have to do all your thinking hats, to make an informed decision. Should I buy now, or wait for a better rate, or should I use the price trend forecast to get recommendations on when to book so that huge fluctuations can be avoided.
Post regulation, airline industry, around 1980 – 2001, was a time of price wars, fluctuating fuel prices, and a general increase in the number of passengers opting to fly, along with great technological advances, between the handful of airline providers. Airlines with conservative overheads could offer low airfares as a blanket to all its customers. But in the long run, could not sustain on low margins and had to shut shop. On the other hand, few airlines, used a scientific and analytical approach in offering reduced pricing over a certain sector, during a particular period and maintained regular premium cost airfares for the remaining slots, and with this approach not only did they survive but also accounted profit.
This article will help you understand how could they do that….
The low airfares could be offered by airlines, by following a dynamic pricing mechanism for tickets, based on the demand and supply trend.
Predictive analytics takes stock of what happened in the past, that is historical data, and the forecasts are done by using scientific methods, and statistical models for reliable information, about possible outcomes in the future. Predictive analysis is most accurate because it tries to understand the behavior of a person or a customer, by not only looking at the past information, but is able to predict what they will also do in the future based on the customer, and about that entire population. Thus by grouping people in segments, predictive analysis will be able to give predictions and recommendations on how people will react in the future.
Predictive Analytics has continued to factor essentially into how airfare prices are decided and fluctuate to maintain airline profitability even today. It is mainly because the airline industry is getting very competitive and is a low margin industry. The ability to forecast travel demand is crucial to gain market share. Predictive analysis tools are no longer the luxury of airlines selling tickets, travel booking sites offering Price Predictors Features, can now be used by consumers to see and make decisions for their own projection. Price predicting feature will help them decide whether the price of a particular route is likely to fluctuate, higher or lower, in the coming weeks.
Keeping in line with the trend Indian government is contemplating on introducing an analytical tool which will present the travelers with future ticket price trends, similar to the travel booking sites which offer price predicting features.
This is a positive step towards addressing the travelers concerns of facing steep fluctuation in airfares. Of course, this proposal is subject to sanction from the civil aviation industry. Since air ticket prices are dependent on supply and demand metrics, this step from the aviation industry will get some amount of transparency in the entire process.
This endeavor is crucial and a part of the digitization wave, under the ‘Digiyatra’ Initiative. Since India’s Domestic aviation market is projected to be the world’s 3^rd largest by 2022, it is only fair that the industry uses analytical tools to provide a digitally unified flying experience to its travellers.
Using ‘Historical Data Analysis’ along with ‘Price Curves with Predictive Data Analytics’ will help travelers better project and plan their vacations, so as to gain maximum advantage over airfares.
In conclusion, it could be said that predictive analysis is touching our lives positively, in ways that we might not be aware, and the penetration is only going to increase with time.
To learn more about Analytics watch this space until next week for the big news!

It is quite tricky to differentiate between a System Analyst and a Business Analyst (BA). While in some organisations the role is quite defined, in others the difference does not exist. The title itself is not so conclusive. The term ‘Business Analyst’ and ‘System Analyst’ can be interchanged and regularly misunderstood. Nevertheless, the fact is that the two roles are very distinct and perform different duties which require specific skill sets, and these roles only get more elaborate, depending upon the size of the organisation you are associated with. Analytical skills, Planning Skills, Team Work, Innovative skills are some overlapping skill sets between a system analyst and a business analyst. According to the Business Analysis Body of Knowledge, there are many jobs or titles that may perform business analysis, like a Business Architect, Business Consultant, Process Analyst, Requirement Analyst. To further give an elaborate……, A Business Analyst working within an organisation will use the business analysis fundamentals to understand the stakeholder’s perspective and requirements and will assess them for possible roadblocks so that eventually an appropriate solution can be achieved. We can take the same definition further and say, a business analyst working in IT, will address the challenges of the business and recommend developing and implementing certain IT systems, as an initiative to improve and enhance productivity and performance. Now enters the System Analyst, who becomes responsible for outlining the technical aspects of the developed IT system or platform, and how easily the developed system or platform is integrated among the company’s requirement. The main objective of a Business Analyst is to understand and detect business opportunities that can positively impact business operations. In addition, it is also expected out of a business analyst to endorse technology wherever applicable, to create better productivity and output for the organisation. Basically, a business analyst has to understand the perspective and language of the stakeholders. A Business Analyst should have a keen observation to pick up inadequacies of a business and become a link between the stakeholder and the IT department. A Business Analyst might not speak in IT jargons but should have enough knowledge to assess and communicate the needs of the stakeholder’s, from business requirements to software requirements. The role of a System Analyst is to use IT systems and platforms in an organisation, and partner with the goal of the stakeholders, helping them to achieve the strategic business objective. Thus a system analyst will have to choose the most appropriate platform or technology to meet all the functional requirements. An SA will use the principles of structured analysis, sampling and accounting to ensure that the offered resolution is effective and financially worthwhile. In terms of knowledge, A Business Analyst is expected to be excellent in understanding the working specifics in the area they are associated with. They need to have a sound understanding of business regulation, customers profile etc…, while it is not expected out of an SA to acquire this information, more so when the associated organisation has a BA in their team. Strong communication skills, analytical know how, modelling techniques, is common amongst the two. It is not expected out of a BA to have sound knowledge of IT infrastructure and system landscape. So to sum it up, while the two roles are separate depending on the organisation, there could be an overlap or an inter-cross while working on a project. Irrespective of the title, it is healthy, that one is aware of the job responsibilities that both the specialists perform and the objective of their role.

Big data has created and is continuing to create a lot of buzz in not only the technology arena but across the globe. It promises big changes, big interventions, big innovations, big integration with and within our daily lives. But how big is Big Data? now that is a question which can be answered with different connotations. How big is big? it is always relative to whom the question is being asked. For example, a big book of hundred pages will look humungous to a child, but present the same book to an adult who loves to read, they will be able to process it in a matter of hours. Now present shocking data statistics like close to 7.5 billion searches on Google, gazillion tweets, more than 40% of the world photographs stored on the internet. Is this information big enough for a data scientist to process for insights, not really, but is it big? of course yes, it is colossal. This is Big Data, not only the accumulation of data, that is the easy part. But processing it for insights, understanding trends, searching titbits within it, knowing what it says, is the tricky part. And if managed with excellence, it can change the world around us.
Also Read: Top Careers to Explore In Big Data Analytics
Industries across verticals are researching methods to create sophisticated methods of analysing and decoding the data, to accurately understand what it is saying, and are developing methods of using this information to enhance business prospects.
This blog puts together some positive impacts of using Big Data Insights, which are going to change the world we live in. Not all the suggested changes will be immediate, some might happen in the immediate future, while others might be gradual, and some might be impacting our current lives as we read this blog.

Healthcare

There is a possibility that in the near future all patient history can be stored on a consolidated mainframe. The advantage is that not only a unique entry can be identified by any doctor, but on putting certain symptoms, your doctor will be able to see every patient in the world who has experienced those symptoms, and their response to treatments. Imagine the benefit it will present in terms of early detection of a disease and its prevention. The medical fraternity is aware of the advantages of Big Data and is ensuring they hop on the bus.

Security Applications

Big Data are being developed, like a pedometer application, that helps identify people on the way they walk, based on their pace. What is promising is that this application is so far in the testing phase and is a 100% accurate. So imagine a time you walk into an ATM and you do not need to enter your password simply because the machine recognises you by your walk. Or another application called ‘Pindrop’, which helps identify the person and location of the caller, by recognising the background sounds.
Facial recognition is another feature of Big data, which is getting better at processing. The capabilities of the machine to pick up our unique features like speech, movements, tonality and marry it with security features is possible, and we might very well enter a phase where forgery with signatures could be extinct.

Decoding the Consumer

With the rise of e-commerce websites, and the onset of all devices connected to the internet, which capture consumer patterns, possibilities in understanding the consumer and their behavioural patterns is on the rise. Companies recognise the value in accurately predicting and influencing consumer behaviour, as it could swing the profit tables upwards. Over a period of time companies will be able to identify not only a group of people based on demographics, but also individual patterns, and will then be able to not only cater to a group but to customize individual products as well, such information can also be used in other areas, like influencing the voter mindset specifically for a family at the time of Elections.
From Urban planning to presenting you with the piece of news you would like to read in the morning, Big Data, its impact and its effect can be felt around us. While it could be fashionable to think and mull over this intervention as an invasion to our privacy, the other bright side to the coin is to consider how the collection and analysis of data can make our lives easier.
Related Article: The Role of Big Data in Digital Marketing

To excel in the data science profession, it becomes imperative that one starts making efforts in sharpening their skills to full potential, by not only understanding the underlying models of data science, but escalating their knowledge in programming tools as well. When it comes to programming, the most talked about the debate is between R and Python enthusiasts, claiming one over the other, with reasons and by listing down the advantages of one over other. This debate needs a change of perspective because whenever we enter a debate you are claiming one is better than the other. And for a true data science enthusiast this would not be beneficial, in fact, if one programming language is chosen over another, in some sense it serves more as a disadvantage. A sound data scientist should definitely know the difference between both the languages, so that he or she is sure when to use what, while coming up with solutions, but beyond that, if the data scientist understands the fundamentals of both R and Python, they can then leverage their knowledge of both languages, based on their understanding of the basic data science concepts.
R and Python are just different tools used to perform different tasks, think about it! As a data scientist you will need a tool that will permit you to perform, statistical computations, data analysis etc…, hence R or Python just become elements of knowledge from disciplines such as statistics, computer science, engineering, essentially a computing tool. Now if you are a carpenter, would it just be good for you to have one wrench in your toolbox or a combination of wrench’s which are dynamic in its capabilities? Similarly, with the knowledge of both the languages, a data scientist will be able to express better in data science projects. This analogy gives you enough reasons to learn both the languages.
Learning both the languages will help you gain confidence in communication while working in the field of data science, you will need to interact with both sets of people, who are users of R and Python. Any company or vertical you associate with will have projects done in both languages. Without the basic knowledge of both these languages, it will be very difficult to appreciate the efforts and the entire working around on the project. An added advantage of fluency in both the languages would be your ability to communicate with audiences that are comfortable with either language while working on a project.
Springboard your career opportunity in the field of data science, some companies, or at times, departments within the companies might be comfortable using either of the languages. The last thing a candidate thinking of pursuing his career in data science should be missing out on an opportunity, due to the lack of knowledge in either R or Python. No one is expected to be a professional in any one language, a data scientist is expected to acquire as many skills and tool as they can to be successful in their career. Hence learning basics of both the languages will give you an upper edge.
The best thing is that both the languages are not very difficult to learn, although R and Python are unique languages, they are very similar in many ways. They do have a different syntax and have their own technical advantages, at the same time are similar while using appropriate Python packages like Pandas etc…,
The world of data science is not getting smaller but will only continue to grow in this data-driven world. At such times it is best to enhance your knowledge where ever applicable. Doing an online course to learn R and Python would thus be a very recommended effort.

Whenever you see a recommendation for you on Amazon, we hope you know it’s not a coincidence! In fact, Amazon has become a case study for a lot of online retailers, to watch and learn about ways to focus heavily on data-driven marketing.

At the crux of it, Amazon’s recommendation is based on many factors, like shopping history of a customer, the items customers select and store in the shopping carts online, items that they desire or like, basically their wish list, and what other customers have viewed and purchased.

This mechanism is the brainchild of Amazon and they have christened it, “Item to item collaborative filtering” and it is this algorithm that Amazon has used aggressively to enhance, and customize the user experience, in turn targeting sales.

Existing recommendation systems could not perform the huge volumes of Amazons products and customer base, hence they decided to build one for themselves. It is purely for this reason that a new mother would see baby products while a sports enthusiast would see gadgets on the Amazon web pages.

Using this methodology, Amazon has generated 102.1 billion in net revenue. 35% of Amazon.com’s revenue is generated by the recommendation engine.

Amazon’s recommendations are two types Online and Offsite.

Want to know how they do it? Well, read along……

Amazon’s On-line Recommendation, based on user shopping history and browsing activity for products, this list is created. The algorithm aims at the ‘wow’ experience. The sentiment, that only Amazon could have come up with this recommendation, and that if the customer was to search, they would never be able to discover the product, is targeted.

Frequently Bought Together, this feature only has one aim and that is to increase the order value. The customer might not need the product but buys them as they see others have. Amazon is able to show these suggestions to the customer, based on what they are about to buy, and cross-sell logical products that go with the primary one.

Your Recently Used Items and Featured Recommendation, this is a very interesting recommendation tool that Amazon uses. Here, based on your browsing history, Amazon shows products with similarity, either different brand same product, same brand different products, products similar in shape, size etc…, with the intention of increasing sales by offering at least something that you find interesting enough to buy.

Off-Site Recommendations, Amazon sends specific and relevant emails to its customers to increase sales. This, of course, is done scientifically. A new mother would not receive hiking gear on sale updates in her inbox as it would be a disconnect. This form of recommendation has a very high conversion ratio.

Recommendations thus remain as one of the biggest innovations in online shopping. The prediction algorithms are very important for virtual stores, as their accuracy has a direct impact on sales. The data on which these algorithms work is updated regularly because the fluency in movement of a virtual customer is hugely based on their usage of the website, changes in preferences, introduction of new products etc…,

‘Collaborative’, ‘Customer Clusters’, ‘Simple Search’, ‘Item to Item’, ‘Bellkor’, are some of the most popular algorithms not only used by Amazon but other e-commerce companies for recommendations.

Predicting Algorithms used for recommendations, if they are fast and close to accuracy, then they highly benefit the sales of the organisation. Amazon’s case study is a testimony to it. To learn more about the Amazon’s predicting Algorithm watch this space until next week for the big news!